[00:01:05] RECOVERY - matomo151 Check unit status of matomo-archiver-1 on matomo151 is OK: OK: Status of the systemd unit matomo-archiver-1 [00:01:28] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/33d758c3e7c525afc7d3f9cc80ccff70d6525267 [00:01:28] 02statichelp/03main 07WikiTideBot 0333d758c Bot: Auto-update Tech namespace pages 2026-01-15 00:01:26 [00:02:29] RECOVERY - matomo151 Check unit status of matomo-archiver-2 on matomo151 is OK: OK: Status of the systemd unit matomo-archiver-2 [00:04:00] RECOVERY - matomo151 Check unit status of matomo-archiver-3 on matomo151 is OK: OK: Status of the systemd unit matomo-archiver-3 [00:05:18] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/6c5f9abfcf5e16d1ad8383f43778b566e90e90d3 [00:05:18] 02ssl/03main 07WikiTideBot 036c5f9ab Bot: Auto-update domain lists [00:35:19] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/6767fd8488823c06d8fa6c27b75a8ea6fbf916e0 [00:35:19] 02ssl/03main 07WikiTideBot 036767fd8 Bot: Auto-update domain lists [00:37:08] RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 67732MiB (14.9% inode=100%); [00:39:34] RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 68858MiB (15.2% inode=100%); [00:44:15] [02mw-config] 07paladox created 03paladox-patch-5 (+1 new commit) 13https://github.com/miraheze/mw-config/commit/5bc9807e2596 [00:44:15] 02mw-config/03paladox-patch-5 07paladox 035bc9807 Set $wgManageWikiServers for test151 [00:44:19] [02mw-config] 07paladox opened pull request #6233: Set $wgManageWikiServers for test151 (03main...03paladox-patch-5) 13https://github.com/miraheze/mw-config/pull/6233 [00:45:01] [02mw-config] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/8a234fa32e9e215292dbdfbcb48cc58a668b86f8 [00:45:01] 02mw-config/03main 07paladox 038a234fa Set $wgManageWikiServers for test151 (#6233) [00:45:01] [02mw-config] 07paladox merged pull request #6233: Set $wgManageWikiServers for test151 (03main...03paladox-patch-5) 13https://github.com/miraheze/mw-config/pull/6233 [00:45:03] [02mw-config] 07paladox 04deleted 03paladox-patch-5 at 035bc9807 13https://api.github.com/repos/miraheze/mw-config/commit/5bc9807 [00:45:12] !log [paladox@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [00:45:14] !log [paladox@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 1s [00:45:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:45:14] miraheze/mw-config - paladox the build passed. [00:45:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:45:59] miraheze/mw-config - paladox the build passed. [01:01:41] RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 69197MiB (15.3% inode=100%); [02:13:12] [02puppet] 07paladox created 03paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/puppet/commit/a4c1052ff369 [02:13:12] 02puppet/03paladox-patch-1 07paladox 03a4c1052 Resolver: reduce ttl for max negative ttl [02:13:17] [02puppet] 07paladox opened pull request #4688: Resolver: reduce ttl for max negative ttl (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4688 [02:13:27] [02puppet] 07coderabbitai[bot] commented on pull request #4688:
[…] 13https://github.com/miraheze/puppet/pull/4688#issuecomment-3752551344 [02:13:31] [02puppet] 07paladox merged pull request #4688: Resolver: reduce ttl for max negative ttl (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4688 [02:13:31] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/956d52be617538794b1e61d9f6d2e31046b398e5 [02:13:31] 02puppet/03main 07paladox 03956d52b Resolver: reduce ttl for max negative ttl (#4688) [02:13:33] [02puppet] 07paladox 04deleted 03paladox-patch-1 at 03a4c1052 13https://api.github.com/repos/miraheze/puppet/commit/a4c1052 [02:20:19] [02puppet] 07paladox created 03revert-4688-paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/puppet/commit/f66d8bdc9787 [02:20:21] 02puppet/03revert-4688-paladox-patch-1 07paladox 03f66d8bd Revert "Resolver: reduce ttl for max negative ttl (#4688)"… [02:20:25] [02puppet] 07paladox opened pull request #4689: Revert "Resolver: reduce ttl for max negative ttl" (03main...03revert-4688-paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4689 [02:20:31] [02puppet] 07coderabbitai[bot] commented on pull request #4689:
[…] 13https://github.com/miraheze/puppet/pull/4689#issuecomment-3752567854 [02:20:34] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/9c8b62baf5600704945bd2c6ef5f4225997cd279 [02:20:34] 02puppet/03main 07paladox 039c8b62b Revert "Resolver: reduce ttl for max negative ttl" (#4689)… [02:20:36] [02puppet] 07paladox merged pull request #4689: Revert "Resolver: reduce ttl for max negative ttl" (03main...03revert-4688-paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4689 [02:20:38] [02puppet] 07paladox 04deleted 03revert-4688-paladox-patch-1 at 03f66d8bd 13https://api.github.com/repos/miraheze/puppet/commit/f66d8bd [02:34:25] PROBLEM - mwtask181 Disk Space on mwtask181 is WARNING: DISK WARNING - free space: / 19872MiB (8% inode=95%); [02:54:25] RECOVERY - mwtask181 Disk Space on mwtask181 is OK: DISK OK - free space: / 36360MiB (16% inode=95%); [05:57:46] PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push [06:05:46] RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push [06:22:03] [02landing] 07dependabot[bot] created 03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0 (+1 new commit) 13https://github.com/miraheze/landing/commit/0abeb32c73e9 [06:22:03] 02landing/03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0 07dependabot[bot] 030abeb32 Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0… [06:22:04] [02landing] 07dependabot[bot] added the label 'dependencies' to pull request #180 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/landing/pull/180 [06:22:05] [02landing] 07dependabot[bot] opened pull request #180: Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0 (03main...03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0) 13https://github.com/miraheze/landing/pull/180 [06:22:07] [02landing] 07dependabot[bot] added the label 'github_actions' to pull request #180 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/landing/pull/180 [06:22:09] [02landing] 07dependabot[bot] added the label 'dependencies' to pull request #180 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/landing/pull/180 [06:22:11] [02landing] 07dependabot[bot] added the label 'github_actions' to pull request #180 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/landing/pull/180 [06:22:12] [02landing] 07coderabbitai[bot] commented on pull request #180: --- […] 13https://github.com/miraheze/landing/pull/180#issuecomment-3753072599 [06:22:46] miraheze/landing - dependabot[bot] the build passed. [06:25:17] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/ddd7143a9909d1e6dbf6c7b17df70786c2900783 [06:25:17] 02ssl/03main 07WikiTideBot 03ddd7143 Bot: Auto-update domain lists [06:52:38] [02phorge-extensions] 07dependabot[bot] created 03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0 (+1 new commit) 13https://github.com/miraheze/phorge-extensions/commit/64eb928d1288 [06:52:38] 02phorge-extensions/03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0 07dependabot[bot] 0364eb928 Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0… [06:52:39] [02phorge-extensions] 07dependabot[bot] added the label 'github_actions' to pull request #65 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/phorge-extensions/pull/65 [06:52:41] [02phorge-extensions] 07dependabot[bot] added the label 'dependencies' to pull request #65 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/phorge-extensions/pull/65 [06:52:43] [02phorge-extensions] 07dependabot[bot] opened pull request #65: Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0 (03main...03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0) 13https://github.com/miraheze/phorge-extensions/pull/65 [06:52:45] [02phorge-extensions] 07dependabot[bot] added the label 'dependencies' to pull request #65 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/phorge-extensions/pull/65 [06:52:47] [02phorge-extensions] 07dependabot[bot] added the label 'github_actions' to pull request #65 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/phorge-extensions/pull/65 [06:52:57] [02phorge-extensions] 07coderabbitai[bot] commented on pull request #65: --- […] 13https://github.com/miraheze/phorge-extensions/pull/65#issuecomment-3753144821 [06:53:18] miraheze/phorge-extensions - dependabot[bot] the build passed. [07:58:40] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 4 backends are down. mw172 mw181 mw153 mw193 [07:58:49] PROBLEM - cp161 Varnish Backends on cp161 is CRITICAL: 4 backends are down. mw162 mw172 mw153 mw202 [07:59:09] PROBLEM - cp171 HTTPS on cp171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [07:59:18] PROBLEM - test151 MediaWiki Rendering on test151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:59:28] No [07:59:31] Not again [07:59:31] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 10 backends are down. mw151 mw152 mw171 mw172 mw182 mw183 mw191 mw192 mw202 mw203 [07:59:32] PROBLEM - mw201 HTTPS on mw201 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [07:59:35] PROBLEM - mw163 MediaWiki Rendering on mw163 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.010 second response time [07:59:37] You're only allowed to work [08:00:14] PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 2 backends are down. mw161 mw193 [08:00:20] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is WARNING: WARNING - NGINX Error Rate is 41% [08:00:40] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [08:00:49] RECOVERY - cp161 Varnish Backends on cp161 is OK: All 31 backends are healthy [08:01:05] RECOVERY - cp171 HTTPS on cp171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4303 bytes in 0.078 second response time [08:01:11] That's better [08:01:13] RECOVERY - test151 MediaWiki Rendering on test151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.851 second response time [08:01:26] RECOVERY - mw201 HTTPS on mw201 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4254 bytes in 0.060 second response time [08:01:31] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [08:01:35] RECOVERY - mw163 MediaWiki Rendering on mw163 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.190 second response time [08:01:43] Now stay like that for the next 9 hours please icinga-miraheze so I can do my day job [08:02:14] RECOVERY - cp191 Varnish Backends on cp191 is OK: All 31 backends are healthy [08:02:20] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 21% [08:08:52] i cast fireball [10:18:14] farting [12:12:56] [02CreateWiki] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/CreateWiki/commit/929c3f1d3f10623625f624265b7f9a07e0873c03 [12:12:56] 02CreateWiki/03main 07translatewiki.net 03929c3f1 Localisation updates from https://translatewiki.net. [12:12:57] [02ErrorPages] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/ErrorPages/commit/48135942fc52a008feefca4521079db88948e245 [12:12:58] 02ErrorPages/03main 07translatewiki.net 034813594 Localisation updates from https://translatewiki.net. [12:12:59] [02MatomoAnalytics] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/MatomoAnalytics/commit/cb200b421812b25b10cadd87adcb865978a61f47 [12:13:00] 02MatomoAnalytics/03main 07translatewiki.net 03cb200b4 Localisation updates from https://translatewiki.net. [12:13:02] [02DataDump] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/DataDump/commit/92486c6e2e7ed68f6f08973256fca9a739cafd0b [12:13:04] [02landing] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/landing/commit/091d5dde4ff873f5f64e0a3133951729019d77a8 [12:13:06] [02ManageWiki] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/ManageWiki/commit/ce55e4d05003bd3b2864ffbf68dfb49d4dd84eba [12:13:08] 02DataDump/03main 07translatewiki.net 0392486c6 Localisation updates from https://translatewiki.net. [12:13:10] [02WikiDiscover] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/WikiDiscover/commit/37c81ea8e467badef86a38e5ebd25318cc91c070 [12:13:12] [02MirahezeMagic] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/MirahezeMagic/commit/8ffd7c6ee251263009a3ea0fd9c0582f47379bc9 [12:13:13] 02landing/03main 07translatewiki.net 03091d5dd Localisation updates from https://translatewiki.net. [12:13:15] 02ManageWiki/03main 07translatewiki.net 03ce55e4d Localisation updates from https://translatewiki.net. [12:13:16] 02WikiDiscover/03main 07translatewiki.net 0337c81ea Localisation updates from https://translatewiki.net. [12:13:18] 02MirahezeMagic/03main 07translatewiki.net 038ffd7c6 Localisation updates from https://translatewiki.net. [12:17:21] miraheze/MirahezeMagic - translatewiki the build has errored. [12:17:23] miraheze/ErrorPages - translatewiki the build passed. [12:17:32] miraheze/landing - translatewiki the build passed. [12:17:36] miraheze/DataDump - translatewiki the build passed. [12:17:41] miraheze/MatomoAnalytics - translatewiki the build passed. [12:18:44] miraheze/ManageWiki - translatewiki the build passed. [12:20:23] miraheze/WikiDiscover - translatewiki the build passed. [12:21:29] miraheze/CreateWiki - translatewiki the build passed. [12:39:57] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php /srv/mediawiki/1.44/extensions/MirahezeMagic/maintenance/CheckWikiDatabases.php --wiki=loginwiki --delete (END - exit=2) [12:39:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:40:26] !log DELETED and DROPPED https://issue-tracker.miraheze.org/P581 [12:40:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:06:28] [02puppet] 07paladox created 03paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/puppet/commit/6e1eade0e736 [14:06:28] 02puppet/03paladox-patch-1 07paladox 036e1eade dns: Set extended-resolution-errors to yes [14:06:33] [02puppet] 07paladox opened pull request #4690: dns: Set extended-resolution-errors to yes (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4690 [14:06:44] [02puppet] 07coderabbitai[bot] commented on pull request #4690:
[…] 13https://github.com/miraheze/puppet/pull/4690#issuecomment-3755066282 [14:08:13] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/31c9ecdb86b24fc088a6ca8a2e43ea291688eb57 [14:08:13] 02puppet/03paladox-patch-1 07paladox 0331c9ecd Update recursor.yml.erb [14:09:15] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/ccea2d5caae568a3c27368faf39bb2f7091feaa6 [14:09:15] 02puppet/03main 07paladox 03ccea2d5 dns: Set extended-resolution-errors to yes (#4690) [14:09:17] [02puppet] 07paladox merged pull request #4690: dns: Set extended-resolution-errors to yes (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4690 [14:09:19] [02puppet] 07paladox 04deleted 03paladox-patch-1 at 0331c9ecd 13https://api.github.com/repos/miraheze/puppet/commit/31c9ecd [14:10:49] [02mw-config] 07dependabot[bot] created 03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0 (+1 new commit) 13https://github.com/miraheze/mw-config/commit/178b7b3e8dd0 [14:10:49] 02mw-config/03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0 07dependabot[bot] 03178b7b3 Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0… [14:10:50] [02mw-config] 07dependabot[bot] added the label 'github_actions' to pull request #6234 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/mw-config/pull/6234 [14:10:50] [02mw-config] 07dependabot[bot] added the label 'dependencies' to pull request #6234 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/mw-config/pull/6234 [14:10:52] [02mw-config] 07dependabot[bot] opened pull request #6234: Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0 (03main...03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0) 13https://github.com/miraheze/mw-config/pull/6234 [14:10:54] [02mw-config] 07dependabot[bot] added the label 'dependencies' to pull request #6234 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/mw-config/pull/6234 [14:10:56] [02mw-config] 07dependabot[bot] added the label 'github_actions' to pull request #6234 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/mw-config/pull/6234 [14:11:08] [02mw-config] 07dependabot[bot] created 03dependabot/composer/phpunit/phpunit-12.5.5 (+1 new commit) 13https://github.com/miraheze/mw-config/commit/6cc1813b4028 [14:11:08] 02mw-config/03dependabot/composer/phpunit/phpunit-12.5.5 07dependabot[bot] 036cc1813 Update phpunit/phpunit requirement from 12.5.4 to 12.5.5… [14:11:10] [02mw-config] 07dependabot[bot] added the label 'dependencies' to pull request #6235 (Update phpunit/phpunit requirement from 12.5.4 to 12.5.5) 13https://github.com/miraheze/mw-config/pull/6235 [14:11:10] [02mw-config] 07dependabot[bot] added the label 'php' to pull request #6235 (Update phpunit/phpunit requirement from 12.5.4 to 12.5.5) 13https://github.com/miraheze/mw-config/pull/6235 [14:11:12] [02mw-config] 07dependabot[bot] opened pull request #6235: Update phpunit/phpunit requirement from 12.5.4 to 12.5.5 (03main...03dependabot/composer/phpunit/phpunit-12.5.5) 13https://github.com/miraheze/mw-config/pull/6235 [14:11:14] [02mw-config] 07dependabot[bot] added the label 'php' to pull request #6235 (Update phpunit/phpunit requirement from 12.5.4 to 12.5.5) 13https://github.com/miraheze/mw-config/pull/6235 [14:11:16] [02mw-config] 07dependabot[bot] added the label 'dependencies' to pull request #6235 (Update phpunit/phpunit requirement from 12.5.4 to 12.5.5) 13https://github.com/miraheze/mw-config/pull/6235 [14:16:26] [02mw-config] 07pskyechology merged 07dependabot[bot]'s pull request #6226: Update mediawiki/mediawiki-codesniffer requirement from 48.0.0 to 49.0.0 (03main...03dependabot/composer/mediawiki/mediawiki-codesniffer-49.0.0) 13https://github.com/miraheze/mw-config/pull/6226 [14:16:27] [02mw-config] 07pskyechology pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/03b52fc2b249f767077dce6af90fafa6bbea36d9 [14:16:27] 02mw-config/03main 07dependabot[bot] 0303b52fc Update mediawiki/mediawiki-codesniffer requirement from 48.0.0 to 49.0.0 (#6226)… [14:16:29] [02mw-config] 07pskyechology 04deleted 03dependabot/composer/mediawiki/mediawiki-codesniffer-49.0.0 at 0383bbf11 13https://api.github.com/repos/miraheze/mw-config/commit/83bbf11 [14:17:34] [02mw-config] 07dependabot[bot] 04force-pushed 03dependabot/composer/mediawiki/minus-x-2.0.0 from 03657a234 to 03747e5bc 13https://api.github.com/repos/miraheze/mw-config/commits/dependabot/composer/mediawiki/minus-x-2.0.0 [14:17:34] 02mw-config/03dependabot/composer/mediawiki/minus-x-2.0.0 07dependabot[bot] 03747e5bc Update mediawiki/minus-x requirement from 1.1.3 to 2.0.0… [14:18:10] [02mw-config] 07pskyechology merged 07dependabot[bot]'s pull request #6235: Update phpunit/phpunit requirement from 12.5.4 to 12.5.5 (03main...03dependabot/composer/phpunit/phpunit-12.5.5) 13https://github.com/miraheze/mw-config/pull/6235 [14:18:11] [02mw-config] 07pskyechology pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/a1d9a14b0a3724aa1da818272af5cd4e55cb154b [14:18:12] 02mw-config/03main 07dependabot[bot] 03a1d9a14 Update phpunit/phpunit requirement from 12.5.4 to 12.5.5 (#6235)… [14:18:14] [02mw-config] 07pskyechology 04deleted 03dependabot/composer/phpunit/phpunit-12.5.5 at 036cc1813 13https://api.github.com/repos/miraheze/mw-config/commit/6cc1813 [14:18:25] [02mw-config] 07dependabot[bot] 04force-pushed 03dependabot/composer/mediawiki/minus-x-2.0.0 from 03747e5bc to 036ad2fce 13https://api.github.com/repos/miraheze/mw-config/commits/dependabot/composer/mediawiki/minus-x-2.0.0 [14:18:25] 02mw-config/03dependabot/composer/mediawiki/minus-x-2.0.0 07dependabot[bot] 036ad2fce Update mediawiki/minus-x requirement from 1.1.3 to 2.0.0… [14:20:10] [02mw-config] 07dependabot[bot] 04force-pushed 03dependabot/composer/mediawiki/minus-x-2.0.0 from 03747e5bc to 036ad2fce 13https://api.github.com/repos/miraheze/mw-config/commits/dependabot/composer/mediawiki/minus-x-2.0.0 [14:20:10] 02mw-config/03dependabot/composer/mediawiki/minus-x-2.0.0 07dependabot[bot] 036ad2fce Update mediawiki/minus-x requirement from 1.1.3 to 2.0.0… [14:24:13] [02puppet] 07paladox created 03paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/puppet/commit/9559e6f144e0 [14:24:13] 02puppet/03paladox-patch-1 07paladox 039559e6f dns: Tweek pdns-recursor config… [14:24:17] [02puppet] 07paladox opened pull request #4691: dns: Tweek pdns-recursor config (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4691 [14:24:26] [02puppet] 07coderabbitai[bot] commented on pull request #4691:
[…] 13https://github.com/miraheze/puppet/pull/4691#issuecomment-3755138731 [14:29:09] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/5254099bc7048322810234ac5bde59c1e9ad5baa [14:29:10] 02puppet/03paladox-patch-1 07paladox 035254099 Update recursor.yml.erb [14:36:11] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/46d0b92180a51d4e34c9db43ee4f5bc75c8426d1 [14:36:11] 02puppet/03paladox-patch-1 07paladox 0346d0b92 Update recursor.yml.erb [14:36:41] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/5f891ac71f3c99e8736d3e01e45b3d4a6fafa297 [14:36:41] 02puppet/03paladox-patch-1 07paladox 035f891ac Update bastion.yaml [14:36:54] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/d100273e7a2fb40345f75852dd530bfa659a1293 [14:36:54] 02puppet/03paladox-patch-1 07paladox 03d100273 Update common.yaml [14:37:28] PROBLEM - mw152 PowerDNS Recursor on mw152 is CRITICAL: No response from DNS ::1 [14:37:29] PROBLEM - mw152 Puppet on mw152 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:38:21] PROBLEM - mw183 Puppet on mw183 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:38:34] PROBLEM - mw182 Puppet on mw182 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:38:35] PROBLEM - mw182 PowerDNS Recursor on mw182 is CRITICAL: No response from DNS ::1 [14:38:50] PROBLEM - mw183 PowerDNS Recursor on mw183 is CRITICAL: No response from DNS ::1 [14:40:12] PROBLEM - swiftproxy161 PowerDNS Recursor on swiftproxy161 is CRITICAL: No response from DNS ::1 [14:40:19] PROBLEM - swiftproxy161 Puppet on swiftproxy161 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:40:23] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/d94332faf59bb281e9907657f5bf96e65d7298bc [14:40:23] 02puppet/03paladox-patch-1 07paladox 03d94332f Update recursor.yml.erb [14:42:00] PROBLEM - mw161 PowerDNS Recursor on mw161 is CRITICAL: No response from DNS ::1 [14:42:13] PROBLEM - mw162 PowerDNS Recursor on mw162 is CRITICAL: No response from DNS ::1 [14:42:16] PROBLEM - mw202 Puppet on mw202 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:42:23] PROBLEM - mw162 Puppet on mw162 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:42:35] PROBLEM - swiftproxy171 PowerDNS Recursor on swiftproxy171 is CRITICAL: No response from DNS ::1 [14:42:37] PROBLEM - swiftproxy171 Check unit status of swift-container-stats_mw-media on swiftproxy171 is CRITICAL: CRITICAL: Status of the systemd unit swift-container-stats_mw-media [14:42:40] PROBLEM - mw202 PowerDNS Recursor on mw202 is CRITICAL: No response from DNS ::1 [14:42:55] PROBLEM - swiftproxy171 Check unit status of swift-account-stats_swift:dispersion on swiftproxy171 is CRITICAL: CRITICAL: Status of the systemd unit swift-account-stats_swift:dispersion [14:43:25] PROBLEM - mw172 PowerDNS Recursor on mw172 is CRITICAL: No response from DNS ::1 [14:43:36] PROBLEM - mw203 PowerDNS Recursor on mw203 is CRITICAL: No response from DNS ::1 [14:43:42] PROBLEM - swiftproxy171 Check unit status of swift-account-stats_admin:admin on swiftproxy171 is CRITICAL: CRITICAL: Status of the systemd unit swift-account-stats_admin:admin [14:43:44] PROBLEM - swiftproxy171 Check unit status of swift-account-stats_mw:media on swiftproxy171 is CRITICAL: CRITICAL: Status of the systemd unit swift-account-stats_mw:media [14:43:45] PROBLEM - swiftproxy171 Puppet on swiftproxy171 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:43:47] PROBLEM - mw172 Puppet on mw172 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:43:48] PROBLEM - mw161 Puppet on mw161 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:43:59] PROBLEM - mwtask171 Puppet on mwtask171 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:44:09] PROBLEM - swiftobject191 Puppet on swiftobject191 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:44:35] PROBLEM - mwtask161 PowerDNS Recursor on mwtask161 is CRITICAL: No response from DNS ::1 [14:44:54] PROBLEM - swiftobject201 Puppet on swiftobject201 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:44:54] PROBLEM - mw203 Puppet on mw203 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:44:58] PROBLEM - mwtask171 PowerDNS Recursor on mwtask171 is CRITICAL: No response from DNS ::1 [14:45:32] PROBLEM - mwtask161 Puppet on mwtask161 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:45:36] PROBLEM - swiftobject201 PowerDNS Recursor on swiftobject201 is CRITICAL: No response from DNS ::1 [14:45:40] PROBLEM - mw173 Puppet on mw173 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:45:46] PROBLEM - swiftobject191 PowerDNS Recursor on swiftobject191 is CRITICAL: No response from DNS ::1 [14:46:00] PROBLEM - swiftobject181 Puppet on swiftobject181 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:46:02] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/9ad838e8a4e9c6431b3704a8af716aae3756a915 [14:46:02] 02puppet/03paladox-patch-1 07paladox 039ad838e Update recursor.yml.erb [14:46:03] PROBLEM - swiftac171 Puppet on swiftac171 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:46:05] PROBLEM - mwtask151 PowerDNS Recursor on mwtask151 is CRITICAL: No response from DNS ::1 [14:46:28] PROBLEM - swiftac171 PowerDNS Recursor on swiftac171 is CRITICAL: No response from DNS ::1 [14:46:32] PROBLEM - db161 PowerDNS Recursor on db161 is CRITICAL: No response from DNS ::1 [14:46:36] PROBLEM - mwtask151 Puppet on mwtask151 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:46:47] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/8081f178d6da2a9c4faec474115f487d29582b73 [14:46:47] 02puppet/03paladox-patch-1 07paladox 038081f17 Update recursor.conf.erb [14:46:58] PROBLEM - mw173 PowerDNS Recursor on mw173 is CRITICAL: No response from DNS ::1 [14:47:07] PROBLEM - swiftobject181 PowerDNS Recursor on swiftobject181 is CRITICAL: No response from DNS ::1 [14:47:22] PROBLEM - db161 Puppet on db161 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:47:24] [02puppet] 07paladox merged pull request #4691: dns: Tweek pdns-recursor config (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4691 [14:47:24] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/71c7dd489211ab1bef85a4871e202dd236147354 [14:47:26] 02puppet/03main 07paladox 0371c7dd4 dns: Tweek pdns-recursor config (#4691)… [14:47:27] [02puppet] 07paladox 04deleted 03paladox-patch-1 at 038081f17 13https://api.github.com/repos/miraheze/puppet/commit/8081f17 [14:47:48] PROBLEM - cloud15 Puppet on cloud15 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:47:51] PROBLEM - cloud16 PowerDNS Recursor on cloud16 is CRITICAL: No response from DNS ::1 [14:48:00] PROBLEM - cloud15 PowerDNS Recursor on cloud15 is CRITICAL: No response from DNS ::1 [14:49:26] PROBLEM - cloud16 Puppet on cloud16 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:49:39] PROBLEM - swiftproxy171 Check unit status of swift_dispersion_stats on swiftproxy171 is CRITICAL: CRITICAL: Status of the systemd unit swift_dispersion_stats [14:50:19] PROBLEM - mw193 PowerDNS Recursor on mw193 is CRITICAL: No response from DNS ::1 [14:50:24] PROBLEM - db171 PowerDNS Recursor on db171 is CRITICAL: No response from DNS ::1 [14:50:33] PROBLEM - mw193 Puppet on mw193 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:50:50] PROBLEM - mw192 Puppet on mw192 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:51:34] PROBLEM - mw153 PowerDNS Recursor on mw153 is CRITICAL: No response from DNS ::1 [14:51:35] PROBLEM - db171 Puppet on db171 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:51:35] PROBLEM - db151 PowerDNS Recursor on db151 is CRITICAL: No response from DNS ::1 [14:51:47] PROBLEM - mw151 PowerDNS Recursor on mw151 is CRITICAL: No response from DNS ::1 [14:51:53] PROBLEM - mw181 Puppet on mw181 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:52:06] PROBLEM - mw153 Puppet on mw153 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:52:17] PROBLEM - mw163 PowerDNS Recursor on mw163 is CRITICAL: No response from DNS ::1 [14:52:18] PROBLEM - db151 Puppet on db151 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:52:25] PROBLEM - mw171 Puppet on mw171 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:52:32] PROBLEM - mw151 Puppet on mw151 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:52:35] PROBLEM - mw191 Puppet on mw191 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:52:36] PROBLEM - cloud17 PowerDNS Recursor on cloud17 is CRITICAL: No response from DNS ::1 [14:52:37] PROBLEM - db181 PowerDNS Recursor on db181 is CRITICAL: No response from DNS ::1 [14:52:48] PROBLEM - mw181 PowerDNS Recursor on mw181 is CRITICAL: No response from DNS ::1 [14:52:58] PROBLEM - swiftproxy161 Swift NGINX SSL check on swiftproxy161 is CRITICAL: connect to address localhost and port 443: Connection refused [14:53:02] reverted ^ [14:53:05] PROBLEM - mw201 PowerDNS Recursor on mw201 is CRITICAL: No response from DNS ::1 [14:53:13] PROBLEM - mw163 Puppet on mw163 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:53:15] PROBLEM - db181 Puppet on db181 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:53:19] PROBLEM - swiftproxy161 HTTPS on swiftproxy161 is CRITICAL: connect to address 10.0.16.135 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [14:53:24] PROBLEM - mw171 PowerDNS Recursor on mw171 is CRITICAL: No response from DNS ::1 [14:53:30] PROBLEM - mw201 Puppet on mw201 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:53:31] PROBLEM - mw191 PowerDNS Recursor on mw191 is CRITICAL: No response from DNS ::1 [14:53:46] PROBLEM - cloud17 Puppet on cloud17 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:54:21] PROBLEM - cloud19 PowerDNS Recursor on cloud19 is CRITICAL: No response from DNS ::1 [14:54:35] RECOVERY - mwtask161 PowerDNS Recursor on mwtask161 is OK: DNS OK: 0.045 seconds response time. mwtask161.fsslc.wtnet returns 10.0.16.157 [14:54:50] RECOVERY - mw192 Puppet on mw192 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:54:54] PROBLEM - cloud19 Puppet on cloud19 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[pdns-recursor] [14:54:58] RECOVERY - mwtask171 PowerDNS Recursor on mwtask171 is OK: DNS OK: 0.038 seconds response time. mwtask171.fsslc.wtnet returns 10.0.17.144 [14:56:05] RECOVERY - mwtask151 PowerDNS Recursor on mwtask151 is OK: DNS OK: 0.038 seconds response time. mwtask151.fsslc.wtnet returns 10.0.15.150 [14:58:36] RECOVERY - cloud17 PowerDNS Recursor on cloud17 is OK: DNS OK: 0.147 seconds response time. cloud17.wikitide.net returns 2602:294:0:b23::100,38.46.218.228 [14:59:51] RECOVERY - cloud16 PowerDNS Recursor on cloud16 is OK: DNS OK: 0.148 seconds response time. cloud16.wikitide.net returns 2602:294:0:b13::100,38.46.218.226 [15:00:00] RECOVERY - cloud15 PowerDNS Recursor on cloud15 is OK: DNS OK: 0.027 seconds response time. cloud15.wikitide.net returns 2602:294:0:c8::100,38.46.218.154 [15:00:21] RECOVERY - cloud19 PowerDNS Recursor on cloud19 is OK: DNS OK: 0.152 seconds response time. cloud19.wikitide.net returns 2602:294:0:b33::100,38.46.218.157 [15:01:29] PROBLEM - swift-lb.wikitide.net - LetsEncrypt on sslhost is CRITICAL: connect to address swift-lb.wikitide.net and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [15:01:42] RECOVERY - swiftproxy171 Check unit status of swift-account-stats_admin:admin on swiftproxy171 is OK: OK: Status of the systemd unit swift-account-stats_admin:admin [15:01:44] RECOVERY - swiftproxy171 Check unit status of swift-account-stats_mw:media on swiftproxy171 is OK: OK: Status of the systemd unit swift-account-stats_mw:media [15:05:25] RECOVERY - mw172 PowerDNS Recursor on mw172 is OK: DNS OK: 0.037 seconds response time. mw172.fsslc.wtnet returns 10.0.17.123 [15:05:28] RECOVERY - mw152 PowerDNS Recursor on mw152 is OK: DNS OK: 0.024 seconds response time. mw152.fsslc.wtnet returns 10.0.15.115 [15:05:34] RECOVERY - mw153 PowerDNS Recursor on mw153 is OK: DNS OK: 0.039 seconds response time. mw153.fsslc.wtnet returns 10.0.15.140 [15:05:36] RECOVERY - mw203 PowerDNS Recursor on mw203 is OK: DNS OK: 0.039 seconds response time. mw203.fsslc.wtnet returns 10.0.20.165 [15:05:42] PROBLEM - swiftproxy171 Check unit status of swift-account-stats_admin:admin on swiftproxy171 is CRITICAL: CRITICAL: Status of the systemd unit swift-account-stats_admin:admin [15:05:44] PROBLEM - swiftproxy171 Check unit status of swift-account-stats_mw:media on swiftproxy171 is CRITICAL: CRITICAL: Status of the systemd unit swift-account-stats_mw:media [15:05:47] RECOVERY - mw151 PowerDNS Recursor on mw151 is OK: DNS OK: 0.036 seconds response time. mw151.fsslc.wtnet returns 10.0.15.114 [15:06:00] RECOVERY - mw161 PowerDNS Recursor on mw161 is OK: DNS OK: 0.033 seconds response time. mw161.fsslc.wtnet returns 10.0.16.132 [15:06:12] RECOVERY - swiftproxy161 PowerDNS Recursor on swiftproxy161 is OK: DNS OK: 0.033 seconds response time. swiftproxy161.fsslc.wtnet returns 10.0.16.135 [15:06:13] RECOVERY - mw162 PowerDNS Recursor on mw162 is OK: DNS OK: 0.043 seconds response time. mw162.fsslc.wtnet returns 10.0.16.133 [15:06:17] RECOVERY - mw163 PowerDNS Recursor on mw163 is OK: DNS OK: 0.036 seconds response time. mw163.fsslc.wtnet returns 10.0.16.151 [15:06:19] RECOVERY - mw193 PowerDNS Recursor on mw193 is OK: DNS OK: 0.030 seconds response time. mw193.fsslc.wtnet returns 10.0.19.164 [15:06:21] RECOVERY - mw171 PowerDNS Recursor on mw171 is OK: DNS OK: 0.042 seconds response time. mw171.fsslc.wtnet returns 10.0.17.122 [15:06:24] RECOVERY - db171 PowerDNS Recursor on db171 is OK: DNS OK: 0.171 seconds response time. db171.fsslc.wtnet returns 10.0.17.119 [15:06:28] RECOVERY - swiftac171 PowerDNS Recursor on swiftac171 is OK: DNS OK: 0.045 seconds response time. swiftac171.fsslc.wtnet returns 10.0.17.125 [15:06:32] RECOVERY - db161 PowerDNS Recursor on db161 is OK: DNS OK: 0.149 seconds response time. db161.fsslc.wtnet returns 10.0.16.128 [15:06:35] RECOVERY - swiftproxy171 PowerDNS Recursor on swiftproxy171 is OK: DNS OK: 0.158 seconds response time. swiftproxy171.fsslc.wtnet returns 10.0.17.108 [15:06:35] RECOVERY - mw182 PowerDNS Recursor on mw182 is OK: DNS OK: 0.039 seconds response time. mw182.fsslc.wtnet returns 10.0.18.105 [15:06:36] RECOVERY - mw191 PowerDNS Recursor on mw191 is OK: DNS OK: 0.038 seconds response time. mw191.fsslc.wtnet returns 10.0.19.160 [15:06:37] RECOVERY - db181 PowerDNS Recursor on db181 is OK: DNS OK: 0.161 seconds response time. db181.fsslc.wtnet returns 10.0.18.102 [15:06:40] RECOVERY - mw202 PowerDNS Recursor on mw202 is OK: DNS OK: 0.035 seconds response time. mw202.fsslc.wtnet returns 10.0.20.163 [15:06:48] RECOVERY - mw181 PowerDNS Recursor on mw181 is OK: DNS OK: 0.040 seconds response time. mw181.fsslc.wtnet returns 10.0.18.104 [15:06:50] RECOVERY - mw183 PowerDNS Recursor on mw183 is OK: DNS OK: 0.036 seconds response time. mw183.fsslc.wtnet returns 10.0.18.155 [15:06:55] RECOVERY - swiftproxy171 Check unit status of swift-account-stats_swift:dispersion on swiftproxy171 is OK: OK: Status of the systemd unit swift-account-stats_swift:dispersion [15:06:58] RECOVERY - mw173 PowerDNS Recursor on mw173 is OK: DNS OK: 0.034 seconds response time. mw173.fsslc.wtnet returns 10.0.17.153 [15:07:05] RECOVERY - mw201 PowerDNS Recursor on mw201 is OK: DNS OK: 0.048 seconds response time. mw201.fsslc.wtnet returns 10.0.20.162 [15:07:07] RECOVERY - swiftobject181 PowerDNS Recursor on swiftobject181 is OK: DNS OK: 0.156 seconds response time. swiftobject181.fsslc.wtnet returns 10.0.18.107 [15:07:35] RECOVERY - db151 PowerDNS Recursor on db151 is OK: DNS OK: 0.046 seconds response time. db151.fsslc.wtnet returns 10.0.15.110 [15:07:36] RECOVERY - swiftobject201 PowerDNS Recursor on swiftobject201 is OK: DNS OK: 0.035 seconds response time. swiftobject201.fsslc.wtnet returns 10.0.20.145 [15:07:42] RECOVERY - swiftproxy171 Check unit status of swift-account-stats_admin:admin on swiftproxy171 is OK: OK: Status of the systemd unit swift-account-stats_admin:admin [15:07:44] RECOVERY - swiftproxy171 Check unit status of swift-account-stats_mw:media on swiftproxy171 is OK: OK: Status of the systemd unit swift-account-stats_mw:media [15:07:46] RECOVERY - swiftobject191 PowerDNS Recursor on swiftobject191 is OK: DNS OK: 0.157 seconds response time. swiftobject191.fsslc.wtnet returns 10.0.19.120 [15:07:48] RECOVERY - swift-lb.wikitide.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wikitide.net' will expire on Fri 13 Mar 2026 03:30:27 PM GMT +0000. [15:07:55] RECOVERY - swiftproxy161 HTTPS on swiftproxy161 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.022 second response time [15:08:03] RECOVERY - swiftproxy161 Swift NGINX SSL check on swiftproxy161 is OK: OK - Certificate 'wikitide.net' will expire on Fri Mar 13 15:30:27 2026 +0000.TCP OK - 0.010 second response time on localhost port 443 [15:09:04] RECOVERY - swiftproxy161 Puppet on swiftproxy161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:09:59] RECOVERY - swiftproxy171 Puppet on swiftproxy171 is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures [15:10:16] RECOVERY - mw202 Puppet on mw202 is OK: OK: Puppet is currently enabled, last run 44 seconds ago with 0 failures [15:10:22] RECOVERY - mw162 Puppet on mw162 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:10:37] RECOVERY - swiftproxy171 Check unit status of swift-container-stats_mw-media on swiftproxy171 is OK: OK: Status of the systemd unit swift-container-stats_mw-media [15:10:38] [02puppet] 07paladox created 03paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/puppet/commit/eacc5b7b8adf [15:10:38] 02puppet/03paladox-patch-1 07paladox 03eacc5b7 dns: Fix setting query-local-address… [15:10:43] [02puppet] 07paladox opened pull request #4692: dns: Fix setting query-local-address (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4692 [15:10:55] [02puppet] 07coderabbitai[bot] commented on pull request #4692:
[…] 13https://github.com/miraheze/puppet/pull/4692#issuecomment-3755333672 [15:11:46] RECOVERY - mw161 Puppet on mw161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:11:47] RECOVERY - mw172 Puppet on mw172 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:11:49] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/b26c320f15c90633389d864e32e7756f89f01683 [15:11:49] 02puppet/03paladox-patch-1 07paladox 03b26c320 Update dns.pp [15:11:59] RECOVERY - mwtask171 Puppet on mwtask171 is OK: OK: Puppet is currently enabled, last run 19 seconds ago with 0 failures [15:12:11] RECOVERY - swiftobject191 Puppet on swiftobject191 is OK: OK: Puppet is currently enabled, last run 44 seconds ago with 0 failures [15:12:54] RECOVERY - swiftobject201 Puppet on swiftobject201 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:12:54] RECOVERY - mw203 Puppet on mw203 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:13:39] RECOVERY - mwtask161 Puppet on mwtask161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:13:42] RECOVERY - mw173 Puppet on mw173 is OK: OK: Puppet is currently enabled, last run 55 seconds ago with 0 failures [15:14:00] RECOVERY - swiftobject181 Puppet on swiftobject181 is OK: OK: Puppet is currently enabled, last run 17 seconds ago with 0 failures [15:14:03] RECOVERY - swiftac171 Puppet on swiftac171 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures [15:14:34] RECOVERY - mwtask151 Puppet on mwtask151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:15:29] RECOVERY - db161 Puppet on db161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:15:46] RECOVERY - cloud15 Puppet on cloud15 is OK: OK: Puppet is currently enabled, last run 39 seconds ago with 0 failures [15:15:49] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/7d87d98ea6932b6d0b3c1074c0c7a9494fdae7fe [15:15:49] 02puppet/03paladox-patch-1 07paladox 037d87d98 Update dns.pp [15:16:33] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/3d9a75b0d75bf9f26a1688832d7ac24dbfe8c305 [15:16:33] 02puppet/03paladox-patch-1 07paladox 033d9a75b Update recursor.conf.erb [15:17:00] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/83ed1f9f968917eda5d104891066fdd087a12527 [15:17:00] 02puppet/03paladox-patch-1 07paladox 0383ed1f9 Update common.yaml [15:17:19] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/ef8f6413d06c45f58b5b1c5ee177f1909a26415c [15:17:19] 02puppet/03paladox-patch-1 07paladox 03ef8f641 Update bastion.yaml [15:17:26] RECOVERY - cloud16 Puppet on cloud16 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:17:39] RECOVERY - swiftproxy171 Check unit status of swift_dispersion_stats on swiftproxy171 is OK: OK: Status of the systemd unit swift_dispersion_stats [15:18:48] RECOVERY - mw193 Puppet on mw193 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:18:52] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/d57583f7fe88d1fc67308dadf2a4b6b5851fd0e7 [15:18:52] 02puppet/03paladox-patch-1 07paladox 03d57583f Update os202.yaml [15:19:06] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/cf841e714feceb5ea2705d49c0ecde0ab1a0bea5 [15:19:06] 02puppet/03paladox-patch-1 07paladox 03cf841e7 Update db192.yaml [15:19:34] RECOVERY - db171 Puppet on db171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:19:35] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/403ded5a4aa10bb00bdc38987e16e5448e6f8d9e [15:19:35] 02puppet/03paladox-patch-1 07paladox 03403ded5 Update kafka182.yaml [15:19:51] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/edf23b9c92180f8a79a932afcdb4334d8f2df06d [15:19:51] 02puppet/03paladox-patch-1 07paladox 03edf23b9 Update db201.yaml [15:20:07] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/655126267b8ee1500742effaeb4ecc32a5df08f7 [15:20:07] 02puppet/03paladox-patch-1 07paladox 036551262 Update rdb191.yaml [15:20:28] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/39a43cf916df57aa2aa201f422adcdc37401c22e [15:20:28] 02puppet/03paladox-patch-1 07paladox 0339a43cf Update changeprop202.yaml [15:20:45] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/49262a987c48af7fe6a8b6c3d9e580187a0d5eba [15:20:45] 02puppet/03paladox-patch-1 07paladox 0349262a9 Update db192.yaml [15:22:01] RECOVERY - cloud17 Puppet on cloud17 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:22:06] RECOVERY - mw153 Puppet on mw153 is OK: OK: Puppet is currently enabled, last run 53 seconds ago with 0 failures [15:22:22] RECOVERY - mw171 Puppet on mw171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:22:35] RECOVERY - mw191 Puppet on mw191 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:22:58] RECOVERY - cloud19 Puppet on cloud19 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:23:31] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/7b8eed9c740fbc6c53ce12ae059ed355c807481b [15:23:31] 02puppet/03paladox-patch-1 07paladox 037b8eed9 Update recursor.yml.erb [15:26:11] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/9e3062c4e04873541df414f8c02e180ebc8632aa [15:26:12] 02puppet/03paladox-patch-1 07paladox 039e3062c Update recursor.conf.erb [15:27:13] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/1882cf5096d66cad85b10a705d64988b93c83e83 [15:27:13] 02puppet/03paladox-patch-1 07paladox 031882cf5 Update recursor.conf.erb [15:28:32] RECOVERY - mw151 Puppet on mw151 is OK: OK: Puppet is currently enabled, last run 19 seconds ago with 0 failures [15:29:14] RECOVERY - mw163 Puppet on mw163 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:29:15] RECOVERY - db181 Puppet on db181 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:29:24] [02puppet] 07paladox merged pull request #4692: dns: Fix setting query-local-address (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4692 [15:29:25] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/1802b3f38cf2781abe9b93423fa5593bb65faf59 [15:29:26] 02puppet/03main 07paladox 031802b3f dns: Fix setting query-local-address (#4692)… [15:29:27] [02puppet] 07paladox 04deleted 03paladox-patch-1 at 031882cf5 13https://api.github.com/repos/miraheze/puppet/commit/1882cf5 [15:32:18] RECOVERY - db151 Puppet on db151 is OK: OK: Puppet is currently enabled, last run 43 seconds ago with 0 failures [15:32:21] RECOVERY - mw183 Puppet on mw183 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [15:32:33] RECOVERY - mw182 Puppet on mw182 is OK: OK: Puppet is currently enabled, last run 46 seconds ago with 0 failures [15:33:29] RECOVERY - mw152 Puppet on mw152 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:33:39] RECOVERY - mw201 Puppet on mw201 is OK: OK: Puppet is currently enabled, last run 18 seconds ago with 0 failures [15:33:53] RECOVERY - mw181 Puppet on mw181 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:01:34] PROBLEM - cp191 Disk Space on cp191 is WARNING: DISK WARNING - free space: / 45326MiB (10.0% inode=100%); [17:02:00] PROBLEM - matomo151 Check unit status of matomo-archiver-3 on matomo151 is CRITICAL: CRITICAL: Status of the systemd unit matomo-archiver-3 [17:02:29] PROBLEM - matomo151 Check unit status of matomo-archiver-2 on matomo151 is CRITICAL: CRITICAL: Status of the systemd unit matomo-archiver-2 [17:03:05] PROBLEM - matomo151 Check unit status of matomo-archiver-1 on matomo151 is CRITICAL: CRITICAL: Status of the systemd unit matomo-archiver-1 [17:45:08] PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 45238MiB (10.0% inode=100%); [17:47:41] PROBLEM - cp171 Disk Space on cp171 is WARNING: DISK WARNING - free space: / 45284MiB (10.0% inode=100%); [17:58:25] PROBLEM - mwtask181 Disk Space on mwtask181 is WARNING: DISK WARNING - free space: / 22462MiB (10% inode=95%); [18:14:25] RECOVERY - mwtask181 Disk Space on mwtask181 is OK: DISK OK - free space: / 35841MiB (16% inode=95%); [18:20:38] [02mw-config] 07Universal-Omega created 03filemaint (+1 new commit) 13https://github.com/miraheze/mw-config/commit/00490e9bf621 [18:20:38] 02mw-config/03filemaint 07CosmicAlpha 0300490e9 Disable file storage for maintenance [18:20:44] [02mw-config] 07Universal-Omega opened pull request #6236: Disable file storage for maintenance (03main...03filemaint) 13https://github.com/miraheze/mw-config/pull/6236 [18:21:44] miraheze/mw-config - Universal-Omega the build passed. [19:42:44] [02.github] 07dependabot[bot] created 03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0 (+1 new commit) 13https://github.com/miraheze/.github/commit/609c4f4d2877 [19:42:46] 02.github/03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0 07dependabot[bot] 03609c4f4 Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0… [19:42:48] [02.github] 07dependabot[bot] opened pull request #83: Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0 (03main...03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0) 13https://github.com/miraheze/.github/pull/83 [19:42:50] [02.github] 07dependabot[bot] added the label 'github_actions' to pull request #83 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/.github/pull/83 [19:42:52] [02.github] 07dependabot[bot] added the label 'dependencies' to pull request #83 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/.github/pull/83 [19:42:54] [02.github] 07dependabot[bot] added the label 'dependencies' to pull request #83 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/.github/pull/83 [19:42:56] [02.github] 07dependabot[bot] added the label 'github_actions' to pull request #83 (Bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/.github/pull/83 [19:42:58] [02.github] 07coderabbitai[bot] commented on pull request #83: --- […] 13https://github.com/miraheze/.github/pull/83#issuecomment-3756565660 [20:00:06] [02mw-config] 07paladox merged 07Universal-Omega's pull request #6236: Disable file storage for maintenance (03main...03filemaint) 13https://github.com/miraheze/mw-config/pull/6236 [20:00:06] [02mw-config] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/a0fa15b8b2327cf9cb87ae137a0e9540c4f74457 [20:00:06] 02mw-config/03main 07CosmicAlpha 03a0fa15b Disable file storage for maintenance (#6236) [20:00:08] [02mw-config] 07paladox 04deleted 03filemaint at 0300490e9 13https://api.github.com/repos/miraheze/mw-config/commit/00490e9 [20:00:18] !log [paladox@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [20:00:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:00:30] !log upgrading swift* to trixie (stopping services) [20:00:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:00:47] !log [paladox@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 30s [20:00:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:01:34] miraheze/mw-config - paladox the build passed. [21:10:14] @orduin hi [21:10:17] are you around? [21:10:32] yup, what's up? [21:11:27] see dm pls [21:55:29] PROBLEM - mw201 HTTPS on mw201 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:55:52] PROBLEM - mw163 MediaWiki Rendering on mw163 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:56:14] PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 9 backends are down. mw171 mw172 mw181 mw182 mw153 mw183 mw193 mw202 mw203 [21:56:18] PROBLEM - mw201 MediaWiki Rendering on mw201 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.010 second response time [21:56:24] uhh [21:56:29] we're down it seems [21:56:40] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 10 backends are down. mw152 mw161 mw162 mw153 mw173 mw183 mw192 mw193 mw202 mw203 [21:56:40] PROBLEM - mw171 HTTPS on mw171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:56:49] PROBLEM - cp161 Varnish Backends on cp161 is CRITICAL: 13 backends are down. mw152 mw161 mw162 mw172 mw181 mw182 mw153 mw173 mw183 mw192 mw193 mw202 mw203 [21:57:03] PROBLEM - mw163 HTTPS on mw163 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:57:15] PROBLEM - cp161 HTTPS on cp161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [21:57:18] PROBLEM - test151 MediaWiki Rendering on test151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:57:19] PROBLEM - cp171 HTTPS on cp171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [21:57:20] Uh oh [21:57:31] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 10 backends are down. mw151 mw171 mw172 mw181 mw182 mw153 mw183 mw191 mw202 mw203 [21:57:46] PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push [21:58:08] test and prod again [21:58:10] Ffs [21:58:20] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is WARNING: WARNING - NGINX Error Rate is 55% [21:58:46] RECOVERY - mw171 HTTPS on mw171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4254 bytes in 9.876 second response time [21:59:04] PROBLEM - mw182 MediaWiki Rendering on mw182 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [21:59:15] mw171 is back [21:59:53] RECOVERY - mw163 MediaWiki Rendering on mw163 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 3.480 second response time [22:00:18] RECOVERY - mw201 MediaWiki Rendering on mw201 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.184 second response time [22:00:40] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [22:00:49] RECOVERY - cp161 Varnish Backends on cp161 is OK: All 31 backends are healthy [22:01:01] RECOVERY - mw182 MediaWiki Rendering on mw182 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.197 second response time [22:01:03] RECOVERY - mw163 HTTPS on mw163 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4254 bytes in 0.057 second response time [22:01:10] RECOVERY - cp161 HTTPS on cp161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4357 bytes in 0.059 second response time [22:01:17] RECOVERY - test151 MediaWiki Rendering on test151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.635 second response time [22:01:21] RECOVERY - cp171 HTTPS on cp171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4303 bytes in 0.064 second response time [22:01:24] RECOVERY - mw201 HTTPS on mw201 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4254 bytes in 0.057 second response time [22:01:31] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [22:02:14] RECOVERY - cp191 Varnish Backends on cp191 is OK: All 31 backends are healthy [22:02:20] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 14% [22:05:46] RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push [23:00:27] PROBLEM - swiftproxy161 HTTP on swiftproxy161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [23:00:27] PROBLEM - swiftproxy171 HTTPS on swiftproxy171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [23:00:27] PROBLEM - swiftproxy171 HTTP on swiftproxy171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [23:01:22] PROBLEM - swiftproxy161 HTTPS on swiftproxy161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [23:13:10] [02python-functions] 07dependabot[bot] created 03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0 (+1 new commit) 13https://github.com/miraheze/python-functions/commit/4b5440d31a8a [23:13:10] 02python-functions/03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0 07dependabot[bot] 034b5440d build(deps): bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0… [23:13:10] [02python-functions] 07dependabot[bot] added the label 'dependencies' to pull request #204 (build(deps): bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/python-functions/pull/204 [23:13:10] [02python-functions] 07dependabot[bot] opened pull request #204: build(deps): bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0 (03main...03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0) 13https://github.com/miraheze/python-functions/pull/204 [23:13:12] [02python-functions] 07dependabot[bot] added the label 'github_actions' to pull request #204 (build(deps): bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/python-functions/pull/204 [23:13:14] [02python-functions] 07dependabot[bot] added the label 'dependencies' to pull request #204 (build(deps): bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/python-functions/pull/204 [23:13:16] [02python-functions] 07dependabot[bot] added the label 'github_actions' to pull request #204 (build(deps): bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0) 13https://github.com/miraheze/python-functions/pull/204 [23:13:19] [02python-functions] 07coderabbitai[bot] commented on pull request #204: --- […] 13https://github.com/miraheze/python-functions/pull/204#issuecomment-3757325945 [23:16:26] [02python-functions] 07github-merge-queue[bot] created 03gh-readonly-queue/main/pr-204-041ad01ad059a1271775dbbcaec4628b05bb0054 (+1 new commit) 13https://github.com/miraheze/python-functions/commit/edb5fda31f07 [23:16:28] 02python-functions/03gh-readonly-queue/main/pr-204-041ad01ad059a1271775dbbcaec4628b05bb0054 07dependabot[bot] 03edb5fda build(deps): bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0 (#204)… [23:16:34] miraheze/python-functions - dependabot[bot] the build passed. [23:19:16] [02python-functions] 07github-merge-queue[bot] merged 03gh-readonly-queue/main/pr-204-041ad01ad059a1271775dbbcaec4628b05bb0054 into 03main 13https://github.com/miraheze/python-functions/compare/041ad01ad059...edb5fda31f07 [23:19:17] [02python-functions] 07github-actions[bot] merged 07dependabot[bot]'s pull request #204: build(deps): bump styfle/cancel-workflow-action from 0.12.1 to 0.13.0 (03main...03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0) 13https://github.com/miraheze/python-functions/pull/204 [23:19:17] [02python-functions] 07github-merge-queue[bot] 04deleted 03gh-readonly-queue/main/pr-204-041ad01ad059a1271775dbbcaec4628b05bb0054 at 03edb5fda 13https://api.github.com/repos/miraheze/python-functions/commit/edb5fda [23:19:18] [02python-functions] 07github-actions[bot] 04deleted 03dependabot/github_actions/styfle/cancel-workflow-action-0.13.0 at 034b5440d 13https://api.github.com/repos/miraheze/python-functions/commit/4b5440d [23:22:25] miraheze/python-functions - github-merge-queue[bot] the build passed. [23:23:46] RECOVERY - swiftproxy171 HTTPS on swiftproxy171 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.031 second response time [23:24:23] RECOVERY - swiftproxy161 HTTP on swiftproxy161 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.023 second response time [23:25:21] RECOVERY - swiftproxy161 HTTPS on swiftproxy161 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 1.520 second response time [23:25:49] RECOVERY - swiftproxy171 HTTP on swiftproxy171 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.015 second response time [23:58:07] PROBLEM - swiftobject171 NTP time on swiftobject171 is WARNING: NTP WARNING: Offset 0.4768909514 secs