[00:00:20] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/7c6b187de8dbf3e484d065ca196d308ed3f12aa1 [00:00:20] 02ssl/03main 07WikiTideBot 037c6b187 Bot: Auto-update domain lists [00:01:40] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/bb430ccf1500283d9bd5dfe06a02348cd0da9674 [00:01:40] 02statichelp/03main 07WikiTideBot 03bb430cc Bot: Auto-update Tech namespace pages 2026-09-03 00:01:37 [02:35:58] PROBLEM - mw173 Current Load on mw173 is WARNING: LOAD WARNING - total load average: 12.65, 20.90, 12.92 [02:37:54] RECOVERY - mw173 Current Load on mw173 is OK: LOAD OK - total load average: 5.97, 15.71, 11.96 [02:38:16] !log [wwr@mwtask151] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php importDump --wiki=kamenriderwiki --no-updates --username-prefix=wikia:kamenrider -- kamenrider_pages_full.xml (END - exit=0) [02:38:17] !log [wwr@mwtask151] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php rebuildall --wiki=kamenriderwiki (START) [02:38:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:38:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:01:09] RECOVERY - db161 Check unit status of sql-backup on db161 is OK: OK: Status of the systemd unit sql-backup [04:21:27] [02ManageWiki] 07lihaohong6 left a file comment in pull request #789 0354f9a3d: This is now done. Not sure if the implementation is sufficiently generic but it works for extensions. 13https://github.com/miraheze/ManageWiki/pull/789#discussion_r3920989873 [04:25:11] [02ManageWiki] 07lihaohong6 left a file comment in pull request #789 0354f9a3d: Resolved by adding an empty check in `permissions` 13https://github.com/miraheze/ManageWiki/pull/789#discussion_r3921006617 [04:25:53] miraheze/ManageWiki - lihaohong6 the build passed. [04:53:05] [02mw-config] 07lihaohong6 left a file comment in pull request #6520 038cbf7a8: This is currently deduplicated through https://github.com/miraheze/MirahezeMagic/blob/5697d92/includes/HookHandlers/ContentSecurityPolicy.php#L29 and once MH Magic is changed the new version would have a similar line. 13https://github.com/miraheze/mw-config/pull/6520#discussion_r3921150627 [04:58:24] [02mw-config] 07lihaohong6 left a file comment in pull request #6520 038cbf7a8: Figuring out the right granularity is hard since connect-src is more risky than img-src, but at the same time having too many services will add bloat to ManageWiki options. There was a split for Wikimedia wikis but I'm not sure if the same should be […] 13https://github.com/miraheze/mw-config/pull/6520#discussion_r3921182127 [05:13:47] [02MirahezeMagic] 07lihaohong6 drafted pull request #666: T15205: allow CSP entries to be toggled via ManageWiki (07miraheze:03main...07lihaohong6:03csp2) 13https://github.com/miraheze/MirahezeMagic/pull/666 [05:19:36] miraheze/MirahezeMagic - lihaohong6 the build passed. [10:10:15] RECOVERY - db151 Check unit status of sql-backup on db151 is OK: OK: Status of the systemd unit sql-backup [11:00:21] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/de79cf7e787977558b9a786eb885d10b8954b48b [11:00:21] 02ssl/03main 07WikiTideBot 03de79cf7 Bot: Auto-update domain lists [11:15:54] [02puppet] 07paladox created 03paladox-patch-2 (+1 new commit) 13https://github.com/miraheze/puppet/commit/c948094c7c36 [11:15:54] 02puppet/03paladox-patch-2 07paladox 03c948094 db201: Disable db backup… [11:15:58] [02puppet] 07paladox opened pull request #5025: db201: Disable db backup (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5025 [11:17:23] [02puppet] 07paladox merged pull request #5025: db201: Disable db backup (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5025 [11:17:24] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/df458e1441b373e8bccf0804ab902af8362fe508 [11:17:24] 02puppet/03main 07paladox 03df458e1 db201: Disable db backup (#5025)… [11:17:25] [02puppet] 07paladox 04deleted 03paladox-patch-2 at 03c948094 13https://api.github.com/repos/miraheze/puppet/commit/c948094 [11:27:38] RECOVERY - db181 Check unit status of sql-backup on db181 is OK: OK: Status of the systemd unit sql-backup [11:58:27] [02puppet] 07paladox created 03paladox-patch-2 (+1 new commit) 13https://github.com/miraheze/puppet/commit/167ce0a16a09 [11:58:27] 02puppet/03paladox-patch-2 07paladox 03167ce0a backup: in wikitide-backup don't strip output in find_backups for sql and mediawiki-xml [11:58:33] [02puppet] 07paladox opened pull request #5026: backup: in wikitide-backup don't strip output in find_backups for sql and mediawiki-xml (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5026 [12:00:18] [02puppet] 07paladox merged pull request #5026: backup: in wikitide-backup don't strip output in find_backups for sql and mediawiki-xml (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5026 [12:00:20] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/ac70569f6809540aa7165e835a453e8bf01a0995 [12:00:22] [02puppet] 07paladox 04deleted 03paladox-patch-2 at 03167ce0a 13https://api.github.com/repos/miraheze/puppet/commit/167ce0a [12:00:24] 02puppet/03main 07paladox 03ac70569 backup: in wikitide-backup don't strip output in find_backups for sql and mediawiki-xml (#5026) [12:05:23] [02puppet] 07paladox created 03paladox-patch-2 (+1 new commit) 13https://github.com/miraheze/puppet/commit/8746f21c35b8 [12:05:23] 02puppet/03paladox-patch-2 07paladox 038746f21 backup: Fix finding objects using a prefix… [12:05:27] [02puppet] 07paladox opened pull request #5027: backup: Fix finding objects using a prefix (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5027 [12:10:15] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-2 13https://github.com/miraheze/puppet/commit/ad062ade2d343145abb3732ad12767c351836c55 [12:10:16] 02puppet/03paladox-patch-2 07paladox 03ad062ad Update wikitide-backup.py.epp [12:13:25] [02GlobalNewFiles] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/GlobalNewFiles/commit/cf2beb6b846cf951e5a84ca16d33be5c230430bf [12:13:25] 02GlobalNewFiles/03main 07translatewiki.net 03cf2beb6 Localisation updates from https://translatewiki.net. [12:13:26] [02ImportDump] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/ImportDump/commit/c641db822a217992030cb282d1d2c1bb5d84f903 [12:13:26] 02ImportDump/03main 07translatewiki.net 03c641db8 Localisation updates from https://translatewiki.net. [12:13:28] [02CreateWiki] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/CreateWiki/commit/433a4650a4ff39759509600d725b4fcef5d36c64 [12:13:30] 02CreateWiki/03main 07translatewiki.net 03433a465 Localisation updates from https://translatewiki.net. [12:13:32] [02MatomoAnalytics] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/MatomoAnalytics/commit/071a8d479e7e0631439221e1521f47ec706a9b50 [12:13:34] [02ManageWiki] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/ManageWiki/commit/ea37334ae712b7f4a2ec40558d544d544eab2d34 [12:13:36] 02MatomoAnalytics/03main 07translatewiki.net 03071a8d4 Localisation updates from https://translatewiki.net. [12:13:37] 02ManageWiki/03main 07translatewiki.net 03ea37334 Localisation updates from https://translatewiki.net. [12:17:40] miraheze/GlobalNewFiles - translatewiki the build passed. [12:18:16] miraheze/MatomoAnalytics - translatewiki the build passed. [12:18:21] miraheze/ImportDump - translatewiki the build passed. [12:18:35] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-2 13https://github.com/miraheze/puppet/commit/fc6fe61a282f3d16ec6cc8b0ab4b45e495a7b21d [12:18:35] 02puppet/03paladox-patch-2 07paladox 03fc6fe61 Update wikitide-backup.py.epp [12:19:13] miraheze/ManageWiki - translatewiki the build passed. [12:19:25] [02puppet] 07paladox merged pull request #5027: backup: Fix finding objects using a prefix (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5027 [12:19:26] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/6976543160ee41207bfd994a167fb268b640f354 [12:19:27] 02puppet/03main 07paladox 036976543 backup: Fix finding objects using a prefix (#5027)… [12:19:29] [02puppet] 07paladox 04deleted 03paladox-patch-2 at 03fc6fe61 13https://api.github.com/repos/miraheze/puppet/commit/fc6fe61 [12:20:38] miraheze/CreateWiki - translatewiki the build passed. [12:30:21] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/bf5d861cf88032125047e6bf6dc5c48e223fb44a [12:30:21] 02ssl/03main 07WikiTideBot 03bf5d861 Bot: Auto-update domain lists [12:35:27] [02puppet] 07paladox created 03paladox-patch-2 (+1 new commit) 13https://github.com/miraheze/puppet/commit/803095785ad7 [12:35:27] 02puppet/03paladox-patch-2 07paladox 038030957 backup: Better ability to download all files based on --prefix [12:35:32] [02puppet] 07paladox opened pull request #5028: backup: Better ability to download all files based on --prefix (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5028 [12:37:03] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-2 13https://github.com/miraheze/puppet/commit/540173b574aa25b0533be7bb8bf8dd70532b48c3 [12:37:03] 02puppet/03paladox-patch-2 07paladox 03540173b Update wikitide-backup.py.epp [12:37:19] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/7bafef600c722612273c7ad038f4a475543014a2 [12:37:21] 02puppet/03main 07paladox 037bafef6 backup: Better ability to download all files based on --prefix (#5028) [12:37:23] [02puppet] 07paladox merged pull request #5028: backup: Better ability to download all files based on --prefix (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5028 [12:37:25] [02puppet] 07paladox 04deleted 03paladox-patch-2 at 03540173b 13https://api.github.com/repos/miraheze/puppet/commit/540173b [12:39:50] [02puppet] 07paladox created 03paladox-patch-2 (+1 new commit) 13https://github.com/miraheze/puppet/commit/ed2b017bff99 [12:39:50] 02puppet/03paladox-patch-2 07paladox 03ed2b017 backup: Missed updating s3_download [12:39:53] [02puppet] 07paladox opened pull request #5029: backup: Missed updating s3_download (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5029 [12:40:24] [02puppet] 07paladox merged pull request #5029: backup: Missed updating s3_download (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5029 [12:40:26] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/4c8c504ae875e466ca989c1a257c53d0ce312232 [12:40:28] [02puppet] 07paladox 04deleted 03paladox-patch-2 at 03ed2b017 13https://api.github.com/repos/miraheze/puppet/commit/ed2b017 [12:40:30] 02puppet/03main 07paladox 034c8c504 backup: Missed updating s3_download (#5029) [12:45:31] [02puppet] 07paladox created 03paladox-patch-2 (+1 new commit) 13https://github.com/miraheze/puppet/commit/cfa188268af8 [12:45:31] 02puppet/03paladox-patch-2 07paladox 03cfa1882 backup: Fix [12:45:35] [02puppet] 07paladox opened pull request #5030: backup: Fix (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5030 [12:45:57] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-2 13https://github.com/miraheze/puppet/commit/d4f8d2ebe25f8b7beaad3cbfc87f8a6a34358465 [12:45:58] 02puppet/03paladox-patch-2 07paladox 03d4f8d2e Update wikitide-backup.py.epp [12:49:03] [02puppet] 07paladox retitled PR #5030: "backup: Fix" ➜ "backup: Fix downloading objects under a prefix" 13https://github.com/miraheze/puppet/pull/5030 [12:49:06] [02puppet] 07paladox merged pull request #5030: backup: Fix downloading objects under a prefix (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5030 [12:49:06] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/1ed35177412481a9b8dc9f353718d134a870a5f5 [12:49:06] 02puppet/03main 07paladox 031ed3517 backup: Fix (#5030) [12:49:08] [02puppet] 07paladox 04deleted 03paladox-patch-2 at 03d4f8d2e 13https://api.github.com/repos/miraheze/puppet/commit/d4f8d2e [12:53:52] RECOVERY - mwtask181 Check unit status of mediawiki_job_generate-sitemap-index on mwtask181 is OK: OK: Status of the systemd unit mediawiki_job_generate-sitemap-index [15:46:02] [02puppet] 07paladox created 03paladox-patch-2 (+1 new commit) 13https://github.com/miraheze/puppet/commit/75c6709e728e [15:46:02] 02puppet/03paladox-patch-2 07paladox 0375c6709 Run mhglobal backup daily [15:46:07] [02puppet] 07paladox opened pull request #5031: Run mhglobal backup daily (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5031 [15:46:43] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-2 13https://github.com/miraheze/puppet/commit/96e59931e5e24acf92b9bc2c0ddca0fa14115a36 [15:46:44] 02puppet/03paladox-patch-2 07paladox 0396e5993 Update db192.yaml with daily_misc role [15:49:40] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-2 13https://github.com/miraheze/puppet/commit/5247ef0dd72b98e36f41b9cdca58692445842361 [15:49:40] 02puppet/03paladox-patch-2 07paladox 035247ef0 Update lifecycle.json [15:49:49] [02puppet] 07paladox merged pull request #5031: Run mhglobal backup daily (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5031 [15:49:50] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/47e1a022c58a4220852e4241d0ba412eeb98150d [15:49:51] 02puppet/03main 07paladox 0347e1a02 Run mhglobal backup daily (#5031) [15:49:53] [02puppet] 07paladox 04deleted 03paladox-patch-2 at 035247ef0 13https://api.github.com/repos/miraheze/puppet/commit/5247ef0 [15:51:34] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/4569c865c98f7811d7915c2a7d415777af9006ae [15:51:34] 02puppet/03main 07paladox 034569c86 Update lifecycle.json [16:00:46] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/262d742957c566ea263587d5bd332b2b7bf8a9d8 [16:00:46] 02puppet/03main 07paladox 03262d742 Fix [16:32:54] [02CreateWiki] 07Universal-Omega requested changes on pull request #852: An initial review but could still be a little more review later. 13https://github.com/miraheze/CreateWiki/pull/852#pullrequestreview-5069591621 [16:32:55] [02CreateWiki] 07Universal-Omega left a file comment in pull request #852 03ae0cea7: I think we can use createChild here? But overall would it make more sense to use WikiImporterFactory here or not? maybe not unless this were made an actual job instead of a script where it is easier to call child scripts in scripts, but it also may […] 13https://github.com/miraheze/CreateWiki/pull/852#discussion_r3897021739 [16:32:55] [02CreateWiki] 07Universal-Omega left a file comment in pull request #852 03ae0cea7: We don't need FakeMaintenance here, as this itself is maint, we can use $this->createChild. 13https://github.com/miraheze/CreateWiki/pull/852#discussion_r3926330170 [16:32:55] [02CreateWiki] 07Universal-Omega left a file comment in pull request #852 03ae0cea7: Probably a left over from ImportDump, for this one didn't this is a script itself, we can probably use createChild( InitSiteStats::class ) with options for this and below SiteStatsUpdate stuff. If we do that, we can also remove $dbw above as well. 13https://github.com/miraheze/CreateWiki/pull/852#discussion_r3926350352 [16:32:57] [02CreateWiki] 07Universal-Omega left a file comment in pull request #852 03ae0cea7: Please do it like: […] 13https://github.com/miraheze/CreateWiki/pull/852#discussion_r3926292522 [16:32:59] [02CreateWiki] 07Universal-Omega left a file comment in pull request #852 03ae0cea7: This seems kinda redundant to me? I think we can just call this directly rather than another method for it here? 13https://github.com/miraheze/CreateWiki/pull/852#discussion_r3926405054 [16:33:01] [02CreateWiki] 07Universal-Omega left a file comment in pull request #852 03ae0cea7: Sounds like a recipe for a race condition? 13https://github.com/miraheze/CreateWiki/pull/852#discussion_r3926414494 [16:33:03] [02CreateWiki] 07Universal-Omega left a file comment in pull request #852 03ae0cea7: 1. Not positive if fatalError is best here or not, probably fine though […] 13https://github.com/miraheze/CreateWiki/pull/852#discussion_r3926495873 [16:33:05] [02CreateWiki] 07Universal-Omega left a file comment in pull request #852 03ae0cea7: For this and other logs, maybe add what wiki? 13https://github.com/miraheze/CreateWiki/pull/852#discussion_r3926498384 [16:33:07] [02CreateWiki] 07Universal-Omega left a file comment in pull request #852 03ae0cea7: Should check for MainConfigNames::DisableSearchUpdate 13https://github.com/miraheze/CreateWiki/pull/852#discussion_r3926463649 [17:30:23] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/b457bea2b9b50fa5762c6eff627c572ba5e1b171 [17:30:23] 02ssl/03main 07WikiTideBot 03b457bea Bot: Auto-update domain lists [18:14:41] !log [wwr@mwtask151] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php rebuildall --wiki=kamenriderwiki (END - exit=0) [18:14:42] !log [wwr@mwtask151] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php initEditCount --wiki=kamenriderwiki (START) [18:14:43] !log [wwr@mwtask151] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php initEditCount --wiki=kamenriderwiki (END - exit=0) [18:14:44] !log [wwr@mwtask151] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php initSiteStats --wiki=kamenriderwiki --update (START) [18:14:46] !log [wwr@mwtask151] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php initSiteStats --wiki=kamenriderwiki --update (END - exit=0) [18:14:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:14:47] !log [wwr@mwtask151] Finished import for kamenriderwiki (XML: kamenrider_pages_full.xml; Images: None) (END - exit=0) [18:14:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:14:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:15:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:15:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:15:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:34:25] [02puppet] 07paladox created 03paladox-patch-2 (+1 new commit) 13https://github.com/miraheze/puppet/commit/14cf288e7a0a [18:34:25] 02puppet/03paladox-patch-2 07paladox 0314cf288 DO NOT MERGE… [18:34:29] [02puppet] 07paladox opened pull request #5032: DO NOT MERGE (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5032 [18:42:19] [02puppet] 07paladox closed pull request #5032: DO NOT MERGE (03main...03paladox-patch-2) 13https://github.com/miraheze/puppet/pull/5032 [18:42:22] [02puppet] 07paladox 04deleted 03paladox-patch-2 at 0314cf288 13https://api.github.com/repos/miraheze/puppet/commit/14cf288 [18:43:48] don't tempt me like that... [22:43:48] PROBLEM - cp191 Current Load on cp191 is CRITICAL: LOAD CRITICAL - total load average: 17.70, 16.28, 7.89 [22:45:48] RECOVERY - cp191 Current Load on cp191 is OK: LOAD OK - total load average: 3.95, 11.46, 7.14 [22:47:55] PROBLEM - cp161 HTTPS on cp161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:48:00] PROBLEM - db151 Current Load on db151 is CRITICAL: LOAD CRITICAL - total load average: 22.83, 12.92, 6.30 [22:48:33] PROBLEM - mw171 HTTPS on mw171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:48:33] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 6 backends are down. mw151 mw162 mw182 mw193 mw202 mw203 [22:48:36] PROBLEM - mw163 MediaWiki Rendering on mw163 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.012 second response time [22:48:36] PROBLEM - mw161 HTTPS on mw161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:48:37] PROBLEM - cp171 Current Load on cp171 is CRITICAL: LOAD CRITICAL - total load average: 39.15, 29.73, 15.15 [22:48:51] PROBLEM - cp201 HTTPS on cp201 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10006 milliseconds with 0 bytes received [22:48:52] PROBLEM - cp191 HTTPS on cp191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10005 milliseconds with 0 bytes received [22:48:56] PROBLEM - mw173 HTTPS on mw173 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:48:57] PROBLEM - cp201 Current Load on cp201 is CRITICAL: LOAD CRITICAL - total load average: 21.55, 18.53, 10.78 [22:49:09] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 9 backends are down. mw152 mw162 mw171 mw172 mw183 mw191 mw192 mw193 mw201 [22:49:19] PROBLEM - mw181 HTTPS on mw181 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:49:23] PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 9 backends are down. mw171 mw182 mw153 mw191 mw192 mw193 mw201 mw202 mw203 [22:49:24] PROBLEM - cp171 HTTPS on cp171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:49:37] PROBLEM - mw161 MediaWiki Rendering on mw161 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.012 second response time [22:49:42] PROBLEM - cp161 Varnish Backends on cp161 is CRITICAL: 7 backends are down. mw151 mw172 mw182 mw153 mw193 mw202 mw203 [22:49:48] PROBLEM - cp191 Current Load on cp191 is CRITICAL: LOAD CRITICAL - total load average: 25.22, 28.50, 16.26 [22:50:14] PROBLEM - mw202 HTTPS on mw202 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:50:33] RECOVERY - mw171 HTTPS on mw171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.281 second response time [22:50:40] RECOVERY - mw163 MediaWiki Rendering on mw163 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 3.929 second response time [22:50:50] PROBLEM - mw163 Current Load on mw163 is WARNING: LOAD WARNING - total load average: 23.99, 17.01, 10.57 [22:50:54] RECOVERY - mw173 HTTPS on mw173 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 1.937 second response time [22:50:56] RECOVERY - cp201 HTTPS on cp201 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1485 bytes in 7.093 second response time [22:51:09] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 32 backends are healthy [22:51:19] RECOVERY - cp171 HTTPS on cp171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 0.088 second response time [22:51:19] RECOVERY - mw181 HTTPS on mw181 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.077 second response time [22:51:23] RECOVERY - cp191 Varnish Backends on cp191 is OK: All 32 backends are healthy [22:51:37] RECOVERY - cp161 Varnish Backends on cp161 is OK: All 32 backends are healthy [22:51:37] RECOVERY - mw161 MediaWiki Rendering on mw161 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.182 second response time [22:51:55] RECOVERY - cp161 HTTPS on cp161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1518 bytes in 0.095 second response time [22:51:57] PROBLEM - db151 Current Load on db151 is WARNING: LOAD WARNING - total load average: 3.67, 10.48, 7.11 [22:52:11] RECOVERY - mw202 HTTPS on mw202 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.075 second response time [22:52:27] RECOVERY - mw161 HTTPS on mw161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.070 second response time [22:52:33] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 32 backends are healthy [22:52:49] RECOVERY - cp191 HTTPS on cp191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 0.073 second response time [22:52:50] RECOVERY - mw163 Current Load on mw163 is OK: LOAD OK - total load average: 8.80, 14.01, 10.29 [22:52:57] PROBLEM - cp201 Current Load on cp201 is WARNING: LOAD WARNING - total load average: 5.13, 13.75, 10.88 [22:53:56] RECOVERY - db151 Current Load on db151 is OK: LOAD OK - total load average: 1.38, 7.47, 6.40 [22:54:37] RECOVERY - cp171 Current Load on cp171 is OK: LOAD OK - total load average: 1.80, 11.31, 11.58 [22:54:57] RECOVERY - cp201 Current Load on cp201 is OK: LOAD OK - total load average: 2.66, 9.87, 9.81 [22:55:48] RECOVERY - cp191 Current Load on cp191 is OK: LOAD OK - total load average: 3.03, 11.96, 12.91 [23:05:49] PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 10.89, 14.15, 7.71 [23:07:49] RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 2.90, 10.11, 7.01 [23:38:54] PROBLEM - db161 mariadb_memory on db161 is CRITICAL: CRIT Memory 96% used. Largest process: mariadbd (501635) = 94.4% [23:39:49] PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 15.72, 10.96, 6.87 [23:41:49] RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 4.36, 8.50, 6.48