[00:05:54] [1/2] Depooling db181 [00:05:54] [2/2] @cosmicalpha Want to check if your mitigation is working? [00:06:01] oh wait it wasn't db161 [00:06:01] https://cdn.discordapp.com/attachments/808001911868489748/1565154706911666186/image.png?ex=6ae3df69&is=6ae28de9&hm=b97a18fccf0fc17cb6d2fe0acd367270ab04fe73646a59e5f8b80504c1ad32d8& [00:06:15] sorry db161 I may have just slandered your good name in #general because I assumed it was db161 [00:06:30] !log [petramagna@mwtask181] Starting deploy of {'config': True, 'force': True} to all [00:06:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029000630-c291307c [00:07:06] !log [petramagna@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 36s [00:07:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029000706-54c0b479 [00:09:01] Repooling [00:09:07] !log [petramagna@mwtask181] Starting deploy of {'config': True, 'force': True} to all [00:09:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029000907-c291307c [00:09:30] !log [petramagna@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 23s [00:09:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029000930-00655a67 [00:10:30] that's been awfully well-behaved recently [00:10:42] not even taken the farm down in at least a few days [00:10:53] yeah but it told every other brother/sister db server [00:11:07] I've seen db151, 171 and 181 throw a hissy fit recently [00:25:08] miraheze/puppet - Universal-Omega the build passed. [00:25:18] miraheze/puppet - Universal-Omega the build passed. [00:25:22] miraheze/puppet - Universal-Omega the build passed. [00:25:57] miraheze/puppet - Universal-Omega the build passed. [00:26:40] miraheze/puppet - Universal-Omega the build passed. [00:26:58] miraheze/puppet - Universal-Omega the build passed. [00:27:36] miraheze/puppet - Universal-Omega the build passed. [00:27:39] miraheze/puppet - Universal-Omega the build passed. [00:29:01] miraheze/puppet - Universal-Omega the build passed. [00:29:58] PROBLEM - db181 Current Load on db181 is WARNING: LOAD WARNING - total load average: 2.68, 4.89, 10.90 [00:31:58] RECOVERY - db181 Current Load on db181 is OK: LOAD OK - total load average: 1.77, 3.87, 9.80 [00:35:09] miraheze/puppet - Universal-Omega the build passed. [00:36:52] miraheze/puppet - Universal-Omega the build passed. [00:37:32] miraheze/puppet - Universal-Omega the build passed. [00:38:24] miraheze/puppet - Universal-Omega the build passed. [00:39:18] miraheze/puppet - Universal-Omega the build passed. [00:43:20] PROBLEM - mon181 Puppet on mon181 is WARNING: WARNING: Puppet is currently disabled, message: Universal Omega, last run 9 minutes ago with 0 failures [00:44:03] PROBLEM - bots171 Puppet on bots171 is WARNING: WARNING: Puppet is currently disabled, message: Universal Omega, last run 20 minutes ago with 0 failures [00:50:50] There does not appear to have been a specific cause this time that I can see. [00:50:56] Everything looks fairly normal in logs. [00:51:31] Either way by the end of this weekend I will finish my work to prevent one DB server from taking the whole farm down. [00:53:26] FYI ircecho (icinga-miraheze) will be down for a bit while I attempt to deploy a new version of it, merging taskbot with it and deploying it on bots171 also. [00:57:06] miraheze/puppet - Universal-Omega the build passed. [00:57:20] RECOVERY - mon181 Puppet on mon181 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [01:00:34] https://cdn.discordapp.com/attachments/808001911868489748/1565168431567732896/db161.png?ex=6ae3ec31&is=6ae29ab1&hm=bf7c1ebe2df3cd70153edaa6540461954e492dbb0bcb7689c1c871b5c485c130& [01:01:03] PROBLEM - mon181 IRCEcho on mon181 is UNKNOWN: NRPE: Command 'check_IRCEcho' not defined [01:01:37] PROBLEM - bots171 Icinga Task Bot Sync on bots171 is UNKNOWN: NRPE: Command 'check_Icinga Task Bot Sync' not defined [01:01:47] [Grafana] FIRING: Notification test [01:02:34] PROBLEM - bots171 Icinga Task Bot on bots171 is UNKNOWN: NRPE: Command 'check_Icinga Task Bot' not defined [01:04:08] RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 86434MiB (19.1% inode=100%); [01:08:30] miraheze/puppet - Universal-Omega the build passed. [01:09:51] miraheze/puppet - Universal-Omega the build passed. [01:11:55] miraheze/puppet - Universal-Omega the build passed. [01:20:41] !log [somerandomdeveloper@test151] Starting deploy of {'versions': 'all', 'upgrade_extensions': 'PageForms'} to test151 (T15578) [01:20:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029012041-c1859b3f [01:21:03] !log [somerandomdeveloper@test151] finished deploy of {'versions': 'all', 'upgrade_extensions': 'PageForms'} to test151 - SUCCESS in 21s (T15578) [01:21:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029012103-069f741c [01:21:13] !log [somerandomdeveloper@mwtask181] Starting deploy of {'versions': 'all', 'upgrade_extensions': 'PageForms'} to all (T15578) [01:21:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029012113-c5abdd61 [01:21:46] miraheze/puppet - Universal-Omega the build passed. [01:22:02] !log [somerandomdeveloper@mwtask181] finished deploy of {'versions': 'all', 'upgrade_extensions': 'PageForms'} to all - SUCCESS in 49s (T15578) [01:22:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029012202-3b413056 [01:27:45] miraheze/puppet - Universal-Omega the build passed. [01:29:03] miraheze/puppet - Universal-Omega the build has errored. [01:29:52] miraheze/puppet - Universal-Omega the build has errored. [01:30:33] miraheze/puppet - Universal-Omega the build has errored. [01:30:47] miraheze/puppet - Universal-Omega the build has errored. [01:31:31] miraheze/puppet - Universal-Omega the build passed. [01:32:11] miraheze/puppet - Universal-Omega the build passed. [01:38:42] miraheze/puppet - Universal-Omega the build passed. [01:54:16] miraheze/dns - Universal-Omega the build passed. [02:37:41] PROBLEM - bast181 Puppet on bast181 is WARNING: WARNING: Puppet is currently disabled, message: Universal Omega, last run 4 minutes ago with 0 failures [02:39:40] RECOVERY - bast181 Puppet on bast181 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:45:20] !log [universalomega@mattermost2] Test [02:45:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029024520-a2fb67c1 [02:48:51] miraheze/dns - Universal-Omega the build passed. [03:59:01] PROBLEM - bots171 Check unit status of pywikibot-metawiki-archivebot-job on bots171 is CRITICAL: CRITICAL: Status of the systemd unit pywikibot-metawiki-archivebot-job [05:55:41] PROBLEM - mw213 Current Load on mw213 is WARNING: LOAD WARNING - total load average: 25.78, 20.07, 15.30 [05:56:01] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.51, 18.61, 14.18 [05:56:02] PROBLEM - mw193 Current Load on mw193 is WARNING: LOAD WARNING - total load average: 21.27, 17.99, 13.85 [05:56:16] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 22.07, 18.57, 14.17 [05:57:41] RECOVERY - mw213 Current Load on mw213 is OK: LOAD OK - total load average: 17.86, 19.21, 15.59 [05:58:01] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 14.59, 16.94, 14.11 [05:58:02] RECOVERY - mw193 Current Load on mw193 is OK: LOAD OK - total load average: 13.91, 16.23, 13.71 [05:58:21] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 15.58, 17.11, 14.16 [06:03:05] PROBLEM - db151 Current Load on db151 is CRITICAL: LOAD CRITICAL - total load average: 13.13, 10.18, 7.66 [06:03:19] PROBLEM - mw211 Current Load on mw211 is WARNING: LOAD WARNING - total load average: 26.52, 23.36, 18.55 [06:03:41] PROBLEM - mw213 Current Load on mw213 is WARNING: LOAD WARNING - total load average: 24.79, 22.50, 18.14 [06:03:44] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 21.09, 19.85, 16.16 [06:05:19] RECOVERY - mw211 Current Load on mw211 is OK: LOAD OK - total load average: 18.04, 21.45, 18.45 [06:05:38] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 14.81, 18.08, 15.96 [06:05:41] RECOVERY - mw213 Current Load on mw213 is OK: LOAD OK - total load average: 14.94, 19.66, 17.64 [06:06:54] PROBLEM - db151 Current Load on db151 is WARNING: LOAD WARNING - total load average: 8.05, 11.66, 9.06 [06:08:49] RECOVERY - db151 Current Load on db151 is OK: LOAD OK - total load average: 6.93, 10.14, 8.81 [07:52:28] PROBLEM - mw171 Current Load on mw171 is WARNING: LOAD WARNING - total load average: 20.80, 18.25, 13.85 [07:52:33] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 21.17, 18.43, 13.96 [07:52:39] PROBLEM - mw153 Current Load on mw153 is WARNING: LOAD WARNING - total load average: 21.42, 18.71, 14.01 [07:52:52] PROBLEM - mw202 Current Load on mw202 is WARNING: LOAD WARNING - total load average: 21.01, 18.43, 13.99 [07:53:07] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 20.86, 18.72, 14.30 [07:53:28] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 21.03, 18.45, 14.16 [07:53:48] PROBLEM - mw161 Current Load on mw161 is WARNING: LOAD WARNING - total load average: 21.73, 19.27, 14.64 [07:54:03] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.97, 19.35, 14.69 [07:54:28] RECOVERY - mw171 Current Load on mw171 is OK: LOAD OK - total load average: 19.56, 18.72, 14.56 [07:54:33] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 19.80, 19.03, 14.73 [07:54:40] RECOVERY - mw153 Current Load on mw153 is OK: LOAD OK - total load average: 18.50, 18.81, 14.64 [07:54:52] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 19.72, 18.99, 14.75 [07:55:07] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 19.83, 19.35, 15.10 [07:55:32] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 18.67, 18.58, 14.74 [07:55:48] RECOVERY - mw161 Current Load on mw161 is OK: LOAD OK - total load average: 17.06, 18.66, 15.00 [07:56:01] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.63, 18.75, 15.05 [07:59:07] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 21.97, 20.25, 16.39 [08:00:33] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 20.86, 19.99, 16.46 [08:01:07] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 20.04, 20.30, 16.89 [08:03:01] RECOVERY - bots171 Check unit status of pywikibot-metawiki-archivebot-job on bots171 is OK: OK: Status of the systemd unit pywikibot-metawiki-archivebot-job [08:26:07] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/a67b1310b375f83ceab97a576eea6409db1ba649 [08:26:07] 02puppet/03main 07CosmicAlpha 03a67b131 mwtask181: temp support IPv6 [08:48:06] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.76, 17.97, 14.19 [08:48:33] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 22.52, 18.23, 14.39 [08:48:39] PROBLEM - mw153 Current Load on mw153 is WARNING: LOAD WARNING - total load average: 22.70, 18.13, 14.22 [08:48:52] PROBLEM - mw202 Current Load on mw202 is WARNING: LOAD WARNING - total load average: 22.62, 18.43, 14.38 [08:48:56] PROBLEM - mw162 Current Load on mw162 is WARNING: LOAD WARNING - total load average: 22.37, 18.31, 14.36 [08:49:06] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 23.22, 18.70, 14.58 [08:49:07] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 22.86, 18.61, 14.61 [08:49:10] PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 23.04, 18.22, 14.31 [08:49:13] PROBLEM - mw212 Current Load on mw212 is WARNING: LOAD WARNING - total load average: 26.48, 21.20, 16.33 [08:49:23] PROBLEM - mw211 Current Load on mw211 is WARNING: LOAD WARNING - total load average: 26.68, 21.25, 16.37 [08:49:34] PROBLEM - mw151 Current Load on mw151 is WARNING: LOAD WARNING - total load average: 22.73, 18.84, 14.66 [08:49:35] PROBLEM - mw201 Current Load on mw201 is WARNING: LOAD WARNING - total load average: 34.31, 26.53, 20.05 [08:49:38] PROBLEM - mw191 Current Load on mw191 is WARNING: LOAD WARNING - total load average: 23.10, 18.70, 14.55 [08:49:46] PROBLEM - mw173 Current Load on mw173 is WARNING: LOAD WARNING - total load average: 22.99, 18.90, 14.71 [08:49:51] PROBLEM - mw161 Current Load on mw161 is WARNING: LOAD WARNING - total load average: 21.90, 18.85, 14.81 [08:49:56] PROBLEM - mw213 Current Load on mw213 is WARNING: LOAD WARNING - total load average: 27.52, 22.43, 17.11 [08:50:02] PROBLEM - mw193 Current Load on mw193 is WARNING: LOAD WARNING - total load average: 22.66, 19.32, 15.05 [08:50:09] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 23.07, 19.55, 15.19 [08:50:28] PROBLEM - mw171 Current Load on mw171 is WARNING: LOAD WARNING - total load average: 22.65, 19.48, 15.14 [08:50:32] PROBLEM - mw163 Current Load on mw163 is WARNING: LOAD WARNING - total load average: 22.83, 19.50, 15.14 [08:50:54] PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 22.52, 19.74, 15.41 [08:53:32] PROBLEM - mw151 Current Load on mw151 is CRITICAL: LOAD CRITICAL - total load average: 25.51, 22.22, 16.95 [08:53:33] RECOVERY - mw201 Current Load on mw201 is OK: LOAD OK - total load average: 30.45, 29.96, 23.02 [08:53:44] RECOVERY - mw173 Current Load on mw173 is OK: LOAD OK - total load average: 19.88, 20.29, 16.29 [08:53:49] RECOVERY - mw161 Current Load on mw161 is OK: LOAD OK - total load average: 19.70, 20.34, 16.39 [08:53:53] RECOVERY - mw213 Current Load on mw213 is OK: LOAD OK - total load average: 21.66, 23.37, 18.78 [08:54:03] RECOVERY - mw193 Current Load on mw193 is OK: LOAD OK - total load average: 18.94, 20.20, 16.47 [08:54:07] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 19.73, 20.38, 16.56 [08:54:32] RECOVERY - mw163 Current Load on mw163 is OK: LOAD OK - total load average: 19.06, 20.06, 16.44 [08:54:33] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 16.32, 19.52, 16.44 [08:54:40] RECOVERY - mw153 Current Load on mw153 is OK: LOAD OK - total load average: 14.75, 18.94, 16.11 [08:54:52] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 14.64, 18.89, 16.19 [08:54:54] RECOVERY - mw172 Current Load on mw172 is OK: LOAD OK - total load average: 14.52, 18.61, 16.09 [08:54:56] RECOVERY - mw162 Current Load on mw162 is OK: LOAD OK - total load average: 15.69, 19.04, 16.20 [08:55:01] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 16.55, 19.60, 16.48 [08:55:07] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 16.40, 19.37, 16.47 [08:55:10] RECOVERY - mw212 Current Load on mw212 is OK: LOAD OK - total load average: 17.70, 21.77, 18.47 [08:55:11] RECOVERY - mw152 Current Load on mw152 is OK: LOAD OK - total load average: 15.81, 18.73, 16.07 [08:55:19] RECOVERY - mw211 Current Load on mw211 is OK: LOAD OK - total load average: 20.51, 22.37, 18.70 [08:55:30] RECOVERY - mw151 Current Load on mw151 is OK: LOAD OK - total load average: 16.77, 19.81, 16.70 [08:55:38] RECOVERY - mw191 Current Load on mw191 is OK: LOAD OK - total load average: 14.91, 18.60, 16.14 [08:56:01] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 16.18, 19.55, 16.89 [08:56:28] RECOVERY - mw171 Current Load on mw171 is OK: LOAD OK - total load average: 13.64, 18.17, 16.26 [08:56:50] PROBLEM - cp191 HTTPS on cp191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10006 milliseconds with 0 bytes received [08:59:44] PROBLEM - cp171 HTTPS on cp171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10006 milliseconds with 0 bytes received [09:01:01] RECOVERY - cp191 HTTPS on cp191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1485 bytes in 3.280 second response time [09:03:49] RECOVERY - cp171 HTTPS on cp171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1485 bytes in 0.074 second response time [09:06:18] [02MirahezeMagic] 07lihaohong6 commented on pull request #674: CI failures are caused by unrelated issues that seem hard to fix due to the 1.46/1.47 version split. Not a serious problem for this PR. Will merge and deploy shortly. 13https://github.com/miraheze/MirahezeMagic/pull/674#issuecomment-6492523020 [09:14:05] [02MirahezeMagic] 07lihaohong6 pushed 1 new commit to 03main 13https://github.com/miraheze/MirahezeMagic/commit/f9f59f1b021e4c01a7679db88e26a48c91fbfad1 [09:14:07] 02MirahezeMagic/03main 07ParasailNumerous 03f9f59f1 Notify wiki users for CSP violations (#674)… [09:14:09] [02MirahezeMagic] 07lihaohong6 merged 07ParasailNumerous's pull request #674: T15875: Notify interface admin/admin/bureaucrat users for CSP violations (07miraheze:03main...07ParasailNumerous:03csp-violation-notify) 13https://github.com/miraheze/MirahezeMagic/pull/674 [09:15:26] !log [petramagna@test151] Starting deploy of {'l10n': True, 'versions': 'all', 'upgrade_extensions': 'MirahezeMagic'} to test151 (T15875) [09:15:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029091526-ec39e280 [09:16:12] !log [petramagna@test151] finished deploy of {'l10n': True, 'versions': 'all', 'upgrade_extensions': 'MirahezeMagic'} to test151 - SUCCESS in 46s (T15875) [09:16:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029091612-90fb4460 [09:17:14] miraheze/MirahezeMagic - lihaohong6 the build has errored. [09:21:08] !log [petramagna@mwtask181] Starting deploy of {'l10n': True, 'versions': 'all', 'upgrade_extensions': 'MirahezeMagic'} to all (T15875) [09:21:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029092108-7db5856b [09:22:26] !log [petramagna@mwtask181] finished deploy of {'l10n': True, 'versions': 'all', 'upgrade_extensions': 'MirahezeMagic'} to all - SUCCESS in 77s (T15875) [09:22:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029092226-72e74479 [09:44:03] [02MirahezeMagic] 07lihaohong6 opened pull request #679: Skip CSP violation notifications for report-only ones (07miraheze:03main...07lihaohong6:03csp3) 13https://github.com/miraheze/MirahezeMagic/pull/679 [09:45:03] !log [petramagna@test151] Starting deploy of {'folders': '1.46/extensions/MirahezeMagic', 'debug': True} to test151 (T15875) [09:45:04] !log [petramagna@test151] finished deploy of {'folders': '1.46/extensions/MirahezeMagic', 'debug': True} to test151 - SUCCESS in 0s (T15875) [09:45:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029094503-bb2c6d7b [09:45:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029094506-bc9c51bb [09:47:17] [02MirahezeMagic] 07lihaohong6 pushed 1 new commit to 03main 13https://github.com/miraheze/MirahezeMagic/commit/e121cdd9712fb1b5ae92f44c9b10a7c26ad07ec2 [09:47:17] 02MirahezeMagic/03main 07Peter Li 03e121cdd Skip CSP violation notifications for report-only ones (#679)… [09:47:18] [02MirahezeMagic] 07lihaohong6 merged pull request #679: Skip CSP violation notifications for report-only ones (07miraheze:03main...07lihaohong6:03csp3) 13https://github.com/miraheze/MirahezeMagic/pull/679 [09:47:19] miraheze/MirahezeMagic - lihaohong6 the build has errored. [09:47:33] !log [petramagna@mwtask181] Starting deploy of {'versions': 'all', 'upgrade_extensions': 'MirahezeMagic'} to all (T15875) [09:47:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029094733-126ae96a [09:48:38] !log [petramagna@mwtask181] finished deploy of {'versions': 'all', 'upgrade_extensions': 'MirahezeMagic'} to all - SUCCESS in 65s (T15875) [09:48:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029094838-4f3aeb25 [09:50:34] miraheze/MirahezeMagic - lihaohong6 the build has errored. [10:42:14] PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 26.82, 13.09, 7.98 [10:50:14] PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 9.00, 11.83, 9.99 [10:54:14] RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 7.19, 9.24, 9.34 [10:57:01] PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 15.73, 13.49, 10.62 [10:57:38] PROBLEM - mw191 Current Load on mw191 is WARNING: LOAD WARNING - total load average: 21.75, 18.22, 14.38 [10:58:16] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 21.91, 18.67, 14.73 [10:58:28] PROBLEM - mw171 Current Load on mw171 is WARNING: LOAD WARNING - total load average: 22.44, 18.90, 14.83 [10:58:32] PROBLEM - mw163 Current Load on mw163 is WARNING: LOAD WARNING - total load average: 22.26, 18.87, 14.78 [10:58:33] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 22.22, 18.97, 14.98 [10:58:37] PROBLEM - mw173 Current Load on mw173 is WARNING: LOAD WARNING - total load average: 21.56, 18.94, 14.94 [10:58:42] PROBLEM - mw212 Current Load on mw212 is WARNING: LOAD WARNING - total load average: 25.97, 21.55, 16.43 [10:59:19] PROBLEM - mw211 Current Load on mw211 is WARNING: LOAD WARNING - total load average: 24.39, 21.40, 16.69 [10:59:44] PROBLEM - mw213 Current Load on mw213 is WARNING: LOAD WARNING - total load average: 24.89, 21.71, 16.94 [11:00:35] PROBLEM - mw201 Current Load on mw201 is WARNING: LOAD WARNING - total load average: 33.32, 27.96, 20.91 [11:01:06] PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 14.26, 15.31, 12.04 [11:02:37] PROBLEM - mw161 Current Load on mw161 is WARNING: LOAD WARNING - total load average: 22.19, 20.14, 16.15 [11:02:40] PROBLEM - mw153 Current Load on mw153 is WARNING: LOAD WARNING - total load average: 21.34, 19.76, 15.99 [11:02:52] PROBLEM - mw202 Current Load on mw202 is WARNING: LOAD WARNING - total load average: 22.56, 20.49, 16.47 [11:02:53] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.98, 20.64, 16.60 [11:02:54] PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 21.87, 20.17, 16.33 [11:02:55] PROBLEM - mw193 Current Load on mw193 is WARNING: LOAD WARNING - total load average: 22.08, 20.08, 16.20 [11:02:56] PROBLEM - mw162 Current Load on mw162 is WARNING: LOAD WARNING - total load average: 21.25, 19.77, 16.16 [11:03:07] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 21.87, 20.51, 16.57 [11:03:10] PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 22.13, 20.07, 16.13 [11:03:19] PROBLEM - mw151 Current Load on mw151 is WARNING: LOAD WARNING - total load average: 22.17, 20.62, 16.53 [11:03:23] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 22.57, 20.52, 16.46 [11:04:16] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 20.18, 20.40, 16.79 [11:04:32] RECOVERY - mw163 Current Load on mw163 is OK: LOAD OK - total load average: 19.52, 20.10, 16.68 [11:04:35] RECOVERY - mw201 Current Load on mw201 is OK: LOAD OK - total load average: 29.86, 29.66, 23.26 [11:04:55] RECOVERY - mw172 Current Load on mw172 is OK: LOAD OK - total load average: 20.39, 20.16, 16.80 [11:04:57] RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 9.93, 12.58, 11.55 [11:05:38] RECOVERY - mw191 Current Load on mw191 is OK: LOAD OK - total load average: 20.40, 19.98, 16.86 [11:08:16] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 19.72, 20.85, 17.86 [11:08:18] RECOVERY - mw161 Current Load on mw161 is OK: LOAD OK - total load average: 17.87, 20.03, 17.43 [11:08:28] RECOVERY - mw171 Current Load on mw171 is OK: LOAD OK - total load average: 17.31, 19.89, 17.50 [11:08:31] RECOVERY - mw173 Current Load on mw173 is OK: LOAD OK - total load average: 17.57, 20.31, 17.79 [11:08:36] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 16.16, 19.71, 17.59 [11:08:38] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 15.81, 19.44, 17.50 [11:08:43] RECOVERY - mw153 Current Load on mw153 is OK: LOAD OK - total load average: 15.47, 19.23, 17.16 [11:08:44] RECOVERY - mw212 Current Load on mw212 is OK: LOAD OK - total load average: 17.36, 22.87, 20.07 [11:08:45] RECOVERY - mw193 Current Load on mw193 is OK: LOAD OK - total load average: 16.30, 19.72, 17.40 [11:08:52] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 17.10, 19.47, 17.44 [11:08:56] RECOVERY - mw162 Current Load on mw162 is OK: LOAD OK - total load average: 17.31, 19.41, 17.28 [11:09:07] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 17.62, 19.49, 17.50 [11:09:10] RECOVERY - mw152 Current Load on mw152 is OK: LOAD OK - total load average: 18.78, 19.72, 17.31 [11:09:15] RECOVERY - mw151 Current Load on mw151 is OK: LOAD OK - total load average: 17.45, 19.67, 17.56 [11:09:19] RECOVERY - mw211 Current Load on mw211 is OK: LOAD OK - total load average: 19.02, 22.51, 20.02 [11:09:20] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 16.90, 19.18, 17.30 [11:09:41] RECOVERY - mw213 Current Load on mw213 is OK: LOAD OK - total load average: 17.05, 21.62, 19.74 [11:10:16] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 15.62, 19.01, 17.56 [11:16:33] PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 6.69, 9.22, 11.71 [11:20:19] RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 4.06, 6.52, 10.06 [11:50:54] PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 14.68, 11.36, 9.08 [11:52:54] RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 9.60, 10.83, 9.18 [11:53:06] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 23.57, 18.60, 14.33 [11:53:11] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 22.53, 18.12, 14.17 [11:53:34] PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 22.54, 18.47, 14.31 [11:53:38] PROBLEM - mw191 Current Load on mw191 is WARNING: LOAD WARNING - total load average: 22.92, 18.62, 14.41 [11:53:53] PROBLEM - mw161 Current Load on mw161 is WARNING: LOAD WARNING - total load average: 22.70, 18.60, 14.42 [11:54:01] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.58, 18.83, 14.63 [11:54:02] PROBLEM - mw193 Current Load on mw193 is WARNING: LOAD WARNING - total load average: 22.16, 18.63, 14.44 [11:55:08] PROBLEM - mw151 Current Load on mw151 is WARNING: LOAD WARNING - total load average: 21.04, 18.61, 14.72 [11:55:10] PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 20.97, 18.89, 14.85 [11:56:02] RECOVERY - mw193 Current Load on mw193 is OK: LOAD OK - total load average: 20.19, 19.00, 15.07 [11:58:17] PROBLEM - mw173 Current Load on mw173 is WARNING: LOAD WARNING - total load average: 22.21, 20.51, 16.32 [11:58:28] PROBLEM - mw171 Current Load on mw171 is WARNING: LOAD WARNING - total load average: 21.84, 20.31, 16.28 [11:58:32] PROBLEM - mw163 Current Load on mw163 is WARNING: LOAD WARNING - total load average: 22.15, 20.20, 16.12 [11:58:52] PROBLEM - mw202 Current Load on mw202 is WARNING: LOAD WARNING - total load average: 21.55, 19.98, 16.05 [11:58:56] PROBLEM - mw162 Current Load on mw162 is WARNING: LOAD WARNING - total load average: 21.61, 20.30, 16.38 [12:00:02] PROBLEM - mw193 Current Load on mw193 is WARNING: LOAD WARNING - total load average: 22.67, 20.81, 16.69 [12:00:03] PROBLEM - mw211 Current Load on mw211 is WARNING: LOAD WARNING - total load average: 26.34, 23.61, 18.72 [12:00:16] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 23.69, 21.53, 17.18 [12:00:33] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 22.00, 20.71, 16.73 [12:00:35] PROBLEM - mw213 Current Load on mw213 is WARNING: LOAD WARNING - total load average: 26.54, 24.31, 19.21 [12:00:40] PROBLEM - mw153 Current Load on mw153 is WARNING: LOAD WARNING - total load average: 22.24, 21.07, 16.99 [12:00:41] PROBLEM - mw212 Current Load on mw212 is WARNING: LOAD WARNING - total load average: 25.52, 23.63, 18.96 [12:01:57] RECOVERY - mw211 Current Load on mw211 is OK: LOAD OK - total load average: 23.31, 23.43, 19.24 [12:02:39] [02ImportDump] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/ImportDump/commit/544b2946b084f1d9c50be01719db5e897b1dcdab [12:02:39] 02ImportDump/03main 07translatewiki.net 03544b294 Localisation updates from https://translatewiki.net. [12:02:40] [02ManageWiki] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/ManageWiki/commit/cd5b9cb4d2569a7c507e5f04ea4b5b43fee05977 [12:02:40] 02ManageWiki/03main 07translatewiki.net 03cd5b9cb Localisation updates from https://translatewiki.net. [12:02:41] [02MatomoAnalytics] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/MatomoAnalytics/commit/30d8f6e7b5f366e61090b3dac124769a5d07c8d9 [12:02:43] 02MatomoAnalytics/03main 07translatewiki.net 0330d8f6e Localisation updates from https://translatewiki.net. [12:02:45] [02MirahezeMagic] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/MirahezeMagic/commit/c2e68a71924edfd0224c021c98f8fc8f02c5dcfa [12:02:47] [02WikiDiscover] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/WikiDiscover/commit/a727e1142997e663a5f357e40d7630fb798857c8 [12:02:48] 02MirahezeMagic/03main 07translatewiki.net 03c2e68a7 Localisation updates from https://translatewiki.net. [12:02:50] 02WikiDiscover/03main 07translatewiki.net 03a727e11 Localisation updates from https://translatewiki.net. [12:04:17] RECOVERY - mw173 Current Load on mw173 is OK: LOAD OK - total load average: 17.28, 19.98, 17.60 [12:04:28] RECOVERY - mw171 Current Load on mw171 is OK: LOAD OK - total load average: 17.84, 20.03, 17.59 [12:04:32] RECOVERY - mw163 Current Load on mw163 is OK: LOAD OK - total load average: 16.91, 19.53, 17.28 [12:04:33] RECOVERY - mw213 Current Load on mw213 is OK: LOAD OK - total load average: 19.52, 22.57, 19.69 [12:04:34] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 18.36, 20.16, 17.46 [12:04:40] RECOVERY - mw153 Current Load on mw153 is OK: LOAD OK - total load average: 17.30, 19.69, 17.39 [12:04:41] RECOVERY - mw212 Current Load on mw212 is OK: LOAD OK - total load average: 17.85, 21.91, 19.40 [12:04:52] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 15.90, 19.48, 17.27 [12:04:57] RECOVERY - mw162 Current Load on mw162 is OK: LOAD OK - total load average: 16.76, 19.57, 17.48 [12:05:05] RECOVERY - mw151 Current Load on mw151 is OK: LOAD OK - total load average: 17.64, 19.66, 17.45 [12:05:10] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 17.83, 19.89, 17.71 [12:05:11] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 18.26, 19.66, 17.50 [12:05:12] RECOVERY - mw152 Current Load on mw152 is OK: LOAD OK - total load average: 16.98, 19.56, 17.47 [12:05:29] RECOVERY - mw172 Current Load on mw172 is OK: LOAD OK - total load average: 16.45, 19.37, 17.45 [12:05:38] RECOVERY - mw191 Current Load on mw191 is OK: LOAD OK - total load average: 14.56, 18.60, 17.18 [12:05:48] RECOVERY - mw161 Current Load on mw161 is OK: LOAD OK - total load average: 14.27, 18.48, 17.16 [12:06:01] miraheze/MatomoAnalytics - translatewiki the build has errored. [12:06:02] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 13.58, 18.30, 17.23 [12:06:02] miraheze/MirahezeMagic - translatewiki the build has errored. [12:06:03] RECOVERY - mw193 Current Load on mw193 is OK: LOAD OK - total load average: 13.77, 18.30, 17.12 [12:06:16] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 14.02, 18.28, 17.32 [12:07:15] miraheze/WikiDiscover - translatewiki the build passed. [12:07:27] miraheze/ImportDump - translatewiki the build passed. [12:07:31] miraheze/ManageWiki - translatewiki the build passed. [14:01:05] PROBLEM - mw151 Current Load on mw151 is WARNING: LOAD WARNING - total load average: 22.65, 18.48, 14.23 [14:01:19] PROBLEM - mw211 Current Load on mw211 is WARNING: LOAD WARNING - total load average: 26.59, 21.19, 16.02 [14:01:38] PROBLEM - mw191 Current Load on mw191 is WARNING: LOAD WARNING - total load average: 23.10, 19.07, 14.57 [14:01:41] PROBLEM - mw213 Current Load on mw213 is WARNING: LOAD WARNING - total load average: 25.63, 20.85, 15.96 [14:01:48] PROBLEM - mw161 Current Load on mw161 is WARNING: LOAD WARNING - total load average: 22.18, 18.72, 14.50 [14:02:01] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.59, 19.10, 14.71 [14:02:02] PROBLEM - mw193 Current Load on mw193 is WARNING: LOAD WARNING - total load average: 22.52, 19.06, 14.63 [14:02:16] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 23.10, 19.45, 14.90 [14:02:18] PROBLEM - mw173 Current Load on mw173 is WARNING: LOAD WARNING - total load average: 22.00, 19.37, 14.91 [14:02:28] PROBLEM - mw171 Current Load on mw171 is WARNING: LOAD WARNING - total load average: 22.71, 19.59, 14.99 [14:02:32] PROBLEM - mw163 Current Load on mw163 is WARNING: LOAD WARNING - total load average: 22.90, 19.88, 15.16 [14:02:33] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 22.37, 19.26, 14.82 [14:02:40] PROBLEM - mw153 Current Load on mw153 is WARNING: LOAD WARNING - total load average: 21.78, 18.98, 14.66 [14:02:41] PROBLEM - mw212 Current Load on mw212 is WARNING: LOAD WARNING - total load average: 26.52, 21.80, 16.46 [14:02:46] PROBLEM - cp191 HTTPS on cp191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [14:02:52] PROBLEM - mw202 Current Load on mw202 is WARNING: LOAD WARNING - total load average: 22.81, 19.69, 15.08 [14:02:54] PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 22.60, 19.75, 15.14 [14:02:56] PROBLEM - mw162 Current Load on mw162 is WARNING: LOAD WARNING - total load average: 22.69, 19.69, 15.14 [14:03:06] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 21.76, 19.33, 15.10 [14:03:11] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 22.42, 19.44, 15.11 [14:03:12] PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 21.58, 19.50, 15.10 [14:03:42] PROBLEM - cp171 HTTPS on cp171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10006 milliseconds with 0 bytes received [14:04:35] PROBLEM - mw201 Current Load on mw201 is WARNING: LOAD WARNING - total load average: 33.35, 28.81, 21.40 [14:04:54] RECOVERY - cp191 HTTPS on cp191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1485 bytes in 9.807 second response time [14:05:45] RECOVERY - cp171 HTTPS on cp171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 6.202 second response time [14:07:49] PROBLEM - matomo151 Current Load on matomo151 is CRITICAL: LOAD CRITICAL - total load average: 16.45, 13.01, 10.38 [14:08:28] RECOVERY - mw171 Current Load on mw171 is OK: LOAD OK - total load average: 16.91, 20.00, 16.84 [14:08:32] RECOVERY - mw163 Current Load on mw163 is OK: LOAD OK - total load average: 17.39, 20.14, 16.98 [14:08:33] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 17.25, 20.09, 16.83 [14:08:35] RECOVERY - mw201 Current Load on mw201 is OK: LOAD OK - total load average: 19.46, 27.05, 22.62 [14:08:39] RECOVERY - mw153 Current Load on mw153 is OK: LOAD OK - total load average: 17.52, 19.98, 16.71 [14:08:46] RECOVERY - mw212 Current Load on mw212 is OK: LOAD OK - total load average: 18.69, 22.52, 18.77 [14:08:56] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 18.07, 20.30, 17.02 [14:08:58] RECOVERY - mw172 Current Load on mw172 is OK: LOAD OK - total load average: 18.52, 19.99, 16.88 [14:08:59] RECOVERY - mw162 Current Load on mw162 is OK: LOAD OK - total load average: 18.67, 20.25, 17.02 [14:09:05] RECOVERY - mw151 Current Load on mw151 is OK: LOAD OK - total load average: 17.42, 19.94, 16.97 [14:09:06] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 16.77, 19.54, 16.80 [14:09:07] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 17.68, 19.78, 16.84 [14:09:10] RECOVERY - mw152 Current Load on mw152 is OK: LOAD OK - total load average: 17.79, 19.81, 16.84 [14:09:19] RECOVERY - mw211 Current Load on mw211 is OK: LOAD OK - total load average: 17.71, 21.91, 18.94 [14:09:21] PROBLEM - matomo211 Current Load on matomo211 is WARNING: LOAD WARNING - total load average: 15.44, 12.89, 9.94 [14:09:38] RECOVERY - mw191 Current Load on mw191 is OK: LOAD OK - total load average: 17.34, 19.45, 16.87 [14:09:41] RECOVERY - mw213 Current Load on mw213 is OK: LOAD OK - total load average: 17.81, 21.68, 18.85 [14:09:48] PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 15.19, 14.10, 11.13 [14:09:49] RECOVERY - mw161 Current Load on mw161 is OK: LOAD OK - total load average: 17.62, 19.22, 16.70 [14:10:02] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 18.56, 19.97, 17.16 [14:10:03] RECOVERY - mw193 Current Load on mw193 is OK: LOAD OK - total load average: 19.80, 20.08, 17.15 [14:10:16] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 19.81, 20.14, 17.30 [14:11:21] RECOVERY - matomo211 Current Load on matomo211 is OK: LOAD OK - total load average: 9.81, 11.78, 9.90 [14:11:47] RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 12.43, 13.34, 11.20 [14:12:18] RECOVERY - mw173 Current Load on mw173 is OK: LOAD OK - total load average: 15.85, 18.75, 17.23 [14:57:44] PROBLEM - cp171 Current Load on cp171 is WARNING: LOAD WARNING - total load average: 14.89, 12.06, 9.58 [14:59:43] RECOVERY - cp171 Current Load on cp171 is OK: LOAD OK - total load average: 13.28, 12.54, 10.06 [16:54:54] PROBLEM - matomo151 Current Load on matomo151 is CRITICAL: LOAD CRITICAL - total load average: 16.55, 14.18, 10.91 [16:56:54] PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 15.11, 13.97, 11.20 [16:58:54] PROBLEM - matomo151 Current Load on matomo151 is CRITICAL: LOAD CRITICAL - total load average: 19.12, 15.79, 12.20 [16:59:21] PROBLEM - matomo211 Current Load on matomo211 is CRITICAL: LOAD CRITICAL - total load average: 16.25, 14.25, 11.08 [17:05:21] PROBLEM - matomo211 Current Load on matomo211 is WARNING: LOAD WARNING - total load average: 12.17, 14.51, 12.36 [17:06:53] PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 14.76, 15.67, 13.77 [17:07:21] RECOVERY - matomo211 Current Load on matomo211 is OK: LOAD OK - total load average: 11.46, 13.59, 12.29 [17:08:54] RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 8.86, 13.50, 13.23 [17:38:44] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importImages --wiki=warhammerhomebrewwiki '--comment=Imported images from https://warhammer-homebrew.fandom.com/wiki/Warhammer_Homebrew_Wiki' --search-recursively /home/reception/warhammer (START) [17:38:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029173843-92f0b3ac [17:56:30] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importImages --wiki=warhammerhomebrewwiki '--comment=Imported images from https://warhammer-homebrew.fandom.com/wiki/Warhammer_Homebrew_Wiki' --search-recursively /home/reception/warhammer (END - exit=0; time=1066s) [17:56:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029175630-274c7bc4 [18:55:01] PROBLEM - mw211 APT on mw211 is CRITICAL: APT CRITICAL: 8 packages available for upgrade (8 critical updates). [18:57:54] PROBLEM - mw192 APT on mw192 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [18:58:59] PROBLEM - mw163 APT on mw163 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:00:26] PROBLEM - mw151 APT on mw151 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:03:23] PROBLEM - mw181 APT on mw181 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:04:00] PROBLEM - mwtask181 APT on mwtask181 is CRITICAL: APT CRITICAL: 11 packages available for upgrade (9 critical updates). [19:04:18] PROBLEM - test151 APT on test151 is CRITICAL: APT CRITICAL: 11 packages available for upgrade (9 critical updates). [19:05:06] PROBLEM - mw201 APT on mw201 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:06:09] PROBLEM - mw152 APT on mw152 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:06:11] PROBLEM - mw183 APT on mw183 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:07:08] PROBLEM - mw182 APT on mw182 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:07:57] PROBLEM - mw212 APT on mw212 is CRITICAL: APT CRITICAL: 8 packages available for upgrade (8 critical updates). [19:09:43] PROBLEM - mw162 APT on mw162 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:09:46] !log [void@puppet181] Upgraded packages on mw163: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:09:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029190946-25b2923c [19:10:04] !log [void@puppet181] Upgraded packages on mw153: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:10:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191004-310aad47 [19:10:22] !log [void@puppet181] Upgraded packages on mw152: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:10:22] PROBLEM - mw171 APT on mw171 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:10:24] PROBLEM - mw193 APT on mw193 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:10:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191022-b14cce92 [19:10:36] PROBLEM - mw172 APT on mw172 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:10:41] PROBLEM - mwtask161 APT on mwtask161 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:10:41] !log [void@puppet181] Upgraded packages on mw151: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:10:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191041-2adaac14 [19:10:57] PROBLEM - mw202 APT on mw202 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:10:58] RECOVERY - mw163 APT on mw163 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:10:58] !log [void@puppet181] Upgraded packages on mw171: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:11:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191059-ed1a2e7b [19:11:14] PROBLEM - mw173 APT on mw173 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:11:17] !log [void@puppet181] Upgraded packages on mw172: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:11:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191117-ac08d84b [19:11:27] PROBLEM - mw213 APT on mw213 is CRITICAL: APT CRITICAL: 8 packages available for upgrade (8 critical updates). [19:11:35] !log [void@puppet181] Upgraded packages on mw192: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:11:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191135-bd9ebfc5 [19:11:39] PROBLEM - mwtask171 APT on mwtask171 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:11:53] PROBLEM - mwtask151 APT on mwtask151 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:11:54] RECOVERY - mw192 APT on mw192 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:11:55] !log [void@puppet181] Upgraded packages on mw161: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:11:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191155-be782808 [19:11:59] PROBLEM - mw203 APT on mw203 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:12:05] PROBLEM - mw191 APT on mw191 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (8 critical updates). [19:12:10] RECOVERY - mw152 APT on mw152 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:12:14] !log [void@puppet181] Upgraded packages on mw202: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:12:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191214-4bda7911 [19:12:23] RECOVERY - mw171 APT on mw171 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:12:32] !log [void@puppet181] Upgraded packages on mw173: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:12:33] RECOVERY - mw151 APT on mw151 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:12:34] RECOVERY - mw172 APT on mw172 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:12:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191232-d2ee2783 [19:12:50] !log [void@puppet181] Upgraded packages on mw213: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:12:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191250-0bd748f7 [19:12:57] RECOVERY - mw202 APT on mw202 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:13:09] !log [void@puppet181] Upgraded packages on mw183: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:13:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191309-de217493 [19:13:13] RECOVERY - mw173 APT on mw173 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:13:26] RECOVERY - mw213 APT on mw213 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [19:13:29] !log [void@puppet181] Upgraded packages on mwtask161: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:13:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191329-04018219 [19:13:47] !log [void@puppet181] Upgraded packages on mw162: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:13:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191347-c43b67c6 [19:14:06] !log [void@puppet181] Upgraded packages on mw191: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:14:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191406-8b939b1c [19:14:20] RECOVERY - mw183 APT on mw183 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:14:25] !log [void@puppet181] Upgraded packages on mw193: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:14:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191425-3c7eb3e5 [19:14:40] RECOVERY - mwtask161 APT on mwtask161 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:14:44] !log [void@puppet181] Upgraded packages on mw182: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:14:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191444-5516db53 [19:15:03] !log [void@puppet181] Upgraded packages on mw203: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:15:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191503-90796bec [19:15:15] RECOVERY - mw182 APT on mw182 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:15:22] !log [void@puppet181] Upgraded packages on mw212: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:15:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191522-da866643 [19:15:40] RECOVERY - mw162 APT on mw162 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:15:41] !log [void@puppet181] Upgraded packages on mwtask151: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:15:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191541-0ed021d8 [19:15:47] RECOVERY - mw212 APT on mw212 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [19:15:56] RECOVERY - mwtask151 APT on mwtask151 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:15:57] RECOVERY - mw203 APT on mw203 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:16:00] !log [void@puppet181] Upgraded packages on mw211: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:16:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191600-20aead60 [19:16:04] RECOVERY - mw191 APT on mw191 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:16:19] !log [void@puppet181] Upgraded packages on mw201: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:16:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191619-e380c3e0 [19:16:24] RECOVERY - mw193 APT on mw193 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:16:38] !log [void@puppet181] Upgraded packages on mwtask171: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:16:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191638-17a00630 [19:16:57] !log [void@puppet181] Upgraded packages on mw181: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, and libmagickcore-7.q16-10-extra [19:16:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191657-59a1d574 [19:17:06] RECOVERY - mw201 APT on mw201 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:17:16] RECOVERY - mw211 APT on mw211 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [19:17:18] !log [void@puppet181] Upgraded packages on mwtask181: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, libmagickcore-7.q16-10-extra, and python3-git [19:17:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191718-ca7f3439 [19:17:26] RECOVERY - mw181 APT on mw181 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:17:39] RECOVERY - mwtask171 APT on mwtask171 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:17:40] !log [void@puppet181] Upgraded packages on test151: libimage-magick-perl, imagemagick-7-common, libmagickcore-7.q16-10, libimage-magick-q16-perl, libmagickwand-7.q16-10, imagemagick-7.q16, imagemagick, libmagickcore-7.q16-10-extra, and python3-git [19:17:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029191740-6d7f3a51 [19:17:58] RECOVERY - mwtask181 APT on mwtask181 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:18:18] RECOVERY - test151 APT on test151 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [19:42:51] [02mw-config] 07Universal-Omega left a file comment in pull request #6617 0375776af: These can just be removed, not need to comment them. 13https://github.com/miraheze/mw-config/pull/6617#discussion_r4417099274 [19:43:02] miraheze/mw-config - MaxStache the build passed. [19:58:55] miraheze/mw-config - MaxStache the build passed. [20:55:21] PROBLEM - matomo211 Current Load on matomo211 is CRITICAL: LOAD CRITICAL - total load average: 16.11, 12.32, 8.65 [20:56:25] PROBLEM - db151 Current Load on db151 is CRITICAL: LOAD CRITICAL - total load average: 13.04, 10.05, 7.28 [20:58:25] PROBLEM - db151 Current Load on db151 is WARNING: LOAD WARNING - total load average: 10.97, 10.03, 7.59 [21:00:25] PROBLEM - db151 Current Load on db151 is CRITICAL: LOAD CRITICAL - total load average: 17.12, 11.76, 8.47 [21:02:25] PROBLEM - db151 Current Load on db151 is WARNING: LOAD WARNING - total load average: 11.07, 11.13, 8.64 [21:04:25] RECOVERY - db151 Current Load on db151 is OK: LOAD OK - total load average: 7.35, 9.73, 8.43 [21:06:57] PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 11.13, 15.40, 12.87 [21:07:21] PROBLEM - matomo211 Current Load on matomo211 is WARNING: LOAD WARNING - total load average: 9.59, 14.48, 12.50 [21:08:56] RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 7.10, 12.26, 12.01 [21:09:21] RECOVERY - matomo211 Current Load on matomo211 is OK: LOAD OK - total load average: 6.25, 11.57, 11.67 [21:27:32] [02mw-config] 07Universal-Omega merged 07MaxStache's pull request #6617: Fix markers not showing in Kartographer (07miraheze:03main...07MaxStache:03MaxStache-kartographer-markerFix) 13https://github.com/miraheze/mw-config/pull/6617 [21:27:32] [02mw-config] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/80583a36cdc3feff4cb5c835713648de8d54d4f6 [21:27:32] 02mw-config/03main 07Max 0380583a3 Fix markers not showing in Kartographer (#6617)… [21:28:03] !log [universalomega@mwtask181] Starting deploy of {'pull': 'config', 'config': True} to all [21:28:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029212803-4d1b4212 [21:28:33] !log [universalomega@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 29s [21:28:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261029212833-32b53518 [21:28:44] miraheze/mw-config - Universal-Omega the build passed.