[00:02:25] [02DataDump] 07Universal-Omega synchronize pull request 03#56: Create a few maintenance scripts - 13https://github.com/miraheze/DataDump/pull/56 [00:02:27] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/18d6b78ef835...fb39636cf07d [00:02:29] [02miraheze/DataDump] 07Universal-Omega 03fb39636 - Update importMissingDumps.php [00:03:39] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.35, 4.56, 3.61 [00:05:34] miraheze/DataDump - Universal-Omega the build passed. [00:13:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.23, 3.77, 3.92 [00:17:30] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.95, 2.34, 3.31 [00:23:07] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/c5f96be9f70a [00:23:08] [02miraheze/puppet] 07Universal-Omega 03c5f96be - nginx: enable stub_status in /server-status [00:23:09] [02puppet] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/puppet [00:23:11] [02puppet] 07Universal-Omega opened pull request 03#3104: nginx: enable stub_status in /server-status - 13https://github.com/miraheze/puppet/pull/3104 [00:23:58] [02puppet] 07paladox commented on commit 03c5f96be9f70adcadf8559fce59a2142b6c15591a - 13https://github.com/miraheze/puppet/commit/c5f96be9f70adcadf8559fce59a2142b6c15591a#commitcomment-94650052 [00:37:23] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.42, 5.31, 3.76 [00:43:07] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.64, 9.23, 8.56 [00:43:20] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.87, 3.79, 3.58 [00:45:02] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.65, 8.73, 8.47 [00:45:19] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.76, 3.10, 3.35 [00:55:28] [02puppet] 07Universal-Omega closed pull request 03#3104: nginx: enable stub_status in /server-status - 13https://github.com/miraheze/puppet/pull/3104 [00:55:30] [02miraheze/puppet] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [00:55:32] [02puppet] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/puppet [00:57:43] PROBLEM - matomo131 Current Load on matomo131 is WARNING: WARNING - load average: 3.41, 2.74, 2.29 [00:59:43] RECOVERY - matomo131 Current Load on matomo131 is OK: OK - load average: 2.84, 2.60, 2.29 [01:32:52] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 6.07, 3.94, 2.18 [01:34:51] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.40, 3.91, 2.41 [01:36:16] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=fictiontopiawiki /home/reception/wikibackups1812/fictiontopiawiki.xml (END - exit=0) [01:36:17] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=fictiontopiawiki (START) [01:36:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:36:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:36:47] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 8.92, 5.42, 3.30 [01:38:51] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.30, 2.82, 2.32 [01:48:41] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.55, 3.67, 3.80 [01:54:37] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.62, 2.46, 3.21 [01:57:07] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 5.68, 9.96, 6.75 [01:59:06] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 1.34, 6.87, 6.00 [02:01:04] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 0.69, 4.80, 5.35 [02:06:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.09, 3.53, 3.19 [02:07:28] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=fictiontopiawiki (END - exit=0) [02:07:29] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=fictiontopiawiki --missing (START) [02:07:32] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=fictiontopiawiki --missing (END - exit=0) [02:07:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:07:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:07:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:08:02] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=fictiontopiawiki --active --update (END - exit=0) [02:08:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:08:58] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.26, 6.10, 5.48 [02:10:57] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 6.58, 6.71, 5.81 [02:12:08] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=fatimamicrowiki /home/reception/wikibackups1812/fatimamicrowiki.xml (START) [02:12:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:14:24] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.04, 3.48, 3.55 [02:16:23] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.53, 4.48, 3.90 [02:18:22] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.15, 4.00, 3.79 [02:22:20] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.42, 2.92, 3.37 [02:29:42] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 14.28, 10.54, 7.83 [02:32:51] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.80, 3.71, 2.19 [02:33:40] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.18, 5.96, 6.64 [02:34:51] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.73, 2.91, 2.08 [02:37:04] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 5.46, 4.13, 3.15 [02:37:28] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=fatimamicrowiki /home/reception/wikibackups1812/fatimamicrowiki.xml (END - exit=0) [02:37:29] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=fatimamicrowiki (START) [02:37:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:37:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:39:13] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=fatimamicrowiki (END - exit=0) [02:39:14] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=fatimamicrowiki --missing (START) [02:39:18] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=fatimamicrowiki --missing (END - exit=0) [02:39:19] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=fatimamicrowiki --active --update (END - exit=0) [02:39:20] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=emulationwiki /home/reception/wikibackups1812/emulationwiki.xml (START) [02:39:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:39:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:39:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:39:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:39:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:50:23] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.95, 3.91, 3.71 [03:01:54] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.32, 3.03, 3.33 [03:02:07] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=emulationwiki /home/reception/wikibackups1812/emulationwiki.xml (END - exit=256) [03:02:08] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=emulationwiki (START) [03:02:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:02:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:03:41] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=emulationwiki (END - exit=0) [03:03:42] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=emulationwiki --missing (START) [03:03:46] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=emulationwiki --missing (END - exit=0) [03:03:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:03:47] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=emulationwiki --active --update (END - exit=0) [03:03:50] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=comprehensibleinputwiki /home/reception/wikibackups1812/comprehensibleinputwiki.xml (START) [03:03:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:04:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:04:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:04:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:08:13] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 6.99, 7.06, 6.31 [03:09:03] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=comprehensibleinputwiki /home/reception/wikibackups1812/comprehensibleinputwiki.xml (END - exit=0) [03:09:04] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=comprehensibleinputwiki (START) [03:09:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:09:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:18:11] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.77, 6.35, 6.47 [03:24:27] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=comprehensibleinputwiki (END - exit=0) [03:24:28] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=comprehensibleinputwiki --missing (START) [03:24:33] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=comprehensibleinputwiki --missing (END - exit=0) [03:24:34] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=comprehensibleinputwiki --active --update (END - exit=0) [03:24:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:24:35] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=ceteaoniawiki /home/reception/wikibackups1812/ceteaoniawiki.xml (START) [03:24:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:24:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:24:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:24:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:25:17] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=ceteaoniawiki /home/reception/wikibackups1812/ceteaoniawiki.xml (END - exit=0) [03:25:18] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=ceteaoniawiki (START) [03:25:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:25:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:25:31] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=ceteaoniawiki (END - exit=0) [03:25:32] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=ceteaoniawiki --missing (START) [03:25:36] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=ceteaoniawiki --missing (END - exit=0) [03:25:37] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=ceteaoniawiki --active --update (END - exit=0) [03:25:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:25:39] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=anewc0dawiki /home/reception/wikibackups1812/anewc0dawiki.xml (START) [03:25:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:25:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:25:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:26:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:30:25] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=anewc0dawiki /home/reception/wikibackups1812/anewc0dawiki.xml (END - exit=0) [03:30:26] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=anewc0dawiki (START) [03:30:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:30:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:30:45] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=anewc0dawiki (END - exit=0) [03:30:46] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=anewc0dawiki --missing (START) [03:30:51] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=anewc0dawiki --missing (END - exit=0) [03:30:52] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=anewc0dawiki --active --update (END - exit=0) [03:30:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:31:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:31:03] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 7.15, 10.60, 8.58 [03:31:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:31:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:40:27] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1006718576 secs [03:46:53] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 4.77, 6.76, 7.90 [03:48:51] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 15.16, 9.69, 8.78 [03:51:54] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09408357739 secs [03:56:10] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.80, 3.00, 2.05 [03:56:46] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 4.17, 6.92, 8.00 [03:57:33] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=emulationwiki /home/reception/wikibackups1812/emulationwiki.xml (START) [03:57:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:58:09] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.08, 2.44, 1.95 [03:58:45] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.63, 9.91, 9.09 [04:02:07] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=emulationwiki /home/reception/wikibackups1812/emulationwiki.xml (END - exit=256) [04:02:08] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=emulationwiki (START) [04:02:11] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=emulationwiki (END - exit=0) [04:02:12] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=emulationwiki --missing (START) [04:02:16] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=emulationwiki --missing (END - exit=0) [04:02:17] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=emulationwiki --active --update (END - exit=0) [04:02:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:02:21] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=emulationwiki /home/reception/wikibackups1812/emulationwiki.xml (START) [04:02:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:02:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:02:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:02:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:02:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:02:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:02:50] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=emulationwiki /home/reception/wikibackups1812/emulationwiki.xml (END - exit=256) [04:02:51] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=emulationwiki (START) [04:02:55] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=emulationwiki (END - exit=0) [04:02:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:02:56] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=emulationwiki --missing (START) [04:03:00] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=emulationwiki --missing (END - exit=0) [04:03:01] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=emulationwiki --active --update (END - exit=0) [04:03:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:03:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:03:09] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=emulationwiki /home/reception/wikibackups1812/emulationwiki.xml (START) [04:03:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:03:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:03:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:03:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:04:03] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=emulationwiki /home/reception/wikibackups1812/emulationwiki.xml (END - exit=0) [04:04:04] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=emulationwiki (START) [04:04:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:04:09] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=emulationwiki (END - exit=0) [04:04:10] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=emulationwiki --missing (START) [04:04:14] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=emulationwiki --missing (END - exit=0) [04:04:15] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=emulationwiki --active --update (END - exit=0) [04:04:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:04:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:04:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:04:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:04:41] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 0.94, 5.42, 7.74 [04:04:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:05:24] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 9.41, 4.90, 3.16 [04:08:38] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 0.32, 2.61, 6.06 [04:15:18] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.45, 3.16, 3.34 [04:16:49] !log [universalomega@mwtask141] Running: sudo -u www-data php /srv/mediawiki/w/extensions/CreateWiki/maintenance/populateMainPage.php --wiki kawkabanwiki [04:16:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:16:58] !log [universalomega@mwtask141] Running: sudo -u www-data php /srv/mediawiki/w/maintenance/createAndPromote.php Apolloqu --bureaucrat --sysop --force --wiki kawkabanwiki [04:17:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:19:58] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=hangoverswiki --no-updates /home/universalomega/ImportDump/hangoverswiki-20221206055555.xml (START) [04:20:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:23:53] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=hangoverswiki --no-updates /home/universalomega/ImportDump/hangoverswiki-20221206055555.xml (END - exit=0) [04:23:54] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=hangoverswiki (START) [04:23:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:24:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:26:34] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=hangoverswiki (END - exit=0) [04:26:35] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=hangoverswiki --active --update (END - exit=0) [04:26:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:26:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:37:03] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.83, 4.10, 3.46 [04:41:01] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.35, 3.61, 3.45 [04:43:00] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.79, 2.94, 3.22 [05:06:47] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.13, 5.43, 3.81 [05:10:45] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.44, 3.43, 3.38 [05:12:44] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.75, 3.04, 3.24 [05:36:36] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 13.42, 7.74, 4.52 [05:44:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.04, 3.59, 3.79 [05:45:59] !log MariaDB [mhglobal]> UPDATE cw_wikis SET wiki_dbcluster = 'c3' WHERE wiki_dbcluster = 'c6' AND wiki_dbname IN ('academiadesusarduwiki', 'agalaxytolookuponwiki', 'angrybirdswiki', 'chakuwikiwiki', 'homevideowiki', 'mockelectionswiki', 'pokeclickerwiki', 'polandballwiki', 'polcompballanarchywiki', 'reversecrappygameswiki', 'romancewikiwiki', 'saramorawiki', 'stampencyclopediawiki', 'thecaserewiki', 'webkinzguidewiki', [05:46:00] 'xomniwiki'); [05:46:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:50:28] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.12, 2.49, 3.29 [05:59:31] !log start rebuildImages.php --missing on all wikis [05:59:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:00:21] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.56, 3.30, 3.23 [06:02:20] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.73, 2.94, 3.09 [06:05:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.92, 10.35, 8.46 [06:06:03] !log [void@db142] create image table for webkinzguidewiki [06:06:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:06:18] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.94, 5.24, 3.93 [06:07:34] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 10.01, 10.20, 8.64 [06:08:10] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (START) [06:08:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:09:29] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (END - exit=65280) [06:09:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:09:48] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (START) [06:09:50] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (END - exit=65280) [06:09:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:10:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:10:43] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (START) [06:10:44] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (END - exit=65280) [06:10:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:10:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:11:27] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (START) [06:11:29] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (END - exit=65280) [06:11:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:11:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:14:17] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.85, 3.92, 3.84 [06:18:17] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.04, 2.70, 3.34 [06:28:17] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.47, 3.74, 3.35 [06:30:17] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.32, 2.86, 3.08 [06:39:54] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.26, 3.22, 2.56 [06:45:54] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.15, 3.57, 2.98 [06:47:37] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1067246199 secs [06:48:18] !log [universalomega@mwtask141] Running: sudo -u www-data php /srv/mediawiki/w/extensions/CreateWiki/maintenance/populateMainPage.php --wiki astrallionwiki [06:48:20] !log [universalomega@mwtask141] Running: sudo -u www-data php /srv/mediawiki/w/maintenance/createAndPromote.php Maverick --bureaucrat --sysop --force --wiki astrallionwiki [06:48:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:48:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:49:54] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.51, 3.06, 2.90 [07:06:00] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.11, 11.13, 9.18 [07:07:58] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.36, 9.58, 8.86 [07:12:54] miraheze/mw-config - Universal-Omega the build passed. [07:13:23] miraheze/mw-config - Universal-Omega the build passed. [07:17:37] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.0236903429 secs [07:21:42] !log [universalomega@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [07:21:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:21:51] !log [universalomega@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 8s [07:21:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:26:33] !log [@test131] starting deploy of {'config': True} to all [07:26:34] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [07:26:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:26:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:08:17] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.05, 2.92, 2.24 [08:10:17] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.41, 3.05, 2.37 [08:12:17] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.43, 2.44, 2.23 [08:31:10] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.68, 2.97, 2.26 [08:33:09] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.71, 3.54, 2.55 [08:34:42] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=elysiamcwiki --missing (END - exit=65280) [08:34:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:41:05] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.30, 3.92, 3.45 [08:43:04] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.35, 3.11, 3.21 [08:51:37] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.102373153 secs [08:58:51] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.48, 3.01, 1.75 [08:59:37] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09985864162 secs [09:02:52] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 2.13, 3.86, 2.48 [09:04:51] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.12, 2.95, 2.32 [09:10:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.71, 8.53, 7.55 [09:12:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 15.44, 11.07, 8.61 [09:14:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.97, 10.22, 8.61 [09:15:37] !log [universalomega@mwtask141] Running: sudo -u www-data php /srv/mediawiki/w/extensions/CreateWiki/maintenance/populateMainPage.php --wiki thelemarnewworldwiki [09:15:39] !log [universalomega@mwtask141] Running: sudo -u www-data php /srv/mediawiki/w/maintenance/createAndPromote.php Michato --bureaucrat --sysop --force --wiki thelemarnewworldwiki [09:15:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:15:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:16:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.05, 9.28, 8.44 [09:29:27] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/extensions/MirahezeMagic/maintenance/assignImportedEdits.php --wiki=comprehensibleinputwiki (END - exit=0) [09:29:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:36:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 8.21, 4.16, 2.45 [09:38:30] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.38, 3.54, 2.42 [09:40:06] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=hangoverswiki --missing (END - exit=65280) [09:40:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:40:29] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.13, 2.97, 2.35 [09:42:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.36, 9.78, 8.70 [09:44:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.64, 8.75, 8.46 [10:38:17] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.36, 3.68, 2.15 [10:46:17] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.91, 3.04, 2.69 [11:35:17] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1187286079 secs [11:45:17] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.08447784185 secs [12:06:11] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.24, 4.71, 2.30 [12:08:11] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.86, 5.18, 2.78 [12:09:38] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 14.90, 6.15, 2.98 [12:13:36] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.71, 3.59, 2.65 [12:15:35] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.49, 2.87, 2.49 [12:41:18] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.87, 3.93, 2.75 [12:43:16] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.54, 4.30, 3.04 [12:51:14] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 0.96, 3.52, 3.61 [12:51:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.26, 11.22, 9.81 [12:53:13] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.83, 3.53, 3.56 [12:53:37] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1213304996 secs [12:55:08] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.83, 10.02, 9.67 [12:55:12] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.32, 2.60, 3.21 [13:07:04] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.55, 3.65, 2.98 [13:11:20] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.92, 10.03, 9.63 [13:13:14] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.48, 9.25, 9.41 [13:18:58] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.35, 3.33, 3.63 [13:20:56] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.28, 2.63, 3.33 [13:21:37] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.0817155242 secs [13:25:17] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1204497218 secs [13:26:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.75, 9.39, 8.99 [13:27:17] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.09051114321 secs [13:28:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.51, 8.33, 8.66 [13:33:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.53, 9.55, 8.71 [13:37:34] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.33, 9.85, 9.14 [13:42:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.37, 10.78, 9.37 [13:46:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.78, 9.37, 9.14 [14:36:34] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 15.33, 11.57, 9.40 [14:38:32] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.98, 10.80, 9.35 [14:42:17] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.12, 12.42, 10.32 [14:42:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.42, 9.77, 9.32 [14:46:05] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.77, 11.12, 10.31 [14:49:54] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.72, 9.93, 9.97 [14:50:13] !log apply https://github.com/wikimedia/mediawiki-extensions-AbuseFilter/blob/REL1_38/db_patches/mysql/patch-split-afl_filter.sql to higyaanipediawiki [14:50:13] [url] mediawiki-extensions-AbuseFilter/patch-split-afl_filter.sql at REL1_38 · wikimedia/mediawiki-extensions-AbuseFilter · GitHub | github.com [14:50:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:06:00] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.28, 10.52, 9.32 [15:07:58] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.82, 10.26, 9.37 [15:09:56] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.46, 8.79, 8.94 [15:12:45] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.79, 10.31, 9.45 [15:14:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.55, 10.28, 9.57 [15:16:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.80, 10.06, 9.56 [15:18:36] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [15:18:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:18:44] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 7s [15:18:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:18:54] !log [paladox@mwtask141] starting deploy of {'config': True} to all [15:19:00] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 5s [15:19:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:19:06] miraheze/mw-config - paladox the build passed. [15:19:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:19:14] miraheze/mw-config - paladox the build passed. [15:25:54] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (END - exit=65280) [15:26:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:26:29] !log [@test131] starting deploy of {'config': True} to all [15:26:30] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [15:26:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:26:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:36:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.47, 4.61, 2.79 [15:46:28] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.86, 3.88, 3.63 [15:48:26] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.61, 3.07, 3.36 [15:55:43] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.nj.cn.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [16:05:52] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.09, 12.36, 10.25 [16:07:50] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.74, 11.60, 10.24 [16:09:48] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.40, 9.98, 9.80 [16:25:26] RECOVERY - wiki.nj.cn.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.nj.cn.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [16:37:00] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.56, 3.32, 2.62 [16:38:58] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.53, 4.08, 3.00 [16:44:55] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.08, 3.86, 3.30 [16:46:54] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.00, 3.26, 3.15 [17:02:51] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.04, 3.61, 2.25 [17:04:51] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.21, 2.76, 2.11 [17:08:40] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.80, 3.97, 2.87 [17:08:41] PROBLEM - wiki.albrusmusic.ru - reverse DNS on sslhost is WARNING: Timeout: The DNS operation timed out after 5.4037604331970215 seconds [17:10:08] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=hangoverswiki --missing (START) [17:10:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:11:37] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=elysiamcwiki --missing (START) [17:11:41] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=elysiamcwiki --missing (END - exit=0) [17:11:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:11:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:20:13] !log [paladox@mwtask141] starting deploy of {'world': True, 'l10n': True} to all [17:20:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:21:48] !log [paladox@test131] starting deploy of {'world': True, 'l10n': True} to all [17:21:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:24:35] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.42, 3.55, 4.00 [17:26:46] !log [paladox@test131] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 298s [17:26:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:28:07] !log apply https://github.com/wikimedia/mediawiki-extensions-AbuseFilter/blob/REL1_38/db_patches/mysql/patch-split-afl_filter.sql to astrobiologywiki [17:28:08] [url] mediawiki-extensions-AbuseFilter/patch-split-afl_filter.sql at REL1_38 · wikimedia/mediawiki-extensions-AbuseFilter · GitHub | github.com [17:28:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:28:32] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.01, 2.09, 3.29 [17:36:17] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.42, 11.38, 10.01 [17:40:32] !log [universalomega@mwtask141] Running: sudo -u www-data php /srv/mediawiki/w/extensions/CreateWiki/maintenance/populateMainPage.php --wiki bristolstudenthousingcoopwiki [17:40:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:40:41] !log [universalomega@mwtask141] Running: sudo -u www-data php /srv/mediawiki/w/maintenance/createAndPromote.php BristolSHC --bureaucrat --sysop --force --wiki bristolstudenthousingcoopwiki [17:40:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:42:11] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.20, 9.70, 9.81 [17:42:13] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.01, 3.41, 2.79 [17:42:39] could somebody please merge https://github.com/miraheze/dns/pull/389 [17:42:39] [url] add acme challenges for orain.org by MacFan4000 · Pull Request #389 · miraheze/dns · GitHub | github.com [17:43:05] Done MacFan4000 [17:44:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.15, 3.36, 2.85 [17:44:55] MacFan4000: puppet ran on ns* also. [17:46:43] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.54, 11.73, 10.09 [17:49:34] miraheze/mw-config - Universal-Omega the build passed. [17:50:00] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.76, 10.80, 10.11 [17:50:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.75, 11.42, 10.41 [17:55:54] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.27, 9.75, 9.85 [17:56:43] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.64, 9.85, 10.03 [17:56:47] !log [universalomega@mwtask141] starting deploy of {'world': True} to all [17:56:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:57:21] miraheze/DataDump - Universal-Omega the build passed. [17:57:29] !log [@test131] starting deploy of {'config': True} to all [17:57:30] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [17:57:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:57:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:01:49] !log [universalomega@mwtask141] finished deploy of {'world': True} to all - SUCCESS in 302s [18:01:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:06:39] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 15.63, 12.07, 10.52 [18:10:17] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.59, 4.34, 2.99 [18:10:35] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.41, 11.60, 10.74 [18:10:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.07, 10.41, 9.72 [18:11:27] !log [@mwtask141] starting deploy of {'config': True} to all [18:11:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:11:37] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 9s [18:11:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:12:17] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.78, 3.37, 2.80 [18:13:13] RECOVERY - poserdazfreebies.orain.org - LetsEncrypt on sslhost is OK: OK - Certificate 'orain.org' will expire on Fri 31 Mar 2023 16:45:32 GMT +0000. [18:16:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.51, 9.81, 10.20 [18:16:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.48, 11.14, 10.36 [18:16:46] RECOVERY - allthetropes.orain.org - LetsEncrypt on sslhost is OK: OK - Certificate 'orain.org' will expire on Fri 31 Mar 2023 16:45:32 GMT +0000. [18:17:29] !log [universalomega@mwtask141] starting deploy of {'world': True} to all [18:17:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:18:04] miraheze/DataDump - Universal-Omega the build passed. [18:18:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.10, 11.76, 10.65 [18:20:22] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.78, 12.34, 11.20 [18:22:11] !log [universalomega@mwtask141] finished deploy of {'world': True} to all - SUCCESS in 281s [18:22:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:22:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.28, 11.51, 11.03 [18:22:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.34, 11.16, 10.67 [18:23:54] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.51, 3.43, 3.07 [18:24:03] RECOVERY - orain.org - LetsEncrypt on sslhost is OK: OK - Certificate 'orain.org' will expire on Fri 31 Mar 2023 16:45:32 GMT +0000. [18:25:54] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.83, 3.21, 3.02 [18:28:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.10, 11.14, 10.64 [18:29:06] RECOVERY - isv.orain.org - LetsEncrypt on sslhost is OK: OK - Certificate 'orain.org' will expire on Fri 31 Mar 2023 16:45:32 GMT +0000. [18:32:10] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.19, 11.62, 10.90 [18:32:57] !log [universalomega@mwtask141] starting deploy of {'world': True} to all [18:33:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:34:38] miraheze/DataDump - Universal-Omega the build passed. [18:35:54] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.32, 3.60, 3.24 [18:37:34] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.44, 6.42, 5.02 [18:37:54] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.92, 3.88, 3.40 [18:38:07] RECOVERY - wiki.albrusmusic.ru - reverse DNS on sslhost is OK: SSL OK - wiki.albrusmusic.ru reverse DNS resolves to cp22.miraheze.org - CNAME OK [18:38:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.07, 11.95, 11.66 [18:38:51] !log [universalomega@mwtask141] finished deploy of {'world': True} to all - SUCCESS in 353s [18:38:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:39:34] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 3.45, 5.43, 4.84 [18:40:01] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.91, 11.09, 11.41 [18:41:54] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.45, 4.07, 3.58 [18:43:54] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.41, 3.82, 3.54 [18:44:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.27, 11.97, 11.64 [18:46:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.83, 10.67, 11.20 [18:51:54] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.19, 3.30, 3.40 [18:55:50] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.59, 3.82, 3.58 [18:57:20] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.56, 9.23, 10.15 [18:57:34] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=hangoverswiki --missing (END - exit=0) [18:57:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:59:39] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.21, 3.69, 3.60 [19:00:35] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.89, 8.30, 9.78 [19:03:29] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.68, 3.09, 3.36 [19:05:05] miraheze/DataDump - Universal-Omega the build passed. [19:08:15] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.31, 3.60, 2.49 [19:10:15] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.47, 3.53, 2.60 [19:12:15] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.13, 3.35, 2.64 [19:28:09] miraheze/DataDump - Universal-Omega the build passed. [19:30:19] !log [universalomega@mwtask141] starting deploy of {'world': True} to all [19:30:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:35:36] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.02, 10.84, 9.09 [19:36:08] !log [universalomega@mwtask141] finished deploy of {'world': True} to all - SUCCESS in 348s [19:36:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:39:36] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.80, 11.21, 9.71 [19:40:41] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 16.40, 12.06, 9.58 [19:45:30] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1599564552 secs [19:45:36] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.49, 9.05, 9.32 [19:46:33] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.60, 11.92, 10.53 [19:56:21] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.35, 9.38, 9.88 [20:05:17] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.70, 11.63, 10.33 [20:07:09] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 9.41, 7.65, 4.34 [20:07:11] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.85, 11.31, 10.37 [20:09:05] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.81, 10.12, 10.05 [20:11:30] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.0520221293 secs [20:13:08] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.45, 3.40, 3.46 [20:15:07] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.09, 2.56, 3.13 [20:29:37] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gensekaiwiki /home/reception/wikibackups1812/gensekaiwiki.xml (START) [20:29:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:30:47] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gensekaiwiki /home/reception/wikibackups1812/gensekaiwiki.xml (END - exit=0) [20:30:48] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=gensekaiwiki (START) [20:30:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:30:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:31:22] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=gensekaiwiki (END - exit=0) [20:31:23] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=gensekaiwiki --missing (START) [20:31:27] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=gensekaiwiki --missing (END - exit=0) [20:31:28] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=gensekaiwiki --active --update (END - exit=0) [20:31:29] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=decodedocdigitizewiki /home/reception/wikibackups1812/decodedocdigitizewiki.xml (START) [20:31:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:31:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:31:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:31:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:31:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:34:47] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.10, 10.49, 9.24 [20:38:36] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.30, 10.09, 9.39 [20:43:14] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.11, 9.73, 9.01 [20:45:12] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 10.04, 10.19, 9.30 [21:05:36] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 15.58, 11.36, 9.33 [21:07:02] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 9.31, 6.05, 3.43 [21:07:36] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.43, 10.44, 9.23 [21:09:36] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.56, 9.74, 9.12 [21:13:02] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.50, 3.51, 3.18 [21:15:02] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.44, 2.85, 2.98 [21:17:52] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=decodedocdigitizewiki /home/reception/wikibackups1812/decodedocdigitizewiki.xml (END - exit=0) [21:17:53] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=decodedocdigitizewiki (START) [21:17:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:18:01] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.08, 3.06, 2.42 [21:18:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:20:01] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.20, 2.72, 2.37 [21:20:25] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.63, 10.47, 9.61 [21:22:23] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.94, 9.34, 9.30 [21:28:13] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.64, 9.77, 9.38 [21:28:16] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 3.02, 4.11, 2.83 [21:29:14] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=decodedocdigitizewiki (END - exit=0) [21:29:15] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=decodedocdigitizewiki --missing (START) [21:29:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:29:22] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=decodedocdigitizewiki --missing (END - exit=0) [21:29:23] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=decodedocdigitizewiki --active --update (END - exit=0) [21:29:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:29:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:29:36] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.27, 9.92, 9.09 [21:29:45] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=evolutionismwiki /home/reception/wikibackups1812/evolutionismwiki.xml (START) [21:29:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:29:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:30:01] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.59, 3.11, 2.62 [21:30:10] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.90, 9.93, 9.50 [21:30:15] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.64, 3.26, 2.67 [21:31:36] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.09, 9.33, 8.98 [21:32:01] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.62, 2.95, 2.62 [21:35:00] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.27, 3.65, 2.98 [21:38:59] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.03, 3.77, 3.21 [21:40:59] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.41, 2.89, 2.95 [21:43:50] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 17.83, 12.21, 10.09 [21:45:36] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.31, 9.69, 9.35 [21:47:36] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.41, 8.95, 9.13 [21:49:43] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.37, 11.07, 10.47 [21:49:58] PROBLEM - mw121 Puppet on mw121 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 5 minutes ago with 0 failures [21:51:41] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.10, 9.89, 10.12 [22:06:01] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 5.68, 4.37, 3.10 [22:07:56] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.88, 3.74, 2.81 [22:07:56] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (START) [22:07:58] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (END - exit=65280) [22:08:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:08:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:09:56] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.80, 3.41, 2.81 [22:10:01] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.38, 3.46, 3.03 [22:12:01] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 6.84, 4.54, 3.46 [22:13:55] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/findBadBlobs.php --wiki=webkinzguidewiki --revisions=252727 (END - exit=0) [22:14:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:30:14] PROBLEM - prometheus131 Puppet on prometheus131 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 7 minutes ago with 0 failures [22:31:33] [Grafana] !sre FIRING: There has been a rise in the MediaWiki exception rate https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [22:32:59] PROBLEM - prometheus131 Prometheus on prometheus131 is CRITICAL: connect to address 2a10:6740::6:402 and port 9090: Connection refused [22:38:50] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/populateContentTables.php --wiki=webkinzguide (END - exit=65280) [22:38:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:38:59] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/populateContentTables.php --wiki=webkinzguidewiki (END - exit=0) [22:39:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:42:59] RECOVERY - prometheus131 Prometheus on prometheus131 is OK: TCP OK - 0.001 second response time on 2a10:6740::6:402 port 9090 [22:46:30] [Grafana] !sre RESOLVED: MediaWiki Exception Rate https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [22:57:13] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.63, 4.28, 2.67 [22:59:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.49, 3.52, 2.58 [23:01:10] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.33, 2.75, 2.41 [23:24:52] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.11, 3.69, 2.42 [23:26:50] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.71, 2.90, 2.28 [23:36:43] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.88, 4.16, 3.02 [23:38:43] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.30, 3.63, 2.96 [23:40:43] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.15, 3.12, 2.86 [23:46:46] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=webkinzguidewiki /home/void/delete.txt -r Corrupted page (START) [23:46:48] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=webkinzguidewiki /home/void/delete.txt -r Corrupted page (END - exit=256) [23:46:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:47:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:48:36] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (START) [23:48:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:50:46] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (END - exit=65280) [23:50:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:54:35] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=webkinzguidewiki /home/void/delete.txt -r Corrupted page (START) [23:54:36] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=webkinzguidewiki /home/void/delete.txt -r Corrupted page (END - exit=256) [23:54:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:54:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:54:47] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (START) [23:54:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:55:17] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (END - exit=65280) [23:55:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:56:35] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=webkinzguidewiki /home/void/delete.txt -r Corrupted page (START) [23:56:37] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=webkinzguidewiki /home/void/delete.txt -r Corrupted page (END - exit=256) [23:56:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:56:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:56:52] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (START) [23:56:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:57:31] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=webkinzguidewiki --missing (END - exit=65280) [23:57:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:57:58] RECOVERY - mw121 Puppet on mw121 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [23:59:25] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.63, 9.67, 8.24 [23:59:57] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=webkinzguidewiki /home/void/delete.txt -r Corrupted page (START) [23:59:58] !log [void@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=webkinzguidewiki /home/void/delete.txt -r Corrupted page (END - exit=256)