[00:01:32] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.95, 3.98, 3.30 [00:07:22] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.02, 4.16, 3.52 [00:09:18] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.05, 3.73, 3.44 [00:11:15] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.75, 3.18, 3.29 [00:12:10] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 4.65, 4.19, 2.67 [00:15:59] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 1.89, 3.86, 2.97 [00:16:03] PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.1910521984 secs [00:17:53] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.68, 3.20, 2.83 [00:24:03] RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset 0.08031085134 secs [00:25:31] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.53, 3.66, 3.27 [00:27:26] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.15, 3.21, 3.15 [00:50:31] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.58, 3.40, 2.77 [00:52:31] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.58, 3.16, 2.76 [01:36:01] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.60, 3.83, 2.90 [01:37:57] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.97, 3.78, 2.99 [01:39:54] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.75, 4.48, 3.36 [01:45:31] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.51, 3.57, 2.89 [01:45:43] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.36, 3.39, 3.34 [01:47:26] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.47, 3.06, 2.78 [02:08:07] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.74, 3.76, 3.06 [02:10:03] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.44, 3.98, 3.24 [02:12:00] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.74, 4.23, 3.41 [02:15:37] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 9.20, 3.59, 1.86 [02:17:37] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 3.75, 3.54, 2.06 [02:19:37] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.97, 3.02, 2.06 [02:23:39] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.33, 3.28, 3.83 [02:26:54] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.77, 4.65, 3.47 [02:27:32] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.46, 3.34, 3.69 [02:29:28] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.55, 2.59, 3.37 [02:34:34] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.83, 3.92, 3.64 [02:36:31] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.91, 3.12, 3.37 [02:52:51] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.19, 3.35, 2.86 [02:54:51] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.66, 3.65, 3.05 [02:56:51] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.89, 2.94, 2.85 [03:00:32] RECOVERY - db142 Backups SQL on db142 is OK: FILE_AGE OK: /var/log/sql-backup.log is 31 seconds old and 0 bytes [03:02:31] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.27, 5.07, 3.63 [03:06:31] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.94, 3.45, 3.31 [03:06:51] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.16, 4.00, 3.14 [03:08:31] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.51, 2.79, 3.08 [03:08:51] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.34, 3.90, 3.22 [03:10:51] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.00, 3.19, 3.04 [03:18:51] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.80, 3.55, 3.24 [03:20:51] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.06, 3.63, 3.31 [03:22:51] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.99, 3.50, 3.31 [03:24:51] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.39, 3.05, 3.16 [03:25:22] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.91, 4.14, 3.31 [03:27:17] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.63, 3.79, 3.27 [03:29:12] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.78, 4.82, 3.73 [03:33:02] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.88, 3.75, 3.57 [03:34:56] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.16, 2.82, 3.25 [03:42:08] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1082151532 secs [03:55:05] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.32, 4.50, 3.60 [04:00:50] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.80, 3.51, 3.50 [04:02:45] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.90, 3.14, 3.38 [04:09:04] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.88, 4.22, 3.12 [04:11:00] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.11, 3.78, 3.09 [04:14:08] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.01016977429 secs [04:14:53] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.52, 2.84, 2.88 [04:26:43] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.10, 3.44, 3.20 [04:28:38] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.08, 3.09, 3.11 [04:37:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.87, 4.78, 3.52 [04:41:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.74, 3.52, 3.32 [04:43:22] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.52, 3.93, 3.48 [04:45:19] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.77, 3.14, 3.25 [04:45:36] [02mw-config] 07AgentIsai opened pull request 03#5073: Enable StopForumSpam globally - 13https://github.com/miraheze/mw-config/pull/5073 [04:46:41] miraheze/mw-config - AgentIsai the build passed. [04:50:31] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.67, 3.08, 2.69 [04:52:31] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 3.05, 2.85, 2.64 [04:56:31] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.49, 3.46, 3.03 [04:58:31] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.43, 2.72, 2.80 [05:06:51] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.02, 4.25, 3.25 [05:18:08] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset -0.1159570217 secs [05:22:51] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.57, 2.95, 3.86 [05:26:51] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.53, 2.41, 3.40 [05:26:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 9.29, 5.20, 3.55 [05:30:08] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.09134054184 secs [05:32:36] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.30, 3.89, 3.50 [05:34:31] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.24, 3.28, 3.31 [06:12:50] PROBLEM - cloud12 Puppet on cloud12 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [06:22:24] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=nirvanainfirewiki --no-updates --username-prefix=mh:thelangyalist /home/reception/ImportDump/nirvanainfirewiki-20230121062111.xml (START) [06:22:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:26:10] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a7519abb16ac...55c1e4d8e0f6 [06:26:11] [02miraheze/ssl] 07Reception123 0355c1e4d - temporarily redirect thelangyalistwiki to nirvanainfirewiki per T10178 [06:27:36] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 6.86, 6.29, 2.97 [06:27:48] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=nirvanainfirewiki --no-updates --username-prefix=mh:thelangyalist /home/reception/ImportDump/nirvanainfirewiki-20230121062111.xml (END - exit=0) [06:27:49] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=nirvanainfirewiki (START) [06:27:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:27:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:28:52] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.72, 3.36, 2.58 [06:30:51] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.64, 2.72, 2.43 [06:31:29] [02mw-config] 07Reception123 commented on pull request 03#5053: T10231: Add wgModerationOnlyInNamespaces and wgModerationIgnoredInNamespaces to ManageWikiSettings.php - 13https://github.com/miraheze/mw-config/pull/5053#issuecomment-1399191886 [06:31:37] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.87, 3.31, 2.49 [06:36:46] [02mw-config] 07Reception123 synchronize pull request 03#5053: T10231: Add wgModerationOnlyInNamespaces and wgModerationIgnoredInNamespaces to ManageWikiSettings.php - 13https://github.com/miraheze/mw-config/pull/5053 [06:37:37] [02mw-config] 07Reception123 edited pull request 03#5053: T10231: Add wgModerationOnlyInNamespaces to ManageWikiNamespaces.php - 13https://github.com/miraheze/mw-config/pull/5053 [06:37:49] miraheze/mw-config - Reception123 the build passed. [06:38:01] [02mw-config] 07Reception123 closed pull request 03#5053: T10231: Add wgModerationOnlyInNamespaces to ManageWikiNamespaces.php - 13https://github.com/miraheze/mw-config/pull/5053 [06:38:04] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/b9b1b7090e6a...a73f2dfc9a0b [06:38:05] [02miraheze/mw-config] 07ugochimobi 03a73f2df - T10231: Add wgModerationOnlyInNamespaces to ManageWikiNamespaces.php (#5053) [06:38:50] RECOVERY - cloud12 Puppet on cloud12 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures [06:39:05] miraheze/mw-config - Reception123 the build passed. [06:40:17] !log [@mwtask141] starting deploy of {'config': True} to all [06:40:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:40:24] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 6s [06:40:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:56:42] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.02, 3.70, 2.91 [06:57:14] !log [@test131] starting deploy of {'config': True} to all [06:57:14] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [06:57:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:57:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:02:39] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.51, 3.35, 3.21 [07:04:08] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=nirvanainfirewiki (END - exit=0) [07:04:09] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=nirvanainfirewiki --active --update (END - exit=0) [07:04:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:04:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:10:36] !log wikibackups for private wikis completed [07:10:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:24:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.24, 3.51, 2.67 [07:28:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.16, 3.89, 3.04 [07:30:39] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.04, 3.21, 2.89 [08:20:26] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.2363108695 secs [08:22:53] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.69, 5.07, 3.09 [08:28:43] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.08, 3.99, 3.34 [08:30:40] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.27, 3.10, 3.10 [08:31:55] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.36, 3.57, 2.26 [08:33:50] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.37, 2.66, 2.08 [08:34:26] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset -0.01532116532 secs [08:35:02] !log reception@mwtask141:~/lhmnwiki$ sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=lhmnwiki --summary="From: lhmnwiki.miraheze.org" --search-recursively /home/reception/lhmnwiki [08:35:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:37:43] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.22, 4.41, 2.98 [08:39:38] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.08, 3.75, 2.89 [08:41:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.34, 3.40, 2.85 [08:44:41] !log DELETED and DROPPED wildriftwiki (wiki reset, T10201) [08:44:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:52:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.58, 2.83, 2.30 [08:54:39] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.16, 2.62, 2.29 [08:58:05] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/dns/compare/e4d9770eb274...a6ab20d9af04 [08:58:06] [02miraheze/dns] 07Reception123 03a6ab20d - add wonderfuleveryday.org zone [09:00:33] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/55c1e4d8e0f6...35d9aff41dba [09:00:35] [02miraheze/ssl] 07MirahezeSSLBot 0335d9aff - Bot: Add SSL cert for wonderfuleveryday.org [09:09:41] [02mw-config] 07ugochimobi commented on pull request 03#5019: Wikibase client and repo settings that are currently missing - 13https://github.com/miraheze/mw-config/pull/5019#issuecomment-1399213070 [09:09:55] [02mw-config] 07ugochimobi edited a comment on pull request 03#5019: Wikibase client and repo settings that are currently missing - 13https://github.com/miraheze/mw-config/pull/5019#issuecomment-1399213070 [09:15:04] [02mw-config] 07ugochimobi commented on pull request 03#5066: make gpcommonswiki the wgGlobalUsageDatabase for gp wikis - 13https://github.com/miraheze/mw-config/pull/5066#issuecomment-1399213884 [09:23:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.67, 3.99, 2.62 [09:27:06] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.67, 4.43, 3.09 [09:29:04] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.52, 3.29, 2.84 [10:00:58] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.53, 3.66, 2.78 [10:02:58] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.41, 2.87, 2.60 [10:53:33] PROBLEM - cloud13 Puppet on cloud13 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [11:06:57] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.62, 2.57, 1.79 [11:08:55] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.18, 2.98, 2.06 [11:21:33] RECOVERY - cloud13 Puppet on cloud13 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:26:47] PROBLEM - mwtask141 Disk Space on mwtask141 is WARNING: DISK WARNING - free space: / 7348 MB (10% inode=90%); [12:07:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.22, 2.98, 1.90 [12:09:30] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.75, 3.19, 2.12 [12:11:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.09, 4.17, 2.63 [12:25:29] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.41, 3.96, 3.80 [12:27:29] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.37, 2.72, 3.36 [13:58:32] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 6.82, 3.81, 2.38 [14:00:32] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.89, 2.98, 2.26 [14:09:57] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.30, 3.64, 2.49 [14:10:43] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 12.85, 14.71, 7.29 [14:13:54] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.35, 2.88, 2.45 [14:14:33] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 0.95, 7.19, 5.84 [14:16:28] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 1.25, 5.28, 5.30 [14:33:37] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.81, 4.76, 3.07 [14:39:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.70, 3.95, 3.23 [14:43:30] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.81, 3.25, 3.10 [14:47:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 9.75, 10.20, 6.20 [14:55:29] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.00, 2.66, 3.97 [14:59:06] !log reception@db121:~$ sudo service mysql restart [14:59:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:01:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.10, 3.16, 3.70 [15:03:29] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.12, 2.91, 3.56 [15:04:31] [02TSPortal] 07redbluegreenhat opened pull request 03#8: T10167: Add ability to copy to clipboard items - 13https://github.com/miraheze/TSPortal/pull/8 [15:05:29] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.99, 2.61, 3.36 [15:06:09] [02TSPortal] 07redbluegreenhat commented on pull request 03#8: T10167: Add ability to copy to clipboard items - 13https://github.com/miraheze/TSPortal/pull/8#issuecomment-1399267996 [15:09:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.19, 3.59, 3.59 [15:13:29] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.19, 3.22, 3.44 [15:14:14] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03Reception123-patch-3 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/2a2926f8e714 [15:14:17] [02miraheze/mw-config] 07Reception123 032a2926f - maxCategoryCount to 7 for dragontamerwiki T10321 [15:14:19] [02mw-config] 07Reception123 created branch 03Reception123-patch-3 - 13https://github.com/miraheze/mw-config [15:14:20] [02mw-config] 07Reception123 opened pull request 03#5074: maxCategoryCount to 7 for dragontamerwiki T10321 - 13https://github.com/miraheze/mw-config/pull/5074 [15:15:22] miraheze/mw-config - Reception123 the build passed. [15:15:29] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.53, 2.64, 3.20 [15:15:47] [02mw-config] 07Reception123 closed pull request 03#5074: maxCategoryCount to 7 for dragontamerwiki T10321 - 13https://github.com/miraheze/mw-config/pull/5074 [15:15:49] [02miraheze/mw-config] 07Reception123 deleted branch 03Reception123-patch-3 [15:15:51] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/a73f2dfc9a0b...f4bd52e5dd49 [15:15:54] [02miraheze/mw-config] 07Reception123 03f4bd52e - maxCategoryCount to 7 for dragontamerwiki T10321 (#5074) [15:15:56] [02mw-config] 07Reception123 deleted branch 03Reception123-patch-3 - 13https://github.com/miraheze/mw-config [15:16:14] !log [reception@mwtask141] starting deploy of {'config': True} to all [15:16:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:16:20] !log [reception@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 5s [15:16:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:16:49] miraheze/mw-config - Reception123 the build passed. [15:16:51] [02puppet] 07RhinosF1 opened pull request 03#3128: prep for 1.38 - 13https://github.com/miraheze/puppet/pull/3128 [15:17:42] [02puppet] 07RhinosF1 synchronize pull request 03#3128: prep for 1.38 - 13https://github.com/miraheze/puppet/pull/3128 [15:18:12] [02puppet] 07RhinosF1 edited pull request 03#3128: prep for 1.39 - 13https://github.com/miraheze/puppet/pull/3128 [15:19:02] [02puppet] 07RhinosF1 opened pull request 03#3129: depool all servers for 1.39 upgrade from deploy - 13https://github.com/miraheze/puppet/pull/3129 [15:22:50] [02puppet] 07RhinosF1 synchronize pull request 03#3128: prep for 1.39 - 13https://github.com/miraheze/puppet/pull/3128 [15:26:30] [02puppet] 07RhinosF1 synchronize pull request 03#3128: prep for 1.39 - 13https://github.com/miraheze/puppet/pull/3128 [15:26:45] !log [@test131] starting deploy of {'config': True} to all [15:26:46] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [15:26:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:26:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:33:22] [02CreateWiki] 07redbluegreenhat commented on pull request 03#357: T9997: Replace Miraheze-specific messages with generic ones - 13https://github.com/miraheze/CreateWiki/pull/357#issuecomment-1399272302 [15:33:54] [02CreateWiki] 07redbluegreenhat edited a comment on pull request 03#357: T9997: Replace Miraheze-specific messages with generic ones - 13https://github.com/miraheze/CreateWiki/pull/357#issuecomment-1399272302 [15:36:24] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.27, 3.75, 3.03 [15:38:23] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.67, 3.59, 3.07 [15:40:34] !log [@mwtask141] starting deploy of {'config': True} to all [15:40:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:40:41] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 6s [15:40:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:42:20] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.06, 3.12, 3.01 [15:52:58] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.54, 4.45, 2.61 [15:56:58] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.48, 3.39, 2.67 [16:19:49] PROBLEM - matomo131 Current Load on matomo131 is WARNING: WARNING - load average: 3.69, 3.10, 2.61 [16:21:49] RECOVERY - matomo131 Current Load on matomo131 is OK: OK - load average: 2.83, 3.02, 2.64 [16:26:58] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.69, 3.21, 2.37 [16:28:58] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.73, 2.92, 2.37 [16:35:07] [02CreateWiki] 07Universal-Omega commented on pull request 03#357: T9997: Replace Miraheze-specific messages with generic ones - 13https://github.com/miraheze/CreateWiki/pull/357#issuecomment-1399284569 [16:36:34] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.03, 3.32, 2.39 [16:38:32] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.19, 2.90, 2.34 [16:49:33] [02puppet] 07Universal-Omega closed pull request 03#3128: prep for 1.39 - 13https://github.com/miraheze/puppet/pull/3128 [16:49:36] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/0eb99a8a1e7c...f9405be8d25b [16:49:37] [02miraheze/puppet] 07RhinosF1 03f9405be - prep for 1.39 (#3128) [16:54:07] [02mw-config] 07Universal-Omega synchronize pull request 03#5070: Update sitenotice and read-only - 13https://github.com/miraheze/mw-config/pull/5070 [16:54:22] [02mw-config] 07Universal-Omega edited pull request 03#5070: Update sitenotice - 13https://github.com/miraheze/mw-config/pull/5070 [16:54:58] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 9.31, 5.41, 3.33 [16:55:01] miraheze/mw-config - Universal-Omega the build passed. [16:58:58] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.71, 3.60, 3.09 [17:00:58] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.58, 3.21, 2.99 [17:02:03] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.zulaclub.net. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [17:10:18] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03update-schema-paths-1.39 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/44120c04b48d [17:10:19] [02miraheze/mw-config] 07Universal-Omega 0344120c0 - Update schema patch paths for 1.39 [17:10:20] [02mw-config] 07Universal-Omega created branch 03update-schema-paths-1.39 - 13https://github.com/miraheze/mw-config [17:10:21] [02mw-config] 07Universal-Omega opened pull request 03#5075: Update schema patch paths for 1.39 - 13https://github.com/miraheze/mw-config/pull/5075 [17:11:04] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.17, 3.75, 2.99 [17:11:19] miraheze/mw-config - Universal-Omega the build passed. [17:13:02] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.97, 3.05, 2.82 [17:16:57] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03update-schema-paths-1.39 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/44120c04b48d...d94ac6c298e3 [17:16:59] [02miraheze/mw-config] 07Universal-Omega 03d94ac6c - Update ManageWikiExtensions.php [17:17:02] [02mw-config] 07Universal-Omega synchronize pull request 03#5075: Update schema patch paths for 1.39 - 13https://github.com/miraheze/mw-config/pull/5075 [17:17:57] miraheze/mw-config - Universal-Omega the build passed. [17:25:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.71, 3.62, 2.88 [17:29:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.27, 2.68, 2.69 [17:30:36] RECOVERY - wiki.zulaclub.net - reverse DNS on sslhost is OK: SSL OK - wiki.zulaclub.net reverse DNS resolves to cp23.miraheze.org - CNAME OK [17:46:27] !log [universalomega@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/OAuth/schema/mysql/patch-oauth_accepted_consumer-timestamp.sql (START) [17:46:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:46:31] !log [universalomega@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/OAuth/schema/mysql/patch-oauth_accepted_consumer-timestamp.sql (END - exit=0) [17:46:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:47:33] !log [universalomega@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/OAuth/schema/mysql/patch-oauth_registered_consumer-timestamp.sql (START) [17:47:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:47:36] !log [universalomega@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/OAuth/schema/mysql/patch-oauth_registered_consumer-timestamp.sql (END - exit=0) [17:47:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:48:31] !log [universalomega@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/CheckUser/schema/mysql/patch-cu_changes-cuc_timestamp.sql (START) [17:48:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:48:37] !log [universalomega@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/CheckUser/schema/mysql/patch-cu_changes-cuc_timestamp.sql (END - exit=0) [17:48:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:49:23] !log [universalomega@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/CheckUser/schema/mysql/patch-cu_log-comment_table_for_reason.sql (START) [17:49:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:49:26] !log [universalomega@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/CheckUser/schema/mysql/patch-cu_log-comment_table_for_reason.sql (END - exit=256) [17:49:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:50:34] !log [universalomega@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/CheckUser/schema/mysql/patch-cu_log-actor.sql (START) [17:50:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:50:38] !log [universalomega@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/CheckUser/schema/mysql/patch-cu_log-actor.sql (END - exit=256) [17:50:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:51:07] !log [universalomega@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/AntiSpoof/sql/mysql/patch-spoofuser-index-su_normname_idx.sql (START) [17:51:09] !log [universalomega@test131] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/beta.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/AntiSpoof/sql/mysql/patch-spoofuser-index-su_normname_idx.sql (END - exit=256) [17:51:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:51:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:52:53] [02puppet] 07Universal-Omega closed pull request 03#3129: depool all servers for 1.39 upgrade from deploy - 13https://github.com/miraheze/puppet/pull/3129 [17:52:55] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/f9405be8d25b...2076480aa872 [17:52:56] [02miraheze/puppet] 07RhinosF1 032076480 - depool all servers for 1.39 upgrade from deploy (#3129) [17:59:15] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/2076480aa872...d73a327062ac [17:59:16] [02miraheze/puppet] 07Universal-Omega 03d73a327 - mediawiki: switch to REL1_39 branch [17:59:50] PROBLEM - matomo131 Current Load on matomo131 is CRITICAL: CRITICAL - load average: 4.25, 3.24, 2.72 [18:01:49] PROBLEM - matomo131 Current Load on matomo131 is WARNING: WARNING - load average: 3.42, 3.24, 2.78 [18:02:27] [02miraheze/MirahezeDebug] 07Universal-Omega pushed 031 commit to 03manifest-version-3 [+0/-0/±1] 13https://github.com/miraheze/MirahezeDebug/compare/3c3e3160b7a5...10937d4bc825 [18:02:28] [02miraheze/MirahezeDebug] 07Universal-Omega 0310937d4 - Update background.js [18:02:31] [02MirahezeDebug] 07Universal-Omega synchronize pull request 03#11: Convert to manifest version 3 - 13https://github.com/miraheze/MirahezeDebug/pull/11 [18:03:50] RECOVERY - matomo131 Current Load on matomo131 is OK: OK - load average: 3.07, 3.18, 2.82 [18:04:13] !log disable puppet on mwtask141 [18:04:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:06:53] PROBLEM - mwtask141 Puppet on mwtask141 is WARNING: WARNING: Puppet is currently disabled, message: 1.39 upgrade -- Universal Omega, last run 5 minutes ago with 0 failures [18:10:03] !log starting MediaWiki 1.39 upgrade (on mwtask141 only — other servers will be done in maintenance window). [18:10:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:10:23] !log sudo -u www-data rm -rf /srv/mediawiki-staging/w on mwtask141 [18:10:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:10:47] RECOVERY - mwtask141 Disk Space on mwtask141 is OK: DISK OK - free space: / 10204 MB (14% inode=94%); [18:17:23] !log REL1_39 cloning into /srv/mediawiki-staging/w on mwtask141 [18:17:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:18:34] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 5.06, 3.60, 2.01 [18:19:16] [02mw-config] 07RhinosF1 reviewed pull request 03#5070 commit - 13https://github.com/miraheze/mw-config/pull/5070#discussion_r1083319507 [18:20:34] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.96, 3.71, 2.25 [18:24:35] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.97, 3.32, 2.43 [18:26:47] PROBLEM - mwtask141 Disk Space on mwtask141 is WARNING: DISK WARNING - free space: / 7176 MB (10% inode=93%); [18:37:53] !log re-enable and run puppet on mwtask141 [18:37:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:38:55] RECOVERY - mwtask141 Puppet on mwtask141 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:52:16] !log stop jobrunner on mwtask141 and disable puppet again [18:52:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:53:02] !log [universalomega@mwtask141] starting deploy of {'world': True, 'l10n': True, 'extensionlist': True, 'force': True, 'ignoretime': True} to mwtask141 [18:53:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:54:53] PROBLEM - mwtask141 Puppet on mwtask141 is WARNING: WARNING: Puppet is currently disabled, message: 1.39 upgrade -- Universal Omega, last run 3 minutes ago with 2 failures [18:54:58] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.95, 3.55, 2.51 [18:55:05] PROBLEM - mwtask141 JobRunner Service on mwtask141 is CRITICAL: PROCS CRITICAL: 0 processes with args 'redisJobRunnerService' [18:56:58] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.95, 3.59, 2.68 [19:00:00] PROBLEM - mwtask141 MediaWiki Rendering on mwtask141 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 8191 bytes in 0.745 second response time [19:00:58] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.83, 2.75, 2.55 [19:03:54] RECOVERY - mwtask141 MediaWiki Rendering on mwtask141 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.392 second response time [19:05:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 3.81, 4.74, 3.46 [19:07:29] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.43, 3.85, 3.28 [19:09:29] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.21, 3.37, 3.17 [19:10:34] !log [universalomega@mwtask141] finished deploy of {'world': True, 'l10n': True, 'extensionlist': True, 'force': True, 'ignoretime': True} to mwtask141 - SUCCESS in 1052s [19:10:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:11:44] [02MirahezeMagic] 07Universal-Omega closed pull request 03#360: Support MediaWiki 1.39+ - 13https://github.com/miraheze/MirahezeMagic/pull/360 [19:11:47] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/MirahezeMagic/compare/af49e0aec63b...6bc9b3b2e6e6 [19:11:49] [02miraheze/MirahezeMagic] 07Universal-Omega 036bc9b3b - Support MediaWiki 1.39+ (#360) [19:12:49] [02mw-config] 07Universal-Omega closed pull request 03#5075: Update schema patch paths for 1.39 - 13https://github.com/miraheze/mw-config/pull/5075 [19:12:50] [02mw-config] 07Universal-Omega deleted branch 03update-schema-paths-1.39 - 13https://github.com/miraheze/mw-config [19:12:53] [02miraheze/mw-config] 07Universal-Omega deleted branch 03update-schema-paths-1.39 [19:12:55] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/mw-config/compare/f4bd52e5dd49...4fff83b4a0af [19:12:56] [02miraheze/mw-config] 07Universal-Omega 034fff83b - Update schema patch paths for 1.39 (#5075) [19:13:48] miraheze/mw-config - Universal-Omega the build passed. [19:16:25] miraheze/MirahezeMagic - Universal-Omega the build has errored. [19:16:54] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/4fff83b4a0af...d94c9f4e8130 [19:16:55] [02miraheze/mw-config] 07Universal-Omega 03d94c9f4 - Disable PdfBook and Slack Notifications [19:17:55] miraheze/mw-config - Universal-Omega the build passed. [19:26:56] !log [@test131] starting deploy of {'config': True} to all [19:26:57] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [19:36:41] !log [universalomega@mwtask141] starting deploy of {'pull': 'config', 'config': True} to mwtask141 [19:36:43] !log [universalomega@mwtask141] finished deploy of {'pull': 'config', 'config': True} to mwtask141 - SUCCESS in 1s [19:36:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:36:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:42:35] !log [universalomega@mwtask141] starting deploy of {'force': True, 'folders': 'w/vendor', 'ignoretime': True} to mwtask141 [19:42:38] !log [universalomega@mwtask141] finished deploy of {'force': True, 'folders': 'w/vendor', 'ignoretime': True} to mwtask141 - SUCCESS in 2s [19:42:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:42:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:43:36] !log [universalomega@mwtask141] starting deploy of {'force': True, 'folders': 'w/vendor', 'ignoretime': True} to mwtask141 [19:43:37] !log [universalomega@mwtask141] finished deploy of {'force': True, 'folders': 'w/vendor', 'ignoretime': True} to mwtask141 - SUCCESS in 0s [19:43:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:43:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:53:07] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.95, 3.16, 2.29 [19:55:04] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 3.39, 3.32, 2.47 [19:55:07] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03revert-3129-patch-50 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/329ab14d1d26 [19:55:09] [02miraheze/puppet] 07Universal-Omega 03329ab14 - Revert "depool all servers for 1.39 upgrade from deploy (#3129)" [19:55:12] [02puppet] 07Universal-Omega created branch 03revert-3129-patch-50 - 13https://github.com/miraheze/puppet [19:55:13] [02puppet] 07Universal-Omega opened pull request 03#3130: Revert "depool all servers for 1.39 upgrade from deploy" - 13https://github.com/miraheze/puppet/pull/3130 [19:57:35] [02SpriteSheet] 07pppery opened pull request 03#13: Updated README - 13https://github.com/miraheze/SpriteSheet/pull/13 [20:07:18] miraheze/SpriteSheet - pppery the build passed. [20:07:29] [02SpriteSheet] 07Universal-Omega closed pull request 03#13: Updated README - 13https://github.com/miraheze/SpriteSheet/pull/13 [20:07:30] [02miraheze/SpriteSheet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/SpriteSheet/compare/717a632a56e3...28d3911ae383 [20:07:32] [02miraheze/SpriteSheet] 07pppery 0328d3911 - Updated README (#13) [20:08:37] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/d73a327062ac...10652abfa62f [20:08:40] [02miraheze/puppet] 07Universal-Omega 0310652ab - Revert "depool all servers for 1.39 upgrade from deploy" (#3130) [20:08:42] [02puppet] 07Universal-Omega closed pull request 03#3130: Revert "depool all servers for 1.39 upgrade from deploy" - 13https://github.com/miraheze/puppet/pull/3130 [20:08:43] [02miraheze/puppet] 07Universal-Omega deleted branch 03revert-3129-patch-50 [20:08:44] [02puppet] 07Universal-Omega deleted branch 03revert-3129-patch-50 - 13https://github.com/miraheze/puppet [20:08:55] [02SpriteSheet] 07pppery opened pull request 03#14: Updated README.wiki from current page on MediaWiki.org - 13https://github.com/miraheze/SpriteSheet/pull/14 [20:10:34] miraheze/SpriteSheet - Universal-Omega the build passed. [20:11:57] [02SpriteSheet] 07Universal-Omega commented on pull request 03#14: Updated README.wiki from current page on MediaWiki.org - 13https://github.com/miraheze/SpriteSheet/pull/14#issuecomment-1399324325 [20:12:05] miraheze/SpriteSheet - pppery the build passed. [20:15:49] [02SpriteSheet] 07pppery synchronize pull request 03#14: Updated README.wiki from current page on MediaWiki.org - 13https://github.com/miraheze/SpriteSheet/pull/14 [20:16:05] [02SpriteSheet] 07pppery edited pull request 03#14: Delete README.wiki - 13https://github.com/miraheze/SpriteSheet/pull/14 [20:16:25] [02SpriteSheet] 07pppery edited pull request 03#14: Delete README.wiki - 13https://github.com/miraheze/SpriteSheet/pull/14 [20:16:41] [02SpriteSheet] 07pppery commented on pull request 03#14: Delete README.wiki - 13https://github.com/miraheze/SpriteSheet/pull/14#issuecomment-1399325085 [20:18:58] miraheze/SpriteSheet - pppery the build passed. [20:20:07] [02SpriteSheet] 07Universal-Omega commented on pull request 03#14: Delete README.wiki - 13https://github.com/miraheze/SpriteSheet/pull/14#issuecomment-1399325608 [20:20:16] [02miraheze/SpriteSheet] 07Universal-Omega pushed 031 commit to 03master [+0/-1/±0] 13https://github.com/miraheze/SpriteSheet/compare/28d3911ae383...0ed21cc03397 [20:20:17] [02miraheze/SpriteSheet] 07pppery 030ed21cc - Delete README.wiki (#14) [20:20:20] [02SpriteSheet] 07Universal-Omega closed pull request 03#14: Delete README.wiki - 13https://github.com/miraheze/SpriteSheet/pull/14 [20:23:15] miraheze/SpriteSheet - Universal-Omega the build passed. [20:23:29] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.61, 2.89, 2.20 [20:25:26] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.42, 2.66, 2.20 [20:27:28] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/a19109fae379 [20:27:29] [02miraheze/mw-config] 07Universal-Omega 03a19109f - Remove configs not needed with 1.39 [20:27:32] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-4 - 13https://github.com/miraheze/mw-config [20:27:35] [02mw-config] 07Universal-Omega opened pull request 03#5076: Remove configs not needed with 1.39 - 13https://github.com/miraheze/mw-config/pull/5076 [20:28:30] miraheze/mw-config - Universal-Omega the build passed. [20:29:33] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/a19109fae379...b4172814591c [20:29:34] [02miraheze/mw-config] 07Universal-Omega 03b417281 - Update GlobalSettings.php [20:29:36] [02mw-config] 07Universal-Omega synchronize pull request 03#5076: Remove configs not needed with 1.39 - 13https://github.com/miraheze/mw-config/pull/5076 [20:30:39] miraheze/mw-config - Universal-Omega the build passed. [20:34:59] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-5 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/ede4421cac92 [20:35:01] [02miraheze/mw-config] 07Universal-Omega 03ede4421 - Use ssl parameter instead of deprecated DBO_SSL [20:35:02] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-5 - 13https://github.com/miraheze/mw-config [20:35:04] [02mw-config] 07Universal-Omega opened pull request 03#5077: Use ssl parameter instead of deprecated DBO_SSL - 13https://github.com/miraheze/mw-config/pull/5077 [20:35:34] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-5 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/ede4421cac92...572e55d9da3c [20:35:35] [02miraheze/mw-config] 07Universal-Omega 03572e55d - Update Database.php [20:35:36] [02mw-config] 07Universal-Omega synchronize pull request 03#5077: Use ssl parameter instead of deprecated DBO_SSL - 13https://github.com/miraheze/mw-config/pull/5077 [20:36:03] miraheze/mw-config - Universal-Omega the build passed. [20:36:07] [02mw-config] 07Universal-Omega edited pull request 03#5077: Database: use ssl parameter instead of deprecated DBO_SSL - 13https://github.com/miraheze/mw-config/pull/5077 [20:36:38] miraheze/mw-config - Universal-Omega the build passed. [20:37:50] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.65, 3.55, 2.75 [20:38:27] [02mw-config] 07Universal-Omega closed pull request 03#5076: Remove configs not needed with 1.39 - 13https://github.com/miraheze/mw-config/pull/5076 [20:38:30] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-4 [20:38:33] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/mw-config/compare/d94c9f4e8130...a5241cb18294 [20:38:36] [02miraheze/mw-config] 07Universal-Omega 03a5241cb - Remove configs not needed with 1.39 (#5076) [20:38:37] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-4 - 13https://github.com/miraheze/mw-config [20:39:31] miraheze/mw-config - Universal-Omega the build passed. [20:39:33] !log [universalomega@mwtask141] starting deploy of {'pull': 'config', 'config': True} to mwtask141 [20:39:34] !log [universalomega@mwtask141] finished deploy of {'pull': 'config', 'config': True} to mwtask141 - SUCCESS in 1s [20:39:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:39:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:39:48] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.63, 4.36, 3.15 [20:41:47] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.38, 3.88, 3.11 [20:41:59] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.41, 9.39, 7.87 [20:43:27] [02puppet] 07Universal-Omega closed pull request 03#3117: Re-enable manageInactiveWikis.php cron - 13https://github.com/miraheze/puppet/pull/3117 [20:43:29] [02miraheze/puppet] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [20:43:31] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/10652abfa62f...87fbc3ff4e8e [20:43:32] [02miraheze/puppet] 07Universal-Omega 0387fbc3f - Re-enable manageInactiveWikis.php cron (#3117) [20:43:35] [02puppet] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/puppet [20:43:45] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.55, 3.04, 2.89 [20:43:59] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.83, 8.28, 7.65 [20:57:04] !log [@test131] starting deploy of {'config': True} to all [20:57:05] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [20:57:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:57:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:09:29] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.57, 3.24, 2.74 [21:11:29] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.94, 2.75, 2.62 [21:17:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.12, 10.34, 9.22 [21:19:40] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.73, 9.17, 8.94 [21:46:00] [02mw-config] 07Universal-Omega edited pull request 03#5070: Update sitenotice - 13https://github.com/miraheze/mw-config/pull/5070 [21:48:34] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/610a9fa85a36 [21:48:35] [02miraheze/mw-config] 07Universal-Omega 03610a9fa - Post-upgrade site notice [21:48:36] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-4 - 13https://github.com/miraheze/mw-config [21:48:38] [02mw-config] 07Universal-Omega opened pull request 03#5078: Post-upgrade site notice - 13https://github.com/miraheze/mw-config/pull/5078 [21:49:44] miraheze/mw-config - Universal-Omega the build passed. [21:56:53] [02mw-config] 07AgentIsai synchronize pull request 03#5070: Update sitenotice - 13https://github.com/miraheze/mw-config/pull/5070 [21:57:56] miraheze/mw-config - AgentIsai the build passed. [22:00:12] !log enable and run puppet on mwtask141 [22:00:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:00:55] PROBLEM - mwtask141 Puppet on mwtask141 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[ipinfo_composer],Exec[translate_composer] [22:02:54] RECOVERY - mwtask141 Puppet on mwtask141 is OK: OK: Puppet is currently enabled, last run 21 seconds ago with 0 failures [22:03:05] RECOVERY - mwtask141 JobRunner Service on mwtask141 is OK: PROCS OK: 1 process with args 'redisJobRunnerService' [22:06:24] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.43, 3.85, 2.85 [22:06:32] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 8191 bytes in 0.351 second response time [22:08:59] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 8191 bytes in 0.387 second response time [22:10:21] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.86, 3.90, 3.09 [22:11:25] PROBLEM - mw131 MediaWiki Rendering on mw131 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 8191 bytes in 0.190 second response time [22:12:18] PROBLEM - mw132 MediaWiki Rendering on mw132 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 8191 bytes in 0.191 second response time [22:12:19] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.48, 3.38, 2.99 [22:15:05] PROBLEM - mw141 MediaWiki Rendering on mw141 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 8191 bytes in 0.206 second response time [22:17:38] PROBLEM - mw142 MediaWiki Rendering on mw142 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 8191 bytes in 0.206 second response time [22:18:32] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.443 second response time [22:18:36] PROBLEM - matomo131 Current Load on matomo131 is WARNING: WARNING - load average: 3.84, 3.49, 2.95 [22:20:59] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.363 second response time [22:21:20] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.02, 3.48, 2.48 [22:22:27] PROBLEM - matomo131 Current Load on matomo131 is CRITICAL: CRITICAL - load average: 4.07, 3.73, 3.16 [22:23:20] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.45, 2.73, 2.33 [22:23:25] RECOVERY - mw131 MediaWiki Rendering on mw131 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.207 second response time [22:24:18] RECOVERY - mw132 MediaWiki Rendering on mw132 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.211 second response time [22:24:22] PROBLEM - matomo131 Current Load on matomo131 is WARNING: WARNING - load average: 3.91, 3.82, 3.26 [22:24:58] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.49, 3.15, 2.43 [22:26:58] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.54, 2.88, 2.42 [22:27:05] RECOVERY - mw141 MediaWiki Rendering on mw141 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.275 second response time [22:30:08] RECOVERY - matomo131 Current Load on matomo131 is OK: OK - load average: 2.44, 3.16, 3.16 [22:31:38] RECOVERY - mw142 MediaWiki Rendering on mw142 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.238 second response time [22:33:05] !log [universalomega@mwtask141] finished deploy of {'world': True, 'l10n': True, 'extensionlist': True, 'force': True, 'servers': 'mw121,mw122,mw131,mw132,mw141,mw142', 'ignoretime': True} to mw121,mw122,mw131,mw132,mw141,mw142 - SUCCESS in 1787s [22:33:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:34:55] !log [universalomega@mwtask141] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/universalomega/semanticmediawiki.json /srv/mediawiki/w/extensions/SemanticMediaWiki/maintenance/setupStore.php (START) [22:34:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:37:53] !log [universalomega@mwtask141] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/universalomega/semanticmediawiki.json /srv/mediawiki/w/extensions/SemanticMediaWiki/maintenance/setupStore.php (END - exit=0) [22:37:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:38:16] !log [universalomega@mwtask141] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/universalomega/semanticmediawiki.json /srv/mediawiki/w/extensions/SemanticMediaWiki/maintenance/setupStore.php (START) [22:38:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:38:50] PROBLEM - matomo131 Current Load on matomo131 is WARNING: WARNING - load average: 3.49, 3.34, 3.25 [22:41:17] !log [universalomega@mwtask141] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/universalomega/semanticmediawiki.json /srv/mediawiki/w/extensions/SemanticMediaWiki/maintenance/setupStore.php (END - exit=0) [22:41:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:41:25] !log [universalomega@mwtask141] starting deploy of {'files': '../mediawiki/w/extensions/SemanticMediaWiki/.smw.json'} to all [22:41:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:41:36] !log [universalomega@mwtask141] finished deploy of {'files': '../mediawiki/w/extensions/SemanticMediaWiki/.smw.json'} to all - SUCCESS in 11s [22:41:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:42:29] !log [universalomega@mwtask141] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/maintenance/archives/patch-user_autocreate_serial.sql (START) [22:42:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:44:35] RECOVERY - matomo131 Current Load on matomo131 is OK: OK - load average: 2.91, 3.27, 3.27 [22:45:30] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [22:46:28] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/runJobs.php --wiki=creaturawiki (START) [22:46:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:51:21] PROBLEM - matomo131 Current Load on matomo131 is WARNING: WARNING - load average: 3.21, 3.54, 3.42 [22:55:11] PROBLEM - matomo131 Current Load on matomo131 is CRITICAL: CRITICAL - load average: 4.16, 3.85, 3.56 [22:57:06] PROBLEM - matomo131 Current Load on matomo131 is WARNING: WARNING - load average: 3.60, 3.77, 3.57 [22:57:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.99, 3.48, 2.82 [22:59:01] PROBLEM - matomo131 Current Load on matomo131 is CRITICAL: CRITICAL - load average: 4.02, 3.89, 3.64 [22:59:49] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.15, 2.96, 2.70 [23:01:58] !log [universalomega@mwtask141] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.json /srv/mediawiki/w/maintenance/migrateLinksTable.php --force --table=templatelinks (START) [23:02:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:02:52] PROBLEM - matomo131 Current Load on matomo131 is WARNING: WARNING - load average: 3.35, 3.82, 3.68 [23:06:02] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.23, 9.92, 8.08 [23:07:57] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.51, 8.92, 7.93 [23:08:38] PROBLEM - matomo131 Current Load on matomo131 is CRITICAL: CRITICAL - load average: 4.05, 3.97, 3.80 [23:10:06] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/runJobs.php --wiki=creaturawiki (END - exit=65280) [23:10:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:11:33] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/runJobs.php --wiki=creaturawiki (START) [23:11:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:18:08] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/b46e89580d97 [23:18:11] [02miraheze/mw-config] 07paladox 03b46e895 - Update wmincProjects to support MW 1.39 [23:18:12] [02mw-config] 07paladox created branch 03paladox-patch-2 - 13https://github.com/miraheze/mw-config [23:18:15] [02mw-config] 07paladox opened pull request 03#5079: Update wmincProjects to support MW 1.39 - 13https://github.com/miraheze/mw-config/pull/5079 [23:19:11] miraheze/mw-config - paladox the build passed. [23:19:38] [02mw-config] 07paladox closed pull request 03#5079: Update wmincProjects to support MW 1.39 - 13https://github.com/miraheze/mw-config/pull/5079 [23:19:41] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-2 [23:19:43] [02mw-config] 07paladox deleted branch 03paladox-patch-2 - 13https://github.com/miraheze/mw-config [23:19:45] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/a5241cb18294...0217f3118bd0 [23:19:47] [02miraheze/mw-config] 07paladox 030217f31 - Update wmincProjects to support MW 1.39 (#5079) [23:20:03] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [23:20:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:20:10] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 7s [23:20:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:20:18] !log [paladox@mwtask141] starting deploy of {'config': True} to all [23:20:20] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/runJobs.php --wiki=creaturawiki (END - exit=2) [23:20:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:20:23] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 5s [23:20:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:20:24] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 8.57, 4.81, 3.36 [23:20:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:20:38] !log [universalomega@mwtask141] starting deploy of {'l10n': True} to all [23:20:40] miraheze/mw-config - paladox the build passed. [23:20:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:20:51] !log [paladox@test131] starting deploy of {'pull': 'config', 'config': True} to test131 [23:20:53] !log [paladox@test131] finished deploy of {'pull': 'config', 'config': True} to test131 - SUCCESS in 0s [23:20:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:20:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:20:58] !log [paladox@test131] starting deploy of {'config': True} to test131 [23:20:59] !log [paladox@test131] finished deploy of {'config': True} to test131 - SUCCESS in 0s [23:21:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:21:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:22:22] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.33, 3.79, 3.17 [23:24:20] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.35, 3.34, 3.08 [23:24:21] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_39 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/90ee686b89fb...2e8ae6ca11bb [23:24:24] [02miraheze/mediawiki] 07paladox 032e8ae6c - Update Citizen [23:25:52] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1532645524 secs [23:27:16] !log [paladox@mwtask141] starting deploy of {'world': True, 'l10n': True} to all [23:27:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:27:29] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/manageJobs.php --wiki=creaturawiki --type refreshLinks --delete (END - exit=256) [23:27:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:27:39] !log [paladox@test131] starting deploy of {'world': True, 'l10n': True} to test131 [23:27:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:28:09] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/manageJobs.php --wiki=creaturawiki --type refreshLinks --action=delete (END - exit=0) [23:28:09] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.03, 3.90, 3.03 [23:28:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:29:52] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.05151161551 secs [23:30:07] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.96, 3.18, 2.87 [23:31:30] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-4 [23:31:32] [02mw-config] 07Universal-Omega closed pull request 03#5078: Post-upgrade site notice - 13https://github.com/miraheze/mw-config/pull/5078 [23:31:33] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/0217f3118bd0...a168e3eb8baf [23:31:34] [02miraheze/mw-config] 07Universal-Omega 03a168e3e - Post-upgrade site notice (#5078) [23:31:35] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-4 - 13https://github.com/miraheze/mw-config [23:32:28] miraheze/mw-config - Universal-Omega the build passed. [23:32:34] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.95, 3.37, 2.73 [23:32:40] !log [universalomega@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [23:32:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:32:47] !log [universalomega@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 7s [23:32:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:34:13] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.83, 3.82, 3.39 [23:34:48] PROBLEM - mw122 NTP time on mw122 is WARNING: NTP WARNING: Offset 0.1059366763 secs [23:36:12] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.75, 3.91, 3.46 [23:38:10] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.99, 3.56, 3.39 [23:38:36] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.18, 3.25, 2.91 [23:40:57] !log [paladox@test131] finished deploy of {'world': True, 'l10n': True} to test131 - SUCCESS in 797s [23:41:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:42:07] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.86, 2.99, 3.22 [23:43:49] PROBLEM - matomo131 Current Load on matomo131 is WARNING: WARNING - load average: 3.68, 3.60, 3.93 [23:44:34] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.68, 3.97, 3.32 [23:45:28] [02miraheze/mediawiki] 07Universal-Omega pushed 032 commits to 03REL1_39 [+0/-0/±2] 13https://github.com/miraheze/mediawiki/compare/2e8ae6ca11bb...3196bb02c58d [23:45:31] [02miraheze/mediawiki] 07Universal-Omega 03c464e56 - Update PortableInfobox [23:45:32] [02miraheze/mediawiki] 07Universal-Omega 033196bb0 - Update TemplateStylesExtender [23:47:38] !log [universalomega@mwtask141] starting deploy of {'world': True} to all [23:47:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:49:23] RECOVERY - mw122 NTP time on mw122 is OK: NTP OK: Offset 0.07498538494 secs [23:49:39] !log [universalomega@mwtask141] finished deploy of {'l10n': True} to all - SUCCESS in 1740s [23:49:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:49:50] PROBLEM - matomo131 Current Load on matomo131 is CRITICAL: CRITICAL - load average: 4.07, 3.91, 3.96 [23:53:49] PROBLEM - matomo131 Current Load on matomo131 is WARNING: WARNING - load average: 3.93, 3.95, 3.98 [23:54:34] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.81, 3.95, 3.82 [23:57:43] !log [@test131] starting deploy of {'config': True} to all [23:57:44] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [23:57:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:57:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:59:50] PROBLEM - matomo131 Current Load on matomo131 is CRITICAL: CRITICAL - load average: 4.41, 3.84, 3.92