[00:01:18] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 30.45, 22.07, 19.32 [00:02:48] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 12.11, 8.40, 7.43 [00:03:02] RECOVERY - cp12 Disk Space on cp12 is OK: DISK OK - free space: / 8335 MB (21% inode=96%); [00:03:16] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.79, 22.76, 19.94 [00:05:17] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 25.58, 24.95, 21.14 [00:05:29] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 11.44, 9.60, 8.02 [00:07:15] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 19.06, 23.15, 20.97 [00:09:30] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.41, 7.79, 7.65 [00:10:47] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.37, 7.39, 7.63 [00:12:47] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.20, 7.60, 7.67 [00:14:47] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.78, 7.45, 7.62 [00:15:12] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.03, 19.44, 20.11 [00:19:28] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.94, 6.07, 6.77 [00:22:47] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.48, 5.90, 6.72 [00:35:49] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.94, 6.89, 6.61 [00:37:48] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.15, 6.57, 6.52 [00:57:23] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.65, 18.21, 17.42 [00:59:21] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 14.48, 16.96, 17.07 [02:59:40] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.80, 6.32, 5.59 [03:03:35] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.92, 7.38, 6.13 [03:05:33] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.55, 7.13, 6.20 [03:06:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.62, 7.63, 6.31 [03:07:31] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.71, 6.78, 6.19 [03:08:10] !log [rhinos@jobrunner4] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=smutstonewiki /home/rhinos/v2smutstonefandomcom-20210720-history.xml --username-prefix wikia:v2smutstone (END - exit=0) [03:08:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:08:20] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.51, 6.79, 6.16 [03:12:12] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.08, 7.16, 6.46 [03:14:04] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.91, 19.01, 16.35 [03:14:26] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.28, 6.82, 6.42 [03:16:06] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.89, 6.67, 6.43 [03:16:24] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.08, 6.64, 6.41 [03:18:04] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 15.69, 17.22, 16.25 [03:50:11] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.60, 6.71, 5.98 [03:51:04] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.93, 6.55, 6.13 [03:52:09] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.68, 6.50, 5.98 [03:53:02] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.15, 6.43, 6.14 [06:14:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.55, 6.93, 6.14 [06:16:17] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.20, 6.29, 6.00 [06:22:10] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.82, 6.57, 6.09 [06:24:05] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.68, 7.05, 6.43 [06:26:03] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.51, 6.51, 6.30 [06:26:08] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.19, 6.37, 6.17 [07:21:47] PROBLEM - mem2 APT on mem2 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:22:08] PROBLEM - mon2 APT on mon2 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:22:41] PROBLEM - cp12 APT on cp12 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [07:23:53] PROBLEM - puppet3 APT on puppet3 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:24:53] PROBLEM - db13 APT on db13 is CRITICAL: APT CRITICAL: 55 packages available for upgrade (5 critical updates). [07:26:46] PROBLEM - gluster4 APT on gluster4 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:27:43] PROBLEM - ns1 APT on ns1 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [07:28:10] PROBLEM - cloud5 APT on cloud5 is CRITICAL: APT CRITICAL: 72 packages available for upgrade (4 critical updates). [07:29:20] PROBLEM - mail2 APT on mail2 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:30:54] PROBLEM - cp13 APT on cp13 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [07:32:50] PROBLEM - graylog2 APT on graylog2 is CRITICAL: APT CRITICAL: 8 packages available for upgrade (5 critical updates). [07:33:46] PROBLEM - phab2 APT on phab2 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:34:23] PROBLEM - mw9 APT on mw9 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:35:15] PROBLEM - jobrunner4 APT on jobrunner4 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:35:24] PROBLEM - test3 APT on test3 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:35:28] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 17.06, 9.96, 6.94 [07:36:08] PROBLEM - mw11 APT on mw11 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:36:19] PROBLEM - mw10 APT on mw10 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:36:33] PROBLEM - jobrunner3 APT on jobrunner3 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:36:39] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.59, 6.81, 5.86 [07:37:00] PROBLEM - mw8 APT on mw8 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:37:05] PROBLEM - mem1 APT on mem1 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:38:36] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.45, 6.32, 5.79 [07:39:12] PROBLEM - ldap2 APT on ldap2 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:39:23] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 4.76, 7.16, 6.47 [07:40:14] PROBLEM - services4 APT on services4 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:43:19] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.62, 6.13, 6.23 [07:44:03] PROBLEM - services3 APT on services3 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:44:14] PROBLEM - cp3 APT on cp3 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [07:44:25] PROBLEM - cloud4 APT on cloud4 is CRITICAL: APT CRITICAL: 72 packages available for upgrade (4 critical updates). [07:50:32] PROBLEM - cloud3 APT on cloud3 is CRITICAL: APT CRITICAL: 108 packages available for upgrade (4 critical updates). [07:51:13] PROBLEM - gluster3 APT on gluster3 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:51:14] PROBLEM - test4 APT on test4 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:51:44] PROBLEM - ns2 APT on ns2 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [07:52:16] PROBLEM - db12 APT on db12 is CRITICAL: APT CRITICAL: 76 packages available for upgrade (5 critical updates). [07:52:49] PROBLEM - db11 APT on db11 is CRITICAL: APT CRITICAL: 67 packages available for upgrade (5 critical updates). [08:32:56] PROBLEM - mw8 APT on mw8 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [08:34:57] PROBLEM - mw8 APT on mw8 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [08:48:21] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/J43Nx [08:48:23] [02miraheze/dns] 07Reception123 03e8362c4 - add journeytheword.wiki zone [08:50:24] PROBLEM - wiki.caker18.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.caker18.com' expires in 15 day(s) (Tue 10 Aug 2021 08:44:29 GMT +0000). [08:54:58] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/J4sLj [08:54:59] [02miraheze/ssl] 07MirahezeSSLBot 03ba2c825 - Bot: Update SSL cert for wiki.caker18.com [09:05:08] !log MariaDB [umbranetwiki]> DROP TABLE cargo__Events; [09:05:11] !log MariaDB [umbranetwiki]> DROP TABLE cargo__Missions; [09:05:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:05:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:15:09] [02mediawiki] 07Reception123 commented on pull request 03#2394: T7617: Install ArticlePlaceholder - 13https://git.io/J4sHw [09:20:14] [02miraheze/ssl] 07Reception123 pushed 032 commits to 03master [+2/-0/±3] 13https://git.io/J4sh3 [09:20:16] [02miraheze/ssl] 07Reception123 034a8a03c - add www.journeytheword.wiki cert [09:20:17] [02miraheze/ssl] 07Reception123 03d9d8785 - add pj-masks-info.cf cert [09:26:16] !log renamed antridotowiki to anthridotewiki [09:26:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:28:06] !log renamed lgbtqiawiki to lgbtqawiki [09:28:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:31:32] RECOVERY - wiki.caker18.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.caker18.com' will expire on Sat 23 Oct 2021 07:54:49 GMT +0000. [10:22:16] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.09, 6.74, 5.51 [10:24:17] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.04, 6.08, 5.42 [10:33:38] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.21, 7.92, 6.38 [10:34:16] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.54, 7.89, 6.50 [10:35:33] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.99, 7.55, 6.43 [10:36:16] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.75, 7.87, 6.68 [10:37:28] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.17, 6.73, 6.26 [10:42:16] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.09, 6.45, 6.49 [10:54:08] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/J4nyW [10:54:09] [02miraheze/ssl] 07Reception123 0333956cc - add wiki.apico.buzz cert [11:01:51] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.20, 7.10, 6.59 [11:03:49] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 12.02, 8.38, 7.09 [11:05:46] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.47, 7.65, 6.99 [11:09:41] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.12, 6.64, 6.71 [11:21:38] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.97, 7.42, 6.87 [11:23:35] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.41, 6.94, 6.76 [11:27:29] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.46, 7.21, 6.88 [11:29:24] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.78, 6.37, 6.62 [11:35:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.49, 7.07, 6.81 [11:37:16] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.69, 6.13, 6.50 [12:15:59] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.75, 6.85, 6.42 [12:17:56] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.28, 7.05, 6.53 [12:21:48] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.52, 6.26, 6.37 [13:05:25] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.33, 6.94, 6.16 [13:07:20] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.33, 6.22, 5.98 [13:50:08] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.62, 6.97, 6.20 [13:52:05] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.44, 6.70, 6.18 [14:05:32] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.35, 6.90, 6.25 [14:07:27] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.27, 6.58, 6.21 [14:14:27] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.93, 6.43, 6.17 [14:16:22] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.67, 6.62, 6.28 [14:20:16] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.38, 7.34, 6.64 [14:22:12] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.91, 6.59, 6.45 [14:30:01] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 11.37, 8.08, 6.95 [14:31:57] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.22, 7.16, 6.74 [14:33:55] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 14.25, 9.20, 7.50 [14:36:29] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.56, 7.23, 6.49 [14:38:25] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.12, 6.78, 6.41 [14:39:44] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.01, 7.85, 7.52 [14:51:18] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.14, 7.66, 7.44 [14:53:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.46, 7.38, 7.35 [15:02:14] PROBLEM - mw11 APT on mw11 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [15:03:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.78, 6.81, 7.00 [15:04:16] PROBLEM - mw11 APT on mw11 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [15:06:37] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.67, 6.98, 6.62 [15:07:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.31, 7.13, 7.16 [15:08:33] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.97, 6.59, 6.52 [15:09:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.03, 7.28, 7.19 [15:12:28] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.22, 7.48, 6.92 [15:13:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.51, 7.13, 7.18 [15:19:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.43, 7.81, 7.46 [15:20:21] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.68, 8.29, 7.42 [15:21:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.74, 7.35, 7.33 [15:26:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.98, 7.29, 7.32 [15:27:14] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.18, 5.78, 6.67 [15:34:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.71, 7.45, 7.24 [15:35:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.46, 8.94, 7.60 [15:38:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.06, 7.18, 7.22 [15:39:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.43, 7.95, 7.48 [15:44:20] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.10, 6.14, 6.71 [15:49:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 11.60, 8.32, 7.52 [15:50:21] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.38, 8.18, 7.34 [15:52:21] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.81, 7.98, 7.37 [15:54:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.69, 8.59, 7.64 [16:01:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.08, 7.50, 7.71 [16:02:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.32, 7.71, 7.80 [16:04:21] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.69, 8.66, 8.12 [16:05:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 11.16, 9.52, 8.46 [16:06:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.80, 7.88, 7.91 [16:08:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.24, 8.14, 7.99 [16:10:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.28, 7.76, 7.87 [16:20:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 12.07, 8.47, 7.88 [16:26:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.11, 7.36, 7.66 [16:31:15] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.92, 7.06, 7.93 [16:33:15] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.47, 7.52, 7.98 [16:34:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 12.66, 8.70, 7.89 [16:40:22] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.66, 7.34, 7.62 [16:45:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.64, 7.30, 7.94 [16:48:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.11, 7.30, 7.41 [16:49:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 11.32, 8.34, 8.13 [16:52:35] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 9.06, 7.11, 6.12 [16:54:35] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.80, 6.56, 6.04 [16:59:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.93, 7.60, 7.96 [17:02:21] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.48, 7.09, 7.65 [17:03:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.61, 8.16, 8.08 [17:04:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.45, 8.36, 8.05 [17:06:21] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.21, 7.82, 7.89 [17:10:22] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.01, 7.66, 7.79 [17:11:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.95, 7.54, 7.93 [17:14:21] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.30, 7.85, 7.88 [17:20:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.78, 7.68, 7.70 [17:21:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.01, 7.85, 7.82 [17:22:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.43, 7.73, 7.73 [17:23:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.82, 7.40, 7.65 [17:33:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.14, 6.80, 7.07 [17:34:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 11.57, 8.46, 7.81 [17:40:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.89, 7.72, 7.65 [17:42:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.12, 7.94, 7.75 [17:44:21] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.44, 7.51, 7.62 [17:47:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.44, 7.81, 7.77 [17:49:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.11, 8.53, 8.03 [17:50:21] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.76, 8.34, 7.83 [17:54:21] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.55, 7.70, 7.66 [17:57:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.49, 7.61, 7.87 [18:02:20] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.34, 6.05, 6.80 [18:03:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.65, 7.98, 7.89 [18:06:21] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.23, 7.84, 7.42 [18:08:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.52, 7.28, 7.26 [18:13:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.10, 7.23, 7.87 [18:14:20] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.55, 6.03, 6.71 [18:22:21] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.52, 6.65, 6.65 [18:24:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.90, 7.33, 6.88 [18:25:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.13, 7.42, 7.35 [18:26:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.36, 7.31, 6.94 [18:27:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.02, 7.11, 7.24 [18:28:20] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.88, 6.77, 6.78 [18:32:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.34, 7.68, 7.17 [18:35:14] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.15, 6.24, 6.73 [18:36:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.12, 7.85, 7.40 [18:41:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.39, 6.89, 6.78 [18:44:22] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 11.16, 8.51, 7.73 [18:48:46] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.45, 6.69, 5.95 [18:49:12] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.87, 7.27, 6.18 [18:51:09] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.87, 6.43, 6.01 [18:52:44] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.81, 6.27, 5.95 [18:56:41] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 5.93, 6.90, 6.37 [18:58:40] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.58, 6.54, 6.30 [19:10:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.70, 7.27, 7.98 [19:13:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.31, 7.08, 7.99 [19:15:59] PROBLEM - wiki.nevillepedia.eu - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.nevillepedia.eu' expires in 15 day(s) (Tue 10 Aug 2021 19:08:35 GMT +0000). [19:19:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.10, 6.52, 7.35 [19:19:21] [02miraheze/RemovePII] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J4zC9 [19:19:23] [02miraheze/RemovePII] 07Universal-Omega 033b3ecbd - Update SpecialRemovePII.php [19:19:24] [02RemovePII] 07Universal-Omega synchronize pull request 03#3: Create SpecialRemovePII (T7286: phase two and phase three) - 13https://git.io/Js0PN [19:19:26] [02RemovePII] 07Universal-Omega synchronize pull request 03#3: Create SpecialRemovePII (T7286: phase two and phase three) - 13https://git.io/Js0PN [19:20:19] miraheze/RemovePII - Universal-Omega the build passed. [19:20:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.51, 7.11, 7.37 [19:21:10] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/J4zlZ [19:21:11] [02miraheze/ssl] 07MirahezeSSLBot 03437bed1 - Bot: Update SSL cert for wiki.nevillepedia.eu [19:21:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.72, 7.36, 7.59 [19:21:31] miraheze/RemovePII - Universal-Omega the build passed. [19:23:14] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 13.29, 9.24, 8.22 [19:25:21] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 10.32, 7.91, 6.46 [19:27:18] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.18, 7.46, 6.46 [19:28:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.17, 7.75, 7.71 [19:29:15] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.29, 6.68, 6.30 [19:32:02] [02RemovePII] 07supertassu reviewed pull request 03#3 commit - 13https://git.io/J4z2Z [19:35:40] [02mediawiki] 07R4356th synchronize pull request 03#2394: T7617: Install ArticlePlaceholder - 13https://git.io/JCgIo [19:36:38] [02mediawiki] 07R4356th synchronize pull request 03#2394: T7617: Install ArticlePlaceholder - 13https://git.io/JCgIo [19:37:01] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.59, 21.31, 18.63 [19:37:02] [02mediawiki] 07R4356th commented on pull request 03#2394: T7617: Install ArticlePlaceholder - 13https://git.io/J4zrA [19:37:04] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.97, 7.29, 6.59 [19:38:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.05, 7.46, 7.38 [19:39:02] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.71, 19.92, 18.47 [19:39:02] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.92, 7.13, 6.62 [19:40:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.02, 7.09, 7.25 [19:41:00] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.67, 6.74, 6.53 [19:41:48] [02mediawiki] 07Universal-Omega closed pull request 03#2394: T7617: Install ArticlePlaceholder - 13https://git.io/JCgIo [19:41:50] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_36 [+1/-0/±1] 13https://git.io/J4zP5 [19:41:51] [02miraheze/mediawiki] 07R4356th 03f5917d8 - T7617: Install ArticlePlaceholder (#2394) [19:42:21] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.93, 7.91, 7.52 [19:44:54] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.99, 8.09, 7.15 [19:47:03] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 51.38.69.175/cpweb, 2001:41d0:801:2000::58af/cpweb [19:47:31] PROBLEM - www.permanentfuturelab.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.permanentfuturelab.wiki could not be found [19:47:31] PROBLEM - adadevelopersacademy.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for adadevelopersacademy.wiki could not be found [19:47:34] PROBLEM - db1.miraheze.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for db1.miraheze.wiki could not be found [19:47:38] PROBLEM - wiki.velaan.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.velaan.org could not be found [19:47:39] PROBLEM - sarovia.graalmilitary.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for sarovia.graalmilitary.com could not be found [19:47:40] PROBLEM - uncyclopediakr.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for uncyclopediakr.miraheze.org could not be found [19:47:42] PROBLEM - wiki.mikrodev.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.mikrodev.com could not be found [19:47:43] PROBLEM - wiki.sdbx.dev - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.sdbx.dev could not be found [19:47:45] PROBLEM - spiral.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for spiral.wiki could not be found [19:47:48] PROBLEM - bebaskanpengetahuan.id - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for bebaskanpengetahuan.id could not be found [19:47:49] PROBLEM - lama.madooa.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lama.madooa.org could not be found [19:47:49] PROBLEM - podpedia.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for podpedia.org could not be found [19:47:50] PROBLEM - cubashpedia.cf - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for cubashpedia.cf could not be found [19:47:58] PROBLEM - you.r-fit.cc - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for you.r-fit.cc could not be found [19:48:02] PROBLEM - wikipariksha.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wikipariksha.com could not be found [19:48:13] PROBLEM - www.petrawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.petrawiki.org could not be found [19:48:20] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for diamowiki.ga could not be found [19:48:22] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.stardewmodding.wiki could not be found [19:48:48] PROBLEM - thelonsdalebattalion.co.uk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for thelonsdalebattalion.co.uk could not be found [19:48:54] PROBLEM - trollpasta.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for trollpasta.com could not be found [19:48:55] PROBLEM - guia.esporo.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for guia.esporo.net could not be found [19:48:56] PROBLEM - infectowiki.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for infectowiki.com could not be found [19:48:57] PROBLEM - wiki.usagihime.ml - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.usagihime.ml could not be found [19:49:02] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [19:49:26] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 10.48, 8.68, 7.20 [19:51:29] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.69, 7.88, 7.08 [19:54:18] RECOVERY - db1.miraheze.wiki - reverse DNS on sslhost is OK: SSL OK - db1.miraheze.wiki reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:54:24] RECOVERY - wiki.velaan.org - reverse DNS on sslhost is OK: SSL OK - wiki.velaan.org reverse DNS resolves to cp13.miraheze.org - CNAME OK [19:54:27] RECOVERY - sarovia.graalmilitary.com - reverse DNS on sslhost is OK: SSL OK - sarovia.graalmilitary.com reverse DNS resolves to cp13.miraheze.org - CNAME OK [19:54:28] RECOVERY - uncyclopediakr.miraheze.org - reverse DNS on sslhost is OK: SSL OK - uncyclopediakr.miraheze.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:54:29] RECOVERY - www.permanentfuturelab.wiki - reverse DNS on sslhost is OK: SSL OK - www.permanentfuturelab.wiki reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:54:29] RECOVERY - adadevelopersacademy.wiki - reverse DNS on sslhost is OK: SSL OK - adadevelopersacademy.wiki reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:54:32] RECOVERY - wiki.mikrodev.com - reverse DNS on sslhost is OK: SSL OK - wiki.mikrodev.com reverse DNS resolves to cp13.miraheze.org - CNAME OK [19:54:34] RECOVERY - wiki.sdbx.dev - reverse DNS on sslhost is OK: SSL OK - wiki.sdbx.dev reverse DNS resolves to cp13.miraheze.org - CNAME OK [19:54:38] RECOVERY - spiral.wiki - reverse DNS on sslhost is OK: SSL OK - spiral.wiki reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:54:40] RECOVERY - you.r-fit.cc - reverse DNS on sslhost is OK: SSL OK - you.r-fit.cc reverse DNS resolves to cp13.miraheze.org - CNAME OK [19:54:45] RECOVERY - bebaskanpengetahuan.id - reverse DNS on sslhost is OK: SSL OK - bebaskanpengetahuan.id reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:54:46] RECOVERY - lama.madooa.org - reverse DNS on sslhost is OK: SSL OK - lama.madooa.org reverse DNS resolves to cp13.miraheze.org - CNAME OK [19:54:47] RECOVERY - podpedia.org - reverse DNS on sslhost is OK: SSL OK - podpedia.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:54:47] RECOVERY - cubashpedia.cf - reverse DNS on sslhost is OK: SSL OK - cubashpedia.cf reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:55:00] RECOVERY - wikipariksha.com - reverse DNS on sslhost is OK: SSL OK - wikipariksha.com reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:55:04] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [19:55:05] RECOVERY - www.stardewmodding.wiki - reverse DNS on sslhost is OK: SSL OK - www.stardewmodding.wiki reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:55:15] RECOVERY - www.petrawiki.org - reverse DNS on sslhost is OK: SSL OK - www.petrawiki.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:55:29] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.24, 6.55, 6.71 [19:55:37] RECOVERY - wiki.usagihime.ml - reverse DNS on sslhost is OK: SSL OK - wiki.usagihime.ml reverse DNS resolves to cp13.miraheze.org - CNAME OK [19:55:48] RECOVERY - thelonsdalebattalion.co.uk - reverse DNS on sslhost is OK: SSL OK - thelonsdalebattalion.co.uk reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:55:53] RECOVERY - trollpasta.com - reverse DNS on sslhost is OK: SSL OK - trollpasta.com reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:55:54] RECOVERY - guia.esporo.net - reverse DNS on sslhost is OK: SSL OK - guia.esporo.net reverse DNS resolves to cp13.miraheze.org - CNAME OK [19:55:55] RECOVERY - infectowiki.com - reverse DNS on sslhost is OK: SSL OK - infectowiki.com reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:56:37] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.32, 7.12, 7.72 [19:58:34] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.78, 7.54, 7.77 [20:02:15] PROBLEM - mw11 APT on mw11 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [20:02:28] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.50, 7.35, 7.69 [20:04:19] PROBLEM - mw11 APT on mw11 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [20:04:33] [02miraheze/RemovePII] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J4ge7 [20:04:35] [02miraheze/RemovePII] 07Universal-Omega 033ac9523 - Drop database param; use `waitForReplication();` [20:04:36] [02RemovePII] 07Universal-Omega created branch 03Universal-Omega-patch-2 - 13https://git.io/Jsfpj [20:04:38] [02RemovePII] 07Universal-Omega opened pull request 03#16: Drop database param; use `waitForReplication();` - 13https://git.io/J4gep [20:04:56] RECOVERY - wiki.nevillepedia.eu - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.nevillepedia.eu' will expire on Sat 23 Oct 2021 18:21:00 GMT +0000. [20:05:35] miraheze/RemovePII - Universal-Omega the build passed. [20:05:38] [02miraheze/RemovePII] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J4gv7 [20:05:40] [02miraheze/RemovePII] 07Universal-Omega 0353cb0a9 - Update SpecialRemovePII.php [20:05:41] [02RemovePII] 07Universal-Omega synchronize pull request 03#16: Drop database param; use `waitForReplication();` - 13https://git.io/J4gep [20:05:59] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.34, 22.98, 19.43 [20:06:29] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.26, 7.66, 7.72 [20:06:49] miraheze/RemovePII - Universal-Omega the build passed. [20:07:58] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 16.77, 20.84, 19.08 [20:08:26] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.51, 7.56, 7.67 [20:09:58] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.34, 19.17, 18.68 [20:10:22] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.33, 7.88, 7.77 [20:11:00] PROBLEM - jobrunner3 Puppet on jobrunner3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [20:12:15] PROBLEM - mw11 Puppet on mw11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [20:12:22] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.00, 7.33, 7.59 [20:12:51] PROBLEM - mw10 Puppet on mw10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [20:13:06] PROBLEM - mw8 Puppet on mw8 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [20:13:42] PROBLEM - mw9 Puppet on mw9 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [20:18:21] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.14, 6.76, 7.17 [20:20:21] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.75, 6.21, 6.92 [20:21:12] PROBLEM - wiki.ssij.se - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.ssij.se could not be found [20:22:46] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/J4gZ2 [20:22:47] [02miraheze/MirahezeMagic] 07Universal-Omega 031f2ac31 - Remove reference on $titleObj [20:23:41] miraheze/MirahezeMagic - Universal-Omega the build passed. [20:24:21] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.57, 6.10, 6.73 [20:27:59] RECOVERY - wiki.ssij.se - reverse DNS on sslhost is OK: SSL OK - wiki.ssij.se reverse DNS resolves to cp13.miraheze.org - CNAME OK [20:30:49] SRE have released a security disclosure regarding a recent incident affecting our mail system. See https://meta.miraheze.org/w/index.php?diff=188538&oldid=188531&rcid=656960 for more information. [20:30:51] [url] Difference between revisions of "Community noticeboard" - Miraheze Meta | meta.miraheze.org [20:34:36] [02RemovePII] 07supertassu reviewed pull request 03#16 commit - 13https://git.io/J4gRy [20:34:38] [02RemovePII] 07supertassu reviewed pull request 03#16 commit - 13https://git.io/J4gRS [20:34:40] [02RemovePII] 07supertassu reviewed pull request 03#16 commit - 13https://git.io/J4gR9 [20:34:45] RECOVERY - mw10 Puppet on mw10 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [20:35:04] RECOVERY - jobrunner3 Puppet on jobrunner3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [20:35:17] RECOVERY - mw8 Puppet on mw8 is OK: OK: Puppet is currently enabled, last run 17 seconds ago with 0 failures [20:35:38] RECOVERY - mw9 Puppet on mw9 is OK: OK: Puppet is currently enabled, last run 2 seconds ago with 0 failures [20:36:08] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.12, 8.09, 6.91 [20:36:19] RECOVERY - mw11 Puppet on mw11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [20:39:28] PROBLEM - biblestrength.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for biblestrength.net could not be found [20:39:42] PROBLEM - froggy.info - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for froggy.info could not be found [20:39:43] PROBLEM - phab.bots.miraheze.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for phab.bots.miraheze.wiki could not be found [20:39:44] PROBLEM - history.sdtef.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for history.sdtef.org could not be found [20:39:47] PROBLEM - wiki.fraterniterebellion.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.fraterniterebellion.com could not be found [20:39:52] PROBLEM - tensegritywiki.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for tensegritywiki.com could not be found [20:39:53] PROBLEM - phabdigests.bots.miraheze.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for phabdigests.bots.miraheze.wiki could not be found [20:39:54] PROBLEM - guia.esporo.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for guia.esporo.net could not be found [20:39:55] PROBLEM - thelonsdalebattalion.co.uk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for thelonsdalebattalion.co.uk could not be found [20:39:55] PROBLEM - trollpasta.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for trollpasta.com could not be found [20:39:56] PROBLEM - pubwiki.lab.co.pl - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for pubwiki.lab.co.pl could not be found [20:39:57] PROBLEM - wiki.slimeapedia.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.slimeapedia.org could not be found [20:39:58] PROBLEM - allthetropes.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for allthetropes.miraheze.org could not be found [20:40:07] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 5.27, 7.11, 6.84 [20:40:24] [02miraheze/RemovePII] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J4gzS [20:40:25] [02miraheze/RemovePII] 07Universal-Omega 03140ec9d - Update RemovePIIJob.php [20:40:27] [02RemovePII] 07Universal-Omega synchronize pull request 03#16: Drop database param; use `waitForReplication();` - 13https://git.io/J4gep [20:40:36] [02RemovePII] 07Universal-Omega reviewed pull request 03#16 commit - 13https://git.io/J4gzd [20:40:45] [02RemovePII] 07Universal-Omega reviewed pull request 03#16 commit - 13https://git.io/J4gzA [20:40:51] [02RemovePII] 07Universal-Omega reviewed pull request 03#16 commit - 13https://git.io/J4ggv [20:41:21] miraheze/RemovePII - Universal-Omega the build passed. [20:46:05] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.53, 6.36, 6.62 [20:46:20] RECOVERY - biblestrength.net - reverse DNS on sslhost is OK: SSL OK - biblestrength.net reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [20:46:22] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 12.20, 8.06, 6.90 [20:46:23] RECOVERY - froggy.info - reverse DNS on sslhost is OK: SSL OK - froggy.info reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [20:46:24] RECOVERY - phab.bots.miraheze.wiki - reverse DNS on sslhost is OK: SSL OK - phab.bots.miraheze.wiki reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [20:46:26] RECOVERY - history.sdtef.org - reverse DNS on sslhost is OK: SSL OK - history.sdtef.org reverse DNS resolves to cp13.miraheze.org - CNAME OK [20:46:35] RECOVERY - wiki.fraterniterebellion.com - reverse DNS on sslhost is OK: SSL OK - wiki.fraterniterebellion.com reverse DNS resolves to cp13.miraheze.org - CNAME OK [20:46:41] RECOVERY - tensegritywiki.com - reverse DNS on sslhost is OK: SSL OK - tensegritywiki.com reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [20:46:50] RECOVERY - pubwiki.lab.co.pl - reverse DNS on sslhost is OK: SSL OK - pubwiki.lab.co.pl reverse DNS resolves to cp13.miraheze.org - CNAME OK [20:46:52] RECOVERY - phabdigests.bots.miraheze.wiki - reverse DNS on sslhost is OK: SSL OK - phabdigests.bots.miraheze.wiki reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [20:46:54] RECOVERY - guia.esporo.net - reverse DNS on sslhost is OK: SSL OK - guia.esporo.net reverse DNS resolves to cp13.miraheze.org - CNAME OK [20:46:55] RECOVERY - trollpasta.com - reverse DNS on sslhost is OK: SSL OK - trollpasta.com reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [20:46:55] RECOVERY - allthetropes.miraheze.org - reverse DNS on sslhost is OK: SSL OK - allthetropes.miraheze.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [20:46:56] RECOVERY - thelonsdalebattalion.co.uk - reverse DNS on sslhost is OK: SSL OK - thelonsdalebattalion.co.uk reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [20:46:57] RECOVERY - wiki.slimeapedia.org - reverse DNS on sslhost is OK: SSL OK - wiki.slimeapedia.org reverse DNS resolves to cp13.miraheze.org - CNAME OK [20:48:21] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.11, 7.01, 6.65 [20:50:05] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 5.73, 7.12, 6.96 [20:50:24] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.60, 6.17, 6.38 [20:52:03] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.33, 6.58, 6.79 [21:00:04] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.63, 6.87, 6.69 [21:00:06] PROBLEM - inourownwords.online - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for inourownwords.online could not be found [21:00:07] PROBLEM - wiki.yumeka.icu - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.yumeka.icu could not be found [21:00:08] PROBLEM - olwest.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for olwest.org could not be found [21:00:30] PROBLEM - wiki.valkyrienskies.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.valkyrienskies.org could not be found [21:00:31] PROBLEM - wiki.counterculturelabs.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.counterculturelabs.org could not be found [21:00:32] PROBLEM - debilepedie.gq - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for debilepedie.gq could not be found [21:00:33] PROBLEM - www.wikimicrofinanza.it - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.wikimicrofinanza.it could not be found [21:00:35] PROBLEM - wiki.gesamtschule-nordkirchen.de - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.gesamtschule-nordkirchen.de could not be found [21:00:35] PROBLEM - guia.cineastas.pt - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for guia.cineastas.pt could not be found [21:00:36] PROBLEM - www.electrowiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.electrowiki.org could not be found [21:00:39] PROBLEM - www.highdeologywiki.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.highdeologywiki.com could not be found [21:00:40] PROBLEM - pa.gyaanipedia.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for pa.gyaanipedia.com could not be found [21:02:02] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.72, 6.41, 6.54 [21:06:54] RECOVERY - wiki.yumeka.icu - reverse DNS on sslhost is OK: SSL OK - wiki.yumeka.icu reverse DNS resolves to cp13.miraheze.org - CNAME OK [21:06:56] RECOVERY - inourownwords.online - reverse DNS on sslhost is OK: SSL OK - inourownwords.online reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [21:06:58] RECOVERY - olwest.org - reverse DNS on sslhost is OK: SSL OK - olwest.org reverse DNS resolves to cp13.miraheze.org - CNAME OK [21:07:18] RECOVERY - wiki.valkyrienskies.org - reverse DNS on sslhost is OK: SSL OK - wiki.valkyrienskies.org reverse DNS resolves to cp13.miraheze.org - CNAME OK [21:07:21] RECOVERY - debilepedie.gq - reverse DNS on sslhost is OK: SSL OK - debilepedie.gq reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [21:07:22] RECOVERY - www.wikimicrofinanza.it - reverse DNS on sslhost is OK: SSL OK - www.wikimicrofinanza.it reverse DNS resolves to cp13.miraheze.org - CNAME OK [21:07:22] RECOVERY - wiki.counterculturelabs.org - reverse DNS on sslhost is OK: SSL OK - wiki.counterculturelabs.org reverse DNS resolves to cp13.miraheze.org - CNAME OK [21:07:25] RECOVERY - wiki.gesamtschule-nordkirchen.de - reverse DNS on sslhost is OK: SSL OK - wiki.gesamtschule-nordkirchen.de reverse DNS resolves to cp13.miraheze.org - CNAME OK [21:07:26] RECOVERY - guia.cineastas.pt - reverse DNS on sslhost is OK: SSL OK - guia.cineastas.pt reverse DNS resolves to cp13.miraheze.org - CNAME OK [21:07:29] RECOVERY - www.electrowiki.org - reverse DNS on sslhost is OK: SSL OK - www.electrowiki.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [21:07:37] RECOVERY - www.highdeologywiki.com - reverse DNS on sslhost is OK: SSL OK - www.highdeologywiki.com reverse DNS resolves to cp13.miraheze.org - CNAME OK [21:07:37] RECOVERY - pa.gyaanipedia.com - reverse DNS on sslhost is OK: SSL OK - pa.gyaanipedia.com reverse DNS resolves to cp13.miraheze.org - CNAME OK [21:13:17] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.42, 7.29, 7.92 [21:15:15] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.58, 8.00, 8.09 [21:18:39] !log [rhinos@jobrunner4] sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=smutstonewiki /home/rhinos/smutfiles --comment-ext=desc --summary=Attribution: https://v2smutstone.fandom.com (END - exit=2) [21:18:40] [url] Smutstone Wiki | Fandom | v2smutstone.fandom.com [21:18:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:20:13] !log restart dump [21:20:23] !log correction: import not dump [21:21:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:21:45] Voidwalker: why MirahezeLogbot so laggy [21:21:57] .status mhmeta on [21:21:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:22:02] wouldn't know [21:22:08] RhinosF1: Success [21:22:15] Voidwalker: meta is laggy [21:22:22] as MirahezeBot also slow [21:22:35] yeah, probably capacity problem [21:22:51] new servers soon? [21:24:03] yeah, reception and I were discussing that, we'll go ahead and try without additional resources [21:26:52] PROBLEM - wiki.insideearth.info - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.insideearth.info could not be found [21:26:53] PROBLEM - bebaskanpengetahuan.id - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for bebaskanpengetahuan.id could not be found [21:26:58] PROBLEM - wlhmn.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wlhmn.tk could not be found [21:27:01] PROBLEM - tools1.miraheze.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for tools1.miraheze.wiki could not be found [21:27:56] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 12.41, 8.53, 7.19 [21:28:22] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.85, 7.58, 6.67 [21:30:00] !log hard stop/start php-fpm on mw11 as workers 100% [21:30:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:31:44] !log same on mw10 to stop it exploding [21:31:53] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 5.59, 7.91, 7.34 [21:32:22] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 3.64, 6.29, 6.41 [21:32:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:32:51] !log mw9 too [21:33:37] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 51.38.69.175/cpweb, 51.222.25.132/cpweb [21:33:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:33:45] RECOVERY - wlhmn.tk - reverse DNS on sslhost is OK: SSL OK - wlhmn.tk reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [21:33:55] RECOVERY - bebaskanpengetahuan.id - reverse DNS on sslhost is OK: SSL OK - bebaskanpengetahuan.id reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [21:33:56] RECOVERY - wiki.insideearth.info - reverse DNS on sslhost is OK: SSL OK - wiki.insideearth.info reverse DNS resolves to cp13.miraheze.org - CNAME OK [21:34:01] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 51.222.25.132/cpweb [21:34:04] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 31.35, 27.20, 21.19 [21:34:27] icinga-miraheze: stop crying [21:34:58] !log mw8 [21:35:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:35:31] i'm looking at grafana and it's not a pretty view [21:35:39] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [21:35:45] PROBLEM - history.sdtef.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for history.sdtef.org could not be found [21:35:47] PROBLEM - phab.bots.miraheze.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for phab.bots.miraheze.wiki could not be found [21:35:48] PROBLEM - wiki.responsibly.ai - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.responsibly.ai could not be found [21:35:56] no it's not [21:36:00] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [21:36:03] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 16.76, 23.17, 20.45 [21:36:12] !log try mw8 again, still 100% worker usage [21:36:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:37:35] !log mw10 [21:38:28] @Agent: Unfortunately as soon as one server goes green it ruins the rest [21:39:20] !log mw8 again, should I just make this loop [21:39:25] !log [rhinos@jobrunner4] sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=smutstonewiki /home/rhinos/smutfiles --comment-ext=desc --summary=Attribution: https://v2smutstone.fandom.com --overwrite (END - exit=0) [21:39:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:39:25] [url] Smutstone Wiki | Fandom | v2smutstone.fandom.com [21:39:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:40:35] RECOVERY - tools1.miraheze.wiki - reverse DNS on sslhost is OK: SSL OK - tools1.miraheze.wiki reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [21:41:57] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.14, 5.74, 6.61 [21:42:02] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 12.99, 18.15, 19.20 [21:42:28] RECOVERY - history.sdtef.org - reverse DNS on sslhost is OK: SSL OK - history.sdtef.org reverse DNS resolves to cp13.miraheze.org - CNAME OK [21:42:29] RECOVERY - phab.bots.miraheze.wiki - reverse DNS on sslhost is OK: SSL OK - phab.bots.miraheze.wiki reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [21:42:34] RECOVERY - wiki.responsibly.ai - reverse DNS on sslhost is OK: SSL OK - wiki.responsibly.ai reverse DNS resolves to cp13.miraheze.org - CNAME OK [21:45:14] [02RemovePII] 07Universal-Omega closed pull request 03#16: Drop database param; use `waitForReplication();` - 13https://git.io/J4gep [21:45:15] [02miraheze/RemovePII] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±2] 13https://git.io/J42T7 [21:45:17] [02miraheze/RemovePII] 07Universal-Omega 039a92b62 - Drop database param; use `waitForReplication();` [21:45:18] [02RemovePII] 07Universal-Omega synchronize pull request 03#3: Create SpecialRemovePII (T7286: phase two and phase three) - 13https://git.io/Js0PN [21:45:20] [02miraheze/RemovePII] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 [21:45:21] [02RemovePII] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 - 13https://git.io/Jsfpj [21:45:37] Universal_Omega: pls don't deploy [21:45:41] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/J42Tp [21:45:42] [02miraheze/mediawiki] 07Universal-Omega 03b500287 - Update Cosmos [21:46:17] miraheze/RemovePII - Universal-Omega the build passed. [21:46:28] poor servers, they're all overloaded. At least an upgrade in capacity is coming soon [21:47:04] ye [21:47:29] @Agent: They're being very grumpy [21:49:38] well, with their load averages, I'd be grumpy too [21:49:39] lol [21:49:54] php-fpm kept running out [21:49:58] it's calming [21:51:59] RECOVERY - test3 Puppet on test3 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [21:52:23] Universal_Omega: did you miss the don't deploy? [21:53:04] RhinosF1: Don't deploy what? [21:53:37] Universal_Omega: anything, we have a live incident [21:54:34] most of the servers calmed down and have a far better load average now, except maybe mw8 [21:54:42] mw8 was weirdest [21:54:46] It's calming [21:55:06] yeah, it's at 8 now, when I looked at it, it was at 10 [21:56:53] !log [rhinos@mw8] Disabling puppet for incident [21:57:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:57:33] scratch that, mw8 at 12 [21:57:51] !log disable puppet on mw* to avoid patch rolling forward [21:58:02] RhinosF1: Sorry, I did miss all the messages... Seen now. [21:58:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:58:07] Universal_Omega: does your patch require i18n rebuild or anything? [21:58:20] RhinosF1: No. [21:58:26] !log cycle php-fpm on mw8 [21:58:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:59:00] Universal_Omega: ack, I just want to be careful to not add load [21:59:15] more I look less I'm sure of source [21:59:19] PROBLEM - miraheze.gq - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for miraheze.gq could not be found [21:59:19] PROBLEM - wiki.fffbr.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.fffbr.org could not be found [21:59:21] PROBLEM - wiki.globasa.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.globasa.net could not be found [21:59:37] PROBLEM - mw9 Puppet on mw9 is WARNING: WARNING: Puppet is currently disabled, message: incident, last run 24 minutes ago with 0 failures [21:59:53] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 16.02, 10.20, 7.74 [22:00:12] PROBLEM - mw11 Puppet on mw11 is WARNING: WARNING: Puppet is currently disabled, message: incident, last run 26 minutes ago with 0 failures [22:00:43] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.39, 7.27, 6.52 [22:00:45] PROBLEM - mw10 Puppet on mw10 is WARNING: WARNING: Puppet is currently disabled, message: incident, last run 27 minutes ago with 0 failures [22:00:53] !log mw11 [22:01:09] PROBLEM - mw8 Puppet on mw8 is WARNING: WARNING: Puppet is currently disabled, message: incident, last run 25 minutes ago with 0 failures [22:01:33] Universal_Omega: can we revert? [22:01:53] just because I want to avoid new stuff until I know what madness is happening [22:03:51] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 4.63, 7.66, 7.32 [22:04:37] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.01, 6.89, 6.57 [22:05:11] RhinosF1: Puppet is running right now, so reverting is not necessary at this point I'd think, right? [22:05:19] Universal_Omega: It's disabled [22:05:25] oh. [22:06:08] RECOVERY - wiki.globasa.net - reverse DNS on sslhost is OK: SSL OK - wiki.globasa.net reverse DNS resolves to cp13.miraheze.org - CNAME OK [22:06:10] RECOVERY - miraheze.gq - reverse DNS on sslhost is OK: SSL OK - miraheze.gq reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [22:06:11] RECOVERY - wiki.fffbr.org - reverse DNS on sslhost is OK: SSL OK - wiki.fffbr.org reverse DNS resolves to cp13.miraheze.org - CNAME OK [22:06:25] Universal_Omega: what do you want to do? [22:06:34] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.48, 6.48, 6.43 [22:08:12] RhinosF1: I will revert the commit if you want. [22:08:17] please [22:08:39] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/J42s6 [22:08:40] [02miraheze/mediawiki] 07Universal-Omega 03aefcf0f - Revert "Update Cosmos" [22:08:45] just adds more commits, I don't think that's necessary [22:08:48] !log depooling mw8 for reboot [22:08:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:09:11] Voidwalker: I'll watch rollout [22:09:53] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 10.21, 7.91, 7.42 [22:10:35] PROBLEM - cp12 Varnish Backends on cp12 is CRITICAL: 1 backends are down. mw8 [22:11:00] !log [rhinos@mw8] enabled puppet [22:11:05] Voidwalker: ^ [22:11:06] go [22:11:07] RECOVERY - mw8 Puppet on mw8 is OK: OK: Puppet is currently enabled, last run 9 seconds ago with 0 failures [22:11:08] PROBLEM - cp13 Varnish Backends on cp13 is CRITICAL: 1 backends are down. mw8 [22:11:22] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 1 backends are down. mw8 [22:11:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:11:46] !log rebooting mw8 [22:11:58] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.58, 7.96, 7.52 [22:12:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:12:22] RECOVERY - mw8 Disk Space on mw8 is OK: DISK OK - free space: / 4280 MB (22% inode=73%); [22:12:24] !log cycle php-fpm on 10 [22:12:25] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 10.46, 8.72, 7.33 [22:12:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:13:07] PROBLEM - savage-wiki.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for savage-wiki.com could not be found [22:13:11] PROBLEM - ref.undu.st - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for ref.undu.st could not be found [22:13:11] PROBLEM - heavyironmodding.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for heavyironmodding.org could not be found [22:13:12] PROBLEM - thelonsdalebattalion.co.uk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for thelonsdalebattalion.co.uk could not be found [22:13:14] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 2.43, 1.05, 0.39 [22:13:16] PROBLEM - en.pornwiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for en.pornwiki.org could not be found [22:13:16] PROBLEM - cnt.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for cnt.miraheze.org could not be found [22:13:17] PROBLEM - wiki.ssij.se - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.ssij.se could not be found [22:13:19] PROBLEM - bots1.miraheze.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for bots1.miraheze.wiki could not be found [22:13:21] PROBLEM - wiki.scvo.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.scvo.org could not be found [22:13:22] PROBLEM - iesd.mobagenie.my.id - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for iesd.mobagenie.my.id could not be found [22:13:23] PROBLEM - www.ethene.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.ethene.wiki could not be found [22:13:57] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.43, 8.57, 7.82 [22:14:15] PROBLEM - www.hoolehistoryheritagesociety.org.uk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.hoolehistoryheritagesociety.org.uk could not be found [22:14:23] PROBLEM - wiki.worlduniversityandschool.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.worlduniversityandschool.org could not be found [22:14:24] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.13, 7.74, 7.12 [22:14:36] !log repool mw8 [22:14:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:14:55] PROBLEM - miraheze.gq - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for miraheze.gq could not be found [22:15:08] RECOVERY - cp13 Varnish Backends on cp13 is OK: All 7 backends are healthy [22:15:22] RECOVERY - cp3 Varnish Backends on cp3 is OK: All 7 backends are healthy [22:15:56] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 5.13, 7.30, 7.45 [22:16:21] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.63, 6.68, 6.80 [22:16:30] RECOVERY - cp12 Varnish Backends on cp12 is OK: All 7 backends are healthy [22:17:48] !log slowly re-enable & run puppet on mw9+, 8 was done while depooled [22:17:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:17:55] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 11.04, 8.42, 7.81 [22:18:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.56, 6.71, 7.86 [22:18:33] !log [rhinos@mw10] enabled puppet [22:18:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:18:44] RECOVERY - mw10 Puppet on mw10 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures [22:19:54] RECOVERY - heavyironmodding.org - reverse DNS on sslhost is OK: SSL OK - heavyironmodding.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [22:20:00] RECOVERY - en.pornwiki.org - reverse DNS on sslhost is OK: SSL OK - en.pornwiki.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [22:20:03] RECOVERY - wiki.ssij.se - reverse DNS on sslhost is OK: SSL OK - wiki.ssij.se reverse DNS resolves to cp13.miraheze.org - CNAME OK [22:20:04] RECOVERY - cnt.miraheze.org - reverse DNS on sslhost is OK: SSL OK - cnt.miraheze.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [22:20:04] RECOVERY - savage-wiki.com - reverse DNS on sslhost is OK: SSL OK - savage-wiki.com reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [22:20:08] RECOVERY - bots1.miraheze.wiki - reverse DNS on sslhost is OK: SSL OK - bots1.miraheze.wiki reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [22:20:08] RECOVERY - ref.undu.st - reverse DNS on sslhost is OK: SSL OK - ref.undu.st reverse DNS resolves to cp13.miraheze.org - CNAME OK [22:20:10] RECOVERY - wiki.scvo.org - reverse DNS on sslhost is OK: SSL OK - wiki.scvo.org reverse DNS resolves to cp13.miraheze.org - CNAME OK [22:20:10] RECOVERY - iesd.mobagenie.my.id - reverse DNS on sslhost is OK: SSL OK - iesd.mobagenie.my.id reverse DNS resolves to cp13.miraheze.org - CNAME OK [22:20:12] RECOVERY - thelonsdalebattalion.co.uk - reverse DNS on sslhost is OK: SSL OK - thelonsdalebattalion.co.uk reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [22:20:15] RECOVERY - www.ethene.wiki - reverse DNS on sslhost is OK: SSL OK - www.ethene.wiki reverse DNS resolves to cp13.miraheze.org - CNAME OK [22:21:00] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.89, 19.10, 17.43 [22:21:01] RECOVERY - www.hoolehistoryheritagesociety.org.uk - reverse DNS on sslhost is OK: SSL OK - www.hoolehistoryheritagesociety.org.uk reverse DNS resolves to cp13.miraheze.org - CNAME OK [22:21:12] RECOVERY - wiki.worlduniversityandschool.org - reverse DNS on sslhost is OK: SSL OK - wiki.worlduniversityandschool.org reverse DNS resolves to cp13.miraheze.org - CNAME OK [22:21:15] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.78, 6.74, 3.27 [22:21:47] RECOVERY - miraheze.gq - reverse DNS on sslhost is OK: SSL OK - miraheze.gq reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [22:22:11] RECOVERY - mw11 Puppet on mw11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:22:16] !log [rhinos@mw11] enabled puppet [22:22:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:23:14] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.85, 6.53, 3.63 [22:23:54] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 5.52, 7.19, 7.57 [22:24:22] !log enable-puppet ran on all appservers, normal puppet usage resumed [22:24:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:24:58] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.91, 17.68, 17.35 [22:25:25] !log [rhinos@mw9] enabled puppet [22:25:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:25:37] RECOVERY - mw9 Puppet on mw9 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures [22:26:20] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 4.45, 2.14, 1.34 [22:28:18] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.77, 1.77, 1.29 [22:29:52] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.55, 5.50, 6.68 [22:30:16] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 0.86, 1.44, 1.22 [22:32:22] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.08, 6.22, 6.88 [22:32:56] !log mw8 100% - restart [22:33:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:33:37] !log 9 too [22:33:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:34:21] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.24, 7.03, 7.13 [22:35:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.59, 7.39, 5.58 [22:36:21] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.02, 7.69, 7.35 [22:37:32] .op [22:37:32] Attempting to OP... [22:39:14] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.15, 6.56, 5.67 [22:50:05] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.31, 8.21, 6.76 [22:52:01] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.87, 7.25, 6.58 [22:52:21] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.79, 7.48, 7.91 [22:53:57] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.26, 6.76, 6.49 [22:58:22] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.12, 6.82, 7.35 [23:00:21] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.01, 6.55, 7.19 [23:00:57] PROBLEM - cp12 Disk Space on cp12 is WARNING: DISK WARNING - free space: / 4232 MB (10% inode=96%); [23:02:23] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 12.81, 8.44, 7.77 [23:03:39] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 11.01, 8.23, 7.10 [23:05:47] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.35, 7.35, 6.27 [23:07:19] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+1/-0/±0] 13https://git.io/J42o4 [23:07:21] [02miraheze/ManageWiki] 07Universal-Omega 037a80d12 - Create ext.managewiki.oouiform.confirmClose.js [23:07:22] [02ManageWiki] 07Universal-Omega created branch 03Universal-Omega-patch-2 - 13https://git.io/vpSns [23:07:30] [02ManageWiki] 07Universal-Omega opened pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:07:46] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.20, 7.49, 6.46 [23:08:02] Universal_Omega: see -security [23:08:34] miraheze/ManageWiki - Universal-Omega the build passed. [23:09:21] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J42Kq [23:09:23] [02miraheze/ManageWiki] 07Universal-Omega 03e2e6a27 - Update ManageWikiFormFactory.php [23:09:24] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:09:25] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.91, 7.71, 7.29 [23:09:46] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.82, 6.74, 6.31 [23:10:15] miraheze/ManageWiki - Universal-Omega the build passed. [23:11:21] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 13.78, 9.60, 7.99 [23:14:33] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J42ic [23:14:35] [02miraheze/ManageWiki] 07Universal-Omega 0304f6f0f - Update ext.managewiki.oouiform.js [23:14:36] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:15:28] miraheze/ManageWiki - Universal-Omega the build passed. [23:16:07] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J42ih [23:16:09] [02miraheze/ManageWiki] 07Universal-Omega 033dbc36d - Update ext.managewiki.oouiform.styles.less [23:16:10] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:17:04] miraheze/ManageWiki - Universal-Omega the build passed. [23:17:26] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J42P1 [23:17:28] [02miraheze/ManageWiki] 07Universal-Omega 037f9f5b8 - Update ManageWikiOOUIForm.php [23:17:29] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:18:27] miraheze/ManageWiki - Universal-Omega the build passed. [23:19:39] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 11.58, 8.57, 6.98 [23:20:58] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.22, 7.17, 6.16 [23:21:38] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J4216 [23:21:40] [02miraheze/ManageWiki] 07Universal-Omega 031b69bbc - Update extension.json [23:21:41] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:22:43] miraheze/ManageWiki - Universal-Omega the build passed. [23:23:13] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J42MB [23:23:15] [02miraheze/ManageWiki] 07Universal-Omega 0392034d6 - Update ext.managewiki.oouiform.js [23:23:16] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:24:18] miraheze/ManageWiki - Universal-Omega the build passed. [23:24:56] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.60, 6.65, 6.20 [23:25:11] dmehus: if you're around, now is probably the worse time to create wikis [23:25:24] Load is super high so chance of it failing is high [23:25:28] Anyway I'm off to sleep [23:25:33] There's a UBN phab task [23:25:34] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.02, 7.46, 7.04 [23:25:38] For php low workers [23:25:56] Oh you're deleting [23:26:01] That's fine [23:28:21] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.40, 6.91, 7.98 [23:29:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 4.97, 7.05, 7.90 [23:29:31] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.49, 6.04, 6.58 [23:32:02] PROBLEM - test3 Puppet on test3 is WARNING: WARNING: Puppet is currently disabled, message: Testing, last run 28 minutes ago with 0 failures [23:33:15] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.82, 7.42, 7.83 [23:33:26] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:33:28] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J429a [23:33:29] [02miraheze/ManageWiki] 07Universal-Omega 0321f08b3 - Update ext.managewiki.oouiform.confirmClose.js [23:33:42] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.52, 7.62, 6.68 [23:34:08] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J4291 [23:34:09] [02miraheze/ManageWiki] 07Universal-Omega 0307546d4 - Update ManageWikiOOUIForm.php [23:34:11] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:34:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.57, 8.10, 8.02 [23:34:27] miraheze/ManageWiki - Universal-Omega the build passed. [23:35:06] miraheze/ManageWiki - Universal-Omega the build passed. [23:35:38] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.02, 7.14, 6.63 [23:36:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.17, 7.42, 7.77 [23:39:33] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.23, 6.80, 6.62 [23:39:48] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J427z [23:39:50] [02miraheze/ManageWiki] 07Universal-Omega 032254301 - Update ManageWikiOOUIForm.php [23:39:52] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:39:54] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.90, 1.91, 1.38 [23:40:43] miraheze/ManageWiki - Universal-Omega the build passed. [23:41:00] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J425v [23:41:02] [02miraheze/ManageWiki] 07Universal-Omega 035ba09e1 - Update ManageWikiOOUIForm.php [23:41:03] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:41:27] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J425Z [23:41:29] [02miraheze/ManageWiki] 07Universal-Omega 03fb830f9 - Update ext.managewiki.oouiform.confirmClose.js [23:41:30] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:41:51] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 0.76, 1.47, 1.28 [23:42:03] miraheze/ManageWiki - Universal-Omega the build passed. [23:42:28] miraheze/ManageWiki - Universal-Omega the build passed. [23:42:45] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:42:46] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J425H [23:42:48] [02miraheze/ManageWiki] 07Universal-Omega 03d7e6e8c - Update ext.managewiki.oouiform.confirmClose.js [23:43:25] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 9.56, 7.14, 6.60 [23:43:40] miraheze/ManageWiki - Universal-Omega the build passed. [23:44:22] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.19, 7.25, 7.38 [23:45:25] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.58, 6.71, 6.53 [23:46:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.87, 6.69, 7.16 [23:46:24] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J42Fx [23:46:25] [02miraheze/ManageWiki] 07Universal-Omega 036052a58 - Update ext.managewiki.oouiform.confirmClose.js [23:46:27] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:47:24] miraheze/ManageWiki - Universal-Omega the build passed. [23:50:00] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J42ND [23:50:02] [02miraheze/ManageWiki] 07Universal-Omega 03200b80b - Update SpecialManageWiki.php [23:50:03] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:50:57] miraheze/ManageWiki - Universal-Omega the build passed. [23:51:15] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.66, 7.02, 7.85 [23:53:15] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.54, 7.65, 7.96 [23:54:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.82, 7.82, 7.37 [23:56:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.59, 7.13, 7.17 [23:56:27] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J42h3 [23:56:28] [02miraheze/ManageWiki] 07Universal-Omega 033dc2183 - Update SpecialManageWiki.php [23:56:30] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:57:11] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.84, 7.60, 6.75 [23:57:30] miraheze/ManageWiki - Universal-Omega the build passed. [23:58:32] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/J42jY [23:58:34] [02miraheze/ManageWiki] 07Universal-Omega 036cc025a - Update SpecialManageWiki.php [23:58:35] [02ManageWiki] 07Universal-Omega synchronize pull request 03#289: Give close confirmation if settings are changed - 13https://git.io/J42o0 [23:59:07] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.50, 6.88, 6.60 [23:59:22] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.86, 6.79, 6.35 [23:59:28] miraheze/ManageWiki - Universal-Omega the build passed.