[00:03:25] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.72, 7.04, 5.84 [00:04:08] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.88, 7.58, 5.97 [00:04:20] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.18, 6.56, 5.38 [00:04:28] PROBLEM - graylog2 Puppet on graylog2 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 4 minutes ago with 10 failures. Failed resources (up to 3 shown): File[/usr/local/bin/nginx-prometheus-exporter],File[/etc/nginx/sites-available/graylog-proxy],File[/etc/sysctl.d],File[/etc/logrotate.d/nginx] [00:06:29] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 27.12, 21.82, 17.46 [00:08:39] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.35, 22.02, 18.14 [00:10:39] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 26.99, 22.96, 18.90 [00:12:23] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.53, 7.75, 6.92 [00:13:47] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.98, 7.85, 7.43 [00:14:18] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.85, 7.79, 7.21 [00:19:45] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 11.15, 8.06, 7.51 [00:20:19] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.15, 7.88, 7.26 [00:20:19] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.43, 7.53, 7.07 [00:20:54] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.58, 23.91, 22.00 [00:21:54] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.60, 7.50, 7.38 [00:22:22] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.08, 6.18, 6.63 [00:22:22] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.85, 6.93, 6.98 [00:24:19] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.53, 6.23, 6.73 [00:25:53] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.19, 5.71, 6.67 [00:29:04] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.69, 17.64, 19.95 [00:30:00] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.19, 3.48, 3.09 [00:32:03] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.21, 3.51, 3.16 [00:32:40] RECOVERY - graylog2 Puppet on graylog2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:34:09] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.72, 3.24, 3.11 [00:35:54] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.20, 7.08, 6.25 [00:36:13] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.59, 6.67, 6.32 [00:36:15] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.84, 7.29, 6.77 [00:36:22] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.16, 6.81, 6.45 [00:37:56] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.37, 6.70, 6.21 [00:38:12] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.71, 6.12, 6.15 [00:38:18] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.59, 6.50, 6.54 [00:38:21] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.13, 6.65, 6.44 [00:47:12] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.61, 6.59, 6.35 [00:49:16] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 6.53, 6.51, 6.36 [00:52:46] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.88, 7.07, 6.37 [00:53:15] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.85, 6.85, 6.46 [00:53:27] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.10, 3.44, 3.11 [00:53:53] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.74, 7.92, 7.06 [00:55:21] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.94, 6.14, 6.25 [00:55:33] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.34, 3.40, 3.13 [00:55:54] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 11.10, 9.56, 7.79 [00:56:52] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.12, 7.66, 6.82 [00:57:14] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.15, 6.97, 6.32 [01:01:04] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.02, 7.77, 7.06 [01:01:57] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.27, 3.64, 3.29 [01:03:39] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.35, 6.65, 6.46 [01:03:52] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 14.76, 9.11, 7.30 [01:04:08] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.73, 3.44, 3.28 [01:04:08] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 5.90, 7.56, 7.59 [01:05:40] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.16, 9.68, 7.56 [01:06:00] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.63, 7.75, 6.88 [01:07:27] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.54, 7.48, 7.33 [01:08:04] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.99, 6.69, 6.59 [01:08:23] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.68, 3.10, 3.18 [01:09:32] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.65, 7.68, 7.39 [01:09:45] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.53, 7.89, 7.32 [01:11:31] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.55, 7.23, 7.24 [01:12:10] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.07, 7.71, 7.72 [01:16:07] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.14, 7.02, 7.18 [01:17:38] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.33, 5.80, 6.59 [01:18:03] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 6.79, 6.87, 7.10 [01:20:22] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.35, 7.34, 7.39 [01:21:44] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.25, 6.14, 6.78 [01:22:21] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 5.18, 6.46, 7.05 [01:24:00] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.45, 6.24, 6.77 [01:24:23] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.84, 6.00, 6.79 [01:33:51] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.76, 6.71, 6.33 [01:34:58] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.75, 7.30, 6.81 [01:35:01] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.26, 7.16, 6.85 [01:35:53] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 12.77, 8.83, 7.14 [01:36:37] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.57, 3.96, 3.48 [01:36:39] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.87, 7.03, 6.66 [01:36:58] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.18, 6.43, 6.00 [01:38:42] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.31, 3.54, 3.38 [01:38:43] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.37, 6.37, 6.46 [01:38:56] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 6.92, 7.75, 7.24 [01:38:56] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.43, 5.90, 5.85 [01:38:59] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.42, 7.18, 6.92 [01:39:59] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.02, 7.96, 7.23 [01:40:47] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.13, 3.35, 3.33 [01:44:51] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 4.54, 6.13, 6.71 [01:45:00] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.59, 6.36, 6.72 [01:48:13] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.83, 6.05, 6.61 [01:53:38] PROBLEM - db13 Current Load on db13 is CRITICAL: CRITICAL - load average: 15.83, 8.73, 4.32 [02:06:11] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.29, 6.64, 5.60 [02:08:08] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.70, 5.91, 5.46 [02:08:57] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.60, 6.90, 5.42 [02:11:03] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 5.92, 6.29, 5.37 [02:38:52] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 8.11, 7.23, 6.47 [02:40:54] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.70, 7.06, 6.47 [02:44:49] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 4.89, 6.13, 6.24 [02:54:08] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.02, 6.40, 5.69 [02:56:05] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.95, 6.26, 5.73 [03:25:27] PROBLEM - db13 APT on db13 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [03:27:32] RECOVERY - db13 APT on db13 is OK: APT OK: 49 packages available for upgrade (0 critical updates). [04:24:55] PROBLEM - lcn.zfc.id.lv - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'lcn.zfc.id.lv' expires in 15 day(s) (Thu 21 Oct 2021 04:19:31 GMT +0000). [04:25:13] PROBLEM - wiki.landev.vn - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.landev.vn' expires in 15 day(s) (Thu 21 Oct 2021 04:18:15 GMT +0000). [04:26:11] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.08, 7.23, 6.70 [04:32:24] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 8.45, 7.27, 6.84 [04:32:34] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 1 backends are down. mw10 [04:34:25] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 6.61, 6.83, 6.72 [04:34:36] RECOVERY - cp3 Varnish Backends on cp3 is OK: All 9 backends are healthy [04:36:21] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 5.77, 6.38, 6.56 [04:46:45] PROBLEM - db13 APT on db13 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [04:53:17] RECOVERY - db13 APT on db13 is OK: APT OK: 49 packages available for upgrade (0 critical updates). [07:13:42] PROBLEM - db13 APT on db13 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [07:15:46] RECOVERY - db13 APT on db13 is OK: APT OK: 49 packages available for upgrade (0 critical updates). [08:11:45] PROBLEM - db13 APT on db13 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [08:18:06] RECOVERY - db13 APT on db13 is OK: APT OK: 49 packages available for upgrade (0 critical updates). [08:48:04] PROBLEM - db13 APT on db13 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [08:50:16] RECOVERY - db13 APT on db13 is OK: APT OK: 49 packages available for upgrade (0 critical updates). [09:47:16] > @Joseph: wiki username? Ugochimobi [09:48:15] @Joseph: I think we discussed. Can you email me and I'll send instructions tonight? [09:48:30] Stupid wifi is annoying here [09:50:01] What's your E-mail address? [09:50:36] rhinosf1@miraheze.org [09:56:46] RhinosF1 Check your spam folder I guess [09:57:07] Got it [09:58:26] Okay, hope to hear from you soon. [10:01:07] @Joseph: think it actually sent [10:01:15] Please follow the instructions [10:01:25] Do not share the information with me [10:02:51] Okay, will check [10:08:53] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 8.21, 7.25, 6.57 [10:10:48] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 6.21, 6.71, 6.44 [11:12:11] PROBLEM - db13 APT on db13 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [11:16:25] RECOVERY - db13 APT on db13 is OK: APT OK: 49 packages available for upgrade (0 critical updates). [12:14:27] PROBLEM - db13 APT on db13 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [12:18:41] RECOVERY - db13 APT on db13 is OK: APT OK: 49 packages available for upgrade (0 critical updates). [12:52:56] !log killed queries on db13 [12:52:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:08:38] PROBLEM - db13 Current Load on db13 is WARNING: WARNING - load average: 2.20, 2.33, 7.93 [13:11:43] RECOVERY - db13 Current Load on db13 is OK: OK - load average: 1.48, 1.89, 6.74 [13:36:35] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.36, 5.71, 5.01 [13:37:43] RhinosF1: Reply mail [13:38:41] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.33, 5.62, 5.07 [13:39:14] PROBLEM - wiki.seamly.net - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:48:24] PROBLEM - housing.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:51:05] PROBLEM - housing.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - housing.wiki All nameservers failed to answer the query. [13:53:28] PROBLEM - wiki.seamly.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.seamly.net' expires in 14 day(s) (Wed 20 Oct 2021 02:47:33 GMT +0000). [13:55:15] PROBLEM - housing.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'housing.wiki' expires in 14 day(s) (Tue 19 Oct 2021 23:12:14 GMT +0000). [13:58:06] PROBLEM - housing.wiki - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns1.dreamhost.com.', 'ns2.dreamhost.com.', 'ns3.dreamhost.com.'], 'CNAME': None} [14:26:42] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JVW8Y [14:26:44] [02miraheze/mw-config] 07RhinosF1 03d800121 - MWE: disable DPL(3) on pages > 30,000 [14:27:16] JohnLewis: ^ [14:28:21] !log [rhinos@test3] starting deploy of {'config': True} to skip [14:28:22] !log [rhinos@test3] finished deploy of {'config': True} to skip - SUCCESS in 0s [14:28:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:28:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:30:15] !log [@mw11] starting deploy of {'config': True} to all [14:30:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:30:26] !log [@mw11] finished deploy of {'config': True} to all - SUCCESS in 11s [14:30:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:36:52] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.94, 3.31, 2.80 [14:37:26] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.13, 6.42, 5.31 [14:37:38] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.53, 6.71, 5.17 [14:38:16] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.03, 6.80, 5.77 [14:39:30] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.43, 6.33, 5.41 [14:39:43] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.99, 6.65, 5.33 [14:40:16] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.08, 7.44, 6.14 [14:41:51] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.37, 6.59, 5.51 [14:42:18] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 6.26, 7.01, 6.15 [14:43:06] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.21, 3.35, 3.01 [14:44:14] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.55, 6.43, 6.03 [14:49:21] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.09, 6.13, 5.22 [14:50:07] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.05, 7.07, 5.99 [14:50:18] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.76, 7.11, 6.42 [14:50:38] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.30, 7.71, 6.07 [14:51:36] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.66, 6.91, 5.81 [14:52:22] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.62, 7.84, 6.79 [14:53:04] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.78, 7.43, 6.39 [14:53:32] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.77, 7.28, 5.98 [14:54:39] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.84, 7.35, 6.31 [14:55:09] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 10.57, 8.52, 6.91 [14:56:30] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.46, 7.88, 6.82 [14:57:35] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.53, 6.77, 6.06 [14:58:43] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.51, 7.54, 6.59 [15:00:44] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 10.22, 8.19, 7.13 [15:00:47] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.00, 7.34, 6.64 [15:01:50] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.50, 7.70, 6.62 [15:02:49] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.09, 7.36, 6.71 [15:04:14] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.71, 8.04, 6.79 [15:08:32] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.42, 7.77, 7.02 [15:10:40] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.31, 7.82, 7.12 [15:13:04] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.38, 7.92, 7.47 [15:14:22] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.82, 7.80, 7.49 [15:15:26] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.15, 3.84, 3.23 [15:16:10] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.19, 7.29, 7.81 [15:16:23] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.38, 7.87, 7.54 [15:17:01] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.08, 7.79, 7.41 [15:17:34] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.09, 3.42, 3.15 [15:17:53] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.07, 7.61, 7.93 [15:18:27] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.78, 7.06, 7.28 [15:21:11] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.07, 8.37, 7.69 [15:21:45] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.13, 4.03, 3.48 [15:22:09] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.49, 7.36, 7.67 [15:23:14] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.96, 7.98, 7.65 [15:23:48] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.42, 3.74, 3.44 [15:24:16] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 5.75, 6.59, 7.34 [15:24:29] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.06, 7.00, 7.35 [15:26:31] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.91, 6.97, 7.33 [15:26:33] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.19, 5.85, 6.78 [15:27:20] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 3.54, 5.72, 6.59 [15:29:58] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.41, 2.92, 3.27 [15:30:26] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.79, 5.86, 6.80 [15:30:30] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.20, 5.57, 6.67 [15:30:45] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 5.23, 6.36, 7.67 [15:33:30] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.94, 5.92, 6.70 [15:34:44] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 10.25, 7.68, 7.84 [15:35:32] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.22, 7.38, 6.80 [15:36:49] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 6.91, 7.56, 7.79 [15:36:53] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.85, 6.99, 6.92 [15:36:57] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.06, 8.27, 7.51 [15:37:32] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.57, 6.95, 6.72 [15:37:51] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.20, 6.68, 6.87 [15:38:54] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.86, 6.14, 6.61 [15:39:00] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.01, 7.57, 7.36 [15:39:30] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.98, 6.66, 6.65 [15:39:55] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.80, 6.21, 6.69 [15:44:30] PROBLEM - gluster3 Disk Space on gluster3 is WARNING: DISK WARNING - free space: / 117630 MB (10% inode=80%); [15:44:37] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 4.15, 5.62, 6.80 [15:47:07] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.20, 5.60, 6.56 [15:55:55] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.92, 8.40, 7.42 [15:57:56] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.24, 7.45, 7.18 [15:58:25] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.84, 3.60, 3.31 [16:00:26] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.52, 3.19, 3.19 [16:06:13] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.83, 6.31, 6.78 [16:07:49] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 6.73, 6.86, 6.39 [16:09:46] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 6.70, 6.80, 6.43 [16:26:50] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.98, 6.92, 6.40 [16:33:02] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.00, 7.30, 6.39 [16:33:10] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 9.33, 7.76, 6.91 [16:35:32] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.87, 7.33, 5.95 [16:35:35] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.71, 7.00, 6.01 [16:35:44] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.12, 7.06, 5.99 [16:37:36] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.05, 6.73, 5.90 [16:37:39] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.09, 6.69, 6.02 [16:37:51] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.70, 6.68, 6.00 [16:39:27] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.54, 7.81, 7.29 [16:41:14] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.03, 7.71, 7.16 [16:42:17] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.50, 3.38, 3.08 [16:44:22] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.86, 3.21, 3.06 [16:45:35] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 6.43, 6.21, 6.71 [16:47:22] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 3.95, 5.96, 6.58 [16:49:53] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.06, 10.13, 7.70 [16:52:22] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 4.79, 7.79, 7.17 [16:52:40] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 9.78, 8.93, 7.32 [16:53:39] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 19.11, 22.00, 18.43 [16:54:25] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 3.55, 6.45, 6.77 [16:55:41] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 13.98, 19.24, 17.88 [16:56:36] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 5.16, 6.88, 6.85 [16:58:32] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 4.20, 5.95, 6.51 [17:22:38] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.05, 3.50, 3.26 [17:24:43] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.68, 3.42, 3.25 [17:26:45] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.27, 3.25, 3.20 [17:35:59] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.21, 6.27, 5.72 [17:37:18] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.58, 3.46, 3.16 [17:38:02] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 6.10, 6.17, 5.75 [17:39:19] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.04, 3.29, 3.13 [17:51:39] !log mon2: update grafana tzdata [17:51:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:02:39] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.56, 7.02, 6.20 [18:06:44] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 12.42, 8.81, 7.03 [18:07:10] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.52, 7.13, 6.50 [18:07:57] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.41, 7.77, 6.65 [18:08:09] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 11.59, 9.12, 7.18 [18:11:14] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.50, 7.53, 6.79 [18:12:10] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 4.99, 6.83, 6.60 [18:12:19] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.92, 7.56, 7.00 [18:12:48] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 6.76, 7.67, 7.16 [18:13:14] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.52, 7.00, 6.68 [18:14:20] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 10.25, 8.20, 7.27 [18:14:44] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.52, 7.87, 7.28 [18:16:15] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.65, 6.65, 6.60 [18:16:41] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.12, 7.46, 7.20 [18:22:35] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.14, 7.88, 7.55 [18:22:35] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.96, 3.53, 3.31 [18:24:46] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.14, 3.78, 3.41 [18:24:50] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 9.72, 8.09, 7.50 [18:25:34] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.29, 6.24, 6.65 [18:26:49] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.46, 7.68, 7.51 [18:26:53] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.38, 3.50, 3.35 [18:28:46] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.57, 7.63, 7.43 [18:28:50] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.25, 6.93, 7.27 [18:28:55] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.63, 3.17, 3.25 [18:36:51] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.70, 6.21, 6.67 [18:38:35] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 6.42, 6.41, 6.78 [19:06:52] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.02, 5.95, 5.46 [19:08:52] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.89, 5.39, 5.31 [20:06:28] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.11, 6.68, 5.69 [20:06:50] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.17, 6.29, 5.71 [20:08:31] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.98, 6.64, 5.81 [20:08:48] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.47, 6.03, 5.69 [20:21:51] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.35, 6.55, 6.02 [20:23:53] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.72, 6.17, 5.95 [20:36:15] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.15, 7.73, 6.45 [20:37:42] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 9.69, 7.75, 6.44 [20:37:44] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.05, 6.97, 5.83 [20:38:11] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.13, 7.90, 6.69 [20:38:21] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.75, 7.41, 6.50 [20:39:40] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 5.94, 6.85, 6.25 [20:39:43] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.22, 6.31, 5.73 [20:42:19] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.51, 7.99, 6.95 [20:44:30] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.80, 7.76, 7.01 [20:44:41] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.54, 6.69, 6.51 [20:45:51] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.49, 7.94, 6.93 [20:46:42] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.65, 7.95, 7.16 [20:47:32] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.68, 7.02, 6.17 [20:47:37] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.48, 3.36, 3.12 [20:47:59] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 6.79, 7.95, 7.10 [20:49:39] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.62, 6.59, 6.13 [20:49:40] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.23, 3.05, 3.04 [20:50:03] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.18, 8.38, 7.38 [20:51:15] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.54, 6.82, 6.59 [20:51:50] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.53, 7.20, 6.19 [20:53:54] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 12.64, 8.79, 7.34 [20:54:33] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.55, 9.26, 7.22 [20:55:09] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 12.73, 9.98, 7.67 [20:55:42] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.90, 7.88, 6.86 [20:55:50] @Bukkit: to get security reviewer you just need to provide good explanations on security tickets [20:55:51] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.92, 23.32, 19.55 [20:56:21] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 3.22, 5.20, 3.77 [20:57:59] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 14.36, 20.14, 18.89 [20:58:24] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.17, 7.73, 7.25 [20:58:27] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 1.42, 3.84, 3.45 [20:58:47] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.05, 7.19, 6.86 [20:59:27] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 5.90, 7.86, 7.36 [20:59:58] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.01, 7.86, 7.81 [21:00:54] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 5.32, 7.42, 7.80 [21:01:40] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.16, 6.52, 6.61 [21:02:21] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.47, 6.37, 6.78 [21:05:36] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.49, 7.19, 7.09 [21:05:48] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.43, 6.89, 6.76 [21:06:33] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.45, 7.52, 7.14 [21:06:52] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 10.22, 8.49, 8.09 [21:07:40] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 5.81, 6.82, 6.98 [21:07:45] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.88, 6.40, 6.61 [21:08:04] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.06, 5.62, 6.73 [21:08:34] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 5.97, 6.97, 6.99 [21:08:45] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.59, 7.93, 7.92 [21:11:45] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.57, 6.01, 6.64 [21:12:42] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.38, 6.17, 6.71 [21:12:44] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.39, 6.51, 6.77 [21:20:33] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.15, 5.79, 6.68 [21:29:30] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.92, 6.93, 6.74 [21:31:28] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 4.76, 6.01, 6.43 [21:50:25] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.47, 3.35, 2.96 [21:52:27] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.58, 2.94, 2.85 [22:24:40] !log experimenting increasing apc.shm_size on mw* [22:24:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:32:59] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JVKE2 [22:33:00] [02miraheze/puppet] 07paladox 03be64b33 - mediawiki: Set apc.shm_size for php [22:33:02] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [22:33:03] [02puppet] 07paladox opened pull request 03#2004: mediawiki: Set apc.shm_size for php - 13https://git.io/JVKEK [22:45:43] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JVK7m [22:45:44] [02miraheze/puppet] 07paladox 03c33a3ac - Update php.pp [22:45:46] [02puppet] 07paladox synchronize pull request 03#2004: mediawiki: Set apc.shm_size for php - 13https://git.io/JVKEK [23:29:56] Yeah, I understand that. I realized I was pretty vague.