[00:00:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 45% [00:00:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 70% [00:02:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 68% [00:03:50] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 17.24, 9.18, 4.12 [00:06:18] PROBLEM - os131 PowerDNS Recursor on os131 is CRITICAL: CRITICAL - Plugin timed out while executing system call [00:06:29] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.89, 3.03, 2.61 [00:06:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 52% [00:07:27] PROBLEM - os131 Current Load on os131 is CRITICAL: LOAD CRITICAL - total load average: 5.97, 3.54, 1.94 [00:07:52] PROBLEM - db131 Current Load on db131 is WARNING: WARNING - load average: 7.04, 5.37, 3.53 [00:08:15] RECOVERY - os131 PowerDNS Recursor on os131 is OK: DNS OK: 0.897 seconds response time. miraheze.org returns 2001:41d0:801:2000::3a18,2001:41d0:801:2000::5d68,51.195.201.140,51.89.139.24 [00:08:25] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.31, 3.37, 2.78 [00:09:52] RECOVERY - db131 Current Load on db131 is OK: OK - load average: 6.64, 6.01, 4.00 [00:10:21] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.91, 3.48, 2.89 [00:10:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 58% [00:11:25] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 2.59, 3.56, 2.37 [00:11:50] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 3.31, 7.14, 5.70 [00:12:17] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.73, 3.22, 2.87 [00:12:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 65% [00:12:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 82% [00:12:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is WARNING: WARNING - NGINX Error Rate is 59% [00:13:25] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 2.40, 3.16, 2.36 [00:13:50] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 2.80, 5.67, 5.33 [00:16:32] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 50% [00:16:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 72% [00:20:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 52% [00:20:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 70% [00:22:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 65% [00:24:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 54% [00:26:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 76% [00:28:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 50% [00:28:40] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 53% [00:28:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is WARNING: WARNING - NGINX Error Rate is 57% [00:30:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 71% [00:30:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 65% [00:32:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 75% [00:32:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is WARNING: WARNING - NGINX Error Rate is 53% [00:34:00] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 6.35, 4.18, 3.08 [00:34:32] PROBLEM - os131 Current Load on os131 is CRITICAL: LOAD CRITICAL - total load average: 5.93, 4.47, 3.28 [00:34:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 77% [00:36:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is WARNING: WARNING - NGINX Error Rate is 57% [00:37:52] PROBLEM - db131 Current Load on db131 is WARNING: WARNING - load average: 7.78, 6.83, 5.03 [00:38:19] PROBLEM - os131 PowerDNS Recursor on os131 is CRITICAL: CRITICAL - Plugin timed out while executing system call [00:38:40] PROBLEM - mw134 Current Load on mw134 is CRITICAL: CRITICAL - load average: 17.54, 11.66, 6.95 [00:38:46] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 11.63, 8.95, 6.21 [00:38:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 65% [00:40:46] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 8.75, 8.44, 6.33 [00:41:28] RECOVERY - Host cloud11 is UP: PING OK - Packet loss = 0%, RTA = 0.45 ms [00:41:55] PROBLEM - db131 Current Load on db131 is CRITICAL: CRITICAL - load average: 9.32, 7.76, 5.82 [00:42:22] RECOVERY - os131 PowerDNS Recursor on os131 is OK: DNS OK: 0.819 seconds response time. miraheze.org returns 2001:41d0:801:2000::3a18,2001:41d0:801:2000::5d68,51.195.201.140,51.89.139.24 [00:42:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 53% [00:42:42] RECOVERY - ping6 on cloud11 is OK: PING OK - Packet loss = 0%, RTA = 1.91 ms [00:42:51] PROBLEM - mw134 Current Load on mw134 is WARNING: WARNING - load average: 9.35, 11.84, 8.25 [00:45:52] PROBLEM - db131 Current Load on db131 is WARNING: WARNING - load average: 6.77, 7.60, 6.20 [00:46:40] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 77% [00:48:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is WARNING: WARNING - NGINX Error Rate is 59% [00:48:48] RECOVERY - mw134 Current Load on mw134 is OK: OK - load average: 4.44, 8.32, 7.92 [00:49:52] RECOVERY - db131 Current Load on db131 is OK: OK - load average: 3.24, 5.69, 5.77 [00:50:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 49% [00:50:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 70% [00:51:57] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 2.92, 3.55, 3.99 [00:52:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 67% [00:53:56] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 2.65, 3.37, 3.83 [00:54:42] PROBLEM - ping6 on cloud11 is CRITICAL: CRITICAL - Host Unreachable (2a10:6740::6:200) [00:54:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is WARNING: WARNING - NGINX Error Rate is 50% [00:55:21] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.03, 3.78, 3.09 [00:55:27] PROBLEM - Host cloud11 is DOWN: CRITICAL - Host Unreachable (2a10:6740::6:200) [00:56:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 57% [00:56:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 68% [00:57:20] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.40, 3.50, 3.06 [00:58:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 77% [00:59:42] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 2.61, 2.67, 3.36 [01:01:20] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.42, 3.71, 3.23 [01:01:56] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 3.10, 2.80, 3.36 [01:02:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 59% [01:03:19] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.37, 3.55, 3.23 [01:04:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 59% [01:04:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is WARNING: WARNING - NGINX Error Rate is 45% [01:05:56] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.82, 3.39, 3.47 [01:06:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 67% [01:06:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 62% [01:06:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 43% [01:06:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 68% [01:07:25] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.40, 3.67, 3.33 [01:07:56] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.15, 2.95, 3.30 [01:08:11] PROBLEM - cities.simulz.kr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - cities.simulz.kr All nameservers failed to answer the query. [01:08:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 68% [01:09:19] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 2.85, 3.48, 3.31 [01:11:18] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.18, 3.06, 3.18 [01:12:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 55% [01:14:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 58% [01:14:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is WARNING: WARNING - NGINX Error Rate is 53% [01:15:19] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.50, 3.23, 3.21 [01:16:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 61% [01:16:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 75% [01:16:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 66% [01:17:19] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.02, 2.80, 3.06 [01:18:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 52% [01:20:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 50% [01:22:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 57% [01:22:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 62% [01:22:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 74% [01:24:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 74% [01:26:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 58% [01:30:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 71% [01:32:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is WARNING: WARNING - NGINX Error Rate is 56% [01:34:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 64% [01:36:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 55% [01:36:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is WARNING: WARNING - NGINX Error Rate is 47% [01:36:53] RECOVERY - cities.simulz.kr - reverse DNS on sslhost is OK: SSL OK - cities.simulz.kr reverse DNS resolves to cp25.miraheze.org - CNAME OK [01:38:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 67% [01:38:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 60% [01:40:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 59% [01:40:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is WARNING: WARNING - NGINX Error Rate is 49% [01:42:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 61% [01:42:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 58% [01:42:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 65% [01:44:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 95% [01:45:28] RECOVERY - Host cloud11 is UP: PING OK - Packet loss = 0%, RTA = 0.36 ms [01:46:06] [02miraheze/mw-config] 07The-Voidwalker pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/a24b405aa9a0...a90feaf55235 [01:46:09] [02miraheze/mw-config] 07The-Voidwalker 03a90feaf - add sitenotice for images issue [01:46:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 56% [01:46:33] RECOVERY - ping6 on cloud11 is OK: PING OK - Packet loss = 0%, RTA = 0.41 ms [01:47:04] miraheze/mw-config - The-Voidwalker the build passed. [01:48:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 86% [01:48:21] !log [void@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [01:48:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:48:34] !log [void@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 13s [01:48:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:48:51] RECOVERY - www.twistwoodtaleswiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'www.twistwoodtaleswiki.com' will expire on Wed 20 Dec 2023 22:08:34 GMT +0000. [01:50:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 52% [01:52:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 73% [01:52:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 56% [01:54:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 76% [01:54:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 59% [01:55:51] !log [@test131] starting deploy of {'config': True} to all [01:55:53] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 1s [01:55:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:56:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:56:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 66% [01:58:39] RECOVERY - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is OK: OK - NGINX Error Rate is 24% [02:02:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 70% [02:06:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 47% [02:08:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 58% [02:10:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 55% [02:10:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 68% [02:10:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 69% [02:12:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 76% [02:12:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 55% [02:14:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 56% [02:14:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 70% [02:16:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 60% [02:16:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 56% [02:18:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 69% [02:20:10] PROBLEM - www.twistwoodtaleswiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address www.twistwoodtaleswiki.com and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [02:22:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 58% [02:24:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 72% [02:28:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 53% [02:30:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 82% [02:36:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 46% [02:36:44] PROBLEM - ping6 on cloud11 is CRITICAL: CRITICAL - Host Unreachable (2a10:6740::6:200) [02:37:27] PROBLEM - Host cloud11 is DOWN: CRITICAL - Host Unreachable (2a10:6740::6:200) [02:38:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 70% [02:44:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 55% [02:44:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 59% [02:45:37] RECOVERY - ewgf.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'ewgf.wiki' will expire on Wed 20 Dec 2023 22:59:05 GMT +0000. [02:46:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 67% [02:48:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 66% [02:48:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 48% [02:50:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 57% [02:50:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 74% [02:52:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 79% [03:00:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 57% [03:01:12] RECOVERY - db142 Backups SQL on db142 is OK: FILE_AGE OK: /var/log/sql-backup.log is 71 seconds old and 0 bytes [03:02:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 56% [03:02:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 83% [03:02:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is WARNING: WARNING - NGINX Error Rate is 53% [03:02:59] PROBLEM - mon141 Backups Grafana on mon141 is WARNING: FILE_AGE WARNING: /var/log/grafana-backup.log is 864158 seconds old and 93 bytes [03:04:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 73% [03:04:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 69% [03:05:28] RECOVERY - Host cloud11 is UP: PING OK - Packet loss = 0%, RTA = 0.43 ms [03:06:39] RECOVERY - ping6 on cloud11 is OK: PING OK - Packet loss = 0%, RTA = 1.45 ms [03:08:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is WARNING: WARNING - NGINX Error Rate is 55% [03:10:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 64% [03:12:39] PROBLEM - ping6 on cloud11 is CRITICAL: CRITICAL - Host Unreachable (2a10:6740::6:200) [03:12:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is WARNING: WARNING - NGINX Error Rate is 59% [03:13:27] PROBLEM - Host cloud11 is DOWN: CRITICAL - Host Unreachable (2a10:6740::6:200) [03:14:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 75% [03:16:17] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ac703cbb0c61...1aa1fe64b7c5 [03:16:19] [02miraheze/ssl] 07MirahezeSSLBot 031aa1fe6 - Bot: Update SSL cert for www.thesimswiki.com [03:16:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is WARNING: WARNING - NGINX Error Rate is 48% [03:17:37] PROBLEM - ewgf.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address ewgf.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [03:18:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 51% [03:18:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is WARNING: WARNING - NGINX Error Rate is 59% [03:18:48] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 71% [03:20:11] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1aa1fe64b7c5...fffaabe8ea53 [03:20:14] [02miraheze/ssl] 07MacFan4000 03fffaabe - add thesimswiki.com redirect [03:20:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 76% [03:20:31] PROBLEM - cp25 HTTP 4xx/5xx ERROR Rate on cp25 is CRITICAL: CRITICAL - NGINX Error Rate is 83% [03:26:01] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/fffaabe8ea53...1bdf50e1fd84 [03:26:02] [02miraheze/ssl] 07MirahezeSSLBot 031bdf50e - Bot: Add SSL cert for rockrevolution.wiki [03:28:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 57% [03:30:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 77% [03:40:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 44% [03:42:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 78% [03:44:39] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 56% [03:46:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 54% [03:46:40] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 73% [03:48:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 71% [03:52:37] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 50% [03:54:17] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 58% [03:54:32] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 65% [03:57:24] RECOVERY - Host cloud11 is UP: PING OK - Packet loss = 0%, RTA = 1.16 ms [03:58:06] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 62% [03:59:17] RECOVERY - ping6 on cloud11 is OK: PING OK - Packet loss = 0%, RTA = 1.12 ms [04:07:59] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 56% [04:09:54] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 71% [04:15:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 52% [04:17:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 65% [04:21:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 54% [04:23:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 85% [04:41:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 53% [04:43:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 73% [04:51:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 51% [04:53:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 71% [04:58:31] PROBLEM - ping6 on cloud11 is CRITICAL: CRITICAL - Host Unreachable (2a10:6740::6:200) [04:59:59] RECOVERY - cloud11 SSH on cloud11 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u2 (protocol 2.0) [05:00:33] RECOVERY - ping6 on cloud11 is OK: PING OK - Packet loss = 0%, RTA = 0.77 ms [05:03:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 59% [05:05:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 64% [05:11:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 57% [05:17:18] RECOVERY - www.twistwoodtaleswiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'www.twistwoodtaleswiki.com' will expire on Wed 20 Dec 2023 22:08:34 GMT +0000. [05:17:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 66% [05:19:07] PROBLEM - db131 Disk Space on db131 is WARNING: DISK WARNING - free space: / 26128 MB (10% inode=96%); [05:21:00] PROBLEM - ping6 on cloud11 is CRITICAL: CRITICAL - Host Unreachable (2a10:6740::6:200) [05:21:39] PROBLEM - Host cloud11 is DOWN: CRITICAL - Host Unreachable (2a10:6740::6:200) [05:21:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 48% [05:23:40] RECOVERY - Host cloud11 is UP: PING OK - Packet loss = 0%, RTA = 0.43 ms [05:23:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 64% [05:25:09] RECOVERY - ping6 on cloud11 is OK: PING OK - Packet loss = 0%, RTA = 0.49 ms [05:27:08] PROBLEM - cp25 APT on cp25 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [05:29:08] PROBLEM - cp25 APT on cp25 is CRITICAL: APT CRITICAL: 76 packages available for upgrade (35 critical updates). [05:31:12] PROBLEM - ping6 on cloud11 is CRITICAL: CRITICAL - Host Unreachable (2a10:6740::6:200) [05:33:15] RECOVERY - ping6 on cloud11 is OK: PING OK - Packet loss = 0%, RTA = 1.53 ms [05:33:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 58% [05:35:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 56% [05:35:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 76% [05:37:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 70% [05:41:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 58% [05:43:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 74% [05:43:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 52% [05:45:31] PROBLEM - ping6 on cloud11 is CRITICAL: CRITICAL - Host Unreachable (2a10:6740::6:200) [05:45:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 72% [05:45:49] PROBLEM - Host cloud11 is DOWN: CRITICAL - Host Unreachable (2a10:6740::6:200) [05:47:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 58% [05:49:07] PROBLEM - www.twistwoodtaleswiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address www.twistwoodtaleswiki.com and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [05:49:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 75% [05:53:47] RECOVERY - Host cloud11 is UP: PING OK - Packet loss = 0%, RTA = 1.42 ms [05:54:04] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 11.34, 10.53, 9.74 [05:55:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 53% [05:55:44] RECOVERY - ping6 on cloud11 is OK: PING OK - Packet loss = 0%, RTA = 0.43 ms [05:57:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 78% [05:58:03] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 8.38, 9.66, 9.59 [05:59:05] RECOVERY - cloud11 Disk Space on cloud11 is OK: DISK OK - free space: / 22721 MB (85% inode=95%); [05:59:07] PROBLEM - cloud11 APT on cloud11 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [05:59:08] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [05:59:10] PROBLEM - cloud11 Puppet on cloud11 is UNKNOWN: NRPE: Unable to read output [05:59:11] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,2] - Reallocated_Sector_Ct is non-zero (31) --- [cciss,3] - Reallocated_Sector_Ct is non-zero (20) --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean --- [cciss,6] - Device is clean| [05:59:12] PROBLEM - cloud11 conntrack_table_size on cloud11 is UNKNOWN: NRPE: Unable to read output [05:59:12] PROBLEM - cloud11 ferm_active on cloud11 is UNKNOWN: NRPE: Unable to read output [05:59:16] RECOVERY - cloud11 Current Load on cloud11 is OK: OK - load average: 0.97, 0.57, 0.27 [05:59:16] RECOVERY - cloud11 NTP time on cloud11 is OK: NTP OK: Offset 0.001296013594 secs [06:00:54] RECOVERY - cloud11 PowerDNS Recursor on cloud11 is OK: DNS OK: 0.054 seconds response time. miraheze.org returns 2001:41d0:801:2000::3a18,2001:41d0:801:2000::5d68,51.195.201.140,51.89.139.24 [06:01:07] RECOVERY - cloud11 APT on cloud11 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [06:01:09] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:01:12] RECOVERY - cloud11 conntrack_table_size on cloud11 is OK: OK: nf_conntrack is 0 % full [06:01:12] RECOVERY - cloud11 ferm_active on cloud11 is OK: OK ferm input default policy is set [06:03:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 51% [06:03:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 53% [06:05:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 65% [06:05:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 74% [06:07:09] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: connect to address 2a10:6740::6:200 port 5666: No route to hostconnect to host 2a10:6740::6:200 port 5666: No route to host [06:08:02] PROBLEM - cloud11 SSH on cloud11 is CRITICAL: connect to address 2a10:6740::6:200 and port 22: No route to host [06:08:03] PROBLEM - ping6 on cloud11 is CRITICAL: CRITICAL - Host Unreachable (2a10:6740::6:200) [06:08:10] PROBLEM - Host cloud11 is DOWN: CRITICAL - Host Unreachable (2a10:6740::6:200) [06:09:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 48% [06:10:10] RECOVERY - Host cloud11 is UP: PING OK - Packet loss = 0%, RTA = 0.64 ms [06:11:11] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,2] - Reallocated_Sector_Ct is non-zero (31) --- [cciss,3] - Reallocated_Sector_Ct is non-zero (20) --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean --- [cciss,6] - Device is clean| [06:11:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 59% [06:11:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 76% [06:11:59] RECOVERY - cloud11 SSH on cloud11 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u2 (protocol 2.0) [06:12:03] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 10.65, 9.95, 9.58 [06:12:09] RECOVERY - ping6 on cloud11 is OK: PING OK - Packet loss = 0%, RTA = 0.46 ms [06:13:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 76% [06:14:03] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 9.01, 9.81, 9.59 [06:14:51] RECOVERY - ewgf.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'ewgf.wiki' will expire on Wed 20 Dec 2023 22:59:05 GMT +0000. [06:19:00] RECOVERY - www.twistwoodtaleswiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'www.twistwoodtaleswiki.com' will expire on Wed 20 Dec 2023 22:08:34 GMT +0000. [06:21:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 47% [06:21:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 54% [06:23:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 74% [06:25:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 68% [06:27:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 59% [06:29:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 83% [06:37:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 59% [06:39:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 73% [06:45:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 48% [06:46:30] PROBLEM - ewgf.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address ewgf.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [06:47:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 73% [06:49:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 51% [06:50:03] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 10.86, 10.13, 9.64 [06:50:49] PROBLEM - www.twistwoodtaleswiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address www.twistwoodtaleswiki.com and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [06:51:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 76% [06:54:03] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 10.13, 10.19, 9.79 [06:55:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 56% [06:59:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 76% [07:05:39] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [07:08:58] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 10.94, 10.54, 10.12 [07:09:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 58% [07:10:54] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 12.28, 11.09, 10.37 [07:11:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 68% [07:12:51] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 10.39, 10.77, 10.34 [07:15:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 57% [07:17:09] PROBLEM - cp25 Current Load on cp25 is WARNING: LOAD WARNING - total load average: 5.77, 7.26, 3.28 [07:17:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 92% [07:18:40] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 12.13, 11.13, 10.58 [07:19:08] RECOVERY - cp25 Current Load on cp25 is OK: LOAD OK - total load average: 0.94, 4.89, 2.89 [07:20:37] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 11.10, 11.16, 10.66 [07:23:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 59% [07:25:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 65% [07:27:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 41% [07:29:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 71% [07:31:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 52% [07:32:15] PROBLEM - mw142 Current Load on mw142 is WARNING: WARNING - load average: 11.96, 9.24, 5.30 [07:33:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 73% [07:34:15] RECOVERY - mw142 Current Load on mw142 is OK: OK - load average: 3.61, 6.92, 4.93 [07:35:28] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [07:36:08] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 9.13, 9.55, 10.06 [07:39:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 58% [07:41:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 61% [07:45:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 57% [07:47:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 71% [07:49:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 59% [07:51:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 68% [07:56:06] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 14.51, 11.91, 10.28 [07:57:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 42% [07:58:03] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 8.93, 10.76, 10.06 [07:59:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 58% [07:59:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 70% [08:02:03] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 9.27, 9.96, 9.90 [08:03:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 65% [08:03:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 52% [08:05:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 76% [08:11:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 53% [08:13:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 62% [08:18:29] PROBLEM - cp24 Current Load on cp24 is WARNING: LOAD WARNING - total load average: 2.17, 7.36, 4.17 [08:20:29] RECOVERY - cp24 Current Load on cp24 is OK: LOAD OK - total load average: 0.60, 5.05, 3.71 [08:23:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 49% [08:25:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 67% [08:25:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 45% [08:27:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 72% [08:31:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 48% [08:33:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 63% [08:37:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 59% [08:39:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 68% [08:45:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 59% [08:47:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 77% [08:47:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 59% [08:49:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 73% [08:53:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 58% [08:55:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 72% [09:01:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 58% [09:05:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 69% [09:05:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 54% [09:07:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 55% [09:07:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 76% [09:09:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 70% [09:15:43] RECOVERY - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is OK: OK - NGINX Error Rate is 36% [09:19:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 62% [09:23:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 51% [09:27:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 41% [09:29:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 64% [09:31:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 62% [09:35:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 47% [09:35:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 56% [09:37:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 60% [09:37:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 68% [09:43:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 57% [09:45:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 72% [09:46:17] RECOVERY - ewgf.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'ewgf.wiki' will expire on Wed 20 Dec 2023 22:59:05 GMT +0000. [09:47:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 50% [09:47:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 57% [09:49:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 64% [09:51:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 74% [10:01:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 57% [10:01:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 54% [10:03:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is CRITICAL: CRITICAL - NGINX Error Rate is 81% [10:03:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 61% [10:09:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is WARNING: WARNING - NGINX Error Rate is 53% [10:13:43] PROBLEM - cp35 HTTP 4xx/5xx ERROR Rate on cp35 is CRITICAL: CRITICAL - NGINX Error Rate is 65% [10:17:31] PROBLEM - cp34 HTTP 4xx/5xx ERROR Rate on cp34 is WARNING: WARNING - NGINX Error Rate is 53% [10:18:15] PROBLEM - ewgf.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address ewgf.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [10:19:56]