[00:00:39] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.22, 1.66, 1.75 [00:01:25] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.44, 3.15, 2.95 [00:01:50] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.85, 2.06, 1.38 [00:02:31] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.91, 3.36, 2.96 [00:02:38] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.42, 1.54, 1.70 [00:03:21] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.01, 3.91, 3.23 [00:03:46] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.53, 1.84, 1.38 [00:03:52] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.34, 4.13, 3.36 [00:04:31] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.72, 4.00, 3.25 [00:05:42] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.06, 1.57, 1.33 [00:08:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.82, 3.95, 3.41 [00:09:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.89, 3.66, 3.41 [00:09:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:11:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.92, 4.45, 3.73 [00:11:34] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [00:12:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.05, 4.20, 3.62 [00:14:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.20, 3.92, 3.59 [00:14:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:15:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.58, 3.89, 3.71 [00:16:26] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.53, 3.30, 3.40 [00:17:48] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.48, 3.74, 3.89 [00:20:30] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.13, 1.93, 1.72 [00:22:29] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.51, 1.76, 1.68 [00:23:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.66, 4.01, 3.71 [00:24:22] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.14, 3.56, 3.44 [00:24:28] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.49, 1.64, 1.64 [00:25:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.44, 3.48, 3.56 [00:28:20] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.91, 3.26, 3.35 [00:29:08] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.66, 2.76, 3.26 [00:35:44] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.35, 2.65, 3.27 [00:41:28] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [00:41:28] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [00:43:08] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.51, 1.73, 1.28 [00:45:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.59, 3.30, 3.07 [00:45:08] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.66, 1.76, 1.35 [00:47:08] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.19, 2.90, 2.95 [00:47:08] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.31, 1.63, 1.35 [00:48:18] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 2.00, 1.73, 1.58 [00:50:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.29, 1.93, 1.68 [00:52:17] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.69, 1.83, 1.67 [00:52:42] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.03, 9.92, 8.63 [00:53:46] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.53, 9.17, 7.44 [00:53:56] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.63, 6.29, 4.72 [00:54:07] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.60, 10.77, 8.67 [00:55:36] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:55:46] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.05, 9.16, 7.66 [00:55:56] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 9.17, 7.27, 5.27 [00:56:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:58:18] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.67, 11.04, 8.78 [00:58:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.97, 11.92, 9.99 [00:59:45] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 22334 bytes in 1.313 second response time [00:59:56] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.26, 7.82, 5.99 [01:01:56] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.58, 6.72, 5.82 [01:04:07] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.40, 11.24, 10.29 [01:04:12] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.37, 1.62, 1.67 [01:04:18] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.29, 9.82, 8.98 [01:04:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.49, 9.89, 9.75 [01:05:36] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.85, 4.21, 3.41 [01:06:07] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.29, 9.54, 9.79 [01:09:08] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.55, 1.91, 1.52 [01:09:35] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.44, 3.52, 3.33 [01:11:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:11:08] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.27, 2.50, 1.78 [01:11:34] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.08, 3.05, 3.18 [01:13:08] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.68, 1.80, 1.61 [01:15:08] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.51, 1.42, 1.49 [01:19:09] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.49, 2.11, 1.83 [01:21:08] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.69, 2.77, 2.03 [01:23:07] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.42, 1.86, 1.81 [01:25:08] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.93, 1.78, 1.79 [01:27:08] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.56, 1.39, 1.65 [01:27:35] [discord] MacFan4000: fixed, needed to update URL [01:28:29] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.33, 3.85, 3.41 [01:28:41] @Orduin can we move the mhbots SAL to #fossbots? [01:28:42] [discord] *should be fixed, I'll point the channel properly in a bit [01:28:46] Ok [01:30:29] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.67, 3.63, 3.37 [01:31:04] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.57, 1.59, 1.70 [01:40:26] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.53, 3.15, 3.33 [01:41:17] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [01:48:56] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.65, 2.02, 1.76 [01:49:08] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.36, 1.86, 1.48 [01:49:23] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.56, 3.88, 3.49 [01:50:55] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.82, 2.00, 1.80 [01:51:08] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.18, 1.65, 1.44 [01:53:22] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.40, 3.52, 3.43 [01:54:54] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.37, 1.64, 1.69 [01:55:22] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.92, 3.21, 3.31 [02:07:18] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.90, 3.50, 3.24 [02:09:17] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.27, 3.56, 3.28 [02:11:17] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.10, 3.38, 3.24 [02:17:15] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.11, 3.53, 3.34 [02:18:43] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.43, 1.87, 1.63 [02:19:15] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.16, 3.27, 3.25 [02:22:42] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.54, 1.96, 1.75 [02:28:39] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.37, 1.60, 1.65 [02:37:22] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.61, 3.58, 2.91 [02:39:09] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.63, 4.02, 3.38 [02:41:08] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.96, 3.85, 3.39 [02:41:20] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.70, 3.19, 2.92 [02:43:08] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.49, 3.29, 3.23 [02:50:29] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.34, 2.02, 1.76 [02:52:28] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.35, 1.78, 1.70 [02:54:27] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.43, 1.68, 1.68 [03:00:25] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.96, 1.78, 1.72 [03:06:22] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.51, 1.68, 1.70 [03:18:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.08, 1.75, 1.67 [03:22:15] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.44, 1.73, 1.70 [03:24:14] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.31, 1.61, 1.66 [03:46:42] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [03:46:42] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [03:50:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.80, 2.07, 1.70 [03:52:11] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.45, 1.81, 1.65 [03:54:11] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.19, 1.60, 1.59 [03:56:47] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.10, 3.70, 3.09 [03:58:46] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.57, 3.60, 3.13 [04:02:45] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.02, 2.90, 2.97 [04:16:37] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [04:19:05] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.18, 1.74, 1.56 [04:21:04] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.89, 1.81, 1.61 [04:25:02] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.53, 1.66, 1.60 [04:46:53] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.68, 1.95, 1.68 [04:52:50] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.69, 1.97, 1.81 [04:58:48] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.64, 1.62, 1.70 [05:02:46] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.98, 1.82, 1.76 [05:06:44] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.19, 2.03, 1.86 [05:08:43] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.90, 1.98, 1.87 [05:10:42] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.05, 2.02, 1.90 [05:26:26] PROBLEM - wiki.digitalcandela.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.digitalcandela.com' expires in 15 day(s) (Sun 22 May 2022 04:59:57 GMT +0000). [05:27:17] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/b52b2d6c28c4...3d96cbbbd362 [05:27:17] [url] Comparing b52b2d6c28c4...3d96cbbbd362 · miraheze/ssl · GitHub | github.com [05:27:18] [02miraheze/ssl] 07MirahezeSSLBot 033d96cbb - Bot: Update SSL cert for wiki.digitalcandela.com [05:28:35] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.67, 1.84, 2.00 [05:35:20] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.93, 3.41, 2.88 [05:35:25] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.99, 3.17, 2.61 [05:37:19] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.04, 3.40, 2.95 [05:39:56] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.78, 7.00, 6.23 [05:41:11] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.41, 2.69, 2.63 [05:41:18] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.50, 3.49, 3.13 [05:41:56] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 10.17, 8.22, 6.77 [05:42:18] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 13.72, 9.91, 7.16 [05:42:30] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.30, 1.41, 1.65 [05:43:17] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.56, 3.23, 3.07 [05:44:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:44:07] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.56, 10.17, 7.55 [05:44:18] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.47, 9.87, 7.47 [05:44:42] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.21, 10.64, 8.51 [05:46:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.30, 9.81, 8.47 [05:48:07] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.69, 9.41, 7.95 [05:48:18] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 5.83, 8.62, 7.60 [05:48:27] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.44, 1.94, 1.79 [05:49:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:49:55] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 4.85, 7.55, 7.42 [05:55:55] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.50, 5.95, 6.72 [05:56:05] RECOVERY - wiki.digitalcandela.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.digitalcandela.com' will expire on Thu 04 Aug 2022 04:27:11 GMT +0000. [05:58:23] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.53, 1.90, 1.93 [06:06:20] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.94, 2.04, 1.92 [06:15:08] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.66, 3.96, 3.13 [06:17:07] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.39, 3.30, 2.98 [06:22:01] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [06:33:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.82, 3.54, 3.20 [06:35:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.45, 3.73, 3.29 [06:36:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.70, 3.70, 3.23 [06:38:30] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.93, 4.59, 3.64 [06:42:21] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.75, 3.73, 3.50 [06:48:07] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.71, 2.83, 3.21 [06:48:59] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.28, 3.71, 3.76 [06:51:56] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [06:52:58] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.19, 2.87, 3.40 [07:42:27] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.30, 3.48, 2.79 [07:44:22] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.65, 3.09, 2.72 [07:47:42] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.10, 4.03, 3.22 [07:49:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.23, 3.64, 3.18 [07:51:42] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.52, 3.12, 3.04 [07:55:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.69, 3.61, 3.29 [08:01:42] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.10, 3.05, 3.20 [08:11:08] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.85, 3.45, 1.80 [08:15:08] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.45, 1.83, 1.51 [08:17:08] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.20, 1.25, 1.33 [08:50:00] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.80, 1.60, 1.19 [08:52:00] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.80, 1.34, 1.14 [09:17:55] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 9.58, 6.72, 5.81 [09:20:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.46, 9.17, 7.38 [09:21:46] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.17, 9.77, 6.81 [09:22:07] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.14, 9.76, 6.85 [09:22:18] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.92, 8.97, 6.51 [09:22:42] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.11, 10.10, 7.94 [09:23:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:23:25] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [09:23:25] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [09:24:18] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 13.32, 10.45, 7.34 [09:24:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.43, 10.22, 8.25 [09:25:46] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.79, 10.49, 7.81 [09:26:18] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.12, 10.07, 7.60 [09:26:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.41, 9.65, 8.27 [09:27:46] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.12, 9.86, 7.90 [09:28:07] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.67, 11.36, 8.60 [09:30:07] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.06, 9.25, 8.15 [09:31:55] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 4.77, 7.27, 7.51 [09:33:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:35:56] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.03, 5.41, 6.69 [10:23:14] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [10:46:11] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.42, 1.51, 1.99 [10:48:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.41, 1.81, 2.04 [10:56:11] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.37, 1.78, 1.99 [11:18:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.61, 1.96, 1.85 [11:26:11] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.46, 1.83, 1.89 [11:31:48] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.49, 3.04, 2.67 [11:32:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.10, 1.77, 1.83 [11:33:48] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.25, 2.94, 2.66 [11:34:11] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.80, 1.76, 1.82 [11:40:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.29, 1.93, 1.86 [11:42:11] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 2.00, 1.98, 1.90 [11:44:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.60, 2.15, 1.97 [11:46:44] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.27, 3.59, 3.13 [11:48:44] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.49, 3.10, 3.00 [12:06:11] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.74, 1.87, 2.00 [12:07:56] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.05, 6.56, 5.90 [12:09:56] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.89, 6.29, 5.88 [12:14:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.01, 1.72, 1.85 [12:21:56] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.64, 7.16, 6.54 [12:27:56] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.28, 6.74, 6.55 [12:31:56] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.14, 6.89, 6.63 [12:32:11] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.69, 1.82, 1.97 [12:34:42] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 14.12, 10.62, 8.38 [12:35:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:36:37] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:36:43] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:37:24] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:37:56] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.58, 6.47, 6.59 [12:39:32] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 22334 bytes in 8.836 second response time [12:40:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:40:30] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.99, 3.81, 3.14 [12:42:29] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.77, 3.41, 3.08 [12:42:46] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14640 bytes in 0.017 second response time [12:42:52] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14634 bytes in 0.146 second response time [12:44:29] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.10, 3.50, 3.13 [12:44:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.45, 11.47, 10.28 [12:46:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.37, 1.92, 1.88 [12:48:28] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.87, 3.31, 3.13 [12:48:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.06, 9.66, 9.84 [13:00:24] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.40, 3.48, 3.20 [13:02:23] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.97, 3.35, 3.19 [13:09:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:11:20] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.35, 3.62, 3.32 [13:13:19] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.52, 3.12, 3.17 [13:14:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:22:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.03, 1.98, 1.20 [13:23:07] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [13:24:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.79, 1.58, 1.15 [13:45:11] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.89, 3.84, 3.19 [13:46:47] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.83, 3.58, 2.93 [13:47:10] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.11, 3.57, 3.16 [13:48:46] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.72, 3.20, 2.87 [13:49:10] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.82, 3.23, 3.08 [13:51:06] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 57 seconds ago with 0 failures [13:54:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:56:42] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.64, 3.86, 3.20 [13:57:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.28, 3.93, 3.46 [13:57:39] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.30, 3.42, 3.00 [13:58:25] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [13:58:25] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [13:58:41] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.83, 3.82, 3.26 [13:59:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.16, 3.95, 3.51 [14:01:06] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.00, 3.67, 3.47 [14:01:37] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.41, 7.09, 6.40 [14:03:33] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.41, 6.89, 6.41 [14:04:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:04:39] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.72, 3.22, 3.17 [14:05:05] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.66, 3.30, 3.37 [14:05:21] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.45, 3.67, 3.26 [14:07:16] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.83, 3.56, 3.26 [14:07:23] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.33, 7.36, 6.69 [14:09:12] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.76, 3.30, 3.20 [14:12:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:13:10] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.74, 7.78, 7.15 [14:17:01] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.62, 6.59, 6.80 [14:17:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.11, 3.94, 3.59 [14:27:09] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.90, 10.26, 8.72 [14:27:31] alerting : [FIRING:2] (mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:28:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.15, 3.65, 3.22 [14:29:06] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.42, 9.72, 8.72 [14:30:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.52, 3.52, 3.22 [14:32:26] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.22, 4.01, 3.46 [14:34:25] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.83, 3.45, 3.31 [14:34:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.75, 10.58, 9.84 [14:37:46] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.17, 4.02, 3.44 [14:38:23] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.59, 4.59, 3.80 [14:38:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.51, 9.82, 9.73 [14:39:42] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.42, 3.78, 3.43 [14:41:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.09, 3.81, 3.47 [14:42:31] alerting : [FIRING:1] (mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:43:33] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.75, 3.80, 3.51 [14:44:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.81, 3.71, 3.70 [14:45:28] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.21, 3.79, 3.53 [14:46:20] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.15, 3.90, 3.76 [14:47:24] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.17, 3.73, 3.55 [14:48:19] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.10, 3.62, 3.68 [14:51:56] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.15, 6.12, 5.85 [14:53:56] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.41, 6.04, 5.85 [14:54:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.94, 10.68, 9.95 [14:56:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.36, 10.08, 9.82 [14:57:31] alerting : [FIRING:1] (mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:00:14] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.44, 3.70, 3.65 [15:02:13] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.99, 3.48, 3.58 [15:02:50] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.51, 3.45, 3.91 [15:03:08] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.64, 3.04, 3.35 [15:05:29] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.50, 11.92, 10.74 [15:06:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.03, 3.25, 3.43 [15:07:31] alerting : [FIRING:2] (mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:08:11] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.12, 3.24, 3.40 [15:10:48] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.11, 3.65, 3.77 [15:11:10] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.12, 11.60, 11.02 [15:12:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.99, 3.32, 3.41 [15:12:48] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.70, 3.56, 3.72 [15:14:11] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.92, 3.07, 3.30 [15:17:31] alerting : [FIRING:1] (mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:17:56] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.39, 6.90, 6.36 [15:20:42] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.75, 10.88, 10.79 [15:22:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.45, 10.87, 10.82 [15:23:34] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.49, 1.63, 1.24 [15:23:56] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.52, 6.73, 6.47 [15:24:45] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.89, 3.75, 3.60 [15:25:30] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.79, 1.23, 1.14 [15:26:11] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.31, 3.98, 3.50 [15:26:29] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.44, 3.69, 3.44 [15:26:45] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.51, 3.73, 3.62 [15:27:55] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.66, 7.10, 6.71 [15:28:09] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [15:28:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.21, 3.85, 3.52 [15:28:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.31, 9.37, 10.14 [15:30:11] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.23, 4.05, 3.64 [15:30:20] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.83, 3.83, 3.53 [15:30:44] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.81, 4.24, 3.79 [15:32:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.00, 3.75, 3.59 [15:32:31] alerting : [FIRING:2] (mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:33:56] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.77, 7.79, 7.09 [15:35:55] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.86, 7.70, 7.14 [15:36:11] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.42, 4.30, 3.87 [15:37:31] alerting : [FIRING:1] (mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:38:02] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.62, 3.78, 3.65 [15:38:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.75, 3.72, 3.70 [15:42:11] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.99, 4.02, 3.80 [15:42:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.02, 3.96, 3.97 [15:44:28] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.16, 1.59, 1.33 [15:44:37] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.25, 1.54, 1.14 [15:44:40] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.20, 4.44, 4.14 [15:46:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.82, 3.70, 3.74 [15:46:27] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.33, 1.45, 1.31 [15:47:31] ok : [RESOLVED] (!sre High Job Queue Backlog yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:48:37] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.01, 1.41, 1.18 [15:51:09] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.26, 11.03, 10.23 [15:51:30] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.55, 2.69, 3.21 [15:53:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:54:57] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.27, 11.35, 10.59 [15:58:11] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.94, 3.00, 3.35 [15:58:37] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.15, 3.62, 3.99 [15:58:44] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.29, 11.35, 10.72 [16:00:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.69, 11.16, 10.71 [16:03:47] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.36, 10.21, 9.68 [16:05:42] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.93, 9.51, 9.47 [16:06:35] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.21, 3.29, 3.56 [16:07:54] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.20, 3.59, 3.36 [16:08:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:09:50] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.85, 3.30, 3.28 [16:09:56] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.83, 6.30, 6.67 [16:10:17] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.73, 1.74, 1.47 [16:10:34] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.64, 3.27, 3.52 [16:12:17] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.12, 1.42, 1.37 [16:14:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.65, 8.89, 9.87 [16:16:33] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.40, 3.05, 3.36 [16:16:40] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/commit/30e7d4f4d616 [16:16:41] [02miraheze/CreateWiki] 07Universal-Omega 0330e7d4f - T9147: Create "on hold" status for wiki requests [16:16:43] [02CreateWiki] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/CreateWiki [16:16:43] [url] Page not found · GitHub · GitHub | github.com [16:16:44] [02CreateWiki] 07Universal-Omega opened pull request 03#320: T9147: Create "on hold" status for wiki requests - 13https://github.com/miraheze/CreateWiki/pull/320 [16:16:45] ... [16:20:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.22, 9.80, 9.94 [16:23:08] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.28, 1.48, 1.03 [16:23:20] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.20, 3.33, 3.13 [16:24:30] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.82, 3.53, 3.45 [16:24:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:24:37] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.25, 1.63, 1.17 [16:24:52] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.91, 9.72, 9.16 [16:25:08] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.93, 1.24, 1.00 [16:25:15] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.27, 2.95, 3.01 [16:26:30] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.21, 3.12, 3.31 [16:26:37] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.16, 1.40, 1.14 [16:26:42] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.03, 10.91, 10.31 [16:26:46] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.31, 9.68, 9.22 [16:26:53] miraheze/CreateWiki - Universal-Omega the build passed. [16:28:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.70, 10.27, 10.17 [16:28:53] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.23, 10.10, 8.86 [16:29:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.53, 4.04, 3.42 [16:30:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.03, 9.80, 10.00 [16:30:51] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.17, 9.25, 8.72 [16:31:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.80, 3.47, 3.28 [16:33:08] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.01, 3.01, 3.14 [16:33:30] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.54, 10.79, 9.86 [16:34:26] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.69, 3.85, 3.55 [16:34:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.01, 10.38, 10.18 [16:35:25] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.59, 10.11, 9.73 [16:38:25] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.16, 4.00, 3.73 [16:38:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.65, 10.15, 10.16 [16:40:25] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.19, 4.14, 3.81 [16:42:11] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.60, 1.70, 1.99 [16:44:52] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.65, 3.43, 3.19 [16:46:51] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.03, 3.23, 3.14 [16:48:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.50, 2.00, 2.01 [16:48:23] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.41, 3.71, 3.86 [16:49:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:56:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:56:22] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.11, 4.67, 4.02 [16:57:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.13, 3.71, 3.27 [16:58:03] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 3.01, 1.98, 1.48 [16:58:45] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 7.38, 5.53, 3.99 [16:59:19] [02mw-config] 07ugochimobi opened pull request 03#4669: set wgInterwikiSortingSort to alphabetically on gratispaideiawiki - 13https://github.com/miraheze/mw-config/pull/4669 [16:59:19] [url] Page not found · GitHub · GitHub | github.com [17:00:02] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.36, 1.70, 1.43 [17:00:36] miraheze/mw-config - ugochimobi the build passed. [17:00:46] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [17:00:47] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [17:01:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [17:02:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.48, 10.08, 9.32 [17:04:11] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.81, 1.92, 2.00 [17:06:07] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.84, 8.69, 8.94 [17:08:30] RhinosF1: please can you run rebuildLinks.php for gratispaideiawiki? [17:10:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.34, 2.09, 2.02 [17:14:11] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.88, 1.96, 1.98 [17:16:07] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.84, 10.02, 9.35 [17:16:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.03, 2.02, 2.00 [17:17:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [17:17:16] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.02, 7.22, 6.80 [17:18:07] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.32, 9.75, 9.33 [17:18:25] ugochimobi: 40 minutes [17:18:57] RhinosF1: whym 40 minutes? [17:19:05] Because I'm mobile [17:19:09] s/whym/wdym [17:19:09] ugochimobi meant to say: RhinosF1: wdym 40 minutes? [17:19:23] RhinosF1: Okay, no problem [17:21:07] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.85, 7.28, 6.94 [17:22:07] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.13, 10.42, 9.64 [17:24:07] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.64, 10.26, 9.67 [17:24:53] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.95, 10.63, 10.08 [17:26:07] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.17, 9.95, 9.64 [17:26:11] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.58, 1.90, 2.00 [17:26:53] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 9.50, 7.90, 7.26 [17:27:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [17:30:44] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.67, 7.56, 7.28 [17:32:07] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.94, 11.54, 10.36 [17:32:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.39, 9.28, 9.77 [17:34:07] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.25, 11.17, 10.38 [17:34:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.01, 1.81, 1.90 [17:35:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [17:36:07] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.42, 10.08, 10.09 [17:36:11] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.81, 1.81, 1.89 [17:36:30] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.66, 7.92, 7.49 [17:36:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.82, 10.47, 10.12 [17:36:46] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.56, 1.71, 1.45 [17:38:25] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.46, 7.76, 7.48 [17:38:46] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.95, 1.50, 1.41 [17:40:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.33, 1.94, 1.92 [17:40:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.09, 9.88, 9.97 [17:42:16] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.09, 7.55, 7.44 [17:44:07] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.63, 10.51, 10.28 [17:44:12] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.10, 7.32, 7.36 [17:45:04] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [17:46:07] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.01, 9.84, 10.07 [17:59:55] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.42, 5.76, 6.54 [18:00:36] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [18:01:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.84, 3.73, 3.95 [18:02:20] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.16, 3.18, 3.87 [18:04:19] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.89, 3.97, 4.08 [18:05:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.42, 3.91, 3.94 [18:23:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.11, 3.55, 3.97 [18:26:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [18:27:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.05, 3.87, 3.98 [18:30:31] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [18:30:31] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [18:31:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.24, 3.86, 3.98 [18:34:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.86, 3.35, 3.85 [18:36:11] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.02, 3.39, 3.79 [18:36:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.38, 10.39, 9.48 [18:37:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.75, 3.68, 3.73 [18:43:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.26, 3.17, 3.52 [18:44:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.42, 3.95, 3.98 [18:44:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.34, 9.97, 9.70 [18:47:08] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.22, 2.82, 3.31 [18:49:25] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/30e7d4f4d616...280385459e6c [18:49:26] [url] Comparing 30e7d4f4d616...280385459e6c · miraheze/CreateWiki · GitHub | github.com [18:49:27] [02miraheze/CreateWiki] 07Universal-Omega 032803854 - Update en.json [18:49:28] [02CreateWiki] 07Universal-Omega synchronize pull request 03#320: T9147: Create "on hold" status for wiki requests - 13https://github.com/miraheze/CreateWiki/pull/320 [18:49:29] [url] Page not found · GitHub · GitHub | github.com [18:50:11] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.66, 4.45, 4.17 [18:51:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.78, 4.11, 3.74 [18:52:10] RhinosF1: 🦏 [18:55:08] [02mw-config] 07Universal-Omega reviewed pull request 03#4669 commit - 13https://github.com/miraheze/mw-config/pull/4669#discussion_r867140930 [18:55:09] [url] set wgInterwikiSortingSort to alphabetically on gratispaideiawiki by ugochimobi · Pull Request #4669 · miraheze/mw-config · GitHub | github.com [18:57:10] ugochimobi: I forgot [18:57:13] Will do later [18:59:24] miraheze/CreateWiki - Universal-Omega the build passed. [19:00:25] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [19:04:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.84, 1.46, 1.06 [19:05:55] RhinosF1: okay [19:06:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.18, 1.37, 1.07 [19:07:15] [02mw-config] 07ugochimobi synchronize pull request 03#4669: set wgInterwikiSortingSort to alphabetically on gratispaideiawiki - 13https://github.com/miraheze/mw-config/pull/4669 [19:07:15] [url] Page not found · GitHub · GitHub | github.com [19:08:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 1.92, 3.30, 3.93 [19:08:22] miraheze/mw-config - ugochimobi the build passed. [19:09:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.23, 3.25, 3.79 [19:10:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.52, 9.80, 9.36 [19:12:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 10.04, 9.67, 9.35 [19:13:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.22, 3.77, 3.84 [19:14:11] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.29, 3.81, 3.92 [19:15:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.40, 3.66, 3.80 [19:16:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:17:55] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.41, 6.84, 6.37 [19:19:56] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.43, 7.41, 6.64 [19:22:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.40, 3.51, 3.87 [19:23:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.79, 3.41, 3.56 [19:23:55] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.06, 7.49, 6.89 [19:25:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.11, 3.42, 3.55 [19:25:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.49, 3.35, 3.91 [19:27:49] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.61, 4.35, 4.22 [19:27:56] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.54, 7.79, 7.12 [19:28:11] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.48, 4.53, 4.10 [19:29:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:29:56] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 5.83, 7.06, 6.94 [19:31:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.01, 4.04, 3.77 [19:31:56] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.84, 6.55, 6.75 [19:32:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.94, 3.93, 3.95 [19:33:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 1.65, 3.12, 3.47 [19:33:47] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.48, 3.57, 3.97 [19:34:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:34:20] apart from Special:DataDump is there any other way to get dumps? [19:34:40] like a URL? or something? that can generate it [19:35:01] especially in a format except xml [19:35:47] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.34, 4.12, 4.13 [19:36:11] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.80, 4.33, 4.03 [19:36:31] ugochimobi: api? [19:36:45] I was thinking of that [19:36:54] I've never tried it though [19:39:46] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.64, 3.70, 3.99 [19:41:45] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.76, 3.99, 4.05 [19:45:44] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.49, 3.80, 3.95 [19:47:08] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.90, 3.17, 3.39 [19:49:43] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.15, 4.16, 4.04 [19:51:08] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.00, 1.81, 1.22 [19:51:43] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.32, 3.81, 3.92 [19:55:08] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.54, 1.79, 1.36 [19:55:42] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.80, 3.59, 3.76 [19:57:08] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.37, 1.96, 1.48 [19:57:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.75, 3.55, 3.73 [19:59:08] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.45, 1.67, 1.43 [20:02:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [20:03:42] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.49, 2.95, 3.40 [20:06:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.26, 9.86, 9.27 [20:08:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.84, 9.10, 9.06 [20:09:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.45, 3.54, 3.52 [20:11:42] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.62, 4.02, 3.70 [20:13:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.23, 3.91, 3.71 [20:15:42] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.11, 3.89, 3.72 [20:16:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.31, 1.74, 1.29 [20:17:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [20:17:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.74, 3.48, 3.59 [20:18:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.57, 1.65, 1.30 [20:20:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.44, 3.21, 3.98 [20:20:56] [02puppet] 07Universal-Omega opened pull request 03#2529: Raise `request_terminate_timeout` to 60 instead of 59 - 13https://github.com/miraheze/puppet/pull/2529 [20:20:57] [url] Page not found · GitHub · GitHub | github.com [20:21:42] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.83, 2.84, 3.33 [20:22:11] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.55, 3.44, 3.95 [20:22:38] paladox: can you merge CosmicAlpha's PR? [20:22:42] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.10, 9.48, 8.81 [20:22:53] [02puppet] 07paladox closed pull request 03#2529: Raise `request_terminate_timeout` to 60 instead of 59 - 13https://github.com/miraheze/puppet/pull/2529 [20:22:54] [url] Page not found · GitHub · GitHub | github.com [20:22:55] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/32460f84687c...d5a85b6adafa [20:22:55] [url] Comparing 32460f84687c...d5a85b6adafa · miraheze/puppet · GitHub | github.com [20:22:56] [02miraheze/puppet] 07Universal-Omega 03d5a85b6 - Raise `request_terminate_timeout` to 60 instead of 59 (#2529) [20:23:04] RhinosF1: how would the request syntax look like? I'm talking about the dump one via API [20:23:16] ugochimobi: look at the docs [20:23:19] Thanks! [20:23:51] the point I'm missing is the &type= param [20:24:00] I can find any type [20:24:03] https://meta.miraheze.org/w/api.php?action=help&modules=viewdumps & https://meta.miraheze.org/w/api.php?action=help&modules=generatedumps [20:24:04] [url] MediaWiki API help - Miraheze Meta | meta.miraheze.org [20:24:06] or what to use there [20:24:06] [url] MediaWiki API help - Miraheze Meta | meta.miraheze.org [20:24:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.26, 3.45, 3.90 [20:24:34] xml,image,managewiki id assume ugochimobi are the options [20:24:36] CosmicAlpha: ^ [20:24:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.85, 9.47, 8.89 [20:24:49] Yep, I think that's right. [20:25:00] looking there `⧼apihelp-generatedumps-param-type⧽` is missing [20:25:06] why though [20:25:14] is that supposed to be i18n? [20:25:29] yes, that would be an i18n message [20:25:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.66, 3.34, 3.47 [20:26:01] but it isn't available anywhere [20:26:08] eh? [20:26:15] ManageWiki is managewiki_backup though. [20:26:24] https://meta.miraheze.org/w/api.php?action=viewdumps&type=xml works fine for me [20:26:24] [url] MediaWiki API result - Miraheze Meta | meta.miraheze.org [20:26:55] hmm [20:26:59] CosmicAlpha: can you fix the broken i18n [20:27:18] ugochimobi: the generate one needs posting and you to get a csrf token [20:27:23] But it should work fine [20:27:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [20:27:45] yes I know of the csrf token [20:27:54] but it still doesn't [20:28:02] let me try again though [20:28:53] RhinosF1: when I created the generate/delete dumps APIs, I guess I forgot the help i18n for type, so it was never added, I'll do so in a bit. Sorry. [20:29:42] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.29, 3.20, 3.37 [20:29:43] no need to sorry [20:29:48] ugochimobi: meta api works fine [20:29:55] I just did it in sandbox [20:32:05] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [20:32:11] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.09, 3.75, 3.71 [20:32:13] erm, sanbox actually works for me [20:32:25] but generating it actually doesn't [20:32:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [20:32:35] s/sanbox/sandbox [20:32:35] ugochimobi meant to say: erm, sandbox actually works for me [20:33:35] ugochimobi: what do you mean generating it actually doesn't? [20:33:37] Where? [20:33:59] [02DataDump] 07Universal-Omega edited pull request 03#49: Make addModuleStyles an array - 13https://github.com/miraheze/DataDump/pull/49 [20:34:00] [url] Page not found · GitHub · GitHub | github.com [20:34:08] [02DataDump] 07Universal-Omega closed pull request 03#49: Make addModuleStyles an array - 13https://github.com/miraheze/DataDump/pull/49 [20:34:09] [url] Page not found · GitHub · GitHub | github.com [20:34:10] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/4b61f0e5be98...dca290fb1555 [20:34:11] [url] Comparing 4b61f0e5be98...dca290fb1555 · miraheze/DataDump · GitHub | github.com [20:34:11] [02miraheze/DataDump] 07Universal-Omega 03dca290f - Make addModuleStyles an array (#49) [20:34:13] [02miraheze/DataDump] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [20:34:14] [02DataDump] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/DataDump [20:34:15] [url] Page not found · GitHub · GitHub | github.com [20:34:26] RhinosF1: via the api.php?generategump URL [20:35:15] ugochimobi: how [20:35:22] That's not valid anyway [20:35:58] https://meta.miraheze.org/w/api.php?action=deletedumps&format=json&type=xml&filename=test-blah-bug&token= [20:36:37] https://meta.miraheze.org/w/api.php?action=generatedumps&format=json&type=xml&token= [20:38:19] miraheze/DataDump - Universal-Omega the build passed. [20:38:45] hmm [20:39:56] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.35, 7.32, 5.95 [20:41:56] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.40, 6.57, 5.84 [20:42:27] {"error":{"code":"mustpostparams","info":"The following parameter was found in the query string, but must be in the POST body: token.","*":"See https://gratisdata.miraheze.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/> for notice of API deprecations and breaking changes."},"servedby":"mw102"} [20:42:29] [url] MediaWiki API help - Gratisdata | gratisdata.miraheze.org [20:42:30] [url] lists.wikimedia.org | lists.wikimedia.org [20:43:21] RhinosF1: see this now, it returns it on running https://gratisdata.miraheze.org/w/api.php?action=generatedumps&format=json&type=xml&token=f7169080c27bd8590358f1946db9ef21627587d9+\\ https://www.irccloud.com/pastebin/ruC6sWB4/ [20:43:38] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.15, 4.58, 3.78 [20:44:42] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.28, 3.88, 3.41 [20:44:52] ugochimobi: yes you have to POST it [20:44:56] CosmicAlpha: got access? [20:45:11] If so, can you please force revoke that token [20:45:24] ugochimobi: never share tokens in public [20:45:49] I really wish we could have something like `https://static.miraheze.org/wikinamewiki/dumps` to see index of dumps per wiki [20:45:49] [url] 404 Not Found | static.miraheze.org [20:45:58] RhinosF1: oh, okay, thanks [20:46:15] RhinosF1: no, mobile right now. [20:46:22] paladox: ping [20:46:30] RhinosF1: hello? [20:46:37] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.46, 3.34, 3.27 [20:46:44] paladox: can you please reset ugochimobi's tokens? [20:47:00] for central auth? [20:47:11] no no [20:47:31] i'm confused, reset what tokens [20:47:37] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.36, 3.71, 3.63 [20:47:55] paladox: the token that was just publicly posted above. [20:48:24] oh, how would i do that? I know nothing about the api token thing. [20:48:25] paladox: meta [20:48:37] It should be the user_tokens in the user table [20:48:52] I believe [20:48:55] CosmicAlpha: ^ [20:48:57] is it his https://github.com/wikimedia/mediawiki/blob/master/maintenance/resetUserTokens.php [20:48:57] [url] mediawiki/resetUserTokens.php at master · wikimedia/mediawiki · GitHub | github.com [20:48:59] ah ok [20:49:43] paladox: yes but only for ugochimobi [20:49:45] not everyone [20:49:50] yeh [20:51:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [20:53:22] RhinosF1: done i think, i just did "$user = User::newFromName( 'ugochimobi' );" "$user->setToken();" "$user->saveSettings();" [20:53:46] !log reset user token for ugochimobi on gratisdatawiki [20:53:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:54:10] paladox: is it gone from the table? [20:54:22] CosmicAlpha: sound good ^ [20:55:21] user_token different token to the one posted above [20:56:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [20:56:45] paladox: good [20:56:59] ugochimobi: please check your account settings look right [20:57:24] And don't ever do that again because I won't disturb paladox, I'll just lock your account until I'm [20:57:26] Few [20:57:28] Free* [21:01:33] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.84, 3.62, 3.58 [21:02:00] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [21:02:00] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [21:02:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [21:02:31] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.72, 7.33, 6.49 [21:03:33] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.76, 3.79, 3.65 [21:04:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.47, 7.60, 6.70 [21:05:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.78, 3.57, 3.34 [21:05:32] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.11, 3.94, 3.70 [21:06:22] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.35, 7.74, 6.85 [21:07:08] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.83, 3.31, 3.27 [21:08:17] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.23, 7.58, 6.90 [21:08:37] RhinosF1: thanks and everything looks fine [21:08:46] paladox: many thanks [21:11:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.24, 3.61, 3.66 [21:12:08] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.41, 6.65, 6.68 [21:13:30] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.66, 4.57, 4.01 [21:17:29] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.03, 3.86, 3.85 [21:19:48] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.91, 3.92, 3.54 [21:21:28] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.02, 4.45, 4.10 [21:23:39] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.90, 3.94, 3.63 [21:25:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 3.80, 4.04, 3.71 [21:27:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [21:27:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.53, 3.92, 3.71 [21:29:25] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.00, 4.44, 3.91 [21:30:34] PROBLEM - theapplesecretswiki.ml - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'theapplesecretswiki.ml' expires in 15 day(s) (Sun 22 May 2022 21:17:41 GMT +0000). [21:31:55] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [21:32:07] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.66, 9.71, 8.53 [21:34:07] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.88, 9.19, 8.47 [21:35:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [21:35:08] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.74, 1.50, 1.27 [21:35:12] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.68, 3.98, 3.86 [21:37:08] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.53, 1.48, 1.29 [21:37:24] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.62, 3.45, 3.99 [21:37:35] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3d96cbbbd362...0ba8db6a274b [21:37:36] [url] Comparing 3d96cbbbd362...0ba8db6a274b · miraheze/ssl · GitHub | github.com [21:37:37] [02miraheze/ssl] 07MirahezeSSLBot 030ba8db6 - Bot: Update SSL cert for theapplesecretswiki.ml [21:39:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.40, 3.80, 3.78 [21:39:24] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.66, 4.41, 4.27 [21:41:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.62, 3.35, 3.62 [21:47:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.10, 3.75, 3.68 [21:53:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.08, 3.74, 3.74 [21:59:06] RECOVERY - theapplesecretswiki.ml - LetsEncrypt on sslhost is OK: OK - Certificate 'theapplesecretswiki.ml' will expire on Thu 04 Aug 2022 20:37:27 GMT +0000. [22:00:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [22:01:08] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.62, 3.00, 3.38 [22:03:44] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [22:05:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 8.08, 4.83, 3.96 [22:11:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.71, 3.84, 3.80 [22:17:57] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.54, 6.38, 5.52 [22:18:08] PROBLEM - mw101 MediaWiki Rendering on mw101 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:18:54] PROBLEM - cp20 Stunnel HTTP for mw101 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:19:26] PROBLEM - cp30 Stunnel HTTP for mw101 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:19:31] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:19:51] PROBLEM - cp21 Stunnel HTTP for mw101 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:20:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [22:20:07] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.66, 10.24, 8.79 [22:21:01] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:21:02] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:21:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.58, 3.80, 3.73 [22:21:08] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.68, 10.64, 8.80 [22:21:56] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 9.74, 7.83, 6.28 [22:22:46] PROBLEM - cp31 Stunnel HTTP for mw101 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:23:08] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.75, 2.05, 1.62 [22:23:22] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.71, 10.15, 8.67 [22:24:52] PROBLEM - cp31 Stunnel HTTP for mw112 on cp31 is CRITICAL: HTTP CRITICAL - No data received from host [22:24:55] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:24:59] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:25:08] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.90, 1.63, 1.52 [22:25:12] PROBLEM - cp20 Stunnel HTTP for mw112 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:25:13] PROBLEM - cp30 Stunnel HTTP for mw112 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:25:14] PROBLEM - mw112 MediaWiki Rendering on mw112 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:25:47] PROBLEM - mw102 MediaWiki Rendering on mw102 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:26:01] PROBLEM - cp21 Stunnel HTTP for mw102 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:26:07] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.60, 11.54, 9.81 [22:26:49] RECOVERY - cp31 Stunnel HTTP for mw112 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14640 bytes in 1.537 second response time [22:26:52] PROBLEM - cp20 Stunnel HTTP for mw102 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:27:02] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.88, 9.98, 9.12 [22:27:09] RECOVERY - cp20 Stunnel HTTP for mw112 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14634 bytes in 0.745 second response time [22:27:10] RECOVERY - cp30 Stunnel HTTP for mw112 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14626 bytes in 1.496 second response time [22:27:14] PROBLEM - cp30 Stunnel HTTP for mw102 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:27:15] RECOVERY - mw112 MediaWiki Rendering on mw112 is OK: HTTP OK: HTTP/1.1 200 OK - 22334 bytes in 2.076 second response time [22:27:53] RECOVERY - mw102 MediaWiki Rendering on mw102 is OK: HTTP OK: HTTP/1.1 200 OK - 22334 bytes in 8.535 second response time [22:28:04] RECOVERY - cp21 Stunnel HTTP for mw102 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14626 bytes in 7.137 second response time [22:28:07] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.43, 11.10, 9.86 [22:28:48] RECOVERY - cp20 Stunnel HTTP for mw102 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14640 bytes in 0.020 second response time [22:29:06] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14634 bytes in 8.729 second response time [22:29:14] RECOVERY - cp30 Stunnel HTTP for mw102 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14626 bytes in 2.055 second response time [22:29:56] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.10, 7.62, 7.01 [22:30:21] RECOVERY - cp21 Stunnel HTTP for mw101 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14634 bytes in 4.792 second response time [22:30:55] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14634 bytes in 6.161 second response time [22:30:55] RECOVERY - cp31 Stunnel HTTP for mw101 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14640 bytes in 6.588 second response time [22:30:56] RECOVERY - mw101 MediaWiki Rendering on mw101 is OK: HTTP OK: HTTP/1.1 200 OK - 22334 bytes in 7.455 second response time [22:31:00] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.21, 9.31, 8.96 [22:31:22] RECOVERY - cp20 Stunnel HTTP for mw101 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14640 bytes in 2.196 second response time [22:31:33] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14640 bytes in 0.015 second response time [22:31:38] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 22334 bytes in 3.418 second response time [22:31:55] RECOVERY - cp30 Stunnel HTTP for mw101 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14640 bytes in 0.323 second response time [22:32:07] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14640 bytes in 0.317 second response time [22:34:07] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.54, 9.79, 9.70 [22:35:56] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.35, 6.52, 6.78 [22:45:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [22:45:56] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.00, 6.54, 6.55 [22:47:55] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.53, 6.67, 6.60 [22:52:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.25, 9.80, 8.99 [22:53:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [22:54:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.25, 8.93, 8.78 [23:00:01] !log [@mwtask111] starting deploy of {'l10nupdate': True} to all [23:00:04] !log [@test101] starting deploy of {'l10nupdate': True} to all [23:00:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:00:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:03:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:06:13] !log [@mwtask111] finished deploy of {'l10nupdate': True} to all - SUCCESS in 371s [23:06:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:11:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:16:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:25:35] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.10, 3.29, 1.77 [23:29:33] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.20, 1.59, 1.41 [23:33:29] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [23:33:29] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [23:41:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:47:25] !log [@test101] finished deploy of {'l10nupdate': True} to all - SUCCESS in 2839s [23:47:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:56:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki