[00:01:24] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.69, 3.42, 3.07 [00:01:51] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.19, 3.69, 3.18 [00:02:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.14, 4.40, 3.87 [00:02:21] PROBLEM - es101 Current Load on es101 is WARNING: WARNING - load average: 3.77, 2.42, 1.62 [00:03:02] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 0.87, 1.10, 1.65 [00:03:33] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:03:50] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.46, 3.08, 3.01 [00:04:21] RECOVERY - es101 Current Load on es101 is OK: OK - load average: 3.07, 2.67, 1.81 [00:05:16] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.20, 3.69, 3.29 [00:06:50] [02ssl] 07AgentIsai synchronize pull request 03#505: Add redirects - 13https://github.com/miraheze/ssl/pull/505 [00:07:04] [02ssl] 07AgentIsai synchronize pull request 03#505: Add redirects - 13https://github.com/miraheze/ssl/pull/505 [00:07:14] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.03, 4.02, 3.46 [00:07:47] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.58, 4.34, 3.53 [00:08:33] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:09:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.20, 3.75, 3.44 [00:12:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.30, 3.70, 3.83 [00:12:28] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.66, 9.61, 8.73 [00:13:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.83, 3.90, 3.70 [00:14:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.30, 4.41, 4.08 [00:14:26] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.09, 8.78, 8.52 [00:15:14] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.20, 3.18, 3.30 [00:16:21] PROBLEM - es101 Current Load on es101 is CRITICAL: CRITICAL - load average: 4.13, 3.17, 2.46 [00:17:42] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.43, 3.74, 3.63 [00:18:22] PROBLEM - es101 Current Load on es101 is WARNING: WARNING - load average: 3.95, 3.52, 2.69 [00:19:41] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.85, 3.39, 3.51 [00:20:22] RECOVERY - es101 Current Load on es101 is OK: OK - load average: 2.81, 3.25, 2.69 [00:21:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.73, 3.55, 3.43 [00:21:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:21:39] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.57, 3.04, 3.37 [00:23:14] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.20, 3.39, 3.38 [00:26:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:27:57] Thanks, Agent! [00:30:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.25, 3.18, 3.87 [00:32:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.71, 3.74, 3.99 [00:36:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.14, 3.26, 3.77 [00:36:41] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [00:36:42] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/5b0ccfecddda...ae74ab6781f1 [00:36:44] [02miraheze/ImportDump] 07Universal-Omega 03ae74ab6 - Update mediawiki-tests.yml [00:37:04] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.87, 2.39, 1.73 [00:39:04] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.12, 1.90, 1.63 [00:40:39] miraheze/ImportDump - Universal-Omega the build passed. [00:41:04] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.14, 1.70, 1.59 [00:44:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.42, 3.52, 3.26 [00:46:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.87, 4.33, 4.02 [00:48:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.01, 3.98, 3.94 [00:48:38] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.65, 3.85, 3.45 [00:50:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.68, 4.73, 4.25 [00:53:04] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.27, 1.82, 1.64 [00:55:04] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.70, 1.60, 1.57 [00:56:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.16, 3.87, 3.68 [00:58:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.36, 3.51, 3.95 [01:00:13] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.19, 3.02, 3.38 [01:00:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:06:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.86, 2.73, 3.40 [01:10:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:11:00] [02ImportDump] 07Universal-Omega edited pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [01:11:55] [02ImportDump] 07Universal-Omega edited pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [01:12:29] [02ImportDump] 07Universal-Omega edited pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [01:14:49] [02ImportDump] 07Universal-Omega edited pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [01:20:44] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.15, 1.83, 1.65 [01:22:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 9.31, 5.12, 3.89 [01:23:01] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.46, 4.76, 3.44 [01:23:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:23:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.04, 4.89, 3.88 [01:24:40] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.83, 1.87, 1.71 [01:24:59] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.42, 3.79, 3.24 [01:25:23] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.75, 3.95, 3.65 [01:26:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.39, 3.28, 3.44 [01:26:38] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.19, 1.58, 1.62 [01:26:58] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.17, 3.16, 3.07 [01:28:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.72, 3.14, 3.37 [01:33:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:35:14] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.01, 2.85, 3.24 [01:45:53] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [01:45:53] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [01:46:18] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.96, 1.87, 1.60 [01:48:16] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.72, 1.55, 1.52 [01:57:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.97, 3.26, 3.22 [01:59:39] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.40, 2.86, 3.07 [02:07:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.38, 3.42, 3.24 [02:08:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:09:18] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.90, 3.91, 3.43 [02:11:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.46, 3.73, 3.42 [02:13:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:14:46] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [02:17:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.76, 3.71, 3.44 [02:19:04] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.72, 3.45, 3.37 [02:21:42] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.32, 1.46, 1.22 [02:22:58] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.48, 3.11, 3.25 [02:23:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:23:40] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.88, 1.26, 1.18 [02:28:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:47:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.39, 3.53, 3.05 [02:49:07] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.87, 3.32, 3.04 [02:50:10] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.80, 2.03, 1.59 [02:55:02] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.43, 3.20, 3.02 [02:56:05] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.40, 2.00, 1.79 [02:57:01] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.79, 2.91, 2.92 [02:58:03] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.94, 2.45, 1.96 [03:02:00] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.17, 1.99, 1.91 [03:07:54] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.24, 1.48, 1.69 [03:45:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:50:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:21:02] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.73, 1.56, 1.31 [04:23:02] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.39, 1.48, 1.31 [04:49:47] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.43, 1.95, 1.50 [04:51:46] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.80, 1.56, 1.41 [04:52:49] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.07, 3.30, 2.66 [04:56:43] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.16, 3.16, 2.75 [05:07:02] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.78, 1.60, 1.38 [05:09:02] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.17, 1.45, 1.35 [05:19:02] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.40, 2.05, 1.66 [05:21:02] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.89, 1.98, 1.68 [05:25:02] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.25, 1.52, 1.55 [05:30:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.74, 3.55, 3.02 [05:38:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.07, 3.14, 3.09 [05:50:18] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.45, 1.84, 1.56 [05:52:14] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.57, 1.73, 1.55 [05:56:05] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.44, 1.63, 1.55 [05:57:12] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.34, 3.83, 3.41 [06:01:06] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.89, 3.14, 3.21 [06:07:43] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.82, 2.11, 1.74 [06:17:11] PROBLEM - wiki.lefrenchmelee.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr [06:17:11] hon3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query lefrenchmelee.fr. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [06:45:52] RECOVERY - wiki.lefrenchmelee.fr - reverse DNS on sslhost is OK: SSL OK - wiki.lefrenchmelee.fr reverse DNS resolves to cp21.miraheze.org - CNAME OK [06:47:26] PROBLEM - wikipanel.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wikipanel.org' expires in 15 day(s) (Sat 07 May 2022 06:36:14 GMT +0000). [06:48:10] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/4c997efe6ef2...7e3a558f5137 [06:48:12] [02miraheze/ssl] 07MirahezeSSLBot 037e3a558 - Bot: Update SSL cert for wikipanel.org [06:56:43] PROBLEM - wiki.controlistes.fr - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.controlistes.fr' expires in 15 day(s) (Sat 07 May 2022 06:37:30 GMT +0000). [06:57:15] PROBLEM - ngsswiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'ngsswiki.com' expires in 15 day(s) (Sat 07 May 2022 06:39:57 GMT +0000). [06:59:49] PROBLEM - mabi.myomi.net - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolve(hos [06:59:49] AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.403451442718506 seconds [07:00:58] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/7e3a558f5137...8cec0c3e1d30 [07:01:00] [02miraheze/ssl] 07MirahezeSSLBot 038cec0c3 - Bot: Update SSL cert for wiki.controlistes.fr [07:06:51] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/8cec0c3e1d30...8ae417d049f7 [07:06:52] [02miraheze/ssl] 07MirahezeSSLBot 038ae417d - Bot: Update SSL cert for ngsswiki.com [07:16:22] RECOVERY - wikipanel.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wikipanel.org' will expire on Wed 20 Jul 2022 05:48:02 GMT +0000. [07:25:55] RECOVERY - wiki.controlistes.fr - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.controlistes.fr' will expire on Wed 20 Jul 2022 06:00:53 GMT +0000. [07:26:54] RECOVERY - ngsswiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'ngsswiki.com' will expire on Wed 20 Jul 2022 06:06:45 GMT +0000. [07:29:49] RECOVERY - mabi.myomi.net - reverse DNS on sslhost is OK: SSL OK - mabi.myomi.net reverse DNS resolves to cp20.miraheze.org - CNAME OK [07:57:04] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.28, 2.17, 1.32 [07:59:04] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.71, 1.66, 1.24 [08:07:13] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.83, 3.10, 2.44 [08:07:14] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.34, 3.18, 2.34 [08:07:15] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.97, 3.29, 2.55 [08:09:10] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.97, 2.78, 2.41 [08:09:14] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.95, 2.83, 2.47 [08:09:14] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.04, 2.83, 2.32 [08:42:15] [02mw-config] 07ugochimobi opened pull request 03#4617: $wmgSharedUploadClientDBname for gpcommonswiki - 13https://github.com/miraheze/mw-config/pull/4617 [08:43:19] miraheze/mw-config - ugochimobi the build passed. [09:37:02] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.55, 1.62, 1.99 [09:49:02] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.13, 1.78, 1.85 [09:51:02] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.73, 1.79, 1.84 [09:57:02] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.35, 1.45, 1.67 [10:19:02] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.26, 1.69, 1.46 [10:21:02] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.40, 1.64, 1.47 [10:45:53] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [10:45:53] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [10:49:43] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.27, 1.69, 1.45 [10:51:39] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 2.00, 1.79, 1.51 [10:53:35] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.51, 1.66, 1.50 [10:56:28] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 3.28, 2.14, 1.57 [11:00:27] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.68, 1.98, 1.64 [11:00:54] [02mw-config] 07ugochimobi commented on pull request 03#4612: override page HTML title with the wikibase label, if available, or else the id - 13https://github.com/miraheze/mw-config/pull/4612#issuecomment-1105059893 [11:06:24] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.13, 1.48, 1.54 [11:09:45] [02miraheze/IncidentReporting] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/IncidentReporting/compare/957bd1f349a9...a7ef0d0c5acd [11:09:46] [02miraheze/IncidentReporting] 07translatewiki 03a7ef0d0 - Localisation updates from https://translatewiki.net. [11:09:48] [02miraheze/CreateWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/CreateWiki/compare/ec36b0cae8dc...59e0bf21d4cc [11:09:49] [02miraheze/CreateWiki] 07translatewiki 0359e0bf2 - Localisation updates from https://translatewiki.net. [11:09:51] [02miraheze/ManageWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/826c8fc28367...16819d4bfb18 [11:09:52] [02miraheze/ManageWiki] 07translatewiki 0316819d4 - Localisation updates from https://translatewiki.net. [11:09:54] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/24f540f44b3f...36632523cc4c [11:09:55] [02miraheze/MirahezeMagic] 07translatewiki 033663252 - Localisation updates from https://translatewiki.net. [11:15:36] miraheze/IncidentReporting - translatewiki the build passed. [11:20:55] miraheze/ManageWiki - translatewiki the build passed. [11:25:10] miraheze/CreateWiki - translatewiki the build passed. [11:26:59] miraheze/MirahezeMagic - translatewiki the build passed. [11:43:53] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [11:44:48] PROBLEM - wiki.dunmoore.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.dunmoore.net' expires in 15 day(s) (Sat 07 May 2022 11:33:49 GMT +0000). [11:47:08] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 3.53, 2.50, 1.70 [11:49:38] [02mw-config] 07ugochimobi synchronize pull request 03#4612: override page HTML title with the wikibase label, if available, or else the id - 13https://github.com/miraheze/mw-config/pull/4612 [11:50:27] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/8ae417d049f7...a4841e348f31 [11:50:29] [02miraheze/ssl] 07MirahezeSSLBot 03a4841e3 - Bot: Update SSL cert for wiki.dunmoore.net [11:50:42] miraheze/mw-config - ugochimobi the build passed. [11:52:08] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.04, 4.59, 3.00 [11:52:31] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.85, 3.79, 2.78 [11:53:08] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 8.86, 5.33, 3.60 [11:55:05] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.77, 1.87, 1.73 [11:56:28] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.55, 3.92, 3.11 [11:57:55] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.48, 3.81, 3.24 [12:01:03] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 3.56, 2.26, 1.89 [12:02:08] PROBLEM - www.recyclerie-uzege.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.recyclerie-uzege.org' expires in 15 day(s) (Sat 07 May 2022 11:40:29 GMT +0000). [12:03:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.75, 4.13, 3.47 [12:04:19] PROBLEM - www.winenjoy.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.winenjoy.net' expires in 15 day(s) (Sat 07 May 2022 11:49:53 GMT +0000). [12:04:42] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a4841e348f31...6ad608f5ea9b [12:04:44] [02miraheze/ssl] 07MirahezeSSLBot 036ad608f - Bot: Update SSL cert for www.recyclerie-uzege.org [12:04:50] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.98, 3.95, 3.83 [12:05:39] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.36, 3.38, 3.27 [12:06:42] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/6ad608f5ea9b...4eae946fce5d [12:06:43] [02miraheze/ssl] 07MirahezeSSLBot 034eae946 - Bot: Update SSL cert for www.winenjoy.net [12:07:01] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.06, 1.76, 1.83 [12:07:09] PROBLEM - staff.polandballwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'staff.polandballwiki.com' expires in 15 day(s) (Sat 07 May 2022 12:02:35 GMT +0000). [12:08:22] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.06, 3.63, 3.34 [12:09:05] [02mw-config] 07RhinosF1 reviewed pull request 03#4612 commit - 13https://github.com/miraheze/mw-config/pull/4612#discussion_r855116102 [12:10:02] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/4eae946fce5d...6a3a693a3472 [12:10:04] [02miraheze/ssl] 07MirahezeSSLBot 036a3a693 - Bot: Update SSL cert for staff.polandballwiki.com [12:10:21] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.76, 3.61, 3.36 [12:10:41] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.90, 4.17, 3.93 [12:10:42] PROBLEM - pt.graalmilitary.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'pt.graalmilitary.com' expires in 15 day(s) (Sat 07 May 2022 12:06:04 GMT +0000). [12:10:59] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.84, 2.16, 1.97 [12:11:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.06, 3.86, 3.52 [12:12:08] PROBLEM - wiki.simorgh.me - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.simorgh.me' expires in 15 day(s) (Sat 07 May 2022 11:46:35 GMT +0000). [12:12:38] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.02, 3.73, 3.80 [12:12:59] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.46, 1.79, 1.84 [12:13:25] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.48, 3.39, 3.39 [12:13:53] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [12:14:57] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/6a3a693a3472...c454a78b6737 [12:14:58] [02miraheze/ssl] 07MirahezeSSLBot 03c454a78 - Bot: Update SSL cert for pt.graalmilitary.com [12:15:23] [02mw-config] 07ugochimobi reviewed pull request 03#4612 commit - 13https://github.com/miraheze/mw-config/pull/4612#discussion_r855121030 [12:16:17] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.95, 3.20, 3.26 [12:16:32] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.21, 4.00, 3.88 [12:16:57] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.76, 1.22, 1.59 [12:17:41] [02mw-config] 07ugochimobi synchronize pull request 03#4612: override page HTML title with the wikibase label, if available, or else the id - 13https://github.com/miraheze/mw-config/pull/4612 [12:18:29] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.89, 3.79, 3.83 [12:18:53] miraheze/mw-config - ugochimobi the build passed. [12:19:33] [02mw-config] 07ugochimobi reviewed pull request 03#4612 commit - 13https://github.com/miraheze/mw-config/pull/4612#discussion_r855124320 [12:20:26] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.14, 4.11, 3.95 [12:21:13] [02mw-config] 07RhinosF1 reviewed pull request 03#4612 commit - 13https://github.com/miraheze/mw-config/pull/4612#discussion_r855125643 [12:21:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.98, 3.71, 3.51 [12:22:23] PROBLEM - pwsc.polandballwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'pwsc.polandballwiki.com' expires in 15 day(s) (Sat 07 May 2022 11:55:45 GMT +0000). [12:22:23] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.63, 3.71, 3.83 [12:22:51] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c454a78b6737...15389df044ab [12:22:52] [02miraheze/ssl] 07MirahezeSSLBot 0315389df - Bot: Update SSL cert for wiki.simorgh.me [12:23:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:23:29] [02mw-config] 07ugochimobi reviewed pull request 03#4612 commit - 13https://github.com/miraheze/mw-config/pull/4612#discussion_r855127556 [12:24:29] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/15389df044ab...deb123805cd7 [12:24:31] [02miraheze/ssl] 07MirahezeSSLBot 03deb1238 - Bot: Update SSL cert for pwsc.polandballwiki.com [12:24:54] [02mw-config] 07ugochimobi synchronize pull request 03#4612: override page HTML title with the wikibase label, if available, or else the id - 13https://github.com/miraheze/mw-config/pull/4612 [12:25:02] PROBLEM - wiki.ct777.cf - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.ct777.cf' expires in 15 day(s) (Sat 07 May 2022 12:12:20 GMT +0000). [12:25:14] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.50, 3.83, 3.57 [12:26:00] miraheze/mw-config - ugochimobi the build passed. [12:26:15] [02mw-config] 07RhinosF1 reviewed pull request 03#4612 commit - 13https://github.com/miraheze/mw-config/pull/4612#discussion_r855129618 [12:26:17] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/deb123805cd7...88302b8b785d [12:26:18] [02miraheze/ssl] 07MirahezeSSLBot 0388302b8 - Bot: Update SSL cert for wiki.ct777.cf [12:26:20] [02mw-config] 07ugochimobi reviewed pull request 03#4612 commit - 13https://github.com/miraheze/mw-config/pull/4612#discussion_r855129701 [12:26:27] PROBLEM - wiki.umbranet.mandelbrotgames.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.umbranet.mandelbrotgames.com' expires in 15 day(s) (Sat 07 May 2022 11:57:48 GMT +0000). [12:27:02] [02mw-config] 07RhinosF1 reviewed pull request 03#4612 commit - 13https://github.com/miraheze/mw-config/pull/4612#discussion_r855130292 [12:29:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.81, 3.48, 3.51 [12:30:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.43, 3.72, 3.75 [12:30:35] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/88302b8b785d...682418f81447 [12:30:37] [02miraheze/ssl] 07MirahezeSSLBot 03682418f - Bot: Update SSL cert for wiki.umbranet.mandelbrotgames.com [12:30:45] RECOVERY - www.recyclerie-uzege.org - LetsEncrypt on sslhost is OK: OK - Certificate 'www.recyclerie-uzege.org' will expire on Wed 20 Jul 2022 11:04:36 GMT +0000. [12:30:51] [02mw-config] 07ugochimobi reviewed pull request 03#4612 commit - 13https://github.com/miraheze/mw-config/pull/4612#discussion_r855133289 [12:32:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.27, 3.60, 3.71 [12:33:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:33:04] PROBLEM - popira.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'popira.info' expires in 15 day(s) (Sat 07 May 2022 12:05:29 GMT +0000). [12:33:37] RECOVERY - www.winenjoy.net - LetsEncrypt on sslhost is OK: OK - Certificate 'www.winenjoy.net' will expire on Wed 20 Jul 2022 11:06:36 GMT +0000. [12:34:49] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.85, 1.69, 1.56 [12:34:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.02, 1.82, 1.39 [12:35:48] [02mw-config] 07RhinosF1 reviewed pull request 03#4612 commit - 13https://github.com/miraheze/mw-config/pull/4612#discussion_r855137358 [12:36:18] RECOVERY - staff.polandballwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'staff.polandballwiki.com' will expire on Wed 20 Jul 2022 11:09:56 GMT +0000. [12:36:48] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/682418f81447...18e2158427db [12:36:49] [02miraheze/ssl] 07MirahezeSSLBot 0318e2158 - Bot: Update SSL cert for popira.info [12:37:04] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.78, 3.85, 3.40 [12:38:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.01, 4.14, 3.87 [12:38:47] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.98, 1.50, 1.52 [12:39:14] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.70, 3.89, 3.61 [12:40:23] RECOVERY - pt.graalmilitary.com - LetsEncrypt on sslhost is OK: OK - Certificate 'pt.graalmilitary.com' will expire on Wed 20 Jul 2022 11:14:51 GMT +0000. [12:41:02] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.72, 3.83, 3.49 [12:42:36] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.13, 1.66, 1.54 [12:43:25] RECOVERY - wiki.dunmoore.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.dunmoore.net' will expire on Wed 20 Jul 2022 10:50:20 GMT +0000. [12:45:00] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.08, 3.68, 3.50 [12:46:59] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.34, 3.76, 3.57 [12:47:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.51, 3.51, 3.63 [12:48:22] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.98, 1.70, 1.56 [12:48:58] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.85, 4.36, 3.80 [12:49:10] [02mw-config] 07ugochimobi synchronize pull request 03#4612: override page HTML title with the wikibase label, if available, or else the id - 13https://github.com/miraheze/mw-config/pull/4612 [12:50:14] miraheze/mw-config - ugochimobi the build passed. [12:50:17] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.71, 1.37, 1.46 [12:50:57] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.01, 3.72, 3.63 [12:51:14] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.72, 2.86, 3.37 [12:53:44] [02mw-config] 07ugochimobi reviewed pull request 03#4612 commit - 13https://github.com/miraheze/mw-config/pull/4612#discussion_r855152926 [12:54:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:55:40] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.45, 1.94, 1.65 [12:56:16] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.23, 9.04, 8.16 [12:56:54] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.31, 3.73, 3.60 [12:57:00] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.30, 10.42, 9.50 [12:57:39] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.52, 1.82, 1.64 [12:58:16] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.10, 8.34, 8.02 [12:58:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.86, 3.65, 3.98 [12:59:38] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.21, 1.67, 1.61 [13:00:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.06, 3.77, 3.97 [13:00:51] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.99, 3.56, 3.59 [13:02:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.34, 3.65, 3.91 [13:02:38] RECOVERY - popira.info - LetsEncrypt on sslhost is OK: OK - Certificate 'popira.info' will expire on Wed 20 Jul 2022 11:36:42 GMT +0000. [13:04:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.51, 3.74, 3.89 [13:04:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:04:40] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.03, 9.82, 9.71 [13:04:49] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.02, 3.48, 3.52 [13:06:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.55, 3.57, 3.81 [13:06:48] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.37, 3.46, 3.51 [13:08:47] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.92, 2.97, 3.33 [13:11:24] RECOVERY - wiki.simorgh.me - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.simorgh.me' will expire on Wed 20 Jul 2022 11:22:45 GMT +0000. [13:12:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.80, 3.60, 3.64 [13:12:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:14:47] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 7.90, 4.64, 3.82 [13:15:53] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [13:16:08] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.56, 4.55, 3.62 [13:21:09] RECOVERY - pwsc.polandballwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'pwsc.polandballwiki.com' will expire on Wed 20 Jul 2022 11:24:24 GMT +0000. [13:23:07] RECOVERY - wiki.ct777.cf - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.ct777.cf' will expire on Wed 20 Jul 2022 11:26:11 GMT +0000. [13:24:27] RECOVERY - wiki.umbranet.mandelbrotgames.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.umbranet.mandelbrotgames.com' will expire on Wed 20 Jul 2022 11:30:28 GMT +0000. [13:25:27] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 3.68, 2.45, 1.83 [13:29:26] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.28, 1.85, 1.72 [13:31:25] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.71, 1.94, 1.76 [13:35:24] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.22, 1.71, 1.72 [13:37:22] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.06, 3.45, 3.99 [13:37:23] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.81, 1.41, 1.61 [13:39:40] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.68, 10.44, 9.93 [13:41:14] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.02, 4.70, 4.35 [13:41:37] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.24, 11.20, 10.27 [13:43:35] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.58, 10.98, 10.31 [13:44:46] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [13:44:46] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [13:47:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.50, 3.47, 3.95 [13:47:30] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.18, 9.46, 9.84 [13:50:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.36, 2.97, 3.82 [13:52:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:55:14] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.98, 3.48, 3.61 [13:57:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.34, 3.42, 3.57 [13:59:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:59:14] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.96, 2.90, 3.36 [14:00:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.09, 3.21, 3.99 [14:06:47] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.94, 3.01, 3.34 [14:08:16] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.45, 10.00, 8.91 [14:09:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:10:16] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 10.13, 9.96, 9.03 [14:11:22] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.09, 9.63, 8.89 [14:13:22] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 13.15, 10.71, 9.36 [14:13:53] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [14:13:59] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.70, 10.65, 9.50 [14:14:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:14:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.42, 4.02, 3.83 [14:15:53] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.22, 10.05, 9.40 [14:17:22] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.57, 10.73, 9.70 [14:19:31] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.11, 10.51, 9.56 [14:20:13] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.40, 10.30, 9.46 [14:21:22] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.62, 11.44, 10.20 [14:21:25] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.40, 9.60, 9.35 [14:22:13] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.96, 9.89, 9.39 [14:23:22] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.56, 11.23, 10.30 [14:24:54] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.62, 4.37, 3.54 [14:25:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.79, 4.74, 3.83 [14:26:55] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-9 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/91869e218010 [14:26:57] [02miraheze/puppet] 07paladox 0391869e2 - DO NOT MERGE [14:26:58] [02puppet] 07paladox created branch 03paladox-patch-9 - 13https://github.com/miraheze/puppet [14:27:00] [02puppet] 07paladox opened pull request 03#2514: DO NOT MERGE - 13https://github.com/miraheze/puppet/pull/2514 [14:27:03] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 2.00, 1.89, 1.52 [14:27:22] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.72, 9.93, 10.02 [14:29:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:29:02] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.22, 1.60, 1.45 [14:29:41] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.52, 3.92, 3.74 [14:30:42] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.83, 3.90, 3.61 [14:31:40] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.22, 4.43, 3.94 [14:32:38] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.41, 4.09, 3.71 [14:36:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.55, 3.93, 3.80 [14:38:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:38:25] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.74, 4.11, 3.89 [14:39:22] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.99, 10.57, 10.05 [14:40:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.59, 3.71, 3.78 [14:41:22] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.55, 11.34, 10.41 [14:41:35] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.81, 3.95, 3.97 [14:42:17] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.42, 4.17, 3.94 [14:43:22] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.48, 11.11, 10.45 [14:43:22] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-9 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/91869e218010...947ed9a47080 [14:43:23] [02miraheze/puppet] 07paladox 03947ed9a - Update default.vcl [14:43:25] [02puppet] 07paladox synchronize pull request 03#2514: DO NOT MERGE - 13https://github.com/miraheze/puppet/pull/2514 [14:44:13] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.66, 3.66, 3.79 [14:45:13] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.02, 10.64, 10.08 [14:45:22] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.08, 10.04, 10.14 [14:46:09] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.61, 3.99, 3.89 [14:47:13] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.89, 10.04, 9.93 [14:47:32] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.33, 3.64, 3.77 [14:48:05] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.12, 3.74, 3.81 [14:49:02] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.99, 1.26, 0.89 [14:50:01] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.39, 3.96, 3.87 [14:50:17] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.31, 10.27, 9.71 [14:51:02] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.07, 1.15, 0.90 [14:51:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.60, 3.64, 3.72 [14:51:57] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.83, 3.88, 3.85 [14:52:13] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.19, 9.79, 9.61 [14:53:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:53:50] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.88, 1.76, 1.62 [14:54:57] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-9 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/947ed9a47080...abd7932d2497 [14:54:58] [02miraheze/puppet] 07paladox 03abd7932 - Update init.pp [14:55:00] [02puppet] 07paladox synchronize pull request 03#2514: DO NOT MERGE - 13https://github.com/miraheze/puppet/pull/2514 [14:57:26] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.98, 4.59, 4.05 [14:57:27] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-9 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/abd7932d2497...cc2c874aa65f [14:57:28] [02miraheze/puppet] 07paladox 03cc2c874 - Update php.pp [14:57:30] [02puppet] 07paladox synchronize pull request 03#2514: DO NOT MERGE - 13https://github.com/miraheze/puppet/pull/2514 [14:57:44] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.26, 4.18, 3.91 [14:57:58] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-9 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/cc2c874aa65f...011354d60ec5 [14:58:00] [02miraheze/puppet] 07paladox 03011354d - Update php.pp [14:58:01] [02puppet] 07paladox synchronize pull request 03#2514: DO NOT MERGE - 13https://github.com/miraheze/puppet/pull/2514 [14:58:13] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.86, 1.47, 1.31 [14:59:48] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.03, 1.45, 1.55 [15:00:06] paladox: why them values? [15:00:13] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.72, 1.30, 1.27 [15:00:18] The 59s is deliberately set to be just lower than nginx [15:00:38] So php times out before nginx does as a stray nginx process is cheaper than a stray php one [15:01:24] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.55, 3.80, 3.89 [15:02:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:02:45] I'm playing around and it does say do not merge implying its a work in progress or may be abandoned :) [15:03:32] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.74, 3.84, 3.89 [15:04:34] paladox: yeah I'm more wondering thought process [15:04:49] Because if there's a chance of use, I wanna know [15:04:58] * RhinosF1 was the last to mess around with php config [15:05:00] i'm having a look at the WMF config [15:05:08] Ah right [15:05:18] I'd be careful doing that as we are not the WMF [15:05:23] And have far less resources [15:05:24] https://github.com/wikimedia/puppet/blob/production/modules/tlsproxy/templates/localssl.erb#L65 [15:05:25] [url] puppet/localssl.erb at production · wikimedia/puppet · GitHub | github.com [15:05:30] yes [15:06:11] I think the timeouts then should be 60s [15:06:29] Although tbh I dislike that and would rather us have different ones per route but that's not possible [15:07:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:07:21] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.06, 4.02, 3.88 [15:07:24] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.84, 4.44, 4.09 [15:11:18] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.44, 3.92, 3.87 [15:15:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:18:24] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/6b4b3263ae2f...54145c91e456 [15:18:26] [02miraheze/puppet] 07paladox 0354145c9 - varnish: set http2_max_field_size to 8k [15:19:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.09, 3.64, 3.95 [15:19:15] err typo... should have been nginx [15:19:29] [02puppet] 07paladox commented on commit 0354145c91e4568abf3ef62295fd486c69a45201aa - 13https://github.com/miraheze/puppet/commit/54145c91e4568abf3ef62295fd486c69a45201aa#commitcomment-71819860 [15:19:38] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.45, 10.42, 10.01 [15:20:37] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.12, 10.52, 9.39 [15:21:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.46, 3.66, 3.67 [15:22:34] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.88, 9.96, 9.30 [15:22:57] PROBLEM - puppet111 Current Load on puppet111 is CRITICAL: CRITICAL - load average: 6.05, 4.01, 2.55 [15:23:12] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.41, 3.33, 3.54 [15:23:14] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.42, 3.58, 3.82 [15:24:57] RECOVERY - puppet111 Current Load on puppet111 is OK: OK - load average: 2.78, 3.54, 2.57 [15:25:11] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.69, 2.97, 3.37 [15:26:13] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.83, 11.24, 9.82 [15:27:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.80, 3.47, 3.69 [15:28:13] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.97, 11.72, 10.16 [15:29:26] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.42, 9.58, 9.93 [15:29:27] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/commit/6d66f183d5cc [15:29:28] [02miraheze/ManageWiki] 07Universal-Omega 036d66f18 - Combine addModuleStyles [15:29:30] [02ManageWiki] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/ManageWiki [15:29:31] [02ManageWiki] 07Universal-Omega opened pull request 03#356: Combine addModuleStyles - 13https://github.com/miraheze/ManageWiki/pull/356 [15:30:13] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.03, 10.75, 10.00 [15:30:23] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/6d66f183d5cc...27fd3918fbe2 [15:30:25] [02miraheze/ManageWiki] 07Universal-Omega 0327fd391 - Update SpecialManageWiki.php [15:30:26] [02ManageWiki] 07Universal-Omega synchronize pull request 03#356: Combine addModuleStyles - 13https://github.com/miraheze/ManageWiki/pull/356 [15:31:14] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.60, 2.86, 3.38 [15:32:13] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.82, 10.20, 9.89 [15:32:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.95, 3.53, 3.96 [15:32:38] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/DataDump/commit/002bc69a088f [15:32:39] [02miraheze/DataDump] 07Universal-Omega 03002bc69 - Make addModuleStyles an array. [15:32:41] [02DataDump] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/DataDump [15:32:42] [02DataDump] 07Universal-Omega opened pull request 03#49: Make addModuleStyles an array. - 13https://github.com/miraheze/DataDump/pull/49 [15:33:06] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 3.55, 4.05, 3.74 [15:33:22] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.09, 10.85, 10.39 [15:33:50] [02miraheze/IncidentReporting] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/IncidentReporting/compare/a312a4812766...8e7e26d977cd [15:33:52] [02miraheze/IncidentReporting] 07Universal-Omega 038e7e26d - Update extension.json [15:33:54] [02IncidentReporting] 07Universal-Omega synchronize pull request 03#40: Modernise IncidentReportingOOUIForm - 13https://github.com/miraheze/IncidentReporting/pull/40 [15:34:06] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.37, 10.42, 9.79 [15:34:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.84, 4.35, 4.23 [15:35:05] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.11, 3.86, 3.72 [15:35:43] miraheze/CreateWiki - Universal-Omega the build passed. [15:36:01] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.86, 9.49, 9.50 [15:36:27] miraheze/ManageWiki - Universal-Omega the build passed. [15:38:13] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.42, 10.86, 10.25 [15:38:50] miraheze/DataDump - Universal-Omega the build passed. [15:39:22] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.47, 10.02, 10.17 [15:40:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.80, 3.57, 3.92 [15:40:33] miraheze/IncidentReporting - Universal-Omega the build passed. [15:41:09] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.12, 10.33, 9.70 [15:43:00] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.88, 3.05, 3.39 [15:43:49] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.41, 10.71, 9.85 [15:44:14] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.18, 2.80, 2.00 [15:44:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.44, 3.91, 3.96 [15:45:04] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.58, 9.46, 9.55 [15:45:49] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.83, 10.02, 9.70 [15:46:13] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.47, 9.37, 9.87 [15:46:57] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.43, 3.28, 3.39 [15:48:54] [02miraheze/IncidentReporting] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/IncidentReporting/compare/8e7e26d977cd...3c7d81ad9ace [15:48:55] [02miraheze/IncidentReporting] 07Universal-Omega 033c7d81a - Update SpecialIncidentReports.php [15:48:56] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.41, 3.51, 3.45 [15:48:57] [02IncidentReporting] 07Universal-Omega synchronize pull request 03#40: Modernise IncidentReportingOOUIForm - 13https://github.com/miraheze/IncidentReporting/pull/40 [15:49:38] [02miraheze/IncidentReporting] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/IncidentReporting/compare/3c7d81ad9ace...5b751270dfea [15:49:39] [02miraheze/IncidentReporting] 07Universal-Omega 035b75127 - Update extension.json [15:49:41] [02IncidentReporting] 07Universal-Omega synchronize pull request 03#40: Modernise IncidentReportingOOUIForm - 13https://github.com/miraheze/IncidentReporting/pull/40 [15:50:05] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/ae74ab6781f1...cb13c05cf2de [15:50:07] [02miraheze/ImportDump] 07Universal-Omega 03cb13c05 - Update extension.json [15:50:08] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [15:50:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:52:12] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/cb13c05cf2de...be5630ce4368 [15:52:13] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.70, 1.80, 1.81 [15:52:13] [02miraheze/ImportDump] 07Universal-Omega 03be5630c - Update ImportDumpRequestViewer.php [15:52:15] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [15:52:59] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/be5630ce4368...eb4231ba1678 [15:53:01] [02miraheze/ImportDump] 07Universal-Omega 03eb4231b - Update SpecialImportDumpRequestQueue.php [15:53:02] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [15:54:13] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.73, 1.36, 1.64 [15:54:42] miraheze/IncidentReporting - Universal-Omega the build passed. [15:54:52] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.75, 3.48, 3.51 [15:55:44] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.08, 2.04, 1.30 [15:57:44] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.10, 1.75, 1.28 [15:58:26] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/eb4231ba1678...c8c4f3b8e7b3 [15:58:27] [02miraheze/ImportDump] 07Universal-Omega 03c8c4f3b - Update SpecialImportDumpRequestQueue.php [15:58:29] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [15:58:30] miraheze/ImportDump - Universal-Omega the build passed. [15:58:50] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.02, 3.72, 3.59 [15:59:44] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.44, 1.33, 1.19 [16:00:29] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/c8c4f3b8e7b3...46ff785513ef [16:00:30] [02miraheze/ImportDump] 07Universal-Omega 0346ff785 - Update ImportDumpRequestViewer.php [16:00:37] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [16:00:49] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.99, 3.82, 3.64 [16:01:24] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.33, 4.27, 3.69 [16:02:16] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/46ff785513ef...40dd470fc4a9 [16:02:17] [02miraheze/ImportDump] 07Universal-Omega 0340dd470 - Update SpecialImportDumpRequestQueue.php [16:02:19] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [16:02:28] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.98, 1.72, 1.42 [16:02:48] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.97, 4.75, 4.01 [16:05:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:06:26] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.40, 1.68, 1.48 [16:07:36] miraheze/ImportDump - Universal-Omega the build passed. [16:15:02] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:18:21] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.10, 1.64, 1.44 [16:19:02] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.54, 1.99, 1.32 [16:20:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:20:20] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.22, 1.41, 1.37 [16:21:02] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.13, 1.71, 1.31 [16:23:02] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 0.96, 1.44, 1.26 [16:23:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.29, 3.27, 3.95 [16:25:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:26:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.98, 3.60, 3.94 [16:28:47] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.02, 3.68, 3.93 [16:32:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.37, 3.31, 3.74 [16:33:39] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/40dd470fc4a9...2b5f12bd53d3 [16:33:41] [02miraheze/ImportDump] 07Universal-Omega 032b5f12b - Update SpecialRequestImportDump.php [16:33:42] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [16:35:14] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.35, 2.77, 3.37 [16:35:28] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/2b5f12bd53d3...76ed4a418028 [16:35:29] [02miraheze/ImportDump] 07Universal-Omega 0376ed4a4 - Update SpecialRequestImportDump.php [16:35:31] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [16:36:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.44, 3.22, 3.94 [16:38:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.75, 3.49, 3.93 [16:40:11] miraheze/ImportDump - Universal-Omega the build passed. [16:40:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.54, 3.46, 3.87 [16:41:22] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 15.20, 11.43, 9.75 [16:42:08] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.32, 3.69, 3.56 [16:42:59] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/76ed4a418028...dc659373631f [16:43:01] [02miraheze/ImportDump] 07Universal-Omega 03dc65937 - Update SpecialRequestImportDump.php [16:43:02] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [16:43:22] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.71, 10.33, 9.56 [16:44:27] [02mw-config] 07Universal-Omega commented on pull request 03#4617: $wmgSharedUploadClientDBname for gpcommonswiki - 13https://github.com/miraheze/mw-config/pull/4617#issuecomment-1105457813 [16:46:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.96, 3.41, 3.66 [16:46:47] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.04, 2.99, 3.33 [16:47:07] miraheze/ImportDump - Universal-Omega the build passed. [16:47:22] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.16, 9.90, 9.58 [16:47:55] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.36, 3.14, 3.39 [16:48:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.84, 3.26, 3.58 [16:49:23] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.77, 1.61, 1.33 [16:49:49] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.33, 9.45, 8.76 [16:51:16] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.69, 2.02, 1.61 [16:51:19] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 0.41, 1.19, 1.21 [16:52:36] [02mw-config] 07ugochimobi opened pull request 03#4618: Update LocalSettings.php - 13https://github.com/miraheze/mw-config/pull/4618 [16:53:02] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:53:14] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.10, 1.79, 1.58 [16:53:43] miraheze/mw-config - ugochimobi the build passed. [16:53:49] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.74, 8.86, 8.76 [16:55:12] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.81, 1.46, 1.48 [16:56:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.89, 2.65, 3.25 [16:56:36] [02mw-config] 07ugochimobi commented on pull request 03#4617: $wmgSharedUploadClientDBname for gpcommonswiki - 13https://github.com/miraheze/mw-config/pull/4617#issuecomment-1105469236 [16:58:02] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [17:00:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.67, 3.30, 3.39 [17:00:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.98, 3.32, 3.19 [17:01:55] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/dc659373631f...55d7062db311 [17:01:56] [02miraheze/ImportDump] 07Universal-Omega 0355d7062 - Update ImportDumpRequestViewer.php [17:01:58] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [17:02:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.12, 3.67, 3.38 [17:02:47] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.87, 3.13, 3.14 [17:04:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.63, 3.73, 3.54 [17:05:12] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.69, 9.30, 8.81 [17:05:49] miraheze/ImportDump - Universal-Omega the build passed. [17:06:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.54, 3.64, 3.36 [17:07:12] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.10, 8.01, 8.39 [17:08:15] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.02, 3.62, 3.50 [17:08:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.41, 3.55, 3.57 [17:09:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [17:10:11] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.26, 3.85, 3.60 [17:10:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.29, 3.91, 3.70 [17:10:47] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.28, 3.28, 3.26 [17:11:04] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.32, 3.31, 2.12 [17:12:06] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.42, 3.71, 3.58 [17:12:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.55, 3.53, 3.60 [17:14:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [17:15:53] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [17:15:58] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.02, 2.83, 3.25 [17:16:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.93, 3.03, 3.40 [17:17:04] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.66, 1.90, 1.82 [17:20:55] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.35, 1.74, 1.48 [17:21:04] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.97, 1.45, 1.66 [17:24:53] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.22, 1.61, 1.50 [17:26:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.65, 3.38, 3.31 [17:29:39] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/55d7062db311...fd0edcf861f1 [17:29:41] [02miraheze/ImportDump] 07Universal-Omega 03fd0edcf - Implement editing functionality [17:29:42] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [17:30:40] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/fd0edcf861f1...d1b471c7e0fe [17:30:41] [02miraheze/ImportDump] 07Universal-Omega 03d1b471c - Implement saving edits [17:30:43] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [17:32:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [17:34:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.13, 3.70, 3.52 [17:35:55] miraheze/ImportDump - Universal-Omega the build passed. [17:37:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [17:38:30] [02ImportDump] 07Universal-Omega edited pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [17:44:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [17:44:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.06, 3.84, 3.81 [17:44:45] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [17:49:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [17:50:01] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.22, 6.30, 5.30 [17:51:29] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [17:51:31] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/d1b471c7e0fe...5669033ce0b7 [17:51:32] [02miraheze/ImportDump] 07Universal-Omega 035669033 - Update .eslintrc.json [17:52:01] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.90, 6.09, 5.34 [17:52:41] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.53, 1.90, 1.58 [17:55:21] miraheze/ImportDump - Universal-Omega the build passed. [17:56:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.27, 3.02, 3.38 [17:56:41] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.31, 1.90, 1.68 [17:57:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [17:58:41] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.17, 1.60, 1.59 [18:02:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [18:11:06] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.64, 3.25, 3.17 [18:12:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [18:13:03] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.29, 3.16, 3.14 [18:15:53] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [18:15:53] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [18:17:02] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.73, 2.00, 1.48 [18:17:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [18:20:45] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.51, 3.70, 3.43 [18:22:04] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [18:24:40] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.17, 3.78, 3.52 [18:26:37] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.49, 3.42, 3.40 [18:26:41] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.12, 1.76, 1.56 [18:28:34] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.62, 3.97, 3.59 [18:29:02] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.89, 2.00, 1.83 [18:29:17] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 3.94, 4.07, 3.33 [18:30:09] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.91, 3.79, 3.08 [18:30:41] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.24, 1.58, 1.53 [18:31:16] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.40, 3.84, 3.33 [18:32:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [18:32:04] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.86, 3.13, 2.93 [18:32:28] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.07, 3.61, 3.53 [18:32:45] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/5669033ce0b7...48a867baa23e [18:32:46] [02miraheze/ImportDump] 07Universal-Omega 0348a867b - Implement updating status functionality [18:32:48] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [18:34:26] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.41, 3.87, 3.63 [18:36:23] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.40, 3.85, 3.67 [18:36:40] miraheze/ImportDump - Universal-Omega the build passed. [18:38:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.63, 4.57, 3.93 [18:39:02] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 3.34, 2.51, 2.08 [18:39:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.21, 3.72, 3.42 [18:40:49] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.48, 3.90, 3.37 [18:41:10] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.82, 3.54, 3.38 [18:42:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [18:42:45] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.31, 4.02, 3.47 [18:43:09] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.71, 3.36, 3.34 [18:44:40] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.41, 3.69, 3.40 [18:46:21] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/48a867baa23e...b215513b1cb2 [18:46:23] [02miraheze/ImportDump] 07Universal-Omega 03b215513 - Implement status updating/handling [18:46:24] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [18:46:36] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.12, 3.60, 3.39 [18:47:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.44, 3.58, 3.44 [18:48:32] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.08, 3.28, 3.29 [18:49:05] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.69, 3.30, 3.36 [18:50:04] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 3 seconds ago with 0 failures [18:50:51] miraheze/ImportDump - Universal-Omega the build passed. [18:50:56] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/b215513b1cb2...b11b8cdf00b0 [18:50:58] [02miraheze/ImportDump] 07Universal-Omega 03b11b8cd - Update ImportDumpRequestViewer.php [18:50:59] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [18:51:13] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.56, 10.15, 9.39 [18:53:56] [02ImportDump] 07Universal-Omega edited pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [18:54:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [18:55:12] [02ImportDump] 07Universal-Omega edited pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [18:55:14] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.02, 11.33, 10.10 [18:55:38] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 1.93, 2.09, 1.77 [18:55:54] miraheze/ImportDump - Universal-Omega the build passed. [18:58:59] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.46, 3.56, 3.40 [18:59:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [18:59:36] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.27, 1.86, 1.77 [19:00:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.81, 3.52, 3.93 [19:00:58] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.00, 3.35, 3.34 [19:01:14] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.41, 11.01, 10.26 [19:01:36] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.13, 2.06, 1.85 [19:03:13] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.87, 10.09, 10.04 [19:03:35] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.79, 1.93, 1.83 [19:05:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:05:08] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [19:05:10] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/b11b8cdf00b0...02cfea96eac9 [19:05:11] [02miraheze/ImportDump] 07Universal-Omega 0302cfea9 - Update ImportDumpRequestViewer.php [19:05:34] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.84, 1.54, 1.70 [19:06:20] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.65, 10.35, 9.69 [19:08:15] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 13.32, 11.07, 10.00 [19:08:58] miraheze/ImportDump - Universal-Omega the build passed. [19:10:13] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.62, 10.86, 10.07 [19:11:02] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.40, 1.71, 1.95 [19:12:13] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.71, 9.62, 9.70 [19:12:52] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [19:12:53] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/02cfea96eac9...87c37e88548d [19:12:55] [02miraheze/ImportDump] 07Universal-Omega 0387c37e8 - Update ImportDumpRequestViewer.php [19:13:53] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [19:14:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.63, 3.40, 3.55 [19:14:56] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/87c37e88548d...dff4440ea7ac [19:14:58] [02miraheze/ImportDump] 07Universal-Omega 03dff4440 - Update en.json [19:14:59] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [19:15:11] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/dff4440ea7ac...4794218f93f2 [19:15:12] [02miraheze/ImportDump] 07Universal-Omega 034794218 - Update qqq.json [19:15:14] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [19:17:02] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.20, 1.83, 1.91 [19:18:58] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [19:19:00] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/4794218f93f2...d080c7ca0dca [19:19:01] [02miraheze/ImportDump] 07Universal-Omega 03d080c7c - Update ImportDumpRequestViewer.php [19:19:25] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/d080c7ca0dca...ca047da11c51 [19:19:26] [02miraheze/ImportDump] 07Universal-Omega 03ca047da - Update qqq.json [19:19:28] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [19:19:35] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/ca047da11c51...110771fb6327 [19:19:36] [02miraheze/ImportDump] 07Universal-Omega 03110771f - Update en.json [19:19:38] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [19:19:52] miraheze/ImportDump - Universal-Omega the build passed. [19:22:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.65, 3.58, 3.15 [19:23:55] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/110771fb6327...426b86538d6a [19:23:56] [02miraheze/ImportDump] 07Universal-Omega 03426b865 - Update qqq.json [19:23:58] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [19:24:00] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/426b86538d6a...93fd3a188022 [19:24:01] [02miraheze/ImportDump] 07Universal-Omega 0393fd3a1 - Update en.json [19:24:03] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [19:24:04] miraheze/ImportDump - Universal-Omega the build passed. [19:24:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.43, 3.54, 3.85 [19:25:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:27:25] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.93, 1.82, 1.61 [19:29:05] miraheze/ImportDump - Universal-Omega the build passed. [19:29:24] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.19, 1.50, 1.51 [19:30:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:32:54] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.94, 10.65, 9.88 [19:33:22] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.73, 1.77, 1.63 [19:34:01] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.64, 2.89, 3.08 [19:34:50] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.59, 9.92, 9.71 [19:36:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.60, 2.88, 3.40 [19:37:21] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.15, 2.04, 1.77 [19:39:20] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.24, 1.76, 1.70 [19:41:19] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.19, 1.57, 1.63 [19:44:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.40, 3.64, 3.51 [19:48:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.55, 3.32, 3.44 [19:50:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.37, 3.18, 3.36 [19:50:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:51:40] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/93fd3a188022...f159633a996f [19:51:42] [02miraheze/ImportDump] 07Universal-Omega 03f159633 - Update ImportDumpRequestViewer.php [19:51:43] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [19:52:13] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 1.55, 3.43, 2.28 [19:53:22] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.42, 3.47, 3.14 [19:53:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.60, 3.92, 3.33 [19:54:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.95, 4.20, 3.79 [19:55:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.10, 3.28, 3.11 [19:55:38] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.26, 3.28, 3.16 [19:55:45] miraheze/ImportDump - Universal-Omega the build passed. [19:56:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.57, 3.86, 3.71 [19:57:13] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.30, 1.74, 1.62 [19:57:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.34, 9.74, 9.17 [19:57:57] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.47, 9.76, 9.50 [19:58:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.24, 4.09, 3.82 [19:59:12] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.35, 1.55, 1.55 [19:59:49] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 10.00, 9.90, 9.30 [20:00:13] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 0.75, 1.57, 1.83 [20:00:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.47, 3.75, 3.72 [20:01:47] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.71, 9.86, 9.64 [20:03:29] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/f159633a996f...7a156dfd945b [20:03:31] [02miraheze/ImportDump] 07Universal-Omega 037a156df - Update ImportDumpRequestQueuePager.php [20:03:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [20:03:32] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [20:07:49] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/7a156dfd945b...bfd84839c861 [20:07:50] [02miraheze/ImportDump] 07Universal-Omega 03bfd8483 - Update ImportDumpRequestQueuePager.php [20:07:52] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [20:08:11] miraheze/ImportDump - Universal-Omega the build passed. [20:08:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.44, 4.05, 3.78 [20:08:41] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/bfd84839c861...91d35579ccdf [20:08:43] [02miraheze/ImportDump] 07Universal-Omega 0391d3557 - Update ImportDumpRequestViewer.php [20:08:44] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [20:10:13] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.82, 1.42, 1.66 [20:12:05] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/91d35579ccdf...c79cc28e9021 [20:12:07] [02miraheze/ImportDump] 07Universal-Omega 03c79cc28 - Update SpecialImportDumpRequestQueue.php [20:12:08] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [20:12:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.30, 3.23, 3.51 [20:15:41] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/c79cc28e9021...8129b3f7e91f [20:15:42] [02miraheze/ImportDump] 07Universal-Omega 038129b3f - Update ImportDumpRequestViewer.php [20:15:44] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [20:15:55] miraheze/ImportDump - Universal-Omega the build passed. [20:16:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.26, 3.89, 3.66 [20:17:04] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 1.84, 3.89, 2.69 [20:18:35] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/8129b3f7e91f...f9895e43790c [20:18:37] [02miraheze/ImportDump] 07Universal-Omega 03f9895e4 - Update ImportDumpRequestViewer.php [20:18:38] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [20:22:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.34, 3.96, 3.83 [20:22:32] miraheze/ImportDump - Universal-Omega the build passed. [20:27:02] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 0.55, 1.24, 1.87 [20:31:02] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 0.89, 1.01, 1.62 [20:32:59] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 0.42, 0.97, 1.82 [20:33:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.46, 3.46, 3.33 [20:33:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [20:33:49] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.73, 9.73, 9.10 [20:35:10] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.90, 3.05, 3.18 [20:35:43] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.19, 9.25, 9.01 [20:36:57] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.84, 0.90, 1.60 [20:37:11] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/f9895e43790c...8c13f9271ede [20:37:12] [02miraheze/ImportDump] 07Universal-Omega 038c13f92 - Update ImportDumpRequestViewer.php [20:37:14] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [20:37:31] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/8c13f9271ede...ba898783a9e6 [20:37:33] [02miraheze/ImportDump] 07Universal-Omega 03ba89878 - Update en.json [20:37:34] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [20:37:39] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/ba898783a9e6...9cf60d81a958 [20:37:40] [02miraheze/ImportDump] 07Universal-Omega 039cf60d8 - Update qqq.json [20:37:42] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [20:38:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.86, 2.86, 3.38 [20:39:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [20:42:31] miraheze/ImportDump - Universal-Omega the build passed. [20:43:43] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/9cf60d81a958...783fd8a08a01 [20:43:44] [02miraheze/ImportDump] 07Universal-Omega 03783fd8a - Update ImportDumpRequestQueuePager.php [20:43:46] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [20:44:09] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/783fd8a08a01...458b30b89a10 [20:44:11] [02miraheze/ImportDump] 07Universal-Omega 03458b30b - Update en.json [20:44:12] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [20:44:25] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/458b30b89a10...07127d325b78 [20:44:26] [02miraheze/ImportDump] 07Universal-Omega 0307127d3 - Update qqq.json [20:44:28] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [20:45:54] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [20:45:54] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [20:46:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.14, 4.18, 3.73 [20:48:02] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.56, 3.29, 3.11 [20:48:33] miraheze/ImportDump - Universal-Omega the build has errored. [20:50:00] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.66, 3.10, 3.06 [20:51:11] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/07127d325b78...735cee5be9f2 [20:51:12] [02miraheze/ImportDump] 07Universal-Omega 03735cee5 - Update ImportDumpRequestViewer.php [20:51:14] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [20:52:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.66, 3.68, 3.72 [20:54:57] miraheze/ImportDump - Universal-Omega the build passed. [20:56:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.10, 3.54, 3.63 [20:58:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.73, 3.32, 3.54 [20:59:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [21:01:46] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/735cee5be9f2...a7a31b080bc8 [21:01:47] [02miraheze/ImportDump] 07Universal-Omega 03a7a31b0 - Update en.json [21:01:49] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [21:01:55] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/a7a31b080bc8...54b4ddfa9246 [21:01:56] [02miraheze/ImportDump] 07Universal-Omega 0354b4ddf - Update qqq.json [21:01:58] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [21:02:08] [02ImportDump] 07Universal-Omega edited pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [21:02:10] [02ImportDump] 07Universal-Omega edited pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [21:02:12] [02ImportDump] 07Universal-Omega edited pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [21:02:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.83, 2.89, 3.30 [21:03:35] [02ImportDump] 07Universal-Omega edited pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [21:07:03] miraheze/ImportDump - Universal-Omega the build passed. [21:08:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.20, 3.47, 3.43 [21:09:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [21:12:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.55, 3.21, 2.92 [21:13:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.50, 3.18, 2.95 [21:14:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.43, 3.84, 3.57 [21:14:43] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [21:14:45] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/54b4ddfa9246...245599d149f8 [21:14:46] [02miraheze/ImportDump] 07Universal-Omega 03245599d - Update SpecialRequestImportDump.php [21:14:47] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.73, 3.55, 3.07 [21:15:13] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/245599d149f8...85d870d6a600 [21:15:14] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.95, 2.66, 2.78 [21:15:14] [02miraheze/ImportDump] 07Universal-Omega 0385d870d - Update extension.json [21:15:16] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [21:16:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.80, 3.51, 3.49 [21:18:12] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.71, 10.04, 9.09 [21:18:47] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.47, 3.06, 2.98 [21:19:44] miraheze/ImportDump - Universal-Omega the build passed. [21:20:10] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.41, 9.73, 9.09 [21:22:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.27, 3.47, 3.45 [21:24:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.81, 3.22, 3.36 [21:29:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [21:31:12] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.69, 3.96, 3.66 [21:32:42] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/85d870d6a600...27c7a9450918 [21:32:44] [02miraheze/ImportDump] 07Universal-Omega 0327c7a94 - Update SpecialRequestImportDump.php [21:32:45] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [21:33:09] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.63, 4.42, 3.86 [21:33:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.90, 10.14, 9.26 [21:34:11] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/27c7a9450918...b94fb393e92f [21:34:12] [02miraheze/ImportDump] 07Universal-Omega 03b94fb39 - Update ImportDumpRequestViewer.php [21:34:14] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [21:34:41] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/b94fb393e92f...6775b3f2af78 [21:34:42] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.95, 3.56, 3.25 [21:34:42] [02miraheze/ImportDump] 07Universal-Omega 036775b3f - Update ImportDumpRequestManager.php [21:34:44] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [21:35:49] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.47, 10.54, 9.48 [21:36:38] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.66, 3.28, 3.19 [21:37:43] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.72, 9.96, 9.41 [21:39:26] miraheze/ImportDump - Universal-Omega the build passed. [21:39:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [21:40:28] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.20, 1.70, 1.53 [21:41:01] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.53, 3.45, 3.69 [21:42:26] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.38, 1.61, 1.52 [21:42:54] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.03, 3.74, 3.77 [21:43:53] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [21:43:57] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/6775b3f2af78...060cea483581 [21:43:58] [02miraheze/ImportDump] 07Universal-Omega 03060cea4 - Update importdump_requests.sql [21:44:00] [02ImportDump] 07Universal-Omega synchronize pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [21:46:48] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.51, 3.60, 3.76 [21:48:06] miraheze/ImportDump - Universal-Omega the build passed. [21:49:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [21:50:05] [02ImportDump] 07Universal-Omega edited pull request 03#1: Create extension - 13https://github.com/miraheze/ImportDump/pull/1 [21:51:32] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.32, 3.47, 3.22 [21:55:30] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.68, 3.35, 3.24 [21:56:34] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.12, 2.95, 3.36 [22:08:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [22:09:10] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.79, 4.09, 3.64 [22:11:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.75, 3.61, 3.52 [22:11:33] PROBLEM - cloud10 Puppet on cloud10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [22:13:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [22:13:04] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.41, 3.23, 3.39 [22:15:53] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [22:15:53] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [22:16:55] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.46, 3.89, 3.60 [22:18:52] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.92, 3.48, 3.48 [22:23:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.41, 3.47, 3.09 [22:24:43] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.40, 3.07, 3.32 [22:25:10] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.41, 3.15, 3.02 [22:37:02] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.99, 3.36, 3.01 [22:39:01] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.98, 3.03, 2.92 [22:39:33] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 58 seconds ago with 0 failures [22:44:46] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [22:54:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.41, 3.16, 3.03 [22:56:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.27, 3.10, 3.02 [23:00:02] !log [@test101] starting deploy of {'l10nupdate': True} to all [23:00:03] !log [@mwtask111] starting deploy of {'l10nupdate': True} to all [23:00:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:00:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:02:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.82, 3.47, 3.24 [23:04:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.80, 3.23, 3.18 [23:13:53] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [23:13:53] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [23:17:02] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.08, 1.78, 1.33 [23:17:55] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 4.25, 3.23, 2.09 [23:20:47] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.29, 3.29, 2.84 [23:25:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.46, 3.20, 2.84 [23:25:55] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 2.81, 3.47, 2.75 [23:26:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.34, 3.61, 3.15 [23:27:35] !log [@mwtask111] finished deploy of {'l10nupdate': True} to all - SUCCESS in 1652s [23:27:39] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.84, 3.10, 2.85 [23:27:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:27:55] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 2.34, 3.05, 2.68 [23:29:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:30:47] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.97, 2.93, 3.00 [23:39:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:42:03] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.57, 10.06, 8.81 [23:42:18] !log [@test101] finished deploy of {'l10nupdate': True} to all - SUCCESS in 2536s [23:42:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:43:02] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.05, 1.76, 1.92 [23:43:53] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [23:44:00] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.28, 9.93, 8.92 [23:44:37] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.29, 3.34, 3.16 [23:46:08] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.94, 3.74, 3.25 [23:47:02] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.00, 1.17, 1.62 [23:48:04] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.85, 3.07, 3.06 [23:48:31] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.80, 2.87, 3.04 [23:53:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:56:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.71, 3.70, 3.23 [23:57:29] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 4.88, 2.54, 1.89 [23:58:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:58:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.81, 3.88, 3.37 [23:58:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.81, 3.80, 3.25 [23:59:41] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.27, 3.46, 3.06