[00:00:30] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.91, 10.10, 9.45 [00:02:30] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.03, 9.58, 9.33 [00:02:31] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.66, 2.09, 1.53 [00:02:36] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.82, 10.09, 10.12 [00:03:51] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.00, 2.08, 1.63 [00:04:29] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.25, 1.91, 1.54 [00:04:46] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:05:05] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:05:08] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:05:15] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:05:29] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:06:36] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.49, 11.03, 10.52 [00:08:26] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.07, 1.49, 1.46 [00:08:36] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.07, 11.18, 10.62 [00:09:45] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.89, 1.88, 1.75 [00:10:36] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.03, 10.74, 10.53 [00:10:42] RECOVERY - cloud12 Puppet on cloud12 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:11:19] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14640 bytes in 8.083 second response time [00:11:27] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14626 bytes in 9.043 second response time [00:11:29] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14634 bytes in 8.032 second response time [00:11:43] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.31, 1.68, 1.69 [00:11:51] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14634 bytes in 3.267 second response time [00:13:01] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 22336 bytes in 0.397 second response time [00:15:40] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [00:18:36] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.30, 10.76, 10.56 [00:20:06] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.97, 1.61, 1.29 [00:20:36] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.56, 10.94, 10.65 [00:22:05] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.36, 1.58, 1.32 [00:23:42] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:23:49] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:24:44] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:24:52] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:25:17] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:25:45] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.08, 10.44, 9.51 [00:31:10] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.82, 9.32, 9.35 [00:31:15] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 22336 bytes in 3.095 second response time [00:31:19] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14626 bytes in 3.196 second response time [00:31:25] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14626 bytes in 2.757 second response time [00:31:32] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14640 bytes in 3.180 second response time [00:31:42] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14634 bytes in 0.464 second response time [00:35:06] PROBLEM - es101 Current Load on es101 is CRITICAL: CRITICAL - load average: 4.41, 3.48, 2.89 [00:35:46] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:37:06] PROBLEM - es101 Current Load on es101 is WARNING: WARNING - load average: 3.93, 3.70, 3.04 [00:37:45] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14640 bytes in 2.184 second response time [00:40:49] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.74, 1.55, 1.48 [00:41:06] RECOVERY - es101 Current Load on es101 is OK: OK - load average: 2.67, 3.23, 3.01 [00:41:38] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.41, 9.81, 9.28 [00:42:49] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.49, 1.49, 1.47 [00:43:35] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.47, 9.61, 9.26 [00:45:13] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.66, 8.53, 9.79 [00:47:27] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.32, 10.47, 9.64 [00:49:25] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.81, 9.68, 9.45 [00:50:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.31, 3.21, 3.90 [00:51:03] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.22, 1.67, 1.35 [00:51:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.48, 10.05, 10.03 [00:52:59] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.74, 1.73, 1.42 [00:54:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.25, 3.21, 3.70 [00:54:55] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.55, 1.70, 1.45 [00:55:03] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.85, 10.10, 10.07 [00:58:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.60, 3.61, 3.76 [01:02:11] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.81, 9.57, 8.90 [01:03:53] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.99, 11.44, 10.62 [01:04:07] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.91, 11.07, 9.53 [01:06:03] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.21, 10.79, 9.59 [01:06:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.01, 3.21, 3.46 [01:07:49] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.16, 11.19, 10.77 [01:08:03] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.21, 10.14, 9.50 [01:08:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.89, 3.29, 3.45 [01:10:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.02, 3.42, 3.47 [01:12:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.66, 3.17, 3.38 [01:13:42] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [01:13:42] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [01:14:07] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.77, 1.85, 1.43 [01:16:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:16:06] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.97, 1.51, 1.35 [01:17:41] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.84, 8.79, 9.87 [01:21:04] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.23, 1.88, 1.63 [01:21:36] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.22, 9.42, 8.94 [01:21:36] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.17, 10.18, 10.21 [01:21:54] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.98, 6.47, 5.97 [01:22:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.45, 3.56, 3.35 [01:23:00] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.61, 1.74, 1.61 [01:23:31] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 13.00, 10.48, 9.37 [01:23:34] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.35, 11.24, 10.59 [01:23:54] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.37, 6.10, 5.89 [01:24:31] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.82, 3.71, 4.00 [01:24:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:24:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.59, 3.29, 3.29 [01:24:56] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.32, 1.59, 1.58 [01:25:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.84, 11.01, 9.70 [01:25:32] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.82, 11.12, 10.61 [01:26:30] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.29, 4.01, 4.08 [01:27:21] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.72, 10.05, 9.50 [01:28:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.78, 3.71, 3.98 [01:28:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.77, 3.67, 3.49 [01:30:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.06, 3.10, 3.30 [01:32:31] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.05, 4.02, 4.04 [01:32:38] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.95, 11.42, 10.41 [01:34:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.70, 3.96, 3.65 [01:34:44] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.79, 1.77, 1.67 [01:35:29] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.61, 11.35, 10.80 [01:36:30] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.75, 9.76, 10.05 [01:36:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.20, 4.13, 3.75 [01:37:28] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.37, 11.40, 10.89 [01:38:36] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.23, 1.59, 1.63 [01:41:09] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.30, 1.85, 1.54 [01:43:03] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [01:43:04] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.73, 1.81, 1.56 [01:43:28] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.31, 11.46, 11.06 [01:44:59] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.85, 2.41, 1.82 [01:45:28] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.67, 10.25, 10.68 [01:46:23] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.82, 1.78, 1.70 [01:46:54] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.03, 1.88, 1.69 [01:48:19] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.26, 1.93, 1.77 [01:48:49] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.93, 1.70, 1.66 [01:50:14] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 2.00, 1.96, 1.81 [01:50:59] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.77, 3.94, 2.45 [01:53:32] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.76, 9.56, 10.16 [01:54:06] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.42, 1.54, 1.66 [01:57:29] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.83, 10.29, 10.33 [01:58:58] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.02, 1.78, 1.94 [02:01:04] [02mw-config] 07ugochimobi opened pull request 03#4625: set Tabular.JsonConfig and Map.JsonConfig local storing to true on gpcommonswiki - 13https://github.com/miraheze/mw-config/pull/4625 [02:02:04] miraheze/mw-config - ugochimobi the build passed. [02:02:54] PROBLEM - mabi.myomi.net - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolve(hos [02:02:54] AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.406557083129883 seconds [02:03:11] [02mw-config] 07ugochimobi opened pull request 03#4626: set JsonConfig configuration for gpcommonswiki - 13https://github.com/miraheze/mw-config/pull/4626 [02:03:31] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 5.58, 3.32, 2.22 [02:04:40] miraheze/mw-config - ugochimobi the build has errored. [02:04:57] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.32, 1.37, 1.70 [02:05:28] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.31, 11.01, 10.62 [02:07:28] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.76, 10.83, 10.61 [02:08:51] [02mw-config] 07ugochimobi synchronize pull request 03#4626: set JsonConfig configuration for gpcommonswiki - 13https://github.com/miraheze/mw-config/pull/4626 [02:08:56] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.95, 1.75, 1.80 [02:09:49] miraheze/mw-config - ugochimobi the build passed. [02:10:59] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.66, 10.24, 8.70 [02:12:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.13, 1.37, 1.62 [02:12:59] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.60, 9.25, 8.53 [02:13:07] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.63, 1.85, 1.99 [02:14:36] [02mw-config] 07ugochimobi reviewed pull request 03#4624 commit - 13https://github.com/miraheze/mw-config/pull/4624#discussion_r856797017 [02:15:02] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.08, 1.83, 1.96 [02:15:40] [02mw-config] 07Universal-Omega reviewed pull request 03#4624 commit - 13https://github.com/miraheze/mw-config/pull/4624#discussion_r856797179 [02:16:57] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.01, 1.58, 1.86 [02:19:50] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.94, 1.62, 1.43 [02:20:48] [02mw-config] 07ugochimobi reviewed pull request 03#4624 commit - 13https://github.com/miraheze/mw-config/pull/4624#discussion_r856806757 [02:21:50] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.29, 1.54, 1.43 [02:22:49] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.76, 1.18, 1.59 [02:24:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:30:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:42:15] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.57, 10.21, 9.33 [02:42:16] PROBLEM - cloud12 Puppet on cloud12 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [02:44:15] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.80, 10.07, 9.39 [02:45:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:49:28] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.37, 11.23, 10.74 [02:50:29] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.19, 1.84, 1.55 [02:51:21] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.08, 11.08, 9.94 [02:51:29] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.40, 11.33, 10.85 [02:52:25] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.32, 1.67, 1.52 [02:53:16] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.43, 10.85, 9.99 [02:54:44] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.50, 10.63, 9.61 [02:55:11] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.94, 9.93, 9.76 [02:56:40] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.34, 9.52, 9.34 [03:00:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:02:01] RECOVERY - mabi.myomi.net - reverse DNS on sslhost is OK: SSL OK - mabi.myomi.net reverse DNS resolves to cp21.miraheze.org - CNAME OK [03:02:32] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.07, 10.45, 9.72 [03:04:28] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.99, 9.47, 9.44 [03:05:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:07:28] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.66, 9.60, 10.17 [03:10:15] RECOVERY - cloud12 Puppet on cloud12 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [03:12:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.81, 3.23, 3.87 [03:12:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:16:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.61, 3.38, 3.94 [03:18:30] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 7.02, 4.53, 4.11 [03:18:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.82, 4.06, 4.12 [03:19:50] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.55, 2.02, 1.66 [03:20:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.56, 10.69, 10.15 [03:20:43] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.79, 9.88, 9.06 [03:21:50] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.85, 1.95, 1.68 [03:22:18] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.97, 10.04, 9.99 [03:22:43] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.17, 9.26, 8.91 [03:27:41] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.16, 1.86, 1.58 [03:27:50] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.45, 1.65, 1.64 [03:31:31] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.55, 1.88, 1.66 [03:33:26] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.84, 2.18, 1.79 [03:33:48] PROBLEM - mabi.myomi.net - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolve(hos [03:33:48] AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.40665078163147 seconds [03:35:22] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.02, 1.70, 1.66 [03:36:03] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.82, 11.42, 10.44 [03:38:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.38, 10.76, 10.31 [03:42:15] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.07, 10.16, 9.26 [03:45:50] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.54, 1.74, 1.61 [03:46:15] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.49, 9.53, 9.30 [03:47:50] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.62, 1.66, 1.59 [03:51:50] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.70, 1.76, 1.65 [03:52:49] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.66, 1.74, 1.51 [03:53:50] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.24, 1.57, 1.60 [03:54:49] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.17, 1.51, 1.46 [03:55:30] [02mw-config] 07Universal-Omega edited pull request 03#4621: Add Wikibase configuration for gratisdatawiki, gratispaideiawiki, gpcommonswiki, benpediawiki - 13https://github.com/miraheze/mw-config/pull/4621 [03:56:11] [02mw-config] 07Universal-Omega closed pull request 03#4621: Add Wikibase configuration for gratisdatawiki, gratispaideiawiki, gpcommonswiki, benpediawiki - 13https://github.com/miraheze/mw-config/pull/4621 [03:56:12] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/45d157083695...ed09091745bc [03:56:14] [02miraheze/mw-config] 07ugochimobi 03ed09091 - Add Wikibase configuration for gratisdatawiki, gratispaideiawiki, gpcommonswiki, benpediawiki (#4621) [03:56:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.13, 3.28, 4.00 [03:57:13] miraheze/mw-config - Universal-Omega the build passed. [03:58:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.40, 3.43, 3.96 [03:58:45] [02mw-config] 07Universal-Omega reviewed pull request 03#4626 commit - 13https://github.com/miraheze/mw-config/pull/4626#discussion_r856838776 [03:59:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.32, 9.56, 10.16 [04:00:30] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.52, 4.00, 4.11 [04:02:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:02:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.37, 3.93, 4.02 [04:03:13] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.39, 10.58, 9.55 [04:04:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.75, 3.84, 3.98 [04:04:43] [02mw-config] 07Universal-Omega synchronize pull request 03#4625: set Tabular.JsonConfig and Map.JsonConfig local storing to true on gpcommonswiki - 13https://github.com/miraheze/mw-config/pull/4625 [04:05:09] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.09, 9.70, 9.36 [04:05:12] [02mw-config] 07Universal-Omega closed pull request 03#4625: set Tabular.JsonConfig and Map.JsonConfig local storing to true on gpcommonswiki - 13https://github.com/miraheze/mw-config/pull/4625 [04:05:13] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/ed09091745bc...dcf55ddedccf [04:05:15] [02miraheze/mw-config] 07ugochimobi 03dcf55dd - set Tabular.JsonConfig and Map.JsonConfig local storing to true on gpcommonswiki (#4625) [04:05:50] miraheze/mw-config - Universal-Omega the build passed. [04:06:08] [02mw-config] 07Universal-Omega reviewed pull request 03#4624 commit - 13https://github.com/miraheze/mw-config/pull/4624#discussion_r856839628 [04:06:29] miraheze/mw-config - Universal-Omega the build passed. [04:08:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.70, 3.56, 3.91 [04:10:30] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.83, 4.14, 4.06 [04:12:43] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [04:12:43] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [04:14:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.96, 3.95, 3.99 [04:20:26] !log [@test101] starting deploy of {'config': True} to all [04:20:27] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 1s [04:20:28] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.62, 2.61, 2.06 [04:20:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:20:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:20:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.49, 2.87, 3.35 [04:21:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.79, 10.22, 9.84 [04:21:36] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.73, 1.35, 1.12 [04:21:56] !log [@mwtask111] starting deploy of {'config': True} to all [04:22:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:22:04] !log [@mwtask111] finished deploy of {'config': True} to all - SUCCESS in 8s [04:22:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:23:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:23:18] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.80, 9.60, 9.65 [04:23:35] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.26, 1.22, 1.10 [04:28:30] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.17, 3.62, 3.62 [04:28:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.54, 3.28, 3.43 [04:30:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.80, 3.60, 3.61 [04:32:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.04, 3.23, 3.38 [04:32:56] RECOVERY - mabi.myomi.net - reverse DNS on sslhost is OK: SSL OK - mabi.myomi.net reverse DNS resolves to cp21.miraheze.org - CNAME OK [04:33:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:34:00] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 0.97, 1.62, 1.89 [04:34:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.30, 3.38, 3.90 [04:38:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.91, 3.83, 3.56 [04:38:49] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.81, 4.18, 4.08 [04:40:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.04, 3.49, 3.47 [04:41:50] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.71, 2.00, 1.92 [04:44:30] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 3.60, 4.01, 3.82 [04:46:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.22, 3.70, 3.72 [04:47:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:48:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.22, 3.12, 3.34 [04:52:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:52:30] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.49, 2.96, 3.37 [04:52:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.06, 3.64, 3.98 [04:53:50] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.43, 1.90, 1.99 [04:58:49] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.10, 3.55, 3.82 [04:59:50] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.16, 1.80, 1.89 [05:02:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.73, 3.35, 3.69 [05:03:50] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.82, 1.84, 1.89 [05:04:41] PROBLEM - mabi.myomi.net - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolve(hos [05:04:41] AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.4048285484313965 seconds [05:04:49] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.92, 3.90, 3.84 [05:06:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.81, 3.91, 3.86 [05:11:26] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [05:13:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.73, 3.75, 3.31 [05:14:49] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.65, 4.00, 3.80 [05:16:33] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.10, 3.54, 3.29 [05:17:32] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.61, 3.80, 3.44 [05:17:50] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.23, 1.80, 1.79 [05:18:30] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.71, 3.23, 3.21 [05:18:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:18:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.52, 3.91, 3.81 [05:20:49] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.67, 4.09, 3.88 [05:21:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.83, 3.84, 3.51 [05:21:50] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.58, 1.85, 1.83 [05:22:30] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.65, 4.38, 3.69 [05:26:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.84, 3.88, 3.69 [05:27:50] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.15, 1.45, 1.65 [05:28:30] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 9.60, 5.83, 4.40 [05:28:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:33:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:40:47] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [05:43:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:49:50] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 2.00, 1.76, 1.60 [05:52:32] [02mw-config] 07ugochimobi reviewed pull request 03#4624 commit - 13https://github.com/miraheze/mw-config/pull/4624#discussion_r856847643 [05:52:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.60, 3.10, 3.96 [05:53:50] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.29, 1.66, 1.62 [05:54:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.65, 3.50, 4.01 [05:55:21] [02mw-config] 07ugochimobi reviewed pull request 03#4626 commit - 13https://github.com/miraheze/mw-config/pull/4626#discussion_r856847858 [05:56:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.18, 3.33, 3.89 [05:56:50] [02mw-config] 07ugochimobi commented on pull request 03#4623: set wgPropertySuggesterMinProbability for gratisdatawiki - 13https://github.com/miraheze/mw-config/pull/4623#issuecomment-1107376251 [05:58:26] [02mw-config] 07ugochimobi commented on pull request 03#4622: set wgLexemeLanguageCodePropertyId and wgLexemeEnableDataTransclusion for gratisdatawiki - 13https://github.com/miraheze/mw-config/pull/4622#issuecomment-1107376675 [05:58:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.82, 3.99, 4.07 [05:59:55] [02mw-config] 07Universal-Omega commented on pull request 03#4622: set wgLexemeLanguageCodePropertyId and wgLexemeEnableDataTransclusion for gratisdatawiki - 13https://github.com/miraheze/mw-config/pull/4622#issuecomment-1107377122 [06:00:18] [02mw-config] 07Universal-Omega commented on pull request 03#4623: set wgPropertySuggesterMinProbability for gratisdatawiki - 13https://github.com/miraheze/mw-config/pull/4623#issuecomment-1107377276 [06:00:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.09, 3.67, 3.95 [06:00:46] [02mw-config] 07Universal-Omega reviewed pull request 03#4626 commit - 13https://github.com/miraheze/mw-config/pull/4626#discussion_r856848308 [06:01:20] [02mw-config] 07ugochimobi commented on pull request 03#4622: set wgLexemeLanguageCodePropertyId and wgLexemeEnableDataTransclusion for gratisdatawiki - 13https://github.com/miraheze/mw-config/pull/4622#issuecomment-1107377632 [06:01:54] [02mw-config] 07Universal-Omega reviewed pull request 03#4624 commit - 13https://github.com/miraheze/mw-config/pull/4624#discussion_r856848399 [06:02:20] [02mw-config] 07Universal-Omega commented on pull request 03#4622: set wgLexemeLanguageCodePropertyId and wgLexemeEnableDataTransclusion for gratisdatawiki - 13https://github.com/miraheze/mw-config/pull/4622#issuecomment-1107377877 [06:03:16] [02mw-config] 07ugochimobi reviewed pull request 03#4626 commit - 13https://github.com/miraheze/mw-config/pull/4626#discussion_r856848527 [06:03:49] RECOVERY - mabi.myomi.net - reverse DNS on sslhost is OK: SSL OK - mabi.myomi.net reverse DNS resolves to cp20.miraheze.org - CNAME OK [06:04:31] [02mw-config] 07ugochimobi commented on pull request 03#4623: set wgPropertySuggesterMinProbability for gratisdatawiki - 13https://github.com/miraheze/mw-config/pull/4623#issuecomment-1107378568 [06:06:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.41, 3.26, 3.86 [06:06:35] [02mw-config] 07ugochimobi commented on pull request 03#4622: set wgLexemeLanguageCodePropertyId and wgLexemeEnableDataTransclusion for gratisdatawiki - 13https://github.com/miraheze/mw-config/pull/4622#issuecomment-1107379137 [06:06:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.87, 2.59, 3.37 [06:07:50] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.30, 1.84, 1.63 [06:09:00] [02mw-config] 07ugochimobi reviewed pull request 03#4624 commit - 13https://github.com/miraheze/mw-config/pull/4624#discussion_r856855946 [06:11:19] [02mw-config] 07Universal-Omega reviewed pull request 03#4624 commit - 13https://github.com/miraheze/mw-config/pull/4624#discussion_r856856616 [06:12:31] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.95, 3.99, 3.86 [06:14:46] [02mw-config] 07ugochimobi reviewed pull request 03#4624 commit - 13https://github.com/miraheze/mw-config/pull/4624#discussion_r856856897 [06:15:00] [02mw-config] 07ugochimobi reviewed pull request 03#4624 commit - 13https://github.com/miraheze/mw-config/pull/4624#discussion_r856856897 [06:15:39] [02mw-config] 07Universal-Omega commented on pull request 03#4622: set wgLexemeLanguageCodePropertyId and wgLexemeEnableDataTransclusion for gratisdatawiki - 13https://github.com/miraheze/mw-config/pull/4622#issuecomment-1107389589 [06:16:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.98, 3.66, 3.79 [06:16:31] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.81, 10.25, 9.18 [06:17:05] [02mw-config] 07ugochimobi commented on pull request 03#4622: set wgLexemeLanguageCodePropertyId and wgLexemeEnableDataTransclusion for gratisdatawiki - 13https://github.com/miraheze/mw-config/pull/4622#issuecomment-1107389896 [06:20:27] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.75, 10.11, 9.37 [06:28:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.85, 3.45, 4.00 [06:30:17] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.41, 10.77, 9.95 [06:30:30] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.37, 2.76, 3.29 [06:32:15] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.09, 9.82, 9.71 [06:36:10] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.50, 10.53, 10.02 [06:38:08] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.50, 9.50, 9.70 [06:38:49] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.05, 2.61, 3.33 [06:56:31] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 8.49, 4.56, 3.15 [06:56:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.30, 3.54, 2.79 [06:56:49] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.37, 4.05, 3.47 [06:57:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:00:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.37, 3.51, 3.05 [07:00:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.51, 3.64, 3.02 [07:00:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.46, 3.69, 3.43 [07:02:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:02:49] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.74, 4.69, 3.81 [07:04:30] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.54, 3.03, 2.96 [07:06:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.76, 3.31, 3.07 [07:06:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.70, 3.98, 3.75 [07:12:16] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.00, 1.87, 1.52 [07:12:49] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.42, 2.66, 3.25 [07:14:15] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.03, 1.60, 1.46 [07:16:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.92, 3.42, 3.44 [07:18:52] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.48, 3.28, 3.40 [07:22:25] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 4.00, 3.39, 3.01 [07:22:30] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.21, 3.53, 3.02 [07:22:49] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.31, 4.18, 3.77 [07:24:20] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.80, 2.76, 2.82 [07:24:30] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.57, 2.78, 2.81 [07:24:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.41, 3.11, 3.42 [07:24:49] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.41, 2.66, 1.64 [07:26:49] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.95, 3.16, 3.40 [07:28:49] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.63, 1.63, 1.45 [07:56:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:01:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:04:22] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 14.20, 11.08, 9.14 [08:06:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.15, 10.92, 9.34 [08:08:18] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.00, 9.71, 9.07 [08:15:50] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.71, 1.71, 1.98 [08:17:50] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.15, 1.85, 2.00 [08:21:50] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.59, 1.86, 1.97 [08:25:47] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.71, 1.44, 1.23 [08:27:47] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.24, 1.29, 1.20 [08:33:50] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 0.63, 1.27, 1.63 [09:08:14] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.03, 3.70, 3.19 [09:10:09] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.62, 3.31, 3.10 [09:25:47] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.31, 1.72, 1.36 [09:27:47] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.85, 1.38, 1.28 [09:35:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:40:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:43:22] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [09:43:22] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [09:45:30] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.43, 3.68, 3.11 [09:47:25] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.97, 4.23, 3.36 [09:51:14] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.39, 3.48, 3.27 [09:53:09] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.71, 2.92, 3.09 [09:56:58] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.78, 3.47, 3.27 [10:00:49] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.95, 3.10, 3.16 [10:12:43] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [11:30:53] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.47, 1.76, 1.49 [11:32:52] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.92, 1.78, 1.52 [11:34:51] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.93, 1.45, 1.43 [11:36:04] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.21, 10.06, 9.15 [11:40:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.98, 10.02, 9.34 [11:44:44] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.87, 1.68, 1.55 [11:46:43] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.35, 1.56, 1.53 [11:50:41] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.60, 1.91, 1.70 [11:52:40] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.03, 1.58, 1.60 [12:13:01] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [12:13:01] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [12:22:43] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.74, 9.75, 8.21 [12:22:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.75, 3.14, 2.78 [12:24:43] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.83, 9.11, 8.17 [12:24:49]