[00:01:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.88, 9.49, 9.09 [00:01:36] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.41, 8.68, 8.35 [00:01:41] RECOVERY - matomo101 Disk Space on matomo101 is OK: DISK OK - free space: / 2154 MB (22% inode=80%); [00:03:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.83, 9.92, 9.28 [00:03:33] PROBLEM - mw101 Puppet on mw101 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:03:34] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.25, 8.03, 8.13 [00:04:20] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.47, 1.78, 1.95 [00:06:33] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.36, 8.03, 8.43 [00:07:33] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.06, 9.28, 8.99 [00:07:43] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.24, 9.25, 8.52 [00:08:20] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.10, 1.21, 1.66 [00:09:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.06, 8.83, 8.87 [00:10:33] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.27, 10.44, 9.31 [00:11:39] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:12:33] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.11, 9.53, 9.12 [00:13:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.78, 9.99, 9.76 [00:13:28] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.79, 9.90, 8.97 [00:14:17] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.56, 8.01, 7.86 [00:15:23] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.54, 9.20, 8.82 [00:16:33] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.13, 9.55, 9.18 [00:16:34] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.80, 8.96, 8.09 [00:16:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:17:33] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.56, 9.64, 9.15 [00:18:12] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.42, 8.21, 8.00 [00:18:28] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.42, 8.18, 7.92 [00:18:33] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.20, 9.01, 9.03 [00:19:13] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.09, 9.45, 8.98 [00:19:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.15, 10.20, 9.76 [00:19:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 6.06, 8.28, 8.71 [00:23:03] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.91, 9.64, 9.21 [00:23:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.41, 9.75, 9.69 [00:27:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.03, 9.83, 9.40 [00:27:33] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.63, 8.87, 8.75 [00:27:58] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.77, 8.48, 8.05 [00:28:33] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.52, 9.07, 8.88 [00:29:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.84, 9.81, 9.44 [00:29:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.53, 9.17, 8.88 [00:29:56] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.98, 8.11, 7.95 [00:30:33] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.21, 9.12, 8.93 [00:31:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.08, 9.56, 9.37 [00:31:33] RECOVERY - mw101 Puppet on mw101 is OK: OK: Puppet is currently enabled, last run 15 seconds ago with 0 failures [00:33:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.45, 9.14, 9.24 [00:35:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.49, 10.16, 9.59 [00:37:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.37, 9.63, 9.45 [00:38:33] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.24, 7.34, 8.23 [00:39:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.57, 9.76, 9.35 [00:41:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.23, 9.26, 9.20 [00:44:33] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.61, 8.58, 8.53 [00:46:33] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.08, 8.89, 8.62 [00:47:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.77, 9.41, 9.28 [00:47:33] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.48, 9.68, 9.25 [00:47:38] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.56, 3.42, 2.91 [00:48:33] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.64, 8.02, 8.34 [00:49:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.21, 8.82, 9.08 [00:49:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.58, 9.25, 9.01 [00:49:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.49, 9.06, 9.06 [00:49:38] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.78, 3.13, 2.87 [00:51:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.95, 9.02, 8.96 [00:53:33] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.16, 9.77, 9.33 [00:55:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.07, 9.53, 9.28 [00:55:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.46, 9.26, 9.20 [00:57:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.19, 9.31, 9.22 [00:59:33] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.33, 9.60, 9.31 [01:01:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.91, 9.01, 9.11 [01:01:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:03:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.23, 7.51, 8.44 [01:03:33] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.15, 9.55, 9.27 [01:05:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.48, 9.04, 8.76 [01:05:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.27, 8.93, 9.08 [01:05:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:09:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.96, 8.58, 8.64 [01:13:06] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.81, 8.80, 8.31 [01:13:33] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.39, 9.18, 8.97 [01:15:03] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.69, 8.50, 8.26 [01:15:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.88, 8.71, 8.81 [01:18:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.93, 3.50, 3.22 [01:19:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.35, 9.71, 9.17 [01:19:30] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.04, 1.65, 1.35 [01:20:10] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.22, 3.34, 3.19 [01:21:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 6.95, 8.59, 8.83 [01:21:26] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.24, 1.47, 1.32 [01:23:26] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 5.99, 7.72, 8.48 [01:29:33] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.58, 7.77, 8.36 [01:30:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:33:34] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 10.05, 9.03, 7.84 [01:35:10] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.54, 8.95, 8.29 [01:35:32] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.32, 8.70, 7.85 [01:35:38] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.49, 3.95, 3.48 [01:36:20] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.95, 1.85, 1.38 [01:37:05] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.25, 8.06, 8.02 [01:37:29] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.22, 8.40, 7.84 [01:37:38] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.57, 3.14, 3.24 [01:37:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:38:20] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.32, 1.57, 1.33 [01:42:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:43:15] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.01, 9.13, 8.41 [01:46:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:47:09] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.63, 9.33, 8.62 [01:47:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.65, 8.64, 8.17 [01:49:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.74, 8.32, 8.02 [01:49:06] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.50, 8.26, 8.31 [01:49:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.95, 10.45, 8.91 [01:51:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.10, 8.38, 8.08 [01:55:27] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.48, 9.87, 9.22 [01:55:52] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.00, 9.14, 8.67 [01:56:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:57:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.36, 10.05, 9.37 [01:57:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.22, 8.91, 8.41 [01:58:04] PROBLEM - diamowiki.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - diamowiki.ga All nameservers failed to answer the query. [01:59:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.83, 9.71, 9.33 [02:00:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:01:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.12, 9.66, 9.35 [02:01:43] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.75, 7.92, 8.15 [02:02:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.74, 3.15, 2.93 [02:03:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.38, 9.26, 9.22 [02:03:59] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.08, 3.16, 2.97 [02:05:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.14, 9.96, 9.48 [02:05:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.09, 8.16, 8.00 [02:05:38] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.98, 8.10, 8.50 [02:07:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.44, 9.70, 9.45 [02:07:29] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.95, 8.87, 8.45 [02:09:24] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.93, 8.39, 8.33 [02:09:33] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.24, 9.29, 8.45 [02:09:33] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.89, 9.20, 8.83 [02:11:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.03, 8.91, 8.40 [02:13:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.20, 9.13, 9.24 [02:15:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.65, 8.78, 9.09 [02:15:33] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.93, 8.20, 8.27 [02:17:06] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.07, 8.80, 8.56 [02:17:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.59, 9.59, 9.28 [02:19:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.12, 10.02, 9.48 [02:19:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.70, 8.83, 8.47 [02:20:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:21:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.95, 9.52, 9.37 [02:21:33] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.89, 9.78, 8.86 [02:23:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.16, 8.20, 8.35 [02:23:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.74, 9.65, 8.92 [02:25:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.31, 3.45, 3.09 [02:27:08] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.62, 9.45, 9.22 [02:27:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.26, 9.88, 9.48 [02:27:40] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.56, 3.19, 3.04 [02:28:01] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolve(hostn [02:28:01] AA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.4108827114105225 seconds [02:28:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:29:05] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 6.67, 8.34, 8.84 [02:29:27] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.02, 9.06, 9.22 [02:29:33] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 4.86, 7.56, 8.35 [02:31:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.93, 8.93, 8.71 [02:32:59] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.27, 7.74, 8.47 [02:33:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.68, 8.19, 8.47 [02:35:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.09, 9.47, 9.33 [02:37:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.71, 8.48, 8.57 [02:39:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.14, 9.78, 9.56 [02:40:45] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.46, 9.48, 8.85 [02:42:40] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.31, 1.71, 1.29 [02:42:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.01, 9.09, 8.79 [02:44:36] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.65, 1.72, 1.35 [02:45:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.85, 7.73, 8.23 [02:46:32] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.88, 2.10, 1.53 [02:47:09] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.31, 8.87, 8.48 [02:48:28] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.52, 1.84, 1.50 [02:48:33] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 5.58, 7.59, 8.29 [02:49:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.94, 9.73, 9.44 [02:51:03] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.37, 8.46, 8.39 [02:52:20] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.40, 2.42, 1.78 [02:56:20] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.77, 1.63, 1.62 [02:57:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.87, 9.11, 8.69 [02:57:48] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.55, 9.68, 8.89 [02:57:55] PROBLEM - diamowiki.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - diamowiki.ga All nameservers failed to answer the query. [02:58:00] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.54, 3.82, 3.20 [02:59:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.16, 9.26, 9.54 [02:59:45] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.42, 8.94, 8.70 [03:00:00] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.52, 3.31, 3.09 [03:01:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.05, 8.31, 8.45 [03:11:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.99, 9.39, 9.22 [03:13:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.51, 8.97, 9.09 [03:13:33] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.66, 8.20, 8.49 [03:16:34] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.00, 8.00, 7.31 [03:17:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.70, 9.53, 9.26 [03:18:02] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.97, 8.78, 8.30 [03:18:34] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.89, 7.44, 7.19 [03:19:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.91, 8.57, 8.55 [03:21:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.16, 9.81, 9.54 [03:23:18] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.00, 8.59, 8.22 [03:23:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:23:54] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.07, 8.99, 8.54 [03:25:33] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.15, 7.98, 8.36 [03:27:08] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.36, 8.28, 8.15 [03:27:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.16, 9.86, 9.60 [03:27:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:27:48] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.49, 9.10, 8.74 [03:29:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.32, 9.78, 9.61 [03:31:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 9.56, 10.03, 9.74 [03:33:52] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.60, 9.25, 8.59 [03:35:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.61, 9.70, 9.73 [03:35:37] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.70, 8.14, 8.47 [03:35:47] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.86, 9.50, 8.77 [03:36:24] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.98, 9.77, 8.97 [03:37:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:38:21] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.02, 9.38, 8.95 [03:39:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.21, 9.82, 9.75 [03:39:33] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.17, 9.12, 8.79 [03:39:38] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.46, 8.29, 8.43 [03:41:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.79, 9.79, 9.76 [03:43:28] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.48, 8.92, 8.87 [03:43:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:48:06] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.92, 7.87, 8.38 [03:49:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.18, 9.56, 9.46 [03:51:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.81, 9.62, 9.49 [03:53:13] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.82, 7.73, 8.39 [04:03:39] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:04:33] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.19, 9.46, 8.68 [04:05:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.44, 9.23, 9.01 [04:06:30] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.38, 8.56, 8.44 [04:07:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.74, 9.05, 8.98 [04:08:27] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.92, 8.43, 8.40 [04:09:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.92, 9.94, 9.32 [04:10:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:12:18] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.28, 9.42, 8.82 [04:13:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.60, 9.42, 8.61 [04:14:00] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.52, 8.08, 7.56 [04:14:15] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.69, 9.20, 8.82 [04:14:20] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.29, 2.09, 1.48 [04:14:36] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.61, 7.75, 7.16 [04:15:33] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.26, 9.18, 8.61 [04:16:14] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.60, 8.80, 8.53 [04:16:20] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.94, 1.89, 1.47 [04:16:34] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.01, 7.62, 7.19 [04:18:00] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.52, 7.84, 7.58 [04:18:09] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.41, 8.12, 8.30 [04:18:20] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.90, 1.57, 1.40 [04:20:06] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.39, 8.12, 8.49 [04:21:25] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.61, 7.90, 8.26 [04:21:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.62, 9.46, 9.49 [04:25:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:26:25] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.48, 1.72, 1.31 [04:27:40] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolve(hostn [04:27:40] AA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.4065492153167725 seconds [04:28:25] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.62, 1.28, 1.19 [04:28:49] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.60, 8.51, 8.46 [04:32:43] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.13, 8.50, 8.46 [04:33:26] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.50, 7.52, 8.37 [04:41:33] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.24, 9.94, 9.09 [04:41:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:43:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.28, 9.54, 9.06 [04:46:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:50:37] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.76, 8.55, 8.13 [04:54:33] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.59, 8.40, 8.20 [04:54:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:57:33] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 5.50, 7.38, 8.23 [04:59:43] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.55, 8.15, 7.17 [05:03:39] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.40, 8.16, 7.44 [05:04:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:08:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.58, 8.41, 7.90 [05:10:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:12:19] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.64, 7.92, 7.85 [05:15:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:18:25] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.71, 1.86, 1.33 [05:20:05] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.92, 8.65, 8.11 [05:20:25] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.83, 1.53, 1.27 [05:20:33] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.89, 8.42, 8.13 [05:22:01] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.03, 8.05, 7.95 [05:22:33] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.82, 7.89, 7.97 [05:27:32] PROBLEM - diamowiki.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - diamowiki.ga All nameservers failed to answer the query. [05:39:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:44:39] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:50:19] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.83, 8.55, 7.79 [05:52:16] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.51, 8.46, 7.86 [05:57:28] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolve(hostn [05:57:28] AA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.406583309173584 seconds [06:06:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:11:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:00:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:05:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:12:33] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.57, 8.20, 7.43 [07:13:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:14:33] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.37, 7.75, 7.37 [07:24:33] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.20, 8.34, 7.70 [07:26:33] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.49, 7.96, 7.64 [07:27:17] PROBLEM - diamowiki.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - diamowiki.ga All nameservers failed to answer the query. [07:33:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:43:31] [02mw-config] 07ugochimobi synchronize pull request 03#4493: Adding well-known properties used to format refs - 13https://github.com/miraheze/mw-config/pull/4493 [07:44:33] miraheze/mw-config - ugochimobi the build passed. [07:51:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:52:37] [02mw-config] 07ugochimobi synchronize pull request 03#4492: T8811: Working on adding article schema to pages - 13https://github.com/miraheze/mw-config/pull/4492 [07:53:33] miraheze/mw-config - ugochimobi the build passed. [07:56:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:57:14] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolve(hostn [07:57:14] AA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.4054036140441895 seconds [08:03:38] [02mw-config] 07ugochimobi opened pull request 03#4495: setting dataRightsUrl for gratisdatawiki - 13https://github.com/miraheze/mw-config/pull/4495 [08:05:13] miraheze/mw-config - ugochimobi the build passed. [08:33:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:43:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:47:38] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.67, 2.91, 2.42 [08:49:38] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.02, 2.71, 2.42 [08:50:20] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.99, 8.76, 7.65 [08:50:33] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.05, 8.66, 7.90 [08:54:33] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.74, 8.30, 7.95 [08:58:05] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.13, 8.30, 7.95 [09:12:43] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.51, 8.54, 7.82 [09:14:40] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.27, 9.10, 8.08 [09:16:37] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.98, 9.07, 8.21 [09:20:31] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.69, 7.83, 7.90 [09:21:15] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [09:21:15] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [09:26:48] [02miraheze/puppet] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/d79f177324b6...aab22cc3d443 [09:26:50] [02miraheze/puppet] 07JohnFLewis 03aab22cc - T8912: Correctly apply CORS header to images only [09:50:40] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [10:01:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:06:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:19:26] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.91, 1.53, 1.25 [10:21:26] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.43, 1.44, 1.25 [10:26:55] PROBLEM - diamowiki.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - diamowiki.ga All nameservers failed to answer the query. [10:49:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.98, 8.87, 7.45 [10:49:26] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.95, 1.62, 1.31 [10:51:26] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.29, 7.80, 7.23 [10:51:26] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.46, 1.57, 1.33 [11:02:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:07:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:24:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:26:44] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolve(hostn [11:26:44] AA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.406866073608398 seconds [11:29:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:35:09] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.22, 8.87, 7.88 [11:37:06] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.67, 8.43, 7.84 [11:53:36] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.97, 8.37, 7.90 [11:55:25] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.37, 8.59, 7.52 [11:55:33] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.75, 7.76, 7.74 [11:57:22] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.27, 7.95, 7.41 [11:57:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.17, 8.08, 7.04 [11:59:26] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.81, 7.61, 7.00 [12:23:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:25:33] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.80, 1.65, 1.28 [12:25:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.22, 8.54, 7.71 [12:28:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:29:03] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.01, 8.76, 7.61 [12:31:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.16, 8.48, 7.64 [12:31:28] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.23, 8.87, 8.08 [12:31:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:33:25] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.98, 8.76, 8.15 [12:33:33] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.43, 1.63, 1.45 [12:35:22] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.65, 8.24, 8.04 [12:38:05] [02mw-config] 07ugochimobi opened pull request 03#4496: Define constraints for various strings - 13https://github.com/miraheze/mw-config/pull/4496 [12:39:12] miraheze/mw-config - ugochimobi the build passed. [12:41:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:43:55] [02mw-config] 07ugochimobi synchronize pull request 03#4496: Define constraints for various strings - 13https://github.com/miraheze/mw-config/pull/4496 [12:44:55] miraheze/mw-config - ugochimobi the build passed. [12:53:33] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.32, 1.73, 1.51 [12:54:31] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.47, 3.14, 2.55 [12:55:04] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 4.05, 2.05, 1.26 [12:55:33] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.63, 1.68, 1.52 [12:57:01] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.97, 1.53, 1.17 [12:58:22] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.70, 3.17, 2.72 [13:03:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:04:51] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.95, 9.27, 8.51 [13:10:42] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.72, 8.44, 8.36 [13:12:52] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.13, 3.40, 3.08 [13:14:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.08, 8.60, 8.46 [13:14:48] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.59, 2.74, 2.88 [13:16:20] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.47, 1.82, 1.21 [13:16:31] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 5.79, 7.49, 8.06 [13:18:20] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.74, 1.45, 1.15 [13:18:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:21:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:26:27] PROBLEM - diamowiki.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - diamowiki.ga All nameservers failed to answer the query. [13:30:05] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.84, 8.44, 8.27 [13:32:02] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.63, 8.18, 8.20 [13:33:59] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.38, 8.59, 7.99 [13:35:53] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.75, 8.96, 8.51 [13:35:56] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.53, 9.61, 8.43 [13:37:53] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.06, 8.64, 8.21 [13:39:47] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.19, 8.38, 8.37 [13:45:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.30, 8.15, 8.21 [13:46:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:47:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.62, 8.56, 8.37 [13:49:31] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.92, 1.59, 1.34 [13:51:27] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.08, 1.40, 1.30 [13:51:33] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 5.99, 7.88, 8.20 [13:52:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:55:33] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.24, 9.40, 8.73 [13:56:54] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [13:56:55] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [13:57:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.97, 9.55, 8.87 [13:57:38] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.61, 8.64, 8.13 [13:57:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:58:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.94, 8.90, 8.53 [13:58:34] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.86, 7.91, 7.15 [13:59:34] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.95, 8.29, 8.05 [14:01:22] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.62, 2.61, 1.81 [14:01:33] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.72, 9.40, 8.90 [14:01:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:02:14] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.43, 9.67, 8.92 [14:02:34] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.62, 7.93, 7.34 [14:03:18]