[00:00:16] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.70, 3.54, 3.21 [00:00:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.81, 9.96, 9.25 [00:00:30] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.65, 3.84, 3.60 [00:01:51] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.46, 2.07, 1.56 [00:02:12] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.43, 4.64, 3.83 [00:02:15] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.71, 4.46, 3.63 [00:04:22] PROBLEM - mw101 Puppet on mw101 is CRITICAL: CRITICAL: Puppet has 574 failures. Last run 3 minutes ago with 574 failures. Failed resources (up to 3 shown): File[wiki.ciptamedia.org_private],File[podpedia.org],File[podpedia.org_private],File[dariawiki.org] [00:04:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.19, 9.77, 9.29 [00:05:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.65, 7.20, 8.29 [00:06:03] PROBLEM - cp31 Stunnel HTTP for mw112 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:07:50] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.07, 1.71, 1.60 [00:08:03] RECOVERY - cp31 Stunnel HTTP for mw112 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 2.562 second response time [00:08:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.26, 8.83, 8.77 [00:08:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.92, 9.50, 9.34 [00:08:46] PROBLEM - rothwell-leeds.co.uk - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/ [00:08:46] on3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query rothwell-leeds.co.uk. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [00:09:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.27, 10.01, 9.49 [00:09:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.59, 9.39, 8.85 [00:09:50] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.20, 1.52, 1.54 [00:10:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.78, 9.00, 8.86 [00:11:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.72, 9.58, 9.00 [00:12:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.14, 9.94, 9.23 [00:13:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.84, 8.65, 7.99 [00:13:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.29, 9.55, 9.46 [00:14:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.31, 9.50, 9.15 [00:15:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.29, 7.91, 7.81 [00:16:12] PROBLEM - mw112 MediaWiki Rendering on mw112 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:17:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.81, 9.97, 9.64 [00:18:09] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.46, 3.72, 3.96 [00:18:09] RECOVERY - mw112 MediaWiki Rendering on mw112 is OK: HTTP OK: HTTP/1.1 200 OK - 22336 bytes in 3.675 second response time [00:19:37] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [00:19:37] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [00:19:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.09, 7.61, 8.37 [00:22:07] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.52, 3.68, 3.85 [00:23:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.78, 9.91, 9.85 [00:24:07] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.69, 3.28, 3.68 [00:24:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.68, 10.71, 9.76 [00:24:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.08, 9.15, 8.96 [00:24:47] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.32, 2.16, 1.92 [00:25:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.05, 10.00, 9.89 [00:26:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.32, 9.02, 8.92 [00:26:47] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.07, 1.75, 1.80 [00:27:25] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.16, 3.18, 3.80 [00:28:14] PROBLEM - thelonsdalebattalion.co.uk - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - thelonsdalebattalion.co.uk All nameservers failed to answer the query. [00:28:46] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.73, 1.45, 1.68 [00:30:19] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.29, 3.14, 3.91 [00:31:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.29, 9.70, 9.89 [00:32:10] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 11.52, 9.44, 8.27 [00:32:38] RECOVERY - mw101 Puppet on mw101 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:33:14] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.14, 3.03, 3.49 [00:33:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.21, 9.81, 9.90 [00:34:03] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.05, 3.00, 3.31 [00:34:08] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.30, 8.81, 8.18 [00:35:10] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.32, 3.08, 3.45 [00:35:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.60, 9.28, 8.83 [00:36:05] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.59, 8.38, 8.10 [00:37:06] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.80, 2.89, 3.33 [00:37:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.24, 9.80, 9.93 [00:37:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.34, 10.10, 9.18 [00:38:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.14, 9.18, 9.69 [00:38:37] RECOVERY - rothwell-leeds.co.uk - reverse DNS on sslhost is OK: SSL OK - rothwell-leeds.co.uk reverse DNS resolves to cp21.miraheze.org - NS RECORDS OK [00:38:45] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.03, 2.23, 1.88 [00:39:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.18, 10.15, 10.03 [00:39:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.45, 9.76, 9.19 [00:40:44] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.22, 1.83, 1.77 [00:41:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.58, 9.97, 9.98 [00:41:59] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.69, 3.30, 3.27 [00:42:44] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.62, 1.45, 1.64 [00:42:54] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.15, 8.80, 8.30 [00:43:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.33, 3.90, 3.55 [00:43:58] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.21, 4.46, 3.70 [00:45:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.54, 9.98, 9.94 [00:45:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.74, 9.81, 9.29 [00:46:13] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.36, 3.80, 3.66 [00:46:48] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.19, 8.45, 8.31 [00:47:57] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.81, 3.64, 3.55 [00:48:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.24, 8.63, 9.04 [00:48:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.70, 3.62, 3.60 [00:49:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.47, 9.83, 9.48 [00:50:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.73, 7.81, 8.40 [00:51:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.50, 3.57, 3.66 [00:52:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 6.87, 8.48, 8.96 [00:53:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.91, 9.77, 9.48 [00:53:54] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.49, 3.05, 3.33 [00:54:10] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.50, 2.64, 3.21 [00:54:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.47, 9.88, 9.08 [00:55:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.94, 9.73, 9.53 [00:56:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.79, 9.40, 9.21 [00:57:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.09, 3.05, 3.38 [00:57:31] RECOVERY - thelonsdalebattalion.co.uk - reverse DNS on sslhost is OK: SSL OK - thelonsdalebattalion.co.uk reverse DNS resolves to cp30.miraheze.org - NS RECORDS OK [00:58:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.49, 9.93, 9.31 [01:00:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.58, 9.53, 9.33 [01:01:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.65, 9.64, 9.47 [01:02:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.21, 9.37, 9.19 [01:03:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.09, 9.44, 9.97 [01:03:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.36, 8.88, 9.20 [01:04:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.45, 10.00, 9.58 [01:06:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.49, 9.88, 9.60 [01:06:11] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.77, 7.76, 7.57 [01:06:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.79, 9.91, 9.48 [01:08:08] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.29, 7.82, 7.61 [01:09:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.17, 9.44, 9.75 [01:09:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.75, 9.67, 9.44 [01:11:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.00, 9.23, 9.64 [01:12:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.20, 10.45, 9.77 [01:13:02] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.79, 3.44, 3.18 [01:13:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.49, 3.73, 3.38 [01:13:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.81, 9.17, 9.34 [01:13:46] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.64, 3.40, 3.22 [01:15:01] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.47, 3.68, 3.29 [01:15:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.53, 3.65, 3.39 [01:15:45] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.02, 3.35, 3.23 [01:15:52] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.34, 8.27, 7.83 [01:16:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.77, 10.18, 9.69 [01:17:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.43, 3.56, 3.29 [01:17:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.63, 3.96, 3.53 [01:17:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.52, 9.91, 9.56 [01:17:48] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.24, 8.13, 7.85 [01:18:59] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.69, 3.18, 3.18 [01:19:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.13, 3.14, 3.27 [01:19:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.42, 9.75, 9.63 [01:20:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.68, 9.63, 9.60 [01:20:55] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.25, 5.96, 5.23 [01:21:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.17, 9.78, 9.62 [01:22:55] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.06, 5.34, 5.10 [01:23:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.05, 3.67, 3.42 [01:23:41] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.46, 3.06, 3.07 [01:24:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.24, 10.38, 9.86 [01:25:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.48, 3.44, 3.36 [01:25:41] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.09, 2.73, 2.95 [01:25:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.46, 10.47, 9.89 [01:26:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.38, 9.61, 9.85 [01:27:03] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.31, 2.93, 3.18 [01:27:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.41, 9.85, 9.75 [01:31:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.13, 9.75, 9.70 [01:32:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.84, 9.98, 9.85 [01:33:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.64, 9.58, 9.66 [01:36:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.19, 9.01, 9.56 [01:37:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.73, 9.46, 9.98 [01:38:20] [02RemovePII] 07dmehus commented on pull request 03#54: Always guarantee realname is removed - 13https://github.com/miraheze/RemovePII/pull/54#issuecomment-1074597496 [01:38:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.26, 9.23, 9.55 [01:38:35] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.74, 2.46, 1.85 [01:39:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.48, 10.27, 10.21 [01:41:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.97, 9.81, 9.57 [01:42:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.56, 9.33, 9.55 [01:42:34] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.41, 1.83, 1.73 [01:44:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.83, 9.23, 9.96 [01:46:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.61, 9.58, 9.57 [01:46:34] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.43, 1.62, 1.67 [01:46:47] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.53, 3.90, 3.39 [01:47:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.54, 9.24, 9.51 [01:48:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.46, 3.50, 3.32 [01:50:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.93, 9.53, 9.60 [01:50:33] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.70, 1.73, 1.71 [01:50:45] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.49, 3.20, 3.23 [01:51:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.33, 9.41, 9.47 [01:52:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.33, 9.92, 9.95 [01:52:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.64, 10.23, 9.83 [01:53:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.43, 9.22, 9.41 [01:54:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.08, 9.01, 9.63 [01:54:32] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.10, 1.59, 1.67 [01:55:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.07, 9.67, 9.54 [01:56:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.73, 9.85, 9.79 [01:59:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.03, 9.59, 9.55 [02:01:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 5.86, 8.86, 9.91 [02:02:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.64, 9.33, 9.50 [02:03:05] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.51, 7.72, 6.91 [02:05:02] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.37, 7.71, 7.01 [02:07:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.38, 10.27, 10.12 [02:07:45] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 9.89, 10.01, 9.73 [02:09:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.48, 9.19, 9.75 [02:10:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.32, 9.99, 9.99 [02:14:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.83, 10.21, 10.05 [02:15:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.21, 9.81, 9.78 [02:17:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.24, 9.58, 9.71 [02:19:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.65, 9.69, 9.92 [02:20:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 6.92, 8.97, 9.68 [02:20:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:21:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.45, 10.13, 9.88 [02:22:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.91, 9.73, 9.54 [02:24:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.85, 9.14, 9.36 [02:26:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.23, 9.67, 9.53 [02:27:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.39, 3.84, 3.22 [02:27:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.45, 9.85, 9.84 [02:29:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.24, 3.57, 3.20 [02:31:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.19, 9.14, 9.67 [02:31:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.12, 9.08, 9.58 [02:32:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.50, 9.43, 9.59 [02:33:29] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.20, 3.26, 3.20 [02:34:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.11, 9.46, 9.48 [02:36:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.23, 9.82, 9.70 [02:36:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.42, 9.62, 9.56 [02:37:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.37, 3.72, 3.45 [02:37:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.55, 9.53, 9.56 [02:38:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.42, 9.37, 9.53 [02:39:29] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.44, 3.11, 3.25 [02:39:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.83, 9.22, 9.45 [02:44:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.36, 9.25, 9.30 [02:46:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.95, 8.51, 9.01 [02:47:36] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.06, 3.65, 3.27 [02:49:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.94, 9.99, 9.55 [02:49:32] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.54, 3.56, 3.28 [02:50:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.43, 9.23, 9.01 [02:51:29] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.86, 3.94, 3.45 [02:51:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.55, 9.93, 9.60 [02:52:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.81, 9.07, 9.00 [02:53:25] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.87, 3.60, 3.39 [02:53:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.02, 9.49, 9.45 [02:54:05] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.72, 3.70, 3.26 [02:54:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.75, 9.44, 9.19 [02:55:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.67, 9.00, 9.27 [02:56:04] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.16, 3.13, 3.11 [02:56:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.41, 8.32, 8.72 [02:57:17] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.60, 3.31, 3.34 [02:57:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.37, 9.09, 9.24 [02:58:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.06, 9.36, 9.20 [03:00:02] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.10, 3.43, 3.24 [03:00:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.85, 9.10, 9.13 [03:02:01] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.36, 3.02, 3.11 [03:02:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.71, 9.84, 9.39 [03:04:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.46, 9.16, 8.88 [03:05:50] PROBLEM - inourownwords.online - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'inourownwords.online' expires in 15 day(s) (Thu 07 Apr 2022 02:58:08 GMT +0000). [03:08:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.76, 8.72, 8.80 [03:09:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.67, 9.91, 9.73 [03:10:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.63, 9.72, 9.68 [03:10:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.20, 9.19, 8.95 [03:11:23] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3ca1c61a7c24...fdc656f902e8 [03:11:25] [02miraheze/ssl] 07MirahezeSSLBot 03fdc656f - Bot: Update SSL cert for inourownwords.online [03:11:26] PROBLEM - wiki.astralprojections.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.astralprojections.org' expires in 15 day(s) (Thu 07 Apr 2022 03:01:23 GMT +0000). [03:11:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.02, 9.98, 9.77 [03:11:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.52, 9.90, 9.25 [03:12:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.24, 8.78, 8.82 [03:13:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.82, 8.94, 8.98 [03:13:45] PROBLEM - wiki.thedev.gq - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.thedev.gq' expires in 15 day(s) (Thu 07 Apr 2022 03:10:42 GMT +0000). [03:15:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.87, 9.19, 9.53 [03:15:47] PROBLEM - wiki.elgeis.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.elgeis.com' expires in 15 day(s) (Thu 07 Apr 2022 03:06:32 GMT +0000). [03:16:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.26, 9.50, 9.54 [03:16:57] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [03:17:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.49, 9.85, 9.72 [03:18:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.85, 9.40, 9.52 [03:18:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.95, 3.63, 3.26 [03:18:57] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/fdc656f902e8...8e0b9f836958 [03:18:58] [02miraheze/ssl] 07MirahezeSSLBot 038e0b9f8 - Bot: Update SSL cert for wiki.astralprojections.org [03:21:41] paladox: please check Miraheze Limited Discord server, #sre-requests :) [03:22:34] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.87, 3.10, 3.16 [03:22:37] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [03:23:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.78, 7.86, 8.47 [03:24:08] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/8e0b9f836958...53799d2048fc [03:24:10] [02miraheze/ssl] 07MirahezeSSLBot 0353799d2 - Bot: Update SSL cert for wiki.elgeis.com [03:24:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.73, 9.06, 8.88 [03:26:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.78, 9.89, 9.66 [03:28:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.69, 9.86, 9.68 [03:28:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.83, 9.23, 9.07 [03:30:49] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.64, 3.75, 3.22 [03:31:08] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.20, 3.91, 3.45 [03:31:16] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.94, 1.81, 1.46 [03:31:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.38, 9.16, 9.80 [03:31:47] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/53799d2048fc...732ced16ed8c [03:31:49] [02miraheze/ssl] 07MirahezeSSLBot 03732ced1 - Bot: Update SSL cert for wiki.thedev.gq [03:34:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.54, 9.36, 9.47 [03:34:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.45, 10.03, 9.40 [03:34:47] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.71, 3.19, 3.11 [03:35:09] RECOVERY - inourownwords.online - LetsEncrypt on sslhost is OK: OK - Certificate 'inourownwords.online' will expire on Mon 20 Jun 2022 02:11:17 GMT +0000. [03:36:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.54, 9.83, 9.65 [03:37:07] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.84, 2.90, 3.16 [03:37:15] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.42, 1.70, 1.54 [03:40:21] RECOVERY - wiki.astralprojections.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.astralprojections.org' will expire on Mon 20 Jun 2022 02:18:51 GMT +0000. [03:40:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.89, 9.56, 9.44 [03:40:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:40:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.55, 3.29, 3.14 [03:41:05] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.84, 8.32, 8.26 [03:42:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.61, 9.51, 9.41 [03:42:59] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.38, 9.06, 8.54 [03:44:53] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.60, 8.95, 8.54 [03:45:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:45:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.77, 9.30, 9.12 [03:46:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.32, 8.87, 9.20 [03:46:43] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.50, 3.21, 3.18 [03:46:47] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.45, 8.38, 8.38 [03:47:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.09, 8.85, 8.98 [03:50:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:50:37] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 12 seconds ago with 0 failures [03:52:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.57, 9.40, 9.07 [03:54:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.71, 9.16, 9.03 [03:54:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.41, 7.64, 8.49 [03:54:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:54:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.81, 3.64, 3.30 [03:55:59] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 10.07, 5.85, 4.22 [03:56:35] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.98, 4.34, 3.50 [03:58:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.34, 8.89, 8.90 [03:58:32] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.07, 3.79, 3.40 [04:00:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.53, 8.76, 8.86 [04:00:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.81, 8.35, 8.46 [04:01:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.37, 8.55, 8.65 [04:03:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.52, 8.17, 8.50 [04:04:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.52, 3.34, 3.36 [04:04:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.84, 9.11, 8.69 [04:04:35] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.02, 3.72, 3.65 [04:06:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.84, 9.34, 8.84 [04:07:10] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.10, 2.60, 1.92 [04:08:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.45, 7.92, 8.46 [04:09:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:11:49] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.14, 1.68, 1.35 [04:12:29] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.68, 8.75, 8.29 [04:13:09] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.78, 1.94, 1.81 [04:13:15] RECOVERY - wiki.thedev.gq - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.thedev.gq' will expire on Mon 20 Jun 2022 02:31:42 GMT +0000. [04:13:47] RECOVERY - wiki.elgeis.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.elgeis.com' will expire on Mon 20 Jun 2022 02:24:03 GMT +0000. [04:13:49] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.68, 1.27, 1.24 [04:14:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.82, 8.24, 8.48 [04:15:08] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.69, 1.47, 1.65 [04:15:52] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.10, 3.19, 3.89 [04:16:17] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.16, 7.81, 8.04 [04:16:30] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.34, 2.92, 3.30 [04:16:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:18:03] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [04:18:03] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [04:20:16] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.10, 9.43, 8.66 [04:22:12] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.16, 9.37, 8.73 [04:23:18] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.08, 8.59, 8.49 [04:23:49] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.02, 3.76, 3.88 [04:25:49] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.06, 8.74, 8.40 [04:27:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.07, 3.60, 3.25 [04:28:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.39, 9.41, 8.55 [04:29:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.95, 3.00, 3.07 [04:30:53] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.78, 8.65, 8.36 [04:30:57] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.55, 7.91, 7.22 [04:32:56] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.70, 9.39, 8.84 [04:32:57] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.63, 7.27, 7.06 [04:33:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.74, 9.60, 8.87 [04:34:51] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.98, 9.44, 8.94 [04:35:05] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.43, 1.83, 1.66 [04:35:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.87, 8.93, 8.71 [04:36:38] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.83, 8.12, 8.28 [04:37:05] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.51, 1.39, 1.52 [04:37:38] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.96, 8.27, 8.47 [04:40:38] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.71, 7.93, 8.40 [04:41:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:41:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.57, 10.06, 9.26 [04:43:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.66, 3.57, 3.29 [04:43:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.23, 9.64, 9.20 [04:45:40] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.06, 3.45, 3.88 [04:46:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:47:40] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.33, 3.86, 3.98 [04:48:13] paladox, you're back? [04:49:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.96, 3.34, 3.28 [04:51:38] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 1.76, 3.47, 3.86 [04:53:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.16, 7.81, 8.43 [04:55:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.37, 3.56, 3.78 [04:57:36] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.68, 3.48, 3.72 [04:59:35] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.61, 3.87, 3.82 [05:01:34] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.90, 3.53, 3.71 [05:05:00] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.23, 8.46, 8.01 [05:05:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.80, 3.38, 3.19 [05:05:33] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.02, 3.38, 3.59 [05:06:56] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.74, 8.12, 7.94 [05:09:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.62, 3.80, 3.37 [05:11:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.73, 3.67, 3.38 [05:13:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.58, 3.99, 3.52 [05:13:58] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.43, 2.27, 1.59 [05:15:30] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.91, 3.81, 3.77 [05:15:58] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.49, 1.93, 1.54 [05:17:05] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.54, 3.41, 3.05 [05:17:09] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp30.miraheze.org - CNAME OK [05:17:58] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.35, 1.70, 1.50 [05:19:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.27, 3.90, 3.68 [05:19:05] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.46, 3.16, 3.01 [05:21:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:21:57] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.89, 1.79, 1.58 [05:23:56] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 4.70, 2.38, 1.79 [05:25:29] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.02, 2.74, 3.31 [05:25:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:25:56] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.43, 1.91, 1.69 [05:27:56] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.62, 1.44, 1.54 [05:30:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:33:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.12, 3.94, 3.68 [05:35:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.34, 3.42, 3.51 [05:37:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.25, 3.71, 3.60 [05:40:55] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.98, 3.61, 3.29 [05:41:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.84, 3.51, 3.56 [05:41:23] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.03, 3.69, 3.46 [05:41:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:43:23] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.34, 4.95, 3.99 [05:45:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.13, 4.20, 3.81 [05:46:53] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.62, 3.14, 3.21 [05:47:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.09, 3.74, 3.69 [05:49:20] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.45, 3.79, 3.84 [05:51:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.68, 3.54, 3.56 [05:53:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.97, 3.56, 3.56 [05:55:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.80, 2.85, 3.30 [05:55:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.08, 2.74, 3.35 [05:56:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:01:15] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.59, 3.37, 3.46 [06:03:14] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.64, 3.87, 3.63 [06:04:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:05:13] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.01, 3.64, 3.58 [06:06:15] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.15, 8.22, 7.49 [06:07:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.29, 4.08, 3.76 [06:08:11] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.15, 8.00, 7.48 [06:09:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:11:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.08, 3.86, 3.81 [06:12:01] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.90, 8.77, 7.78 [06:13:55] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.17, 9.34, 8.11 [06:15:09] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.11, 4.37, 3.99 [06:15:49] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.63, 8.53, 7.95 [06:15:54] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.79, 1.53, 1.05 [06:17:41] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.31, 4.06, 3.55 [06:17:43] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.85, 8.17, 7.87 [06:17:54] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.43, 1.48, 1.09 [06:19:08] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.43, 4.00, 3.93 [06:19:37] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.32, 3.75, 3.49 [06:21:34] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.02, 3.16, 3.31 [06:21:54] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.96, 2.11, 1.42 [06:24:43] [02mediawiki] 07Reception123 closed pull request 03#5946: Bump extensions/DynamicPageList3 from `e7113a2` to `b4f58aa` - 13https://github.com/miraheze/mediawiki/pull/5946 [06:24:44] [02mediawiki] 07Reception123 deleted branch 03dependabot/submodules/REL1_37/extensions/DynamicPageList3-b4f58aa - 13https://github.com/miraheze/mediawiki [06:24:46] [02miraheze/mediawiki] 07Reception123 deleted branch 03dependabot/submodules/REL1_37/extensions/DynamicPageList3-b4f58aa [06:24:47] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/d3eca0f69127...12e3e716bdb2 [06:24:49] [02miraheze/mediawiki] 07dependabot[bot] 0312e3e71 - Bump extensions/DynamicPageList3 from `e7113a2` to `b4f58aa` (#5946) [06:25:06] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.25, 2.58, 3.29 [06:25:54] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.60, 1.92, 1.51 [06:26:42] !log [reception@mwtask111] starting deploy of {'world': True, 'l10n': True} to all [06:26:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:27:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:29:54] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.02, 1.94, 1.61 [06:33:54] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.64, 1.92, 1.70 [06:37:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:37:54] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.31, 1.56, 1.60 [06:40:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.95, 8.28, 7.50 [06:42:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.96, 7.83, 7.44 [06:42:30] !log [reception@mwtask111] starting deploy of {'world': True, 'l10n': True} to all [06:42:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:44:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:44:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.18, 8.57, 8.21 [06:47:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.12, 9.39, 8.45 [06:48:43] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.45, 9.29, 8.56 [06:49:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.56, 8.96, 8.41 [06:51:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.06, 7.83, 8.06 [06:52:33] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.92, 9.36, 8.72 [06:52:38] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.74, 1.69, 1.59 [06:54:32] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.39, 1.57, 1.55 [06:56:54] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.05, 8.96, 8.12 [06:58:18] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.60, 9.25, 8.82 [06:58:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.50, 8.93, 8.12 [06:58:49] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.29, 9.14, 8.29 [06:59:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.66, 8.91, 8.43 [07:00:13] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 5.35, 7.88, 8.38 [07:01:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.78, 8.33, 8.28 [07:02:41] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.85, 8.39, 8.19 [07:04:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.56, 8.34, 8.18 [07:04:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:09:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:14:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:18:22] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [07:18:22] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [07:20:04] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.66, 8.36, 7.86 [07:22:00] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.93, 8.22, 7.87 [07:23:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:27:49] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.66, 8.41, 8.04 [07:28:59] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.54, 9.05, 8.27 [07:30:53] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.04, 8.80, 8.27 [07:32:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.56, 8.34, 7.94 [07:32:47] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.13, 8.50, 8.22 [07:33:36] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.70, 8.06, 8.09 [07:34:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.53, 8.35, 8.01 [07:38:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:40:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.78, 8.36, 8.17 [07:42:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:44:12] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.43, 7.66, 7.95 [07:47:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:47:56] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [07:55:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.80, 8.93, 8.35 [07:56:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:57:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.52, 8.26, 8.17 [08:02:25] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.65, 8.37, 7.95 [08:04:21] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.71, 8.07, 7.88 [08:16:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:23:54] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 3.00, 1.94, 1.12 [08:25:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.22, 3.43, 2.71 [08:27:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.78, 2.78, 2.56 [08:27:54] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.72, 1.93, 1.32 [08:30:33] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.22, 8.37, 7.72 [08:31:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:31:54] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.28, 1.65, 1.35 [08:32:28] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.70, 7.83, 7.60 [08:41:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:50:02] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [08:50:03] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:53:59] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.39, 1.97, 1.65 [08:55:54] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.74, 1.94, 1.69 [08:59:54] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.28, 1.57, 1.59 [09:19:35] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [09:21:54] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.05, 1.85, 1.62 [09:23:54] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 0.71, 1.44, 1.50 [09:49:09] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [09:49:09] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [10:12:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:18:43] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [10:22:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:29:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.40, 9.71, 8.44 [10:31:32] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.98, 8.59, 7.89 [10:33:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:35:23] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.06, 8.44, 7.97 [10:37:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.74, 8.03, 8.20 [10:43:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:46:57] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.72, 8.97, 8.29 [10:47:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:48:53] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.46, 8.49, 8.20 [10:50:54] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 1.62, 2.04, 1.33 [10:52:17] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 4.22, 2.38, 1.31 [10:52:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.22, 8.39, 7.85 [10:52:54] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.48, 1.44, 1.19 [10:54:13] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 0.93, 1.73, 1.20 [10:54:27] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.87, 7.64, 7.63 [10:56:10] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 5.83, 3.20, 1.78 [10:57:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:02:00] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.54, 1.63, 1.53 [11:02:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:07:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:20:34] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.11, 8.67, 7.99 [11:24:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:24:51] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.24, 7.97, 7.55 [11:28:43] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.57, 9.02, 8.06 [11:30:31] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.34, 9.18, 8.37 [11:30:38] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.38, 9.05, 8.11 [11:30:38] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.82, 8.75, 8.09 [11:32:25] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.92, 8.76, 8.32 [11:32:33] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.76, 8.43, 8.05 [11:32:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.10, 9.31, 8.33 [11:34:29] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.79, 10.07, 8.71 [11:36:24] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.72, 8.82, 8.35 [11:37:55] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.32, 9.34, 8.77 [11:38:08] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.75, 9.52, 8.74 [11:38:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.36, 9.43, 8.75 [11:39:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:39:51] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.47, 9.06, 8.73 [11:40:02] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.91, 9.39, 8.77 [11:40:14] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.81, 9.54, 8.70 [11:41:47] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.66, 9.75, 9.03 [11:42:09] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.79, 9.19, 8.67 [11:43:42] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.66, 9.66, 9.08 [11:43:50] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.17, 10.12, 9.21 [11:44:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:45:44] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.46, 9.32, 9.01 [11:46:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 5.33, 7.60, 8.18 [11:47:34] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.74, 9.71, 9.20 [11:48:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 5.82, 7.86, 8.44 [11:49:23] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [11:49:23] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [11:53:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.88, 9.56, 9.37 [11:53:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.74, 9.57, 9.09 [11:54:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.59, 8.54, 8.31 [11:54:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:55:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.73, 8.78, 8.87 [11:55:48] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.94, 1.55, 1.21 [11:56:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.88, 8.09, 8.19 [11:56:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.43, 9.08, 8.68 [11:57:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:57:48] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.56, 1.64, 1.27 [11:58:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.29, 9.10, 8.74 [11:59:43] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset -0.02391836047 secs [11:59:48] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.82, 1.35, 1.21 [12:01:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 5.92, 7.31, 8.22 [12:02:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.86, 8.13, 8.40 [12:06:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.35, 8.49, 8.46 [12:06:31] PROBLEM - ns1 NTP time on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:07:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:07:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.88, 7.35, 8.34 [12:08:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.26, 8.20, 8.37 [12:10:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:11:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.30, 9.34, 8.72 [12:13:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.27, 8.62, 8.54 [12:18:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.88, 8.36, 8.29 [12:19:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.49, 8.43, 8.46 [12:20:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.84, 7.63, 8.02 [12:23:54] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 2.00, 1.71, 1.28 [12:27:02] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.80, 9.02, 8.46 [12:28:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.65, 8.86, 8.46 [12:29:04] uh [12:29:11] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.60, 3.73, 2.99 [12:29:20] i think the bot is having a stroke [12:29:54] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.40, 1.65, 1.40 [12:30:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.10, 8.30, 8.30 [12:30:32] Guest51: it rarely isn't [12:30:43] Nothing to worry about [12:30:49] what is it doing [12:31:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.12, 3.53, 3.00 [12:31:14] Telling us about things breaking Guest51 [12:31:30] thats alot of warings [12:31:39] is that normal [12:32:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.94, 8.62, 8.41 [12:32:49] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 4.57, 7.65, 8.14 [12:33:10] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.37, 3.32, 2.98 [12:36:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.08, 10.00, 8.99 [12:36:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.28, 8.57, 8.43 [12:36:28] UH [12:38:02] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.66, 8.14, 8.14 [12:38:04] This is normal, nothing is very broken [12:38:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.05, 9.27, 8.83 [12:38:25] oh [12:38:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.16, 8.32, 8.35 [12:39:56] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.37, 8.20, 8.16 [12:40:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:43:06] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.65, 3.32, 3.11 [12:43:45] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.71, 8.60, 8.35 [12:44:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:45:05] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.63, 2.91, 2.98 [12:45:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.01, 8.78, 8.43 [12:47:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.81, 8.77, 8.47 [12:48:29] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [12:49:03] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.08, 3.58, 3.28 [12:51:02] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.99, 3.03, 3.12 [12:51:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.84, 9.10, 8.25 [12:53:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.44, 9.16, 8.41 [12:53:54] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.14, 1.81, 1.56 [12:55:55] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.57, 1.72, 1.56 [12:56:44] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset -0.1037641168 secs [12:57:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.88, 9.60, 8.72 [12:57:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.35, 8.42, 8.43 [12:57:54] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.14, 1.51, 1.50 [12:58:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.70, 9.09, 8.77 [12:59:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.17, 9.38, 8.75 [12:59:37] PROBLEM - ns1 NTP time on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:59:51] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.58, 9.08, 8.49 [13:00:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.14, 8.46, 8.59 [13:00:57] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.10, 4.09, 3.56 [13:02:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.56, 8.10, 8.45 [13:02:56] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.93, 3.77, 3.51 [13:03:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.26, 9.14, 8.71 [13:03:42] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.11, 9.58, 8.76 [13:04:56] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.61, 3.94, 3.59 [13:05:38] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.44, 9.38, 8.77 [13:05:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.96, 9.15, 8.76 [13:06:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.36, 9.16, 8.82 [13:07:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.51, 8.11, 8.39 [13:07:34] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.04, 9.98, 9.06 [13:07:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.90, 9.74, 9.02 [13:09:29] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.61, 9.76, 9.11 [13:11:15] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.05494469404 secs [13:11:25] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.29, 10.18, 9.36 [13:11:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.76, 9.10, 8.96 [13:12:19] RECOVERY - ns1 Disk Space on ns1 is OK: DISK OK - free space: / 3698 MB (51% inode=82%); [13:13:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.57, 8.46, 8.45 [13:14:52] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.58, 3.72, 3.72 [13:15:57] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.19, 4.42, 3.40 [13:16:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.15, 7.96, 8.39 [13:17:11] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.19, 9.84, 9.50 [13:17:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.02, 9.39, 8.78 [13:17:33] PROBLEM - ns1 Current Load on ns1 is UNKNOWN: NRPE: Call to fork() failed [13:17:43] PROBLEM - ns1 ferm_active on ns1 is UNKNOWN: NRPE: Call to fork() failed [13:19:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.04, 10.55, 9.62 [13:20:29] PROBLEM - ns1 Current Load on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:20:41] PROBLEM - ns1 ferm_active on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:21:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.52, 9.53, 9.02 [13:21:51] PROBLEM - ns1 NTP time on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:22:01] PROBLEM - ns1 Disk Space on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:23:43] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.63, 3.46, 3.35 [13:24:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.99, 9.25, 8.81 [13:24:53] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.69, 9.64, 9.34 [13:25:39] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.66, 2.83, 3.13 [13:25:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 6.60, 9.00, 9.34 [13:26:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.44, 9.70, 9.03 [13:26:49] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.16, 9.05, 9.16 [13:27:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.97, 9.59, 9.17 [13:30:46] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.65, 3.61, 3.55 [13:31:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.14, 9.91, 9.44 [13:31:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.74, 9.73, 9.55 [13:32:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.82, 9.27, 9.13 [13:32:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.43, 3.19, 3.41 [13:34:32] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.77, 9.67, 9.35 [13:34:45] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.93, 2.78, 3.23 [13:35:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.69, 10.00, 9.74 [13:36:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.80, 9.09, 9.19 [13:39:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.34, 9.84, 9.67 [13:42:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.54, 9.30, 9.10 [13:44:10] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.28, 3.67, 3.19 [13:44:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.08, 8.63, 8.88 [13:44:40] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.59, 3.40, 3.30 [13:46:40] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.14, 4.04, 3.56 [13:47:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.24, 9.23, 9.16 [13:48:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.31, 3.68, 3.33 [13:48:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.75, 7.66, 8.44 [13:49:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.20, 9.32, 9.22 [13:50:40] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.71, 3.53, 3.47 [13:51:55] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.76, 3.31, 3.26 [13:52:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.60, 9.28, 8.98 [13:54:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.46, 8.50, 8.59 [13:54:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.90, 9.13, 8.96 [13:54:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.68, 3.54, 3.17 [13:56:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.00, 7.70, 8.30 [13:56:38] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.15, 4.20, 3.73 [13:57:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.86, 9.41, 9.81 [13:58:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.55, 3.81, 3.64 [14:00:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.58, 8.25, 8.38 [14:00:28] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.65, 3.26, 3.21 [14:02:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.58, 7.92, 8.24 [14:02:35] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.40, 4.16, 3.81 [14:04:35] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.18, 3.79, 3.72 [14:06:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.62, 8.98, 8.64 [14:07:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.25, 8.24, 8.49 [14:07:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.11, 9.43, 9.54 [14:08:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.38, 8.26, 8.41 [14:08:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.54, 9.64, 9.18 [14:10:32] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.09, 3.62, 3.62 [14:11:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.49, 8.71, 8.66 [14:12:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.91, 9.93, 9.40 [14:12:32] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.68, 3.67, 3.64 [14:13:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.27, 7.84, 8.35 [14:15:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.14, 4.25, 3.54 [14:16:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.89, 8.73, 8.64 [14:16:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.43, 9.96, 9.53 [14:16:30] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.27, 4.55, 3.97 [14:17:12] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.42, 3.90, 3.49 [14:18:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.92, 9.87, 9.55 [14:19:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.10, 8.62, 8.57 [14:19:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.67, 9.62, 9.92 [14:21:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.78, 7.71, 8.23 [14:23:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.10, 3.36, 3.37 [14:23:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.10, 9.57, 9.78 [14:25:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.98, 9.56, 9.74 [14:28:22] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 10.08, 8.62, 7.71 [14:28:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.96, 3.67, 4.00 [14:29:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.61, 3.42, 3.37 [14:29:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.20, 8.86, 8.55 [14:30:19] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.81, 8.60, 7.81 [14:30:26] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.88, 3.89, 4.02 [14:31:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.23, 3.37, 3.36 [14:31:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.41, 9.22, 8.71 [14:32:16] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.55, 8.04, 7.68 [14:33:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.75, 8.75, 8.61 [14:34:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.24, 9.39, 9.46 [14:36:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.61, 8.85, 9.24 [14:37:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.07, 9.17, 9.32 [14:39:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.10, 8.33, 8.47 [14:40:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.54, 9.42, 9.36 [14:43:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.10, 8.79, 8.67 [14:44:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.45, 9.27, 8.95 [14:44:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 1.40, 2.93, 3.79 [14:44:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.50, 9.00, 9.20 [14:46:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.59, 8.50, 8.70 [14:47:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.97, 9.76, 9.06 [14:52:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.50, 9.32, 8.93 [14:52:18] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.60, 2.27, 3.17 [14:52:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.73, 9.75, 9.34 [14:57:55] PROBLEM - wiki.bikinibottomdiscord.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.bikinibottomdiscord.org All nameservers failed to answer the query. [14:59:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:00:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.17, 9.35, 9.20 [15:03:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.05, 9.81, 9.91 [15:04:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.04, 8.84, 8.96 [15:04:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.69, 3.83, 3.46 [15:06:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.14, 8.63, 8.87 [15:06:11] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.50, 3.24, 3.28 [15:06:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:09:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.99, 9.18, 9.97 [15:11:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.45, 9.63, 10.04 [15:12:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.47, 9.76, 9.88 [15:14:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.58, 10.22, 10.04 [15:15:20] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.55, 3.03, 2.96 [15:15:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.28, 8.91, 9.07 [15:17:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.85, 8.95, 9.08 [15:19:12] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.27, 3.68, 3.23 [15:19:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.41, 9.11, 9.82 [15:20:03] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.50, 3.45, 2.99 [15:20:05] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.49, 4.15, 3.57 [15:20:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.77, 9.44, 9.81 [15:21:09] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.03, 3.50, 3.22 [15:21:28] .op [15:21:29] Attempting to OP... [15:22:02] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.59, 3.12, 2.93 [15:22:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.64, 3.95, 3.56 [15:22:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.41, 10.01, 9.95 [15:22:46] MacFan4000: ns1 shouldn't impact users [15:23:05] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.89, 3.33, 3.20 [15:23:14] Slightly slower dns response might happen but we're talking milliseconds every 5 minutes [15:23:28] it should fail straight over [15:23:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.08, 9.08, 9.59 [15:24:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.23, 9.60, 9.81 [15:25:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.56, 9.48, 9.69 [15:26:03] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.54, 4.07, 3.67 [15:26:49] RECOVERY - wiki.bikinibottomdiscord.org - reverse DNS on sslhost is OK: SSL OK - wiki.bikinibottomdiscord.org reverse DNS resolves to cp21.miraheze.org - CNAME OK [15:27:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.21, 9.66, 9.40 [15:28:02] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.74, 3.87, 3.64 [15:29:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.48, 9.68, 9.45 [15:30:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.57, 9.82, 9.74 [15:32:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.40, 9.24, 9.54 [15:34:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.72, 8.85, 8.75 [15:34:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.85, 9.84, 9.71 [15:35:59] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.95, 4.03, 3.73 [15:36:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.36, 9.47, 9.02 [15:36:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.43, 9.67, 9.67 [15:38:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.14, 9.94, 9.78 [15:40:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.69, 9.41, 9.60 [15:41:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.41, 8.99, 8.91 [15:41:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.63, 10.06, 9.59 [15:43:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.31, 8.61, 8.78 [15:43:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.93, 9.84, 9.56 [15:45:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.36, 7.75, 8.44 [15:47:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.43, 3.56, 3.07 [15:47:54] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 4.00, 3.95, 3.80 [15:48:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.62, 7.61, 8.32 [15:49:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.16, 3.46, 3.09 [15:51:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.75, 8.81, 8.68 [15:51:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.72, 9.83, 9.57 [15:53:52] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.35, 2.92, 3.36 [15:55:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.38, 3.14, 3.11 [15:55:31] PROBLEM - wiki.wisrail.de - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.wisrail.de' expires in 15 day(s) (Thu 07 Apr 2022 15:25:46 GMT +0000). [15:55:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.91, 8.16, 8.44 [15:56:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.87, 8.81, 8.59 [15:58:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.50, 8.38, 8.46 [15:58:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.69, 9.68, 9.44 [15:59:49] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.49, 3.79, 3.53 [16:02:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.16, 8.87, 8.67 [16:03:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.25, 9.56, 8.91 [16:03:47] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.48, 3.88, 3.66 [16:04:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.93, 9.31, 8.84 [16:05:47] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.06, 4.07, 3.76 [16:06:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.00, 9.64, 9.03 [16:07:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.17, 9.67, 9.87 [16:09:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.97, 9.89, 9.44 [16:10:02] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.32, 1.51, 1.24 [16:10:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 5.42, 7.73, 8.43 [16:10:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.86, 8.86, 9.53 [16:11:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.39, 9.59, 9.70 [16:11:44] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.55, 3.91, 3.83 [16:12:01] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.35, 1.40, 1.23 [16:13:44] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.63, 4.20, 3.94 [16:14:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.00, 8.97, 8.74 [16:15:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.42, 3.90, 3.86 [16:16:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.96, 9.00, 8.76 [16:16:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.33, 9.64, 9.58 [16:16:41] !log rebooting ns1 vis 1and1 portal [16:16:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:18:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.86, 9.49, 9.52 [16:19:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.07, 9.56, 9.22 [16:19:41] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.86, 4.06, 3.94 [16:20:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.65, 9.19, 8.86 [16:20:17] PROBLEM - ping6 on ns1 is CRITICAL: PING CRITICAL - Packet loss = 100% [16:20:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.42, 10.22, 9.79 [16:20:40] PROBLEM - ns1 Auth DNS on ns1 is CRITICAL: CRITICAL - Plugin timed out while executing system call [16:21:07] PROBLEM - Host ns1 is DOWN: PING CRITICAL - Packet loss = 100% [16:21:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.98, 9.22, 9.17 [16:21:50] PROBLEM - mail121 Current Load on mail121 is CRITICAL: CRITICAL - load average: 4.42, 3.67, 1.77 [16:23:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.43, 9.37, 9.79 [16:24:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.27, 9.56, 9.10 [16:24:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.54, 9.68, 9.71 [16:25:50] PROBLEM - mail121 Current Load on mail121 is WARNING: WARNING - load average: 0.29, 1.76, 1.40 [16:27:35] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.89, 3.32, 2.96 [16:27:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.47, 9.53, 9.71 [16:27:50] RECOVERY - mail121 Current Load on mail121 is OK: OK - load average: 0.30, 1.30, 1.27 [16:28:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.28, 9.70, 9.19 [16:29:34] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.26, 2.93, 2.86 [16:29:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.42, 9.37, 9.63 [16:31:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.74, 9.73, 9.22 [16:31:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.33, 9.70, 9.72 [16:33:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.93, 9.16, 9.06 [16:34:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.04, 9.77, 9.43 [16:34:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.82, 10.12, 9.68 [16:35:45] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.85, 3.92, 3.49 [16:36:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.86, 9.94, 9.67 [16:36:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.96, 3.62, 3.18 [16:37:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.41, 9.88, 9.84 [16:38:02] RECOVERY - Host ns1 is UP: PING OK - Packet loss = 0%, RTA = 102.72 ms [16:38:02] RECOVERY - ns1 SSH on ns1 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [16:38:07] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.03812018037 secs [16:38:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.16, 9.60, 9.41 [16:38:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.50, 10.13, 9.77 [16:38:29] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.72, 3.32, 3.13 [16:38:51] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [16:38:53] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 13 seconds ago with 0 failures [16:38:54] RECOVERY - ns1 ferm_active on ns1 is OK: OK ferm input default policy is set [16:39:24] RECOVERY - ns1 Disk Space on ns1 is OK: DISK OK - free space: / 3700 MB (51% inode=82%); [16:39:37] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.09, 3.78, 3.56 [16:39:39] RECOVERY - ns1 Auth DNS on ns1 is OK: DNS OK: 0.245 seconds response time. miraheze.org returns 198.244.148.90,2001:41d0:801:2000::1b80,2001:41d0:801:2000::4c25,51.195.220.68 [16:39:41] RECOVERY - ns1 conntrack_table_size on ns1 is OK: OK: nf_conntrack is 0 % full [16:39:52] RECOVERY - ping6 on ns1 is OK: PING OK - Packet loss = 0%, RTA = 103.83 ms [16:40:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.71, 9.21, 9.29 [16:40:09] RECOVERY - ns1 Current Load on ns1 is OK: OK - load average: 0.15, 0.14, 0.06 [16:40:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.08, 9.86, 9.73 [16:42:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.35, 10.43, 9.95 [17:01:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.90, 3.41, 3.97 [17:03:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.20, 3.68, 4.00 [17:03:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.30, 9.86, 9.65 [17:04:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.60, 9.80, 9.48 [17:04:57] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.18, 8.55, 8.18 [17:05:30] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.75, 3.30, 3.82 [17:05:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.77, 9.02, 9.36 [17:05:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.09, 10.07, 9.78 [17:07:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.80, 9.61, 9.53 [17:08:57] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.55, 7.85, 7.99 [17:09:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.47, 3.15, 3.05 [17:09:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 7.00, 4.04, 3.93 [17:10:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.52, 9.87, 9.68 [17:11:16] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.13, 3.21, 3.09 [17:13:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.55, 9.86, 9.73 [17:14:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.17, 9.17, 9.96 [17:15:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.77, 3.85, 3.95 [17:16:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.22, 9.57, 10.01 [17:17:30] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.34, 4.23, 4.12 [17:19:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.62, 10.33, 9.81 [17:19:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.01, 9.34, 9.83 [17:20:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.94, 9.73, 9.99 [17:22:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.68, 10.30, 10.15 [17:23:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.89, 9.82, 9.87 [17:25:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.12, 9.54, 9.68 [17:27:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.52, 3.65, 3.93 [17:29:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.96, 3.54, 3.22 [17:29:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.10, 4.18, 4.09 [17:29:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.28, 9.83, 9.97 [17:31:11] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.79, 2.87, 3.02 [17:31:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.71, 3.69, 3.93 [17:31:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.19, 9.97, 9.77 [17:31:49] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 1.39, 2.08, 1.45 [17:33:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.72, 4.14, 4.09 [17:33:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.66, 8.98, 9.43 [17:33:49] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.46, 1.48, 1.31 [17:34:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.13, 9.19, 8.95 [17:34:26] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.88, 8.04, 7.62 [17:36:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.76, 9.55, 9.12 [17:36:23] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.59, 7.85, 7.61 [17:37:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.19, 3.66, 3.91 [17:37:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.74, 9.54, 9.46 [17:39:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.44, 8.86, 9.23 [17:39:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.33, 10.00, 9.92 [17:43:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.77, 9.86, 9.92 [17:44:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.22, 9.66, 9.96 [17:46:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.75, 7.34, 8.20 [17:47:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.57, 8.59, 8.84 [17:49:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.22, 3.31, 3.50 [17:50:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.61, 9.66, 9.84 [17:51:09] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.74, 1.55, 1.47 [17:51:29] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.57, 3.05, 3.39 [17:52:05] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [17:52:05] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [17:52:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.47, 8.52, 8.40 [17:52:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.14, 9.09, 9.64 [17:53:09] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.60, 1.49, 1.45 [17:54:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.23, 8.97, 8.57 [17:55:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.46, 9.77, 9.40 [17:55:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.80, 9.56, 9.64 [17:56:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.89, 8.91, 8.57 [17:56:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.65, 10.17, 9.93 [17:57:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.95, 9.41, 9.58 [17:58:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.79, 9.70, 9.82 [18:01:57] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.64, 8.50, 8.05 [18:04:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.56, 10.48, 9.99 [18:05:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.43, 9.63, 9.36 [18:05:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.88, 9.53, 9.37 [18:06:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.47, 9.66, 9.02 [18:07:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 5.88, 8.25, 8.90 [18:07:48] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.13, 8.11, 8.08 [18:08:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.57, 9.53, 9.05 [18:08:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.44, 9.51, 9.72 [18:09:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.95, 9.54, 9.42 [18:10:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.63, 10.43, 10.03 [18:11:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.68, 9.09, 9.05 [18:11:39] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.17, 8.58, 8.28 [18:11:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.27, 10.71, 9.88 [18:14:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.42, 9.77, 9.29 [18:14:14] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.86, 8.78, 8.01 [18:16:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.61, 9.29, 9.17 [18:16:11] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 10.27, 9.19, 8.25 [18:18:08] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.42, 8.18, 7.99 [18:18:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.61, 9.53, 9.85 [18:20:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.82, 10.06, 9.51 [18:20:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.03, 9.93, 9.97 [18:21:38] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [18:22:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.47, 9.44, 9.78 [18:24:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.01, 9.81, 9.54 [18:24:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.42, 10.21, 10.02 [18:25:12] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.52, 2.16, 1.55 [18:25:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.99, 9.96, 9.99 [18:25:55] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.03, 6.78, 6.20 [18:26:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.84, 9.98, 9.95 [18:27:14] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 5.76, 7.53, 8.21 [18:27:50] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.74, 7.04, 6.39 [18:29:00] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.83, 1.99, 1.61 [18:29:45] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.89, 6.25, 6.17 [18:31:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.19, 9.97, 9.98 [18:32:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.35, 9.95, 9.85 [18:32:49] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.35, 1.65, 1.56 [18:33:02] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.62, 8.71, 8.54 [18:34:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.39, 9.61, 9.48 [18:35:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.03, 9.83, 9.94 [18:38:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.23, 9.52, 9.79 [18:40:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.24, 9.79, 9.65 [18:40:23] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.04, 8.61, 8.13 [18:40:57] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.45, 8.09, 8.44 [18:42:19] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.92, 8.28, 8.07 [18:44:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.44, 10.60, 9.99 [18:44:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.76, 9.68, 9.70 [18:46:09] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 10.76, 9.13, 8.42 [18:47:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.74, 9.95, 9.73 [18:48:06] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.84, 9.23, 8.54 [18:48:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.93, 9.89, 9.81 [18:48:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.09, 9.69, 9.76 [18:50:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.82, 10.10, 9.88 [18:50:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.82, 9.91, 9.81 [18:51:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.98, 9.94, 9.84 [18:51:59] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.71, 8.27, 8.29 [18:52:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.40, 9.31, 9.64 [18:52:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.27, 9.59, 9.71 [18:53:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.16, 10.50, 10.06 [18:53:34] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.53, 3.67, 3.27 [18:53:54] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.93, 1.74, 1.57 [18:54:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.30, 10.17, 9.92 [18:54:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.41, 2.94, 2.68 [18:55:30] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.15, 3.51, 3.26 [18:55:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.73, 9.88, 9.89 [18:55:54] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.27, 1.51, 1.50 [18:56:43] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.46, 2.74, 2.64 [18:57:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.98, 10.26, 10.03 [18:57:44] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.63, 8.67, 8.29 [18:59:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.65, 9.16, 9.95 [18:59:43] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.15, 8.56, 8.35 [19:00:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.36, 9.77, 9.91 [19:00:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.26, 9.47, 9.46 [19:01:19] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.24, 3.93, 3.51 [19:01:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.99, 9.74, 9.89 [19:01:37] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 11.06, 9.69, 8.77 [19:02:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.66, 10.03, 9.98 [19:02:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.02, 8.73, 9.18 [19:03:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.78, 3.62, 3.46 [19:03:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.46, 9.93, 9.94 [19:03:34] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.40, 9.59, 8.85 [19:03:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.81, 10.08, 10.10 [19:04:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.82, 9.95, 9.57 [19:05:12] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.23, 4.06, 3.65 [19:05:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.86, 9.58, 9.81 [19:05:33] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.05, 7.84, 8.21 [19:06:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:06:38] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.88, 3.44, 2.99 [19:07:05] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.83, 3.67, 3.26 [19:07:08] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.86, 3.68, 3.56 [19:08:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.57, 9.83, 9.63 [19:08:38] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.35, 3.77, 3.17 [19:09:04] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.33, 4.02, 3.44 [19:10:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.52, 9.88, 9.99 [19:10:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.35, 3.59, 3.17 [19:13:03] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.07, 3.47, 3.48 [19:13:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.43, 8.94, 8.50 [19:14:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 9.68, 10.07, 10.06 [19:15:16] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.91, 8.02, 8.47 [19:15:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.16, 8.30, 8.33 [19:16:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.35, 9.82, 9.97 [19:16:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.17, 10.15, 9.71 [19:17:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.82, 4.00, 3.74 [19:18:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.80, 10.23, 10.10 [19:18:34] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.57, 3.21, 3.21 [19:19:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.86, 8.65, 8.47 [19:19:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 14.03, 10.55, 9.77 [19:21:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.53, 8.07, 8.26 [19:22:59] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.63, 3.55, 3.77 [19:25:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.98, 9.66, 9.69 [19:26:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.63, 9.77, 9.95 [19:26:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.20, 9.83, 9.91 [19:27:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.75, 2.84, 3.29 [19:28:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.17, 9.98, 9.99 [19:28:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.56, 10.29, 10.08 [19:28:56] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.70, 4.36, 3.96 [19:30:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.49, 9.50, 9.79 [19:30:56] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.23, 3.84, 3.81 [19:33:43] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.99, 8.59, 8.42 [19:34:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.54, 9.68, 9.73 [19:34:54] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.25, 3.79, 3.79 [19:36:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.81, 9.83, 9.80 [19:36:58] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.61, 3.64, 3.73 [19:37:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.62, 9.27, 9.17 [19:38:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.28, 9.70, 9.73 [19:40:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.23, 8.92, 9.43 [19:41:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.64, 9.76, 9.45 [19:41:45] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.89, 8.61, 8.14 [19:42:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.26, 9.80, 9.66 [19:42:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.29, 9.28, 9.80 [19:42:56] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.33, 3.02, 3.38 [19:43:27] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.49, 8.22, 8.42 [19:43:42] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.32, 7.85, 7.93 [19:44:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.40, 9.10, 9.43 [19:45:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.08, 9.27, 9.28 [19:47:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.75, 9.09, 9.21 [19:48:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.12, 9.86, 9.85 [19:49:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.17, 9.52, 9.35 [19:51:12] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.39, 8.65, 8.50 [19:51:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.63, 9.17, 9.25 [19:52:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.51, 9.48, 9.42 [19:53:09] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.23, 8.18, 8.35 [19:53:18] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [19:53:18] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [19:53:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.06, 9.70, 9.44 [19:53:49] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.45, 2.24, 1.53 [19:56:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.44, 9.56, 9.86 [19:57:49] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.91, 1.64, 1.47 [19:58:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.99, 9.85, 9.65 [19:59:57] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 11.15, 9.16, 8.62 [20:00:02] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.53, 9.13, 8.57 [20:01:54] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.15, 9.06, 8.65 [20:03:49] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.15, 1.80, 1.60 [20:03:51] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.60, 8.45, 8.47 [20:03:56] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.88, 7.93, 8.22 [20:04:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.19, 9.71, 9.64 [20:04:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.79, 10.01, 9.90 [20:05:49] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.54, 1.62, 1.55 [20:06:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.81, 9.36, 9.69 [20:08:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.50, 9.80, 9.78 [20:08:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.42, 9.86, 9.84 [20:09:42] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.37, 8.32, 8.24 [20:09:49] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 1.87, 2.41, 1.92 [20:10:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.48, 3.14, 3.02 [20:10:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.53, 9.74, 9.80 [20:10:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.66, 3.78, 3.47 [20:11:49] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 0.71, 1.84, 1.77 [20:12:11] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.51, 2.92, 2.96 [20:13:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.21, 9.69, 9.97 [20:13:36] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.31, 7.76, 8.07 [20:13:49] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.67, 1.45, 1.64 [20:14:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.12, 9.62, 9.69 [20:16:27] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.68, 8.54, 8.25 [20:16:41] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.48, 3.16, 3.32 [20:17:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.41, 9.81, 9.95 [20:18:24] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.08, 8.20, 8.15 [20:18:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.07, 9.77, 9.77 [20:21:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.45, 9.95, 10.00 [20:22:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.66, 3.26, 2.97 [20:22:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.98, 9.47, 9.20 [20:22:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.67, 10.17, 9.88 [20:23:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.92, 8.36, 8.10 [20:23:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.86, 10.29, 10.12 [20:24:05] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.19, 3.13, 2.95 [20:26:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.58, 9.26, 9.25 [20:27:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.96, 9.72, 9.95 [20:28:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.57, 9.63, 9.37 [20:28:35] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.16, 3.65, 3.47 [20:29:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.33, 10.77, 10.31 [20:30:35] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.64, 2.96, 3.24 [20:30:57] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 10.36, 8.90, 8.45 [20:32:57] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.69, 8.33, 8.29 [20:33:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.91, 8.31, 8.32 [20:33:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.85, 9.65, 10.00 [20:34:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.87, 9.45, 9.42 [20:35:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 9.84, 9.82, 10.02 [20:39:30] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.60, 3.87, 3.56 [20:41:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.98, 9.06, 9.72 [20:45:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.05, 9.44, 9.65 [20:47:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.84, 3.74, 3.41 [20:47:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.74, 9.07, 8.42 [20:47:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.07, 3.57, 3.51 [20:47:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.89, 9.34, 9.60 [20:48:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.53, 9.31, 9.03 [20:50:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.86, 8.99, 8.96 [20:51:03] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.48, 3.29, 3.31 [20:51:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.75, 8.01, 8.16 [20:52:24] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [20:52:46] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 4.82, 2.85, 1.73 [20:52:53] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.44, 3.83, 3.23 [20:54:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 5.79, 7.69, 8.48 [20:54:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.83, 9.71, 9.99 [20:54:52] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.62, 3.40, 3.14 [20:56:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.76, 10.11, 10.10 [20:58:45] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.75, 1.74, 1.63 [20:58:50] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.33, 3.72, 3.36 [21:00:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 6.35, 8.74, 9.59 [21:00:30] PROBLEM - cp21 Stunnel HTTP for mw121 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:00:46] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.87, 1.44, 1.53 [21:00:52] PROBLEM - cp31 Stunnel HTTP for mw121 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:00:58] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:01:08] PROBLEM - cp20 Stunnel HTTP for mw121 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:01:12] PROBLEM - cp30 Stunnel HTTP for mw121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:02:48] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.44, 3.74, 3.42 [21:03:11] RECOVERY - cp20 Stunnel HTTP for mw121 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 8.664 second response time [21:03:17] RECOVERY - cp30 Stunnel HTTP for mw121 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 7.534 second response time [21:04:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.47, 8.84, 9.35 [21:04:32] RECOVERY - cp21 Stunnel HTTP for mw121 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 4.001 second response time [21:04:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.73, 3.68, 3.44 [21:04:54] RECOVERY - cp31 Stunnel HTTP for mw121 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 4.078 second response time [21:04:58] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 22336 bytes in 3.088 second response time [21:06:48] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.63, 9.18, 8.54 [21:07:57] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.97, 3.67, 3.42 [21:08:43] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.01, 8.37, 8.33 [21:08:46] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.13, 3.39, 3.38 [21:11:34] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.71, 9.19, 8.94 [21:12:34] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.31, 8.67, 8.47 [21:14:29] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.22, 9.03, 8.61 [21:15:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.19, 8.47, 7.78 [21:16:25] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.90, 8.93, 8.62 [21:17:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.40, 9.55, 9.19 [21:17:39] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.58, 4.32, 3.78 [21:19:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.87, 9.95, 9.38 [21:20:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.08, 9.77, 9.93 [21:21:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.99, 8.49, 8.07 [21:21:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.08, 3.56, 3.59 [21:21:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 6.23, 8.49, 8.92 [21:23:58] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [21:23:58] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [21:26:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.44, 9.36, 8.93 [21:26:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.48, 9.92, 9.88 [21:27:42] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 10.27, 9.06, 8.02 [21:28:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.73, 9.19, 9.62 [21:29:17] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.59, 2.97, 3.32 [21:29:39] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.82, 8.92, 8.10 [21:30:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.52, 9.21, 9.00 [21:30:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.22, 9.75, 9.75 [21:31:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [21:31:49] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.46, 2.00, 1.37 [21:32:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.32, 9.10, 9.51 [21:33:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.06, 9.33, 9.07 [21:33:33] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.22, 8.31, 8.07 [21:33:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.75, 9.09, 9.84 [21:34:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.14, 9.46, 9.10 [21:34:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [21:35:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.72, 9.64, 9.94 [21:35:49] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.39, 1.64, 1.35 [21:37:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.51, 9.54, 9.27 [21:37:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.33, 9.64, 9.92 [21:38:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.03, 9.47, 9.23 [21:38:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.35, 9.76, 9.59 [21:39:03] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.10, 3.28, 3.16 [21:39:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.45, 10.25, 9.56 [21:40:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.24, 9.53, 9.53 [21:41:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.94, 2.97, 3.05 [21:41:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.77, 3.31, 3.89 [21:41:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.65, 9.61, 9.41 [21:41:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.63, 10.45, 10.17 [21:44:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.36, 9.40, 9.43 [21:46:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.77, 9.59, 9.50 [21:47:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.18, 8.63, 8.14 [21:47:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.59, 9.89, 9.57 [21:48:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.15, 9.37, 9.27 [21:49:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.30, 9.38, 9.88 [21:50:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.72, 9.79, 9.46 [21:51:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.92, 8.46, 8.18 [21:51:27] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.85, 3.62, 3.15 [21:51:29] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.97, 2.81, 3.29 [21:52:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.19, 9.99, 9.55 [21:53:26] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.78, 3.60, 3.20 [21:54:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.55, 9.90, 9.58 [21:55:26] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.35, 3.79, 3.32 [21:55:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.73, 3.48, 3.44 [21:55:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.31, 9.55, 9.70 [21:55:50] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.88, 4.72, 3.70 [21:57:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.86, 8.69, 8.37 [21:57:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.72, 3.47, 3.43 [21:57:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.09, 9.80, 9.87 [21:58:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.71, 9.87, 9.61 [21:59:24] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.75, 3.76, 3.45 [21:59:29] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.91, 3.28, 3.37 [22:00:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.64, 9.65, 9.56 [22:01:45] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.23, 8.68, 7.99 [22:02:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.50, 10.19, 9.76 [22:03:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.29, 8.29, 8.31 [22:03:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.41, 3.51, 3.46 [22:03:35] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.66, 3.80, 3.72 [22:03:42] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.57, 8.27, 7.93 [22:03:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.06, 9.63, 9.83 [22:05:22] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.81, 3.26, 3.33 [22:05:29] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.36, 3.32, 3.39 [22:05:32] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.48, 4.11, 3.83 [22:05:57] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 1.39, 2.21, 1.49 [22:06:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.54, 9.38, 9.29 [22:07:28] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 4.00, 3.88, 3.77 [22:07:57] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.55, 1.67, 1.38 [22:08:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.20, 9.41, 9.32 [22:09:25] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.09, 3.83, 3.76 [22:10:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.76, 9.69, 9.42 [22:10:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.49, 9.69, 9.71 [22:10:54] PROBLEM - www.lab612.at - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/lib/pyt [22:10:54] t-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query lab612.at. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [22:11:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.11, 3.73, 3.74 [22:11:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.36, 9.21, 9.17 [22:11:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.32, 11.12, 10.21 [22:12:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.00, 10.54, 10.01 [22:14:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.77, 9.81, 9.52 [22:17:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.07, 9.46, 9.40 [22:17:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.23, 9.68, 9.82 [22:18:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.44, 3.78, 3.45 [22:19:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.54, 9.94, 9.58 [22:19:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.34, 9.92, 9.88 [22:20:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 6.35, 8.95, 9.60 [22:21:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 6.96, 8.85, 9.23 [22:21:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.77, 9.45, 9.72 [22:22:24] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.21, 4.31, 3.70 [22:22:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.60, 9.47, 9.70 [22:24:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.07, 9.35, 9.63 [22:25:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.42, 9.14, 9.22 [22:29:03] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.95, 3.17, 3.40 [22:29:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.65, 9.37, 9.35 [22:29:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.11, 9.43, 9.50 [22:30:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.06, 9.22, 8.91 [22:31:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.72, 9.58, 9.41 [22:31:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.65, 9.20, 9.41 [22:32:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.64, 3.24, 3.03 [22:32:57] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 11.77, 9.00, 7.98 [22:33:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.45, 3.74, 3.61 [22:33:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.92, 9.71, 9.56 [22:34:11] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.03, 3.07, 2.99 [22:34:20] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.51, 3.43, 3.71 [22:34:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.45, 9.25, 9.33 [22:34:57] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.19, 8.40, 7.90 [22:35:03] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.65, 3.93, 3.68 [22:35:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.55, 9.41, 9.40 [22:37:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.62, 3.50, 3.55 [22:38:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.06, 4.24, 3.92 [22:40:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.09, 9.70, 9.58 [22:42:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.17, 9.33, 9.50 [22:43:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.37, 3.83, 3.63 [22:43:29] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.46, 1.95, 1.54 [22:43:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.04, 9.59, 9.47 [22:44:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.01, 3.62, 3.31 [22:45:29] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.77, 1.46, 1.41 [22:45:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.49, 9.21, 9.32 [22:45:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.49, 9.26, 9.68 [22:50:04] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.70, 3.18, 3.27 [22:51:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.71, 9.79, 9.72 [22:52:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.50, 7.50, 8.45 [22:52:37] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [22:53:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.84, 3.97, 3.94 [22:54:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.46, 8.99, 8.94 [22:55:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.01, 4.04, 3.98 [22:56:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.57, 8.97, 8.81 [22:57:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.71, 3.89, 3.93 [22:57:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.85, 7.85, 8.47 [22:57:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.54, 9.99, 9.86 [22:58:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.19, 8.55, 8.67 [22:58:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.33, 9.00, 8.96 [23:00:02] !log [@test101] starting deploy of {'l10nupdate': True} to all [23:00:03] !log [@mwtask111] starting deploy of {'l10nupdate': True} to all [23:00:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:00:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:03:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.02, 4.18, 4.02 [23:05:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.45, 3.50, 3.79 [23:05:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.39, 9.03, 9.32 [23:06:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.45, 9.59, 8.99 [23:08:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.87, 9.71, 9.11 [23:08:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.63, 8.10, 8.50 [23:10:18] RECOVERY - www.lab612.at - reverse DNS on sslhost is OK: SSL OK - www.lab612.at reverse DNS resolves to cp20.miraheze.org - CNAME OK [23:11:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.16, 2.35, 3.21 [23:11:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.98, 9.82, 9.66 [23:14:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.03, 8.83, 8.67 [23:17:57] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.53, 3.61, 3.48 [23:20:02] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.56, 3.46, 3.98 [23:20:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.92, 9.02, 8.90 [23:20:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.99, 1.60, 0.95 [23:21:29] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.43, 1.86, 1.31 [23:21:50] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.62, 3.97, 3.64 [23:21:52] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.66, 8.72, 8.32 [23:22:01] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.08, 4.71, 4.38 [23:22:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.34, 8.18, 8.60 [23:22:24] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.33, 1.47, 0.99 [23:22:55] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.90, 7.58, 5.88 [23:23:23] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.84, 1.83, 1.37 [23:23:46] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.80, 3.70, 3.59 [23:23:48] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.10, 8.33, 8.24 [23:24:06] !log [@test101] finished deploy of {'l10nupdate': True} to all - SUCCESS in 1443s [23:24:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.32, 9.15, 8.89 [23:24:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:24:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:24:55] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.62, 7.33, 5.98 [23:25:18] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 0.92, 1.52, 1.31 [23:26:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.77, 9.06, 8.88 [23:27:41] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.36, 9.43, 8.76 [23:28:22] !log [@mwtask111] finished deploy of {'l10nupdate': True} to all - SUCCESS in 1699s [23:28:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:28:55] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 3.89, 6.09, 5.84 [23:30:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.22, 7.88, 8.45 [23:30:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:31:33] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.89, 9.40, 8.85 [23:33:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.08, 9.22, 8.86 [23:33:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.29, 9.37, 9.03 [23:34:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.89, 9.35, 8.97 [23:35:24] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.01, 3.08, 3.32 [23:35:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.90, 9.67, 9.19 [23:35:56] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.50, 3.19, 3.90 [23:36:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.05, 9.52, 9.10 [23:36:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.93, 8.71, 8.66 [23:37:55] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 8.05, 4.94, 4.44 [23:38:53] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.79, 8.24, 8.49 [23:39:19] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.22, 4.04, 3.66 [23:39:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.69, 9.87, 9.36 [23:39:43] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.69, 3.48, 3.26 [23:41:15] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.36, 3.34, 3.45 [23:41:42] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.08, 3.20, 3.18 [23:41:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.76, 9.57, 9.32 [23:42:44] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.75, 8.93, 8.75 [23:44:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.59, 7.90, 8.50 [23:45:08] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.95, 3.89, 3.63 [23:45:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.39, 9.59, 9.11 [23:45:35] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.40, 8.42, 7.54 [23:45:52] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.05, 3.16, 3.86 [23:47:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.24, 9.42, 9.11 [23:47:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.55, 9.84, 9.51 [23:47:52] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.08, 3.35, 3.83 [23:48:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.20, 9.38, 8.95 [23:48:29] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.20, 9.24, 8.93 [23:49:29] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.14, 8.09, 7.62 [23:49:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.12, 10.32, 9.47 [23:49:51] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.16, 2.94, 3.62 [23:51:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.35, 9.92, 9.66 [23:53:49] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.07, 3.18, 3.53 [23:55:49] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.44, 3.30, 3.54 [23:56:09] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.83, 9.69, 9.40 [23:57:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.79, 3.98, 3.82 [23:57:24] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.74, 2.72, 1.79 [23:57:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.34, 9.92, 9.71 [23:57:48] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.98, 3.84, 3.68 [23:58:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.52, 10.28, 9.65 [23:59:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.75, 8.44, 7.95 [23:59:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.71, 9.43, 9.60