[00:00:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.63, 9.60, 9.47 [00:01:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.97, 4.07, 3.87 [00:01:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.48, 8.05, 7.87 [00:01:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.17, 9.87, 9.79 [00:01:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.48, 3.61, 3.62 [00:02:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.08, 10.02, 9.63 [00:03:32] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.49, 3.74, 3.29 [00:03:46] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.52, 4.07, 3.79 [00:04:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.37, 9.54, 9.49 [00:04:21] PROBLEM - mw101 Puppet on mw101 is CRITICAL: CRITICAL: Puppet has 387 failures. Last run 3 minutes ago with 387 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.gpg],File[/usr/local/bin/puppet-enabled],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [00:05:23] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.03, 1.87, 1.81 [00:06:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.59, 9.41, 9.77 [00:07:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.61, 9.88, 9.75 [00:07:44] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.04, 3.61, 3.68 [00:09:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.36, 3.78, 3.53 [00:09:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.66, 9.65, 9.69 [00:09:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.58, 9.55, 9.60 [00:11:22] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.83, 1.31, 1.59 [00:12:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.45, 10.41, 9.91 [00:13:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.36, 9.76, 9.69 [00:15:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.29, 3.51, 3.98 [00:15:28] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.48, 3.18, 3.38 [00:17:03] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.42, 3.82, 4.03 [00:17:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.68, 9.99, 9.83 [00:19:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.08, 3.22, 3.79 [00:19:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 9.45, 10.06, 9.90 [00:19:40] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.42, 3.01, 3.35 [00:19:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.86, 9.51, 9.82 [00:21:16] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [00:21:17] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [00:21:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.19, 10.01, 9.95 [00:23:20] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.71, 1.82, 1.68 [00:23:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.58, 3.86, 3.61 [00:23:43] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.57, 9.66, 9.85 [00:25:36] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.64, 3.65, 3.56 [00:25:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.31, 9.92, 9.92 [00:26:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.02, 9.69, 9.95 [00:27:19] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.27, 1.56, 1.61 [00:27:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.90, 9.99, 9.95 [00:29:03] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.24, 3.47, 3.54 [00:29:20] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.67, 3.65, 3.48 [00:30:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.02, 9.38, 9.09 [00:31:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.92, 3.33, 3.49 [00:31:19] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.99, 3.34, 3.38 [00:31:27] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.60, 8.41, 7.82 [00:31:34] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.85, 3.80, 3.62 [00:32:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.70, 8.80, 8.92 [00:32:30] RECOVERY - mw101 Puppet on mw101 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:33:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.81, 4.14, 3.78 [00:33:24] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.63, 7.55, 7.57 [00:34:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.04, 9.62, 9.66 [00:34:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.43, 9.62, 9.21 [00:35:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.40, 3.91, 3.74 [00:35:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.86, 3.74, 3.59 [00:36:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.52, 9.54, 9.23 [00:37:32] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.96, 3.52, 3.60 [00:38:17] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.33, 1.68, 1.55 [00:39:43] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.66, 9.73, 9.79 [00:41:15] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.27, 2.96, 3.31 [00:41:28] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.56, 8.44, 7.67 [00:43:25] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.18, 7.60, 7.46 [00:43:30] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.37, 3.07, 3.39 [00:44:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.40, 9.69, 9.84 [00:44:16] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.67, 1.88, 1.70 [00:44:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.78, 9.14, 8.95 [00:45:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.59, 3.06, 3.39 [00:46:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.01, 9.51, 9.74 [00:46:16] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.35, 1.67, 1.65 [00:48:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.38, 9.33, 9.08 [00:48:57] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 7.99, 8.51, 8.06 [00:50:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.31, 9.44, 9.65 [00:50:15] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.74, 1.86, 1.78 [00:50:20] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.76, 1.80, 1.52 [00:51:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.73, 3.32, 3.33 [00:52:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.50, 9.82, 9.76 [00:52:14] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.92, 1.47, 1.43 [00:52:15] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.84, 1.48, 1.65 [00:52:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.03, 9.37, 9.10 [00:52:57] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.82, 7.81, 7.89 [00:53:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.59, 4.03, 3.58 [00:58:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.30, 9.85, 9.51 [00:59:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.55, 3.39, 3.19 [00:59:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.27, 3.39, 3.50 [01:01:07] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.96, 3.18, 3.13 [01:01:29] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.62, 3.06, 3.36 [01:01:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.44, 9.19, 9.89 [01:03:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.12, 9.58, 9.94 [01:03:48] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.28, 8.52, 8.10 [01:05:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.77, 9.34, 9.79 [01:05:45] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.37, 8.49, 8.14 [01:06:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.50, 9.94, 9.61 [01:07:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.58, 3.78, 3.54 [01:07:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.34, 9.88, 9.92 [01:08:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.53, 9.83, 9.62 [01:09:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.42, 9.63, 9.82 [01:11:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.61, 3.99, 3.70 [01:11:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.38, 10.93, 10.29 [01:12:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.74, 9.96, 9.69 [01:13:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.74, 4.41, 3.88 [01:14:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.91, 9.14, 9.42 [01:16:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.77, 9.01, 9.86 [01:16:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.86, 9.64, 9.56 [01:17:10] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.70, 1.73, 1.65 [01:17:14] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.57, 3.54, 3.31 [01:18:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.37, 9.96, 10.10 [01:18:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.33, 9.23, 9.43 [01:19:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.80, 3.78, 3.78 [01:19:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.68, 9.45, 9.91 [01:20:23] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [01:21:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.08, 4.63, 3.75 [01:21:10] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.64, 2.31, 1.89 [01:21:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.81, 4.04, 3.86 [01:23:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.12, 3.61, 3.48 [01:25:03] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.01, 3.70, 3.52 [01:25:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.31, 9.48, 9.70 [01:27:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.46, 3.72, 3.56 [01:27:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.61, 9.46, 9.66 [01:28:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.51, 9.53, 9.39 [01:29:08] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.30, 1.93, 1.92 [01:30:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.78, 9.66, 9.87 [01:31:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.41, 3.78, 3.59 [01:31:08] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.19, 1.93, 1.92 [01:32:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.30, 8.99, 9.26 [01:32:52] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.76, 3.68, 3.39 [01:33:08] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.19, 1.68, 1.83 [01:33:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.50, 9.83, 9.74 [01:34:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.14, 9.75, 9.50 [01:34:51] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.63, 3.28, 3.28 [01:35:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.90, 3.53, 3.55 [01:35:07] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.73, 1.91, 1.88 [01:35:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.68, 9.55, 9.67 [01:36:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.95, 10.51, 10.07 [01:37:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.32, 3.13, 3.40 [01:37:07] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.65, 1.77, 1.83 [01:37:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.89, 10.06, 9.84 [01:40:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.83, 9.73, 9.86 [01:41:07] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.28, 2.02, 1.91 [01:42:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.25, 10.68, 10.21 [01:42:35] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 10.92, 9.08, 8.17 [01:43:06] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.19, 1.69, 1.81 [01:44:32] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.51, 9.02, 8.28 [01:45:06] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.83, 1.41, 1.69 [01:45:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.60, 8.93, 9.83 [01:45:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.94, 9.44, 9.73 [01:46:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.57, 9.36, 9.81 [01:46:29] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 10.19, 9.39, 8.50 [01:48:26] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 7.25, 8.56, 8.30 [01:49:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.96, 9.88, 9.96 [01:50:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.32, 9.42, 9.69 [01:50:22] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.93, 8.10, 8.14 [01:51:57] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [01:51:57] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [01:52:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.79, 9.59, 9.73 [01:52:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.87, 9.52, 9.89 [01:52:43] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.82, 3.26, 3.12 [01:53:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.71, 3.40, 3.92 [01:53:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.48, 9.19, 9.36 [01:53:43] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.41, 3.33, 3.24 [01:54:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.08, 9.69, 9.91 [01:54:42] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.69, 3.05, 3.06 [01:54:57] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.17, 8.01, 7.40 [01:55:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.45, 3.85, 4.02 [01:56:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.21, 9.32, 9.74 [01:56:57] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.21, 7.79, 7.40 [01:57:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.33, 3.64, 3.92 [01:57:35] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.43, 3.13, 3.20 [01:57:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.50, 9.67, 9.54 [01:58:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.54, 9.37, 9.48 [01:59:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.69, 3.74, 3.90 [02:00:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.86, 9.19, 9.38 [02:01:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.27, 3.69, 3.87 [02:03:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 6.53, 9.08, 9.91 [02:05:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.80, 4.12, 3.97 [02:06:02] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.05, 1.78, 1.64 [02:07:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.66, 8.98, 9.60 [02:08:02] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.47, 1.56, 1.57 [02:10:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.80, 9.48, 9.50 [02:10:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:11:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.95, 3.69, 3.85 [02:11:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.70, 9.23, 9.59 [02:11:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.15, 9.65, 9.31 [02:12:01] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.87, 1.95, 1.73 [02:13:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:13:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.61, 9.77, 9.41 [02:14:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.10, 7.44, 8.37 [02:14:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.78, 9.48, 9.55 [02:17:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.16, 9.44, 9.50 [02:18:00] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.96, 2.38, 1.95 [02:20:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.32, 8.64, 8.60 [02:21:29] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.86, 2.61, 3.22 [02:21:30] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [02:21:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.65, 9.44, 9.56 [02:22:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.11, 8.27, 8.46 [02:24:00] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.17, 1.87, 1.86 [02:25:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.09, 9.82, 9.42 [02:26:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 6.95, 8.56, 8.62 [02:27:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.86, 9.64, 9.40 [02:27:59] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.12, 1.44, 1.67 [02:28:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.45, 9.21, 8.83 [02:30:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.10, 9.25, 8.90 [02:30:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.70, 3.36, 3.30 [02:31:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.30, 10.10, 9.78 [02:32:26] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.59, 3.14, 3.23 [02:33:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.83, 9.68, 9.66 [02:34:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.15, 9.82, 9.18 [02:34:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.97, 9.86, 9.31 [02:35:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.13, 9.97, 9.77 [02:36:23] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.46, 3.83, 3.52 [02:36:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.29, 9.48, 9.24 [02:37:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 6.81, 9.01, 9.45 [02:38:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.71, 9.23, 9.14 [02:38:23] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.68, 4.45, 3.76 [02:38:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:40:22] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.03, 3.72, 3.57 [02:45:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:46:20] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.86, 3.20, 3.39 [02:48:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.48, 8.68, 8.79 [02:49:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.48, 9.23, 9.16 [02:51:03] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [02:52:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.29, 9.52, 9.16 [02:55:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.67, 9.89, 9.57 [02:55:52] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.48, 3.96, 3.25 [02:57:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.38, 8.91, 8.84 [02:58:15] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.99, 3.66, 3.09 [02:58:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.55, 9.47, 9.18 [02:59:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.44, 10.06, 9.69 [02:59:45] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.71, 3.77, 3.33 [03:00:14] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.13, 3.03, 2.93 [03:00:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.97, 9.74, 9.33 [03:01:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.40, 9.82, 9.65 [03:02:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.79, 9.91, 9.43 [03:03:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.73, 8.88, 8.95 [03:04:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.68, 10.00, 9.53 [03:05:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:06:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.05, 9.17, 9.28 [03:07:30] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.01, 2.83, 3.07 [03:07:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.67, 10.36, 9.85 [03:07:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.65, 9.37, 9.10 [03:08:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 6.65, 9.00, 9.31 [03:09:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.29, 9.15, 9.05 [03:11:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:12:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.12, 9.43, 9.41 [03:13:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.53, 9.85, 9.89 [03:14:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.91, 9.65, 9.50 [03:16:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.23, 7.38, 8.33 [03:17:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.24, 9.57, 9.74 [03:20:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.39, 8.61, 8.63 [03:22:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.46, 9.83, 9.08 [03:22:20] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.31, 1.79, 1.40 [03:23:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.84, 9.97, 9.99 [03:24:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.43, 9.36, 9.01 [03:24:14] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.23, 1.70, 1.42 [03:25:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.23, 8.00, 8.46 [03:26:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:29:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.21, 10.10, 9.96 [03:29:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.36, 8.96, 8.75 [03:31:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.02, 9.97, 9.95 [03:32:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.27, 7.83, 8.45 [03:33:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:33:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.64, 8.18, 8.47 [03:34:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.54, 9.38, 8.98 [03:36:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.04, 8.99, 8.80 [03:36:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.45, 9.10, 8.91 [03:38:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.06, 9.35, 9.01 [03:40:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.33, 7.61, 8.28 [03:40:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.52, 8.92, 8.89 [03:42:25] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.63, 9.80, 9.08 [03:43:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:44:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.01, 9.57, 9.07 [03:46:13] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.78, 9.37, 9.06 [03:49:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.78, 8.94, 8.92 [03:50:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:51:46] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.36, 1.80, 1.44 [03:52:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.92, 9.59, 8.76 [03:52:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.52, 7.75, 8.35 [03:53:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.63, 9.42, 9.15 [03:53:46] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.48, 1.76, 1.47 [03:53:50] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.17, 7.83, 8.48 [03:55:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:55:45] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.48, 1.56, 1.42 [03:56:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.73, 9.16, 8.79 [03:57:48] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.29, 3.53, 3.24 [04:00:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.87, 7.90, 8.36 [04:00:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:01:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.78, 7.35, 8.25 [04:01:47] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.71, 2.91, 3.09 [04:07:44] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.45, 3.07, 3.07 [04:09:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.50, 2.59, 2.90 [04:10:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:13:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.00, 8.48, 8.09 [04:15:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.79, 10.44, 8.83 [04:19:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:23:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.61, 9.65, 9.41 [04:27:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.22, 9.31, 9.28 [04:28:35] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.88, 3.41, 3.06 [04:29:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:29:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 6.69, 8.60, 9.04 [04:30:35] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.64, 3.08, 2.97 [04:35:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.38, 7.22, 8.30 [04:36:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:36:48] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.44, 8.64, 8.01 [04:38:31] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 7.93, 4.80, 3.60 [04:38:43] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.61, 8.19, 7.92 [04:39:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.66, 3.66, 3.07 [04:41:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:41:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.99, 8.97, 8.65 [04:43:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.88, 3.59, 3.17 [04:44:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.58, 3.51, 3.42 [04:45:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.85, 4.24, 3.44 [04:47:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.40, 3.86, 3.40 [04:47:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:48:27] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.21, 3.31, 3.36 [04:49:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.19, 8.22, 8.50 [04:51:03] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.85, 3.82, 3.44 [04:52:16] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [04:52:16] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [04:52:37] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [04:53:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.06, 3.03, 3.19 [04:57:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:59:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.55, 3.27, 3.20 [05:00:22] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.80, 3.56, 3.37 [05:01:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.59, 2.99, 3.11 [05:02:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.93, 3.01, 3.19 [05:03:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:06:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.63, 8.88, 8.15 [05:11:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.71, 8.98, 8.38 [05:12:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.11, 8.30, 8.19 [05:13:16] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.94, 8.75, 8.36 [05:13:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:17:05] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 5.64, 7.51, 7.98 [05:20:37] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:21:50] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [05:31:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:41:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:54:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:59:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:04:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:19:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:24:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:34:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:35:54] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.96, 1.67, 1.36 [06:37:54] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.26, 1.54, 1.35 [06:57:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.71, 8.57, 7.82 [06:59:21] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.53, 8.87, 8.02 [06:59:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:05:08] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.35, 8.44, 8.12 [07:06:28] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.22, 8.35, 7.58 [07:08:23] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.86, 8.24, 7.64 [07:11:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.55, 8.83, 8.36 [07:12:55] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.63, 9.41, 8.39 [07:13:50] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.84, 8.46, 8.29 [07:14:49] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.12, 9.19, 8.43 [07:16:43] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.55, 8.42, 8.25 [07:19:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:22:26] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.68, 2.93, 2.51 [07:24:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.57, 8.12, 8.01 [07:24:26] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.41, 2.31, 2.33 [07:24:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:26:14] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.35, 8.28, 8.08 [07:34:19] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.62, 3.43, 3.00 [07:34:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:35:46] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.45, 8.76, 8.28 [07:36:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.63, 4.30, 3.37 [07:37:12] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.61, 9.18, 8.35 [07:37:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.41, 8.18, 8.12 [07:38:17] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.22, 3.76, 3.28 [07:41:02] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 5.59, 7.52, 7.89 [07:41:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.07, 9.07, 8.47 [07:42:20] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.87, 2.87, 3.04 [07:43:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.55, 9.40, 8.67 [07:49:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.63, 8.28, 8.43 [07:55:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.27, 8.18, 7.55 [07:58:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.72, 8.97, 7.95 [07:59:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.55, 8.04, 7.71 [08:02:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.23, 9.31, 8.27 [08:04:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.45, 8.81, 8.27 [08:04:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.42, 8.78, 8.34 [08:04:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.29, 9.16, 8.33 [08:05:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:05:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.02, 8.53, 8.06 [08:06:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.54, 8.11, 8.10 [08:06:13] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.60, 8.08, 8.15 [08:07:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.16, 8.06, 7.95 [08:10:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.74, 8.29, 8.21 [08:10:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:11:56] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.94, 8.66, 8.31 [08:15:44] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.80, 8.30, 8.27 [08:20:53] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.52, 8.32, 7.90 [08:21:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.56, 8.65, 8.14 [08:22:59] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.27, 6.63, 5.66 [08:23:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:24:43] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.41, 8.01, 7.89 [08:24:55] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 3.70, 5.70, 5.45 [08:27:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.29, 8.25, 8.18 [08:28:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:42:34] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.69, 8.57, 8.18 [08:44:28] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.03, 7.93, 7.99 [08:51:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:54:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.50, 8.55, 8.15 [08:56:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.85, 8.04, 8.02 [08:56:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:00:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:02:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.67, 8.23, 7.70 [09:06:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.89, 8.18, 7.80 [09:15:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:38:56] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.20, 1.81, 1.29 [09:39:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:40:56] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.79, 1.41, 1.20 [09:41:02] PROBLEM - cloud10 Puppet on cloud10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [09:44:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:47:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:56:43] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 1.97, 2.27, 1.39 [09:57:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:00:37] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.78, 1.56, 1.29 [10:07:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:11:01] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [10:12:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:28:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:33:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:57:56] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [10:57:56] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [11:00:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:03:59] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.93, 3.36, 2.68 [11:05:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:05:58] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.42, 3.00, 2.63 [11:11:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:11:56] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.57, 9.20, 7.89 [11:13:50] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.39, 8.05, 7.62 [11:26:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:27:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.46, 8.21, 7.18 [11:30:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:31:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.82, 8.24, 7.45 [11:35:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:38:14] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.93, 8.79, 8.06 [11:40:02] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.46, 9.01, 8.30 [11:41:56] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.20, 9.28, 8.50 [11:42:05] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.84, 8.43, 8.07 [11:45:45] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.22, 8.47, 8.37 [11:46:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:47:57] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 3.22, 5.00, 2.70 [11:53:42] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.23, 8.51, 8.25 [11:53:57] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 0.75, 1.84, 1.96 [11:55:38] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.47, 8.43, 8.25 [11:57:02] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [11:59:30] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.68, 8.92, 8.53 [12:00:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.34, 8.84, 8.14 [12:01:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.18, 9.25, 8.69 [12:01:57] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.33, 1.51, 1.69 [12:02:40] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.86, 4.19, 3.26 [12:02:48] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.60, 3.43, 2.93 [12:03:52] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.42, 9.86, 8.65 [12:03:57] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.53, 1.40, 1.62 [12:04:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.19, 8.98, 8.38 [12:04:39] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.76, 3.55, 3.14 [12:04:47] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.57, 2.72, 2.73 [12:05:47] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.89, 9.59, 8.71 [12:06:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.38, 10.01, 8.84 [12:06:38] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.10, 3.26, 3.07 [12:07:12] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.95, 9.82, 9.13 [12:08:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.38, 9.62, 8.83 [12:09:08] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 9.32, 10.01, 9.30 [12:09:32] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.94, 8.33, 7.51 [12:11:03] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.44, 9.49, 9.20 [12:11:28] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 10.36, 8.81, 7.77 [12:11:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.50, 9.43, 8.69 [12:13:25] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 5.87, 7.50, 7.40 [12:13:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.76, 9.42, 8.77 [12:14:54] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.40, 9.73, 9.32 [12:16:50] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.73, 9.30, 9.21 [12:17:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.51, 7.94, 8.42 [12:18:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.89, 8.99, 8.69 [12:19:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.28, 7.90, 8.32 [12:21:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.96, 9.11, 8.84 [12:22:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.53, 9.09, 8.81 [12:22:55] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.95, 6.09, 5.28 [12:24:55] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.39, 6.18, 5.41 [12:29:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.37, 7.76, 8.39 [12:30:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.26, 8.12, 8.46 [12:32:26] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.11, 3.65, 3.19 [12:34:25] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.82, 3.40, 3.16 [12:37:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.70, 9.21, 8.54 [12:38:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.44, 9.13, 8.85 [12:38:23] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.02, 4.26, 3.55 [12:38:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.31, 8.20, 8.50 [12:39:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.52, 8.13, 8.22 [12:42:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.76, 7.92, 8.42 [12:46:20] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.95, 3.93, 3.79 [12:47:27] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.36, 3.92, 3.33 [12:48:19] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.41, 4.81, 4.12 [12:49:18] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.18, 4.08, 3.49 [12:49:27] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.35, 3.81, 3.37 [12:51:14] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.09, 3.88, 3.49 [12:51:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.60, 9.31, 8.62 [12:52:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.97, 3.80, 3.87 [12:53:11] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.54, 8.69, 8.47 [12:53:54] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.29, 1.93, 1.58 [12:55:05] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.33, 8.91, 8.55 [12:55:07] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.99, 2.99, 3.23 [12:55:25] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.03, 2.96, 3.17 [12:55:55] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.56, 1.77, 1.57 [12:57:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.79, 9.90, 9.06 [12:57:54] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.46, 1.63, 1.54 [12:58:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.95, 9.26, 8.45 [13:00:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.68, 8.62, 8.45 [13:00:14] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.79, 2.97, 3.38 [13:00:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 5.95, 8.42, 8.27 [13:00:47] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 5.52, 8.07, 8.42 [13:01:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 5.87, 8.61, 8.82 [13:02:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.65, 8.00, 8.25 [13:03:00] PROBLEM - cp30 Stunnel HTTP for mw102 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:03:31] PROBLEM - mw102 MediaWiki Rendering on mw102 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:03:53] PROBLEM - cp20 Stunnel HTTP for mw102 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:04:01] PROBLEM - cp31 Stunnel HTTP for mw102 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:04:57] RECOVERY - cp30 Stunnel HTTP for mw102 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 1.247 second response time [13:05:27] RECOVERY - mw102 MediaWiki Rendering on mw102 is OK: HTTP OK: HTTP/1.1 200 OK - 22336 bytes in 0.313 second response time [13:05:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.85, 8.84, 8.81 [13:05:51] RECOVERY - cp20 Stunnel HTTP for mw102 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 0.115 second response time [13:06:00] RECOVERY - cp31 Stunnel HTTP for mw102 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 0.312 second response time [13:08:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.75, 9.23, 8.74 [13:09:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.51, 9.44, 9.10 [13:10:08] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.75, 1.22, 1.01 [13:10:18] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.45, 9.46, 8.74 [13:11:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.07, 10.00, 9.37 [13:12:07] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.80, 1.05, 0.98 [13:12:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.48, 9.69, 8.95 [13:12:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.79, 9.55, 8.62 [13:13:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.48, 9.37, 9.21 [13:14:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.00, 9.37, 8.93 [13:14:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.10, 8.49, 8.34 [13:16:01] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.35, 9.91, 9.25 [13:17:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.67, 9.66, 9.30 [13:18:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.61, 9.49, 9.05 [13:18:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.26, 8.80, 8.49 [13:20:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.84, 8.42, 8.39 [13:22:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.27, 9.44, 9.15 [13:25:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.13, 9.42, 9.06 [13:26:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.28, 9.47, 9.20 [13:28:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.48, 9.19, 9.11 [13:28:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.94, 9.27, 8.79 [13:29:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.03, 9.42, 9.16 [13:32:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.95, 9.79, 9.10 [13:33:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.76, 9.91, 9.40 [13:34:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.41, 10.41, 9.66 [13:35:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.95, 9.71, 9.84 [13:37:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.92, 8.56, 7.77 [13:37:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.40, 9.55, 9.45 [13:38:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.10, 9.65, 9.56 [13:38:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.63, 9.84, 9.44 [13:39:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.10, 8.15, 7.71 [13:39:58] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.12, 3.52, 2.99 [13:40:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.20, 9.68, 9.57 [13:40:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.32, 10.24, 9.65 [13:41:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.16, 9.87, 9.59 [13:42:14] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.60, 8.20, 7.60 [13:43:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.28, 8.66, 8.06 [13:43:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.31, 9.29, 9.41 [13:45:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.74, 8.39, 8.04 [13:46:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.32, 9.73, 9.67 [13:46:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.93, 9.97, 9.76 [13:47:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.66, 9.69, 9.71 [13:47:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.90, 9.42, 9.41 [13:47:55] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.55, 3.66, 3.37 [13:48:05] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 11.99, 9.53, 8.28 [13:48:52] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.02, 4.11, 3.40 [13:49:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.70, 9.79, 9.75 [13:49:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.68, 9.15, 9.32 [13:49:54] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.24, 4.33, 3.68 [13:50:02] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.67, 8.93, 8.19 [13:50:48] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.84, 3.75, 3.34 [13:51:02] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.66, 3.27, 2.82 [13:51:49] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.24, 1.84, 1.22 [13:51:53] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.57, 3.99, 3.63 [13:52:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.61, 9.29, 9.39 [13:52:45] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.39, 3.40, 3.27 [13:53:02] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.28, 3.05, 2.81 [13:53:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.46, 9.84, 9.53 [13:53:49] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.62, 1.43, 1.14 [13:53:56] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.03, 8.16, 8.06 [13:54:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 6.10, 8.30, 9.03 [13:54:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.35, 10.00, 9.76 [13:55:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.34, 9.56, 9.60 [13:57:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.43, 8.75, 9.30 [13:57:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.12, 9.64, 9.53 [13:58:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.47, 8.90, 9.04 [13:59:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.18, 9.54, 9.49 [13:59:50] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.60, 4.07, 3.71 [14:00:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.53, 8.47, 8.88 [14:02:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.97, 9.65, 9.83 [14:03:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.99, 9.73, 9.60 [14:04:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.04, 9.20, 9.09 [14:04:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.32, 9.88, 9.89 [14:06:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.71, 9.28, 9.15 [14:06:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.00, 9.85, 9.87 [14:07:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.23, 9.04, 9.01 [14:07:47] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.37, 3.75, 3.82 [14:09:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.94, 8.76, 8.92 [14:11:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.92, 9.32, 9.09 [14:11:35] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.87, 8.38, 7.87 [14:12:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.17, 9.75, 9.79 [14:13:31] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.43, 8.13, 7.86 [14:13:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.48, 9.35, 9.13 [14:13:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.18, 9.79, 9.51 [14:14:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.54, 9.69, 9.76 [14:15:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.95, 9.20, 9.31 [14:16:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.50, 9.29, 9.04 [14:16:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.13, 9.85, 9.81 [14:17:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.08, 9.55, 9.26 [14:17:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.19, 3.86, 3.81 [14:18:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.03, 8.78, 8.88 [14:18:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.96, 9.78, 9.78 [14:19:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.44, 9.77, 9.39 [14:19:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.63, 3.70, 3.75 [14:22:51] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.77, 3.76, 3.44 [14:23:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.71, 10.49, 9.77 [14:24:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.81, 9.97, 9.34 [14:26:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.51, 9.62, 9.28 [14:26:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 9.74, 10.12, 9.89 [14:26:43] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.02, 3.76, 3.51 [14:28:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.04, 9.89, 9.84 [14:28:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.80, 3.44, 3.42 [14:29:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.93, 9.48, 9.59 [14:30:36] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.07, 3.55, 3.45 [14:32:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.47, 9.89, 9.84 [14:32:32] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.88, 3.23, 3.34 [14:34:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.77, 9.61, 9.75 [14:35:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.26, 3.03, 3.39 [14:35:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.96, 9.31, 9.11 [14:39:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.90, 7.60, 8.50 [14:39:34] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 7.28, 4.37, 3.79 [14:39:43] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.10, 4.05, 3.38 [14:41:33] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.94, 4.00, 3.72 [14:41:42] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.44, 3.88, 3.41 [14:43:18] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.59, 3.96, 3.62 [14:43:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.37, 8.61, 8.70 [14:43:33] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.01, 4.06, 3.76 [14:43:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.59, 9.77, 9.50 [14:44:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.21, 8.73, 8.75 [14:45:14] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.59, 3.79, 3.60 [14:45:32] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.23, 3.58, 3.61 [14:45:40] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.11, 3.05, 3.18 [14:46:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.31, 8.88, 8.80 [14:46:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.27, 10.15, 9.72 [14:47:10] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.16, 3.14, 3.38 [14:47:31] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.08, 3.03, 3.40 [14:47:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.31, 10.15, 9.68 [14:48:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.35, 9.70, 9.11 [14:49:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.30, 9.69, 9.05 [14:50:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.56, 9.81, 9.23 [14:51:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:51:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.32, 9.37, 9.49 [14:54:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 9.93, 10.06, 9.48 [14:55:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:55:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.40, 9.48, 9.17 [14:56:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.96, 9.17, 9.21 [14:58:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.25, 10.11, 9.56 [15:00:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.68, 9.64, 9.46 [15:01:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.20, 9.63, 9.52 [15:02:22] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [15:02:22] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [15:02:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.12, 9.91, 9.95 [15:03:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.57, 9.59, 9.26 [15:03:50] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.92, 3.27, 3.12 [15:04:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.91, 10.40, 10.13 [15:05:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.39, 4.11, 3.54 [15:05:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.95, 9.54, 9.28 [15:05:32] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.74, 3.62, 3.12 [15:05:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.63, 9.65, 9.56 [15:05:46] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.06, 3.21, 3.12 [15:06:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.14, 9.27, 9.22 [15:06:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.17, 9.27, 9.76 [15:07:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.86, 3.93, 3.54 [15:07:31] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.70, 3.29, 3.06 [15:08:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.27, 9.03, 9.12 [15:09:30] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.02, 3.90, 3.57 [15:09:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.86, 9.23, 9.34 [15:11:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.64, 3.46, 3.45 [15:13:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.97, 9.04, 8.19 [15:14:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.22, 9.25, 9.44 [15:15:29] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.37, 3.06, 3.30 [15:15:46] PROBLEM - housing.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'housing.wiki' expires in 15 day(s) (Fri 08 Apr 2022 14:59:32 GMT +0000). [15:16:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.64, 9.08, 9.36 [15:17:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.40, 9.15, 9.01 [15:18:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.29, 9.20, 9.04 [15:19:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.86, 8.46, 8.20 [15:19:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.98, 9.15, 9.04 [15:21:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.51, 9.96, 9.96 [15:22:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.39, 8.81, 8.95 [15:22:17] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/732ced16ed8c...1ae1d417c0b5 [15:22:18] [02miraheze/ssl] 07MirahezeSSLBot 031ae1d41 - Bot: Update SSL cert for housing.wiki [15:22:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.15, 9.49, 9.45 [15:23:43] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.66, 10.72, 10.25 [15:24:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.12, 9.27, 9.36 [15:28:11] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.47, 8.71, 8.15 [15:28:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.62, 9.69, 9.48 [15:30:08] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 10.28, 9.19, 8.40 [15:30:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.00, 9.54, 9.44 [15:32:05] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.39, 9.17, 8.50 [15:32:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.80, 10.13, 9.65 [15:35:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.49, 9.46, 9.89 [15:35:59] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.04, 7.82, 8.14 [15:36:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.15, 9.46, 9.57 [15:37:01] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.51, 8.50, 8.27 [15:37:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.03, 7.82, 8.42 [15:37:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.60, 10.66, 10.28 [15:38:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.97, 10.12, 9.79 [15:38:58] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.03, 7.95, 8.09 [15:39:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.50, 9.65, 9.95 [15:41:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.26, 8.75, 8.69 [15:41:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.34, 10.51, 10.21 [15:43:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.60, 9.67, 9.94 [15:44:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.63, 9.19, 8.92 [15:44:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.66, 9.25, 9.58 [15:46:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.25, 8.65, 8.75 [15:48:35] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.45, 8.83, 8.48 [15:52:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.11, 9.33, 9.43 [15:52:28] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.73, 8.29, 8.40 [15:53:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.43, 9.15, 9.36 [15:54:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 6.95, 8.42, 9.08 [15:55:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [15:55:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.18, 9.65, 9.18 [15:57:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.24, 9.04, 9.02 [15:59:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.73, 9.85, 9.67 [16:00:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.25, 9.13, 9.07 [16:00:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:01:28] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [16:01:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.41, 9.43, 9.13 [16:01:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.82, 10.22, 9.83 [16:02:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.80, 9.46, 9.20 [16:03:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.72, 9.02, 9.03 [16:04:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.91, 10.13, 9.18 [16:05:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.57, 9.37, 9.59 [16:07:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.16, 9.53, 9.18 [16:08:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.90, 9.36, 9.13 [16:09:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.11, 9.91, 9.73 [16:11:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.50, 9.27, 9.22 [16:11:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.79, 9.96, 9.79 [16:13:46] RECOVERY - housing.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'housing.wiki' will expire on Tue 21 Jun 2022 14:22:10 GMT +0000. [16:14:55] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 14.33, 9.51, 6.48 [16:15:08] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.79, 3.04, 2.70 [16:16:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.37, 9.24, 9.08 [16:17:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.78, 9.16, 9.12 [16:18:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.86, 9.21, 9.10 [16:18:55] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 5.47, 7.64, 6.45 [16:19:03] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.51, 3.10, 2.83 [16:19:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.88, 9.58, 9.54 [16:21:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.49, 9.68, 9.43 [16:22:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.35, 9.50, 9.20 [16:22:55] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.13, 6.48, 6.26 [16:23:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.91, 10.38, 9.73 [16:23:32] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.39, 8.63, 8.04 [16:23:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.56, 9.52, 9.57 [16:24:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.73, 9.17, 9.11 [16:25:29] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.31, 8.02, 7.88 [16:25:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.66, 9.78, 9.60 [16:25:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.82, 10.15, 9.79 [16:27:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.23, 9.30, 9.51 [16:28:05] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.87, 3.90, 3.37 [16:29:50] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.94, 3.42, 3.02 [16:30:04] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.59, 3.38, 3.24 [16:30:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.60, 9.25, 9.09 [16:31:46] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.19, 3.16, 2.96 [16:33:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 5.74, 6.87, 8.23 [16:34:02] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.57, 3.98, 3.55 [16:34:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.27, 7.75, 8.39 [16:34:37] PROBLEM - cp21 Stunnel HTTP for mw122 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:34:43] PROBLEM - cp30 Stunnel HTTP for mw122 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:36:02] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.68, 3.91, 3.56 [16:36:36] RECOVERY - cp21 Stunnel HTTP for mw122 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 0.020 second response time [16:36:44] RECOVERY - cp30 Stunnel HTTP for mw122 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 1.790 second response time [16:38:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.36, 9.19, 8.88 [16:38:55] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.41, 3.45, 3.09 [16:39:34] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 9.59, 5.35, 3.86 [16:40:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.39, 9.95, 9.70 [16:40:55] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.19, 4.13, 3.38 [16:42:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.60, 7.80, 8.38 [16:42:16] PROBLEM - cp21 Stunnel HTTP for mw112 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:43:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.13, 9.69, 9.26 [16:44:13] RECOVERY - cp21 Stunnel HTTP for mw112 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 0.015 second response time [16:44:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.16, 9.22, 9.39 [16:44:53] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.48, 3.54, 3.34 [16:45:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.63, 9.07, 8.69 [16:45:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.23, 9.78, 9.37 [16:46:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.97, 9.00, 9.29 [16:46:52] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.97, 3.22, 3.26 [16:47:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.00, 9.75, 8.99 [16:47:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.22, 10.07, 9.53 [16:50:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.48, 8.92, 8.57 [16:51:13] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.47, 3.69, 3.96 [16:51:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.28, 9.48, 9.43 [16:52:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.56, 9.64, 9.45 [16:53:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.41, 9.89, 9.57 [16:56:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.15, 9.59, 9.55 [16:57:54] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.24, 3.27, 3.97 [16:58:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.96, 9.92, 9.66 [16:59:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.33, 2.59, 3.30 [16:59:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.34, 9.89, 9.82 [17:00:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.71, 9.89, 9.53 [17:00:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.85, 9.91, 9.70 [17:02:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.18, 10.05, 9.63 [17:02:35] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [17:02:35] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [17:03:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.38, 9.53, 9.83 [17:03:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.26, 9.83, 9.79 [17:04:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.07, 9.75, 9.58 [17:04:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.31, 10.66, 9.99 [17:06:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.52, 9.91, 9.65 [17:06:21] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.03, 8.61, 8.20 [17:07:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.77, 8.81, 8.08 [17:08:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.20, 9.30, 9.43 [17:09:50] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.21, 3.87, 3.75 [17:10:15] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.50, 7.91, 8.04 [17:11:24] PROBLEM - cp21 Stunnel HTTP for mw121 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:11:49] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.40, 3.47, 3.61 [17:12:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.08, 9.99, 9.97 [17:13:18] RECOVERY - cp21 Stunnel HTTP for mw121 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 0.021 second response time [17:13:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.30, 7.84, 7.93 [17:15:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.24, 4.05, 3.59 [17:16:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.02, 8.44, 8.79 [17:16:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.82, 10.20, 10.04 [17:17:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.48, 3.92, 3.61 [17:18:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.87, 8.48, 8.76 [17:19:46] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.53, 2.88, 3.32 [17:20:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.57, 9.18, 8.98 [17:21:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.17, 8.68, 8.21 [17:22:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.00, 8.51, 8.75 [17:23:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.22, 8.39, 8.15 [17:26:06] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.83, 1.48, 1.23 [17:27:42] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.81, 3.50, 3.41 [17:28:06] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.18, 1.37, 1.22 [17:29:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.14, 4.04, 3.68 [17:29:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.56, 3.40, 3.09 [17:30:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.99, 9.08, 8.84 [17:31:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.73, 3.66, 3.59 [17:31:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.69, 8.28, 8.50 [17:31:34] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.68, 2.84, 2.93 [17:31:41] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.91, 2.97, 3.24 [17:32:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.53, 8.66, 8.73 [17:32:08] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [17:36:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.51, 9.75, 9.14 [17:38:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.76, 9.28, 9.04 [17:39:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.70, 3.12, 3.38 [17:39:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.66, 8.39, 8.23 [17:40:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.46, 9.48, 9.13 [17:40:55] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.81, 6.46, 5.77 [17:42:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.38, 9.08, 9.03 [17:42:55] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.01, 6.30, 5.80 [17:44:18] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.75, 8.84, 8.66 [17:45:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.80, 8.36, 8.35 [17:46:14] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.02, 8.17, 8.44 [17:47:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.77, 3.72, 3.46 [17:48:56] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/645ff2ec3d8d [17:48:57] [02miraheze/mediawiki] 07Universal-Omega 03645ff2e - Fix path for foreground [17:48:59] [02mediawiki] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/mediawiki [17:49:00] [02mediawiki] 07Universal-Omega opened pull request 03#5947: Fix path for foreground - 13https://github.com/miraheze/mediawiki/pull/5947 [17:49:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.42, 3.41, 3.37 [17:49:32] [02mediawiki] 07RhinosF1 closed pull request 03#5947: Fix path for foreground - 13https://github.com/miraheze/mediawiki/pull/5947 [17:49:34] [02mediawiki] 07RhinosF1 deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/mediawiki [17:49:35] [02miraheze/mediawiki] 07RhinosF1 deleted branch 03Universal-Omega-patch-1 [17:49:37] [02miraheze/mediawiki] 07RhinosF1 pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/12e3e716bdb2...ce5900f4fc34 [17:49:38] [02miraheze/mediawiki] 07Universal-Omega 03ce5900f - Fix path for foreground (#5947) [17:50:07] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.71, 9.46, 8.89 [17:51:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.56, 3.14, 3.28 [17:51:03] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_38 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/79adf3eebdd0...02d5e17d3b36 [17:51:04] [02miraheze/mediawiki] 07Universal-Omega 0302d5e17 - Fix path for foreground [17:52:03] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.13, 8.81, 8.71 [17:54:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.67, 7.91, 8.48 [17:55:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [17:55:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.78, 8.46, 9.74 [17:55:55] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.38, 8.07, 8.43 [17:58:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.63, 9.37, 10.00 [17:59:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.52, 9.98, 10.04 [18:00:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.58, 8.32, 8.48 [18:01:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.52, 9.44, 9.83 [18:01:57] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.10, 4.05, 3.54 [18:02:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.02, 8.06, 8.36 [18:02:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.37, 3.70, 3.29 [18:02:35] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [18:03:20] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.61, 3.65, 3.19 [18:03:40] PROBLEM - hi.famepedia.org - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/lib/ [18:03:40] dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query famepedia.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [18:04:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.53, 9.55, 9.70 [18:04:26] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.61, 3.34, 3.21 [18:04:57] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/db9722a5d7e8 [18:04:59] [02miraheze/mediawiki] 07Universal-Omega 03db9722a - Fix more paths in submodules [18:05:00] [02mediawiki] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/mediawiki [18:05:02] [02mediawiki] 07Universal-Omega opened pull request 03#5948: Fix more paths in submodules - 13https://github.com/miraheze/mediawiki/pull/5948 [18:05:20] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.63, 2.89, 2.97 [18:05:35] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.63, 8.35, 8.31 [18:05:50] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.00, 3.67, 3.54 [18:07:00] [02mediawiki] 07RhinosF1 closed pull request 03#5948: Fix more paths in submodules - 13https://github.com/miraheze/mediawiki/pull/5948 [18:07:01] [02miraheze/mediawiki] 07RhinosF1 deleted branch 03Universal-Omega-patch-1 [18:07:03] [02mediawiki] 07RhinosF1 deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/mediawiki [18:07:04] [02miraheze/mediawiki] 07RhinosF1 pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/ce5900f4fc34...ea4bff04fc9c [18:07:06] [02miraheze/mediawiki] 07Universal-Omega 03ea4bff0 - Fix more paths in submodules (#5948) [18:08:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.55, 9.59, 9.70 [18:08:57] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/ea4bff04fc9c...8cea63da7f68 [18:08:58] [02miraheze/mediawiki] 07Universal-Omega 038cea63d - Fix SecureLinkFixer path [18:10:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.41, 10.05, 9.85 [18:11:30] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_38 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/02d5e17d3b36...081b53133836 [18:11:32] [02miraheze/mediawiki] 07Universal-Omega 03081b531 - Fix more paths in submodules [18:11:56] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.41, 1.99, 1.37 [18:12:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.09, 9.67, 9.74 [18:13:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.94, 8.17, 8.30 [18:13:36] !log rhinos@test101:/srv/mediawiki-staging/w$ sudo -u www-data git submodule sync --recursive [18:13:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:13:56] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.53, 1.48, 1.26 [18:14:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.64, 8.86, 8.59 [18:15:13] !log [rhinos@test101] starting deploy of {'world': True} to all [18:15:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:16:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.08, 8.34, 8.44 [18:19:17] !log same on deployment.mh.o [18:19:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:19:33] !log [rhinos@mwtask111] starting deploy of {'world': True} to all [18:19:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:21:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.28, 3.22, 3.39 [18:21:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.21, 9.47, 9.37 [18:21:55] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.73, 1.93, 1.31 [18:22:21] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.37, 9.04, 8.56 [18:23:43] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.13, 9.07, 9.24 [18:24:16] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.79, 8.81, 8.54 [18:24:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.32, 8.78, 8.50 [18:26:12] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.18, 8.47, 8.44 [18:26:18] !log [rhinos@test101] finished deploy of {'world': True} to all - SUCCESS in 665s [18:26:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:26:30] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.80, 7.28, 6.24 [18:27:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.48, 3.66, 3.30 [18:28:04] !log [rhinos@mwtask111] finished deploy of {'world': True} to all - SUCCESS in 510s [18:28:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:28:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.41, 9.21, 9.23 [18:28:48] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.07, 9.39, 8.80 [18:29:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.51, 3.75, 3.38 [18:29:55] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.16, 1.98, 1.71 [18:30:05] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.52, 8.67, 8.54 [18:30:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.18, 3.56, 3.39 [18:30:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.38, 8.54, 8.99 [18:30:43] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.43, 8.98, 8.73 [18:31:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.35, 9.04, 9.07 [18:32:01] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.24, 9.29, 8.79 [18:32:04] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.73, 3.21, 3.28 [18:32:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.64, 9.47, 9.28 [18:33:11] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.94, 2.92, 3.14 [18:33:55] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.03, 1.54, 1.59 [18:33:57] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.57, 8.98, 8.73 [18:34:33] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.91, 9.51, 8.94 [18:35:53] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.37, 9.54, 8.97 [18:37:48] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.00, 9.39, 8.99 [18:38:23] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.78, 9.26, 9.02 [18:38:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.40, 9.86, 9.64 [18:39:51] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.77, 3.41, 3.26 [18:39:55] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 9.40, 7.80, 6.95 [18:40:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.17, 3.72, 3.30 [18:40:42] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.84, 1.64, 1.34 [18:41:47] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.30, 3.62, 3.35 [18:41:50] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.41, 7.28, 6.86 [18:42:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.55, 3.46, 3.25 [18:42:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.19, 9.93, 9.72 [18:42:42] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.70, 1.56, 1.34 [18:43:45] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.32, 6.43, 6.59 [18:44:11] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.01, 3.35, 3.24 [18:45:55] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.11, 1.75, 1.60 [18:47:36] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.11, 3.50, 3.42 [18:47:55] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.58, 1.70, 1.60 [18:48:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.10, 9.05, 8.95 [18:48:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.27, 9.52, 9.71 [18:49:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.12, 9.71, 9.13 [18:49:33] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.38, 3.26, 3.35 [18:49:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.83, 9.23, 9.77 [18:51:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.39, 9.29, 9.06 [18:52:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.03, 9.20, 9.06 [18:53:28] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.46, 3.71, 3.52 [18:53:43] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.17, 9.45, 9.70 [18:55:24] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.75, 3.23, 3.36 [18:55:43] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.81, 9.65, 9.77 [18:58:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.22, 9.44, 9.14 [18:58:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.98, 10.41, 9.92 [18:59:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.32, 10.27, 10.02 [19:00:05] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.99, 3.54, 3.18 [19:00:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.65, 9.52, 9.21 [19:00:57] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 10.74, 8.63, 7.75 [19:01:02] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.69, 3.34, 3.26 [19:01:39] RECOVERY - hi.famepedia.org - reverse DNS on sslhost is OK: SSL OK - hi.famepedia.org reverse DNS resolves to cp20.miraheze.org - NS RECORDS OK [19:01:51] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.74, 8.02, 7.78 [19:02:04] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.66, 3.21, 3.11 [19:02:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.80, 9.94, 9.41 [19:02:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.50, 9.81, 9.81 [19:02:57] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.77, 8.76, 7.91 [19:03:01] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.39, 3.55, 3.33 [19:03:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.72, 9.52, 9.82 [19:03:48] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.05, 7.96, 7.79 [19:04:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.97, 8.94, 9.09 [19:04:48] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [19:04:57] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.95, 8.13, 7.78 [19:05:01] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.98, 3.51, 3.33 [19:07:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.26, 3.30, 3.27 [19:09:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.79, 9.56, 9.02 [19:11:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.64, 9.12, 8.93 [19:11:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.09, 9.81, 9.67 [19:13:57] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.43, 3.56, 3.36 [19:15:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.49, 9.67, 9.67 [19:15:54] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.52, 3.20, 3.26 [19:16:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.21, 9.08, 9.20 [19:17:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.84, 10.11, 9.83 [19:20:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.71, 9.44, 9.37 [19:21:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.26, 9.63, 9.73 [19:25:38] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 4.00, 3.33, 3.22 [19:25:48] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 5.19, 2.36, 1.44 [19:27:34] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.51, 2.99, 3.11 [19:27:35] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.53, 1.75, 1.38 [19:27:43] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.02, 9.07, 9.39 [19:27:48] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.59, 1.92, 1.39 [19:28:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.33, 9.14, 8.88 [19:28:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.62, 9.72, 9.51 [19:29:34] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.18, 1.53, 1.35 [19:29:48] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.98, 1.53, 1.31 [19:30:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.91, 9.00, 8.83 [19:32:47] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.26, 3.64, 3.40 [19:32:57] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.54, 8.05, 7.65 [19:34:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.05, 9.34, 8.99 [19:34:21] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [19:34:21] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [19:34:46] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.48, 4.06, 3.59 [19:34:57] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.40, 8.13, 7.73 [19:37:18] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.69, 3.56, 3.24 [19:37:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.73, 8.12, 8.48 [19:39:14] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 8.36, 4.82, 3.69 [19:39:32] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.36, 1.84, 1.49 [19:40:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 6.58, 8.84, 9.01 [19:40:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.86, 3.47, 3.03 [19:41:08] PROBLEM - cp20 Stunnel HTTP for mw121 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:41:11] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:41:24] PROBLEM - cp21 Stunnel HTTP for mw121 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:41:32] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.74, 1.78, 1.51 [19:41:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.44, 9.70, 9.93 [19:42:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.60, 9.60, 9.86 [19:42:46] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.60, 3.16, 2.97 [19:43:09] RECOVERY - cp20 Stunnel HTTP for mw121 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 7.147 second response time [19:45:09] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 22336 bytes in 0.461 second response time [19:45:22] RECOVERY - cp21 Stunnel HTTP for mw121 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 0.274 second response time [19:45:31] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.18, 1.64, 1.53 [19:45:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.19, 9.64, 9.80 [19:47:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 6.93, 8.69, 9.44 [19:48:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.22, 7.59, 8.30 [19:48:57] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.69, 7.76, 7.39 [19:50:57] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.91, 7.51, 7.35 [19:51:30] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.71, 1.94, 1.71 [19:53:30] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.78, 1.49, 1.57 [19:56:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.19, 7.55, 8.44 [19:57:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.44, 8.95, 9.07 [19:59:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.52, 3.64, 3.84 [19:59:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.88, 9.00, 9.09 [20:03:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.42, 9.25, 9.14 [20:04:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.50, 8.32, 8.36 [20:05:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.92, 9.05, 9.11 [20:06:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.94, 8.32, 8.37 [20:06:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.67, 9.67, 8.74 [20:07:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.07, 9.76, 9.35 [20:08:55] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.45, 9.53, 8.80 [20:09:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.31, 8.06, 7.42 [20:10:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.94, 9.59, 8.87 [20:11:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.99, 3.49, 3.25 [20:12:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.33, 9.30, 8.85 [20:13:03] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.86, 3.60, 3.65 [20:13:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.04, 8.06, 7.58 [20:14:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.33, 9.50, 8.96 [20:14:40] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.14, 9.91, 9.08 [20:15:32] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.25, 3.77, 3.41 [20:16:01] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.50, 9.11, 8.50 [20:16:36] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.00, 9.15, 8.90 [20:17:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.81, 3.04, 3.45 [20:17:31] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.97, 3.07, 3.20 [20:17:57] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.17, 9.61, 8.74 [20:18:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 1.44, 2.87, 3.93 [20:18:31] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.19, 9.62, 9.11 [20:19:03] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.79, 2.77, 3.31 [20:20:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.21, 9.60, 9.17 [20:20:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 6.89, 9.05, 9.04 [20:21:48] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.12, 9.78, 9.05 [20:22:19] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.52, 7.21, 5.93 [20:23:28] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.66, 3.67, 3.35 [20:23:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.70, 9.60, 9.94 [20:24:14] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.38, 6.38, 5.77 [20:24:16] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.11, 9.07, 8.98 [20:24:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.97, 3.46, 3.79 [20:25:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.03, 3.76, 3.53 [20:25:28] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.05, 3.47, 3.32 [20:25:40] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.61, 9.87, 9.23 [20:26:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.41, 10.20, 9.50 [20:27:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.87, 3.83, 3.59 [20:27:27] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.06, 3.99, 3.52 [20:27:35] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.31, 9.43, 9.14 [20:28:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.98, 3.33, 3.66 [20:29:26] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.62, 3.30, 3.32 [20:29:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.09, 9.96, 9.84 [20:30:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.58, 9.77, 9.39 [20:31:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.14, 3.67, 3.56 [20:31:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.42, 9.88, 9.84 [20:32:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.96, 10.08, 9.53 [20:33:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.02, 3.35, 3.45 [20:33:27] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.94, 8.89, 8.19 [20:33:28] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [20:33:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.29, 10.22, 9.98 [20:33:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.72, 1.68, 1.29 [20:34:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.97, 9.56, 9.41 [20:35:03] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.68, 3.21, 3.39 [20:35:49] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.75, 1.41, 1.24 [20:36:23] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.27, 2.94, 3.36 [20:39:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.59, 3.34, 3.43 [20:39:17] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.94, 8.12, 8.09 [20:40:20] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.09, 3.30, 3.42 [20:41:03] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.48, 3.85, 3.61 [20:42:20] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.03, 3.98, 3.68 [20:43:01] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.46, 8.83, 8.31 [20:43:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.45, 3.84, 3.64 [20:44:19] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.86, 3.70, 3.62 [20:45:21] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.37, 6.53, 6.02 [20:46:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.05, 9.33, 9.18 [20:47:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.95, 4.35, 3.89 [20:47:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.01, 8.96, 8.99 [20:48:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.46, 3.55, 3.35 [20:48:17] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.61, 3.97, 3.73 [20:49:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.53, 3.73, 3.72 [20:49:11] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 3.92, 5.55, 5.78 [20:49:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.91, 9.45, 9.17 [20:50:16] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.50, 3.13, 3.22 [20:50:16] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 1.76, 3.20, 3.48 [20:50:48] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.95, 8.14, 8.23 [20:51:54] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.28, 1.52, 0.94 [20:52:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.62, 9.85, 9.53 [20:52:16] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.61, 2.70, 3.26 [20:53:03] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.98, 2.82, 3.36 [20:53:54] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.93, 1.69, 1.08 [20:54:38] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.14, 8.56, 8.42 [20:54:58] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.91, 7.64, 6.58 [20:55:54] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.58, 1.65, 1.14 [20:56:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 6.96, 9.11, 9.90 [20:56:35] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.50, 8.36, 8.38 [20:56:55] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.19, 7.36, 6.60 [20:57:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.80, 9.83, 10.00 [20:58:55] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.24, 6.39, 6.35 [20:59:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.20, 10.47, 10.19 [21:00:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.08, 9.45, 9.27 [21:01:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.15, 9.81, 10.00 [21:02:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.20, 8.90, 9.09 [21:04:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.78, 9.40, 9.23 [21:04:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.50, 9.90, 9.86 [21:05:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.74, 9.74, 9.36 [21:06:26] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.36, 8.75, 8.32 [21:07:05] PROBLEM - cp20 HTTP 4xx/5xx ERROR Rate on cp20 is CRITICAL: CRITICAL - NGINX Error Rate is 61% [21:07:39] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 5.33, 3.14, 1.77 [21:07:54] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.31, 1.82, 1.34 [21:07:57] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.56, 1.85, 1.15 [21:08:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.37, 9.52, 9.33 [21:08:23] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.64, 8.49, 8.29 [21:08:57] PROBLEM - cp21 HTTP 4xx/5xx ERROR Rate on cp21 is WARNING: WARNING - NGINX Error Rate is 55% [21:09:03] PROBLEM - cp20 HTTP 4xx/5xx ERROR Rate on cp20 is WARNING: WARNING - NGINX Error Rate is 55% [21:09:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.33, 9.42, 9.36 [21:10:09] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.46, 4.02, 3.36 [21:10:54] PROBLEM - cp21 HTTP 4xx/5xx ERROR Rate on cp21 is CRITICAL: CRITICAL - NGINX Error Rate is 72% [21:11:28] Why high error rate [21:11:29] :( [21:11:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.66, 9.90, 9.54 [21:11:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.95, 9.78, 9.80 [21:12:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.89, 9.70, 9.87 [21:13:00] PROBLEM - cp20 HTTP 4xx/5xx ERROR Rate on cp20 is CRITICAL: CRITICAL - NGINX Error Rate is 68% [21:13:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.17, 4.25, 3.49 [21:13:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.79, 9.01, 9.25 [21:13:39] [02puppet] 07revi commented on pull request 03#2423: Allow static access from same domain - 13https://github.com/miraheze/puppet/pull/2423#issuecomment-1076826360 [21:13:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.91, 9.35, 9.64 [21:13:43] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.87, 4.17, 3.57 [21:13:51] PROBLEM - cp21 Stunnel HTTP for mw102 on cp21 is CRITICAL: HTTP CRITICAL - No data received from host [21:14:48] PROBLEM - cp21 Stunnel HTTP for mail121 on cp21 is CRITICAL: HTTP CRITICAL - No data received from host [21:15:30] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 328 bytes in 0.020 second response time [21:15:52] RECOVERY - cp21 Stunnel HTTP for mw102 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 1.884 second response time [21:15:53] PROBLEM - cp21 Varnish Backends on cp21 is CRITICAL: 7 backends are down. mw101 mw102 mw111 mw112 mw121 mw122 mediawiki [21:16:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 3.59, 6.56, 8.14 [21:16:18] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.87, 1.89, 1.53 [21:16:43] RECOVERY - cp21 Stunnel HTTP for mail121 on cp21 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 427 bytes in 0.014 second response time [21:17:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.74, 3.86, 3.55 [21:17:35] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.85, 3.81, 3.60 [21:17:55] PROBLEM - cp30 Varnish Backends on cp30 is CRITICAL: 1 backends are down. mw111 [21:18:01] PROBLEM - cp31 Varnish Backends on cp31 is CRITICAL: 1 backends are down. mw111 [21:18:18] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.89, 1.54, 1.44 [21:18:28] PROBLEM - cp20 Varnish Backends on cp20 is CRITICAL: 1 backends are down. mw122 [21:18:54] PROBLEM - cp20 HTTP 4xx/5xx ERROR Rate on cp20 is WARNING: WARNING - NGINX Error Rate is 45% [21:19:26] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 1.103 second response time [21:19:32] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.01, 3.91, 3.66 [21:19:55] RECOVERY - cp30 Varnish Backends on cp30 is OK: All 12 backends are healthy [21:20:28] RECOVERY - cp20 Varnish Backends on cp20 is OK: All 12 backends are healthy [21:20:52] PROBLEM - cp20 HTTP 4xx/5xx ERROR Rate on cp20 is CRITICAL: CRITICAL - NGINX Error Rate is 61% [21:21:54] PROBLEM - cp31 Stunnel HTTP for mw102 on cp31 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 328 bytes in 0.300 second response time [21:23:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 3.80, 6.14, 8.03 [21:23:53] RECOVERY - cp31 Stunnel HTTP for mw102 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 0.323 second response time [21:23:55] PROBLEM - cp30 Varnish Backends on cp30 is CRITICAL: 1 backends are down. mw102 [21:24:28] PROBLEM - cp20 Varnish Backends on cp20 is CRITICAL: 4 backends are down. mw101 mw102 mw111 mw121 [21:25:27] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 198.244.148.90/cpweb, 2001:41d0:801:2000::1b80/cpweb [21:25:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.49, 7.66, 8.40 [21:26:01] RECOVERY - cp31 Varnish Backends on cp31 is OK: All 12 backends are healthy [21:26:22] PROBLEM - cp21 HTTP 4xx/5xx ERROR Rate on cp21 is WARNING: WARNING - NGINX Error Rate is 47% [21:27:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.33, 3.67, 3.55 [21:27:22] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [21:28:19] PROBLEM - cp21 HTTP 4xx/5xx ERROR Rate on cp21 is CRITICAL: CRITICAL - NGINX Error Rate is 80% [21:28:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.03, 8.85, 8.97 [21:29:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.80, 3.64, 3.55 [21:29:13] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.75, 3.92, 3.88 [21:29:53] RECOVERY - cp21 Varnish Backends on cp21 is OK: All 12 backends are healthy [21:29:55] RECOVERY - cp30 Varnish Backends on cp30 is OK: All 12 backends are healthy [21:30:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.71, 8.63, 8.87 [21:31:11] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.94, 4.55, 4.11 [21:34:07] PROBLEM - cp21 HTTP 4xx/5xx ERROR Rate on cp21 is WARNING: WARNING - NGINX Error Rate is 58% [21:34:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.04, 8.87, 8.87 [21:35:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.75, 4.19, 3.72 [21:36:03] PROBLEM - cp21 HTTP 4xx/5xx ERROR Rate on cp21 is CRITICAL: CRITICAL - NGINX Error Rate is 71% [21:36:22] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.84, 8.65, 8.78 [21:36:32] PROBLEM - mw112 MediaWiki Rendering on mw112 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 1595 bytes in 0.009 second response time [21:36:44] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 328 bytes in 0.587 second response time [21:36:59] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.99, 3.82, 3.90 [21:38:02] PROBLEM - cp21 HTTP 4xx/5xx ERROR Rate on cp21 is WARNING: WARNING - NGINX Error Rate is 58% [21:38:30] RECOVERY - mw112 MediaWiki Rendering on mw112 is OK: HTTP OK: HTTP/1.1 200 OK - 22336 bytes in 0.551 second response time [21:38:36] PROBLEM - cp20 HTTP 4xx/5xx ERROR Rate on cp20 is WARNING: WARNING - NGINX Error Rate is 44% [21:38:47] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 4.345 second response time [21:38:52] PROBLEM - cp31 Varnish Backends on cp31 is CRITICAL: 1 backends are down. mw112 [21:39:12] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.60, 3.91, 3.72 [21:39:55] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.51, 1.79, 2.00 [21:40:28] RECOVERY - cp20 Varnish Backends on cp20 is OK: All 12 backends are healthy [21:42:02] PROBLEM - cp21 HTTP 4xx/5xx ERROR Rate on cp21 is CRITICAL: CRITICAL - NGINX Error Rate is 65% [21:42:46] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.90, 8.42, 7.90 [21:43:52] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.60, 8.90, 8.43 [21:43:54] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 0.31, 1.04, 1.65 [21:44:02] PROBLEM - cp21 HTTP 4xx/5xx ERROR Rate on cp21 is WARNING: WARNING - NGINX Error Rate is 54% [21:44:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.01, 9.01, 8.84 [21:44:28] PROBLEM - cp20 Varnish Backends on cp20 is CRITICAL: 1 backends are down. mw111 [21:44:31] PROBLEM - cp20 HTTP 4xx/5xx ERROR Rate on cp20 is CRITICAL: CRITICAL - NGINX Error Rate is 64% [21:44:37] RECOVERY - cp31 Varnish Backends on cp31 is OK: All 12 backends are healthy [21:44:41] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 5.57, 7.39, 7.58 [21:45:03] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.06, 4.36, 4.01 [21:45:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.66, 3.73, 3.65 [21:46:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.23, 8.83, 8.81 [21:46:28] RECOVERY - cp20 Varnish Backends on cp20 is OK: All 12 backends are healthy [21:47:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.33, 9.48, 8.77 [21:48:02] PROBLEM - cp21 HTTP 4xx/5xx ERROR Rate on cp21 is CRITICAL: CRITICAL - NGINX Error Rate is 71% [21:49:02] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.87, 9.72, 8.46 [21:49:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.50, 3.61, 3.81 [21:49:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.70, 3.69, 3.69 [21:50:02] PROBLEM - cp21 HTTP 4xx/5xx ERROR Rate on cp21 is WARNING: WARNING - NGINX Error Rate is 57% [21:50:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.39, 9.53, 9.09 [21:50:27] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.22, 8.17, 7.78 [21:50:58] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.95, 9.18, 8.43 [21:51:03] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.02, 3.82, 3.86 [21:51:56] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-8 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/82e37efb7f2c [21:51:58] [02miraheze/puppet] 07paladox 0382e37ef - base::firewall add a way to block traffic for abuse [21:51:59] [02puppet] 07paladox created branch 03paladox-patch-8 - 13https://github.com/miraheze/puppet [21:52:01] [02puppet] 07paladox opened pull request 03#2454: base::firewall add a way to block traffic for abuse - 13https://github.com/miraheze/puppet/pull/2454 [21:52:02] PROBLEM - cp21 HTTP 4xx/5xx ERROR Rate on cp21 is CRITICAL: CRITICAL - NGINX Error Rate is 60% [21:52:23] PROBLEM - cp20 HTTP 4xx/5xx ERROR Rate on cp20 is WARNING: WARNING - NGINX Error Rate is 59% [21:52:25] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-8 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/82e37efb7f2c...7e740050cbd8 [21:52:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.86, 9.23, 9.04 [21:52:27] [02miraheze/puppet] 07paladox 037e74005 - Update firewall.pp [21:52:28] [02puppet] 07paladox synchronize pull request 03#2454: base::firewall add a way to block traffic for abuse - 13https://github.com/miraheze/puppet/pull/2454 [21:53:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.87, 3.97, 3.92 [21:53:12] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.46, 3.75, 3.68 [21:53:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 6.75, 9.39, 9.12 [21:53:57] PROBLEM - cp30 Varnish Backends on cp30 is CRITICAL: 1 backends are down. mw121 [21:54:02] PROBLEM - cp21 HTTP 4xx/5xx ERROR Rate on cp21 is WARNING: WARNING - NGINX Error Rate is 40% [21:55:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.77, 3.76, 3.70 [21:55:55] RECOVERY - cp30 Varnish Backends on cp30 is OK: All 12 backends are healthy [21:56:45] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.22, 8.48, 8.36 [21:56:54] PROBLEM - cp30 Stunnel HTTP for mw122 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:57:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.28, 4.06, 3.82 [21:57:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.16, 9.39, 9.13 [21:58:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.76, 8.47, 8.20 [21:58:56] RECOVERY - cp30 Stunnel HTTP for mw122 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 2.960 second response time [21:59:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.31, 3.65, 3.69 [21:59:24] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [21:59:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.65, 9.07, 9.05 [22:00:02] RECOVERY - cp21 HTTP 4xx/5xx ERROR Rate on cp21 is OK: OK - NGINX Error Rate is 29% [22:01:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.28, 3.97, 3.91 [22:01:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.27, 3.91, 3.78 [22:01:23] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [22:02:14] RECOVERY - cp20 HTTP 4xx/5xx ERROR Rate on cp20 is OK: OK - NGINX Error Rate is 25% [22:02:35] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [22:03:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.52, 3.72, 3.73 [22:04:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.07, 9.17, 9.00 [22:05:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.41, 3.97, 3.96 [22:05:22] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::1b80/cpweb [22:06:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.81, 8.94, 8.93 [22:07:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.15, 3.96, 3.95 [22:07:21] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [22:08:03] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 198.244.148.90/cpweb, 2001:41d0:801:2000::1b80/cpweb [22:08:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.80, 8.20, 8.06 [22:09:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.00, 3.69, 3.86 [22:09:08] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.50, 1.75, 1.57 [22:10:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.84, 8.03, 8.01 [22:11:03] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.26, 4.12, 4.01 [22:11:08] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.03, 1.48, 1.49 [22:11:19] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 198.244.148.90/cpweb, 2001:41d0:801:2000::1b80/cpweb [22:15:41] PROBLEM - ldap111 Puppet on ldap111 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 5 minutes ago with 0 failures [22:15:42] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 5.15, 7.21, 8.18 [22:15:47] PROBLEM - mw111 Puppet on mw111 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 10 minutes ago with 0 failures [22:15:48] PROBLEM - mon111 Puppet on mon111 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 17 minutes ago with 0 failures [22:15:50] PROBLEM - ns1 Puppet on ns1 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 21 minutes ago with 0 failures [22:15:51] PROBLEM - mw121 Puppet on mw121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 4 minutes ago with 0 failures [22:15:55] PROBLEM - cp20 Puppet on cp20 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 17 minutes ago with 0 failures [22:15:59] PROBLEM - gluster111 Puppet on gluster111 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 18 minutes ago with 0 failures [22:16:01] PROBLEM - jobchron121 Puppet on jobchron121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 29 minutes ago with 0 failures [22:16:02] PROBLEM - cp30 Puppet on cp30 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 23 minutes ago with 0 failures [22:16:06] PROBLEM - es121 Puppet on es121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 2 minutes ago with 0 failures [22:16:07] PROBLEM - mw112 Puppet on mw112 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 26 minutes ago with 0 failures [22:16:13] PROBLEM - mw102 Puppet on mw102 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 5 minutes ago with 0 failures [22:16:16] PROBLEM - mw101 Puppet on mw101 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 15 minutes ago with 0 failures [22:16:16] PROBLEM - es101 Puppet on es101 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 29 minutes ago with 0 failures [22:16:17] PROBLEM - prometheus101 Puppet on prometheus101 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 26 minutes ago with 0 failures [22:16:22] PROBLEM - cloud12 Puppet on cloud12 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 7 minutes ago with 0 failures [22:16:33] PROBLEM - db112 Puppet on db112 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 15 minutes ago with 0 failures [22:16:37] PROBLEM - cloud11 Puppet on cloud11 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 26 minutes ago with 0 failures [22:16:38] PROBLEM - mw122 Puppet on mw122 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 12 minutes ago with 0 failures [22:16:41] PROBLEM - db111 Puppet on db111 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 8 minutes ago with 0 failures [22:16:42] PROBLEM - graylog121 Puppet on graylog121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 19 minutes ago with 0 failures [22:16:43] PROBLEM - cp31 Puppet on cp31 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 26 minutes ago with 0 failures [22:16:48] PROBLEM - db121 Puppet on db121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 27 minutes ago with 0 failures [22:16:49] PROBLEM - db101 Puppet on db101 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 26 minutes ago with 0 failures [22:16:52] PROBLEM - cp21 Puppet on cp21 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 18 minutes ago with 0 failures [22:16:53] PROBLEM - matomo101 Puppet on matomo101 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 27 minutes ago with 0 failures [22:16:55] PROBLEM - gluster101 Puppet on gluster101 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 2 minutes ago with 0 failures [22:16:58] PROBLEM - mem101 Puppet on mem101 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 27 minutes ago with 0 failures [22:17:01] PROBLEM - cloud10 Puppet on cloud10 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 7 minutes ago with 0 failures [22:17:09] PROBLEM - mwtask111 Puppet on mwtask111 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 24 minutes ago with 0 failures [22:17:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.20, 4.04, 3.79 [22:17:17] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [22:17:24] PROBLEM - mail121 Puppet on mail121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 4 minutes ago with 0 failures [22:17:25] PROBLEM - puppet111 Puppet on puppet111 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 23 minutes ago with 0 failures [22:17:25] PROBLEM - bast121 Puppet on bast121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 7 minutes ago with 0 failures [22:17:28] PROBLEM - phab121 Puppet on phab121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 8 minutes ago with 0 failures [22:17:30] PROBLEM - bast101 Puppet on bast101 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 14 minutes ago with 0 failures [22:17:34] PROBLEM - ns2 Puppet on ns2 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 29 minutes ago with 0 failures [22:17:36] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [22:17:39] PROBLEM - es111 Puppet on es111 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 24 minutes ago with 0 failures [22:17:40] PROBLEM - mem121 Puppet on mem121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 16 minutes ago with 0 failures [22:17:47] PROBLEM - gluster121 Puppet on gluster121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 20 minutes ago with 0 failures [22:18:01] RECOVERY - jobchron121 Puppet on jobchron121 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:18:01] RECOVERY - cp30 Puppet on cp30 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:18:06] RECOVERY - es121 Puppet on es121 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:18:13] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/46df8614d805...795ac20914c7 [22:18:14] [02miraheze/puppet] 07paladox 03795ac20 - base::firewall add a way to block traffic for abuse (#2454) [22:18:16] [02puppet] 07paladox closed pull request 03#2454: base::firewall add a way to block traffic for abuse - 13https://github.com/miraheze/puppet/pull/2454 [22:18:16] RECOVERY - es101 Puppet on es101 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:18:17] [02puppet] 07paladox deleted branch 03paladox-patch-8 - 13https://github.com/miraheze/puppet [22:18:19] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-8 [22:18:22] RECOVERY - cloud12 Puppet on cloud12 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:18:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.27, 8.03, 8.46 [22:18:33] RECOVERY - db112 Puppet on db112 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:18:37] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:18:42] RECOVERY - db111 Puppet on db111 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:18:42] RECOVERY - graylog121 Puppet on graylog121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:18:43] RECOVERY - cp31 Puppet on cp31 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:18:48] RECOVERY - db121 Puppet on db121 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:18:49] RECOVERY - db101 Puppet on db101 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:18:52] RECOVERY - cp21 Puppet on cp21 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:18:53] RECOVERY - matomo101 Puppet on matomo101 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:18:56] RECOVERY - gluster101 Puppet on gluster101 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:18:59] RECOVERY - mem101 Puppet on mem101 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:19:01] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:19:24] RECOVERY - mail121 Puppet on mail121 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:19:25] RECOVERY - bast121 Puppet on bast121 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:19:30] RECOVERY - bast101 Puppet on bast101 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:19:39] RECOVERY - es111 Puppet on es111 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:19:41] RECOVERY - ldap111 Puppet on ldap111 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:19:47] RECOVERY - gluster121 Puppet on gluster121 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:19:56] RECOVERY - cp20 Puppet on cp20 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:19:59] RECOVERY - gluster111 Puppet on gluster111 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:21:01] PROBLEM - puppet111 Current Load on puppet111 is CRITICAL: CRITICAL - load average: 7.76, 4.11, 1.99 [22:21:49] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.62, 2.41, 1.58 [22:22:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.13, 8.74, 8.03 [22:22:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.31, 8.82, 8.70 [22:22:28] PROBLEM - puppet111 Puppet on puppet111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [22:23:49] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.30, 1.99, 1.53 [22:24:20] PROBLEM - puppet111 SSH on puppet111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:24:25] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.89, 9.25, 8.34 [22:24:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.09, 9.52, 8.96 [22:24:32] PROBLEM - puppet111 PowerDNS Recursor on puppet111 is CRITICAL: CRITICAL - Plugin timed out while executing system call [22:25:49] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.06, 1.70, 1.48 [22:25:53] PROBLEM - puppet111 NTP time on puppet111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:26:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.95, 8.95, 8.83 [22:27:52] PROBLEM - puppet111 ferm_active on puppet111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [22:28:16] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.24, 9.62, 8.73 [22:28:25] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.12, 9.04, 8.63 [22:28:50] PROBLEM - puppet111 conntrack_table_size on puppet111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [22:29:12] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.19, 3.78, 3.92 [22:29:12] PROBLEM - puppet111 Puppet on puppet111 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 35 minutes ago with 0 failures [22:29:29] RECOVERY - phab121 Puppet on phab121 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:29:33] PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:29:40] RECOVERY - mem121 Puppet on mem121 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:29:47] RECOVERY - puppet111 ferm_active on puppet111 is OK: OK ferm input default policy is set [22:29:47] PROBLEM - mw111 Puppet on mw111 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:29:48] PROBLEM - mon111 Puppet on mon111 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:29:50] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:29:51] PROBLEM - mw121 Puppet on mw121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:29:57] RECOVERY - puppet111 PowerDNS Recursor on puppet111 is OK: DNS OK: 0.104 seconds response time. miraheze.org returns 198.244.148.90,2001:41d0:801:2000::1b80,2001:41d0:801:2000::4c25,51.195.220.68 [22:30:07] PROBLEM - mw112 Puppet on mw112 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:30:13] PROBLEM - mw102 Puppet on mw102 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:30:16] PROBLEM - mw101 Puppet on mw101 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:30:17] PROBLEM - prometheus101 Puppet on prometheus101 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:30:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.91, 9.89, 9.18 [22:30:35] RECOVERY - puppet111 SSH on puppet111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [22:30:38] PROBLEM - mw122 Puppet on mw122 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:30:45] RECOVERY - puppet111 conntrack_table_size on puppet111 is OK: OK: nf_conntrack is 0 % full [22:30:53] RECOVERY - puppet111 NTP time on puppet111 is OK: NTP OK: Offset -0.0009894967079 secs [22:31:09] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:31:12] PROBLEM - puppet111 Puppet on puppet111 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:31:24] PROBLEM - mail121 Puppet on mail121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:31:25] PROBLEM - bast121 Puppet on bast121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:31:31] PROBLEM - bast101 Puppet on bast101 is CRITICAL: CRITICAL: Puppet has 21 failures. Last run 2 minutes ago with 21 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.gpg],File[/usr/local/bin/puppet-enabled],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [22:31:39] PROBLEM - es111 Puppet on es111 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:31:41] PROBLEM - ldap111 Puppet on ldap111 is CRITICAL: CRITICAL: Puppet has 25 failures. Last run 2 minutes ago with 25 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.gpg],File[/usr/local/bin/puppet-enabled],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [22:31:47] PROBLEM - gluster121 Puppet on gluster121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:31:55] PROBLEM - cp20 Puppet on cp20 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:31:59] PROBLEM - gluster111 Puppet on gluster111 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:32:02] PROBLEM - cp30 Puppet on cp30 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:32:02] PROBLEM - jobchron121 Puppet on jobchron121 is CRITICAL: CRITICAL: Puppet has 24 failures. Last run 2 minutes ago with 24 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.gpg],File[/usr/local/bin/puppet-enabled],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [22:32:06] PROBLEM - es121 Puppet on es121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:32:08] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.33, 10.04, 9.09 [22:32:16] PROBLEM - es101 Puppet on es101 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:32:22] PROBLEM - cloud12 Puppet on cloud12 is CRITICAL: CRITICAL: Puppet has 23 failures. Last run 3 minutes ago with 23 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.gpg],File[/usr/local/bin/puppet-enabled],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [22:32:38] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CRITICAL: Puppet has 23 failures. Last run 3 minutes ago with 23 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.gpg],File[/usr/local/bin/puppet-enabled],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [22:32:42] PROBLEM - graylog121 Puppet on graylog121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:32:42] PROBLEM - db111 Puppet on db111 is CRITICAL: CRITICAL: Puppet has 23 failures. Last run 3 minutes ago with 23 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.gpg],File[/usr/local/bin/puppet-enabled],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [22:32:48] PROBLEM - db121 Puppet on db121 is CRITICAL: CRITICAL: Puppet has 23 failures. Last run 3 minutes ago with 23 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.gpg],File[/usr/local/bin/puppet-enabled],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [22:32:49] PROBLEM - db101 Puppet on db101 is CRITICAL: CRITICAL: Puppet has 23 failures. Last run 3 minutes ago with 23 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.gpg],File[/usr/local/bin/puppet-enabled],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [22:32:52] PROBLEM - cp21 Puppet on cp21 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:32:53] PROBLEM - matomo101 Puppet on matomo101 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:32:55] PROBLEM - gluster101 Puppet on gluster101 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:32:58] PROBLEM - test101 Puppet on test101 is CRITICAL: CRITICAL: Puppet has 676 failures. Last run 2 minutes ago with 676 failures. Failed resources (up to 3 shown): File[wiki.thunis.eu_private],File[commons.gyaanipedia.com],File[commons.gyaanipedia.com_private],File[wiki.pablojprieto.com.es] [22:32:58] PROBLEM - mem101 Puppet on mem101 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:33:02] PROBLEM - cloud10 Puppet on cloud10 is CRITICAL: CRITICAL: Puppet has 23 failures. Last run 3 minutes ago with 23 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.gpg],File[/usr/local/bin/puppet-enabled],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [22:33:12] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.08, 3.90, 3.93 [22:33:28] PROBLEM - phab121 Puppet on phab121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:34:04] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.89, 9.91, 9.15 [22:34:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.62, 9.20, 9.10 [22:34:45] PROBLEM - cp31 Puppet on cp31 is CRITICAL: CRITICAL: Puppet has 487 failures. Last run 3 minutes ago with 487 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.gpg],File[/usr/local/bin/puppet-enabled],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [22:35:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.41, 3.88, 3.93 [22:35:31] RECOVERY - bast101 Puppet on bast101 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:35:50] RECOVERY - mw111 Puppet on mw111 is OK: OK: Puppet is currently enabled, last run 12 seconds ago with 0 failures [22:35:55] RECOVERY - cp20 Puppet on cp20 is OK: OK: Puppet is currently enabled, last run 44 seconds ago with 0 failures [22:35:59] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.84, 10.39, 9.43 [22:36:38] RECOVERY - mw122 Puppet on mw122 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:36:43] RECOVERY - cp31 Puppet on cp31 is OK: OK: Puppet is currently enabled, last run 50 seconds ago with 0 failures [22:36:52] RECOVERY - cp21 Puppet on cp21 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:37:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.48, 4.08, 3.99 [22:37:55] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.10, 9.86, 9.34 [22:37:56] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.40, 8.18, 8.47 [22:37:57] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 0.75, 1.05, 1.98 [22:38:02] RECOVERY - cp30 Puppet on cp30 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:38:41] RECOVERY - db111 Puppet on db111 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [22:39:01] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 21 seconds ago with 0 failures [22:39:28] RECOVERY - phab121 Puppet on phab121 is OK: OK: Puppet is currently enabled, last run 34 seconds ago with 0 failures [22:40:22] RECOVERY - cloud12 Puppet on cloud12 is OK: OK: Puppet is currently enabled, last run 45 seconds ago with 0 failures [22:41:21] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 0.62, 0.92, 1.89 [22:41:25] RECOVERY - bast121 Puppet on bast121 is OK: OK: Puppet is currently enabled, last run 53 seconds ago with 0 failures [22:41:41] RECOVERY - ldap111 Puppet on ldap111 is OK: OK: Puppet is currently enabled, last run 16 seconds ago with 0 failures [22:41:51] RECOVERY - mw121 Puppet on mw121 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [22:41:57] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.54, 0.77, 1.65 [22:42:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.58, 9.68, 9.69 [22:42:13] RECOVERY - mw102 Puppet on mw102 is OK: OK: Puppet is currently enabled, last run 39 seconds ago with 0 failures [22:43:27] RECOVERY - mail121 Puppet on mail121 is OK: OK: Puppet is currently enabled, last run 31 seconds ago with 0 failures [22:43:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.97, 8.70, 8.55 [22:44:06] RECOVERY - es121 Puppet on es121 is OK: OK: Puppet is currently enabled, last run 45 seconds ago with 0 failures [22:44:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.27, 9.94, 9.78 [22:44:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.27, 9.39, 9.27 [22:44:56] RECOVERY - gluster101 Puppet on gluster101 is OK: OK: Puppet is currently enabled, last run 54 seconds ago with 0 failures [22:45:20] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.31, 0.61, 1.54 [22:46:01] RECOVERY - jobchron121 Puppet on jobchron121 is OK: OK: Puppet is currently enabled, last run 13 seconds ago with 0 failures [22:46:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.93, 9.47, 9.31 [22:47:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.76, 8.21, 7.61 [22:48:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.74, 9.53, 9.65 [22:48:17] RECOVERY - es101 Puppet on es101 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:48:58] RECOVERY - test101 Puppet on test101 is OK: OK: Puppet is currently enabled, last run 16 seconds ago with 0 failures [22:49:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.25, 3.75, 3.97 [22:49:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.27, 9.52, 9.17 [22:49:34] RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:50:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.69, 10.03, 9.83 [22:50:07] RECOVERY - mw112 Puppet on mw112 is OK: OK: Puppet is currently enabled, last run 52 seconds ago with 0 failures [22:50:37] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 9 seconds ago with 0 failures [22:50:48] RECOVERY - db121 Puppet on db121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:50:54] RECOVERY - matomo101 Puppet on matomo101 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:51:00] RECOVERY - mem101 Puppet on mem101 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:51:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.00, 4.19, 4.13 [22:51:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.85, 8.02, 7.68 [22:52:17] RECOVERY - prometheus101 Puppet on prometheus101 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:52:49] RECOVERY - db101 Puppet on db101 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:53:09] RECOVERY - mwtask111 Puppet on mwtask111 is OK: OK: Puppet is currently enabled, last run 20 seconds ago with 0 failures [22:53:09] RECOVERY - puppet111 Puppet on puppet111 is OK: OK: Puppet is currently enabled, last run 2 seconds ago with 0 failures [22:53:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.73, 9.98, 9.46 [22:53:39] RECOVERY - es111 Puppet on es111 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:55:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.15, 9.51, 9.13 [22:55:50] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 53 seconds ago with 0 failures [22:56:01] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.05, 1.66, 1.48 [22:56:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 6.68, 9.15, 9.65 [22:56:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.88, 9.79, 9.38 [22:57:49] RECOVERY - gluster121 Puppet on gluster121 is OK: OK: Puppet is currently enabled, last run 10 seconds ago with 0 failures [22:57:59] RECOVERY - gluster111 Puppet on gluster111 is OK: OK: Puppet is currently enabled, last run 42 seconds ago with 0 failures [22:58:01] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.63, 1.73, 1.52 [22:58:42] RECOVERY - graylog121 Puppet on graylog121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:59:00] PROBLEM - puppet111 Current Load on puppet111 is WARNING: WARNING - load average: 0.67, 1.22, 5.64 [22:59:32] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.09, 9.96, 9.48 [22:59:49] RECOVERY - mon111 Puppet on mon111 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:00:00] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.30, 1.58, 1.50 [23:00:02] !log [@test101] starting deploy of {'l10nupdate': True} to all [23:00:03] !log [@mwtask111] starting deploy of {'l10nupdate': True} to all [23:00:38] PROBLEM - cp31 Stunnel HTTP for mw112 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:01:00] RECOVERY - puppet111 Current Load on puppet111 is OK: OK - load average: 0.57, 1.07, 5.06 [23:01:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:01:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.92, 3.61, 4.00 [23:01:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:02:16] RECOVERY - mw101 Puppet on mw101 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:02:36] RECOVERY - cp31 Stunnel HTTP for mw112 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 0.407 second response time [23:03:32] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.91, 9.42, 9.42 [23:03:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.23, 9.73, 9.55 [23:05:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.16, 10.25, 9.75 [23:06:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.59, 9.22, 9.20 [23:06:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.81, 9.61, 9.70 [23:07:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.79, 9.14, 9.40 [23:08:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.41, 8.58, 8.95 [23:11:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.66, 3.78, 3.79 [23:13:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.93, 3.48, 3.69 [23:13:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.55, 9.81, 9.62 [23:15:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.08, 3.74, 3.73 [23:15:32] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.38, 7.33, 8.32 [23:15:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.55, 9.43, 9.51 [23:16:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.37, 7.55, 8.42 [23:17:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.67, 3.57, 3.67 [23:17:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.26, 3.11, 3.94 [23:20:28] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.78, 1.55, 1.29 [23:21:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.13, 8.32, 7.74 [23:22:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.31, 10.09, 9.23 [23:22:22] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.79, 1.26, 1.22 [23:22:38] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.85, 1.64, 1.18 [23:23:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.93, 7.71, 7.59 [23:24:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.28, 9.18, 9.00 [23:24:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.32, 7.57, 8.36 [23:24:32] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.38, 1.59, 1.22 [23:24:57] !log [@mwtask111] finished deploy of {'l10nupdate': True} to all - SUCCESS in 1494s [23:25:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:26:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.25, 9.36, 9.06 [23:27:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.44, 3.56, 3.56 [23:28:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.67, 9.33, 9.08 [23:28:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.78, 8.63, 8.57 [23:29:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.80, 3.55, 3.54 [23:30:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.07, 7.66, 8.22 [23:31:06] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.51, 3.45, 3.93 [23:31:11] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.47, 3.14, 3.39 [23:33:12] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.71, 6.50, 5.63 [23:33:46] !log [@test101] finished deploy of {'l10nupdate': True} to all - SUCCESS in 2024s [23:34:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:35:07] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.57, 6.59, 5.77 [23:35:30] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.24, 3.71, 3.69 [23:36:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.09, 9.62, 9.18 [23:37:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.05, 3.58, 3.66 [23:38:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.30, 9.57, 9.22 [23:39:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.58, 3.96, 3.78 [23:39:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.54, 8.49, 8.65 [23:39:48] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [23:39:48] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [23:41:18] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.21, 9.15, 8.57 [23:41:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.15, 3.71, 3.71 [23:42:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.39, 9.55, 9.24 [23:43:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.47, 9.30, 8.97 [23:44:05] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.94, 9.17, 8.49 [23:44:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.40, 9.06, 9.11 [23:45:09] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.42, 9.75, 8.99 [23:47:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.28, 10.41, 9.49 [23:47:57] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.75, 9.85, 8.86 [23:48:07] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.77, 3.47, 3.32 [23:49:03] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.05, 3.90, 3.76 [23:49:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.70, 9.41, 9.24 [23:49:52] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.69, 9.13, 8.73 [23:50:06] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.74, 3.17, 3.22 [23:50:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.59, 9.35, 9.10 [23:50:56] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.11, 10.29, 9.42 [23:51:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.71, 3.46, 3.62 [23:51:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.05, 3.63, 3.65 [23:53:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.97, 3.68, 3.65 [23:53:42] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.06, 9.85, 9.47 [23:57:42] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.82, 8.96, 9.20