[00:07:25] [Grafana] !tech RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [00:16:03] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 60%, RTA = 161.22 ms [00:18:02] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 166.06 ms [00:23:25] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [00:52:50] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [01:03:59] PROBLEM - cp41 HTTPS on cp41 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 35 - Recv failure: Connection reset by peer [01:04:17] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 109.123.230.163/cpweb, 2400:d320:2161:9775::1/cpweb [01:05:09] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 80%, RTA = 1181.41 ms [01:05:34] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 109.123.230.163/cpweb, 2400:d320:2161:9775::1/cpweb [01:06:10] PROBLEM - cp41 Nginx Backend for mwtask171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [01:06:14] PROBLEM - cp41 ferm_active on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:06:32] PROBLEM - cp41 Nginx Backend for mw181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:06:32] PROBLEM - cp41 Nginx Backend for reports171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:06:40] PROBLEM - cp41 NTP time on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:06:44] PROBLEM - cp41 Nginx Backend for mw162 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:06:45] PROBLEM - cp41 Nginx Backend for swiftproxy171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:06:50] PROBLEM - cp41 Nginx Backend for test151 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:06:51] PROBLEM - cp41 Puppet on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [01:06:52] PROBLEM - cp41 conntrack_table_size on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:06:56] PROBLEM - cp41 Nginx Backend for puppet181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:06:56] PROBLEM - cp41 Nginx Backend for mwtask181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:07:02] PROBLEM - cp41 Nginx Backend for mw161 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [01:07:04] PROBLEM - cp41 PowerDNS Recursor on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:07:24] PROBLEM - cp41 Nginx Backend for mw171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:07:24] PROBLEM - cp41 Nginx Backend for mon181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:07:26] PROBLEM - cp41 Nginx Backend for mw152 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:07:27] PROBLEM - cp41 SSH on cp41 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:08:19] PROBLEM - cp41 Nginx Backend for mw182 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:08:21] PROBLEM - cp41 Varnish Backends on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:08:33] PROBLEM - cp41 Disk Space on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:09:17] PROBLEM - cp41 Nginx Backend for mw151 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:09:33] PROBLEM - cp41 Current Load on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:09:47] PROBLEM - cp41 Nginx Backend for swiftproxy161 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:10:00] PROBLEM - cp41 Nginx Backend for matomo151 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:10:08] PROBLEM - cp41 Nginx Backend for mw172 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [01:12:15] PROBLEM - cp41 Nginx Backend for phorge171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [01:13:03] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:16:02] RECOVERY - cp41 conntrack_table_size on cp41 is OK: OK: nf_conntrack is 0 % full [01:20:50] RECOVERY - cp41 Current Load on cp41 is OK: LOAD OK - total load average: 0.09, 0.08, 0.15 [01:21:33] RECOVERY - cp41 Nginx Backend for mw161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8115 [01:21:50] RECOVERY - cp41 Nginx Backend for mon181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8201 [01:22:43] PROBLEM - cp41 conntrack_table_size on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [01:27:22] PROBLEM - cp41 Nginx Backend for mw161 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [01:28:01] PROBLEM - cp41 Current Load on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:28:37] PROBLEM - cp41 Nginx Backend for mon181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:29:06] RECOVERY - cp41 Nginx Backend for puppet181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8204 [01:30:26] RECOVERY - cp41 Nginx Backend for mw172 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8118 [01:33:41] RECOVERY - cp41 Nginx Backend for reports171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8205 [01:33:42] RECOVERY - cp41 Nginx Backend for mw152 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8114 [01:33:42] RECOVERY - cp41 Nginx Backend for phorge171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8202 [01:33:42] RECOVERY - cp41 Nginx Backend for swiftproxy161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8206 [01:33:43] RECOVERY - cp41 Nginx Backend for mw151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8113 [01:33:44] RECOVERY - cp41 Nginx Backend for mw161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8115 [01:33:46] RECOVERY - cp41 Nginx Backend for mon181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8201 [01:37:09] RECOVERY - cp41 SSH on cp41 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [01:37:36] PROBLEM - cp41 Nginx Backend for mw172 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:38:28] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 121.72 ms [01:39:48] PROBLEM - cp41 Nginx Backend for reports171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:40:11] PROBLEM - cp41 Nginx Backend for mw151 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:40:32] PROBLEM - cp41 Nginx Backend for mon181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:40:34] PROBLEM - cp41 Nginx Backend for swiftproxy161 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:40:36] PROBLEM - cp41 Nginx Backend for mw161 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:40:36] PROBLEM - cp41 Nginx Backend for mw152 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:40:37] PROBLEM - cp41 Nginx Backend for puppet181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:41:17] PROBLEM - cp41 Nginx Backend for phorge171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:43:34] PROBLEM - cp41 SSH on cp41 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:48:46] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 100% [01:51:09] PROBLEM - Host cp41 is DOWN: PING CRITICAL - Packet loss = 100% [02:24:30] RECOVERY - Host cp41 is UP: PING OK - Packet loss = 68%, RTA = 1171.17 ms [02:25:04] RECOVERY - cp41 Nginx Backend for mw182 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8120 [02:25:15] RECOVERY - cp41 Nginx Backend for mw171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8117 [02:25:47] RECOVERY - cp41 Nginx Backend for phorge171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8202 [02:26:40] RECOVERY - cp41 Nginx Backend for mw151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8113 [02:26:52] RECOVERY - cp41 Nginx Backend for mw161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8115 [02:27:25] RECOVERY - cp41 Nginx Backend for mw172 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8118 [02:28:14] RECOVERY - cp41 Nginx Backend for mw152 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8114 [02:29:09] RECOVERY - cp41 Nginx Backend for swiftproxy161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8206 [02:30:15] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset -0.4591873884 secs [02:31:05] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is WARNING: WARNING - NGINX Error Rate is 55% [02:31:53] PROBLEM - cp41 Nginx Backend for mw182 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:32:45] PROBLEM - cp41 Nginx Backend for phorge171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:33:14] PROBLEM - cp41 NTP time on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:33:15] PROBLEM - cp41 Nginx Backend for mw172 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [02:33:48] RECOVERY - cp41 Nginx Backend for mon181 on cp41 is OK: TCP OK - 0.002 second response time on localhost port 8201 [02:35:45] RECOVERY - cp41 Nginx Backend for phorge171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8202 [02:36:07] RECOVERY - cp41 Nginx Backend for test151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8181 [02:36:09] RECOVERY - cp41 Nginx Backend for reports171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8205 [02:38:35] PROBLEM - cp41 Nginx Backend for mw151 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [02:38:52] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:39:08] RECOVERY - cp41 Nginx Backend for swiftproxy171 on cp41 is OK: TCP OK - 0.001 second response time on localhost port 8207 [02:39:37] RECOVERY - cp41 Disk Space on cp41 is OK: DISK OK - free space: / 15756MiB (16% inode=98%); [02:39:47] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset 0.005908548832 secs [02:39:51] PROBLEM - cp41 Nginx Backend for mw171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [02:40:05] RECOVERY - cp41 ferm_active on cp41 is OK: OK ferm input default policy is set [02:41:21] PROBLEM - cp41 Nginx Backend for phorge171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:41:31] PROBLEM - cp41 Nginx Backend for swiftproxy161 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:41:40] RECOVERY - cp41 Nginx Backend for mw172 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8118 [02:42:10] PROBLEM - cp41 Nginx Backend for mw152 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:42:37] RECOVERY - cp41 conntrack_table_size on cp41 is OK: OK: nf_conntrack is 0 % full [02:42:54] RECOVERY - cp41 Nginx Backend for mw181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8119 [02:42:55] RECOVERY - cp41 Nginx Backend for puppet181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8204 [02:43:55] RECOVERY - cp41 Nginx Backend for mw151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8113 [02:44:23] PROBLEM - cp41 Nginx Backend for swiftproxy171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [02:44:29] PROBLEM - cp41 NTP time on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [02:46:45] RECOVERY - cp41 Nginx Backend for swiftproxy161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8206 [02:46:58] PROBLEM - cp41 ferm_active on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:47:22] RECOVERY - cp41 Nginx Backend for phorge171 on cp41 is OK: TCP OK - 0.001 second response time on localhost port 8202 [02:47:57] RECOVERY - cp41 Nginx Backend for matomo151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8203 [02:48:55] PROBLEM - cp41 Nginx Backend for mw161 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:49:10] PROBLEM - cp41 Nginx Backend for test151 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [02:49:33] PROBLEM - cp41 Disk Space on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [02:49:45] PROBLEM - cp41 Nginx Backend for mon181 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [02:50:02] PROBLEM - cp41 Nginx Backend for puppet181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:50:10] PROBLEM - cp41 Nginx Backend for reports171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [02:50:16] PROBLEM - cp41 Nginx Backend for mw181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:50:20] PROBLEM - cp41 conntrack_table_size on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:51:00] PROBLEM - cp41 Nginx Backend for mw151 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:51:11] RECOVERY - cp41 Nginx Backend for mw152 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8114 [02:52:13] RECOVERY - cp41 Nginx Backend for swiftproxy171 on cp41 is OK: TCP OK - 0.001 second response time on localhost port 8207 [02:52:36] RECOVERY - cp41 Nginx Backend for reports171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8205 [02:52:52] PROBLEM - cp41 Nginx Backend for swiftproxy161 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:53:16] RECOVERY - cp41 Nginx Backend for mw181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8119 [02:53:20] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is WARNING: WARNING - NGINX Error Rate is 57% [02:55:06] RECOVERY - cp41 Nginx Backend for mw182 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8120 [02:55:48] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [02:56:02] RECOVERY - cp41 SSH on cp41 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [02:56:22] RECOVERY - cp41 Nginx Backend for mw151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8113 [02:57:16] RECOVERY - cp41 Nginx Backend for mon181 on cp41 is OK: TCP OK - 0.001 second response time on localhost port 8201 [02:57:55] PROBLEM - cp41 Nginx Backend for reports171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [02:58:03] PROBLEM - cp41 Nginx Backend for mw172 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:58:35] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is WARNING: WARNING - NGINX Error Rate is 49% [02:59:21] PROBLEM - cp41 Nginx Backend for matomo151 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:59:27] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset 0.09732788801 secs [03:00:23] PROBLEM - cp41 Nginx Backend for mw181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:00:26] PROBLEM - cp41 SSH on cp41 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:01:25] RECOVERY - cp41 Nginx Backend for mw162 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8116 [03:01:25] PROBLEM - cp41 Nginx Backend for phorge171 on cp41 is CRITICAL: connect to address 2400:d320:2161:9775::1 port 5666: No route to hostconnect to host 2400:d320:2161:9775::1 port 5666: No route to host [03:01:32] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:01:45] PROBLEM - cp41 Nginx Backend for mw152 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:01:55] PROBLEM - cp41 Nginx Backend for mw182 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:02:31] RECOVERY - cp41 SSH on cp41 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [03:03:24] PROBLEM - cp41 Disk Space on cp41 is WARNING: DISK WARNING - free space: / 7979MiB (8% inode=98%); [03:03:54] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is WARNING: WARNING - NGINX Error Rate is 53% [03:04:18] RECOVERY - cp41 Nginx Backend for reports171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8205 [03:04:24] RECOVERY - cp41 Nginx Backend for mw152 on cp41 is OK: TCP OK - 0.001 second response time on localhost port 8114 [03:04:25] RECOVERY - cp41 Nginx Backend for mw172 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8118 [03:05:21] PROBLEM - cp41 Nginx Backend for swiftproxy171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:06:18] PROBLEM - cp41 Disk Space on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:06:19] RECOVERY - cp41 Nginx Backend for puppet181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8204 [03:07:24] PROBLEM - cp41 Nginx Backend for mw162 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:07:41] RECOVERY - cp41 Nginx Backend for mw182 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8120 [03:08:16] RECOVERY - cp41 Nginx Backend for phorge171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8202 [03:09:04] PROBLEM - cp41 Nginx Backend for mw172 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:09:10] RECOVERY - cp41 Nginx Backend for mwtask171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8161 [03:09:30] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:09:51] PROBLEM - cp41 NTP time on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:10:03] RECOVERY - cp41 Nginx Backend for swiftproxy161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8206 [03:10:05] PROBLEM - cp41 SSH on cp41 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:10:05] PROBLEM - cp41 Nginx Backend for mw151 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:10:31] PROBLEM - cp41 Nginx Backend for mw152 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:11:01] PROBLEM - cp41 Nginx Backend for reports171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:11:13] PROBLEM - cp41 Nginx Backend for mon181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:12:03] PROBLEM - cp41 Nginx Backend for puppet181 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:12:29] RECOVERY - cp41 Nginx Backend for mw181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8119 [03:12:35] PROBLEM - cp41 Current Load on cp41 is WARNING: LOAD WARNING - total load average: 3.89, 4.10, 7.90 [03:14:07] PROBLEM - cp41 Nginx Backend for mw182 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:15:17] PROBLEM - cp41 Nginx Backend for mwtask171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:15:23] RECOVERY - cp41 Nginx Backend for mw152 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8114 [03:15:33] PROBLEM - cp41 Current Load on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:16:00] RECOVERY - cp41 Nginx Backend for mw151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8113 [03:16:14] PROBLEM - cp41 Nginx Backend for swiftproxy161 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:17:08] RECOVERY - cp41 Nginx Backend for swiftproxy171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8207 [03:17:55] RECOVERY - cp41 conntrack_table_size on cp41 is OK: OK: nf_conntrack is 0 % full [03:18:26] PROBLEM - cp41 Nginx Backend for mw181 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:18:43] RECOVERY - cp41 Nginx Backend for matomo151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8203 [03:19:27] RECOVERY - cp41 Nginx Backend for mw182 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8120 [03:20:24] RECOVERY - cp41 Nginx Backend for mwtask181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8160 [03:20:31] RECOVERY - cp41 Nginx Backend for mw171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8117 [03:21:10] RECOVERY - cp41 Nginx Backend for mw181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8119 [03:21:37] RECOVERY - cp41 Nginx Backend for mw162 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8116 [03:22:49] RECOVERY - cp41 SSH on cp41 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [03:23:37] PROBLEM - cp41 Nginx Backend for phorge171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:24:24] PROBLEM - cp41 Nginx Backend for mw152 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:25:11] PROBLEM - cp41 Nginx Backend for mw182 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:25:21] PROBLEM - cp41 Nginx Backend for mw151 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:26:01] RECOVERY - cp41 Current Load on cp41 is OK: LOAD OK - total load average: 4.14, 4.01, 5.55 [03:26:06] RECOVERY - cp41 Nginx Backend for phorge171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8202 [03:26:38] PROBLEM - cp41 conntrack_table_size on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:26:51] RECOVERY - cp41 Nginx Backend for swiftproxy161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8206 [03:26:54] PROBLEM - cp41 Nginx Backend for mw171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:27:19] PROBLEM - cp41 Nginx Backend for mwtask181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:28:44] PROBLEM - cp41 Nginx Backend for mw162 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:28:49] PROBLEM - cp41 Nginx Backend for matomo151 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:29:16] PROBLEM - cp41 Nginx Backend for swiftproxy171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:29:53] RECOVERY - cp41 Nginx Backend for mw171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8117 [03:29:58] RECOVERY - cp41 Nginx Backend for mwtask181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8160 [03:30:14] PROBLEM - cp41 Nginx Backend for mw181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:31:09] RECOVERY - cp41 Nginx Backend for mw162 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8116 [03:31:31] PROBLEM - cp41 SSH on cp41 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:31:46] RECOVERY - cp41 Nginx Backend for mon181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8201 [03:32:00] PROBLEM - cp41 Nginx Backend for phorge171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:32:06] PROBLEM - cp41 Current Load on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:33:01] RECOVERY - cp41 ferm_active on cp41 is OK: OK ferm input default policy is set [03:33:04] RECOVERY - cp41 Nginx Backend for mw151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8113 [03:33:57] PROBLEM - cp41 Nginx Backend for swiftproxy161 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:35:15] RECOVERY - cp41 conntrack_table_size on cp41 is OK: OK: nf_conntrack is 0 % full [03:35:15] RECOVERY - cp41 Nginx Backend for test151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8181 [03:35:43] RECOVERY - cp41 SSH on cp41 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [03:36:07] RECOVERY - cp41 Nginx Backend for mw182 on cp41 is OK: TCP OK - 0.005 second response time on localhost port 8120 [03:36:33] RECOVERY - cp41 Nginx Backend for mw161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8115 [03:36:44] RECOVERY - cp41 Nginx Backend for matomo151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8203 [03:37:17] PROBLEM - cp41 Nginx Backend for mw162 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:37:19] PROBLEM - cp41 Nginx Backend for mw171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:38:08] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset 0.007228463888 secs [03:38:15] RECOVERY - cp41 Nginx Backend for mw172 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8118 [03:38:19] PROBLEM - cp41 Nginx Backend for mw151 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:38:32] PROBLEM - cp41 ferm_active on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:39:37] RECOVERY - cp41 Nginx Backend for mw152 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8114 [03:40:07] PROBLEM - cp41 SSH on cp41 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:41:09] RECOVERY - cp41 Nginx Backend for swiftproxy171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8207 [03:41:43] PROBLEM - cp41 Nginx Backend for test151 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:42:03] RECOVERY - cp41 Nginx Backend for mwtask171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8161 [03:42:23] PROBLEM - cp41 Nginx Backend for mw182 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:42:24] RECOVERY - cp41 Nginx Backend for swiftproxy161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8206 [03:43:24] PROBLEM - cp41 Nginx Backend for mon181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:43:24] RECOVERY - cp41 Nginx Backend for puppet181 on cp41 is OK: TCP OK - 0.001 second response time on localhost port 8204 [03:43:54] RECOVERY - cp41 ferm_active on cp41 is OK: OK ferm input default policy is set [03:44:19] RECOVERY - cp41 SSH on cp41 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [03:44:20] RECOVERY - cp41 Nginx Backend for reports171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8205 [03:45:08] PROBLEM - cp41 Nginx Backend for mw161 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:45:27] PROBLEM - cp41 Nginx Backend for mwtask181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:45:51] RECOVERY - cp41 Nginx Backend for mon181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8201 [03:46:05] RECOVERY - cp41 Nginx Backend for phorge171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8202 [03:46:38] PROBLEM - cp41 conntrack_table_size on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:47:47] RECOVERY - cp41 Nginx Backend for mw161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8115 [03:47:58] PROBLEM - cp41 NTP time on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:48:03] PROBLEM - cp41 Nginx Backend for swiftproxy171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:48:46] PROBLEM - cp41 Nginx Backend for matomo151 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:49:33] RECOVERY - cp41 conntrack_table_size on cp41 is OK: OK: nf_conntrack is 0 % full [03:49:47] PROBLEM - cp41 Nginx Backend for reports171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:51:17] RECOVERY - cp41 Nginx Backend for mw171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8117 [03:51:22] PROBLEM - cp41 Nginx Backend for swiftproxy161 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [03:51:28] RECOVERY - cp41 Nginx Backend for matomo151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8203 [03:51:43] RECOVERY - cp41 Nginx Backend for mwtask181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8160 [03:51:51] RECOVERY - cp41 Current Load on cp41 is OK: LOAD OK - total load average: 3.96, 4.03, 4.30 [03:51:54] RECOVERY - cp41 Nginx Backend for mw162 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8116 [03:51:59] PROBLEM - cp41 SSH on cp41 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:53:49] RECOVERY - cp41 Nginx Backend for swiftproxy161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8206 [03:54:06] RECOVERY - cp41 Nginx Backend for test151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8181 [03:55:00] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset -0.1640518904 secs [03:55:19] RECOVERY - cp41 Nginx Backend for reports171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8205 [03:55:42] RECOVERY - cp41 Nginx Backend for swiftproxy171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8207 [03:56:36] PROBLEM - cp41 Nginx Backend for mw161 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:57:07] PROBLEM - cp41 Current Load on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:57:24] PROBLEM - cp41 Nginx Backend for puppet181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:57:28] RECOVERY - cp41 Nginx Backend for mw181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8119 [03:57:55] PROBLEM - cp41 Nginx Backend for mon181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:57:57] RECOVERY - cp41 Nginx Backend for mw151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8113 [03:59:23] RECOVERY - cp41 Current Load on cp41 is OK: LOAD OK - total load average: 4.72, 4.27, 4.30 [04:00:15] RECOVERY - cp41 Nginx Backend for mon181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8201 [04:00:41] PROBLEM - cp41 Nginx Backend for reports171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:01:03] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset -0.07208821177 secs [04:01:05] RECOVERY - cp41 Nginx Backend for mw182 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8120 [04:01:08] PROBLEM - cp41 Nginx Backend for phorge171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:01:35] PROBLEM - cp41 Nginx Backend for swiftproxy171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:01:57] RECOVERY - cp41 Nginx Backend for puppet181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8204 [04:02:38] PROBLEM - cp41 ferm_active on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:02:55] PROBLEM - cp41 Nginx Backend for mw162 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:03:53] RECOVERY - cp41 Nginx Backend for mw161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8115 [04:04:02] RECOVERY - cp41 Nginx Backend for phorge171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8202 [04:05:18] RECOVERY - cp41 PowerDNS Recursor on cp41 is OK: DNS OK: 3.214 seconds response time. wikitide.net returns 109.123.230.163,2400:d320:2161:9775::1 [04:06:21] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is WARNING: WARNING - NGINX Error Rate is 56% [04:06:51] RECOVERY - cp41 SSH on cp41 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [04:07:15] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset 0.1646781266 secs [04:07:38] PROBLEM - cp41 Nginx Backend for mw172 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:07:42] PROBLEM - cp41 Nginx Backend for mw171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:07:56] PROBLEM - cp41 conntrack_table_size on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:08:40] RECOVERY - cp41 Nginx Backend for mw162 on cp41 is OK: TCP OK - 0.001 second response time on localhost port 8116 [04:09:55] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset -0.03314751387 secs [04:10:06] RECOVERY - cp41 Nginx Backend for mw172 on cp41 is OK: TCP OK - 0.001 second response time on localhost port 8118 [04:10:19] PROBLEM - cp41 Nginx Backend for test151 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:10:21] PROBLEM - cp41 Nginx Backend for mw161 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:10:26] PROBLEM - cp41 Nginx Backend for phorge171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:11:15] PROBLEM - cp41 SSH on cp41 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:11:27] RECOVERY - cp41 Nginx Backend for reports171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8205 [04:11:30] PROBLEM - cp41 Nginx Backend for mw152 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:11:43] PROBLEM - cp41 PowerDNS Recursor on cp41 is CRITICAL: No response from DNS ::1 [04:12:00] PROBLEM - cp41 Nginx Backend for mw151 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:13:01] RECOVERY - cp41 Nginx Backend for test151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8181 [04:13:38] RECOVERY - cp41 Nginx Backend for phorge171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8202 [04:13:58] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:14:12] RECOVERY - cp41 Nginx Backend for mw151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8113 [04:14:17] RECOVERY - cp41 Nginx Backend for swiftproxy171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8207 [04:14:51] PROBLEM - cp41 Nginx Backend for mwtask171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:15:13] PROBLEM - cp41 Nginx Backend for mw162 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:15:31] PROBLEM - cp41 Nginx Backend for swiftproxy161 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:15:40] PROBLEM - cp41 Nginx Backend for matomo151 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:15:45] RECOVERY - cp41 Nginx Backend for mw171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8117 [04:16:24] PROBLEM - cp41 NTP time on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:16:42] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is WARNING: WARNING - NGINX Error Rate is 56% [04:18:09] PROBLEM - cp41 Nginx Backend for puppet181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:18:37] RECOVERY - cp41 Nginx Backend for swiftproxy161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8206 [04:18:41] RECOVERY - cp41 Nginx Backend for mw161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8115 [04:18:48] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:18:49] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset -0.0346712172 secs [04:18:50] PROBLEM - cp41 Nginx Backend for mw172 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:19:07] PROBLEM - cp41 Nginx Backend for mw182 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:21:33] PROBLEM - cp41 Nginx Backend for phorge171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:21:45] RECOVERY - cp41 Nginx Backend for mw152 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8114 [04:22:02] PROBLEM - cp41 Nginx Backend for mw171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:22:02] RECOVERY - cp41 conntrack_table_size on cp41 is OK: OK: nf_conntrack is 0 % full [04:22:02] PROBLEM - cp41 Current Load on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:23:11] RECOVERY - cp41 Nginx Backend for matomo151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8203 [04:23:16] PROBLEM - cp41 Nginx Backend for mw181 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:24:42] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset -0.1126922965 secs [04:24:42] PROBLEM - cp41 Nginx Backend for mw161 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:25:10] PROBLEM - cp41 Nginx Backend for mwtask181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:25:14] PROBLEM - cp41 Nginx Backend for swiftproxy161 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:25:22] PROBLEM - cp41 Nginx Backend for swiftproxy171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:25:37] PROBLEM - cp41 Nginx Backend for test151 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:27:00] PROBLEM - cp41 Nginx Backend for mon181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:27:08] RECOVERY - cp41 Nginx Backend for mw161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8115 [04:27:24] RECOVERY - cp41 Nginx Backend for puppet181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8204 [04:27:28] RECOVERY - cp41 Nginx Backend for mwtask181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8160 [04:27:40] PROBLEM - cp41 NTP time on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:27:43] RECOVERY - cp41 Nginx Backend for swiftproxy171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8207 [04:28:17] RECOVERY - cp41 Nginx Backend for test151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8181 [04:28:32] RECOVERY - cp41 ferm_active on cp41 is OK: OK ferm input default policy is set [04:28:39] PROBLEM - cp41 Nginx Backend for mw152 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:29:27] RECOVERY - cp41 Nginx Backend for mwtask171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8161 [04:30:03] RECOVERY - cp41 SSH on cp41 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [04:30:26] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset -0.2208278775 secs [04:30:29] PROBLEM - cp41 conntrack_table_size on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:30:38] RECOVERY - cp41 Nginx Backend for mw171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8117 [04:31:08] RECOVERY - cp41 Nginx Backend for swiftproxy161 on cp41 is OK: TCP OK - 0.001 second response time on localhost port 8206 [04:32:16] RECOVERY - cp41 Nginx Backend for mw162 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8116 [04:32:53] RECOVERY - cp41 Nginx Backend for mw172 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8118 [04:32:54] RECOVERY - cp41 conntrack_table_size on cp41 is OK: OK: nf_conntrack is 0 % full [04:33:10] PROBLEM - cp41 Nginx Backend for swiftproxy171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:33:32] PROBLEM - cp41 Nginx Backend for mwtask181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:33:36] PROBLEM - cp41 Nginx Backend for mw161 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:33:40] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is WARNING: WARNING - NGINX Error Rate is 56% [04:34:03] PROBLEM - cp41 Nginx Backend for mwtask171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:34:27] PROBLEM - cp41 SSH on cp41 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:34:32] PROBLEM - cp41 Nginx Backend for puppet181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:34:35] PROBLEM - cp41 Nginx Backend for matomo151 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:35:00] RECOVERY - cp41 Nginx Backend for phorge171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8202 [04:35:21] RECOVERY - cp41 Nginx Backend for mw182 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8120 [04:35:46] RECOVERY - cp41 Nginx Backend for mwtask181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8160 [04:35:59] PROBLEM - cp41 NTP time on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:36:27] RECOVERY - cp41 Nginx Backend for mwtask171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8161 [04:37:14] PROBLEM - cp41 ferm_active on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:38:10] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset -0.4423289001 secs [04:38:23] RECOVERY - cp41 Nginx Backend for mon181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8201 [04:39:15] PROBLEM - cp41 Nginx Backend for swiftproxy161 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:39:27] RECOVERY - cp41 Nginx Backend for mw161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8115 [04:39:29] RECOVERY - cp41 Nginx Backend for puppet181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8204 [04:40:23] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset -0.01423409581 secs [04:40:38] RECOVERY - cp41 Nginx Backend for mw152 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8114 [04:41:27] RECOVERY - cp41 Nginx Backend for swiftproxy161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8206 [04:42:53] RECOVERY - cp41 SSH on cp41 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [04:43:10] PROBLEM - cp41 Nginx Backend for mw182 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:44:03] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:45:19] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset 0.3673155606 secs [04:45:30] RECOVERY - cp41 Nginx Backend for mw182 on cp41 is OK: TCP OK - 0.001 second response time on localhost port 8120 [04:45:43] PROBLEM - cp41 Nginx Backend for mw151 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:46:44] PROBLEM - cp41 Nginx Backend for puppet181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:46:57] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is WARNING: WARNING - NGINX Error Rate is 56% [04:47:12] PROBLEM - cp41 Nginx Backend for swiftproxy161 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:47:17] PROBLEM - cp41 Nginx Backend for reports171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:47:17] PROBLEM - cp41 SSH on cp41 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:47:43] RECOVERY - cp41 Current Load on cp41 is OK: LOAD OK - total load average: 4.12, 4.13, 4.17 [04:48:10] RECOVERY - cp41 ferm_active on cp41 is OK: OK ferm input default policy is set [04:48:18] PROBLEM - cp41 NTP time on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:48:32] RECOVERY - cp41 Nginx Backend for mw151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8113 [04:49:25] PROBLEM - cp41 Nginx Backend for mw172 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:49:57] RECOVERY - cp41 Nginx Backend for swiftproxy161 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8206 [04:51:02] RECOVERY - cp41 Nginx Backend for mw181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8119 [04:51:54] PROBLEM - cp41 Nginx Backend for mwtask171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:52:07] RECOVERY - cp41 Nginx Backend for puppet181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8204 [04:53:36] RECOVERY - cp41 SSH on cp41 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [04:53:54] PROBLEM - cp41 Nginx Backend for phorge171 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:54:47] RECOVERY - cp41 Nginx Backend for reports171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8205 [04:55:18] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:55:20] PROBLEM - cp41 Nginx Backend for mw152 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [04:55:22] RECOVERY - cp41 Nginx Backend for mw172 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8118 [04:55:28] PROBLEM - cp41 Nginx Backend for mwtask181 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:55:39] PROBLEM - cp41 Nginx Backend for test151 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:55:56] RECOVERY - cp41 Nginx Backend for swiftproxy171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8207 [04:56:34] PROBLEM - cp41 conntrack_table_size on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:56:49] PROBLEM - cp41 Nginx Backend for swiftproxy161 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:57:02] PROBLEM - cp41 Nginx Backend for mw161 on cp41 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:57:37] RECOVERY - cp41 Nginx Backend for matomo151 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8203 [04:57:44] RECOVERY - cp41 Nginx Backend for mwtask181 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8160 [04:58:02] PROBLEM - cp41 SSH on cp41 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:58:28] RECOVERY - cp41 Nginx Backend for test151 on cp41 is OK: TCP OK - 0.001 second response time on localhost port 8181 [04:59:16] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset 0.3130190074 secs [04:59:46] PROBLEM - cp41 Current Load on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [05:00:08] PROBLEM - cp41 HTTP 4xx/5xx ERROR Rate on cp41 is WARNING: WARNING - NGINX Error Rate is 56% [05:00:22] PROBLEM - cp41 Nginx Backend for reports171 on cp41 is CRITICAL: CHECK_NRPE: Error - Could not connect to 2400:d320:2161:9775::1: Connection reset by peer [05:02:06]