[00:00:32] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 18.94, 6.85, 4.81 [00:03:19] PROBLEM - es141 Current Load on es141 is CRITICAL: CRITICAL - load average: 6.62, 3.38, 1.77 [00:04:29] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 4.746 second response time [00:05:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 11.56, 6.89, 3.59 [00:08:02] PROBLEM - mw141 Current Load on mw141 is WARNING: WARNING - load average: 11.12, 9.39, 7.40 [00:10:02] RECOVERY - mw141 Current Load on mw141 is OK: OK - load average: 7.73, 8.83, 7.45 [00:11:15] PROBLEM - es141 Current Load on es141 is WARNING: WARNING - load average: 2.80, 3.94, 2.82 [00:12:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 1.88, 3.27, 3.77 [00:13:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 5.07, 7.54, 5.65 [00:13:15] RECOVERY - es141 Current Load on es141 is OK: OK - load average: 1.70, 3.14, 2.66 [00:15:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.66, 6.19, 5.38 [00:16:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.09, 3.78, 3.85 [00:18:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.37, 3.52, 3.74 [00:24:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.29, 4.24, 3.97 [00:26:37] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:28:08] PROBLEM - es141 Current Load on es141 is CRITICAL: CRITICAL - load average: 5.16, 3.71, 3.07 [00:28:43] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 8.782 second response time [00:29:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 13.22, 9.42, 6.95 [00:29:42] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 6.08, 3.09, 1.96 [00:30:05] PROBLEM - es141 Current Load on es141 is WARNING: WARNING - load average: 3.61, 3.54, 3.08 [00:30:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.09, 3.85, 3.97 [00:32:01] PROBLEM - es141 Current Load on es141 is CRITICAL: CRITICAL - load average: 4.65, 4.00, 3.31 [00:32:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 3.97, 4.01, 4.01 [00:35:38] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:35:47] PROBLEM - cp23 APT on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:37:07] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.3935116231 secs [00:37:39] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 2.18, 3.41, 2.67 [00:37:47] RECOVERY - cp23 APT on cp23 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:39:38] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.81, 2.49, 2.42 [00:44:13] PROBLEM - es141 PowerDNS Recursor on es141 is CRITICAL: CRITICAL - Plugin timed out while executing system call [00:46:14] RECOVERY - es141 PowerDNS Recursor on es141 is OK: DNS OK: 2.103 seconds response time. miraheze.org returns 109.228.51.216,217.174.247.33,2a00:da00:1800:326::1,2a00:da00:1800:328::1 [00:47:42] PROBLEM - es141 Current Load on es141 is WARNING: WARNING - load average: 2.21, 3.49, 3.74 [00:48:08] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 9.471 second response time [00:50:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 2.59, 3.54, 3.89 [00:54:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.20, 4.03, 3.97 [00:55:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 4.87, 6.26, 7.73 [00:55:31] RECOVERY - es141 Current Load on es141 is OK: OK - load average: 2.83, 2.97, 3.37 [01:01:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.35, 7.35, 7.61 [01:01:07] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.02883097529 secs [01:02:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [01:03:23] PROBLEM - es141 Current Load on es141 is WARNING: WARNING - load average: 3.86, 3.47, 3.39 [01:05:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 2.20, 5.49, 6.89 [01:05:20] RECOVERY - es141 Current Load on es141 is OK: OK - load average: 2.27, 2.96, 3.21 [01:06:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 2.86, 3.64, 3.94 [01:09:01] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.92, 7.87, 7.49 [01:14:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.48, 3.21, 3.92 [01:14:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.03, 3.32, 3.60 [01:16:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.00, 3.28, 3.56 [01:18:53] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [01:18:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 2.17, 2.76, 3.32 [01:22:00] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [01:22:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.79, 4.76, 4.19 [01:24:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.20, 3.44, 3.97 [01:25:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 5.17, 6.58, 7.87 [01:26:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.73, 3.63, 3.95 [01:28:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.03, 3.20, 3.74 [01:29:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.18, 6.89, 7.62 [01:30:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.86, 4.02, 3.49 [01:31:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.58, 6.89, 7.52 [01:34:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.71, 3.94, 3.58 [01:35:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 11.24, 8.78, 8.11 [01:36:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.39, 4.43, 4.08 [01:36:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.80, 4.47, 3.81 [01:37:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 3.92, 6.97, 7.54 [01:40:59] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:43:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.62, 5.06, 6.55 [01:47:45] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [01:51:48] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.33, 7.90, 7.14 [01:53:03] PROBLEM - es141 PowerDNS Recursor on es141 is CRITICAL: CRITICAL - Plugin timed out while executing system call [01:53:30] PROBLEM - es141 Current Load on es141 is CRITICAL: CRITICAL - load average: 4.92, 4.07, 3.46 [01:54:30] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [01:55:01] RECOVERY - es141 PowerDNS Recursor on es141 is OK: DNS OK: 0.433 seconds response time. miraheze.org returns 109.228.51.216,217.174.247.33,2a00:da00:1800:326::1,2a00:da00:1800:328::1 [01:55:27] RECOVERY - es141 Current Load on es141 is OK: OK - load average: 1.10, 2.94, 3.12 [01:55:37] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 2.90, 6.59, 6.90 [01:56:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.04, 3.50, 3.95 [01:56:35] !log [void@cp32] restart varnish [01:57:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:57:31] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.87, 6.02, 6.64 [02:01:19] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 4.56, 7.19, 7.16 [02:01:31] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 1.793 second response time [02:03:13] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.43, 5.16, 6.42 [02:03:35] hmm, something seems up with mw121 and mw122, they seem to be maxing out CPU [02:04:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.42, 2.57, 3.28 [02:06:19] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:06:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.04, 2.71, 3.81 [02:08:19] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 2.662 second response time [02:12:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.36, 3.86, 3.95 [02:13:21] !log [void@cloud14] reboot mw122 [02:13:35] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 0.88, 0.18, 0.06 [02:14:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.23, 3.21, 3.70 [02:14:56] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:17:13] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 3.98, 4.28, 3.61 [02:17:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 19.03, 10.56, 4.34 [02:19:09] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.19, 3.85, 3.53 [02:20:09] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [02:20:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.75, 2.51, 3.27 [02:21:06] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.96, 3.19, 3.32 [02:24:30] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [02:26:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.54, 4.06, 3.68 [02:27:16] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 6.105 second response time [02:30:46] PROBLEM - cp23 APT on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:32:45] RECOVERY - cp23 APT on cp23 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [02:33:33] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:35:28] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 1.165 second response time [02:38:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 2.43, 3.24, 3.99 [02:39:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.12, 11.64, 11.10 [02:40:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.09, 3.49, 3.75 [02:44:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.71, 2.26, 3.30 [02:45:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.88, 8.67, 9.97 [02:46:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.16, 4.49, 3.98 [02:48:48] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [02:56:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.82, 3.25, 3.06 [02:57:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.38, 11.53, 10.36 [02:58:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 3.26, 3.19, 3.06 [02:59:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.59, 10.23, 10.02 [03:01:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.10, 5.36, 3.32 [03:01:21] RECOVERY - db121 Backups SQL on db121 is OK: FILE_AGE OK: /var/log/sql-backup.log is 79 seconds old and 275 bytes [03:01:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.39, 9.12, 9.63 [03:03:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 6.35, 5.79, 3.74 [03:09:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.19, 10.89, 10.25 [03:09:37] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.09, 3.55, 3.25 [03:11:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.33, 10.01, 10.01 [03:11:31] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.44, 3.66, 3.34 [03:11:49] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:13:25] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.47, 4.35, 3.63 [03:15:19] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.63, 3.79, 3.50 [03:15:47] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.461 second response time [03:17:13] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 2.25, 3.38, 3.39 [03:21:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.92, 9.89, 11.66 [03:22:55] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.91, 4.30, 3.72 [03:23:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.31, 11.23, 11.93 [03:30:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.19, 3.96, 3.92 [03:31:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.78, 11.34, 11.93 [03:32:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.55, 3.07, 3.90 [03:35:07] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.477530241 secs [03:36:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 3.02, 2.73, 3.33 [03:37:07] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.0002438724041 secs [03:38:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.00, 2.19, 3.22 [03:39:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.39, 10.81, 11.25 [03:45:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.17, 10.65, 11.09 [03:53:12] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.05, 4.44, 3.81 [03:53:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.96, 11.34, 11.09 [03:54:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.97, 3.22, 3.04 [03:55:48] PROBLEM - elsterclopedie.marithaime.nl - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - elsterclopedie.marithaime.nl All nameservers failed to answer the query. [03:56:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 3.14, 3.17, 3.04 [03:59:07] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.53, 3.93, 3.90 [04:03:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 6.95, 10.25, 11.01 [04:10:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.58, 3.25, 3.09 [04:11:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 15.70, 12.00, 11.32 [04:11:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.70, 10.65, 9.37 [04:12:28] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 8.70, 7.89, 6.07 [04:12:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.22, 3.59, 3.23 [04:12:57] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.12, 2.68, 3.27 [04:13:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.66, 9.38, 9.06 [04:14:27] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 7.57, 7.71, 6.21 [04:14:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.91, 3.02, 3.07 [04:15:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.48, 11.37, 11.26 [04:17:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.95, 12.32, 11.61 [04:18:26] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 5.51, 6.29, 5.96 [04:19:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.31, 5.28, 2.89 [04:21:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.43, 3.72, 2.60 [04:21:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.58, 11.72, 10.00 [04:22:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.88, 4.43, 3.53 [04:25:07] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1909710765 secs [04:25:23] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 7.41, 7.43, 6.63 [04:25:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.15, 11.85, 10.58 [04:25:33] RECOVERY - elsterclopedie.marithaime.nl - reverse DNS on sslhost is OK: SSL OK - elsterclopedie.marithaime.nl reverse DNS resolves to cp23.miraheze.org - CNAME OK [04:26:43] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.44, 4.31, 3.70 [04:27:22] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 9.04, 8.04, 6.95 [04:28:42] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.24, 3.84, 3.60 [04:30:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 1.62, 3.50, 3.58 [04:31:21] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 5.43, 7.16, 6.90 [04:31:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.08, 10.01, 10.13 [04:34:37] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.25, 4.40, 3.81 [04:36:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.81, 2.98, 3.38 [04:37:18] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 5.39, 6.52, 6.78 [04:37:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.57, 10.89, 11.91 [04:38:34] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.97, 3.85, 3.73 [04:40:32] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.88, 4.52, 3.98 [04:41:16] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 10.19, 8.14, 7.36 [04:43:16] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 6.70, 7.63, 7.28 [04:43:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.50, 11.86, 11.96 [04:44:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 3.86, 4.54, 3.19 [04:44:58] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:45:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.01, 10.01, 11.27 [04:46:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.41, 3.39, 2.93 [04:46:28] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.89, 3.97, 3.95 [04:47:07] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09465527534 secs [04:47:57] RECOVERY - cp33 APT on cp33 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [04:48:53] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [04:49:14] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 9.98, 8.28, 7.62 [04:49:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 17.39, 12.43, 11.82 [04:49:58] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.nj.cn.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [04:50:25] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.22, 3.94, 3.93 [04:51:13] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 5.63, 7.34, 7.36 [04:51:19] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.09, 4.20, 3.73 [04:53:13] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.43, 3.84, 3.64 [04:53:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.87, 11.72, 11.74 [04:57:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.41, 11.76, 11.71 [04:58:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.89, 3.52, 4.00 [04:59:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.28, 10.12, 11.12 [05:00:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.49, 2.64, 3.20 [05:01:10] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 4.57, 5.53, 6.54 [05:05:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.35, 8.40, 9.98 [05:06:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.43, 3.15, 3.23 [05:08:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.43, 3.68, 3.79 [05:10:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.88, 3.71, 3.42 [05:12:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.24, 3.83, 3.84 [05:13:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 17.39, 13.57, 11.58 [05:13:57] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.54, 10.87, 9.55 [05:14:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 2.79, 3.63, 3.49 [05:15:54] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.23, 10.65, 9.62 [05:16:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.69, 2.77, 3.40 [05:16:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 3.36, 3.32, 3.38 [05:17:45] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [05:17:52] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.96, 10.16, 9.58 [05:19:44] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [05:23:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.31, 10.66, 11.35 [05:24:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.46, 3.31, 3.29 [05:25:00] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 7.23, 7.28, 6.61 [05:28:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.86, 3.74, 3.51 [05:28:41] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.26, 2.84, 1.71 [05:30:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.68, 3.12, 3.32 [05:30:42] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.64, 2.31, 1.65 [05:34:56] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 5.65, 6.56, 6.65 [05:36:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.95, 11.33, 10.14 [05:37:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 18.70, 13.71, 12.12 [05:38:27] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.45, 3.71, 3.32 [05:38:55] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 10.54, 7.72, 7.03 [05:40:22] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 2.91, 3.35, 3.23 [05:40:24] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.46, 11.08, 10.44 [05:40:36] PROBLEM - wiki.otir.nl - reverse DNS on sslhost is WARNING: Timeout: The DNS operation timed out after 5.406635522842407 seconds [05:40:54] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 7.31, 7.62, 7.09 [05:42:21] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.41, 10.01, 10.11 [05:42:53] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 4.45, 6.40, 6.71 [05:45:03] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:46:51] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 6.83, 6.88, 6.86 [05:46:52] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [05:47:05] RECOVERY - cp33 APT on cp33 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [05:47:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.69, 10.75, 11.60 [05:47:58] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.93, 3.35, 3.17 [05:48:51] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 5.73, 6.39, 6.68 [05:49:29] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.nj.cn.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [05:49:52] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.02, 3.77, 3.34 [05:51:18] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.37, 4.71, 3.57 [05:51:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 16.13, 12.22, 11.88 [05:52:07] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.40, 10.88, 10.24 [05:52:49] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 7.22, 7.94, 7.34 [05:54:04] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.45, 9.78, 9.93 [05:55:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.02, 10.76, 11.48 [05:55:34] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.02, 3.84, 3.61 [05:56:47] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 10.28, 8.55, 7.67 [05:57:28] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.39, 3.95, 3.67 [05:59:22] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 2.51, 3.41, 3.50 [06:01:12] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.46, 3.73, 3.69 [06:02:45] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 7.49, 7.74, 7.58 [06:05:09] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.47, 2.85, 3.34 [06:06:58] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 2.48, 2.98, 3.31 [06:07:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.16, 8.75, 10.05 [06:09:55] RECOVERY - wiki.otir.nl - reverse DNS on sslhost is OK: SSL OK - wiki.otir.nl reverse DNS resolves to cp23.miraheze.org - CNAME OK [06:19:14] RECOVERY - wiki.nj.cn.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.nj.cn.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [06:20:39] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 4.95, 6.12, 6.68 [06:21:54] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.71, 3.48, 3.26 [06:23:52] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.28, 4.51, 3.66 [06:25:50] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.30, 3.96, 3.56 [06:27:33] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 9.79, 7.58, 3.76 [06:27:49] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.14, 4.07, 3.62 [06:29:47] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.91, 3.89, 3.64 [06:31:31] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 1.15, 3.90, 3.09 [06:31:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.17, 3.12, 3.37 [06:33:30] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.97, 2.98, 2.85 [06:49:14] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.209364444 secs [06:51:14] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.09310010076 secs [06:52:27] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 7.13, 6.35, 5.66 [06:54:26] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 4.37, 5.75, 5.53 [06:57:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.06, 3.87, 3.20 [06:59:21] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.39, 3.32, 3.08 [07:03:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 3.13, 5.16, 4.01 [07:05:21] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.23, 3.81, 3.65 [07:09:18] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.08, 3.45, 3.46 [07:11:17] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.73, 3.21, 3.38 [07:24:04] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.91, 4.68, 3.72 [07:26:03] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.94, 3.95, 3.55 [07:31:59] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.17, 2.54, 3.10 [07:39:07] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1331394315 secs [07:55:07] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09635406733 secs [07:55:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.64, 3.83, 2.90 [07:57:37] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.70, 3.42, 2.87 [07:59:36] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.57, 3.94, 3.13 [08:01:34] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.73, 3.80, 3.17 [08:03:33] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.33, 2.90, 2.92 [08:06:25] PROBLEM - test131 MediaWiki Rendering on test131 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:08:20] RECOVERY - test131 MediaWiki Rendering on test131 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.351 second response time [08:22:53] PROBLEM - cp33 SSH on cp33 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:23:12] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 74.208.203.152/cpweb, 2607:f1c0:1800:26f::1/cpweb [08:23:39] PROBLEM - cp33 PowerDNS Recursor on cp33 is CRITICAL: CRITICAL - Plugin timed out while executing system call [08:23:44] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 37.96, 21.52, 10.00 [08:25:09] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [08:25:41] PROBLEM - cp33 Puppet on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [08:26:55] RECOVERY - cp33 SSH on cp33 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [08:27:10] RECOVERY - cp33 APT on cp33 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [08:27:12] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [08:27:38] RECOVERY - cp33 Puppet on cp33 is OK: OK: Puppet is currently enabled, last run 25 minutes ago with 0 failures [08:27:54] RECOVERY - cp33 PowerDNS Recursor on cp33 is OK: DNS OK: 9.571 seconds response time. miraheze.org returns 108.175.15.182,2607:f1c0:1800:26f::1,2607:f1c0:1800:8100::1,74.208.203.152 [08:49:04] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:53:26] PROBLEM - cloud14 Puppet on cloud14 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [09:01:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 0.91, 2.16, 3.67 [09:03:21] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.23, 1.76, 3.34 [09:09:16] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.16, 3.83, 3.69 [09:11:14] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.73, 2.98, 3.40 [09:12:51] PROBLEM - gs.sidem.wiki - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query sidem.wiki. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [09:14:44] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1266077757 secs [09:17:56] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [09:21:26] RECOVERY - cloud14 Puppet on cloud14 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:23:07] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1552859545 secs [09:32:44] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.08869645 secs [09:37:07] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.07638832927 secs [09:42:41] RECOVERY - gs.sidem.wiki - reverse DNS on sslhost is OK: SSL OK - gs.sidem.wiki reverse DNS resolves to cp23.miraheze.org - CNAME OK [09:43:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.81, 10.40, 7.60 [09:45:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 5.68, 8.47, 7.23 [09:56:35] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.97, 4.36, 2.95 [09:58:33] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.96, 3.22, 2.70 [10:44:05] PROBLEM - www.obeymewiki.com - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query obeymewiki.com. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [10:49:07] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.2143098712 secs [11:01:07] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.01444897056 secs [11:05:48] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.06, 3.68, 2.51 [11:11:44] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.38, 3.75, 3.02 [11:13:39] RECOVERY - www.obeymewiki.com - reverse DNS on sslhost is OK: SSL OK - www.obeymewiki.com reverse DNS resolves to cp22.miraheze.org - NS RECORDS OK [11:13:42] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.36, 2.95, 2.82 [11:23:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.78, 10.53, 8.86 [11:25:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.33, 9.47, 8.66 [12:11:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 19.33, 12.61, 9.68 [12:11:48] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.53, 9.95, 8.08 [12:13:46] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.60, 9.07, 7.98 [12:15:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 6.61, 10.52, 9.64 [12:16:01] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:17:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.77, 9.71, 9.45 [12:18:02] RECOVERY - cp33 APT on cp33 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [12:21:10] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [12:27:42] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.79, 6.30, 3.63 [12:29:07] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:31:01] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.84, 4.71, 2.50 [12:33:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.15, 6.33, 3.38 [12:33:37] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.57, 3.54, 3.25 [12:34:08] RECOVERY - cp33 APT on cp33 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [12:35:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.54, 6.18, 3.70 [12:35:36] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.47, 2.81, 3.01 [12:39:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 18.10, 13.50, 10.67 [12:39:51] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 19.10, 13.34, 10.83 [12:42:18] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:43:07] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.102180928 secs [12:44:18] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 5.882 second response time [12:47:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.99, 11.13, 10.99 [12:50:02] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [12:50:07] PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.1863345802 secs [12:53:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.03, 10.63, 11.66 [12:55:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.90, 4.15, 2.99 [12:55:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.89, 9.11, 10.10 [13:01:07] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09956637025 secs [13:03:17] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.35, 3.12, 3.16 [13:04:07] RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset 0.09139612317 secs [13:05:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.33, 9.97, 10.44 [13:08:13] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 19.73, 13.84, 11.36 [13:12:09] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.47, 10.85, 10.73 [13:15:19] PROBLEM - gs.sidem.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - gs.sidem.wiki All nameservers failed to answer the query. [13:17:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.84, 10.74, 11.25 [13:18:01] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.34, 9.27, 10.14 [13:19:41] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:21:44] RECOVERY - cp33 APT on cp33 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [13:25:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.01, 10.71, 10.81 [13:27:47] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.09, 11.87, 10.78 [13:31:42] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.39, 11.91, 11.06 [13:33:39] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 16.08, 13.34, 11.68 [13:35:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.12, 5.65, 2.96 [13:37:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.71, 5.43, 3.23 [13:37:35] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 6.99, 10.95, 11.17 [13:43:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.44, 10.83, 11.73 [13:43:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.45, 8.76, 10.06 [13:45:08] PROBLEM - gs.sidem.wiki - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query sidem.wiki. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [13:56:18] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.21, 11.08, 10.49 [13:58:29] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.98, 3.95, 2.89 [14:00:27] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.17, 3.07, 2.68 [14:02:11] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.90, 9.61, 10.06 [14:05:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.78, 11.39, 11.16 [14:06:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.07, 11.19, 10.65 [14:07:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.64, 5.28, 2.96 [14:07:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.80, 11.49, 11.23 [14:09:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.50, 4.27, 2.89 [14:11:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 17.32, 13.08, 11.80 [14:13:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.64, 11.88, 11.57 [14:13:54] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.91, 9.43, 10.01 [14:15:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.39, 7.20, 4.42 [14:15:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.79, 12.19, 11.68 [14:17:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.20, 5.22, 4.03 [14:17:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.84, 11.75, 11.57 [14:19:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.23, 11.80, 11.57 [14:21:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.65, 12.00, 11.70 [14:23:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.77, 11.63, 11.55 [14:24:24] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.72, 9.08, 5.33 [14:25:19]