[00:03:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.13, 10.15, 10.37 [00:05:14] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset -0.01057472825 secs [00:05:25] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.10, 3.83, 3.87 [00:07:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.38, 9.31, 10.04 [00:11:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.91, 6.14, 3.75 [00:13:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.75, 6.37, 4.16 [00:13:19] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.53, 2.81, 3.33 [00:14:48] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 4.91, 3.36, 1.92 [00:16:48] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 2.00, 2.69, 1.84 [00:17:14] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.39, 3.56, 3.50 [00:19:13] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.65, 3.44, 3.46 [00:21:12] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.58, 4.06, 3.68 [00:27:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.35, 7.53, 5.46 [00:30:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.85, 3.15, 3.92 [00:34:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.55, 4.52, 4.23 [00:35:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 4.57, 7.89, 6.80 [00:35:56] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.33, 10.73, 9.88 [00:37:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.49, 7.59, 6.79 [00:37:54] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.94, 10.55, 9.90 [00:38:59] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.71, 3.38, 3.94 [00:39:52] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.96, 9.42, 9.56 [00:41:51] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.96, 11.55, 9.86 [00:43:46] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.89, 10.23, 9.59 [00:45:40] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.29, 9.45, 9.37 [00:48:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.10, 2.52, 3.25 [00:52:28] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.62, 3.42, 3.92 [00:54:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.15, 4.89, 4.05 [00:58:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.16, 4.02, 3.88 [00:59:42] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 6.86, 4.27, 2.16 [00:59:59] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.40, 10.15, 9.57 [01:00:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.82, 3.41, 3.67 [01:01:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.00, 7.33, 7.86 [01:01:29] PROBLEM - cp23 APT on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:01:53] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.67, 9.93, 9.59 [01:03:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 12.74, 9.39, 8.54 [01:03:31] RECOVERY - cp23 APT on cp23 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [01:03:33] PROBLEM - es141 Current Load on es141 is CRITICAL: CRITICAL - load average: 5.47, 4.08, 3.26 [01:04:40] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.82, 3.40, 3.90 [01:05:29] PROBLEM - es141 Current Load on es141 is WARNING: WARNING - load average: 3.97, 3.98, 3.32 [01:05:40] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.69, 2.89, 2.33 [01:06:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.96, 3.78, 3.66 [01:07:37] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.40, 10.17, 9.75 [01:09:22] RECOVERY - es141 Current Load on es141 is OK: OK - load average: 1.40, 2.76, 2.99 [01:09:32] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.37, 10.65, 9.95 [01:10:28] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.54, 3.99, 3.82 [01:11:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 1.56, 5.83, 7.58 [01:12:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.48, 4.87, 4.18 [01:12:33] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.93, 2.58, 3.36 [01:13:06] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.83, 10.37, 9.72 [01:13:20] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.80, 11.44, 10.51 [01:15:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 0.53, 3.21, 6.13 [01:16:29] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.56, 5.09, 4.15 [01:18:59] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.79, 10.13, 9.88 [01:20:26] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.82, 3.72, 3.82 [01:20:49] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [01:21:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.38, 9.74, 10.11 [01:22:26] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.65, 4.18, 3.96 [01:22:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.06, 3.25, 3.83 [01:24:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.78, 4.11, 4.07 [01:25:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.76, 10.16, 10.21 [01:27:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.53, 11.02, 10.50 [01:29:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.82, 10.67, 10.45 [01:30:39] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.87, 11.65, 10.58 [01:31:43] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 30.50, 13.69, 6.00 [01:33:14] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset -0.1074703336 secs [01:34:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.77, 3.32, 3.92 [01:34:34] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.13, 11.66, 10.86 [01:35:14] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset -0.08916395903 secs [01:35:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.84, 9.65, 10.12 [01:38:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.29, 2.23, 3.33 [01:40:27] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.41, 9.04, 9.92 [01:41:39] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 0.74, 2.81, 3.89 [01:44:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.07, 3.94, 3.65 [01:45:37] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.65, 1.63, 3.15 [01:46:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.63, 3.44, 3.51 [01:50:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.94, 3.81, 3.61 [01:52:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.10, 3.08, 3.85 [01:54:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.80, 3.29, 3.81 [01:56:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.10, 2.84, 3.58 [01:56:39] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:58:48] RECOVERY - cp33 APT on cp33 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [02:02:28] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 13.48, 5.27, 4.16 [02:05:51] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.94, 10.47, 9.80 [02:07:48] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.34, 10.06, 9.74 [02:15:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.41, 6.03, 3.39 [02:16:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.69, 2.92, 3.94 [02:17:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.36, 4.19, 3.04 [02:18:09] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [02:24:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.39, 4.23, 3.97 [02:26:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.85, 3.23, 3.93 [02:28:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.50, 3.76, 4.02 [02:30:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.41, 3.49, 3.88 [02:30:44] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1646915972 secs [02:33:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.91, 11.26, 9.59 [02:34:16] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 15.91, 11.98, 9.90 [02:34:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.28, 4.13, 4.00 [02:35:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.88, 10.40, 9.46 [02:36:13] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.49, 11.28, 9.90 [02:37:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.87, 9.78, 9.34 [02:38:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.46, 3.92, 3.99 [02:40:08] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.65, 10.10, 9.76 [02:42:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.91, 4.04, 3.95 [02:43:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.37, 10.37, 9.67 [02:43:39] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:44:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.98, 3.50, 3.76 [02:44:44] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.09779259562 secs [02:45:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.06, 9.63, 9.47 [02:45:42] RECOVERY - cp33 APT on cp33 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [02:46:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.39, 3.26, 3.94 [02:47:56] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 16.52, 12.25, 10.58 [02:49:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 16.35, 13.42, 11.07 [02:49:54] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.45, 11.73, 10.65 [02:51:52] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.78, 12.45, 11.00 [02:52:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.48, 3.12, 3.60 [02:52:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.80, 2.85, 3.37 [02:54:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.64, 2.86, 3.45 [02:56:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.94, 3.71, 3.69 [02:56:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.14, 3.38, 3.49 [02:58:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.88, 3.94, 3.68 [02:59:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.19, 11.43, 11.37 [02:59:42] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.92, 11.47, 11.30 [03:00:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.31, 3.67, 3.70 [03:00:59] RECOVERY - db131 Backups SQL on db131 is OK: FILE_AGE OK: /var/log/sql-backup.log is 57 seconds old and 275 bytes [03:02:35] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 27.56, 11.13, 6.32 [03:03:07] PROBLEM - db112 Backups SQL on db112 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864185 seconds old and 363 bytes [03:03:46] PROBLEM - db112 Backups SQL on db112 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864224 seconds old and 363 bytes [03:11:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 16.75, 11.98, 11.03 [03:13:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 10.00, 9.70, 10.20 [03:15:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.37, 11.14, 10.86 [03:21:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.51, 11.25, 10.91 [03:21:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.58, 9.84, 9.96 [03:23:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.17, 9.94, 10.47 [03:25:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.11, 9.46, 9.83 [03:27:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.82, 9.37, 10.13 [03:29:07] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.3322731853 secs [03:30:54] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 11.92, 5.92, 2.92 [03:34:52] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.64, 3.06, 2.44 [03:35:07] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.06753394008 secs [03:35:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.44, 10.49, 9.88 [03:37:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.07, 10.51, 9.97 [03:37:45] PROBLEM - citizenwiki.cn - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for citizenwiki.cn could not be found [03:40:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.07, 2.69, 3.91 [03:41:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.00, 9.79, 9.81 [03:42:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.90, 3.30, 3.97 [03:44:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.51, 3.39, 3.93 [03:48:21] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.61, 10.20, 9.99 [03:50:18] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.28, 11.24, 10.41 [03:50:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.08, 4.00, 3.96 [03:51:07] PROBLEM - citizenwiki.cn - LetsEncrypt on sslhost is CRITICAL: connect to address citizenwiki.cn and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [03:52:16] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.60, 9.58, 9.89 [03:52:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.76, 2.58, 3.73 [03:56:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.96, 2.46, 3.40 [04:01:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.58, 5.86, 3.38 [04:03:15] PROBLEM - es141 Current Load on es141 is CRITICAL: CRITICAL - load average: 4.42, 2.54, 1.33 [04:04:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.61, 2.89, 3.78 [04:05:15] PROBLEM - es141 Current Load on es141 is WARNING: WARNING - load average: 3.89, 3.03, 1.67 [04:07:15] RECOVERY - es141 Current Load on es141 is OK: OK - load average: 2.73, 2.85, 1.77 [04:11:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.47, 5.90, 5.26 [04:11:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.88, 10.27, 9.04 [04:12:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.15, 2.59, 3.30 [04:14:44] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1267847717 secs [04:15:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.11, 10.04, 9.26 [04:20:43] RECOVERY - citizenwiki.cn - LetsEncrypt on sslhost is OK: OK - Certificate 'citizenwiki.cn' will expire on Tue 30 May 2023 18:32:14 GMT +0000. [04:24:24] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 14.25, 6.52, 4.39 [04:31:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.07, 10.06, 9.60 [04:32:02] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.93, 3.37, 2.90 [04:33:59] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.43, 4.66, 3.40 [04:35:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.41, 10.94, 9.98 [04:35:56] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.68, 3.82, 3.25 [04:37:13] RECOVERY - citizenwiki.cn - reverse DNS on sslhost is OK: SSL OK - citizenwiki.cn reverse DNS resolves to cp22.miraheze.org - CNAME OK [04:37:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.47, 9.82, 9.68 [04:37:52] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 3.71, 4.15, 3.45 [04:39:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.80, 9.95, 9.91 [04:39:49] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.60, 3.72, 3.38 [04:41:46] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.01, 3.17, 3.22 [04:42:44] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.002646744251 secs [04:50:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.14, 2.58, 3.66 [04:51:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.43, 10.50, 10.00 [04:52:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.50, 3.80, 3.97 [04:53:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.97, 10.36, 10.12 [04:55:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.12, 9.58, 9.76 [04:59:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.85, 9.71, 9.94 [05:00:41] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 6.89, 3.82, 1.99 [05:02:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.93, 3.61, 3.38 [05:02:41] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 2.87, 3.25, 2.01 [05:04:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.03, 4.01, 3.54 [05:07:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.84, 10.79, 10.32 [05:10:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.60, 3.26, 3.90 [05:11:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.99, 11.43, 10.42 [05:12:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.59, 3.85, 3.91 [05:13:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.69, 11.01, 10.38 [05:16:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.07, 2.96, 3.47 [05:17:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.95, 9.99, 10.15 [05:18:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.73, 2.41, 3.20 [05:18:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.85, 2.71, 3.38 [05:19:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.01, 11.29, 10.77 [05:21:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.04, 10.14, 10.22 [05:22:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.62, 3.42, 3.46 [05:23:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.27, 11.19, 10.89 [05:24:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.41, 3.81, 3.60 [05:25:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.96, 9.77, 10.14 [05:27:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.22, 11.50, 11.04 [05:28:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.85, 3.87, 3.73 [05:29:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.07, 11.23, 11.00 [05:30:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 3.22, 4.14, 3.68 [05:32:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.31, 3.01, 3.40 [05:32:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.27, 3.55, 3.52 [05:34:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.35, 4.20, 3.75 [05:36:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.84, 3.80, 3.64 [05:40:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.09, 2.95, 3.33 [05:41:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.64, 11.07, 10.84 [05:43:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.85, 10.09, 10.50 [05:45:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.53, 9.21, 10.13 [05:55:17] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.15, 4.66, 3.58 [05:59:14] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.79, 3.96, 3.55 [06:01:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.85, 5.51, 2.84 [06:01:12] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.57, 3.14, 3.30 [06:03:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.44, 4.55, 2.81 [06:09:12] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.76, 4.44, 3.51 [06:13:06] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.50, 3.80, 3.46 [06:15:02] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.61, 3.30, 3.31 [06:15:58] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.90, 3.81, 3.38 [06:20:55] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.05, 4.75, 3.89 [06:22:52] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.52, 3.55, 3.55 [06:26:45] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.46, 4.29, 3.80 [06:27:38] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:28:42] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.20, 3.75, 3.66 [06:28:48] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 3.61, 6.29, 3.43 [06:32:39] RECOVERY - cp33 APT on cp33 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [06:32:48] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 1.30, 3.64, 2.98 [06:34:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 17.93, 7.02, 4.70 [06:34:48] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.67, 2.63, 2.69 [06:35:44] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.18, 2.93, 3.82 [06:41:39] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.23, 2.28, 3.26 [06:56:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.03, 2.81, 3.82 [07:04:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.87, 2.51, 3.30 [07:23:20] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.41, 3.61, 3.16 [07:24:42] PROBLEM - cp32 APT on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [07:26:43] RECOVERY - cp32 APT on cp32 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [07:27:13] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.29, 3.25, 3.13 [07:35:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.49, 10.62, 9.17 [07:37:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.84, 11.31, 9.43 [07:37:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.59, 10.07, 9.15 [07:38:57] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.11, 3.92, 3.34 [07:39:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.04, 10.19, 9.26 [07:42:50] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.95, 3.48, 3.38 [07:44:47] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.21, 2.70, 3.10 [07:44:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.42, 3.66, 2.48 [07:45:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 16.72, 14.40, 11.29 [07:46:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 2.00, 3.10, 2.43 [07:49:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.74, 10.52, 10.40 [07:51:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.53, 9.62, 10.09 [07:52:37] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.22, 4.10, 3.38 [07:54:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.92, 3.97, 3.42 [07:56:30] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.71, 3.31, 3.23 [08:16:11] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.14, 10.37, 8.84 [08:18:08] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.34, 8.89, 8.49 [08:26:24] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.42, 4.59, 3.28 [08:34:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.18, 3.84, 3.62 [08:36:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.14, 2.87, 3.29 [08:38:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.57, 3.69, 2.65 [08:40:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.99, 3.26, 2.61 [08:58:18] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.37, 10.78, 9.13 [09:00:16] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.71, 9.58, 8.89 [09:04:08] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 16.13, 11.98, 9.91 [09:06:06] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.58, 10.33, 9.57 [09:08:04] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.60, 9.12, 9.22 [09:19:32] PROBLEM - vise.dayid.org - reverse DNS on sslhost is WARNING: Timeout: The DNS operation timed out after 5.4041876792907715 seconds [09:26:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.26, 4.22, 2.87 [09:28:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.67, 3.39, 2.72 [09:28:41] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.71, 2.40, 1.58 [09:30:41] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.60, 2.04, 1.55 [09:31:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.21, 10.54, 9.23 [09:33:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 5.91, 8.83, 8.76 [09:41:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 19.92, 12.91, 10.14 [09:42:18] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.29, 10.04, 8.78 [09:42:18] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:42:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 6.07, 3.83, 2.68 [09:44:13] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.729 second response time [09:44:16] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.38, 9.52, 8.77 [09:44:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 3.03, 3.40, 2.66 [09:47:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.29, 11.19, 10.59 [09:48:51] RECOVERY - vise.dayid.org - reverse DNS on sslhost is OK: SSL OK - vise.dayid.org reverse DNS resolves to cp22.miraheze.org - CNAME OK [09:49:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.37, 9.62, 10.09 [09:54:02] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.87, 3.91, 3.01 [09:56:19] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [09:57:59] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.29, 3.42, 3.00 [09:58:39] RECOVERY - cp33 APT on cp33 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [09:59:54] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.52, 9.86, 8.89 [09:59:57] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.04, 3.22, 3.00 [10:01:52] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.64, 9.00, 8.69 [10:20:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.14, 3.32, 2.41 [10:21:04] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.25, 3.61, 2.53 [10:21:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.97, 11.70, 9.19 [10:21:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.42, 10.57, 9.12 [10:22:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 2.04, 2.74, 2.30 [10:23:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.30, 11.23, 9.31 [10:23:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.99, 9.21, 8.80 [10:24:57] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.95, 3.64, 2.87 [10:25:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.96, 10.13, 9.14 [10:26:54] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.00, 2.77, 2.64 [10:40:11] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.56, 10.61, 9.55 [10:42:09] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.88, 9.69, 9.34 [10:47:59] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.42, 9.93, 9.49 [10:49:56] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.81, 9.33, 9.33 [10:59:42] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 16.70, 11.06, 9.60 [11:01:40] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.91, 9.89, 9.34 [11:05:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.97, 10.38, 9.58 [11:05:32] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.59, 12.21, 10.50 [11:07:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.50, 9.06, 9.20 [11:07:30] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 6.93, 10.39, 10.04 [11:09:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.87, 9.29, 9.68 [11:11:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 16.75, 12.16, 10.32 [11:13:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.41, 11.05, 10.14 [11:15:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.41, 9.26, 9.59 [11:19:15] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.01, 2.76, 2.36 [11:21:14] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.42, 2.62, 2.36 [11:33:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.23, 11.57, 10.13 [11:35:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.76, 10.18, 9.79 [12:00:42] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 6.22, 4.10, 2.36 [12:02:06] PROBLEM - cp23 APT on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:04:05] RECOVERY - cp23 APT on cp23 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [12:04:41] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 1.94, 3.69, 2.66 [12:06:41] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.93, 2.73, 2.44 [12:10:30] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:12:33] RECOVERY - cp33 APT on cp33 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [12:28:42] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.89, 3.36, 2.37 [12:30:42] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.25, 2.52, 2.17 [12:32:21] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.72, 11.06, 9.71 [12:33:30] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.83, 11.12, 10.05 [12:34:16] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.45, 10.39, 9.63 [12:36:10] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.99, 9.98, 9.57 [12:39:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.56, 9.34, 9.63 [13:07:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.22, 9.84, 8.81 [13:09:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.48, 9.27, 8.73 [13:11:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.46, 6.20, 3.23 [13:13:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.56, 4.92, 3.11 [13:20:10] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [13:25:32] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.83, 3.92, 2.91 [13:28:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.64, 3.68, 2.41 [13:29:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.20, 10.25, 8.81 [13:29:29] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.45, 3.79, 3.15 [13:30:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.94, 3.56, 2.51 [13:31:27] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.11, 3.27, 3.03 [13:31:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.05, 10.92, 9.18 [13:32:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 6.01, 4.45, 2.96 [13:33:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.97, 11.68, 9.70 [13:34:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 1.93, 3.41, 2.76 [13:35:14] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1506089568 secs [13:35:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.56, 10.28, 9.42 [13:36:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.96, 2.83, 2.62 [13:37:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 5.53, 8.66, 8.94 [13:37:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 6.84, 10.72, 10.04 [13:41:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.34, 9.22, 9.60 [13:48:49] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [13:53:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.56, 10.31, 9.75 [13:55:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.01, 9.80, 9.63 [14:07:14] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.06723636389 secs [14:09:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 17.49, 11.94, 9.84 [14:10:13] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.18, 12.96, 11.05 [14:11:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.02, 11.35, 9.88 [14:12:11] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.37, 11.83, 10.86 [14:17:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.20, 9.47, 9.58 [14:18:09] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [14:18:52] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [14:20:01] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.34, 11.52, 10.90 [14:21:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 18.43, 13.04, 10.89 [14:21:43] PROBLEM - wiki.albrusmusic.ru - reverse DNS on sslhost is WARNING: Timeout: The DNS operation timed out after 5.405840635299683 seconds [14:22:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.63, 3.03, 2.62 [14:24:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.50, 2.44, 2.46 [14:25:42] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.03, 3.52, 2.58 [14:27:25] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.53, 3.45, 2.82 [14:27:40] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.39, 3.01, 2.50 [14:27:52] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.00, 11.23, 11.50 [14:29:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.68, 11.13, 11.18 [14:29:22] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.99, 3.40, 2.89 [14:35:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.13, 8.97, 10.08 [14:39:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.07, 10.47, 10.51 [14:41:35] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.89, 8.94, 10.14 [14:43:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.66, 9.39, 10.10 [14:49:01] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.49, 6.50, 4.31 [14:51:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.61, 5.21, 4.12 [14:51:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.06, 11.31, 10.39 [14:51:40] RECOVERY - wiki.albrusmusic.ru - reverse DNS on sslhost is OK: SSL OK - wiki.albrusmusic.ru reverse DNS resolves to cp22.miraheze.org - CNAME OK [14:53:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.09, 10.77, 10.28 [14:54:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.81, 4.33, 3.12 [14:55:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.93, 9.52, 9.87 [15:04:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.21, 3.77, 3.85 [15:10:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.22, 2.51, 3.31 [15:14:08] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.88, 10.77, 9.51 [15:14:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.75, 3.13, 2.49 [15:15:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.41, 11.12, 9.64 [15:16:02] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.89, 10.12, 9.42 [15:16:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 2.50, 3.04, 2.54 [15:23:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 16.93, 13.10, 10.84 [15:23:40] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.68, 12.00, 10.30 [15:25:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.61, 11.71, 10.59 [15:25:35] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.25, 10.57, 9.98 [15:25:50] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.41, 3.64, 3.09 [15:27:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 15.58, 13.22, 11.27 [15:27:29] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 17.39, 13.10, 10.96 [15:28:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.52, 4.01, 3.16 [15:29:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.63, 11.58, 10.93 [15:29:44] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.09, 2.89, 2.93 [15:30:33] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=hotararewiki --no-updates --username-prefix=w /home/macfan/ImportDump/hotararewiki-20230318111339.xml (START) [15:30:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:30:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 2.94, 3.52, 3.08 [15:31:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.35, 10.48, 10.43 [15:31:42] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=hotararewiki --no-updates --username-prefix=w /home/macfan/ImportDump/hotararewiki-20230318111339.xml (END - exit=0) [15:31:43] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=hotararewiki (START) [15:31:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:31:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:32:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.73, 3.04, 2.96 [15:33:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.50, 9.57, 10.10 [15:35:17] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.68, 3.69, 3.30 [15:37:13] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 10.91, 5.56, 2.42 [15:37:16] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.31, 3.03, 3.10 [15:37:28] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.24, 9.19, 9.98 [15:37:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.82, 3.61, 3.25 [15:39:30] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.00, 4.01, 3.45 [15:41:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.15, 6.90, 3.51 [15:41:26] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.51, 3.37, 3.27 [15:43:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 6.41, 7.40, 4.15 [15:45:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.29, 7.69, 4.61 [15:45:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 17.36, 12.24, 10.68 [15:46:02] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 16.53, 13.56, 11.41 [15:46:52] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [15:46:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.61, 3.72, 3.05 [15:47:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.25, 6.59, 4.55 [15:48:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 2.32, 3.24, 2.96 [15:49:28] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.31, 10.57, 10.42 [15:53:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 17.57, 13.32, 11.47 [15:54:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.33, 3.55, 3.16 [15:56:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.95, 4.12, 3.39 [16:00:18] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:02:21] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 8.507 second response time [16:05:14] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1153309941 secs [16:06:46] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.96, 3.98, 3.23 [16:06:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.38, 3.75, 3.74 [16:07:13] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.43, 3.55, 3.99 [16:07:14] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.08450651169 secs [16:08:42] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.32, 3.55, 3.15 [16:08:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.42, 4.38, 3.97 [16:09:13] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 8.62, 5.24, 4.54 [16:10:39] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.31, 4.52, 3.57 [16:14:32] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.23, 3.67, 3.44 [16:16:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.41, 3.78, 3.49 [16:18:28] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.85, 3.52, 3.43 [16:20:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.78, 2.97, 3.25 [16:20:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 1.55, 3.28, 3.84 [16:21:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.25, 5.82, 4.06 [16:23:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.66, 5.89, 4.32 [16:24:17] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=hotararewiki (END - exit=0) [16:24:18] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=hotararewiki --active --update (END - exit=0) [16:24:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:24:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:24:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.16, 3.60, 3.78 [16:26:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.88, 3.67, 3.79 [16:29:13] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 0.73, 2.09, 3.64 [16:31:13] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 0.30, 1.50, 3.23 [16:33:30] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=klausclanwiki --no-updates --username-prefix=starcitizen /home/macfan/ImportDump/klausclanwiki-20230318151628.xml (START) [16:33:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:34:27] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.04, 3.81, 2.91 [16:34:28] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=klausclanwiki --no-updates --username-prefix=starcitizen /home/macfan/ImportDump/klausclanwiki-20230318151628.xml (END - exit=0) [16:34:29] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=klausclanwiki (START) [16:34:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:34:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:34:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.99, 3.90, 3.78 [16:36:26] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.80, 3.69, 2.99 [16:42:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.75, 4.26, 3.35 [16:44:21] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:44:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.77, 3.64, 3.24 [16:46:21] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 6.126 second response time [16:46:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.15, 4.17, 3.49 [16:50:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.10, 3.50, 3.41 [16:51:10] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.35, 3.77, 3.39 [16:52:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.79, 3.72, 3.48 [16:53:07] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.41, 3.30, 3.26 [16:54:08] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=klausclanwiki (END - exit=0) [16:54:09] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=klausclanwiki --active --update (END - exit=0) [16:54:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:54:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:02:54] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.49, 3.50, 3.26 [17:04:50] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.20, 3.64, 3.33 [17:04:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 1.79, 3.11, 3.90 [17:08:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.37, 2.12, 3.31 [17:18:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.81, 4.56, 3.86 [17:20:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.83, 3.22, 3.90 [17:22:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.31, 3.41, 3.87 [17:24:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.67, 3.15, 3.83 [17:32:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.93, 3.40, 3.99 [17:34:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.84, 4.75, 4.12 [17:40:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.58, 2.79, 3.40 [17:41:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [17:43:42] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:47:40] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.447 second response time [17:50:09] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query en.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [17:55:57] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:56:12] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.91, 4.79, 3.81 [18:00:04] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 5.746 second response time [18:04:22] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:04:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.22, 3.34, 3.90 [18:06:05] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.72, 3.88, 3.88 [18:06:22] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 6.217 second response time [18:06:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.87, 3.58, 3.91 [18:09:21] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:11:25] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 6.189 second response time [18:12:01] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.87, 2.72, 3.35 [18:17:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.38, 5.87, 3.18 [18:17:55] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.86, 4.43, 3.74 [18:18:48] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [18:18:57] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:19:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.12, 4.96, 3.17 [18:21:01] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 9.499 second response time [18:22:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.67, 2.57, 3.72 [18:25:14] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:29:26] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 7.173 second response time [18:29:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.20, 3.48, 3.82 [18:30:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.48, 2.72, 3.34 [18:32:27] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:33:43] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.12, 4.02, 3.92 [18:34:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.47, 3.93, 3.67 [18:40:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.30, 3.92, 3.84 [18:41:37] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.37, 3.52, 3.81 [18:44:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.85, 2.75, 3.37 [18:44:36] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 5.819 second response time [18:47:59] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:49:31] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 9.62, 6.51, 4.84 [18:51:23] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.41, 4.11, 3.69 [18:52:08] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 4.225 second response time [18:54:55] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:57:13] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.83, 3.24, 3.50 [18:59:10] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.62, 3.92, 3.72 [19:07:18] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 9.844 second response time [19:09:16] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.03, 3.42, 3.99 [19:11:36] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:19:15] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.4618157446 secs [19:21:07] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.92, 2.88, 3.37 [19:22:30] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.19, 3.11, 3.94 [19:23:14] PROBLEM - cp33 NTP time on cp33 is CRITICAL: NTP CRITICAL: Offset 0.5931141675 secs [19:26:00] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [19:26:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.54, 2.99, 3.65 [19:28:12] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 9.883 second response time [19:28:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.82, 2.98, 3.57 [19:28:57] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 3.94, 4.08, 3.78 [19:29:14] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset -0.002134114504 secs [19:30:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.32, 5.74, 4.58 [19:30:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.85, 3.52, 3.60 [19:36:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.34, 3.99, 3.75 [19:38:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.16, 3.73, 4.00 [19:38:33] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:38:50] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.99, 3.77, 3.71 [19:42:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.12, 4.14, 4.11 [19:44:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.24, 3.59, 3.92 [19:46:44] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.45, 2.98, 3.37 [19:48:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.90, 3.85, 3.91 [19:50:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.66, 3.33, 3.71 [19:53:36] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.92, 4.31, 3.79 [19:55:02] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 7.725 second response time [19:56:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [19:58:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.18, 2.31, 3.12 [19:59:31] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.39, 3.56, 3.68 [20:03:29] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.35, 4.39, 3.96 [20:06:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.93, 3.40, 3.26 [20:08:28] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.33, 4.47, 3.67 [20:13:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.72, 7.10, 4.85 [20:13:11] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:13:21] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.68, 3.30, 3.83 [20:15:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 6.68, 6.80, 5.02 [20:15:20] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.02, 3.39, 3.78 [20:16:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.85, 3.18, 3.47 [20:17:18] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.68, 2.70, 3.48 [20:18:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.73, 2.84, 3.32 [20:21:00] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [20:21:15] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.87, 2.66, 3.32 [20:24:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.66, 3.66, 3.54 [20:25:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.57, 3.32, 3.46 [20:25:31] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 5.457 second response time [20:28:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.12, 3.62, 3.52 [20:29:48] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:30:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.67, 3.34, 3.43 [20:33:05] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.90, 3.99, 3.70 [20:34:28] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.78, 3.72, 3.53 [20:35:03] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.63, 3.53, 3.58 [20:36:56] PROBLEM - cp32 APT on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [20:39:00] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.70, 3.97, 3.73 [20:39:20] RECOVERY - cp32 APT on cp32 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [20:40:58] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.24, 3.47, 3.56 [20:44:55] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.16, 2.83, 3.29 [20:46:22] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 8.207 second response time [20:50:40] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:51:30] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [20:52:50] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.91, 4.06, 3.59 [20:54:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.11, 2.89, 3.85 [21:00:43] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.88, 3.77, 3.87 [21:07:08] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 6.517 second response time [21:12:34] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.42, 3.20, 3.42 [21:13:30] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:16:31] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.37, 2.97, 3.31 [21:18:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.90, 2.97, 3.32 [21:18:53] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [21:19:38] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 6.009 second response time [21:20:27] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.49, 4.55, 3.84 [21:23:41] PROBLEM - cloud13 Puppet on cloud13 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [21:25:23] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.72, 3.75, 3.53 [21:27:52] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:29:50] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 4.325 second response time [21:31:13] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.72, 3.71, 3.66 [21:37:03] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.11, 4.12, 3.80 [21:42:54] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.37, 3.90, 3.91 [21:46:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.87, 3.43, 3.94 [21:47:45] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [21:49:28] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:49:41] RECOVERY - cloud13 Puppet on cloud13 is OK: OK: Puppet is currently enabled, last run 2 seconds ago with 0 failures [21:50:41] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.78, 3.70, 3.67 [21:52:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.67, 3.96, 3.95 [21:53:33] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 7.045 second response time [21:54:34] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.53, 3.16, 3.48 [21:56:31] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.74, 3.02, 3.38 [22:08:28] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.69, 4.01, 3.62 [22:10:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.74, 3.26, 3.40 [22:11:20] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:12:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.19, 2.79, 3.89 [22:15:22] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 4.048 second response time [22:18:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.87, 3.39, 3.74 [22:18:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.03, 3.79, 3.42 [22:19:03] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:20:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.80, 3.86, 3.90 [22:21:08] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 7.810 second response time [22:22:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.66, 3.83, 3.57 [22:23:07] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1011859477 secs [22:24:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.81, 3.80, 3.82 [22:27:07] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09995731711 secs [22:27:38] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:28:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.36, 3.52, 3.74 [22:28:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.15, 2.97, 3.30 [22:29:35] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.13, 2.85, 1.71 [22:31:34] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.84, 2.52, 1.73 [22:32:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.64, 2.76, 3.39 [22:36:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.86, 4.51, 4.01 [22:36:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 14.71, 6.66, 4.50 [22:38:01] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 8.17, 7.88, 4.08 [22:39:04] !log db121: MariaDB [(none)]> set global read_only = 0; [22:39:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:40:00] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 2.93, 5.99, 3.85 [22:42:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.75, 3.69, 3.84 [22:46:15] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 8.770 second response time [22:48:52] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [22:52:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.08, 3.49, 3.59 [23:00:25] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:04:22] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 8.981 second response time [23:05:07] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:07:07] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 2.233 second response time [23:08:40] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:10:37] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 2.444 second response time [23:11:47] PROBLEM - cp32 Varnish Backends on cp32 is CRITICAL: 1 backends are down. mw121 [23:13:22] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:13:47] RECOVERY - cp32 Varnish Backends on cp32 is OK: All 14 backends are healthy [23:17:33] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 6.233 second response time [23:17:45] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [23:30:46] PROBLEM - cp23 APT on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [23:32:45] RECOVERY - cp23 APT on cp23 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [23:32:54] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 2.71, 3.33, 3.98 [23:34:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.35, 3.15, 3.88 [23:36:28] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 11.29, 5.34, 4.54 [23:41:30] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [23:42:54] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.94, 2.49, 3.26 [23:50:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.94, 2.96, 3.83 [23:52:54] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.82, 3.75, 3.41 [23:54:23] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [23:56:18] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:58:23] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 7 = Critical]