[00:01:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.23, 7.80, 5.53 [00:02:45] RECOVERY - mw133 Puppet on mw133 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:03:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.22, 7.25, 5.62 [00:06:45] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1670830846 secs [00:09:03] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 5.01, 7.07, 7.64 [00:09:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.79, 7.56, 6.23 [00:11:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.75, 7.14, 6.24 [00:15:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 4.95, 6.23, 6.11 [00:21:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.05, 7.13, 6.46 [00:22:54] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 5.76, 5.88, 6.63 [00:23:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.54, 7.26, 6.59 [00:23:42] PROBLEM - os131 Current Load on os131 is WARNING: WARNING - load average: 3.62, 3.09, 2.21 [00:25:42] RECOVERY - os131 Current Load on os131 is OK: OK - load average: 2.73, 3.14, 2.35 [00:28:35] RECOVERY - betaheze.org - LetsEncrypt on sslhost is OK: OK - Certificate 'betaheze.org' will expire on Mon 18 Sep 2023 22:56:05 GMT +0000. [00:30:19] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.09874331951 secs [00:33:55] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.17, 5.95, 3.99 [00:35:55] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 6.42, 6.07, 4.27 [00:36:01] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.26, 7.01, 5.76 [00:38:00] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.05, 7.01, 5.91 [00:39:58] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 6.70, 6.70, 5.92 [00:45:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 6.22, 6.68, 6.80 [00:45:45] RECOVERY - wiki.nj.cn.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.nj.cn.eu.org reverse DNS resolves to cp22.miraheze.org - CNAME OK [00:53:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.18, 7.27, 6.99 [00:56:56] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.42, 3.62, 3.26 [00:57:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.19, 7.47, 7.11 [00:59:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.60, 7.43, 7.13 [01:00:03] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.33, 7.07, 5.80 [01:00:51] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.23, 3.51, 3.31 [01:02:53] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 1.92, 2.99, 3.15 [01:03:55] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 6.43, 6.79, 5.99 [01:09:34] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 10.50, 12.96, 6.11 [01:10:43] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 6.09, 4.40, 3.66 [01:13:23] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 1.16, 6.55, 4.98 [01:14:39] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 0.95, 2.98, 3.28 [01:15:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.98, 7.85, 7.37 [01:17:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.42, 7.65, 7.36 [01:19:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.12, 7.94, 7.50 [01:19:37] PROBLEM - swiftobject121 APT on swiftobject121 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [01:20:44] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.07, 2.94, 2.22 [01:21:38] PROBLEM - swiftobject121 APT on swiftobject121 is CRITICAL: APT CRITICAL: 52 packages available for upgrade (7 critical updates). [01:22:40] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.35, 2.64, 2.19 [01:25:34] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.99, 7.28, 6.49 [01:27:35] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 4.51, 6.35, 6.25 [01:29:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.19, 7.86, 7.81 [01:30:21] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 9.16, 6.30, 5.19 [01:34:16] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.21, 7.37, 5.95 [01:37:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.33, 7.86, 7.80 [01:39:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.23, 7.76, 7.78 [01:40:07] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.12, 6.43, 6.04 [01:45:56] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 5.80, 7.26, 6.58 [01:49:50] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 4.86, 6.40, 6.41 [01:56:41] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [01:58:35] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.042 second response time [02:01:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.14, 7.28, 7.20 [02:03:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.91, 7.21, 7.19 [02:13:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 5.05, 6.11, 6.77 [02:21:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.45, 7.48, 7.06 [02:23:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.37, 7.30, 7.04 [02:27:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.93, 7.76, 7.26 [02:29:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.14, 7.53, 7.24 [02:34:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.45, 6.73, 7.83 [02:35:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.90, 7.64, 7.32 [02:36:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.15, 7.06, 7.80 [02:37:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.99, 7.47, 7.31 [02:38:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.59, 7.42, 7.86 [02:40:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.24, 7.75, 7.92 [02:42:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.86, 7.71, 7.88 [02:43:45] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query en.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [02:44:16] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1116527021 secs [02:48:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.03, 7.63, 7.73 [02:48:47] PROBLEM - swiftobject121 APT on swiftobject121 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [02:49:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 9.21, 7.69, 7.29 [02:50:48] PROBLEM - swiftobject121 APT on swiftobject121 is CRITICAL: APT CRITICAL: 52 packages available for upgrade (7 critical updates). [02:53:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.65, 7.68, 7.38 [02:58:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.34, 7.90, 7.99 [03:00:40] RECOVERY - db142 Backups SQL on db142 is OK: FILE_AGE OK: /var/log/sql-backup.log is 38 seconds old and 275 bytes [03:03:03] PROBLEM - mon141 Backups Grafana on mon141 is WARNING: FILE_AGE WARNING: /var/log/grafana-backup.log is 864148 seconds old and 118 bytes [03:03:28] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.66, 3.48, 2.45 [03:04:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.18, 7.52, 7.75 [03:05:28] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.18, 3.44, 2.57 [03:07:28] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.57, 2.78, 2.44 [03:08:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.77, 7.49, 7.73 [03:10:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.32, 7.69, 7.76 [03:11:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 5.49, 6.21, 6.66 [03:12:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.89, 7.38, 7.64 [03:18:15] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.09787291288 secs [03:19:13] PROBLEM - swiftobject121 APT on swiftobject121 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [03:20:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.03, 7.80, 7.60 [03:21:14] PROBLEM - swiftobject121 APT on swiftobject121 is CRITICAL: APT CRITICAL: 52 packages available for upgrade (7 critical updates). [03:21:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 9.81, 8.23, 7.26 [03:22:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.43, 7.61, 7.55 [03:24:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.34, 8.06, 7.71 [03:27:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.71, 7.72, 7.39 [03:29:22] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.52, 7.08, 6.14 [03:29:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.17, 7.98, 7.53 [03:31:21] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.52, 7.25, 6.32 [03:31:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.60, 7.67, 7.46 [03:33:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 9.98, 8.51, 7.79 [03:34:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.04, 6.69, 6.20 [03:35:21] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.87, 6.77, 6.37 [03:36:27] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 5.20, 6.23, 6.10 [03:37:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.06, 7.46, 7.55 [03:40:30] PROBLEM - cp32 APT on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:42:31] RECOVERY - cp32 APT on cp32 is OK: APT OK: 35 packages available for upgrade (0 critical updates). [03:43:04] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [03:47:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.35, 7.41, 7.40 [03:49:29] PROBLEM - swiftobject121 APT on swiftobject121 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [03:51:30] PROBLEM - swiftobject121 APT on swiftobject121 is CRITICAL: APT CRITICAL: 52 packages available for upgrade (7 critical updates). [03:53:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.58, 7.62, 7.60 [03:56:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.65, 7.26, 7.96 [04:04:56] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [04:06:50] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.034 second response time [04:07:56] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.49, 3.68, 2.69 [04:10:25] PROBLEM - swiftproxy131 HTTP on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [04:11:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.63, 7.21, 7.06 [04:11:56] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.32, 3.00, 2.70 [04:12:25] RECOVERY - swiftproxy131 HTTP on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.037 second response time [04:13:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.63, 6.98, 7.00 [04:17:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.45, 7.62, 7.24 [04:18:05] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.2113012969 secs [04:21:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.42, 7.42, 7.29 [04:22:05] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset -0.01121720672 secs [04:24:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.09, 7.35, 7.34 [04:25:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.33, 7.34, 7.24 [04:26:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.83, 7.62, 7.45 [04:27:21] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.65, 6.92, 5.94 [04:27:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 5.18, 6.64, 7.00 [04:29:21] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.53, 6.34, 5.85 [04:29:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 5.08, 6.04, 6.73 [04:32:37] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 3.69, 5.64, 6.67 [04:33:55] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.17, 5.21, 3.39 [04:34:52] PROBLEM - mw142 MediaWiki Rendering on mw142 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:35:55] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.60, 5.51, 3.74 [04:36:49] RECOVERY - mw142 MediaWiki Rendering on mw142 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.612 second response time [04:38:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.82, 6.12, 5.52 [04:41:21] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.06, 6.76, 6.01 [04:44:23] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 4.53, 5.76, 5.62 [04:44:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.00, 6.83, 6.59 [04:45:21] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.12, 7.68, 6.60 [04:47:21] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.06, 6.76, 6.40 [04:50:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.73, 7.95, 7.13 [04:54:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.02, 7.68, 7.22 [04:55:28] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.66, 3.14, 2.21 [04:57:28] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.34, 2.76, 2.18 [04:57:52] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.08, 6.66, 6.58 [05:00:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.63, 7.79, 7.37 [05:01:39] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.44, 7.45, 6.91 [05:02:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.62, 7.68, 7.37 [05:03:33] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.55, 7.50, 7.00 [05:10:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.85, 7.96, 7.46 [05:11:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.52, 7.75, 7.26 [05:12:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.99, 7.40, 7.32 [05:16:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.83, 7.84, 7.48 [05:18:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.86, 7.11, 7.26 [05:20:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.27, 7.86, 7.52 [05:22:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.35, 7.33, 7.37 [05:23:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.26, 7.94, 7.78 [05:25:55] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 7.81, 8.29, 6.35 [05:27:55] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 5.64, 7.28, 6.21 [05:31:55] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.16, 8.67, 6.97 [05:33:55] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 5.60, 7.63, 6.81 [05:35:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.21, 7.39, 7.46 [05:37:55] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.69, 7.95, 7.06 [05:38:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.38, 7.39, 7.18 [05:41:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.39, 7.88, 7.70 [05:42:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.00, 7.36, 7.22 [05:45:55] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 6.04, 7.99, 7.77 [05:47:55] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.06, 9.14, 8.22 [05:49:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.95, 7.69, 7.55 [05:51:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.72, 7.68, 7.57 [05:51:55] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 4.96, 6.95, 7.54 [05:55:55] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.99, 5.37, 6.75 [06:02:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.53, 8.27, 7.50 [06:07:57] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 3.74, 4.44, 2.82 [06:08:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.06, 7.76, 7.58 [06:09:56] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.62, 3.40, 2.64 [06:12:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.19, 7.91, 7.66 [06:14:02] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.27, 6.66, 6.10 [06:14:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.97, 7.64, 7.60 [06:17:59] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 5.90, 6.48, 6.19 [06:19:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.70, 7.66, 7.33 [06:23:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.69, 7.91, 7.53 [06:25:55] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.05, 6.70, 6.04 [06:27:55] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.13, 7.44, 6.40 [06:28:28] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 3.48, 2.14, 1.27 [06:29:22] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 2.08, 3.49, 2.02 [06:29:55] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.45, 6.75, 6.28 [06:30:23] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.41, 1.77, 1.23 [06:31:22] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.86, 2.52, 1.83 [06:32:37] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.55, 6.07, 6.79 [06:43:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 6.78, 6.47, 6.79 [06:46:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.02, 7.39, 6.99 [06:47:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.50, 7.07, 6.97 [06:48:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.32, 7.00, 6.90 [06:50:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.33, 7.46, 7.08 [06:52:21] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.62, 6.72, 6.12 [06:52:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.82, 7.24, 7.05 [06:53:55] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.70, 5.77, 5.13 [06:54:18] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 6.16, 6.46, 6.09 [06:55:55] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 6.86, 6.15, 5.35 [06:57:55] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.20, 6.97, 5.75 [06:58:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.22, 7.16, 7.02 [06:59:55] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 6.33, 6.70, 5.80 [07:02:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.90, 7.32, 7.11 [07:03:55] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 12.24, 9.00, 6.89 [07:05:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 5.83, 6.20, 6.72 [07:10:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.99, 7.35, 7.13 [07:11:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.83, 7.61, 7.13 [07:11:55] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 5.85, 7.76, 7.34 [07:13:01] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [07:13:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.51, 7.50, 7.14 [07:14:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.92, 7.32, 7.17 [07:14:55] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.036 second response time [07:17:55] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.68, 7.05, 7.10 [07:18:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.86, 7.94, 7.42 [07:19:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.67, 7.53, 7.20 [07:19:55] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.99, 6.23, 6.79 [07:20:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.23, 7.53, 7.33 [07:23:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.69, 7.41, 7.26 [07:29:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 4.77, 6.00, 6.71 [07:30:05] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.1233124733 secs [07:30:37] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.97, 6.05, 6.72 [07:36:05] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.07810139656 secs [07:40:05] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.02, 7.17, 6.91 [07:41:58] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.96, 7.50, 7.07 [07:43:26] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.76, 6.88, 6.85 [07:45:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.50, 8.13, 7.28 [07:47:15] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.81, 7.50, 7.15 [07:49:10] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.12, 7.80, 7.28 [07:51:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.50, 7.68, 7.30 [07:57:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 6.35, 6.32, 6.72 [08:01:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.24, 7.07, 6.98 [08:01:58] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.54, 7.21, 6.00 [08:05:55] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.17, 6.54, 6.13 [08:06:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.78, 7.73, 7.40 [08:07:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.58, 7.36, 7.06 [08:08:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.41, 7.27, 7.27 [08:11:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.90, 7.28, 7.11 [08:16:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.48, 8.06, 7.64 [08:20:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.42, 7.64, 7.61 [08:22:27] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [08:23:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 5.91, 6.32, 6.74 [08:24:21] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.027 second response time [08:30:37] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.01, 5.50, 6.59 [08:33:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.98, 7.39, 6.98 [08:35:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.86, 7.55, 7.09 [08:39:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.21, 7.35, 7.06 [08:41:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.90, 7.49, 7.15 [08:42:59] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [08:44:55] RECOVERY - swiftproxy111 HTTP on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.025 second response time [08:48:03] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.26, 6.91, 6.19 [08:48:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.31, 7.62, 7.15 [08:50:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.94, 8.28, 7.46 [08:51:58] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.85, 6.62, 6.25 [08:52:59] PROBLEM - cloud13 Puppet on cloud13 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [08:54:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.35, 7.72, 7.45 [08:56:16] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1233148873 secs [09:11:21] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.nj.cn.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [09:19:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.12, 7.31, 7.06 [09:20:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.32, 7.75, 7.38 [09:20:58] RECOVERY - cloud13 Puppet on cloud13 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:21:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.07, 7.26, 7.08 [09:22:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.84, 7.69, 7.40 [09:30:04] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.191465348 secs [09:32:12] PROBLEM - bast141 APT on bast141 is CRITICAL: APT CRITICAL: 39 packages available for upgrade (2 critical updates). [09:33:04] PROBLEM - db112 APT on db112 is CRITICAL: APT CRITICAL: 56 packages available for upgrade (2 critical updates). [09:33:29] PROBLEM - db131 APT on db131 is CRITICAL: APT CRITICAL: 56 packages available for upgrade (2 critical updates). [09:34:23] PROBLEM - graylog131 APT on graylog131 is CRITICAL: APT CRITICAL: 48 packages available for upgrade (4 critical updates). [09:36:07] PROBLEM - ldap141 APT on ldap141 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (2 critical updates). [09:36:37] PROBLEM - mw143 APT on mw143 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [09:37:13] PROBLEM - mw134 APT on mw134 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [09:37:55] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 6.83, 5.38, 4.08 [09:39:07] PROBLEM - cp32 APT on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [09:39:28] PROBLEM - os141 APT on os141 is CRITICAL: APT CRITICAL: 44 packages available for upgrade (4 critical updates). [09:39:55] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.84, 4.43, 3.90 [09:40:20] PROBLEM - puppet141 APT on puppet141 is CRITICAL: APT CRITICAL: 8 packages available for upgrade (4 critical updates). [09:40:47] PROBLEM - swiftproxy131 APT on swiftproxy131 is CRITICAL: APT CRITICAL: 48 packages available for upgrade (2 critical updates). [09:40:49] RECOVERY - wiki.nj.cn.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.nj.cn.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [09:41:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.99, 7.34, 7.06 [09:41:37] PROBLEM - cloud10 APT on cloud10 is CRITICAL: APT CRITICAL: 104 packages available for upgrade (2 critical updates). [09:41:45] PROBLEM - cloud12 APT on cloud12 is CRITICAL: APT CRITICAL: 103 packages available for upgrade (2 critical updates). [09:42:39] PROBLEM - bast121 APT on bast121 is CRITICAL: APT CRITICAL: 39 packages available for upgrade (2 critical updates). [09:43:07] PROBLEM - db142 APT on db142 is CRITICAL: APT CRITICAL: 47 packages available for upgrade (2 critical updates). [09:43:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.50, 7.26, 7.06 [09:43:37] PROBLEM - swiftproxy111 APT on swiftproxy111 is CRITICAL: APT CRITICAL: 39 packages available for upgrade (2 critical updates). [09:45:11] PROBLEM - cp22 APT on cp22 is CRITICAL: APT CRITICAL: 37 packages available for upgrade (2 critical updates). [09:46:33] PROBLEM - mem141 APT on mem141 is CRITICAL: APT CRITICAL: 48 packages available for upgrade (2 critical updates). [09:48:11] PROBLEM - os131 APT on os131 is CRITICAL: APT CRITICAL: 44 packages available for upgrade (4 critical updates). [09:48:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.23, 7.29, 7.11 [09:49:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.47, 7.37, 7.10 [09:50:44] PROBLEM - ns2 APT on ns2 is CRITICAL: APT CRITICAL: 39 packages available for upgrade (2 critical updates). [09:51:27] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.98, 7.12, 6.06 [09:51:47] PROBLEM - db121 APT on db121 is CRITICAL: APT CRITICAL: 56 packages available for upgrade (2 critical updates). [09:51:52] PROBLEM - cloud13 APT on cloud13 is CRITICAL: APT CRITICAL: 103 packages available for upgrade (2 critical updates). [09:52:34] PROBLEM - swiftobject122 APT on swiftobject122 is CRITICAL: APT CRITICAL: 48 packages available for upgrade (2 critical updates). [09:52:47] PROBLEM - cloud11 APT on cloud11 is CRITICAL: APT CRITICAL: 57 packages available for upgrade (3 critical updates). [09:53:04] PROBLEM - cloud14 APT on cloud14 is CRITICAL: APT CRITICAL: 113 packages available for upgrade (2 critical updates). [09:53:10] PROBLEM - db101 APT on db101 is CRITICAL: APT CRITICAL: 56 packages available for upgrade (2 critical updates). [09:53:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.99, 7.43, 7.19 [09:53:25] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.37, 6.98, 6.14 [09:54:16] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.09791710973 secs [09:54:20] PROBLEM - cp33 APT on cp33 is CRITICAL: APT CRITICAL: 37 packages available for upgrade (2 critical updates). [09:55:02] PROBLEM - prometheus131 APT on prometheus131 is CRITICAL: APT CRITICAL: 39 packages available for upgrade (2 critical updates). [09:55:22] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 4.29, 5.99, 5.88 [09:56:04] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.05317947268 secs [09:56:32] PROBLEM - mem131 APT on mem131 is CRITICAL: APT CRITICAL: 48 packages available for upgrade (2 critical updates). [09:57:50] PROBLEM - ns1 APT on ns1 is CRITICAL: APT CRITICAL: 38 packages available for upgrade (2 critical updates). [09:58:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.33, 7.92, 7.83 [09:59:04] PROBLEM - cp23 APT on cp23 is CRITICAL: APT CRITICAL: 37 packages available for upgrade (2 critical updates). [10:01:10] PROBLEM - mw133 APT on mw133 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [10:06:14] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.68, 6.63, 6.17 [10:08:12] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 5.75, 6.46, 6.17 [10:18:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.09, 7.27, 7.27 [10:18:44] PROBLEM - swiftobject121 APT on swiftobject121 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [10:19:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 9.86, 8.19, 7.56 [10:20:45] PROBLEM - swiftobject121 APT on swiftobject121 is CRITICAL: APT CRITICAL: 52 packages available for upgrade (7 critical updates). [10:21:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.12, 7.85, 7.52 [10:24:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.56, 7.92, 7.63 [10:24:56] PROBLEM - swiftac111 APT on swiftac111 is CRITICAL: APT CRITICAL: 39 packages available for upgrade (2 critical updates). [10:30:29] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [10:32:23] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.036 second response time [10:32:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.06, 7.73, 7.55 [10:34:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.69, 7.28, 7.41 [10:41:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.57, 7.32, 7.08 [10:43:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.53, 7.29, 7.09 [10:49:06] PROBLEM - swiftobject121 APT on swiftobject121 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [10:49:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.05, 7.27, 7.12 [10:51:07] PROBLEM - swiftobject121 APT on swiftobject121 is CRITICAL: APT CRITICAL: 52 packages available for upgrade (7 critical updates). [10:51:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.94, 7.47, 7.21 [11:01:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 6.55, 6.52, 6.78 [11:04:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.13, 7.46, 7.33 [11:05:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.72, 7.26, 7.04 [11:06:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.31, 7.43, 7.34 [11:11:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 4.94, 6.22, 6.70 [11:12:08] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query cn.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [11:18:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.46, 7.73, 7.41 [11:19:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.62, 7.22, 6.83 [11:21:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.20, 6.83, 6.73 [11:23:21] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.37, 6.79, 5.84 [11:23:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 6.35, 6.62, 6.66 [11:25:21] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 6.50, 6.70, 5.92 [11:26:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.76, 7.79, 7.67 [11:30:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.60, 7.91, 7.72 [11:32:16] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset -0.1032065749 secs [11:32:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.49, 7.79, 7.71 [11:34:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.10, 8.11, 7.85 [11:36:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.54, 7.81, 7.76 [11:37:35] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.14, 6.34, 5.82 [11:38:15] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset -0.09999901056 secs [11:39:34] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 6.44, 6.31, 5.87 [11:41:35] RECOVERY - wiki.nj.cn.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.nj.cn.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [11:46:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.47, 7.63, 7.62 [11:48:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.44, 7.50, 7.58 [11:51:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.83, 6.84, 6.65 [11:55:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 6.57, 6.70, 6.64 [12:00:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.15, 6.85, 7.20 [12:02:10] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [12:02:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.44, 7.11, 7.26 [12:04:03] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.035 second response time [12:05:22] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 6.93, 6.34, 5.80 [12:06:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.70, 8.19, 7.58 [12:07:21] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 6.14, 6.35, 5.88 [12:08:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.87, 7.73, 7.47 [12:12:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.95, 8.42, 7.77 [12:13:21] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [12:15:18] RECOVERY - swiftproxy111 HTTP on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.024 second response time [12:18:05] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.95, 7.04, 6.68 [12:19:39] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [12:19:58] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.52, 7.51, 6.89 [12:21:52] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.48, 7.17, 6.85 [12:25:53] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.17, 6.32, 5.96 [12:27:51] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [12:29:27] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 6.18, 6.64, 6.77 [12:29:47] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 6.27, 6.56, 6.17 [12:29:47] RECOVERY - swiftproxy111 HTTP on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.021 second response time [12:33:10] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.05, 6.24, 5.94 [12:33:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.02, 6.85, 6.82 [12:39:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 5.76, 6.52, 6.73 [12:43:01] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 5.77, 6.47, 6.33 [12:43:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.63, 7.56, 7.08 [12:47:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.45, 7.64, 7.23 [12:49:18] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [12:51:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.64, 8.09, 7.50 [12:59:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 5.77, 7.27, 7.45 [13:03:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.07, 7.30, 7.38 [13:04:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.86, 6.57, 7.81 [13:05:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.27, 7.34, 7.39 [13:12:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.94, 7.69, 7.73 [13:15:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.37, 7.52, 7.35 [13:16:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.61, 7.85, 7.80 [13:17:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.99, 7.76, 7.46 [13:18:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.76, 8.31, 7.96 [13:18:58] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [13:28:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.67, 7.72, 7.98 [13:32:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.73, 8.00, 8.01 [13:35:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.87, 7.56, 7.32 [13:37:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.99, 7.71, 7.41 [13:41:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.10, 7.67, 7.42 [13:43:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.38, 7.64, 7.45 [13:46:04] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1058773994 secs [14:00:59] PROBLEM - wiki.thedev.gq - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.thedev.gq All nameservers failed to answer the query. [14:03:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.07, 7.12, 7.15 [14:05:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.84, 6.95, 7.08 [14:05:25] PROBLEM - swiftproxy131 HTTP on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [14:08:26] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [14:09:55] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 11.35, 6.24, 3.06 [14:10:04] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.0756008327 secs [14:10:20] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.036 second response time [14:12:21] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.23, 2.86, 1.53 [14:13:07] RECOVERY - swiftproxy131 HTTP on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.045 second response time [14:13:55] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.08, 6.62, 4.08 [14:14:21] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 1.55, 2.58, 1.61 [14:14:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.26, 7.48, 7.99 [14:15:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 3.68, 5.50, 6.43 [14:18:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.29, 8.12, 8.09 [14:21:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.73, 6.99, 6.76 [14:23:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.21, 7.27, 6.88 [14:25:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.59, 7.10, 6.88 [14:27:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 5.75, 6.64, 6.74 [14:29:54] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [14:31:54] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.030 second response time [14:32:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.02, 7.49, 7.86 [14:36:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.69, 8.40, 8.11 [14:42:27] PROBLEM - mem141 NTP time on mem141 is WARNING: NTP WARNING: Offset 0.1168345511 secs [14:48:55] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.93, 7.87, 7.20 [14:52:42] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.71, 7.67, 7.30 [14:56:20] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [14:56:30] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.71, 7.59, 7.30 [14:58:14] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.047 second response time [14:58:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.51, 7.57, 7.33 [15:01:21] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.45, 6.72, 6.11 [15:03:21] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 6.47, 6.62, 6.16 [15:09:46] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 6.28, 6.24, 6.80 [15:13:33] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.95, 6.94, 6.95 [15:14:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.75, 7.44, 7.98 [15:18:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.71, 8.18, 8.10 [15:18:59] PROBLEM - swiftobject121 APT on swiftobject121 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [15:19:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 9.54, 8.10, 7.40 [15:21:00] PROBLEM - swiftobject121 APT on swiftobject121 is CRITICAL: APT CRITICAL: 52 packages available for upgrade (7 critical updates). [15:22:30] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/e88e47fa8769 [15:22:33] [02miraheze/mw-config] 07paladox 03e88e47f - Disable file uploads temporarily due to swift outage [15:22:34] [02mw-config] 07paladox created branch 03paladox-patch-2 - 13https://github.com/miraheze/mw-config [15:22:37] [02mw-config] 07paladox opened pull request 03#5256: Disable file uploads temporarily due to swift outage - 13https://github.com/miraheze/mw-config/pull/5256 [15:22:39] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-2 [15:22:42] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/a5309f5384b4...0f0f7ecf442c [15:22:43] [02miraheze/mw-config] 07paladox 030f0f7ec - Disable file uploads temporarily due to swift outage (#5256) [15:22:46] [02mw-config] 07paladox closed pull request 03#5256: Disable file uploads temporarily due to swift outage - 13https://github.com/miraheze/mw-config/pull/5256 [15:22:47] [02mw-config] 07paladox deleted branch 03paladox-patch-2 - 13https://github.com/miraheze/mw-config [15:23:00] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [15:23:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:23:12] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 11s [15:23:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:23:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.35, 7.88, 7.48 [15:23:27] !log [paladox@mwtask141] starting deploy of {'config': True} to all [15:23:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:23:35] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 7s [15:23:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:23:40] miraheze/mw-config - paladox the build passed. [15:23:57] miraheze/mw-config - paladox the build passed. [15:26:43] !log [@test131] starting deploy of {'config': True} to all [15:26:44] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [15:26:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:26:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:49:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.38, 7.61, 7.38 [15:53:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.54, 7.71, 7.53 [16:12:03] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [16:12:27] RECOVERY - mem141 NTP time on mem141 is OK: NTP OK: Offset -0.03224125504 secs [16:13:57] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.036 second response time [16:17:06] PROBLEM - wiki.thundamoo.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.thundamoo.com could not be found [16:20:22] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.83, 3.10, 1.62 [16:21:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.70, 8.07, 7.60 [16:23:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.17, 7.78, 7.56 [16:24:22] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 1.94, 3.09, 2.03 [16:29:54] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [16:31:54] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.026 second response time [16:34:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.15, 7.44, 7.97 [16:36:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.77, 7.95, 8.09 [16:45:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.95, 7.15, 6.32 [16:47:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.83, 7.48, 7.30 [16:47:37] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [16:47:49] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 4.42, 6.25, 6.10 [16:49:26] PROBLEM - swiftproxy131 HTTP on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [16:51:21] RECOVERY - swiftproxy131 HTTP on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.020 second response time [16:51:37] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.030 second response time [16:53:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.61, 7.80, 7.59 [16:55:54] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [16:57:54] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.035 second response time [17:03:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 9.18, 7.24, 7.16 [17:05:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.90, 7.17, 7.15 [17:05:34] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.44, 6.56, 6.15 [17:07:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.54, 7.81, 7.39 [17:07:34] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 6.32, 6.50, 6.18 [17:09:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.63, 7.29, 7.25 [17:11:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.91, 7.98, 7.51 [17:13:02] RECOVERY - www.polandballwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'www.polandballwiki.com' will expire on Mon 24 Jul 2023 13:42:47 GMT +0000. [17:15:21] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.37, 7.15, 6.47 [17:15:54] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [17:17:21] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.29, 7.11, 6.53 [17:17:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.09, 7.61, 7.59 [17:17:54] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.040 second response time [17:19:12] PROBLEM - swiftobject121 APT on swiftobject121 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [17:19:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.40, 7.78, 7.64 [17:21:13] PROBLEM - swiftobject121 APT on swiftobject121 is CRITICAL: APT CRITICAL: 52 packages available for upgrade (7 critical updates). [17:23:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 5.43, 6.92, 7.36 [17:25:21] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.58, 6.37, 6.47 [17:29:23] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [17:31:19] RECOVERY - swiftproxy111 HTTP on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.018 second response time [17:31:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.55, 7.44, 7.35 [17:35:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.82, 7.40, 7.37 [17:35:37] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [17:37:37] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.037 second response time [17:44:43] PROBLEM - www.polandballwiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address www.polandballwiki.com and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [17:47:21] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.99, 6.59, 6.42 [17:49:21] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 4.96, 5.98, 6.21 [17:49:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.34, 7.51, 7.37 [17:53:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 5.61, 7.40, 7.44 [18:02:33] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [18:04:33] RECOVERY - swiftproxy111 HTTP on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.018 second response time [18:08:22] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.08, 2.45, 1.29 [18:10:21] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 1.46, 2.09, 1.31 [18:13:16] PROBLEM - swiftproxy131 HTTP on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [18:15:11] RECOVERY - swiftproxy131 HTTP on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.028 second response time [18:18:00] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [18:19:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 10.50, 8.06, 7.45 [18:21:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.97, 7.35, 7.26 [18:22:00] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [18:24:20] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [18:26:15] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.337 second response time [18:29:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 10.27, 8.10, 7.49 [18:33:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.53, 7.87, 7.55 [18:33:54] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [18:37:54] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.035 second response time [18:39:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 9.22, 8.23, 7.75 [18:44:01] PROBLEM - spcodex.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address spcodex.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [18:45:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.31, 7.53, 7.68 [18:47:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 9.42, 8.20, 7.89 [18:49:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.34, 7.69, 7.73 [18:53:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.29, 7.95, 7.83 [18:54:21] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.69, 3.36, 2.44 [18:55:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.62, 7.74, 7.76 [18:56:21] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.46, 2.96, 2.41 [18:59:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.14, 7.64, 7.70 [19:05:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.22, 7.87, 7.83 [19:07:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.30, 8.07, 7.91 [19:09:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.62, 7.77, 7.81 [19:11:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 9.22, 8.30, 8.00 [19:13:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.31, 7.89, 7.89 [19:13:37] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [19:13:56] RECOVERY - spcodex.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'spcodex.wiki' will expire on Sun 17 Sep 2023 16:21:32 GMT +0000. [19:15:37] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.030 second response time [19:19:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.37, 7.41, 7.58 [19:21:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.67, 7.55, 7.62 [19:23:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.53, 7.81, 7.69 [19:24:29] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [19:26:22] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.024 second response time [19:27:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.94, 7.70, 7.67 [19:35:34] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.56, 6.32, 5.90 [19:37:34] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 6.11, 6.18, 5.91 [19:47:05] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.91, 6.81, 6.23 [19:47:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 11.04, 8.01, 7.47 [19:48:30] PROBLEM - swiftobject121 APT on swiftobject121 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [19:49:02] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 6.12, 6.45, 6.15 [19:50:31] PROBLEM - swiftobject121 APT on swiftobject121 is CRITICAL: APT CRITICAL: 52 packages available for upgrade (7 critical updates). [19:53:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.05, 7.75, 7.59 [20:06:26] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.37, 6.13, 5.71 [20:12:01] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+1/-0/±0] 13https://github.com/miraheze/puppet/commit/8891330c0e3f [20:12:04] [02miraheze/puppet] 07paladox 038891330 - swift: prevent full disk scenario with rsync [20:12:06] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [20:12:07] [02puppet] 07paladox opened pull request 03#3241: swift: prevent full disk scenario with rsync - 13https://github.com/miraheze/puppet/pull/3241 [20:13:33] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.2441841364 secs [20:14:16] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1762295067 secs [20:14:20] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 6.32, 6.52, 6.10 [20:19:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.39, 7.63, 7.39 [20:21:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.98, 7.80, 7.49 [20:21:36] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/8891330c0e3f...0383006403ad [20:21:37] [02miraheze/puppet] 07paladox 030383006 - Update storage.pp [20:21:38] [02puppet] 07paladox synchronize pull request 03#3241: swift: prevent full disk scenario with rsync - 13https://github.com/miraheze/puppet/pull/3241 [20:22:29] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/0383006403ad...cdc67fb58034 [20:22:30] [02miraheze/puppet] 07paladox 03cdc67fb - Update common.yaml [20:22:33] [02puppet] 07paladox synchronize pull request 03#3241: swift: prevent full disk scenario with rsync - 13https://github.com/miraheze/puppet/pull/3241 [20:23:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.79, 8.01, 7.59 [20:25:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.23, 7.53, 7.45 [20:25:35] [02puppet] 07paladox synchronize pull request 03#3241: swift: prevent full disk scenario with rsync - 13https://github.com/miraheze/puppet/pull/3241 [20:25:38] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/cdc67fb58034...b42e8f7ca1bc [20:25:39] [02miraheze/puppet] 07paladox 03b42e8f7 - Update storage.pp [20:26:34] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+1/-0/±0] 13https://github.com/miraheze/puppet/compare/b42e8f7ca1bc...d2bf65bbbd05 [20:26:35] [02miraheze/puppet] 07paladox 03d2bf65b - Create devicecheck.cron [20:26:36] [02puppet] 07paladox synchronize pull request 03#3241: swift: prevent full disk scenario with rsync - 13https://github.com/miraheze/puppet/pull/3241 [20:29:24] [02puppet] 07paladox synchronize pull request 03#3241: swift: prevent full disk scenario with rsync - 13https://github.com/miraheze/puppet/pull/3241 [20:29:26] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/d2bf65bbbd05...50b2809db904 [20:29:27] [02miraheze/puppet] 07paladox 0350b2809 - Update disable_rsync.py [20:31:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.22, 7.69, 7.50 [20:33:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.26, 7.56, 7.47 [20:33:29] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/50b2809db904...4b0e7f06e537 [20:33:30] [02miraheze/puppet] 07paladox 034b0e7f0 - Update object-server.conf.erb [20:33:31] [02puppet] 07paladox synchronize pull request 03#3241: swift: prevent full disk scenario with rsync - 13https://github.com/miraheze/puppet/pull/3241 [20:33:54] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [20:35:33] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.0210686028 secs [20:35:54] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.030 second response time [20:37:59] [02puppet] 07paladox closed pull request 03#3241: swift: prevent full disk scenario with rsync - 13https://github.com/miraheze/puppet/pull/3241 [20:38:01] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+2/-0/±3] 13https://github.com/miraheze/puppet/compare/1484b2c593ec...c841ad714a8f [20:38:04] [02miraheze/puppet] 07paladox 03c841ad7 - swift: prevent full disk scenario with rsync (#3241) [20:38:07] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [20:38:09] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [20:43:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 9.90, 8.27, 7.69 [20:48:38] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [20:50:32] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.029 second response time [20:55:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.12, 7.68, 7.73 [20:59:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.27, 7.93, 7.80 [21:00:03] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/a52de5760be0 [21:00:04] [02miraheze/puppet] 07paladox 03a52de57 - swift: update path to store disable_rsync rsync conf [21:00:04] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1012465954 secs [21:00:06] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [21:00:11] [02puppet] 07paladox opened pull request 03#3242: swift: update path to store disable_rsync rsync conf - 13https://github.com/miraheze/puppet/pull/3242 [21:00:15] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset -0.0421500802 secs [21:00:36] [02puppet] 07paladox closed pull request 03#3242: swift: update path to store disable_rsync rsync conf - 13https://github.com/miraheze/puppet/pull/3242 [21:00:39] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/c841ad714a8f...db980b4243f8 [21:00:41] [02miraheze/puppet] 07paladox 03db980b4 - swift: update path to store disable_rsync rsync conf (#3242) [21:00:43] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [21:00:44] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [21:01:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.54, 7.80, 7.78 [21:02:04] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.07510018349 secs [21:02:05] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.2476913035 secs [21:03:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.05, 8.03, 7.88 [21:05:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.82, 7.93, 7.86 [21:14:05] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset -0.02424263954 secs [21:15:08] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/9b83a3ac3f44 [21:15:11] [02miraheze/puppet] 07paladox 039b83a3a - swift: fix disabling rsync [21:15:13] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [21:15:15] [02puppet] 07paladox opened pull request 03#3243: swift: fix disabling rsync - 13https://github.com/miraheze/puppet/pull/3243 [21:15:49] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/9b83a3ac3f44...2f6fd1a4de67 [21:15:50] [02miraheze/puppet] 07paladox 032f6fd1a - Update common.yaml [21:15:53] [02puppet] 07paladox synchronize pull request 03#3243: swift: fix disabling rsync - 13https://github.com/miraheze/puppet/pull/3243 [21:19:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.15, 7.67, 7.51 [21:19:34] PROBLEM - swiftobject122 Puppet on swiftobject122 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [21:19:52] PROBLEM - swiftobject111 Puppet on swiftobject111 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [21:19:58] PROBLEM - swiftobject113 Puppet on swiftobject113 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [21:20:13] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/2f6fd1a4de67...6be9b4b1c73d [21:20:16] [02miraheze/puppet] 07paladox 036be9b4b - Update storage.pp [21:20:17] [02puppet] 07paladox synchronize pull request 03#3243: swift: fix disabling rsync - 13https://github.com/miraheze/puppet/pull/3243 [21:20:29] PROBLEM - swiftobject112 Puppet on swiftobject112 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [21:20:42] PROBLEM - swiftobject121 Puppet on swiftobject121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [21:21:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.96, 7.85, 7.61 [21:21:34] RECOVERY - swiftobject122 Puppet on swiftobject122 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures [21:21:53] RECOVERY - swiftobject111 Puppet on swiftobject111 is OK: OK: Puppet is currently enabled, last run 28 seconds ago with 0 failures [21:21:58] RECOVERY - swiftobject113 Puppet on swiftobject113 is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures [21:22:15] PROBLEM - swiftproxy131 HTTP on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [21:22:29] RECOVERY - swiftobject112 Puppet on swiftobject112 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [21:22:43] RECOVERY - swiftobject121 Puppet on swiftobject121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [21:26:05] RECOVERY - swiftproxy131 HTTP on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.030 second response time [21:29:24] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 5.08, 5.69, 6.65 [21:34:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.17, 7.28, 7.98 [21:41:04] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [21:46:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.45, 7.19, 7.43 [21:52:36] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [21:52:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.67, 7.72, 7.72 [21:53:06] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query zulaclub.net. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [21:54:31] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.046 second response time [21:59:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 9.85, 7.04, 6.11 [22:00:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.28, 7.24, 7.36 [22:07:29] PROBLEM - matomo121 HTTPS on matomo121 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 358 bytes in 0.009 second response time [22:07:55] PROBLEM - mail121 webmail.miraheze.org HTTPS on mail121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:10:30] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [22:14:01] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 4.03, 2.49, 1.37 [22:16:01] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 2.33, 2.60, 1.57 [22:18:30] PROBLEM - swiftobject121 APT on swiftobject121 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [22:20:31] PROBLEM - swiftobject121 APT on swiftobject121 is CRITICAL: APT CRITICAL: 52 packages available for upgrade (7 critical updates). [22:21:33] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [22:23:30] RECOVERY - swiftproxy111 HTTP on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.032 second response time [22:31:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 5.84, 7.29, 7.86 [22:43:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.80, 7.79, 7.77 [22:45:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.25, 7.18, 7.54 [22:49:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.58, 7.61, 7.59 [22:51:07] RECOVERY - wiki.zulaclub.net - reverse DNS on sslhost is OK: SSL OK - wiki.zulaclub.net reverse DNS resolves to cp22.miraheze.org - CNAME OK [22:53:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.05, 7.05, 7.39 [22:55:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.30, 7.59, 7.55 [22:57:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.47, 7.44, 7.49 [23:13:25] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 4.97, 5.98, 6.80 [23:17:20] PROBLEM - wiki.consid.vn - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.consid.vn. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [23:21:37] PROBLEM - db112 MariaDB Connections on db112 is WARNING: Warning connection usage: 81.8%Current connections: 409 [23:27:37] RECOVERY - db112 MariaDB Connections on db112 is OK: OK connection usage: 8.8%Current connections: 44 [23:27:47] RECOVERY - mail121 webmail.miraheze.org HTTPS on mail121 is OK: HTTP OK: HTTP/1.1 200 OK - 6281 bytes in 1.077 second response time [23:28:58] RECOVERY - db112 Disk Space on db112 is OK: DISK OK - free space: / 21689 MB (16% inode=99%); [23:29:30] RECOVERY - matomo121 HTTPS on matomo121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 1.151 second response time [23:41:05] [02mw-config] 07paladox closed pull request 03#5255: Allow wiki requests again - 13https://github.com/miraheze/mw-config/pull/5255 [23:41:07] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/mw-config/compare/0f0f7ecf442c...45453977ff41 [23:41:09] [02miraheze/mw-config] 07paladox 034545397 - Allow wiki requests again (#5255) [23:41:10] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1 [23:41:13] [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/mw-config [23:41:31] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [23:41:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:41:40] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 9s [23:41:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:41:52] !log [paladox@mwtask141] starting deploy of {'config': True} to all [23:41:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:41:58] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 6s [23:42:02] PROBLEM - tudien.como.vn - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query como.vn. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [23:42:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:42:03] miraheze/mw-config - paladox the build passed. [23:47:52] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.82, 7.01, 6.67 [23:49:46] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 6.34, 6.69, 6.59 [23:53:33] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.89, 7.41, 6.88 [23:55:27] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.68, 7.28, 6.89 [23:57:20] !log [@test131] starting deploy of {'config': True} to all [23:57:21] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [23:57:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:57:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:59:24] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.22, 7.02, 6.82 [23:59:54] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable