[00:00:54] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.82, 3.74, 3.89 [00:01:03] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03mediawiki-rewrite-paths [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/8319638f6677...f4bb789b2457 [00:01:03] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.86, 4.06, 3.80 [00:01:03] [url] Comparing 8319638f6677...f4bb789b2457 · miraheze/puppet · GitHub | github.com [00:01:06] [02miraheze/puppet] 07Universal-Omega 03f4bb789 - Update mediawiki-includes.conf.erb [00:01:07] [02puppet] 07Universal-Omega synchronize pull request 03#2942: mediawiki: rewrite ^/page to ^/wiki/page - 13https://github.com/miraheze/puppet/pull/2942 [00:01:07] [url] Page not found · GitHub · GitHub | github.com [00:01:57] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.50, 3.33, 2.68 [00:02:26] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.17, 3.31, 3.12 [00:02:54] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.82, 3.51, 3.77 [00:03:55] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.93, 3.25, 2.75 [00:04:40] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [00:04:53] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 6.90, 4.48, 4.07 [00:06:53] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.59, 3.63, 3.81 [00:07:03] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.40, 3.97, 3.88 [00:07:25] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 6.92, 4.74, 2.91 [00:07:38] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:07:50] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.51, 3.80, 3.06 [00:08:52] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.36, 3.81, 3.84 [00:09:03] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.71, 4.13, 3.94 [00:09:48] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.21, 3.64, 3.10 [00:10:52] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.04, 3.09, 3.57 [00:11:03] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.64, 3.90, 3.87 [00:11:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.80, 3.36, 3.06 [00:12:52] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.28, 3.54, 3.66 [00:13:03] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.18, 3.99, 3.90 [00:13:22] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [00:13:24] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 2.07, 3.61, 3.02 [00:14:52] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.66, 3.12, 3.49 [00:15:24] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.41, 2.87, 2.82 [00:15:25] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.31, 10.19, 9.46 [00:16:20] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:16:52] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.10, 3.91, 3.76 [00:17:25] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.77, 9.52, 9.30 [00:18:51] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.96, 3.44, 3.60 [00:23:32] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.21, 4.34, 3.55 [00:24:50] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.35, 3.78, 3.66 [00:25:03] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.02, 3.69, 3.94 [00:27:03] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 5.07, 4.72, 4.31 [00:28:50] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.63, 3.42, 3.57 [00:30:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.24, 3.30, 4.00 [00:31:26] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.07, 3.17, 3.56 [00:32:49] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.67, 3.06, 3.38 [00:33:24] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.52, 2.59, 3.29 [00:34:10] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [00:34:26] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.52, 3.13, 3.71 [00:37:09] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:39:57] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [00:42:25] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 17.92, 12.23, 10.08 [00:42:55] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:43:14] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 30.05, 13.58, 7.01 [00:43:24] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 27.81, 11.53, 5.28 [00:44:15] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.96, 4.39, 3.16 [00:44:23] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.18, 10.82, 9.85 [00:44:37] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 23.05, 14.55, 6.73 [00:46:10] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 8 datacenters are down: 217.174.247.33/cpweb, 109.228.51.216/cpweb, 2a00:da00:1800:326::1/cpweb, 2a00:da00:1800:328::1/cpweb, 108.175.15.182/cpweb, 74.208.203.152/cpweb, 2607:f1c0:1800:8100::1/cpweb, 2607:f1c0:1800:26f::1/cpweb [00:46:22] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.16, 9.12, 9.34 [00:46:31] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 8 datacenters are down: 217.174.247.33/cpweb, 109.228.51.216/cpweb, 2a00:da00:1800:326::1/cpweb, 2a00:da00:1800:328::1/cpweb, 108.175.15.182/cpweb, 74.208.203.152/cpweb, 2607:f1c0:1800:8100::1/cpweb, 2607:f1c0:1800:26f::1/cpweb [00:46:32] PROBLEM - cp33 HTTPS on cp33 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:47:03] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 1.73, 3.34, 3.92 [00:47:16] PROBLEM - cp32 HTTPS on cp32 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:48:02] PROBLEM - cp22 HTTPS on cp22 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:48:48] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 1.90, 2.99, 3.80 [00:49:17] RECOVERY - cp32 HTTPS on cp32 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3420 bytes in 4.206 second response time [00:49:59] RECOVERY - cp22 HTTPS on cp22 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3418 bytes in 0.091 second response time [00:50:10] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [00:50:23] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [00:50:33] RECOVERY - cp33 HTTPS on cp33 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3417 bytes in 0.428 second response time [00:50:48] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 10.81, 6.74, 5.12 [00:51:03] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 9.19, 5.87, 4.72 [00:52:44] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.27, 4.48, 3.70 [00:56:02] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 2.19, 3.52, 3.66 [00:57:44] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [00:58:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.99, 11.16, 9.81 [00:59:58] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.55, 2.82, 3.37 [01:00:42] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:02:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.59, 10.26, 9.79 [01:04:22] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.69, 10.15, 9.81 [01:05:46] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.2378701568 secs [01:08:37] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 0.78, 0.88, 3.63 [01:09:24] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 0.65, 1.27, 3.97 [01:09:46] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset -0.02171683311 secs [01:10:36] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.52, 0.84, 3.28 [01:12:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.86, 12.30, 10.81 [01:14:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.85, 11.72, 10.78 [01:15:24] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.73, 1.34, 3.18 [01:20:44] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.69, 2.26, 3.96 [01:22:43] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.29, 2.93, 4.00 [01:26:22] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.43, 9.27, 9.98 [01:29:19] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 11.35, 19.49, 10.51 [01:36:08] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [01:39:06] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:42:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.35, 11.72, 10.29 [01:42:25] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.26, 11.21, 10.32 [01:44:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.38, 10.77, 10.13 [01:46:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.95, 11.38, 10.41 [01:46:36] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 0.65, 1.13, 3.71 [01:48:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.92, 10.44, 10.19 [01:48:37] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.31, 0.87, 3.30 [01:50:22] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.15, 10.00, 10.06 [01:51:58] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.06, 9.89, 10.10 [01:53:44] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [01:54:07] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 21.19, 7.49, 3.75 [01:56:42] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:57:55] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 0.80, 3.82, 3.07 [01:58:29] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.08, 3.08, 3.97 [01:59:37] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.25, 10.08, 10.01 [01:59:49] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.20, 2.65, 2.73 [02:01:24] PROBLEM - cp23 SSH on cp23 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:01:32] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.21, 9.46, 9.78 [02:02:29] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.78, 3.55, 3.91 [02:02:31] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 109.228.51.216/cpweb, 2a00:da00:1800:328::1/cpweb [02:02:57] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 109.228.51.216/cpweb, 2a00:da00:1800:328::1/cpweb [02:03:18] RECOVERY - cp23 SSH on cp23 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [02:04:02] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 47.20, 37.93, 17.78 [02:04:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.66, 10.18, 9.70 [02:04:27] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [02:04:28] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.18, 3.12, 3.71 [02:04:54] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [02:06:22] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.62, 9.85, 9.63 [02:06:28] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.34, 3.55, 3.79 [02:07:25] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.34, 10.69, 10.30 [02:08:24] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [02:10:28] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.92, 3.41, 3.69 [02:11:22] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:11:58] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.68, 3.69, 3.21 [02:12:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.44, 11.19, 10.21 [02:13:56] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 2.84, 3.34, 3.14 [02:14:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.26, 10.64, 10.13 [02:14:28] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 44.05, 12.15, 6.43 [02:15:25] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.48, 11.05, 10.48 [02:18:22] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.88, 9.79, 9.93 [02:19:25] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.73, 11.39, 10.75 [02:23:10] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [02:23:25] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.93, 9.42, 10.07 [02:24:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.14, 2.40, 3.88 [02:26:08] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:26:26] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.17, 2.98, 3.88 [02:28:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.14, 2.95, 3.76 [02:28:28] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.96, 2.83, 3.85 [02:28:46] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 0.42, 0.76, 3.91 [02:32:26] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.43, 5.62, 4.64 [02:32:29] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.05, 3.13, 3.69 [02:32:36] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.32, 0.55, 3.12 [02:34:28] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.16, 2.77, 3.49 [02:36:28] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.60, 2.64, 3.35 [02:39:41] RECOVERY - wiki.slimeapedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.slimeapedia.org' will expire on Fri 23 Dec 2022 04:36:40 GMT +0000. [02:40:55] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [02:42:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.72, 10.51, 9.66 [02:43:53] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:44:22] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.02, 9.35, 9.33 [02:46:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.14, 2.77, 3.95 [02:47:24] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 4.31, 3.80, 2.72 [02:53:24] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 0.76, 3.78, 3.32 [02:55:24] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.52, 2.68, 2.97 [02:56:25] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.26, 2.54, 3.35 [02:59:33] PROBLEM - cp23 PowerDNS Recursor on cp23 is CRITICAL: CRITICAL - Plugin timed out while executing system call [03:01:24] PROBLEM - cp23 SSH on cp23 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:01:29] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 29.95, 18.27, 9.05 [03:03:23] PROBLEM - cp23 Puppet on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:03:23] PROBLEM - cp23 Varnish Backends on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:03:53] PROBLEM - cp23 conntrack_table_size on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:04:29] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 109.228.51.216/cpweb [03:04:31] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 109.228.51.216/cpweb [03:05:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 19.71, 23.60, 12.19 [03:06:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.46, 3.42, 3.26 [03:07:25] RECOVERY - cp23 SSH on cp23 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [03:08:13] RECOVERY - cp23 Puppet on cp23 is OK: OK: Puppet is currently enabled, last run 41 minutes ago with 0 failures [03:08:21] RECOVERY - cp23 Varnish Backends on cp23 is OK: All 14 backends are healthy [03:08:23] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [03:08:24] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [03:08:26] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.83, 4.08, 3.53 [03:08:56] RECOVERY - cp23 conntrack_table_size on cp23 is OK: OK: nf_conntrack is 0 % full [03:09:05] RECOVERY - cp23 PowerDNS Recursor on cp23 is OK: DNS OK: 0.152 seconds response time. miraheze.org returns 109.228.51.216,217.174.247.33,2a00:da00:1800:326::1,2a00:da00:1800:328::1 [03:10:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.60, 3.74, 3.49 [03:11:13] PROBLEM - wiki.slimeapedia.org - LetsEncrypt on sslhost is CRITICAL: connect to address wiki.slimeapedia.org and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [03:16:26] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.15, 2.88, 3.18 [03:31:24] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [03:34:22] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:34:36] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 0.20, 0.44, 3.95 [03:37:39] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 31.32, 14.28, 7.13 [03:38:10] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 108.175.15.182/cpweb, 2607:f1c0:1800:8100::1/cpweb [03:38:36] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.91, 0.59, 3.19 [03:40:10] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [03:40:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.79, 2.84, 3.97 [03:42:18] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.89, 3.41, 3.88 [03:46:18] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.62, 4.02, 4.02 [03:46:34] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.35, 2.26, 3.38 [03:50:29] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.50, 4.51, 4.04 [03:54:16] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.39, 3.65, 3.95 [03:56:28] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.85, 3.73, 3.90 [03:58:00] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.85, 3.25, 2.69 [03:58:36] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.28, 4.82, 3.19 [04:00:28] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.74, 2.41, 3.30 [04:00:36] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.97, 3.36, 2.86 [04:01:56] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 3.11, 3.32, 2.86 [04:04:14] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 11.77, 5.77, 4.54 [04:16:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.95, 2.75, 3.96 [04:26:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 1.97, 3.18, 3.83 [04:26:26] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.24, 2.74, 3.37 [04:32:12] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 1.93, 2.66, 3.40 [04:34:25] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.96, 10.92, 6.79 [04:36:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 5.17, 3.95, 3.75 [04:38:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.42, 3.35, 3.56 [04:42:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 6.11, 4.17, 3.80 [04:42:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.58, 10.18, 9.15 [04:44:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.86, 3.98, 3.77 [04:44:22] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.29, 8.99, 8.83 [04:44:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 0.88, 2.27, 4.00 [04:45:24] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 4.38, 2.68, 1.69 [04:46:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 5.01, 4.61, 4.04 [04:46:28] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.83, 2.72, 2.24 [04:48:28] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.40, 2.63, 2.27 [04:50:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.01, 3.34, 3.66 [04:50:26] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.61, 1.75, 3.20 [04:51:52] PROBLEM - cp22 SSH on cp22 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:51:58] PROBLEM - cp22 HTTPS on cp22 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:52:42] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 217.174.247.33/cpweb, 2a00:da00:1800:326::1/cpweb [04:53:09] PROBLEM - cp22 PowerDNS Recursor on cp22 is CRITICAL: CRITICAL - Plugin timed out while executing system call [04:53:28] PROBLEM - cp22 Puppet on cp22 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:53:38] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 217.174.247.33/cpweb, 2a00:da00:1800:326::1/cpweb [04:53:59] RECOVERY - cp22 HTTPS on cp22 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3421 bytes in 4.420 second response time [04:54:22] PROBLEM - cp22 Varnish Backends on cp22 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:54:39] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [04:55:13] PROBLEM - swiftobject114 Current Load on swiftobject114 is WARNING: WARNING - load average: 0.44, 2.29, 4.00 [04:55:24] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.01, 3.89, 3.03 [04:55:26] PROBLEM - cp22 conntrack_table_size on cp22 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:57:20] PROBLEM - cp22 HTTP 4xx/5xx ERROR Rate on cp22 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:57:22] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.26, 3.30, 2.92 [04:57:23] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [04:57:27] PROBLEM - cp22 ferm_active on cp22 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:58:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.34, 3.63, 3.61 [04:59:13] RECOVERY - swiftobject114 Current Load on swiftobject114 is OK: OK - load average: 0.12, 1.07, 3.10 [05:00:12] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 1.88, 2.92, 3.35 [05:00:22] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:02:24] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 4 datacenters are down: 217.174.247.33/cpweb, 2a00:da00:1800:326::1/cpweb, 74.208.203.152/cpweb, 2607:f1c0:1800:26f::1/cpweb [05:02:29] PROBLEM - cp22 HTTPS on cp22 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:02:39] RECOVERY - cp22 PowerDNS Recursor on cp22 is OK: DNS OK: 0.266 seconds response time. miraheze.org returns 109.228.51.216,2a00:da00:1800:328::1 [05:02:40] RECOVERY - cp22 HTTP 4xx/5xx ERROR Rate on cp22 is OK: OK - NGINX Error Rate is 18% [05:02:41] RECOVERY - cp22 ferm_active on cp22 is OK: OK ferm input default policy is set [05:02:53] RECOVERY - cp22 Varnish Backends on cp22 is OK: All 14 backends are healthy [05:03:05] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [05:03:15] RECOVERY - cp22 conntrack_table_size on cp22 is OK: OK: nf_conntrack is 0 % full [05:03:36] PROBLEM - cp33 SSH on cp33 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:04:14] RECOVERY - cp22 Puppet on cp22 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures [05:04:24] RECOVERY - cp22 SSH on cp22 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [05:04:26] RECOVERY - cp22 HTTPS on cp22 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3405 bytes in 0.093 second response time [05:04:40] PROBLEM - cp33 HTTPS on cp33 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:06:04] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:06:25] PROBLEM - cp33 PowerDNS Recursor on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:06:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.90, 3.50, 3.03 [05:06:40] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:07:03] PROBLEM - cp33 conntrack_table_size on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:07:10] PROBLEM - cp33 Puppet on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:07:10] PROBLEM - cp33 Varnish Backends on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:07:42] PROBLEM - cp33 HTTP 4xx/5xx ERROR Rate on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:07:50] PROBLEM - cp33 Stunnel for puppet141 on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:08:24] RECOVERY - cp33 PowerDNS Recursor on cp33 is OK: DNS OK: 0.492 seconds response time. miraheze.org returns 108.175.15.182,2607:f1c0:1800:8100::1 [05:08:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 29.80, 9.93, 5.27 [05:08:41] RECOVERY - cp33 HTTPS on cp33 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3421 bytes in 0.455 second response time [05:08:42] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [05:09:02] RECOVERY - cp33 conntrack_table_size on cp33 is OK: OK: nf_conntrack is 1 % full [05:09:07] RECOVERY - cp33 Varnish Backends on cp33 is OK: All 14 backends are healthy [05:09:10] RECOVERY - cp33 Puppet on cp33 is OK: OK: Puppet is currently enabled, last run 10 minutes ago with 0 failures [05:09:38] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [05:09:41] RECOVERY - cp33 HTTP 4xx/5xx ERROR Rate on cp33 is OK: OK - NGINX Error Rate is 4% [05:09:46] RECOVERY - cp33 Stunnel for puppet141 on cp33 is OK: TCP OK - 0.000 second response time on localhost port 8204 [05:09:51] RECOVERY - cp33 SSH on cp33 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [05:10:13] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [05:11:41] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:12:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.19, 3.55, 3.26 [05:16:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 0.87, 2.96, 3.64 [05:17:30] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [05:18:26] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.64, 2.52, 3.39 [05:20:28] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:26:07] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [05:26:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.69, 3.62, 3.74 [05:29:05] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:29:06] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 30.93, 18.53, 7.88 [05:30:12] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.50, 2.84, 3.39 [05:31:18] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.01, 3.71, 2.34 [05:31:50] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [05:33:17] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 1.27, 2.84, 2.19 [05:34:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.47, 3.77, 3.67 [05:34:26] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 3.68, 4.63, 3.77 [05:34:49] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:36:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.31, 3.92, 3.73 [05:37:05] PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.1889836788 secs [05:38:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.57, 3.42, 3.57 [05:39:47] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.58, 2.96, 2.49 [05:40:36] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 0.33, 2.11, 3.88 [05:40:39] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [05:41:14] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.23, 10.65, 9.06 [05:41:44] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 2.86, 2.75, 2.47 [05:43:12] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.66, 10.62, 9.25 [05:43:19] [02mw-config] 07Reception123 reviewed pull request 03#4935 commit - 13https://github.com/miraheze/mw-config/pull/4935#discussion_r1017451267 [05:43:20] [url] T9847: Set wgShortDescriptionEnableTagline to false on slymodswiki by ugochimobi · Pull Request #4935 · miraheze/mw-config · GitHub | github.com [05:43:25] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 2.24, 1.27, 3.99 [05:43:37] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:44:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 5.92, 4.16, 3.77 [05:44:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.53, 3.43, 3.83 [05:44:28] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 0.86, 1.38, 3.95 [05:44:36] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.54, 1.16, 3.08 [05:45:09] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.94, 9.97, 9.17 [05:47:24] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.41, 0.83, 3.19 [05:48:26] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.10, 2.54, 3.35 [05:49:19] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [05:50:29] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.70, 1.29, 3.12 [05:51:00] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.09, 10.37, 9.62 [05:51:19] [02mw-config] 07Reception123 reviewed pull request 03#4935 commit - 13https://github.com/miraheze/mw-config/pull/4935#discussion_r1017451267 [05:51:20] [url] T9847: Set wgShortDescriptionEnableTagline to false on slymodswiki by ugochimobi · Pull Request #4935 · miraheze/mw-config · GitHub | github.com [05:51:31] [02mw-config] 07Reception123 closed pull request 03#4935: T9847: Set wgShortDescriptionEnableTagline to false on slymodswiki - 13https://github.com/miraheze/mw-config/pull/4935 [05:51:31] [url] Page not found · GitHub · GitHub | github.com [05:51:32] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/mw-config/compare/602793b45215...48ddd4b63c04 [05:51:32] [url] Comparing 602793b45215...48ddd4b63c04 · miraheze/mw-config · GitHub | github.com [05:51:34] [02miraheze/mw-config] 07ugochimobi 0348ddd4b - T9847: Set wgShortDescriptionEnableTagline to false on slymodswiki (#4935) [05:52:17] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:52:36] miraheze/mw-config - Reception123 the build passed. [05:52:58] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.81, 8.99, 9.20 [05:55:07] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [05:56:40] !log [@test131] starting deploy of {'config': True} to all [05:56:41] !log [@test131] DEPLOY ABORTED: Canary check failed for beta.betaheze.org@localhost [05:56:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:56:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:57:06] RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset 0.0579186976 secs [05:58:05] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:59:29] PROBLEM - test131 Puppet on test131 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[MediaWiki Config Sync] [06:00:44] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [06:03:42] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:04:26] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.52, 4.31, 3.51 [06:06:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.14, 3.19, 3.73 [06:06:23] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [06:06:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.90, 3.74, 3.40 [06:09:21] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:10:06] !log [@mwtask141] starting deploy of {'config': True} to all [06:10:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:10:16] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 10s [06:10:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:10:25] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.62, 2.53, 3.03 [06:12:12] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [06:15:10] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:17:57] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [06:20:55] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:22:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.91, 3.64, 3.67 [06:23:34] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [06:24:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.40, 3.60, 3.66 [06:26:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.03, 3.59, 3.64 [06:26:32] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:27:14] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 9.02, 6.28, 3.95 [06:27:26] RECOVERY - test131 Puppet on test131 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures [06:28:12] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.80, 3.27, 3.51 [06:32:14] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [06:35:07] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 0.71, 3.05, 3.56 [06:35:13] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:35:49] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.92, 10.66, 9.92 [06:37:05] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.59, 2.22, 3.18 [06:37:43] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.15, 10.19, 9.85 [06:38:12] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.45, 3.14, 3.39 [06:40:54] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [06:41:33] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.59, 10.40, 10.00 [06:42:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 17.67, 13.44, 10.87 [06:43:52] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:46:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.95, 11.42, 10.62 [06:46:37] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [06:47:25] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.74, 11.60, 10.57 [06:49:25] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.80, 10.86, 10.43 [06:50:10] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.36, 3.55, 3.45 [06:51:25] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.44, 9.87, 10.11 [06:52:09] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 7.04, 4.50, 3.80 [06:52:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.89, 11.23, 10.75 [06:54:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.82, 10.20, 10.43 [06:55:06] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:56:09] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.35, 3.50, 3.55 [06:58:01] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [06:58:22] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.71, 9.39, 10.06 [06:58:45] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.29, 3.61, 2.93 [06:58:45] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 10.83, 5.19, 2.61 [06:59:03] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.35, 3.53, 3.97 [07:00:43] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.68, 2.84, 2.73 [07:00:59] PROBLEM - m.miraheze.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:01:00] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [07:01:03] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 5.25, 3.92, 4.04 [07:01:24] PROBLEM - cp23 SSH on cp23 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:01:38] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 109.228.51.216/cpweb, 2a00:da00:1800:328::1/cpweb [07:02:10] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 109.228.51.216/cpweb, 2a00:da00:1800:328::1/cpweb [07:02:54] PROBLEM - m.miraheze.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'm.miraheze.org' expires in 13 day(s) (Tue 22 Nov 2022 14:02:45 GMT +0000). [07:03:18] RECOVERY - cp23 SSH on cp23 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [07:03:38] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [07:03:46] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [07:04:07] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 1.55, 2.76, 3.26 [07:04:10] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [07:05:34] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.45, 3.22, 2.58 [07:07:34] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.60, 2.65, 2.45 [07:07:52] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.05, 4.52, 3.14 [07:09:28] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [07:09:47] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.05, 3.43, 2.90 [07:10:15] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.72, 10.45, 9.91 [07:10:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.50, 11.16, 10.20 [07:11:42] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.19, 2.70, 2.69 [07:15:11] PROBLEM - cloud11 SMART on cloud11 is WARNING: WARNING: [cciss,6] - No health status line found, Self-test log contains errors --- [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean| [07:17:53]