[00:00:22] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 6.49, 3.63, 1.79 [00:00:58] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.46, 6.37, 4.39 [00:02:22] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.57, 3.63, 2.01 [00:04:22] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.40, 3.75, 2.22 [00:06:22] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 2.60, 2.93, 2.09 [00:06:52] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.23, 3.64, 3.86 [00:09:55] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 9.31, 5.49, 2.67 [00:12:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.49, 2.64, 3.38 [00:12:49] PROBLEM - cp22 NTP time on cp22 is CRITICAL: NTP CRITICAL: Offset 2.186362982 secs [00:13:51] PROBLEM - test131 NTP time on test131 is CRITICAL: NTP CRITICAL: Offset 0.7365761697 secs [00:15:55] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 1.72, 3.63, 2.85 [00:17:54] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.45, 2.88, 2.67 [00:24:30] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.50, 4.96, 4.02 [00:24:49] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.001181274652 secs [00:25:13] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 2.22, 3.47, 2.70 [00:25:39] PROBLEM - cp32 NTP time on cp32 is CRITICAL: NTP CRITICAL: Offset 0.5612648129 secs [00:27:11] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 2.71, 3.11, 2.66 [00:27:51] RECOVERY - test131 NTP time on test131 is OK: NTP OK: Offset -0.006503373384 secs [00:29:40] RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset -0.005289286375 secs [00:36:05] [02mw-config] 07AgentIsai opened pull request 03#5070: Update sitenotice and read-only - 13https://github.com/miraheze/mw-config/pull/5070 [00:36:18] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.53, 3.39, 3.96 [00:36:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.77, 4.15, 3.04 [00:37:10] miraheze/mw-config - AgentIsai the build passed. [00:40:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.69, 3.85, 3.18 [00:42:12] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.20, 3.14, 3.60 [00:42:31] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.91, 3.12, 2.99 [00:43:28] [02mw-config] 07AgentIsai synchronize pull request 03#5070: Update sitenotice and read-only - 13https://github.com/miraheze/mw-config/pull/5070 [00:44:31] miraheze/mw-config - AgentIsai the build passed. [00:46:08] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.96, 3.62, 3.72 [00:52:03] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.58, 3.67, 3.66 [00:57:57] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.09, 3.52, 3.74 [00:59:31] [02puppet] 07AgentIsai opened pull request 03#3126: mediawiki: use REL1_39 - 13https://github.com/miraheze/puppet/pull/3126 [01:01:05] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1255384088 secs [01:01:53] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.17, 2.66, 3.32 [01:06:49] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1087563634 secs [01:13:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.36, 3.53, 3.08 [01:15:25] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.41, 2.74, 2.84 [01:17:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.54, 4.83, 3.57 [01:19:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.19, 3.53, 3.24 [01:21:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 3.19, 3.17, 3.12 [01:25:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.89, 4.89, 3.78 [01:31:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.23, 3.64, 3.61 [01:32:49] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09032875299 secs [01:33:05] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.08091336489 secs [01:35:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.04, 2.90, 3.32 [01:36:32] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.70, 4.47, 3.46 [01:48:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.85, 3.81, 3.92 [01:49:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.96, 3.52, 3.16 [01:51:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 3.11, 3.32, 3.12 [01:56:32] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.15, 3.39, 3.60 [02:00:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.59, 3.47, 3.58 [02:01:05] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1175702512 secs [02:02:34] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.60, 4.38, 3.65 [02:04:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.07, 4.42, 3.86 [02:04:32] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.30, 3.87, 3.55 [02:05:05] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.08712485433 secs [02:08:28] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.00, 3.12, 3.33 [02:14:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.13, 3.39, 3.81 [02:21:56] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 8.79, 4.71, 2.52 [02:22:31] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.50, 2.63, 3.30 [02:24:05] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.09, 4.08, 3.67 [02:27:54] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.83, 3.30, 2.76 [02:31:57] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.02, 3.60, 3.72 [02:33:05] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset -0.1096646488 secs [02:33:55] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.04, 4.07, 3.86 [02:37:04] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.92, 4.74, 3.84 [02:37:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.90, 3.22, 3.58 [02:41:05] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset -0.09256258607 secs [02:41:47] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.59, 2.33, 3.14 [02:56:30] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.22, 3.21, 3.80 [03:00:31] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.94, 2.32, 3.28 [03:03:31] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 18.10, 10.41, 5.66 [03:06:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.80, 3.31, 3.30 [03:15:19] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.42, 2.93, 3.86 [03:16:01] [02CreateWiki] 07AgentIsai opened pull request 03#363: Display read-only errors when database is in read-only - 13https://github.com/miraheze/CreateWiki/pull/363 [03:17:06] [02CreateWiki] 07AgentIsai synchronize pull request 03#363: Display read-only errors when database is in read-only - 13https://github.com/miraheze/CreateWiki/pull/363 [03:18:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.32, 3.17, 3.75 [03:18:47] [02CreateWiki] 07AgentIsai synchronize pull request 03#363: Display read-only errors when database is in read-only - 13https://github.com/miraheze/CreateWiki/pull/363 [03:21:13] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.10, 3.69, 3.84 [03:22:31] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.67, 2.61, 3.36 [03:23:02] [02CreateWiki] 07AgentIsai synchronize pull request 03#363: Display read-only errors when database is in read-only - 13https://github.com/miraheze/CreateWiki/pull/363 [03:23:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.84, 3.68, 3.81 [03:24:10] miraheze/CreateWiki - AgentIsai the build passed. [03:27:07] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.63, 4.54, 4.08 [03:27:14] miraheze/CreateWiki - AgentIsai the build passed. [03:27:59] miraheze/CreateWiki - AgentIsai the build passed. [03:30:07] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query cn.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [03:32:52] miraheze/CreateWiki - AgentIsai the build passed. [03:33:01] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.69, 3.27, 3.71 [03:36:57] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.22, 2.27, 3.20 [03:40:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.41, 3.92, 3.29 [03:41:54] [02CreateWiki] 07Universal-Omega reviewed pull request 03#363 commit - 13https://github.com/miraheze/CreateWiki/pull/363#discussion_r1071709771 [03:42:27] [02CreateWiki] 07AgentIsai synchronize pull request 03#363: Display read-only errors when database is in read-only - 13https://github.com/miraheze/CreateWiki/pull/363 [03:42:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.87, 3.61, 3.27 [03:42:34] [02CreateWiki] 07AgentIsai reviewed pull request 03#363 commit - 13https://github.com/miraheze/CreateWiki/pull/363#discussion_r1071710100 [03:44:31] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.54, 2.90, 3.05 [03:45:43] [02CreateWiki] 07Universal-Omega reviewed pull request 03#363 commit - 13https://github.com/miraheze/CreateWiki/pull/363#discussion_r1071711115 [03:46:09] [02CreateWiki] 07Universal-Omega reviewed pull request 03#363 commit - 13https://github.com/miraheze/CreateWiki/pull/363#discussion_r1071711282 [03:47:22] [02CreateWiki] 07Universal-Omega reviewed pull request 03#363 commit - 13https://github.com/miraheze/CreateWiki/pull/363#discussion_r1071711658 [03:47:45] [02CreateWiki] 07Universal-Omega reviewed pull request 03#363 commit - 13https://github.com/miraheze/CreateWiki/pull/363#discussion_r1071711282 [03:48:15] [02CreateWiki] 07Universal-Omega reviewed pull request 03#363 commit - 13https://github.com/miraheze/CreateWiki/pull/363#discussion_r1071711658 [03:48:44] [02CreateWiki] 07AgentIsai reviewed pull request 03#363 commit - 13https://github.com/miraheze/CreateWiki/pull/363#discussion_r1071712155 [03:49:55] [02CreateWiki] 07Universal-Omega reviewed pull request 03#363 commit - 13https://github.com/miraheze/CreateWiki/pull/363#discussion_r1071712589 [03:50:10] [02CreateWiki] 07Universal-Omega reviewed pull request 03#363 commit - 13https://github.com/miraheze/CreateWiki/pull/363#discussion_r1071712719 [03:50:43] miraheze/CreateWiki - AgentIsai the build passed. [03:53:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.34, 4.30, 3.45 [03:55:38] [02CreateWiki] 07AgentIsai reviewed pull request 03#363 commit - 13https://github.com/miraheze/CreateWiki/pull/363#discussion_r1071714570 [03:59:30] PROBLEM - swiftac111 PowerDNS Recursor on swiftac111 is CRITICAL: CRITICAL - Plugin timed out while executing system call [04:00:03] PROBLEM - swiftac111 Current Load on swiftac111 is CRITICAL: CRITICAL - load average: 23.82, 11.84, 5.87 [04:00:30] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.88, 2.94, 1.53 [04:02:00] PROBLEM - swiftac111 SSH on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:02:28] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 6.27, 4.27, 2.19 [04:04:19] PROBLEM - swiftac111 Puppet on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:05:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.01, 3.39, 3.75 [04:07:53] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 401 Unauthorized [04:08:01] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 401 Unauthorized [04:08:06] PROBLEM - swiftproxy131 HTTP on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 401 Unauthorized [04:08:13] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 401 Unauthorized [04:08:42] Did Swift just go down? [04:09:26] PROBLEM - swiftac111 ferm_active on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:09:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.47, 3.94, 3.86 [04:10:12] PROBLEM - swiftac111 NTP time on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:10:25] PROBLEM - swiftac111 conntrack_table_size on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:11:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.24, 3.66, 3.78 [04:12:22] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 0.75, 3.45, 3.18 [04:12:54] PROBLEM - swiftac111 Swift Account Service on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:13:47] Considering that I cant currently view images, i'd say so [04:14:22] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.02, 2.59, 2.89 [04:14:32] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.1635723412 secs [04:15:14] PROBLEM - swiftac111 Disk Space on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:15:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.54, 2.68, 3.36 [04:16:21] seems like it is possibly an issue with swiftac [04:16:39] meaning nothing can authenticate [04:20:07] paladox: ^^ [04:22:14] seems to be working again [04:23:12] RECOVERY - swiftac111 SSH on swiftac111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [04:23:12] RECOVERY - swiftac111 Disk Space on swiftac111 is OK: DISK OK - free space: / 112715 MB (91% inode=96%); [04:23:16] RECOVERY - swiftac111 ferm_active on swiftac111 is OK: OK ferm input default policy is set [04:23:23] RECOVERY - swiftac111 Swift Account Service on swiftac111 is OK: TCP OK - 0.001 second response time on 2a10:6740::6:202 port 6002 [04:23:24] RECOVERY - swiftproxy131 HTTP on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.057 second response time [04:23:24] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.023 second response time [04:23:53] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.023 second response time [04:23:55] RECOVERY - swiftac111 conntrack_table_size on swiftac111 is OK: OK: nf_conntrack is 0 % full [04:24:11] RECOVERY - swiftac111 NTP time on swiftac111 is OK: NTP OK: Offset -0.005662232637 secs [04:24:13] RECOVERY - swiftproxy111 HTTP on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.029 second response time [04:24:20] RECOVERY - swiftac111 Puppet on swiftac111 is OK: OK: Puppet is currently enabled, last run 59 minutes ago with 0 failures [04:24:38] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.01, 5.24, 4.06 [04:24:56] RECOVERY - swiftac111 PowerDNS Recursor on swiftac111 is OK: DNS OK: 0.222 seconds response time. miraheze.org returns 109.228.51.216,217.174.247.33,2a00:da00:1800:326::1,2a00:da00:1800:328::1 [04:28:21] RECOVERY - wiki.nj.cn.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.nj.cn.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [04:28:25] PROBLEM - swiftac111 Puppet on swiftac111 is WARNING: WARNING: Puppet last ran 1 hour ago [04:30:22] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.63, 3.08, 2.39 [04:30:32] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.68, 3.79, 3.87 [04:32:22] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 3.31, 3.15, 2.51 [04:32:30] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.23, 4.49, 4.11 [04:34:21] RECOVERY - swiftac111 Puppet on swiftac111 is OK: OK: Puppet is currently enabled, last run 15 seconds ago with 0 failures [04:34:28] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.00, 3.65, 3.86 [04:36:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.84, 3.76, 2.86 [04:38:24] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.41, 2.56, 3.37 [04:40:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.04, 3.62, 3.08 [04:42:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 14.32, 7.40, 4.49 [04:42:33] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.05745670199 secs [04:48:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.26, 3.54, 3.63 [04:50:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 3.05, 4.29, 3.96 [04:52:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.23, 3.24, 3.62 [04:54:31] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.16, 2.50, 3.30 [04:55:45] PROBLEM - swiftac111 Current Load on swiftac111 is WARNING: WARNING - load average: 1.62, 1.86, 7.66 [04:59:45] RECOVERY - swiftac111 Current Load on swiftac111 is OK: OK - load average: 1.16, 1.51, 6.19 [05:02:50] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 6.36, 7.79, 4.33 [05:06:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.83, 3.33, 2.98 [05:08:43] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.49, 3.12, 3.36 [05:12:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.93, 3.84, 3.34 [05:14:31] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.85, 3.19, 3.17 [05:25:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.48, 3.91, 3.17 [05:29:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.13, 3.36, 3.15 [05:31:52] PROBLEM - swiftac111 Current Load on swiftac111 is WARNING: WARNING - load average: 7.63, 7.06, 4.94 [05:34:10] PROBLEM - swiftac111 Current Load on swiftac111 is CRITICAL: CRITICAL - load average: 25.39, 13.56, 7.61 [05:35:44] PROBLEM - swiftac111 PowerDNS Recursor on swiftac111 is CRITICAL: CRITICAL - Plugin timed out while executing system call [05:37:58] PROBLEM - swiftac111 Puppet on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:38:01] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [05:38:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.02, 3.60, 2.69 [05:39:01] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [05:39:04] PROBLEM - swiftac111 SSH on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:39:16] PROBLEM - swiftproxy131 HTTP on swiftproxy131 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:39:22] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:40:31] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.48, 2.70, 2.47 [05:43:06] RECOVERY - swiftproxy131 HTTP on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.044 second response time [05:43:13] RECOVERY - swiftproxy111 HTTP on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.016 second response time [05:43:59] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.037 second response time [05:48:59] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [05:49:09] PROBLEM - swiftproxy131 HTTP on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [05:49:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 2.59, 4.34, 3.46 [05:49:56] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [05:51:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.58, 3.35, 3.20 [05:52:01] PROBLEM - swiftac111 conntrack_table_size on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:52:54] PROBLEM - swiftac111 Swift Account Service on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:52:55] PROBLEM - swiftac111 ferm_active on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:54:00] RECOVERY - swiftac111 SSH on swiftac111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [05:54:01] RECOVERY - swiftac111 conntrack_table_size on swiftac111 is OK: OK: nf_conntrack is 0 % full [05:54:01] RECOVERY - swiftac111 PowerDNS Recursor on swiftac111 is OK: DNS OK: 1.792 second response time. miraheze.org returns 109.228.51.216,217.174.247.33,2a00:da00:1800:326::1,2a00:da00:1800:328::1 [05:54:42] RECOVERY - swiftproxy111 HTTP on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.077 second response time [05:54:52] RECOVERY - swiftac111 Swift Account Service on swiftac111 is OK: TCP OK - 0.001 second response time on 2a10:6740::6:202 port 6002 [05:54:53] RECOVERY - swiftac111 ferm_active on swiftac111 is OK: OK ferm input default policy is set [05:55:01] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.042 second response time [05:55:02] PROBLEM - swiftac111 Puppet on swiftac111 is WARNING: WARNING: Puppet last ran 1 hour ago [05:55:06] RECOVERY - swiftproxy131 HTTP on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.018 second response time [05:55:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.44, 4.88, 3.88 [05:55:53] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.021 second response time [06:03:07] RECOVERY - swiftac111 Puppet on swiftac111 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:03:16] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 6.22, 3.60, 2.06 [06:07:09] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.66, 2.93, 2.43 [06:07:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 0.94, 2.88, 3.68 [06:09:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.26, 2.32, 3.37 [06:10:58] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.63, 3.07, 2.63 [06:13:05] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.34, 2.85, 2.95 [06:15:11] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 31.33, 15.96, 7.73 [06:15:36] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 108.175.15.182/cpweb, 2607:f1c0:1800:8100::1/cpweb [06:16:01] PROBLEM - cp32 PowerDNS Recursor on cp32 is CRITICAL: CRITICAL - Plugin timed out while executing system call [06:17:36] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [06:17:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.63, 3.50, 3.36 [06:17:59] RECOVERY - cp32 PowerDNS Recursor on cp32 is OK: DNS OK: 0.201 seconds response time. miraheze.org returns 108.175.15.182,2607:f1c0:1800:26f::1,74.208.203.152 [06:18:42] PROBLEM - swiftac111 Current Load on swiftac111 is WARNING: WARNING - load average: 1.14, 1.77, 7.73 [06:19:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 3.04, 3.16, 3.25 [06:22:37] RECOVERY - swiftac111 Current Load on swiftac111 is OK: OK - load average: 1.86, 1.81, 6.41 [06:27:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 3.86, 4.51, 3.80 [06:31:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.36, 3.97, 3.78 [06:33:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.87, 2.87, 3.40 [06:42:49] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.309695214 secs [06:46:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 0.90, 2.67, 3.97 [06:50:31] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.92, 1.67, 3.26 [06:55:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 9.40, 5.38, 3.49 [07:01:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.28, 3.32, 3.24 [07:02:49] PROBLEM - cp22 NTP time on cp22 is CRITICAL: NTP CRITICAL: Offset 0.708581239 secs [07:08:29] PROBLEM - db142 Backups SQL on db142 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864144 seconds old and 48 bytes [07:12:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 13.89, 7.01, 4.41 [07:14:49] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.001124590635 secs [07:18:30] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.01, 2.98, 3.42 [07:20:30] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.85, 2.31, 3.12 [07:20:49] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.278945595 secs [07:22:49] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.02535569668 secs [07:24:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.65, 3.88, 3.13 [07:28:03] PROBLEM - swiftac111 Current Load on swiftac111 is WARNING: WARNING - load average: 7.47, 5.50, 3.31 [07:30:00] RECOVERY - swiftac111 Current Load on swiftac111 is OK: OK - load average: 2.85, 4.59, 3.26 [07:30:33] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.09, 2.88, 3.02 [07:33:20] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.79, 4.01, 3.32 [07:35:15] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.48, 3.85, 3.35 [07:39:03] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.16, 3.28, 3.22 [08:04:01] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 6.46, 3.55, 2.03 [08:04:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 17.93, 7.88, 4.25 [08:07:54] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.03, 3.76, 2.82 [08:08:49] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.4907660484 secs [08:09:52] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.32, 3.24, 2.74 [08:12:30] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.16, 3.15, 3.57 [08:14:31] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.67, 2.31, 3.20 [08:16:49] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.00204500556 secs [08:22:32] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.263233304 secs [08:27:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.22, 4.20, 3.00 [08:30:01] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 2.15, 2.79, 3.90 [08:34:01] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.02, 1.94, 3.30 [08:38:32] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset -0.01332631707 secs [08:47:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.03, 2.10, 3.73 [08:51:47] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.41, 2.37, 3.40 [09:01:22] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.46, 3.98, 2.77 [09:05:12] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.84, 3.68, 2.93 [09:07:30] PROBLEM - cp33 PowerDNS Recursor on cp33 is CRITICAL: CRITICAL - Plugin timed out while executing system call [09:09:30] RECOVERY - cp33 PowerDNS Recursor on cp33 is OK: DNS OK: 0.174 seconds response time. miraheze.org returns 108.175.15.182,2607:f1c0:1800:26f::1,2607:f1c0:1800:8100::1,74.208.203.152 [09:10:55] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.83, 2.56, 2.77 [09:21:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.98, 2.26, 3.96 [09:27:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.67, 1.50, 3.10 [09:30:22] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.89, 4.35, 2.48 [09:30:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.70, 2.74, 2.29 [09:32:22] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 2.16, 3.61, 2.44 [09:32:30] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.70, 2.42, 2.24 [09:34:22] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.53, 2.54, 2.19 [09:40:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.90, 3.49, 2.79 [09:44:30] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.51, 2.66, 2.61 [09:58:09] PROBLEM - swiftac111 Current Load on swiftac111 is CRITICAL: CRITICAL - load average: 19.25, 10.01, 4.97 [10:02:02] PROBLEM - swiftac111 PowerDNS Recursor on swiftac111 is CRITICAL: CRITICAL - Plugin timed out while executing system call [10:02:07] PROBLEM - swiftac111 SSH on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:02:15] PROBLEM - swiftac111 Puppet on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [10:03:01] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [10:03:59] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 401 Unauthorized [10:04:06] PROBLEM - swiftproxy131 HTTP on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 401 Unauthorized [10:04:14] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 401 Unauthorized [10:06:11] PROBLEM - swiftac111 conntrack_table_size on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [10:06:54] PROBLEM - swiftac111 Swift Account Service on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:06:57] PROBLEM - swiftac111 NTP time on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:07:14] PROBLEM - swiftac111 ferm_active on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [10:07:53] RECOVERY - swiftproxy131 HTTP on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.023 second response time [10:07:56] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.038 second response time [10:08:13] RECOVERY - swiftproxy111 HTTP on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.019 second response time [10:08:52] RECOVERY - swiftac111 Swift Account Service on swiftac111 is OK: TCP OK - 0.001 second response time on 2a10:6740::6:202 port 6002 [10:09:02] RECOVERY - swiftac111 conntrack_table_size on swiftac111 is OK: OK: nf_conntrack is 0 % full [10:09:18] RECOVERY - swiftac111 NTP time on swiftac111 is OK: NTP OK: Offset -0.003201574087 secs [10:11:46] PROBLEM - swiftproxy131 HTTP on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 401 Unauthorized [10:12:04] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:12:13] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 401 Unauthorized [10:13:16] PROBLEM - swiftac111 Swift Account Service on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:16:08] PROBLEM - swiftac111 conntrack_table_size on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [10:16:53] PROBLEM - swiftac111 NTP time on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [10:18:20] PROBLEM - swiftac111 Disk Space on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [10:18:26] PROBLEM - swiftac111 Swift Container Service on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:20:23] RECOVERY - swiftac111 Swift Container Service on swiftac111 is OK: TCP OK - 0.001 second response time on 2a10:6740::6:202 port 6001 [10:22:30] RECOVERY - swiftac111 NTP time on swiftac111 is OK: NTP OK: Offset -0.002741008997 secs [10:23:01] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.032 second response time [10:23:09] RECOVERY - swiftproxy131 HTTP on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.034 second response time [10:23:23] RECOVERY - swiftac111 SSH on swiftac111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [10:23:23] RECOVERY - swiftac111 Disk Space on swiftac111 is OK: DISK OK - free space: / 112682 MB (91% inode=96%); [10:23:45] RECOVERY - swiftac111 Swift Account Service on swiftac111 is OK: TCP OK - 0.001 second response time on 2a10:6740::6:202 port 6002 [10:23:50] RECOVERY - swiftac111 conntrack_table_size on swiftac111 is OK: OK: nf_conntrack is 0 % full [10:23:59] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.093 second response time [10:24:13] RECOVERY - swiftproxy111 HTTP on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.015 second response time [10:27:01] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [10:27:06] PROBLEM - swiftproxy131 HTTP on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [10:27:48] PROBLEM - swiftac111 SSH on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:27:56] RECOVERY - swiftac111 ferm_active on swiftac111 is OK: OK ferm input default policy is set [10:28:07] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [10:28:12] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [10:28:47] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.01, 2.53, 1.63 [10:30:45] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.51, 2.77, 1.83 [10:31:01] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.036 second response time [10:31:06] RECOVERY - swiftproxy131 HTTP on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.040 second response time [10:31:46] RECOVERY - swiftac111 PowerDNS Recursor on swiftac111 is OK: DNS OK: 0.956 seconds response time. miraheze.org returns 109.228.51.216,217.174.247.33,2a00:da00:1800:326::1,2a00:da00:1800:328::1 [10:31:54] RECOVERY - swiftac111 SSH on swiftac111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [10:32:06] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.081 second response time [10:32:13] RECOVERY - swiftproxy111 HTTP on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.016 second response time [10:32:42] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.50, 3.97, 2.39 [10:34:40] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 2.11, 3.39, 2.38 [10:35:20] RECOVERY - swiftac111 Puppet on swiftac111 is OK: OK: Puppet is currently enabled, last run 21 seconds ago with 0 failures [10:55:07] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.72, 3.37, 2.17 [10:57:05] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.52, 3.00, 2.18 [10:58:54] PROBLEM - swiftac111 Current Load on swiftac111 is WARNING: WARNING - load average: 1.44, 1.90, 7.27 [11:00:52] RECOVERY - swiftac111 Current Load on swiftac111 is OK: OK - load average: 1.39, 1.73, 6.57 [11:29:30] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.95, 4.92, 2.95 [11:38:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.77, 3.70, 2.68 [11:40:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.15, 5.03, 3.34 [11:41:18] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.64, 3.60, 3.62 [11:42:30] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.21, 3.92, 3.14 [11:43:16] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.73, 2.64, 3.26 [11:44:30] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.74, 2.80, 2.82 [11:55:02] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.73, 3.51, 3.03 [11:57:00] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.94, 3.37, 3.04 [12:02:08] PROBLEM - swiftac111 Current Load on swiftac111 is CRITICAL: CRITICAL - load average: 28.56, 14.96, 7.30 [12:03:45] PROBLEM - swiftac111 Puppet on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:04:51] PROBLEM - swiftac111 PowerDNS Recursor on swiftac111 is CRITICAL: CRITICAL - Plugin timed out while executing system call [12:05:08] PROBLEM - swiftac111 SSH on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:05:57] PROBLEM - swiftproxy131 HTTP on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [12:06:03] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [12:06:18] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 401 Unauthorized [12:06:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.24, 2.63, 2.00 [12:06:48] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [12:07:34] PROBLEM - swiftac111 conntrack_table_size on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:08:54] PROBLEM - swiftac111 Swift Account Service on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:09:29] PROBLEM - swiftac111 Swift Container Service on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:11:26] RECOVERY - swiftac111 Swift Container Service on swiftac111 is OK: TCP OK - 0.001 second response time on 2a10:6740::6:202 port 6001 [12:11:53] PROBLEM - swiftac111 NTP time on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:12:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.48, 3.52, 2.68 [12:13:16] PROBLEM - swiftac111 Disk Space on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:13:24] PROBLEM - swiftac111 ferm_active on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:14:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.46, 4.32, 3.05 [12:18:30] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.08, 3.15, 2.92 [12:20:24] [02puppet] 07redbluegreenhat opened pull request 03#3127: add missing = sign - 13https://github.com/miraheze/puppet/pull/3127 [12:21:36] PROBLEM - swiftac111 Swift Container Service on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:23:33] RECOVERY - swiftac111 Swift Container Service on swiftac111 is OK: TCP OK - 0.001 second response time on 2a10:6740::6:202 port 6001 [12:23:39] RECOVERY - swiftac111 Swift Account Service on swiftac111 is OK: TCP OK - 0.001 second response time on 2a10:6740::6:202 port 6002 [12:24:13] RECOVERY - swiftproxy111 HTTP on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.027 second response time [12:24:16] RECOVERY - swiftac111 conntrack_table_size on swiftac111 is OK: OK: nf_conntrack is 0 % full [12:24:23] RECOVERY - swiftac111 Disk Space on swiftac111 is OK: DISK OK - free space: / 112671 MB (91% inode=96%); [12:24:43] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.044 second response time [12:25:02] RECOVERY - swiftac111 ferm_active on swiftac111 is OK: OK ferm input default policy is set [12:28:25] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.94, 4.94, 2.95 [12:29:11] RECOVERY - swiftproxy131 HTTP on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.020 second response time [12:29:14] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.029 second response time [12:31:07] RECOVERY - swiftac111 NTP time on swiftac111 is OK: NTP OK: Offset 0.001924723387 secs [12:33:15] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [12:33:17] PROBLEM - swiftproxy131 HTTP on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [12:33:47] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 503 Service Unavailable [12:34:15] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [12:35:12] PROBLEM - cp33 PowerDNS Recursor on cp33 is CRITICAL: CRITICAL - Plugin timed out while executing system call [12:37:13] RECOVERY - cp33 PowerDNS Recursor on cp33 is OK: DNS OK: 0.913 seconds response time. miraheze.org returns 108.175.15.182,2607:f1c0:1800:26f::1,2607:f1c0:1800:8100::1,74.208.203.152 [12:39:17] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 8.413 second response time [12:39:43] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.025 second response time [12:40:12] RECOVERY - swiftproxy111 HTTP on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.017 second response time [12:42:13] PROBLEM - swiftac111 Swift Container Service on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:43:08] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 401 Unauthorized [12:43:49] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 401 Unauthorized [12:44:13] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 401 Unauthorized [12:44:57] PROBLEM - swiftac111 conntrack_table_size on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:45:18] PROBLEM - swiftac111 ferm_active on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:45:46] PROBLEM - swiftac111 Swift Account Service on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:46:34] PROBLEM - swiftac111 Disk Space on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:46:53] PROBLEM - swiftac111 NTP time on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:49:03] RECOVERY - swiftac111 Disk Space on swiftac111 is OK: DISK OK - free space: / 112670 MB (91% inode=96%); [12:50:29] RECOVERY - swiftac111 Swift Container Service on swiftac111 is OK: TCP OK - 0.001 second response time on 2a10:6740::6:202 port 6001 [12:52:00] RECOVERY - swiftac111 Swift Account Service on swiftac111 is OK: TCP OK - 0.001 second response time on 2a10:6740::6:202 port 6002 [12:56:23] PROBLEM - swiftac111 Swift Account Service on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:00:02] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.80, 2.66, 3.86 [13:03:57] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.72, 1.67, 3.19 [13:04:52] PROBLEM - swiftac111 Disk Space on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:18:26] PROBLEM - swiftac111 Swift Container Service on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:18:44] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 4.18, 3.28, 2.06 [13:20:39]