[00:01:16] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 11.27, 6.32, 4.18 [00:01:16] miraheze/ManageWiki - Universal-Omega the build has errored. [00:01:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.35, 8.45, 7.87 [00:03:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.66, 7.13, 7.46 [00:03:48] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.34, 6.46, 3.87 [00:09:38] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 6.23, 7.12, 5.08 [00:10:34] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.35, 3.60, 3.80 [00:10:51] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.50, 3.63, 3.97 [00:11:34] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.58, 6.10, 4.95 [00:15:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.59, 8.59, 7.74 [00:16:30] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.75, 2.52, 3.25 [00:18:31] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.01, 2.59, 3.38 [00:21:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.36, 7.38, 7.58 [00:23:11] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [00:26:13] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.64, 3.24, 3.28 [00:27:03] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.39, 7.68, 6.21 [00:31:03] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.08, 7.26, 6.40 [00:31:58] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.39, 3.16, 3.25 [00:33:03] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.91, 5.62, 5.89 [00:33:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.88, 7.52, 7.38 [00:35:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.04, 7.25, 7.30 [00:37:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.17, 7.16, 7.24 [00:39:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.67, 7.68, 7.45 [00:43:03] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 12.24, 7.97, 6.40 [00:43:35] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.83, 3.37, 2.60 [00:44:00] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 5.57, 4.32, 3.37 [00:44:05] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.05, 3.57, 3.26 [00:45:35] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.30, 2.94, 2.54 [00:45:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.08, 7.07, 7.18 [00:46:03] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 3.29, 3.30, 3.18 [00:47:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.65, 6.57, 6.98 [00:49:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.76, 7.62, 7.28 [00:49:54] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 2.55, 3.77, 3.48 [00:49:59] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.04, 4.15, 3.53 [00:51:03] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.45, 6.48, 6.66 [00:51:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.48, 7.13, 7.15 [00:51:51] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.29, 3.28, 3.33 [00:51:57] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.03, 3.76, 3.47 [00:52:32] RECOVERY - wiki.nj.cn.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.nj.cn.eu.org reverse DNS resolves to cp22.miraheze.org - CNAME OK [00:53:56] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.00, 3.07, 3.24 [00:55:46] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.67, 3.64, 3.47 [00:59:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.65, 7.61, 7.36 [00:59:52] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 6.65, 6.97, 6.80 [01:01:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.06, 7.03, 7.18 [01:01:38] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.63, 3.22, 3.34 [01:01:49] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.54, 6.15, 6.52 [01:05:37] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.29, 5.88, 6.67 [01:09:24] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.57, 3.48, 3.45 [01:13:17] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.04, 2.83, 3.20 [01:15:55] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.79, 3.73, 3.23 [01:17:13] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.63, 8.13, 7.12 [01:17:19] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.58, 3.17, 3.23 [01:17:55] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.40, 3.19, 3.09 [01:18:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.07, 7.33, 6.92 [01:19:09] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.63, 6.67, 6.70 [01:19:16] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.55, 2.90, 3.12 [01:21:55] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.79, 5.05, 3.88 [01:22:10] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.20, 7.74, 7.22 [01:24:05] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.17, 8.01, 7.37 [01:26:00] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.93, 7.48, 7.24 [01:29:55] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.31, 3.50, 3.76 [01:31:55] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.91, 3.84, 3.82 [01:37:38] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.59, 7.57, 7.24 [01:39:38] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.20, 6.99, 7.07 [01:39:55] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.74, 3.52, 3.84 [01:43:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.31, 3.73, 3.37 [01:43:55] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.30, 3.53, 3.73 [01:45:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.18, 8.26, 7.50 [01:45:55] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.66, 3.85, 3.85 [01:47:09] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.93, 3.02, 3.16 [01:47:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.41, 7.34, 7.24 [01:49:56] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.59, 3.70, 3.74 [01:51:55] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.64, 3.68, 3.73 [01:55:55] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.16, 2.73, 3.32 [02:06:42] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.49, 3.41, 3.26 [02:07:03] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.98, 6.23, 3.54 [02:11:03] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.25, 5.23, 3.86 [02:11:38] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.72, 7.33, 7.04 [02:12:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.20, 2.77, 3.04 [02:13:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.40, 7.33, 7.10 [02:22:49] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.90, 3.69, 3.06 [02:26:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.27, 3.32, 3.08 [02:29:37] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.95, 6.37, 6.75 [02:35:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.54, 6.62, 6.66 [02:37:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.37, 7.03, 6.77 [02:39:37] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.14, 6.44, 6.58 [02:45:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.80, 7.29, 6.86 [02:53:37] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.36, 6.54, 6.75 [03:01:15] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.95, 4.16, 3.23 [03:03:55] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.00, 3.47, 2.84 [03:07:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.52, 3.66, 3.42 [03:09:09] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.05, 3.08, 3.24 [03:09:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.80, 6.97, 6.87 [03:13:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.77, 7.94, 7.31 [03:15:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.84, 7.24, 7.11 [03:15:55] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.49, 3.19, 3.48 [03:17:55] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.21, 2.53, 3.20 [03:21:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.07, 8.82, 7.75 [03:27:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.46, 7.38, 7.44 [03:41:37] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.72, 6.16, 6.78 [03:48:46] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.03, 3.45, 3.23 [03:49:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.05, 6.95, 6.79 [03:50:41] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.93, 2.96, 3.08 [03:51:37] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.17, 6.60, 6.68 [04:03:12] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.77, 3.82, 3.51 [04:03:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.14, 6.58, 6.55 [04:05:07] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.88, 4.55, 3.83 [04:07:02] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.96, 3.90, 3.67 [04:07:37] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.05, 6.29, 6.43 [04:10:52] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.03, 2.83, 3.27 [04:48:21] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.65, 5.14, 3.87 [04:54:07] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.52, 3.87, 3.67 [04:57:05] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.86, 3.77, 3.13 [04:57:57] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.14, 3.07, 3.39 [05:01:05] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.55, 2.94, 2.96 [05:08:42] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.95, 3.41, 3.30 [05:10:37] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.67, 2.95, 3.13 [05:16:23] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.94, 3.49, 3.29 [05:18:18] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.79, 3.16, 3.18 [05:24:11] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.94, 6.77, 6.44 [05:26:06] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.45, 6.51, 6.39 [05:34:46] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.72, 6.90, 6.61 [05:36:41] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.33, 6.57, 6.54 [05:53:29] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.41, 4.44, 2.88 [05:56:52] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.24, 7.60, 6.86 [05:57:26] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.41, 3.98, 3.08 [05:58:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.99, 7.19, 6.79 [06:00:43] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.42, 6.54, 6.61 [06:01:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.41, 3.18, 2.95 [06:11:35] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.53, 2.89, 2.19 [06:13:35] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.03, 2.90, 2.28 [06:17:06] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.25, 7.00, 6.76 [06:20:56] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.79, 7.52, 7.01 [06:22:51] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.04, 7.17, 6.93 [06:24:46] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.85, 6.54, 6.72 [06:36:56] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.04, 3.53, 2.96 [06:40:46] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.38, 2.99, 2.92 [06:43:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.67, 6.96, 6.72 [06:43:23] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/3d3a505fa3e4...3e34b003117e [06:43:24] [02miraheze/ssl] 07MirahezeSSLBot 033e34b00 - Bot: Add SSL cert for glitchcity.wiki [06:44:57] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.93, 7.57, 6.97 [06:45:03] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.80, 5.27, 3.09 [06:46:52] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.04, 7.52, 7.04 [06:47:03] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.72, 5.57, 3.48 [06:52:01] PROBLEM - cloud13 Puppet on cloud13 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [06:55:42] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.48, 4.37, 3.45 [06:55:52] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1078236401 secs [06:56:29] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.04, 8.57, 7.60 [06:57:39] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.30, 3.48, 3.23 [06:58:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.42, 7.98, 7.49 [06:59:36] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.35, 2.82, 3.02 [07:00:11] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.45, 2.81, 1.74 [07:00:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.91, 8.13, 7.60 [07:02:16] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.73, 7.27, 7.35 [07:04:07] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.39, 2.86, 2.08 [07:04:12] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.96, 8.87, 7.91 [07:08:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.81, 7.63, 7.66 [07:15:46] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.54, 7.34, 7.36 [07:19:41] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.0905123055 secs [07:20:48] RECOVERY - cloud13 Puppet on cloud13 is OK: OK: Puppet is currently enabled, last run 55 seconds ago with 0 failures [07:21:34] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.04, 7.85, 7.58 [07:27:18] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.09, 10.31, 11.99 [07:27:20] PROBLEM - www.lab612.at - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.lab612.at All nameservers failed to answer the query. [07:27:21] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 3.84, 5.69, 6.71 [07:29:17] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.09, 11.83, 12.34 [07:29:36] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 3.79, 4.85, 2.90 [07:31:34] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 1.63, 3.59, 2.67 [07:33:32] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.85, 2.63, 2.42 [07:38:59] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.92, 6.85, 6.72 [07:42:51] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.99, 6.54, 6.62 [07:59:16] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.49, 3.09, 2.12 [08:01:16] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.51, 2.40, 1.98 [08:05:29] PROBLEM - with.cpt-ra.bid - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query cpt-ra.bid. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:07:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.06, 11.21, 11.83 [08:16:15] PROBLEM - cp22 Current Load on cp22 is WARNING: WARNING - load average: 3.71, 2.85, 1.76 [08:16:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.73, 6.92, 6.43 [08:18:15] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.72, 2.36, 1.71 [08:19:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.05, 11.78, 11.58 [08:19:52] PROBLEM - db131 Backups SQL on db131 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864175 seconds old and 1879 bytes [08:26:12] RECOVERY - www.lab612.at - reverse DNS on sslhost is OK: SSL OK - www.lab612.at reverse DNS resolves to cp22.miraheze.org - CNAME OK [08:26:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.78, 6.80, 6.67 [08:31:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.05, 11.92, 11.99 [08:33:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.97, 12.71, 12.25 [08:34:14] RECOVERY - with.cpt-ra.bid - reverse DNS on sslhost is OK: SSL OK - with.cpt-ra.bid reverse DNS resolves to cp23.miraheze.org - CNAME OK [08:39:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.99, 11.16, 11.92 [08:49:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.06, 11.75, 11.77 [08:50:13] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.14, 7.46, 6.70 [08:51:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.49, 11.74, 11.77 [08:52:09] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.38, 7.03, 6.63 [08:53:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.67, 12.34, 12.00 [08:54:04] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.84, 6.25, 6.39 [08:55:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.89, 11.60, 11.78 [08:59:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.61, 12.00, 11.84 [09:22:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.27, 7.35, 6.65 [09:24:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.21, 6.87, 6.56 [09:26:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.71, 6.31, 6.38 [09:41:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.01, 10.94, 11.94 [09:47:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.92, 10.77, 11.45 [09:49:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.57, 10.33, 11.21 [09:51:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.82, 11.71, 11.61 [09:53:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.80, 11.09, 11.41 [09:57:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.49, 11.48, 11.48 [09:59:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.31, 11.22, 11.37 [10:03:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 15.19, 11.89, 11.50 [10:13:32] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.79, 7.06, 6.56 [10:27:02] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.29, 6.29, 6.70 [10:34:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.57, 7.03, 6.71 [10:36:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.28, 6.90, 6.70 [10:38:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.22, 7.48, 6.94 [10:40:45] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1569522023 secs [10:40:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.72, 7.27, 6.90 [10:42:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.55, 7.43, 6.98 [10:44:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.61, 6.24, 6.59 [10:56:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.97, 7.83, 7.07 [11:04:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.32, 6.23, 6.63 [11:08:45] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.09015223384 secs [11:23:20] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.73, 5.84, 3.07 [11:25:20] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.99, 5.70, 3.36 [11:30:24] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 7.32, 8.22, 4.37 [11:35:53] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.08, 7.22, 6.68 [11:36:17] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.93, 3.28, 3.34 [11:36:28] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.84, 3.88, 3.02 [11:37:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.26, 6.77, 6.58 [11:38:24] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.55, 3.78, 3.09 [11:41:15] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:43:18] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 5.066 second response time [11:44:12] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.12, 3.38, 3.12 [11:45:34] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.24, 7.39, 6.76 [11:47:30] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.58, 7.34, 6.81 [11:49:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.33, 7.88, 7.08 [11:51:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.95, 7.05, 6.87 [11:53:17] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.14, 6.62, 6.72 [12:00:30] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [12:09:43] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.26, 7.67, 7.02 [12:11:39] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 8.00, 7.54, 7.03 [12:17:22] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:21:18] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.74, 6.54, 6.79 [12:27:52] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 2.800 second response time [12:30:16] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.36, 2.99, 2.03 [12:32:14] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.58, 2.36, 1.92 [12:35:39] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:37:20] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.76, 6.30, 3.12 [12:39:20] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.14, 6.83, 3.73 [12:39:42] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.92, 7.48, 7.12 [12:39:45] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 6.542 second response time [12:41:20] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.04, 5.10, 3.48 [12:44:15] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 4.42, 3.14, 1.94 [12:46:15] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.55, 2.53, 1.86 [12:47:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.59, 3.39, 3.05 [12:47:44] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:49:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.05, 7.54, 7.17 [12:53:19] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.86, 3.21, 3.09 [12:58:59] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.91, 7.26, 7.54 [13:00:27] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 8.194 second response time [13:00:45] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1750264764 secs [13:06:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.70, 5.74, 6.80 [13:08:45] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.08338701725 secs [13:13:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.69, 7.10, 7.01 [13:18:17] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.42, 3.22, 2.90 [13:19:31] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.60, 7.53, 7.13 [13:21:27] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.73, 7.52, 7.18 [13:22:09] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.39, 3.25, 2.98 [13:24:29] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:30:43] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 1.042 second response time [13:31:05] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.15, 7.26, 7.13 [13:33:01] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.03, 6.89, 7.01 [13:33:47] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.78, 3.38, 3.10 [13:35:43] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.27, 3.32, 3.11 [13:36:59] PROBLEM - miraheze.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'miraheze.wiki' expires in 15 day(s) (Tue 13 Jun 2023 13:28:46 GMT +0000). [13:40:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.50, 7.67, 7.29 [13:48:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.06, 7.92, 7.88 [13:50:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 7.47, 8.07, 7.96 [13:52:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.53, 7.20, 7.66 [13:56:22] PROBLEM - index.pdcommunity.ir - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - index.pdcommunity.ir All nameservers failed to answer the query. [14:00:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.24, 7.40, 7.47 [14:02:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.83, 6.48, 7.13 [14:04:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.01, 5.84, 6.80 [14:14:45] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1553801 secs [14:14:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.87, 7.35, 7.01 [14:16:45] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.07993203402 secs [14:16:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.12, 7.22, 7.01 [14:18:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 7.82, 8.09, 7.39 [14:20:24] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:20:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.88, 7.63, 7.32 [14:22:24] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 2.824 second response time [14:24:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.43, 8.41, 7.67 [14:25:05] RECOVERY - index.pdcommunity.ir - reverse DNS on sslhost is OK: SSL OK - index.pdcommunity.ir reverse DNS resolves to cp22.miraheze.org - CNAME OK [14:26:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.26, 7.68, 7.50 [14:30:45] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset -0.1301112771 secs [14:34:45] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.07950556278 secs [14:38:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.11, 7.43, 7.20 [14:40:51] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.59, 7.21, 7.14 [14:52:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.32, 8.53, 7.64 [14:54:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.87, 7.44, 7.35 [14:56:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.18, 8.25, 7.68 [14:58:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.19, 7.61, 7.52 [15:00:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.69, 8.43, 7.83 [15:01:38] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.78, 3.23, 2.91 [15:02:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.84, 7.63, 7.60 [15:03:38] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.38, 3.32, 2.99 [15:07:38] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.60, 3.41, 3.08 [15:10:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.31, 7.46, 7.39 [15:11:20] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.66, 5.24, 2.78 [15:11:38] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.23, 3.29, 3.12 [15:12:51] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.61, 7.34, 7.37 [15:13:20] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.42, 5.04, 3.05 [15:15:38] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.05, 3.62, 3.29 [15:17:38] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.21, 3.37, 3.24 [15:18:11] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:18:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.66, 8.34, 7.68 [15:20:17] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 7.920 second response time [15:20:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.20, 7.88, 7.59 [15:26:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 13.83, 9.46, 8.15 [15:29:16] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.98, 3.23, 2.13 [15:30:30] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [15:31:16] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.39, 2.57, 2.02 [15:59:36] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.20, 3.56, 2.76 [16:00:30] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [16:01:33] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.96, 3.34, 2.78 [16:04:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.77, 7.46, 7.97 [16:05:38] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.84, 3.16, 2.89 [16:06:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.08, 8.31, 8.20 [16:07:38] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.28, 3.08, 2.89 [16:12:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.34, 7.46, 7.97 [16:20:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.56, 7.76, 7.86 [16:22:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.46, 7.21, 7.64 [16:25:21] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.92, 5.54, 2.80 [16:26:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.58, 8.03, 7.85 [16:27:20] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.43, 5.00, 2.97 [16:28:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.26, 7.89, 7.84 [16:34:17] PROBLEM - isv.orain.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'orain.org' expires in 15 day(s) (Tue 13 Jun 2023 16:26:38 GMT +0000). [16:37:39] PROBLEM - wikipedia.yogfront.ooo - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wikipedia.yogfront.ooo All nameservers failed to answer the query. [16:38:16] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 6.16, 2.88, 1.45 [16:40:14] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 1.65, 2.24, 1.38 [16:42:22] PROBLEM - allthetropes.orain.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'orain.org' expires in 15 day(s) (Tue 13 Jun 2023 16:26:38 GMT +0000). [16:42:31] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.59, 3.40, 2.84 [16:42:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.63, 6.02, 6.71 [16:44:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.42, 3.03, 2.78 [16:45:38] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.57, 3.21, 2.96 [16:47:39] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.02, 3.48, 3.09 [16:49:38] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.29, 3.41, 3.11 [16:50:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.91, 7.40, 6.93 [16:50:58] PROBLEM - orain.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'orain.org' expires in 15 day(s) (Tue 13 Jun 2023 16:26:38 GMT +0000). [16:52:04] PROBLEM - poserdazfreebies.orain.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'orain.org' expires in 15 day(s) (Tue 13 Jun 2023 16:26:38 GMT +0000). [16:52:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.86, 7.30, 6.94 [16:53:38] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.03, 3.66, 3.28 [16:55:38] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.18, 3.47, 3.25 [16:56:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.83, 8.28, 7.37 [16:57:38] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.03, 3.32, 3.23 [17:03:38] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.05, 3.75, 3.42 [17:04:05] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.66, 3.72, 2.92 [17:04:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.85, 7.55, 7.43 [17:05:38] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 2.75, 3.44, 3.35 [17:07:38] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.42, 3.04, 3.21 [17:07:59] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.71, 4.31, 3.30 [17:10:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 13.12, 8.49, 7.71 [17:14:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.37, 7.71, 7.63 [17:15:47] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.81, 3.99, 3.61 [17:17:03] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:18:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.03, 7.31, 7.44 [17:19:41] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.48, 2.89, 3.26 [17:22:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.64, 6.63, 7.16 [17:23:20] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.03, 8.25, 4.66 [17:25:20] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.63, 5.61, 4.12 [17:29:34] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 6.682 second response time [17:30:30] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [17:32:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.08, 6.35, 6.78 [17:36:44] PROBLEM - wikipedia.yogfront.ooo - reverse DNS on sslhost is WARNING: Timeout: The DNS operation timed out after 5.4064621925354 seconds [17:43:25] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:45:30] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 8.112 second response time [17:50:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.53, 7.62, 6.98 [17:51:07] PROBLEM - matomo121 HTTPS on matomo121 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 358 bytes in 0.010 second response time [17:51:52] PROBLEM - mail121 webmail.miraheze.org HTTPS on mail121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:51:56] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.99, 7.80, 5.32 [17:52:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 8.00, 7.65, 7.05 [17:53:52] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.66, 3.57, 2.96 [17:53:52] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.85, 5.41, 4.73 [17:55:49] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.04, 2.87, 2.77 [18:00:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.80, 7.61, 7.17 [18:02:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.75, 7.75, 7.28 [18:06:12] RECOVERY - wikipedia.yogfront.ooo - reverse DNS on sslhost is OK: SSL OK - wikipedia.yogfront.ooo reverse DNS resolves to cp22.miraheze.org - CNAME OK [18:08:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.20, 6.14, 6.73 [18:11:19] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.12, 2.97, 2.35 [18:12:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.42, 6.82, 6.88 [18:13:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.53, 3.18, 2.51 [18:14:19] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:15:19] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.14, 3.08, 2.55 [18:16:19] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 2.608 second response time [18:18:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.76, 7.67, 7.19 [18:20:15] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.69, 3.83, 3.05 [18:20:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.72, 7.77, 7.29 [18:22:12] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.34, 3.24, 2.93 [18:24:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.70, 3.91, 3.17 [18:28:08] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.51, 3.29, 3.08 [18:28:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 13.41, 8.87, 7.75 [18:30:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.48, 7.71, 7.46 [18:31:25] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 14.43, 10.50, 4.97 [18:32:11] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.49, 6.25, 4.31 [18:34:20] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.89, 4.63, 3.70 [18:37:18] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 0.96, 3.99, 3.74 [18:38:18] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 5.87, 3.34, 1.94 [18:41:16] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.99, 2.53, 3.21 [18:41:41] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.98, 3.31, 2.82 [18:42:12] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.25, 3.12, 2.19 [18:43:38] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.42, 2.92, 2.74 [18:43:48] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.55, 5.75, 5.70 [18:44:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.12, 5.94, 6.59 [18:48:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.54, 6.78, 6.79 [18:49:39] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.99, 3.84, 3.94 [19:02:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.63, 7.53, 7.25 [19:04:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.70, 6.95, 7.05 [19:05:19] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.24, 3.15, 3.38 [19:06:51] PROBLEM - wiki.susqu.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.susqu.org' expires in 15 day(s) (Tue 13 Jun 2023 18:53:54 GMT +0000). [19:06:52] PROBLEM - wiki.rsf.world - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.rsf.world' expires in 15 day(s) (Tue 13 Jun 2023 18:58:23 GMT +0000). [19:07:28] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3e34b003117e...083cb7a2194b [19:07:30] [02miraheze/ssl] 07MirahezeSSLBot 03083cb7a - Bot: Update SSL cert for wiki.susqu.org [19:07:44] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/083cb7a2194b...d8559aa881a7 [19:07:47] [02miraheze/ssl] 07MirahezeSSLBot 03d8559aa - Bot: Update SSL cert for wiki.rsf.world [19:08:01] PROBLEM - wiki.mc.marochuru.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.mc.marochuru.net' expires in 15 day(s) (Tue 13 Jun 2023 18:56:42 GMT +0000). [19:08:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.67, 7.40, 7.15 [19:10:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.53, 7.06, 7.06 [19:11:08] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.39, 3.76, 3.30 [19:11:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.92, 3.43, 3.40 [19:12:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.75, 7.61, 7.26 [19:13:05] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.04, 3.10, 3.11 [19:15:19] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d8559aa881a7...2f492802697b [19:15:22] [02miraheze/ssl] 07MirahezeSSLBot 032f49280 - Bot: Update SSL cert for wiki.mc.marochuru.net [19:15:35] PROBLEM - db112 MariaDB Connections on db112 is WARNING: Warning connection usage: 82.4%Current connections: 412 [19:17:19] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 1.95, 3.12, 3.32 [19:23:35] PROBLEM - db112 MariaDB Connections on db112 is CRITICAL: Critical connection usage: 91.2%Current connections: 456 [19:25:35] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.59, 3.72, 3.20 [19:27:32] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.71, 3.03, 3.01 [19:35:41] RECOVERY - wiki.susqu.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.susqu.org' will expire on Sat 26 Aug 2023 18:07:21 GMT +0000. [19:36:28] RECOVERY - wiki.rsf.world - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.rsf.world' will expire on Sat 26 Aug 2023 18:07:37 GMT +0000. [19:36:31] RECOVERY - wiki.mc.marochuru.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.mc.marochuru.net' will expire on Sat 26 Aug 2023 18:15:12 GMT +0000. [19:37:22] PROBLEM - phab121 phabricator.miraheze.org HTTPS on phab121 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 4167 bytes in 0.110 second response time [19:37:36] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.12, 3.43, 3.07 [19:38:24] PROBLEM - phab121 phab.miraheze.wiki HTTPS on phab121 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 500 Internal Server Error [19:39:15] PROBLEM - db112 MariaDB on db112 is CRITICAL: Too many connections [19:39:33] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.10, 3.97, 3.32 [19:41:30] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.00, 3.18, 3.11 [19:45:03] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.93, 4.25, 3.32 [19:54:46] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.47, 3.94, 3.81 [19:54:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.67, 7.02, 7.87 [19:58:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.10, 7.11, 7.66 [20:00:24] RECOVERY - phab121 phab.miraheze.wiki HTTPS on phab121 is OK: HTTP OK: Status line output matched "HTTP/1.1 200" - 17684 bytes in 0.237 second response time [20:00:39] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.62, 2.64, 3.31 [20:00:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.12, 6.59, 7.40 [20:00:51] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [20:01:14] RECOVERY - db112 MariaDB on db112 is OK: Uptime: 5664772 Threads: 43 Questions: 3105333023 Slow queries: 38104185 Opens: 678724 Open tables: 2536 Queries per second avg: 548.183 [20:01:22] RECOVERY - phab121 phabricator.miraheze.org HTTPS on phab121 is OK: HTTP OK: HTTP/1.1 200 OK - 18991 bytes in 0.211 second response time [20:01:35] RECOVERY - db112 MariaDB Connections on db112 is OK: OK connection usage: 8.6%Current connections: 43 [20:05:51] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [20:06:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.50, 6.69, 7.06 [20:08:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.69, 6.58, 6.97 [20:14:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.87, 5.80, 6.55 [20:20:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.34, 7.99, 7.22 [20:24:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.86, 3.74, 3.25 [20:26:39] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.66, 3.08, 3.07 [20:30:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.15, 7.94, 7.71 [20:33:36] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.09, 4.21, 3.45 [20:34:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.35, 7.92, 7.74 [20:37:29] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.81, 3.83, 3.50 [20:41:24] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.82, 2.96, 3.23 [20:42:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.32, 7.72, 7.76 [20:56:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.72, 7.73, 7.55 [20:58:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.21, 7.48, 7.48 [21:04:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.01, 7.58, 7.45 [21:06:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.72, 6.81, 7.18 [21:18:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.47, 7.41, 7.16 [21:22:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.99, 7.55, 7.29 [21:25:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.45, 3.40, 2.73 [21:25:36] PROBLEM - db112 MariaDB Connections on db112 is WARNING: Warning connection usage: 86.4%Current connections: 432 [21:27:07] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.67, 4.06, 3.06 [21:29:04] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.46, 3.44, 2.95 [21:31:01] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 3.11, 3.29, 2.95 [21:35:36] PROBLEM - db112 MariaDB Connections on db112 is CRITICAL: Critical connection usage: 93.2%Current connections: 466 [21:36:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.22, 3.85, 3.33 [21:40:21] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.13, 3.26, 3.22 [21:40:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.61, 7.90, 7.37 [21:44:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.23, 7.61, 7.45 [21:47:23] PROBLEM - phab121 phabricator.miraheze.org HTTPS on phab121 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 4167 bytes in 0.091 second response time [21:48:41] PROBLEM - phab121 phab.miraheze.wiki HTTPS on phab121 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 500 Internal Server Error [21:49:12] PROBLEM - biblestrength.net - reverse DNS on sslhost is UNKNOWN: [21:49:36] PROBLEM - db112 MariaDB on db112 is CRITICAL: Too many connections [21:51:01] PROBLEM - db112 MariaDB Connections on db112 is UNKNOWN: [21:51:24] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.94, 8.14, 7.63 [21:51:55] PROBLEM - mon141 grafana.miraheze.org HTTPS on mon141 is CRITICAL: CRITICAL - Socket timeout after 10 seconds PROBLEM - ns1 Auth DNS on ns1 is CRITICAL: CRITICAL - Plugin timed out while executing system call PROBLEM - ns1 Auth DNS on ns1 is CRITICAL: CRITICAL - Plugin timed out while executing system call [21:52:22] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.40, 6.14, 3.29 [21:52:47] PROBLEM - mon141 Current Load on mon141 is CRITICAL: CRITICAL - load average: 46.44, 28.26, 12.38 [21:53:19] PROBLEM - db112 MariaDB Connections on db112 is CRITICAL: PHP Warning: mysqli_real_connect(): (08004/1040): Too many connections in /usr/lib/nagios/plugins/check_mysql_connections.php on line 66Warning: mysqli_real_connect(): (08004/1040): Too many connections in /usr/lib/nagios/plugins/check_mysql_connections.php on line 66Connection failed: Too many connections [21:53:24] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.92, 7.04, 7.30 [21:53:34] RECOVERY - ns1 Auth DNS on ns1 is OK: DNS OK: 0.239 seconds response time. miraheze.org returns 109.228.51.216,217.174.247.33,2a00:da00:1800:326::1,2a00:da00:1800:328::1 [21:53:39] RECOVERY - mon141 grafana.miraheze.org HTTPS on mon141 is OK: HTTP OK: HTTP/1.1 200 OK - 40891 bytes in 0.139 second response time [21:54:17] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.83, 4.96, 3.22 [21:55:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 15.69, 10.13, 8.38 [22:01:08] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.92, 7.46, 7.73 [22:14:25] PROBLEM - mon141 Current Load on mon141 is WARNING: WARNING - load average: 0.44, 0.79, 3.70 [22:14:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.92, 7.45, 7.46 [22:16:23] RECOVERY - mon141 Current Load on mon141 is OK: OK - load average: 0.27, 0.61, 3.28 [22:17:45] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.97, 3.66, 3.29 [22:18:31] PROBLEM - biblestrength.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for biblestrength.net could not be found [22:18:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.56, 7.47, 7.47 [22:19:42] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.58, 3.19, 3.15 [22:22:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 12.13, 8.90, 7.97 [22:23:38] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.07, 3.80, 3.45 [22:25:35] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.67, 3.25, 3.28 [22:26:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.04, 7.30, 7.52 [22:29:32] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.17, 3.47, 3.38 [22:31:29] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.12, 3.34, 3.34 [22:34:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.16, 7.14, 7.22 [22:36:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.95, 6.56, 6.99 [22:40:20] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.64, 4.68, 3.90 [22:42:17] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.73, 3.58, 3.59 [22:44:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.43, 6.29, 6.78 [22:46:11] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.53, 2.77, 3.27 [22:48:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.82, 6.81, 6.87 [22:50:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.01, 7.49, 7.08 [22:52:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.50, 7.08, 6.99 [22:56:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.31, 7.45, 7.14 [22:57:20] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.73, 5.34, 2.65 [22:58:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.41, 6.96, 7.00 [22:58:53] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.71, 10.65, 11.81 [22:59:20] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.43, 5.89, 3.22 [23:00:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.07, 7.23, 7.08 [23:02:51] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.52, 12.08, 12.10 [23:04:50] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.13, 4.25, 3.64 [23:04:50] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.13, 4.25, 3.64 [23:04:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.25, 7.16, 7.16 [23:06:24] RECOVERY - phab121 phab.miraheze.wiki HTTPS on phab121 is OK: HTTP OK: Status line output matched "HTTP/1.1 200" - 17684 bytes in 0.273 second response time [23:06:28] RECOVERY - mail121 webmail.miraheze.org HTTPS on mail121 is OK: HTTP OK: HTTP/1.1 200 OK - 6281 bytes in 0.877 second response time [23:06:47] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.67, 3.67, 3.50 [23:06:52] RECOVERY - db112 MariaDB Connections on db112 is OK: OK connection usage: 15.8%Current connections: 79 [23:07:14] RECOVERY - db112 MariaDB on db112 is OK: Uptime: 5675932 Threads: 40 Questions: 3106482659 Slow queries: 38126657 Opens: 678739 Open tables: 2540 Queries per second avg: 547.307 [23:07:22] RECOVERY - phab121 phabricator.miraheze.org HTTPS on phab121 is OK: HTTP OK: HTTP/1.1 200 OK - 18991 bytes in 0.230 second response time [23:07:30] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [23:08:27] PROBLEM - db112 Disk Space on db112 is WARNING: DISK WARNING - free space: / 9131 MB (6% inode=99%); [23:08:44] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.37, 4.58, 3.83 [23:09:15] RECOVERY - matomo121 HTTPS on matomo121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.306 second response time [23:10:26] RECOVERY - db112 Disk Space on db112 is OK: DISK OK - free space: / 15272 MB (11% inode=99%); [23:10:41] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.73, 3.38, 3.48 [23:12:30] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [23:12:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.37, 8.26, 7.60 [23:14:35] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.33, 3.05, 3.33 [23:14:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.06, 7.39, 7.36 [23:16:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 14.28, 9.26, 7.99 [23:17:04] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.73, 3.66, 2.94 [23:19:00] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 3.38, 3.33, 2.89 [23:24:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.99, 7.46, 7.74 [23:34:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.74, 8.06, 7.77 [23:36:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.59, 7.51, 7.61 [23:38:04] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.99, 3.55, 3.08 [23:38:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.77, 7.82, 7.69 [23:40:32] PROBLEM - wiki.litek.top - reverse DNS on sslhost is UNKNOWN: [23:41:10] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.32, 2.83, 2.90 [23:44:58] PROBLEM - www.electrowiki.org - reverse DNS on sslhost is UNKNOWN: PROBLEM - o.kaoyu666.cn - reverse DNS on sslhost is UNKNOWN: [23:46:16] PROBLEM - mon141 Current Load on mon141 is CRITICAL: CRITICAL - load average: 22.56, 21.00, 10.65 [23:47:08] [02mw-config] 07brandon-wm opened pull request 03#5238: T10794: Add OOJSPlus - 13https://github.com/miraheze/mw-config/pull/5238 [23:47:26] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.87, 7.55, 7.55 [23:48:00] miraheze/mw-config - brandon-wm the build passed. [23:49:22] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.21, 7.63, 7.56 [23:50:49] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.15, 3.25, 2.99 [23:51:18] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.01, 7.26, 7.46 [23:53:13] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.41, 3.34, 3.06 [23:55:56] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.23, 3.58, 3.19 [23:56:48] [02mediawiki] 07brandon-wm opened pull request 03#15266: T10794: Add Extension:OOJSPlus - 13https://github.com/miraheze/mediawiki/pull/15266 [23:57:25] [02mediawiki] 07brandon-wm synchronize pull request 03#15266: T10794: Add Extension:OOJSPlus - 13https://github.com/miraheze/mediawiki/pull/15266 [23:57:40] [02mediawiki] 07brandon-wm synchronize pull request 03#15266: T10794: Add Extension:OOJSPlus - 13https://github.com/miraheze/mediawiki/pull/15266 [23:59:01] PROBLEM - legacygt.wiki - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.denby.tech - reverse DNS on sslhost is UNKNOWN: PROBLEM - tl.awiki.org - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.umbranet.mandelbrotgames.com - reverse DNS on sslhost is UNKNOWN: exceeded.> PROBLEM - wiki.teessidehackspace.org.uk - reverse DNS on sslhost is UNKNOWN: PROBLEM - rct.wiki - reverse DNS on sslhost is UNKNOWN: PROBLEM - obeymewiki.com - reverse DNS on sslhost is UNKNOWN: PROBLEM - pt.polandbal [23:59:02] .com - reverse DNS on sslhost is UNKNOWN: