[00:02:58] PROBLEM - matomo121 HTTPS on matomo121 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [00:03:16] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 15.75, 8.15, 4.16 [00:05:00] RECOVERY - matomo121 HTTPS on matomo121 is OK: HTTP OK: HTTP/2 200 - 552 bytes in 6.333 second response time [00:12:03] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.95, 6.57, 6.40 [00:14:03] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.31, 6.10, 6.24 [00:19:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.83, 8.28, 7.67 [00:20:34] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.88, 9.79, 8.39 [00:21:16] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 2.15, 6.02, 7.79 [00:21:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.38, 7.87, 7.60 [00:22:03] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.74, 10.12, 8.79 [00:22:29] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 9.59, 9.67, 8.52 [00:23:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 7.75, 8.01, 7.70 [00:24:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 14.57, 12.03, 9.62 [00:25:16] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 2.25, 3.91, 6.51 [00:25:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.93, 11.21, 9.47 [00:25:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.39, 7.60, 7.60 [00:26:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.25, 11.08, 9.47 [00:26:20] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.98, 10.82, 9.24 [00:28:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 8.95, 10.69, 9.55 [00:28:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.73, 11.59, 10.03 [00:29:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.33, 11.38, 9.90 [00:32:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.77, 11.05, 9.95 [00:32:03] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.80, 6.66, 6.31 [00:32:24] PROBLEM - os131 Current Load on os131 is CRITICAL: LOAD CRITICAL - total load average: 4.68, 3.71, 2.79 [00:32:39] PROBLEM - mw142 Current Load on mw142 is WARNING: LOAD WARNING - total load average: 10.35, 9.08, 7.41 [00:32:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.29, 11.18, 10.17 [00:33:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.92, 11.54, 10.28 [00:34:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.81, 11.30, 10.17 [00:34:03] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.77, 6.72, 6.37 [00:34:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.35, 11.02, 10.22 [00:35:21] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.19, 3.64, 2.92 [00:35:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.77, 11.64, 10.44 [00:35:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.50, 7.50, 7.49 [00:35:54] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 8.34, 10.16, 9.82 [00:36:36] PROBLEM - mw142 Current Load on mw142 is CRITICAL: LOAD CRITICAL - total load average: 14.69, 10.29, 8.20 [00:37:21] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.74, 3.69, 3.03 [00:37:23] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.24, 3.94, 3.06 [00:38:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.36, 11.51, 10.45 [00:38:14] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 3.53, 3.94, 3.21 [00:38:36] RECOVERY - mw142 Current Load on mw142 is OK: LOAD OK - total load average: 7.50, 9.33, 8.13 [00:38:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.12, 11.84, 10.74 [00:39:21] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.54, 3.25, 2.95 [00:39:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 9.79, 11.22, 10.57 [00:40:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 9.27, 10.44, 10.18 [00:40:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.34, 11.54, 10.76 [00:40:59] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.01, 7.50, 6.86 [00:41:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.71, 7.65, 7.72 [00:42:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 8.25, 9.87, 10.01 [00:42:16] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 3.10, 3.38, 3.14 [00:42:56] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.23, 6.67, 6.64 [00:43:16] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.53, 3.26, 3.05 [00:45:31] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 7.52, 9.39, 9.98 [00:48:47] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.00, 6.44, 6.47 [00:49:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.07, 8.03, 7.82 [00:50:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.94, 11.15, 10.23 [00:50:44] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 9.09, 7.43, 6.83 [00:51:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.40, 7.60, 7.70 [00:52:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.23, 10.44, 10.07 [00:52:40] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.19, 7.05, 6.75 [00:52:46] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 9.13, 9.52, 10.08 [00:54:23] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.72, 10.52, 10.26 [00:56:10] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.15, 4.26, 3.53 [00:56:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.72, 11.08, 10.65 [00:58:08] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.01, 3.71, 3.42 [00:58:29] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 4.68, 6.16, 6.50 [01:00:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.75, 11.19, 10.51 [01:00:07] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.26, 4.17, 3.63 [01:00:09] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 16.40, 12.72, 11.18 [01:02:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 7.32, 9.78, 10.09 [01:02:04] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 7.95, 11.05, 10.77 [01:03:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.65, 7.17, 7.26 [01:04:01] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.40, 3.99, 3.73 [01:04:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.33, 10.88, 10.57 [01:05:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.83, 6.67, 7.07 [01:05:58] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.01, 4.26, 3.85 [01:07:22] PROBLEM - mw143 Current Load on mw143 is WARNING: LOAD WARNING - total load average: 11.91, 10.05, 8.08 [01:08:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.10, 11.69, 11.06 [01:09:19] RECOVERY - mw143 Current Load on mw143 is OK: LOAD OK - total load average: 10.15, 9.77, 8.19 [01:10:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.52, 11.38, 10.52 [01:10:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 13.86, 11.90, 11.17 [01:11:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.37, 7.05, 7.06 [01:12:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.70, 11.38, 10.63 [01:12:19] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 12.96, 11.56, 10.31 [01:12:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.32, 11.20, 11.00 [01:14:14] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 9.13, 11.07, 10.31 [01:15:32] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.56, 11.37, 11.03 [01:15:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.45, 7.50, 7.26 [01:16:08] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 12.33, 12.12, 10.82 [01:17:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 8.14, 10.45, 10.75 [01:18:03] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.62, 11.70, 10.80 [01:18:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.94, 11.26, 10.97 [01:20:40] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [01:20:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 9.38, 10.46, 10.71 [01:24:46] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 9.03, 9.25, 10.16 [01:25:07] PROBLEM - os141 PowerDNS Recursor on os141 is CRITICAL: CRITICAL - Plugin timed out while executing system call [01:25:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.98, 7.79, 7.36 [01:26:13] PROBLEM - mw141 Current Load on mw141 is WARNING: LOAD WARNING - total load average: 9.59, 10.31, 9.46 [01:26:36] PROBLEM - mw142 Current Load on mw142 is WARNING: LOAD WARNING - total load average: 8.82, 11.53, 10.05 [01:27:05] RECOVERY - os141 PowerDNS Recursor on os141 is OK: DNS OK: 0.525 seconds response time. miraheze.org returns 2001:41d0:801:2000::3a18,2001:41d0:801:2000::5d68,51.195.201.140,51.89.139.24 [01:27:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.42, 10.94, 10.78 [01:27:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.29, 7.49, 7.29 [01:28:07] RECOVERY - mw141 Current Load on mw141 is OK: LOAD OK - total load average: 8.04, 9.43, 9.23 [01:28:36] RECOVERY - mw142 Current Load on mw142 is OK: LOAD OK - total load average: 6.75, 9.92, 9.64 [01:28:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.12, 10.79, 10.62 [01:29:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.26, 10.24, 10.51 [01:29:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.02, 7.54, 7.33 [01:30:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 8.48, 9.73, 10.16 [01:30:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.57, 11.40, 10.86 [01:34:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.60, 11.10, 10.57 [01:34:03] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.80, 7.31, 6.68 [01:35:17] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 7.41, 9.29, 10.01 [01:35:31] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 9.17, 9.56, 10.08 [01:36:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 7.87, 10.99, 11.02 [01:38:02] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 11.13, 7.83, 6.94 [01:40:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.27, 11.87, 11.20 [01:40:03] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.81, 7.25, 6.84 [01:41:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.88, 10.23, 10.14 [01:41:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.45, 7.57, 7.94 [01:42:02] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.70, 6.68, 6.68 [01:43:19] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 2.51, 3.20, 3.81 [01:43:31] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 9.71, 10.04, 10.08 [01:45:00] PROBLEM - os141 PowerDNS Recursor on os141 is CRITICAL: CRITICAL - Plugin timed out while executing system call [01:45:17] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 6.30, 4.32, 4.14 [01:46:59] RECOVERY - os141 PowerDNS Recursor on os141 is OK: DNS OK: 0.577 seconds response time. miraheze.org returns 2001:41d0:801:2000::3a18,2001:41d0:801:2000::5d68,51.195.201.140,51.89.139.24 [01:47:36] PROBLEM - mw141 Current Load on mw141 is WARNING: LOAD WARNING - total load average: 10.62, 10.33, 9.57 [01:49:15] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 1.83, 3.44, 3.87 [01:49:36] RECOVERY - mw141 Current Load on mw141 is OK: LOAD OK - total load average: 7.13, 9.29, 9.29 [01:52:46] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 5.07, 8.25, 9.71 [01:53:16] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.23, 3.32, 3.67 [01:55:15] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 1.30, 2.60, 3.37 [01:56:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.50, 9.82, 10.04 [01:57:21] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.18, 10.24, 9.60 [01:58:46] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 9.70, 9.46, 9.87 [01:59:16] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 8.96, 9.86, 9.55 [02:00:23] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.61, 10.84, 9.99 [02:01:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.06, 7.31, 7.31 [02:04:14] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 9.32, 10.15, 9.90 [02:06:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 13.50, 11.41, 10.48 [02:07:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.36, 7.44, 7.40 [02:08:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.96, 11.36, 10.59 [02:09:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.01, 8.03, 7.61 [02:10:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.48, 11.55, 10.89 [02:10:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.09, 11.60, 10.77 [02:11:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.22, 7.78, 7.57 [02:12:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.44, 11.52, 10.96 [02:12:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 9.65, 11.20, 10.75 [02:13:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.31, 10.31, 9.69 [02:13:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.61, 7.86, 7.62 [02:15:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.37, 7.69, 7.59 [02:16:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 15.02, 12.77, 11.53 [02:17:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 13.26, 11.40, 10.24 [02:18:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.85, 11.68, 11.01 [02:19:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.83, 11.19, 10.31 [02:21:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 12.52, 11.55, 10.54 [02:22:32] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 16.01, 12.70, 10.83 [02:23:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.84, 11.48, 10.65 [02:28:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 8.14, 10.87, 11.63 [02:28:18] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 9.24, 11.26, 10.86 [02:29:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.34, 7.57, 7.38 [02:30:14] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 14.33, 12.16, 11.21 [02:30:40] [Grafana] !sre RESOLVED: High Job Queue Backlog https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [02:32:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.54, 12.23, 12.01 [02:32:09] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.00, 11.74, 11.18 [02:33:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 9.32, 9.63, 10.03 [02:34:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.28, 11.29, 11.69 [02:35:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.03, 7.03, 7.17 [02:36:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 15.87, 12.75, 12.14 [02:37:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.24, 10.27, 10.20 [02:38:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.98, 11.97, 11.93 [02:39:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 7.93, 9.56, 9.96 [02:39:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.05, 8.01, 7.57 [02:42:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.02, 12.21, 12.04 [02:43:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 12.55, 11.20, 10.53 [02:43:41] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 13.78, 12.26, 11.54 [02:45:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.18, 11.26, 10.64 [02:47:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 13.86, 12.27, 11.08 [02:51:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.93, 7.79, 7.92 [02:57:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.25, 7.44, 7.59 [03:01:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.19, 7.22, 7.49 [03:09:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.10, 7.16, 7.25 [03:11:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.35, 7.20, 7.26 [03:19:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.30, 7.52, 7.33 [03:21:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.52, 7.19, 7.24 [03:26:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [03:29:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 12.76, 8.31, 7.50 [03:32:22] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.94, 6.65, 6.11 [03:34:19] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.74, 6.60, 6.14 [03:35:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.35, 7.34, 7.38 [03:36:00] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [03:47:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.76, 6.38, 6.80 [03:51:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.75, 6.91, 6.93 [03:55:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.41, 7.41, 7.07 [04:01:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.81, 7.22, 7.17 [04:03:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 3.84, 6.12, 6.78 [04:09:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.37, 7.06, 6.97 [04:15:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.02, 6.45, 6.74 [04:19:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.61, 7.67, 7.12 [04:23:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.46, 11.39, 11.95 [04:23:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.80, 7.48, 7.32 [04:25:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 11.88, 11.74, 12.01 [04:25:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.66, 8.04, 7.52 [04:29:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.22, 11.69, 11.96 [04:29:51] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.54, 7.69, 7.50 [04:39:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 12.62, 12.11, 11.90 [04:45:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.01, 6.10, 6.80 [04:49:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.39, 7.74, 7.29 [04:49:53] PROBLEM - mirabeta.org - LetsEncrypt on sslhost is CRITICAL: connect to address mirabeta.org and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [04:50:03] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.91, 7.00, 6.26 [04:51:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.24, 11.74, 11.98 [04:53:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 12.64, 12.16, 12.11 [04:53:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.89, 7.81, 7.43 [04:56:03] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.46, 6.70, 6.40 [04:57:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 9.78, 11.42, 11.85 [05:03:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.03, 10.50, 11.97 [05:07:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 14.19, 10.53, 10.83 [05:07:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.09, 10.82, 11.69 [05:09:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 9.08, 10.42, 11.46 [05:09:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.87, 6.05, 6.78 [05:11:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 9.06, 10.28, 10.71 [05:14:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 7.66, 9.69, 11.50 [05:15:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 6.33, 8.86, 10.08 [05:15:31] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 5.81, 7.78, 9.96 [05:16:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 5.06, 9.07, 11.45 [05:19:52] PROBLEM - mirabeta.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'mirabeta.org' expires in 15 day(s) (Mon 15 Jan 2024 10:51:10 GMT +0000). [05:20:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.01, 10.79, 11.30 [05:22:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.72, 11.33, 11.55 [05:25:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.23, 10.81, 10.47 [05:26:19] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.68, 10.20, 10.05 [05:30:09] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.16, 10.88, 10.35 [05:30:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.46, 11.44, 11.56 [05:32:04] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 8.97, 10.09, 10.13 [05:32:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 13.62, 12.07, 11.76 [05:33:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 13.08, 11.28, 10.75 [05:34:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.05, 11.46, 11.57 [05:35:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.18, 11.16, 10.78 [05:35:56] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 9.32, 10.45, 10.35 [05:36:50] [02miraheze/ManageWiki] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/4600541d8ea6...b30793e3833e [05:36:52] [02miraheze/ManageWiki] 07AgentIsai 03b30793e - ManageWikiRequirements: don't check 'permissions' if we are in CLI mode [05:39:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 7.11, 8.93, 9.94 [05:42:09] miraheze/ManageWiki - AgentIsai the build passed. [05:43:45] [02mediawiki] 07AgentIsai commented on pull request 03#17591: Bump extensions/ManageWiki from `8c58f21` to `a20129a` - 13https://github.com/miraheze/mediawiki/pull/17591#issuecomment-1872677751 [05:43:48] [02mediawiki] 07dependabot[bot] edited pull request 03#17591: Bump extensions/ManageWiki from `8c58f21` to `a20129a` - 13https://github.com/miraheze/mediawiki/pull/17591 [05:45:31] [02mediawiki] 07dependabot[bot] edited pull request 03#17591: Bump extensions/ManageWiki from `8c58f21` to `a20129a` - 13https://github.com/miraheze/mediawiki/pull/17591 [05:45:32] [02miraheze/ManageWiki] 07AgentIsai pushed 031 commit to 03AgentIsai-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/commit/3352d72a0e2e [05:45:33] [02miraheze/ManageWiki] 07AgentIsai 033352d72 - Backport ManageWikiDefaultPermissions changes from WikiForge/ManageWiki [05:45:33] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.67, 11.61, 10.83 [05:45:34] [02ManageWiki] 07AgentIsai created branch 03AgentIsai-patch-1 - 13https://github.com/miraheze/ManageWiki [05:45:41] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_40/extensions/ManageWiki-b30793e [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/9e8f7940fd24 [05:45:42] [02miraheze/mediawiki] 07dependabot[bot] 039e8f794 - Bump extensions/ManageWiki from `8c58f21` to `b30793e` [05:45:43] [02mediawiki] 07dependabot[bot] opened pull request 03#17592: Bump extensions/ManageWiki from `8c58f21` to `b30793e` - 13https://github.com/miraheze/mediawiki/pull/17592 [05:45:45] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_40/extensions/ManageWiki-b30793e - 13https://github.com/miraheze/mediawiki [05:45:48] [02mediawiki] 07dependabot[bot] commented on pull request 03#17591: Bump extensions/ManageWiki from `8c58f21` to `a20129a` - 13https://github.com/miraheze/mediawiki/pull/17591#issuecomment-1872678024 [05:45:51] [02mediawiki] 07dependabot[bot] closed pull request 03#17591: Bump extensions/ManageWiki from `8c58f21` to `a20129a` - 13https://github.com/miraheze/mediawiki/pull/17591 [05:45:52] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_40/extensions/ManageWiki-a20129a [05:45:55] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_40/extensions/ManageWiki-a20129a - 13https://github.com/miraheze/mediawiki [05:46:02] [02mediawiki] 07github-actions[bot] labeled pull request 03#17592: Bump extensions/ManageWiki from `8c58f21` to `b30793e` - 13https://github.com/miraheze/mediawiki/pull/17592 [05:46:05] [02mediawiki] 07github-actions[bot] labeled pull request 03#17592: Bump extensions/ManageWiki from `8c58f21` to `b30793e` - 13https://github.com/miraheze/mediawiki/pull/17592 [05:46:07] [02ManageWiki] 07AgentIsai opened pull request 03#438: Backport changes from WikiForge/ManageWiki - 13https://github.com/miraheze/ManageWiki/pull/438 [05:47:20] [02miraheze/ManageWiki] 07github-actions[bot] pushed 031 commit to 03AgentIsai-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/3352d72a0e2e...2968385698d3 [05:47:22] [02miraheze/ManageWiki] 07github-actions 032968385 - CI: lint code to MediaWiki standards [05:47:23] [02ManageWiki] 07github-actions[bot] synchronize pull request 03#438: Backport changes from WikiForge/ManageWiki - 13https://github.com/miraheze/ManageWiki/pull/438 [05:47:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.75, 11.50, 10.87 [05:48:35] miraheze/ManageWiki - AgentIsai the build has errored. [05:48:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.43, 11.06, 11.16 [05:49:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 13.31, 11.69, 10.99 [05:50:17] [02miraheze/ManageWiki] 07AgentIsai pushed 031 commit to 03AgentIsai-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/2968385698d3...a63379a78a73 [05:50:20] [02miraheze/ManageWiki] 07AgentIsai 03a63379a - Individualize ManageWiki rights [05:50:22] [02ManageWiki] 07AgentIsai synchronize pull request 03#438: Backport changes from WikiForge/ManageWiki - 13https://github.com/miraheze/ManageWiki/pull/438 [05:52:27] [02miraheze/ManageWiki] 07AgentIsai pushed 031 commit to 03AgentIsai-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/a63379a78a73...220236506b07 [05:52:30] [02miraheze/ManageWiki] 07AgentIsai 032202365 - Check managewiki-* rights [05:52:32] [02ManageWiki] 07AgentIsai synchronize pull request 03#438: Backport changes from WikiForge/ManageWiki - 13https://github.com/miraheze/ManageWiki/pull/438 [05:52:41] miraheze/ManageWiki - AgentIsai the build has errored. [05:54:59] miraheze/ManageWiki - AgentIsai the build has errored. [05:55:25] [02miraheze/ManageWiki] 07AgentIsai pushed 031 commit to 03AgentIsai-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/220236506b07...22e7101363f6 [05:55:26] [02miraheze/ManageWiki] 07AgentIsai 0322e7101 - Add managewiki-* rights [05:55:27] [02ManageWiki] 07AgentIsai synchronize pull request 03#438: Backport changes from WikiForge/ManageWiki - 13https://github.com/miraheze/ManageWiki/pull/438 [05:56:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 7.79, 11.00, 11.42 [05:57:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 9.90, 11.50, 11.32 [05:57:58] miraheze/ManageWiki - AgentIsai the build has errored. [05:58:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 9.03, 10.88, 11.81 [06:02:12] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.01, 6.45, 6.07 [06:02:46] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 5.40, 7.89, 9.95 [06:03:31] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 3.79, 7.62, 9.76 [06:04:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 6.25, 7.38, 9.92 [06:04:07] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.40, 6.05, 5.95 [06:19:59] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppetdb] [06:42:13] [02miraheze/ManageWiki] 07AgentIsai pushed 031 commit to 03AgentIsai-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/22e7101363f6...bcc1cc85e0b4 [06:42:14] [02miraheze/ManageWiki] 07AgentIsai 03bcc1cc8 - Update i18n names [06:42:15] [02ManageWiki] 07AgentIsai synchronize pull request 03#438: Backport changes from WikiForge/ManageWiki - 13https://github.com/miraheze/ManageWiki/pull/438 [06:42:20] [02miraheze/ManageWiki] 07AgentIsai pushed 031 commit to 03AgentIsai-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/bcc1cc85e0b4...924f339821f5 [06:42:21] [02miraheze/ManageWiki] 07AgentIsai 03924f339 - Add i18n [06:42:24] [02ManageWiki] 07AgentIsai synchronize pull request 03#438: Backport changes from WikiForge/ManageWiki - 13https://github.com/miraheze/ManageWiki/pull/438 [06:47:37] miraheze/ManageWiki - AgentIsai the build passed. [06:51:30] [02ManageWiki] 07AgentIsai closed pull request 03#438: Backport changes from WikiForge/ManageWiki - 13https://github.com/miraheze/ManageWiki/pull/438 [06:51:33] [02miraheze/ManageWiki] 07AgentIsai pushed 031 commit to 03master [+0/-0/±5] 13https://github.com/miraheze/ManageWiki/compare/b30793e3833e...c88229f29ee0 [06:51:35] [02miraheze/ManageWiki] 07AgentIsai 03c88229f - Backport changes from WikiForge/ManageWiki (#438) [06:54:31] !log [agent@test131] starting deploy of {'folders': '../mediawiki-staging/w/extensions/ManageWiki'} to all [06:54:32] !log [agent@test131] finished deploy of {'folders': '../mediawiki-staging/w/extensions/ManageWiki'} to all - SUCCESS in 1s [06:54:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:54:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:54:47] PROBLEM - barengreza.my.id - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - barengreza.my.id All nameservers failed to answer the query. [06:56:50] miraheze/ManageWiki - AgentIsai the build passed. [06:56:53] [02mediawiki] 07AgentIsai commented on pull request 03#17592: Bump extensions/ManageWiki from `8c58f21` to `b30793e` - 13https://github.com/miraheze/mediawiki/pull/17592#issuecomment-1872729915 [06:56:56] [02mediawiki] 07dependabot[bot] commented on pull request 03#17592: Bump extensions/ManageWiki from `8c58f21` to `b30793e` - 13https://github.com/miraheze/mediawiki/pull/17592#issuecomment-1872729972 [06:57:22] [02mediawiki] 07AgentIsai commented on pull request 03#17590: Bump extensions/ManageWiki from `8c58f21` to `a20129a` - 13https://github.com/miraheze/mediawiki/pull/17590#issuecomment-1872730613 [06:57:26] [02mediawiki] 07dependabot[bot] edited pull request 03#17590: Bump extensions/ManageWiki from `8c58f21` to `a20129a` - 13https://github.com/miraheze/mediawiki/pull/17590 [06:58:19] [02mediawiki] 07dependabot[bot] edited pull request 03#17590: Bump extensions/ManageWiki from `8c58f21` to `a20129a` - 13https://github.com/miraheze/mediawiki/pull/17590 [06:58:29] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_41/extensions/ManageWiki-c88229f [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/0e6e749f6203 [06:58:30] [02miraheze/mediawiki] 07dependabot[bot] 030e6e749 - Bump extensions/ManageWiki from `8c58f21` to `c88229f` [06:58:33] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_41/extensions/ManageWiki-c88229f - 13https://github.com/miraheze/mediawiki [06:58:35] [02mediawiki] 07dependabot[bot] opened pull request 03#17593: Bump extensions/ManageWiki from `8c58f21` to `c88229f` - 13https://github.com/miraheze/mediawiki/pull/17593 [06:58:36] [02mediawiki] 07dependabot[bot] commented on pull request 03#17590: Bump extensions/ManageWiki from `8c58f21` to `a20129a` - 13https://github.com/miraheze/mediawiki/pull/17590#issuecomment-1872731932 [06:58:39] [02mediawiki] 07dependabot[bot] closed pull request 03#17590: Bump extensions/ManageWiki from `8c58f21` to `a20129a` - 13https://github.com/miraheze/mediawiki/pull/17590 [06:58:40] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_41/extensions/ManageWiki-a20129a [06:58:42] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_41/extensions/ManageWiki-a20129a - 13https://github.com/miraheze/mediawiki [06:58:51] [02mediawiki] 07github-actions[bot] labeled pull request 03#17593: Bump extensions/ManageWiki from `8c58f21` to `c88229f` - 13https://github.com/miraheze/mediawiki/pull/17593 [06:58:52] [02mediawiki] 07github-actions[bot] labeled pull request 03#17593: Bump extensions/ManageWiki from `8c58f21` to `c88229f` - 13https://github.com/miraheze/mediawiki/pull/17593 [06:59:41] [02mediawiki] 07AgentIsai closed pull request 03#17593: Bump extensions/ManageWiki from `8c58f21` to `c88229f` - 13https://github.com/miraheze/mediawiki/pull/17593 [06:59:42] [02miraheze/mediawiki] 07AgentIsai deleted branch 03dependabot/submodules/REL1_41/extensions/ManageWiki-c88229f [06:59:43] [02miraheze/mediawiki] 07AgentIsai pushed 032 commits to 03REL1_41 [+0/-0/±2] 13https://github.com/miraheze/mediawiki/compare/aa5900d8a6db...818f01b62f44 [06:59:44] [02miraheze/mediawiki] 07dependabot[bot] 030e6e749 - Bump extensions/ManageWiki from `8c58f21` to `c88229f` [06:59:46] [02miraheze/mediawiki] 07AgentIsai 03818f01b - Bump extensions/ManageWiki from `8c58f21` to `c88229f` [06:59:48] [02mediawiki] 07AgentIsai deleted branch 03dependabot/submodules/REL1_41/extensions/ManageWiki-c88229f - 13https://github.com/miraheze/mediawiki [07:03:31] [02mediawiki] 07AgentIsai commented on pull request 03#17592: Bump extensions/ManageWiki from `8c58f21` to `b30793e` - 13https://github.com/miraheze/mediawiki/pull/17592#issuecomment-1872738748 [07:03:35] [02mediawiki] 07dependabot[bot] edited pull request 03#17592: Bump extensions/ManageWiki from `8c58f21` to `b30793e` - 13https://github.com/miraheze/mediawiki/pull/17592 [07:04:25] [02mediawiki] 07dependabot[bot] edited pull request 03#17592: Bump extensions/ManageWiki from `8c58f21` to `b30793e` - 13https://github.com/miraheze/mediawiki/pull/17592 [07:04:37] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_40/extensions/ManageWiki-c88229f [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/f08549728f78 [07:04:39] [02miraheze/mediawiki] 07dependabot[bot] 03f085497 - Bump extensions/ManageWiki from `8c58f21` to `c88229f` [07:04:40] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_40/extensions/ManageWiki-c88229f - 13https://github.com/miraheze/mediawiki [07:04:42] [02mediawiki] 07dependabot[bot] opened pull request 03#17594: Bump extensions/ManageWiki from `8c58f21` to `c88229f` - 13https://github.com/miraheze/mediawiki/pull/17594 [07:04:44] [02mediawiki] 07dependabot[bot] commented on pull request 03#17592: Bump extensions/ManageWiki from `8c58f21` to `b30793e` - 13https://github.com/miraheze/mediawiki/pull/17592#issuecomment-1872740355 [07:04:47] [02mediawiki] 07dependabot[bot] closed pull request 03#17592: Bump extensions/ManageWiki from `8c58f21` to `b30793e` - 13https://github.com/miraheze/mediawiki/pull/17592 [07:04:50] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_40/extensions/ManageWiki-b30793e [07:04:53] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_40/extensions/ManageWiki-b30793e - 13https://github.com/miraheze/mediawiki [07:05:02] [02mediawiki] 07github-actions[bot] labeled pull request 03#17594: Bump extensions/ManageWiki from `8c58f21` to `c88229f` - 13https://github.com/miraheze/mediawiki/pull/17594 [07:05:03] [02mediawiki] 07github-actions[bot] labeled pull request 03#17594: Bump extensions/ManageWiki from `8c58f21` to `c88229f` - 13https://github.com/miraheze/mediawiki/pull/17594 [07:07:59] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [07:09:06] RECOVERY - barengreza.my.id - LetsEncrypt on sslhost is OK: OK - Certificate 'barengreza.my.id' will expire on Sun 17 Mar 2024 17:33:19 GMT +0000. [07:09:14] [02mediawiki] 07AgentIsai closed pull request 03#17594: Bump extensions/ManageWiki from `8c58f21` to `c88229f` - 13https://github.com/miraheze/mediawiki/pull/17594 [07:09:17] [02miraheze/mediawiki] 07AgentIsai pushed 032 commits to 03REL1_40 [+0/-0/±2] 13https://github.com/miraheze/mediawiki/compare/2a32c1a42a1f...5bddd304888e [07:09:20] [02miraheze/mediawiki] 07dependabot[bot] 03f085497 - Bump extensions/ManageWiki from `8c58f21` to `c88229f` [07:09:21] [02miraheze/mediawiki] 07AgentIsai 035bddd30 - Bump extensions/ManageWiki from `8c58f21` to `c88229f` [07:09:23] [02miraheze/mediawiki] 07AgentIsai deleted branch 03dependabot/submodules/REL1_40/extensions/ManageWiki-c88229f [07:09:25] [02mediawiki] 07AgentIsai deleted branch 03dependabot/submodules/REL1_40/extensions/ManageWiki-c88229f - 13https://github.com/miraheze/mediawiki [07:24:42] PROBLEM - barengreza.my.id - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for barengreza.my.id could not be found [07:30:53] !log [agent@mwtask141] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.json /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/extensions/MirahezeMagic/maintenance/resetWikiCaches.php (START) [07:30:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:40:30] [02miraheze/mw-config] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/256c2350540b...bb6a219668da [07:40:31] [02miraheze/mw-config] 07AgentIsai 03bb6a219 - Replace managewiki standalone right with new versions [07:40:45] !log [@mwtask141] starting deploy of {'config': True} to all [07:41:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:41:31] miraheze/mw-config - AgentIsai the build passed. [07:41:35] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 19.93, 12.85, 8.81 [07:42:24] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 99s [07:42:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 14.67, 11.70, 8.55 [07:42:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:43:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 15.19, 11.55, 8.64 [07:44:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.20, 11.25, 8.79 [07:45:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.78, 10.90, 8.75 [07:46:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.35, 12.22, 9.47 [07:47:08] PROBLEM - os131 Current Load on os131 is CRITICAL: LOAD CRITICAL - total load average: 4.45, 3.46, 2.23 [07:48:45] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 20.68, 13.71, 9.51 [07:49:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 18.13, 13.04, 10.02 [07:49:52] PROBLEM - mirabeta.org - LetsEncrypt on sslhost is CRITICAL: connect to address mirabeta.org and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [07:51:27] PROBLEM - cp24 Varnish Backends on cp24 is CRITICAL: 3 backends are down. mw131 mw132 mw133 [07:51:34] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.58, 3.80, 2.98 [07:51:58] PROBLEM - cp35 Varnish Backends on cp35 is CRITICAL: 4 backends are down. mw131 mw132 mw133 mw143 [07:54:07] PROBLEM - cp25 Varnish Backends on cp25 is CRITICAL: 3 backends are down. mw131 mw132 mw133 [07:54:11] PROBLEM - cp34 Varnish Backends on cp34 is CRITICAL: 3 backends are down. mw131 mw132 mw133 [07:55:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 8.54, 11.96, 10.70 [07:55:27] RECOVERY - cp24 Varnish Backends on cp24 is OK: All 15 backends are healthy [07:55:58] RECOVERY - cp35 Varnish Backends on cp35 is OK: All 15 backends are healthy [07:56:06] RECOVERY - cp25 Varnish Backends on cp25 is OK: All 15 backends are healthy [07:56:11] RECOVERY - cp34 Varnish Backends on cp34 is OK: All 15 backends are healthy [07:57:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 13.67, 12.86, 11.19 [07:57:25] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.63, 3.92, 3.31 [07:59:42] !log [@test131] starting deploy of {'config': True} to all [07:59:43] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 1s [07:59:58] PROBLEM - cp35 Varnish Backends on cp35 is CRITICAL: 1 backends are down. mw131 [08:00:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:00:04] PROBLEM - cp25 Varnish Backends on cp25 is CRITICAL: 1 backends are down. mw131 [08:00:10] PROBLEM - cp34 Varnish Backends on cp34 is CRITICAL: 1 backends are down. mw131 [08:00:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:01:14] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 8.40, 11.78, 11.20 [08:01:18] PROBLEM - mw142 Current Load on mw142 is CRITICAL: LOAD CRITICAL - total load average: 27.49, 13.47, 8.78 [08:01:39] PROBLEM - mw141 Current Load on mw141 is CRITICAL: LOAD CRITICAL - total load average: 22.78, 13.16, 8.52 [08:03:06] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 1.62, 3.29, 3.37 [08:03:22] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.56, 3.31, 3.24 [08:03:37] PROBLEM - mw141 Current Load on mw141 is WARNING: LOAD WARNING - total load average: 9.48, 11.67, 8.58 [08:03:56] PROBLEM - mw143 Current Load on mw143 is WARNING: LOAD WARNING - total load average: 7.57, 10.47, 7.88 [08:03:58] RECOVERY - cp35 Varnish Backends on cp35 is OK: All 15 backends are healthy [08:04:02] RECOVERY - cp25 Varnish Backends on cp25 is OK: All 15 backends are healthy [08:04:09] RECOVERY - cp34 Varnish Backends on cp34 is OK: All 15 backends are healthy [08:05:07] PROBLEM - mw142 Current Load on mw142 is WARNING: LOAD WARNING - total load average: 8.36, 11.72, 9.23 [08:05:53] RECOVERY - mw143 Current Load on mw143 is OK: LOAD OK - total load average: 9.65, 9.99, 7.99 [08:07:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 6.77, 8.77, 10.09 [08:08:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 9.11, 11.24, 11.89 [08:08:55] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.51, 7.08, 6.47 [08:08:55] RECOVERY - mw142 Current Load on mw142 is OK: LOAD OK - total load average: 7.59, 9.60, 8.92 [08:10:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.43, 6.73, 6.41 [08:11:37] PROBLEM - mw141 Current Load on mw141 is CRITICAL: LOAD CRITICAL - total load average: 15.06, 11.66, 9.56 [08:11:57] [02miraheze/ManageWiki] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/c88229f29ee0...046057bce80a [08:12:00] [02miraheze/ManageWiki] 07AgentIsai 03046057b - Fix permission check [08:12:02] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.30, 3.19, 2.11 [08:12:13] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 8.96, 10.00, 11.91 [08:12:46] [02mediawiki] 07AgentIsai commented on pull request 03#17594: Bump extensions/ManageWiki from `8c58f21` to `c88229f` - 13https://github.com/miraheze/mediawiki/pull/17594#issuecomment-1872832485 [08:12:48] [02mediawiki] 07dependabot[bot] commented on pull request 03#17594: Bump extensions/ManageWiki from `8c58f21` to `c88229f` - 13https://github.com/miraheze/mediawiki/pull/17594#issuecomment-1872832548 [08:13:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 9.80, 10.18, 12.00 [08:13:39] [02mediawiki] 07AgentIsai commented on pull request 03#17291: Bump extensions/ManageWiki from `4fb5fd1` to `809a1e0` - 13https://github.com/miraheze/mediawiki/pull/17291#issuecomment-1872833554 [08:13:43] [02mediawiki] 07dependabot[bot] edited pull request 03#17291: Bump extensions/ManageWiki from `4fb5fd1` to `809a1e0` - 13https://github.com/miraheze/mediawiki/pull/17291 [08:13:58] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 3.03, 3.12, 2.22 [08:14:37] [02mediawiki] 07dependabot[bot] edited pull request 03#17291: Bump extensions/ManageWiki from `4fb5fd1` to `809a1e0` - 13https://github.com/miraheze/mediawiki/pull/17291 [08:14:42] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.95, 7.39, 6.76 [08:14:48] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_40/extensions/ManageWiki-046057b [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/6301a68a1a39 [08:14:50] [02miraheze/mediawiki] 07dependabot[bot] 036301a68 - Bump extensions/ManageWiki from `c88229f` to `046057b` [08:14:51] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_40/extensions/ManageWiki-046057b - 13https://github.com/miraheze/mediawiki [08:14:52] [02mediawiki] 07dependabot[bot] opened pull request 03#17595: Bump extensions/ManageWiki from `c88229f` to `046057b` - 13https://github.com/miraheze/mediawiki/pull/17595 [08:15:12] [02mediawiki] 07github-actions[bot] labeled pull request 03#17595: Bump extensions/ManageWiki from `c88229f` to `046057b` - 13https://github.com/miraheze/mediawiki/pull/17595 [08:15:34] [02mediawiki] 07AgentIsai commented on pull request 03#17291: Bump extensions/ManageWiki from `4fb5fd1` to `809a1e0` - 13https://github.com/miraheze/mediawiki/pull/17291#issuecomment-1872835760 [08:15:37] [02mediawiki] 07dependabot[bot] edited pull request 03#17291: Bump extensions/ManageWiki from `4fb5fd1` to `809a1e0` - 13https://github.com/miraheze/mediawiki/pull/17291 [08:15:38] PROBLEM - mw141 Current Load on mw141 is WARNING: LOAD WARNING - total load average: 9.29, 11.70, 10.17 [08:16:26] [02mediawiki] 07dependabot[bot] edited pull request 03#17291: Bump extensions/ManageWiki from `4fb5fd1` to `809a1e0` - 13https://github.com/miraheze/mediawiki/pull/17291 [08:16:27] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_40/extensions/ManageWiki-809a1e0 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/e7d1f5fc537c...4d0d2b6adf84 [08:16:30] [02miraheze/mediawiki] 07dependabot[bot] 034d0d2b6 - Bump extensions/ManageWiki from `4fb5fd1` to `809a1e0` [08:16:32] [02mediawiki] 07dependabot[bot] synchronize pull request 03#17291: Bump extensions/ManageWiki from `4fb5fd1` to `809a1e0` - 13https://github.com/miraheze/mediawiki/pull/17291 [08:16:34] [02mediawiki] 07dependabot[bot] edited pull request 03#17291: Bump extensions/ManageWiki from `4fb5fd1` to `809a1e0` - 13https://github.com/miraheze/mediawiki/pull/17291 [08:16:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 13.71, 11.16, 11.33 [08:16:47] [02mediawiki] 07github-actions[bot] labeled pull request 03#17291: Bump extensions/ManageWiki from `4fb5fd1` to `809a1e0` - 13https://github.com/miraheze/mediawiki/pull/17291 [08:17:08] miraheze/ManageWiki - AgentIsai the build passed. [08:17:14] [02mediawiki] 07AgentIsai closed pull request 03#17291: Bump extensions/ManageWiki from `4fb5fd1` to `809a1e0` - 13https://github.com/miraheze/mediawiki/pull/17291 [08:17:15] [02miraheze/mediawiki] 07AgentIsai deleted branch 03dependabot/submodules/REL1_40/extensions/ManageWiki-809a1e0 [08:17:16] [02miraheze/mediawiki] 07AgentIsai pushed 032 commits to 03REL1_40 [+0/-0/±2] 13https://github.com/miraheze/mediawiki/compare/5bddd304888e...e0033ff022e1 [08:17:18] [02miraheze/mediawiki] 07dependabot[bot] 034d0d2b6 - Bump extensions/ManageWiki from `4fb5fd1` to `809a1e0` [08:17:20] [02miraheze/mediawiki] 07AgentIsai 03e0033ff - Bump extensions/ManageWiki from `4fb5fd1` to `809a1e0` [08:17:21] [02mediawiki] 07AgentIsai deleted branch 03dependabot/submodules/REL1_40/extensions/ManageWiki-809a1e0 - 13https://github.com/miraheze/mediawiki [08:17:24] [02mediawiki] 07dependabot[bot] edited pull request 03#17291: Bump extensions/ManageWiki from `4fb5fd1` to `809a1e0` - 13https://github.com/miraheze/mediawiki/pull/17291 [08:18:26] !log [agent@mwtask141] starting deploy of {'world': True} to all [08:18:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:18:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 9.89, 10.42, 11.03 [08:19:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.82, 9.89, 9.90 [08:19:54] PROBLEM - mirabeta.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'mirabeta.org' expires in 15 day(s) (Mon 15 Jan 2024 10:51:10 GMT +0000). [08:20:27] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.75, 6.34, 6.48 [08:21:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 9.11, 9.97, 9.95 [08:21:36] PROBLEM - mw141 Current Load on mw141 is CRITICAL: LOAD CRITICAL - total load average: 15.54, 11.86, 10.54 [08:23:54] PROBLEM - cp25 Varnish Backends on cp25 is CRITICAL: 2 backends are down. mw142 mw143 [08:23:58] PROBLEM - cp35 Varnish Backends on cp35 is CRITICAL: 2 backends are down. mw142 mw143 [08:24:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 15.42, 11.61, 11.63 [08:24:08] PROBLEM - mw142 Current Load on mw142 is CRITICAL: LOAD CRITICAL - total load average: 11.47, 13.23, 11.06 [08:24:53] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 14.58, 11.26, 11.00 [08:25:42] PROBLEM - barengreza.my.id - reverse DNS on sslhost is UNKNOWN: [08:25:44] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 14.38, 11.69, 11.45 [08:25:50] PROBLEM - mon141 Current Load on mon141 is CRITICAL: LOAD CRITICAL - total load average: 25.27, 9.39, 4.17 [08:26:57] PROBLEM - mw143 Current Load on mw143 is CRITICAL: LOAD CRITICAL - total load average: 18.14, 15.77, 11.38 [08:28:08] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 12.42, 12.73, 11.18 [08:28:56] RECOVERY - cp25 Varnish Backends on cp25 is OK: All 15 backends are healthy [08:29:00] RECOVERY - cp35 Varnish Backends on cp35 is OK: All 15 backends are healthy [08:29:33] PROBLEM - ns1 Auth DNS on ns1 is CRITICAL: CRITICAL - Plugin timed out while executing system call PROBLEM - ns1 Auth DNS on ns1 is CRITICAL: CRITICAL - Plugin timed out while executing system call [08:30:08] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.64, 11.92, 11.46 [08:30:31] PROBLEM - polcompball.miraheze.org - reverse DNS on sslhost is UNKNOWN: [08:30:31] PROBLEM - es.countryhumans.polandball.wiki - reverse DNS on sslhost is UNKNOWN: PROBLEM - es.countryhumans.polandball.wiki - reverse DNS on sslhost is UNKNOWN: [08:30:35] PROBLEM - wikipwaadia.works - reverse DNS on sslhost is UNKNOWN: [08:31:17] PROBLEM - wiki.queenscourt.games - reverse DNS on sslhost is UNKNOWN: PROBLEM - pwsc.polandballwiki.com - reverse DNS on sslhost is UNKNOWN: PROBLEM - miraheze.wiki - reverse DNS on sslhost is UNKNOWN: [08:31:17] PROBLEM - wiki.teessidehackspace.org.uk - reverse DNS on sslhost is UNKNOWN: [08:31:21] PROBLEM - vise.dayid.org - reverse DNS on sslhost is UNKNOWN: [08:31:22] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 8.51, 11.30, 10.96 [08:31:25] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 8.89, 11.56, 11.70 [08:31:32] PROBLEM - mon141 HTTPS on mon141 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10001 milliseconds with 0 bytes received [08:31:49] PROBLEM - reviwiki.info - reverse DNS on sslhost is UNKNOWN: [08:31:49] PROBLEM - wiki.moores.tech - reverse DNS on sslhost is UNKNOWN: PROBLEM - www.sekaipedia.org - reverse DNS on sslhost is UNKNOWN: PROBLEM - fc.songcontests.eu - reverse DNS on sslhost is UNKNOWN: [08:31:50] PROBLEM - permanentfuturelab.wiki - reverse DNS on sslhost is UNKNOWN: [08:32:04] PROBLEM - db101 MariaDB Connections on db101 is UNKNOWN: [08:32:20] PROBLEM - mon141 Puppet on mon141 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [08:32:24] PROBLEM - wiki.graalmilitary.com - reverse DNS on sslhost is UNKNOWN: [08:32:24] PROBLEM - www.hebammenwiki.de - reverse DNS on sslhost is UNKNOWN: [08:32:27] PROBLEM - thesimswiki.com - reverse DNS on sslhost is UNKNOWN: [08:32:27] PROBLEM - www.zenbuddhism.info - reverse DNS on sslhost is UNKNOWN: [08:32:28] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is UNKNOWN: [08:32:28] PROBLEM - ecole.science - reverse DNS on sslhost is UNKNOWN: [08:32:28] PROBLEM - db131 MariaDB Connections on db131 is UNKNOWN: [08:32:37] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 9.50, 11.06, 11.67 [08:32:45] PROBLEM - wiki.insideearth.info - reverse DNS on sslhost is UNKNOWN: [08:32:47] PROBLEM - wiki.counterculturelabs.org - reverse DNS on sslhost is UNKNOWN: [08:32:47] PROBLEM - wiki.seamly.io - reverse DNS on sslhost is UNKNOWN: [08:32:47] PROBLEM - corru.wiki - reverse DNS on sslhost is UNKNOWN: [08:33:30] PROBLEM - mw143 Current Load on mw143 is WARNING: LOAD WARNING - total load average: 8.57, 10.72, 10.56 PROBLEM - threedomwiki.pcast.site - reverse DNS on sslhost is UNKNOWN: [08:33:30] PROBLEM - wiki.globasa.net - reverse DNS on sslhost is UNKNOWN: PROBLEM - evilgeniuswiki.com - reverse DNS on sslhost is UNKNOWN: [08:33:31] PROBLEM - wiki.umbranet.mandelbrotgames.com - reverse DNS on sslhost is UNKNOWN: [08:33:31] PROBLEM - enc.for.uz - reverse DNS on sslhost is UNKNOWN: [08:33:35] PROBLEM - yokaiwatchwiki.com - reverse DNS on sslhost is UNKNOWN: [08:33:47] PROBLEM - decyclopedia.tk - reverse DNS on sslhost is UNKNOWN: [08:33:52] PROBLEM - wiki.yahyabd.xyz - reverse DNS on sslhost is UNKNOWN: [08:33:52] PROBLEM - wiki.yahyabd.xyz - reverse DNS on sslhost is UNKNOWN: [08:33:55] PROBLEM - ewgf.wiki - reverse DNS on sslhost is UNKNOWN: [08:33:58] PROBLEM - roll-of-arms.com - reverse DNS on sslhost is UNKNOWN: [08:33:59] PROBLEM - abeyancewiki.bumblebee-studios.se - reverse DNS on sslhost is UNKNOWN: [08:33:59] PROBLEM - vitriol.spoon.army - reverse DNS on sslhost is UNKNOWN: [08:34:04] PROBLEM - wiki.aoemods.com - reverse DNS on sslhost is UNKNOWN: [08:34:32] PROBLEM - wikicompliance.com - reverse DNS on sslhost is UNKNOWN: [08:34:38] PROBLEM - alphabirdswiki.gq - reverse DNS on sslhost is UNKNOWN: [08:34:38] PROBLEM - wiki.thehall.xyz - reverse DNS on sslhost is UNKNOWN: [08:34:38] PROBLEM - www.vanillaanarchy.net - reverse DNS on sslhost is UNKNOWN: [08:34:38] PROBLEM - wiki.hsdfz.org.cn - reverse DNS on sslhost is UNKNOWN: [08:34:39] PROBLEM - www.twistwoodtaleswiki.com - reverse DNS on sslhost is UNKNOWN: [08:34:59] PROBLEM - 9b9t.wiki - reverse DNS on sslhost is UNKNOWN: [08:35:10] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is UNKNOWN: [08:35:51] PROBLEM - www.winenjoy.net - reverse DNS on sslhost is UNKNOWN: [08:35:55] PROBLEM - wiki.ricochetuniverse.com - reverse DNS on sslhost is UNKNOWN: [08:35:57] PROBLEM - housing.wiki - reverse DNS on sslhost is UNKNOWN: [08:35:59] PROBLEM - commonswiki.miraheze.org - reverse DNS on sslhost is UNKNOWN: [08:36:07] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is UNKNOWN: [08:36:07] PROBLEM - wiki.yuanpi.eu.org - reverse DNS on sslhost is UNKNOWN: [08:36:27] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.23, 11.30, 11.59 [08:36:28] PROBLEM - cities.simulz.kr - reverse DNS on sslhost is UNKNOWN: [08:36:48] PROBLEM - txt-dex.com - reverse DNS on sslhost is UNKNOWN: [08:37:15] PROBLEM - mw143 Current Load on mw143 is CRITICAL: LOAD CRITICAL - total load average: 14.95, 13.13, 11.56 [08:37:15] PROBLEM - wiki.vinesh.eu.org - reverse DNS on sslhost is UNKNOWN: [08:37:16] PROBLEM - psycho.engineering - reverse DNS on sslhost is UNKNOWN: [08:37:23] PROBLEM - staratlaswiki.org - reverse DNS on sslhost is UNKNOWN: [08:37:27] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is UNKNOWN: [08:38:08] PROBLEM - cyberlaw.ccdcoe.org - reverse DNS on sslhost is UNKNOWN: PROBLEM - wikipedia.yogfront.ooo - reverse DNS on sslhost is UNKNOWN: [08:38:09] PROBLEM - civwiki.org - reverse DNS on sslhost is UNKNOWN: PROBLEM - small.polandballwiki.com - reverse DNS on sslhost is UNKNOWN: [08:38:14] PROBLEM - wiki.bluevertigo.org - reverse DNS on sslhost is UNKNOWN: [08:38:18] PROBLEM - cheeseepedia.org - reverse DNS on sslhost is UNKNOWN: [08:38:37] PROBLEM - mon141 icinga.miraheze.org HTTPS on mon141 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:38:38] PROBLEM - mon141 PowerDNS Recursor on mon141 is CRITICAL: CRITICAL - Plugin timed out while executing system call [08:39:22] PROBLEM - data.nonbinary.wiki - reverse DNS on sslhost is UNKNOWN: [08:39:22] PROBLEM - wiki.esufranchise.com - reverse DNS on sslhost is UNKNOWN: [08:40:30] that's not good [08:40:45] PROBLEM - wizardia.wiki - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.auraxis.co - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.pulsus.cc - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.jeremyboyd.org - reverse DNS on sslhost is UNKNOWN: Terminated by signal 15 (Terminated).> PROBLEM - wizardia.wiki - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.veloren.net - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.cubestudios.xyz - reverse DNS on sslhost is UNKNOWN: PROBLEM - mhdh.pj568.eu.org - revers [08:40:45] on sslhost is UNKNOWN: [08:40:47] PROBLEM - n64brew.dev - reverse DNS on sslhost is UNKNOWN: [08:40:55] No it isn't good @Agent [08:40:56] PROBLEM - www.programming.red - reverse DNS on sslhost is UNKNOWN: [08:41:15] test [08:41:27] PROBLEM - wiki.maxshub.net - reverse DNS on sslhost is UNKNOWN: [08:41:35] huh, the relay isn't relaying us to the other side [08:41:44] Oh lol [08:41:46] PROBLEM - mon141 grafana.miraheze.org HTTPS on mon141 is CRITICAL: CRITICAL - Socket timeout after 10 seconds PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.85, 10.23, 10.96 PROBLEM - freedomplanetwiki.com - reverse DNS on sslhost is UNKNOWN: PROBLEM - yoshi.miraheze.org - reverse DNS on sslhost is UNKNOWN: 15 (Terminated).> [08:41:48] PROBLEM - wiki.cyberfurs.org - reverse DNS on sslhost is UNKNOWN: [08:41:50] PROBLEM - schizoidnightmares.miraheze.org - reverse DNS on sslhost is UNKNOWN: [08:41:51] PROBLEM - wiki.infomedia.co.uk - reverse DNS on sslhost is UNKNOWN: [08:41:51] PROBLEM - tech.exputra.com - reverse DNS on sslhost is UNKNOWN: [08:41:55] PROBLEM - wiki.theatlas.live - reverse DNS on sslhost is UNKNOWN: [08:41:56] PROBLEM - wiki.metaiitgn.org - reverse DNS on sslhost is UNKNOWN: [08:41:56] PROBLEM - wiki.digitalcandela.com - reverse DNS on sslhost is UNKNOWN: [08:42:27] RECOVERY - db131 MariaDB Connections on db131 is OK: OK connection usage: 11%Current connections: 55 [08:42:27] RECOVERY - db101 MariaDB Connections on db101 is OK: OK connection usage: 2.2%Current connections: 11 [08:42:28] oh it does, it's just a little slow [08:42:30] Sure it is lol [08:42:52] pong [08:42:53] ping [08:42:59] lol [08:43:04] impressive [08:43:10] PROBLEM - wiki.puucraft.net - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.openhatch.org - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.solarconquest.net - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.aetherexplorers.science - reverse DNS on sslhost is [08:43:10] WN: PROBLEM - petrawiki.org - reverse DNS on sslhost is UNKNOWN: [08:43:10] pong before ping came through [08:43:13] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.79, 5.63, 4.04 [08:43:14] PROBLEM - removededm.com - reverse DNS on sslhost is UNKNOWN: [08:43:17] PROBLEM - fanon.polandballwiki.com - reverse DNS on sslhost is UNKNOWN: [08:43:20] what the hell is happening here [08:43:26] PROBLEM - wiki.scvo.org - reverse DNS on sslhost is UNKNOWN: [08:43:33] PROBLEM - grayravens.com - reverse DNS on sslhost is UNKNOWN: [08:43:34] PROBLEM - wiki.yunachannel.com - reverse DNS on sslhost is UNKNOWN: [08:43:37] PROBLEM - wiki.lefrenchmelee.fr - reverse DNS on sslhost is UNKNOWN: [08:43:37] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 6.32, 8.02, 9.57 [08:43:55] PROBLEM - wiki.mastodon.kr - reverse DNS on sslhost is UNKNOWN: [08:44:08] PROBLEM - you.r-fit.cc - reverse DNS on sslhost is UNKNOWN: [08:44:09] PROBLEM - pandorastale.wiki - reverse DNS on sslhost is UNKNOWN: [08:44:09] PROBLEM - mon141 Check correctness of the icinga configuration on mon141 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [08:44:11] PROBLEM - talumn.neptune.wiki - reverse DNS on sslhost is UNKNOWN: [08:44:13] PROBLEM - itadminwiki.pl - reverse DNS on sslhost is UNKNOWN: [08:44:24] PROBLEM - wiki.rlab.org.uk - reverse DNS on sslhost is UNKNOWN: [08:44:24] RECOVERY - mon141 icinga.miraheze.org HTTPS on mon141 is OK: HTTP OK: HTTP/1.1 302 Found - 297 bytes in 0.164 second response time [08:44:37] PROBLEM - wiki.slimelab.net - reverse DNS on sslhost is UNKNOWN: [08:44:38] PROBLEM - witchingworld.org - reverse DNS on sslhost is UNKNOWN: [08:44:43] PROBLEM - wikimas.kr - reverse DNS on sslhost is UNKNOWN: [08:45:06] PROBLEM - wiki.ivy.cm - reverse DNS on sslhost is UNKNOWN: [08:45:12] PROBLEM - nl.xliving.org - reverse DNS on sslhost is UNKNOWN: [08:45:13] PROBLEM - www.tintinpedia.fr - reverse DNS on sslhost is UNKNOWN: [08:45:13] PROBLEM - steem.wiki - reverse DNS on sslhost is UNKNOWN: [08:45:16] PROBLEM - project-patterns.com - reverse DNS on sslhost is UNKNOWN: [08:45:16] PROBLEM - electowiki.org - reverse DNS on sslhost is UNKNOWN: [08:45:18] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 7.47, 9.01, 9.98 [08:45:28] PROBLEM - wiki.litek.top - reverse DNS on sslhost is UNKNOWN: [08:46:01] PROBLEM - dragonquest2.miraheze.org - reverse DNS on sslhost is UNKNOWN: SSL OK - dragonquest2.miraheze.org reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [08:46:33] PROBLEM - wiki.geoparkcorumbatai.com.br - reverse DNS on sslhost is UNKNOWN: [08:46:33] PROBLEM - trains.cnrailwiki.org - reverse DNS on sslhost is UNKNOWN: [08:46:39] PROBLEM - glitchcity.wiki - reverse DNS on sslhost is UNKNOWN: [08:46:39] PROBLEM - wiki.make717.org - reverse DNS on sslhost is UNKNOWN: [08:46:39] PROBLEM - glitchcity.wiki - reverse DNS on sslhost is UNKNOWN: [08:46:45] PROBLEM - mw141 Current Load on mw141 is WARNING: LOAD WARNING - total load average: 8.58, 10.95, 11.91 [08:47:02] PROBLEM - test2.specialchat.org - reverse DNS on sslhost is UNKNOWN: PROBLEM - squareenixwiki.com - reverse DNS on sslhost is UNKNOWN: SSL OK - squareenixwiki.com reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [08:47:47] PROBLEM - publictestwiki.com - reverse DNS on sslhost is UNKNOWN: [08:47:49] !log [@mwtask141] starting deploy of {'config': True} to all [08:47:49] PROBLEM - touhouplace.wiki - reverse DNS on sslhost is UNKNOWN: [08:47:52] RECOVERY - mon141 HTTPS on mon141 is OK: HTTP OK: HTTP/2 200 - 336 bytes in 6.419 second response time [08:47:54] RECOVERY - mon141 grafana.miraheze.org HTTPS on mon141 is OK: HTTP OK: HTTP/1.1 200 OK - 43491 bytes in 0.148 second response time [08:48:10] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 20s [08:49:05] PROBLEM - wiki.zamnhacking.net - reverse DNS on sslhost is UNKNOWN: SSL OK - wiki.zamnhacking.net reverse DNS resolves to cp24.miraheze.org - CNAME OK PROBLEM - wiki.iurevar.world - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.valkyrienskies.org - reverse DNS on sslhost is UNKNOWN: Terminated).> [08:49:05] PROBLEM - wiki.kachow.org - reverse DNS on sslhost is UNKNOWN: PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: LOAD WARNING - total load average: 8.41, 10.22, 9.74 [08:49:09] PROBLEM - www.mh142.com - reverse DNS on sslhost is UNKNOWN: [08:49:10] PROBLEM - pandorastale.miraheze.org - reverse DNS on sslhost is UNKNOWN: [08:50:28] PROBLEM - echoes-wiki.com - reverse DNS on sslhost is UNKNOWN: [08:50:28] PROBLEM - opendatascot.org - reverse DNS on sslhost is UNKNOWN: PROBLEM - christipedia.nl - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.denby.tech - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.controlistes.fr - reverse DNS on sslhost is UNKNOWN: ed.> PROBLEM - www.johanloopmans.nl - reverse DNS on sslhost is UNKNOWN: PROBLEM - pornwiki.org - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.seamly.net - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.candelabrem.com - [08:50:28] se DNS on sslhost is UNKNOWN: PROBLEM - wikislamica.org - reverse DNS on sslhost is UNKNOWN: [08:50:28] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is UNKNOWN: [08:51:41] PROBLEM - fallofsanctuary.com - reverse DNS on sslhost is UNKNOWN: PROBLEM - developer-and-testers.wiki.feign-wiki.tk - reverse DNS on sslhost is UNKNOWN: [08:51:41] PROBLEM - www.dariawiki.org - reverse DNS on sslhost is UNKNOWN: PROBLEM - wariopedia.com - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.bloxotel.tk - reverse DNS on sslhost is UNKNOWN: PROBLEM - minehut.tarna.dev - reverse DNS on sslhost is UNKNOWN: .> PROBLEM - www.christipedia.nl - reverse DNS on sslhost is UNKNOWN: PROBLEM - www.moiswiki.net - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.mostly.vet - reverse DNS on sslhost is UNKNOWN: [08:51:48] PROBLEM - o.kaoyu666.cn - reverse DNS on sslhost is UNKNOWN: [08:52:15] RECOVERY - mon141 Check correctness of the icinga configuration on mon141 is OK: Icinga configuration is correct [08:52:16] !log [agent@mwtask141] finished deploy of {'world': True} to all - SUCCESS in 2030s [08:52:35] PROBLEM - runzeppelin.ru - reverse DNS on sslhost is UNKNOWN: PROBLEM - wiki.anempireofdreams.com - reverse DNS on sslhost is UNKNOWN: [08:52:45] RECOVERY - mon141 PowerDNS Recursor on mon141 is OK: DNS OK: 0.583 seconds response time. miraheze.org returns 2001:41d0:801:2000::3a18,2001:41d0:801:2000::5d68,51.195.201.140,51.89.139.24 [08:52:46] PROBLEM - wiki.closai.com - reverse DNS on sslhost is UNKNOWN: SSL OK - wiki.closai.com reverse DNS resolves to cp24.miraheze.org - CNAME OK [08:52:46] PROBLEM - rare.miraheze.org - reverse DNS on sslhost is UNKNOWN: SSL OK - rare.miraheze.org reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [08:52:48] PROBLEM - worldtobuild.wiki - reverse DNS on sslhost is UNKNOWN: SSL OK - worldtobuild.wiki reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [08:52:58] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 10.11, 8.83, 9.69 [08:53:53] PROBLEM - mw142 Current Load on mw142 is WARNING: LOAD WARNING - total load average: 5.38, 8.60, 11.07 [08:54:27] PROBLEM - mw143 Current Load on mw143 is WARNING: LOAD WARNING - total load average: 6.48, 9.66, 11.39 [08:54:27] RECOVERY - mwtask141 Current Load on mwtask141 is OK: LOAD OK - total load average: 5.73, 8.25, 9.11 [08:54:42] RECOVERY - ns1 Auth DNS on ns1 is OK: DNS OK: 0.910 seconds response time. miraheze.org returns 2001:41d0:801:2000::3a18,2001:41d0:801:2000::5d68,51.195.201.140,51.89.139.24 [08:55:05] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 0.86, 2.66, 3.61 [08:55:29] RECOVERY - mon141 Puppet on mon141 is OK: OK: Puppet is currently enabled, last run 28 minutes ago with 0 failures [08:56:10] PROBLEM - crustypedia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'crustypedia.org' expires in 15 day(s) (Tue 16 Jan 2024 08:18:21 GMT +0000). [08:56:20] RECOVERY - mw141 Current Load on mw141 is OK: LOAD OK - total load average: 5.31, 7.65, 9.89 [08:56:21] PROBLEM - barengreza.my.id - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for barengreza.my.id could not be found [08:56:24] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/f78e4b394a89...ec3dabc0f56d [08:56:25] [02miraheze/ssl] 07MirahezeSSLBot 03ec3dabc - Bot: Update SSL cert for crustypedia.org [08:57:00] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 0.62, 2.01, 3.26 [08:58:16] RECOVERY - mw142 Current Load on mw142 is OK: LOAD OK - total load average: 5.74, 7.11, 9.83 [08:58:23] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.47, 10.71, 10.67 [08:59:33] RECOVERY - polcompball.miraheze.org - reverse DNS on sslhost is OK: SSL OK - polcompball.miraheze.org reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [08:59:34] RECOVERY - es.countryhumans.polandball.wiki - reverse DNS on sslhost is OK: SSL OK - es.countryhumans.polandball.wiki reverse DNS resolves to cp24.miraheze.org - CNAME OK [08:59:38] RECOVERY - wikipwaadia.works - reverse DNS on sslhost is OK: SSL OK - wikipwaadia.works reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:00:27] RECOVERY - mw143 Current Load on mw143 is OK: LOAD OK - total load average: 6.57, 7.53, 9.84 [09:00:31] RECOVERY - wiki.queenscourt.games - reverse DNS on sslhost is OK: SSL OK - wiki.queenscourt.games reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:00:33] RECOVERY - wiki.teessidehackspace.org.uk - reverse DNS on sslhost is OK: SSL OK - wiki.teessidehackspace.org.uk reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:00:34] RECOVERY - pwsc.polandballwiki.com - reverse DNS on sslhost is OK: SSL OK - pwsc.polandballwiki.com reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:00:38] RECOVERY - miraheze.wiki - reverse DNS on sslhost is OK: SSL OK - miraheze.wiki reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:00:47] RECOVERY - vise.dayid.org - reverse DNS on sslhost is OK: SSL OK - vise.dayid.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:00:56] RECOVERY - wiki.graalmilitary.com - reverse DNS on sslhost is OK: SSL OK - wiki.graalmilitary.com reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:00:56] RECOVERY - www.hebammenwiki.de - reverse DNS on sslhost is OK: SSL OK - www.hebammenwiki.de reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:01:01] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.38, 3.34, 2.91 [09:01:05] RECOVERY - www.zenbuddhism.info - reverse DNS on sslhost is OK: SSL OK - www.zenbuddhism.info reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:01:07] RECOVERY - thesimswiki.com - reverse DNS on sslhost is OK: SSL OK - thesimswiki.com reverse DNS resolves to cp34.miraheze.org - CNAME FLAT [09:01:09] RECOVERY - ecole.science - reverse DNS on sslhost is OK: SSL OK - ecole.science reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:01:09] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: SSL OK - lcn.zfc.id.lv reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:01:28] RECOVERY - wiki.moores.tech - reverse DNS on sslhost is OK: SSL OK - wiki.moores.tech reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:01:31] RECOVERY - fc.songcontests.eu - reverse DNS on sslhost is OK: SSL OK - fc.songcontests.eu reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:01:31] RECOVERY - wiki.insideearth.info - reverse DNS on sslhost is OK: SSL OK - wiki.insideearth.info reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:01:32] RECOVERY - reviwiki.info - reverse DNS on sslhost is OK: SSL OK - reviwiki.info reverse DNS resolves to cp25.miraheze.org - CNAME FLAT [09:01:32] RECOVERY - wiki.counterculturelabs.org - reverse DNS on sslhost is OK: SSL OK - wiki.counterculturelabs.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:01:34] RECOVERY - wiki.seamly.io - reverse DNS on sslhost is OK: SSL OK - wiki.seamly.io reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:01:34] RECOVERY - www.sekaipedia.org - reverse DNS on sslhost is OK: SSL OK - www.sekaipedia.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:01:34] RECOVERY - corru.wiki - reverse DNS on sslhost is OK: SSL OK - corru.wiki reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:01:36] RECOVERY - permanentfuturelab.wiki - reverse DNS on sslhost is OK: SSL OK - permanentfuturelab.wiki reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:01:41] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.52, 10.50, 9.63 [09:02:00] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.88, 10.61, 9.95 [09:02:22] RECOVERY - threedomwiki.pcast.site - reverse DNS on sslhost is OK: SSL OK - threedomwiki.pcast.site reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:02:25] RECOVERY - wiki.globasa.net - reverse DNS on sslhost is OK: SSL OK - wiki.globasa.net reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:02:26] RECOVERY - evilgeniuswiki.com - reverse DNS on sslhost is OK: SSL OK - evilgeniuswiki.com reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:02:28] RECOVERY - wiki.umbranet.mandelbrotgames.com - reverse DNS on sslhost is OK: SSL OK - wiki.umbranet.mandelbrotgames.com reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:02:29] RECOVERY - enc.for.uz - reverse DNS on sslhost is OK: SSL OK - enc.for.uz reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:02:35] RECOVERY - yokaiwatchwiki.com - reverse DNS on sslhost is OK: SSL OK - yokaiwatchwiki.com reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:03:01] RECOVERY - wiki.yahyabd.xyz - reverse DNS on sslhost is OK: SSL OK - wiki.yahyabd.xyz reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:03:01] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 2.76, 3.19, 2.92 [09:03:10] PROBLEM - wiki.thehall.xyz - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.thehall.xyz could not be found [09:03:14] PROBLEM - alphabirdswiki.gq - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for alphabirdswiki.gq could not be found [09:03:21] RECOVERY - wiki.aoemods.com - reverse DNS on sslhost is OK: SSL OK - wiki.aoemods.com reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:03:22] RECOVERY - roll-of-arms.com - reverse DNS on sslhost is OK: SSL OK - roll-of-arms.com reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:03:24] PROBLEM - decyclopedia.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for decyclopedia.tk could not be found [09:03:25] RECOVERY - abeyancewiki.bumblebee-studios.se - reverse DNS on sslhost is OK: SSL OK - abeyancewiki.bumblebee-studios.se reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:03:25] RECOVERY - vitriol.spoon.army - reverse DNS on sslhost is OK: SSL OK - vitriol.spoon.army reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:03:29] PROBLEM - www.vanillaanarchy.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.vanillaanarchy.net could not be found [09:03:29] PROBLEM - wiki.hsdfz.org.cn - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.hsdfz.org.cn could not be found [09:03:38] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.19, 10.87, 9.86 [09:03:39] PROBLEM - www.twistwoodtaleswiki.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.twistwoodtaleswiki.com could not be found [09:03:49] PROBLEM - ewgf.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for ewgf.wiki could not be found [09:03:56] RECOVERY - wikicompliance.com - reverse DNS on sslhost is OK: SSL OK - wikicompliance.com reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:04:12] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.74, 11.69, 11.17 [09:04:38] RECOVERY - www.winenjoy.net - reverse DNS on sslhost is OK: SSL OK - www.winenjoy.net reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:04:48] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.zulaclub.net could not be found [09:04:52] RECOVERY - wiki.ricochetuniverse.com - reverse DNS on sslhost is OK: SSL OK - wiki.ricochetuniverse.com reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:04:53] RECOVERY - 9b9t.wiki - reverse DNS on sslhost is OK: SSL OK - 9b9t.wiki reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:04:57] RECOVERY - housing.wiki - reverse DNS on sslhost is OK: SSL OK - housing.wiki reverse DNS resolves to cp35.miraheze.org - CNAME FLAT [09:05:03] RECOVERY - commonswiki.miraheze.org - reverse DNS on sslhost is OK: SSL OK - commonswiki.miraheze.org reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:05:24] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:05:25] RECOVERY - wiki.yuanpi.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.yuanpi.eu.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:05:36] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.20, 10.63, 9.89 [09:05:50] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 8.86, 10.13, 9.94 [09:05:52] RECOVERY - cities.simulz.kr - reverse DNS on sslhost is OK: SSL OK - cities.simulz.kr reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:06:00] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [09:06:09] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 15.49, 12.97, 11.69 [09:06:12] PROBLEM - db142 Backups SQL on db142 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864228 seconds old and 82142 bytes [09:06:13] PROBLEM - txt-dex.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for txt-dex.com could not be found [09:06:43] RECOVERY - civwiki.org - reverse DNS on sslhost is OK: SSL OK - civwiki.org reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:06:47] PROBLEM - wikipedia.yogfront.ooo - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wikipedia.yogfront.ooo could not be found [09:06:49] RECOVERY - wiki.vinesh.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.vinesh.eu.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:06:52] RECOVERY - cyberlaw.ccdcoe.org - reverse DNS on sslhost is OK: SSL OK - cyberlaw.ccdcoe.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:06:53] RECOVERY - small.polandballwiki.com - reverse DNS on sslhost is OK: SSL OK - small.polandballwiki.com reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:06:57] RECOVERY - psycho.engineering - reverse DNS on sslhost is OK: SSL OK - psycho.engineering reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:07:04] RECOVERY - staratlaswiki.org - reverse DNS on sslhost is OK: SSL OK - staratlaswiki.org reverse DNS resolves to cp24.miraheze.org - CNAME FLAT [09:07:05] RECOVERY - wiki.bluevertigo.org - reverse DNS on sslhost is OK: SSL OK - wiki.bluevertigo.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:07:09] RECOVERY - cheeseepedia.org - reverse DNS on sslhost is OK: SSL OK - cheeseepedia.org reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:07:35] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 7.87, 10.06, 9.79 [09:08:05] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 8.29, 11.09, 11.16 [09:08:26] RECOVERY - data.nonbinary.wiki - reverse DNS on sslhost is OK: SSL OK - data.nonbinary.wiki reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:08:27] RECOVERY - wiki.esufranchise.com - reverse DNS on sslhost is OK: SSL OK - wiki.esufranchise.com reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:09:04] RECOVERY - wiki.auraxis.co - reverse DNS on sslhost is OK: SSL OK - wiki.auraxis.co reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:09:05] RECOVERY - wiki.jeremyboyd.org - reverse DNS on sslhost is OK: SSL OK - wiki.jeremyboyd.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:09:15] RECOVERY - wiki.cubestudios.xyz - reverse DNS on sslhost is OK: SSL OK - wiki.cubestudios.xyz reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:09:16] RECOVERY - wiki.veloren.net - reverse DNS on sslhost is OK: SSL OK - wiki.veloren.net reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:09:20] RECOVERY - wiki.pulsus.cc - reverse DNS on sslhost is OK: SSL OK - wiki.pulsus.cc reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:09:25] RECOVERY - mhdh.pj568.eu.org - reverse DNS on sslhost is OK: SSL OK - mhdh.pj568.eu.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:09:35] RECOVERY - n64brew.dev - reverse DNS on sslhost is OK: SSL OK - n64brew.dev reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:09:49] RECOVERY - www.programming.red - reverse DNS on sslhost is OK: SSL OK - www.programming.red reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:10:00] PROBLEM - wiki.maxshub.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.maxshub.net could not be found [09:10:22] RECOVERY - wizardia.wiki - reverse DNS on sslhost is OK: SSL OK - wizardia.wiki reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:10:53] RECOVERY - yoshi.miraheze.org - reverse DNS on sslhost is OK: SSL OK - yoshi.miraheze.org reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:10:54] RECOVERY - freedomplanetwiki.com - reverse DNS on sslhost is OK: SSL OK - freedomplanetwiki.com reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:10:54] RECOVERY - wiki.cyberfurs.org - reverse DNS on sslhost is OK: SSL OK - wiki.cyberfurs.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:11:09] RECOVERY - schizoidnightmares.miraheze.org - reverse DNS on sslhost is OK: SSL OK - schizoidnightmares.miraheze.org reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:11:10] RECOVERY - wiki.infomedia.co.uk - reverse DNS on sslhost is OK: SSL OK - wiki.infomedia.co.uk reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:11:12] RECOVERY - wiki.theatlas.live - reverse DNS on sslhost is OK: SSL OK - wiki.theatlas.live reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:11:13] RECOVERY - wiki.metaiitgn.org - reverse DNS on sslhost is OK: SSL OK - wiki.metaiitgn.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:11:14] RECOVERY - wiki.digitalcandela.com - reverse DNS on sslhost is OK: SSL OK - wiki.digitalcandela.com reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:11:50] RECOVERY - removededm.com - reverse DNS on sslhost is OK: SSL OK - removededm.com reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:11:51] RECOVERY - tech.exputra.com - reverse DNS on sslhost is OK: SSL OK - tech.exputra.com reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:11:52] RECOVERY - fanon.polandballwiki.com - reverse DNS on sslhost is OK: SSL OK - fanon.polandballwiki.com reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:12:08] RECOVERY - wiki.scvo.org - reverse DNS on sslhost is OK: SSL OK - wiki.scvo.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:12:24] RECOVERY - grayravens.com - reverse DNS on sslhost is OK: SSL OK - grayravens.com reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:12:25] RECOVERY - wiki.aetherexplorers.science - reverse DNS on sslhost is OK: SSL OK - wiki.aetherexplorers.science reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:12:27] RECOVERY - petrawiki.org - reverse DNS on sslhost is OK: SSL OK - petrawiki.org reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:12:30] RECOVERY - wiki.yunachannel.com - reverse DNS on sslhost is OK: SSL OK - wiki.yunachannel.com reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:12:32] RECOVERY - wiki.lefrenchmelee.fr - reverse DNS on sslhost is OK: SSL OK - wiki.lefrenchmelee.fr reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:12:47] RECOVERY - wiki.mastodon.kr - reverse DNS on sslhost is OK: SSL OK - wiki.mastodon.kr reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:12:48] RECOVERY - wiki.puucraft.net - reverse DNS on sslhost is OK: SSL OK - wiki.puucraft.net reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:12:53] RECOVERY - wiki.solarconquest.net - reverse DNS on sslhost is OK: SSL OK - wiki.solarconquest.net reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:13:04] RECOVERY - wiki.openhatch.org - reverse DNS on sslhost is OK: SSL OK - wiki.openhatch.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:13:06] RECOVERY - you.r-fit.cc - reverse DNS on sslhost is OK: SSL OK - you.r-fit.cc reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:13:06] RECOVERY - pandorastale.wiki - reverse DNS on sslhost is OK: SSL OK - pandorastale.wiki reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:13:11] RECOVERY - talumn.neptune.wiki - reverse DNS on sslhost is OK: SSL OK - talumn.neptune.wiki reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:13:14] RECOVERY - itadminwiki.pl - reverse DNS on sslhost is OK: SSL OK - itadminwiki.pl reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:13:51] PROBLEM - wiki.rlab.org.uk - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns39.domaincontrol.com.', 'ns40.domaincontrol.com.'], 'CNAME': 'mw-ib.miraheze.org.'} [09:13:55] RECOVERY - wikimas.kr - reverse DNS on sslhost is OK: SSL OK - wikimas.kr reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:14:00] RECOVERY - witchingworld.org - reverse DNS on sslhost is OK: SSL OK - witchingworld.org reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:14:05] PROBLEM - wiki.litek.top - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.408 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [09:14:14] RECOVERY - wiki.ivy.cm - reverse DNS on sslhost is OK: SSL OK - wiki.ivy.cm reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:14:22] PROBLEM - wiki.slimelab.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.slimelab.net could not be found [09:14:39] RECOVERY - nl.xliving.org - reverse DNS on sslhost is OK: SSL OK - nl.xliving.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:14:47] RECOVERY - www.tintinpedia.fr - reverse DNS on sslhost is OK: SSL OK - www.tintinpedia.fr reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:14:49] RECOVERY - project-patterns.com - reverse DNS on sslhost is OK: SSL OK - project-patterns.com reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:14:49] RECOVERY - electowiki.org - reverse DNS on sslhost is OK: SSL OK - electowiki.org reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:15:13] RECOVERY - steem.wiki - reverse DNS on sslhost is OK: SSL OK - steem.wiki reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:15:35] RECOVERY - glitchcity.wiki - reverse DNS on sslhost is OK: SSL OK - glitchcity.wiki reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:15:35] RECOVERY - wiki.make717.org - reverse DNS on sslhost is OK: SSL OK - wiki.make717.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:15:52] RECOVERY - dragonquest2.miraheze.org - reverse DNS on sslhost is OK: SSL OK - dragonquest2.miraheze.org reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:16:00] PROBLEM - trains.cnrailwiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for trains.cnrailwiki.org could not be found [09:16:06] RECOVERY - squareenixwiki.com - reverse DNS on sslhost is OK: SSL OK - squareenixwiki.com reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:16:26] RECOVERY - wiki.geoparkcorumbatai.com.br - reverse DNS on sslhost is OK: SSL OK - wiki.geoparkcorumbatai.com.br reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:17:00] RECOVERY - test2.specialchat.org - reverse DNS on sslhost is OK: SSL OK - test2.specialchat.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:17:07] RECOVERY - publictestwiki.com - reverse DNS on sslhost is OK: SSL OK - publictestwiki.com reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:17:26] RECOVERY - wiki.valkyrienskies.org - reverse DNS on sslhost is OK: SSL OK - wiki.valkyrienskies.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:17:27] RECOVERY - wiki.iurevar.world - reverse DNS on sslhost is OK: SSL OK - wiki.iurevar.world reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:17:29] RECOVERY - wiki.zamnhacking.net - reverse DNS on sslhost is OK: SSL OK - wiki.zamnhacking.net reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:17:33] RECOVERY - touhouplace.wiki - reverse DNS on sslhost is OK: SSL OK - touhouplace.wiki reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:17:58] RECOVERY - wiki.kachow.org - reverse DNS on sslhost is OK: SSL OK - wiki.kachow.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:17:58] RECOVERY - pandorastale.miraheze.org - reverse DNS on sslhost is OK: SSL OK - pandorastale.miraheze.org reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:18:03] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.60, 11.09, 10.89 [09:18:04] RECOVERY - www.mh142.com - reverse DNS on sslhost is OK: SSL OK - www.mh142.com reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:19:13] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:19:14] RECOVERY - pornwiki.org - reverse DNS on sslhost is OK: SSL OK - pornwiki.org reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:19:18] RECOVERY - opendatascot.org - reverse DNS on sslhost is OK: SSL OK - opendatascot.org reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:19:30] RECOVERY - wiki.denby.tech - reverse DNS on sslhost is OK: SSL OK - wiki.denby.tech reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:19:35] RECOVERY - wikislamica.org - reverse DNS on sslhost is OK: SSL OK - wikislamica.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:19:36] RECOVERY - echoes-wiki.com - reverse DNS on sslhost is OK: SSL OK - echoes-wiki.com reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:19:37] RECOVERY - christipedia.nl - reverse DNS on sslhost is OK: SSL OK - christipedia.nl reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:19:47] RECOVERY - www.johanloopmans.nl - reverse DNS on sslhost is OK: SSL OK - www.johanloopmans.nl reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:19:48] RECOVERY - wiki.controlistes.fr - reverse DNS on sslhost is OK: SSL OK - wiki.controlistes.fr reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:20:01] RECOVERY - wiki.candelabrem.com - reverse DNS on sslhost is OK: SSL OK - wiki.candelabrem.com reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:20:06] PROBLEM - wiki.seamly.net - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns1.dreamhost.com.', 'ns2.dreamhost.com.', 'ns3.dreamhost.com.'], 'CNAME': 'wiki.seamly.io.'} [09:20:07] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.33, 10.88, 10.83 [09:20:11] PROBLEM - wiki.bloxotel.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.bloxotel.tk could not be found [09:20:17] PROBLEM - developer-and-testers.wiki.feign-wiki.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for developer-and-testers.wiki.feign-wiki.tk could not be found [09:20:19] PROBLEM - minehut.tarna.dev - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for minehut.tarna.dev could not be found [09:20:25] PROBLEM - wariopedia.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wariopedia.com All nameservers failed to answer the query. [09:21:11] RECOVERY - wiki.mostly.vet - reverse DNS on sslhost is OK: SSL OK - wiki.mostly.vet reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:21:12] RECOVERY - fallofsanctuary.com - reverse DNS on sslhost is OK: SSL OK - fallofsanctuary.com reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:21:12] RECOVERY - www.dariawiki.org - reverse DNS on sslhost is OK: SSL OK - www.dariawiki.org reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:21:22] RECOVERY - www.christipedia.nl - reverse DNS on sslhost is OK: SSL OK - www.christipedia.nl reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:21:22] RECOVERY - rare.miraheze.org - reverse DNS on sslhost is OK: SSL OK - rare.miraheze.org reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:21:22] RECOVERY - wiki.closai.com - reverse DNS on sslhost is OK: SSL OK - wiki.closai.com reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:21:24] RECOVERY - worldtobuild.wiki - reverse DNS on sslhost is OK: SSL OK - worldtobuild.wiki reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK [09:21:27] RECOVERY - www.moiswiki.net - reverse DNS on sslhost is OK: SSL OK - www.moiswiki.net reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:21:38] RECOVERY - o.kaoyu666.cn - reverse DNS on sslhost is OK: SSL OK - o.kaoyu666.cn reverse DNS resolves to cp25.miraheze.org - CNAME OK [09:22:03] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.14, 10.98, 10.84 [09:22:11] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 17.86, 13.10, 10.95 [09:22:26] RECOVERY - runzeppelin.ru - reverse DNS on sslhost is OK: SSL OK - runzeppelin.ru reverse DNS resolves to cp24.miraheze.org - NS RECORDS OK [09:22:27] RECOVERY - wiki.anempireofdreams.com - reverse DNS on sslhost is OK: SSL OK - wiki.anempireofdreams.com reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:23:32] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.74, 10.85, 10.05 [09:24:03] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.31, 11.51, 11.08 [09:25:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.46, 10.61, 9.78 [09:26:07] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.34, 11.97, 10.96 [09:27:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 12.42, 11.14, 10.07 [09:28:04] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.82, 12.61, 11.33 [09:29:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 7.99, 9.95, 9.77 [09:30:02] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 8.81, 11.38, 11.04 [09:30:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.04, 11.82, 11.36 [09:32:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 9.46, 10.76, 11.02 [09:33:57] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.22, 11.58, 11.15 [09:34:35] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [09:35:31] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 9.41, 9.84, 10.13 [09:39:39] PROBLEM - mon141 Current Load on mon141 is WARNING: LOAD WARNING - total load average: 0.36, 0.72, 3.64 [09:41:39] RECOVERY - mon141 Current Load on mon141 is OK: LOAD OK - total load average: 0.92, 0.77, 3.29 [09:42:45] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.31, 10.00, 9.66 [09:44:40] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 9.81, 9.74, 9.59 [09:45:43] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.99, 11.97, 11.86 [09:49:01] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.18, 6.24, 5.63 [09:50:56] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.23, 6.00, 5.62 [09:54:03] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 8.67, 9.48, 10.12 [09:55:20] RECOVERY - crustypedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'crustypedia.org' will expire on Sat 30 Mar 2024 07:56:17 GMT +0000. [09:56:49] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.17, 10.50, 10.22 [09:58:44] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 9.12, 10.15, 10.14 [10:03:21] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 7.45, 8.27, 9.82 [10:09:11] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.17, 10.05, 10.06 [10:11:09] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 8.68, 9.25, 9.75 [10:19:59] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppetdb] [10:26:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.32, 10.37, 9.42 [10:28:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 9.51, 10.38, 9.56 [10:30:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 8.24, 9.59, 9.37 [10:34:03] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 9.23, 10.21, 9.74 [10:38:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 9.70, 9.90, 9.72 [10:45:23] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.22, 10.15, 9.11 [10:47:21] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 9.52, 9.94, 9.16 [10:53:11] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 9.92, 10.31, 9.50 [10:54:57] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.79, 11.48, 10.37 [10:55:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.37, 10.50, 9.10 [10:55:09] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 13.06, 11.37, 9.98 [10:56:54] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 9.53, 10.76, 10.23 [10:57:06] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.17, 11.23, 10.09 [10:57:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 7.70, 9.66, 8.97 [10:57:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.07, 10.29, 9.00 [10:58:50] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 15.24, 12.34, 10.87 [10:59:04] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 13.55, 11.86, 10.44 [11:00:46] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 8.38, 10.79, 10.47 [11:01:01] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 6.46, 9.99, 9.94 [11:01:31] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 7.78, 9.92, 9.26 [11:04:39] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 8.23, 9.82, 10.15 [11:07:59] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:36:46] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 9.26, 10.38, 9.97 [11:37:10] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.31, 11.43, 10.15 [11:38:42] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 8.66, 9.60, 9.72 [11:41:06] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 8.98, 9.61, 9.67 [11:50:10] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.11, 9.59, 9.19 [11:54:00] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 9.03, 9.61, 9.32 [11:54:51] PROBLEM - cloud14 Puppet on cloud14 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [12:06:55] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.95, 10.73, 9.81 [12:07:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.90, 9.51, 8.62 [12:09:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 8.40, 9.09, 8.58 [12:11:07] PROBLEM - barengreza.my.id - LetsEncrypt on sslhost is CRITICAL: connect to address barengreza.my.id and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [12:14:16] PROBLEM - mail121 Current Load on mail121 is WARNING: LOAD WARNING - total load average: 1.97, 1.46, 0.80 [12:16:16] RECOVERY - mail121 Current Load on mail121 is OK: LOAD OK - total load average: 0.46, 1.05, 0.73 [12:18:33] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 9.06, 9.84, 9.91 [12:20:54] RECOVERY - cloud14 Puppet on cloud14 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures [12:31:12] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.76, 11.15, 10.33 [12:32:35] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.43, 10.93, 9.75 [12:33:08] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.77, 11.92, 10.72 [12:33:53] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.75, 11.33, 9.97 [12:34:30] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.43, 11.13, 9.96 [12:35:46] PROBLEM - mw143 Current Load on mw143 is WARNING: LOAD WARNING - total load average: 11.45, 9.49, 7.74 [12:37:01] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.12, 11.54, 10.83 [12:37:43] RECOVERY - mw143 Current Load on mw143 is OK: LOAD OK - total load average: 5.85, 8.10, 7.44 [12:38:21] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 9.56, 10.12, 9.81 [12:42:13] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.24, 10.84, 10.19 [12:43:41] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.13, 11.88, 11.10 [12:43:48] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.05, 10.34, 9.63 [12:45:38] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.38, 11.98, 11.22 [12:47:36] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 8.78, 11.17, 11.04 [12:47:58] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 6.61, 9.52, 9.85 [12:48:39] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 6.84, 9.02, 9.98 [12:49:32] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 9.74, 9.94, 9.71 [12:53:46] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 9.86, 10.67, 10.28 [12:58:25] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 8.47, 10.17, 10.25 [12:59:32] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 9.26, 9.76, 9.99 [13:00:21] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 7.88, 9.38, 9.94 [13:01:19] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 7.33, 9.16, 10.10 [13:07:09] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.85, 10.31, 10.27 [13:08:10] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 15.44, 11.59, 10.40 [13:09:07] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 8.74, 9.56, 10.00 [13:10:06] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 9.85, 11.21, 10.43 [13:14:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 8.81, 9.88, 10.07 [13:16:55] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.01, 10.24, 10.02 [13:18:52] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 7.90, 9.43, 9.75 [13:24:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.24, 11.21, 10.33 [13:26:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 14.09, 11.51, 10.53 [13:28:41] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.80, 11.19, 10.09 [13:32:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 9.77, 11.11, 10.36 [13:35:42] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.07, 10.29, 9.37 [13:37:37] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 9.93, 10.16, 9.43 [13:40:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.22, 11.85, 11.59 [13:41:27] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.63, 10.74, 9.81 [13:42:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.01, 12.09, 11.72 [13:44:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.74, 11.62, 11.59 [13:45:17] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 14.83, 12.09, 10.49 [13:45:58] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.92, 11.85, 10.94 [13:46:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.21, 12.06, 11.76 [13:46:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 9.47, 10.93, 11.75 [13:47:54] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.94, 11.63, 10.98 [13:51:44] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 13.32, 11.76, 11.12 [13:53:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 8.37, 11.25, 10.95 [13:53:39] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 9.85, 11.32, 11.05 [13:54:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 15.82, 12.48, 11.91 [13:55:35] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 13.18, 12.08, 11.36 [13:57:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 13.09, 11.80, 11.21 [13:59:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 9.12, 10.89, 10.96 [14:00:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 9.72, 11.33, 11.71 [14:02:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 13.14, 11.85, 11.84 [14:05:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 9.77, 11.79, 11.78 [14:07:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 14.85, 13.33, 12.36 [14:11:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 13.08, 12.06, 11.31 [14:15:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.65, 11.74, 11.97 [14:19:59] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppetdb] [14:21:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.93, 12.12, 12.00 [14:23:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.46, 11.93, 11.82 [14:25:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 12.05, 12.01, 11.86 [14:27:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.43, 11.36, 11.64 [14:29:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 7.56, 10.96, 11.72 [14:30:57] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.47, 3.05, 2.13 [14:31:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 12.25, 10.73, 11.22 [14:31:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 15.61, 13.07, 12.42 [14:32:54] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.60, 2.81, 2.14 [14:35:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.50, 10.94, 11.22 [14:38:44] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 6.34, 4.07, 2.79 [14:40:41] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.44, 3.72, 2.81 [14:42:38] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.65, 3.40, 2.81 [14:47:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.00, 11.19, 11.93 [14:48:28] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.23, 4.12, 3.29 [14:49:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 12.05, 11.09, 11.02 [14:49:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.57, 11.83, 12.09 [14:50:27] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.34, 3.77, 3.26 [14:51:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.30, 11.26, 11.11 [14:52:31] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 7.58, 4.85, 3.69 [14:53:17] [02mw-config] 07R4356th reviewed pull request 03#5410 commit - 13https://github.com/miraheze/mw-config/pull/5410#discussion_r1438894787 [14:54:11] PROBLEM - os141 PowerDNS Recursor on os141 is CRITICAL: CRITICAL - Plugin timed out while executing system call [14:54:40] PROBLEM - mw142 Current Load on mw142 is CRITICAL: LOAD CRITICAL - total load average: 12.76, 11.55, 9.30 [14:55:27] [02mw-config] 07R4356th reviewed pull request 03#5410 commit - 13https://github.com/miraheze/mw-config/pull/5410#discussion_r1438895092 [14:56:11] RECOVERY - os141 PowerDNS Recursor on os141 is OK: DNS OK: 0.993 seconds response time. miraheze.org returns 2001:41d0:801:2000::3a18,2001:41d0:801:2000::5d68,51.195.201.140,51.89.139.24 [14:56:36] RECOVERY - mw142 Current Load on mw142 is OK: LOAD OK - total load average: 6.18, 9.46, 8.80 [14:58:49] [02mw-config] 07R4356th reviewed pull request 03#5410 commit - 13https://github.com/miraheze/mw-config/pull/5410#discussion_r1438895490 [15:01:29] [02mw-config] 07R4356th reviewed pull request 03#5410 commit - 13https://github.com/miraheze/mw-config/pull/5410#discussion_r1438895769 [15:01:36] PROBLEM - mw141 Current Load on mw141 is WARNING: LOAD WARNING - total load average: 11.25, 10.56, 9.21 [15:03:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 9.32, 10.99, 11.87 [15:03:37] RECOVERY - mw141 Current Load on mw141 is OK: LOAD OK - total load average: 9.11, 10.16, 9.23 [15:07:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 9.90, 9.61, 10.08 [15:07:36] PROBLEM - mw141 Current Load on mw141 is WARNING: LOAD WARNING - total load average: 10.36, 10.87, 9.79 [15:07:59] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:11:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.84, 11.71, 11.79 [15:11:36] RECOVERY - mw141 Current Load on mw141 is OK: LOAD OK - total load average: 8.11, 9.81, 9.68 [15:12:12] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 1.46, 3.14, 3.76 [15:13:57] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 13.21, 10.58, 10.22 [15:15:52] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.42, 10.81, 10.37 [15:16:11] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 1.98, 2.47, 3.32 [15:17:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.32, 11.61, 11.79 [15:17:47] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 7.36, 9.68, 10.02 [15:21:32] [02mw-config] 07R4356th synchronize pull request 03#5408: Add $wgDiscordName to ManageWiki (T11459) - 13https://github.com/miraheze/mw-config/pull/5408 [15:22:30] miraheze/mw-config - R4356th the build has errored. [15:23:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 13.53, 12.15, 11.90 [15:23:32] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.54, 10.40, 10.17 [15:25:27] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 8.88, 10.10, 10.10 [15:25:55] [02mw-config] 07R4356th synchronize pull request 03#5408: Add $wgDiscordName to ManageWiki (T11459) - 13https://github.com/miraheze/mw-config/pull/5408 [15:26:52] miraheze/mw-config - R4356th the build has errored. [15:30:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 8.73, 10.44, 11.73 [15:30:32] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.20, 6.74, 6.04 [15:32:27] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.95, 6.44, 6.00 [15:32:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 7.90, 9.97, 11.72 [15:35:27] [02mw-config] 07R4356th synchronize pull request 03#5408: Add $wgDiscordName to ManageWiki (T11459) - 13https://github.com/miraheze/mw-config/pull/5408 [15:35:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.12, 11.68, 11.97 [15:36:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 16.59, 12.83, 12.17 [15:36:18] miraheze/mw-config - R4356th the build passed. [15:36:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 14.06, 11.94, 12.09 [15:38:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 8.43, 10.42, 11.51 [15:40:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.94, 11.72, 11.83 [15:45:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.65, 11.21, 11.39 [15:47:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 9.22, 10.33, 11.05 [15:50:46] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 7.96, 8.81, 10.19 [15:52:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 9.23, 8.79, 10.09 [15:55:31] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 7.93, 8.79, 10.02 [16:01:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.25, 6.59, 6.20 [16:03:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.91, 6.22, 6.10 [16:10:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.33, 10.62, 10.11 [16:12:43] PROBLEM - cloud10 Puppet on cloud10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [16:16:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 8.36, 9.71, 9.95 [16:32:03] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.63, 6.58, 5.85 [16:34:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.90, 10.96, 9.86 [16:35:00] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 13.57, 10.99, 10.02 [16:36:02] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 4.86, 5.96, 5.79 [16:36:05] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.73, 10.60, 9.85 [16:38:00] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 8.98, 10.07, 9.74 [16:38:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 9.72, 10.98, 10.18 [16:38:55] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.83, 11.02, 10.25 [16:40:43] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:40:53] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 13.35, 12.28, 10.82 [16:41:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.74, 9.95, 8.99 [16:41:52] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 9.08, 10.75, 10.17 [16:42:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 7.78, 10.10, 10.11 [16:42:17] [02mw-config] 07Tali64 synchronize pull request 03#5410: Improvements to CW canned messages - 13https://github.com/miraheze/mw-config/pull/5410 [16:42:18] [02mw-config] 07Tali64 reviewed pull request 03#5410 commit - 13https://github.com/miraheze/mw-config/pull/5410#discussion_r1438907386 [16:45:42] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.39, 11.10, 10.37 [16:46:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 9.05, 10.43, 10.29 [16:46:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 6.75, 10.25, 10.52 [16:47:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 6.64, 8.93, 8.92 [16:47:38] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.62, 10.64, 10.28 [16:48:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 9.25, 9.74, 10.04 [16:48:46] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 8.14, 9.46, 10.18 [16:49:33] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.43, 11.04, 10.45 [16:49:54] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.69, 8.19, 6.89 [16:50:48] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.24, 7.15, 6.33 [16:51:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.51, 11.23, 10.63 [16:51:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.29, 7.96, 6.98 [16:52:45] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 9.27, 7.62, 6.58 [16:52:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.95, 10.54, 10.46 [16:54:42] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.24, 6.69, 6.36 [16:55:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.63, 6.34, 6.51 [16:56:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 13.11, 11.45, 10.82 [16:56:54] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 14.13, 11.25, 10.47 [16:58:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.43, 11.06, 10.73 [16:58:50] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.67, 11.05, 10.50 [16:59:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.02, 7.03, 6.77 [17:03:31] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 7.65, 9.15, 10.04 [17:03:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.64, 6.38, 6.57 [17:06:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.39, 10.88, 10.69 [17:08:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 8.03, 10.29, 10.54 [17:10:28] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 15.04, 11.49, 10.81 [17:12:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 14.50, 11.41, 10.84 [17:14:19] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.47, 10.67, 10.33 [17:18:09] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.99, 11.38, 10.67 [17:20:04] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.85, 11.38, 10.75 [17:20:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.51, 11.87, 11.45 [17:23:59] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.23, 10.50, 9.95 [17:24:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 15.17, 13.04, 11.97 [17:25:54] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 12.10, 11.46, 10.39 [17:26:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.17, 11.89, 11.68 [17:27:49] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 9.96, 11.28, 10.47 [17:28:33] !log [agent@mwtask141] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.json /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/extensions/MirahezeMagic/maintenance/resetWikiCaches.php (END - exit=0) [17:28:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:29:41] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.16, 11.81, 11.20 [17:29:43] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 8.31, 10.13, 10.14 [17:31:36] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.88, 11.57, 11.17 [17:32:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.12, 11.43, 11.52 [17:34:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.68, 11.35, 11.50 [17:36:01] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.71, 3.00, 2.73 [17:38:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 7.50, 9.60, 11.83 [17:39:31] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 9.51, 9.41, 10.10 [17:39:50] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 3.07, 3.19, 2.89 [17:40:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.97, 11.02, 12.08 [17:40:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.55, 10.53, 10.79 [17:41:14] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.67, 9.86, 9.69 [17:43:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 13.01, 11.44, 10.76 [17:44:57] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.14, 6.99, 6.30 [17:45:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 9.37, 9.79, 9.70 [17:45:32] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.38, 11.08, 10.70 [17:46:53] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.45, 6.77, 6.30 [17:47:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 14.56, 12.27, 11.17 [17:49:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.37, 11.84, 11.15 [17:50:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.45, 11.94, 11.59 [17:52:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.21, 12.54, 11.88 [17:53:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.54, 11.78, 11.26 [17:55:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.01, 11.65, 11.28 [17:59:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 16.63, 13.21, 11.92 [18:02:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.37, 11.91, 11.86 [18:04:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.23, 11.81, 11.82 [18:07:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.38, 11.92, 11.94 [18:08:30] PROBLEM - wiki.zymonic.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.zymonic.com All nameservers failed to answer the query. [18:08:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.57, 11.91, 11.97 [18:12:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.70, 11.79, 11.88 [18:13:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.83, 10.50, 9.93 [18:13:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.19, 11.49, 11.68 [18:14:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.15, 11.78, 11.88 [18:15:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.07, 11.16, 11.56 [18:17:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 7.62, 9.46, 9.69 [18:18:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.50, 11.09, 11.68 [18:19:59] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppetdb] [18:20:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.53, 11.75, 11.84 [18:20:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.40, 11.10, 11.42 [18:21:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.42, 11.48, 11.55 [18:22:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.79, 11.58, 11.78 [18:22:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.28, 11.06, 11.39 [18:23:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 10.53, 10.42, 10.03 [18:23:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.12, 11.14, 11.40 [18:24:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.67, 11.65, 11.76 [18:24:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 13.05, 11.56, 11.51 [18:26:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.26, 11.83, 11.84 [18:26:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.41, 11.42, 11.47 [18:29:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 7.12, 9.26, 9.74 [18:36:26] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.08, 6.90, 6.16 [18:37:28] PROBLEM - wiki.zymonic.com - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [18:37:31] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 6.67, 8.28, 9.94 [18:38:22] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.03, 6.63, 6.14 [18:40:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 9.37, 8.92, 10.19 [18:40:46] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 7.03, 8.34, 9.85 [18:56:00] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.14, 10.11, 9.65 [18:56:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.72, 11.24, 10.09 [18:57:55] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 8.60, 9.80, 9.60 [18:58:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.14, 11.47, 10.45 [18:59:53] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 9.91, 10.22, 9.31 [19:00:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 9.90, 10.77, 10.31 [19:00:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.26, 11.27, 10.48 [19:01:47] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.98, 11.25, 10.20 [19:05:38] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 8.10, 9.87, 9.60 [19:05:38] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 8.21, 9.53, 9.73 [19:06:27] PROBLEM - wiki.zymonic.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.zymonic.com All nameservers failed to answer the query. [19:07:59] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:08:46] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 4.22, 8.30, 9.62 [19:10:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 6.79, 9.06, 9.90 [19:32:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.42, 10.24, 9.65 [19:34:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 8.63, 9.43, 9.41 [19:36:28] PROBLEM - wiki.zymonic.com - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.406 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [19:51:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.83, 9.54, 8.70 [19:52:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 15.32, 10.96, 9.62 [19:52:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.26, 7.19, 6.34 [19:52:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.32, 11.08, 9.84 [19:53:31] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 8.16, 9.39, 8.78 [19:54:00] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.39, 6.48, 6.18 [19:54:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 7.83, 9.37, 9.19 [19:54:46] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 8.01, 9.77, 9.50 [20:15:15] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.40, 2.88, 1.77 [20:17:15] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 1.05, 2.08, 1.60 [20:22:00] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.43, 7.03, 6.59 [20:23:55] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.29, 6.68, 6.54 [20:32:28] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.10, 6.42, 6.05 [20:33:48] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.18, 10.91, 10.22 [20:35:12] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.49, 10.66, 9.81 [20:35:45] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.06, 10.87, 10.30 [20:36:20] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.84, 6.19, 6.05 [20:36:22] PROBLEM - wiki.zymonic.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.zymonic.com All nameservers failed to answer the query. [20:41:01] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 7.25, 9.52, 9.66 [20:41:38] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 6.72, 9.49, 9.98 [20:45:18] PROBLEM - developer-and-testers.wiki.feign-wiki.tk - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - developer-and-testers.wiki.feign-wiki.tk All nameservers failed to answer the query. [20:51:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.50, 6.83, 6.18 [20:52:44] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.29, 11.13, 10.29 [20:53:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.33, 6.11, 5.99 [20:54:18] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.22, 10.18, 9.91 [20:54:40] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.64, 11.29, 10.45 [20:56:16] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 8.90, 9.64, 9.74 [20:56:36] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 7.97, 10.09, 10.12 [21:06:02] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 13.49, 11.36, 10.35 [21:06:28] PROBLEM - wiki.zymonic.com - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.407 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [21:07:19] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 9.65, 10.40, 10.22 [21:07:59] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 8.56, 10.34, 10.10 [21:11:12] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 7.97, 9.62, 9.98 [21:11:54] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 6.82, 9.46, 9.87 [21:15:14] PROBLEM - developer-and-testers.wiki.feign-wiki.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for developer-and-testers.wiki.feign-wiki.tk could not be found [21:35:20] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 17.20, 11.19, 9.46 [21:37:26] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.20, 10.88, 9.58 [21:39:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.56, 10.26, 8.48 [21:39:15] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 9.86, 10.99, 9.84 [21:39:22] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 7.28, 9.56, 9.26 [21:41:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 8.17, 9.36, 8.36 [21:42:55] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [21:43:11] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 8.49, 10.11, 9.75 [21:46:55] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [21:50:56] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 9.62, 10.37, 10.00 [21:52:00] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.67, 10.05, 9.08 [21:52:54] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 13.11, 11.22, 10.34 [21:53:55] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 13.60, 11.01, 9.52 [21:55:50] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 9.54, 10.66, 9.59 [21:56:49] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.22, 11.62, 10.75 [21:56:53] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.11, 10.91, 10.00 [21:57:46] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 6.82, 9.41, 9.26 [22:00:46] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 8.31, 10.13, 9.90 [22:04:41] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 9.73, 10.29, 10.01 [22:04:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.29, 11.22, 10.78 [22:06:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.14, 11.42, 10.96 [22:08:33] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 8.59, 9.83, 9.93 [22:12:46] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 7.84, 8.96, 9.99 [22:20:14] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.14, 10.88, 9.99 [22:21:59] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppetdb] [22:28:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.71, 11.63, 10.84 [22:34:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.29, 11.31, 10.85 [22:34:34] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [22:37:17] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.59, 10.12, 9.96 [22:41:12] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.94, 11.20, 10.41 [22:42:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 9.68, 11.18, 11.10 [22:43:09] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 8.63, 9.95, 10.03 [22:45:07] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 14.20, 10.71, 9.83 [22:47:03] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 8.79, 10.34, 9.83 [22:48:58] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 8.16, 9.59, 9.61 [22:50:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 8.98, 9.23, 10.20 [22:58:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.64, 10.01, 10.10 [23:00:02] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 9.14, 9.49, 9.89 [23:04:34] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [23:07:59] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:12:57] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.22, 10.35, 10.17 [23:13:29] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.99, 10.78, 10.14 [23:13:58] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.86, 10.29, 9.57 [23:14:54] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 9.80, 10.11, 10.10 [23:15:53] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 8.92, 9.79, 9.47 [23:17:24] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.32, 11.35, 10.53 [23:23:36] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.57, 10.56, 9.90 [23:25:31] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 8.18, 9.88, 9.75 [23:27:12] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 14.75, 12.11, 11.12 [23:29:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.50, 11.69, 10.55 [23:30:29] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.64, 10.52, 10.18 [23:31:32] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.21, 11.77, 10.75 [23:32:25] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.69, 11.60, 10.61 [23:33:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.45, 10.97, 9.68 [23:35:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 12.12, 11.81, 11.01 [23:36:27] PROBLEM - wiki.zymonic.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.zymonic.com All nameservers failed to answer the query. [23:37:31] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 10.42, 11.22, 10.89 [23:40:10] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 7.17, 11.10, 10.96 [23:40:55] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 5.46, 9.87, 11.15 [23:41:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 7.56, 9.27, 9.52 [23:42:06] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.08, 11.38, 11.04 [23:43:31] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 7.75, 9.31, 10.13 [23:44:03] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.06, 11.22, 11.02 [23:48:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 14.63, 11.91, 11.26 [23:49:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.57, 10.89, 10.22 [23:50:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.54, 11.69, 11.26 [23:52:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.21, 10.39, 10.71 [23:54:02] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.70, 11.43, 11.19 [23:54:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.71, 10.24, 10.61 [23:55:07] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 12.57, 11.31, 10.57 [23:56:02] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 10.65, 11.76, 11.39 [23:57:07] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 9.01, 10.38, 10.32 [23:58:46] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 9.54, 9.43, 10.17 [23:59:07] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 7.26, 9.56, 10.05