[00:00:23] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.10, 7.49, 6.99 [00:04:19] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.30, 7.41, 7.14 [00:05:20] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.35, 7.71, 7.28 [00:07:36] PROBLEM - cp24 HTTPS on cp24 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [00:09:10] PROBLEM - cp34 HTTPS on cp34 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [00:09:26] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.61, 3.13, 2.34 [00:09:33] PROBLEM - cp24 HTTPS on cp24 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.653 second response time [00:10:12] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.15, 7.45, 7.19 [00:10:26] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 12.95, 10.63, 8.16 [00:10:27] PROBLEM - os131 Current Load on os131 is CRITICAL: LOAD CRITICAL - total load average: 6.49, 4.59, 2.60 [00:10:55] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.59, 10.92, 8.21 [00:11:05] PROBLEM - cp34 HTTPS on cp34 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.854 second response time [00:11:26] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 3.30, 3.08, 2.41 [00:12:26] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 8.48, 9.59, 8.08 [00:12:55] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 9.38, 9.86, 8.12 [00:16:16] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.17, 6.58, 5.97 [00:16:48] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.91, 7.39, 7.13 [00:18:14] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 6.22, 6.21, 5.90 [00:20:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 5.20, 6.84, 7.02 [00:22:22] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 2.83, 3.65, 3.26 [00:23:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.93, 7.70, 7.76 [00:24:22] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 2.68, 3.33, 3.19 [00:24:43] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.45, 6.22, 6.73 [00:30:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.31, 6.79, 6.73 [00:32:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.10, 6.95, 6.80 [00:36:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 10.64, 7.73, 7.10 [00:39:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.62, 7.56, 7.26 [00:40:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.26, 7.63, 7.24 [00:40:51] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.91, 3.44, 2.74 [00:41:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.29, 7.05, 7.11 [00:42:47] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.91, 3.05, 2.68 [00:44:23] PROBLEM - os131 Current Load on os131 is CRITICAL: LOAD CRITICAL - total load average: 8.10, 4.88, 3.58 [00:44:43] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 4.44, 6.21, 6.79 [00:44:56] PROBLEM - mw134 HTTPS on mw134 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Connection timed out after 10004 milliseconds [00:45:08] PROBLEM - mw143 HTTPS on mw143 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [00:45:10] PROBLEM - cp34 HTTPS on cp34 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [00:45:26] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 5.03, 3.67, 2.97 [00:46:26] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.35, 10.01, 7.89 [00:46:47] PROBLEM - cp35 HTTPS on cp35 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [00:46:59] PROBLEM - mw134 HTTPS on mw134 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 3.069 second response time [00:47:04] PROBLEM - cp34 HTTPS on cp34 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.593 second response time [00:47:07] PROBLEM - mw143 HTTPS on mw143 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.696 second response time [00:47:19] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 16.12, 10.90, 7.83 [00:47:59] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 15.23, 12.12, 8.91 [00:48:26] PROBLEM - cp25 HTTPS on cp25 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10001 milliseconds with 0 bytes received [00:48:33] PROBLEM - mw141 HTTPS on mw141 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [00:49:08] PROBLEM - mw134 HTTPS on mw134 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [00:49:22] PROBLEM - mw142 HTTPS on mw142 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [00:49:36] PROBLEM - cp24 HTTPS on cp24 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [00:50:01] PROBLEM - mw133 HTTPS on mw133 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10000 milliseconds with 0 bytes received [00:50:26] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 16.77, 13.13, 9.55 [00:50:32] PROBLEM - mw141 HTTPS on mw141 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.605 second response time [00:50:37] PROBLEM - cp25 Varnish Backends on cp25 is CRITICAL: 6 backends are down. mw131 mw132 mw141 mw142 mw133 mw143 [00:50:41] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.91, 4.06, 3.26 [00:50:48] PROBLEM - cp35 HTTPS on cp35 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.580 second response time [00:50:48] PROBLEM - os141 PowerDNS Recursor on os141 is CRITICAL: CRITICAL - Plugin timed out while executing system call [00:51:13] PROBLEM - cp35 Varnish Backends on cp35 is CRITICAL: 1 backends are down. mw134 [00:51:16] PROBLEM - mw142 HTTPS on mw142 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.509 second response time [00:51:17] PROBLEM - mw134 HTTPS on mw134 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 9.382 second response time [00:51:19] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 8.34, 11.35, 8.82 [00:51:34] PROBLEM - cp24 HTTPS on cp24 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 1.448 second response time [00:51:59] PROBLEM - mw133 HTTPS on mw133 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 1.766 second response time [00:52:22] PROBLEM - cp25 HTTPS on cp25 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.834 second response time [00:52:26] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 7.15, 10.25, 8.90 [00:52:37] RECOVERY - cp25 Varnish Backends on cp25 is OK: All 19 backends are healthy [00:52:41] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.45, 3.83, 3.28 [00:52:48] RECOVERY - os141 PowerDNS Recursor on os141 is OK: DNS OK: 2.219 seconds response time. miraheze.org returns 2001:41d0:801:2000::3a18,2001:41d0:801:2000::5d68,51.195.201.140,51.89.139.24 [00:53:13] RECOVERY - cp35 Varnish Backends on cp35 is OK: All 19 backends are healthy [00:53:19] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 4.85, 9.03, 8.27 [00:53:26] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 2.44, 3.79, 3.45 [00:53:55] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 5.89, 9.97, 9.34 [00:54:26] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 7.86, 9.33, 8.71 [00:55:26] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 1.84, 3.11, 3.24 [00:55:58] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.29, 6.14, 6.64 [00:57:10] PROBLEM - cp34 HTTPS on cp34 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [00:57:12] PROBLEM - mw143 HTTPS on mw143 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [00:58:22] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 1.78, 3.24, 4.00 [00:59:04] PROBLEM - cp34 HTTPS on cp34 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.629 second response time [00:59:10] PROBLEM - mw143 HTTPS on mw143 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.510 second response time [00:59:59] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.86, 7.05, 6.89 [01:00:29] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.92, 3.33, 3.27 [01:01:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.81, 7.99, 7.28 [01:03:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.53, 7.92, 7.35 [01:04:24] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.50, 4.48, 3.75 [01:04:34] PROBLEM - mw141 HTTPS on mw141 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10000 milliseconds with 0 bytes received [01:05:34] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.96, 6.83, 6.58 [01:06:33] PROBLEM - mw141 HTTPS on mw141 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.704 second response time [01:07:29] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.81, 7.51, 6.84 [01:07:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.40, 8.76, 7.81 [01:09:23] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.01, 6.47, 6.54 [01:09:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.14, 7.74, 7.56 [01:10:26] PROBLEM - cp25 HTTPS on cp25 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [01:10:40] PROBLEM - mw141 HTTPS on mw141 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10005 milliseconds with 0 bytes received [01:12:22] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 2.47, 2.75, 3.26 [01:12:24] PROBLEM - cp25 HTTPS on cp25 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.755 second response time [01:12:39] PROBLEM - mw141 HTTPS on mw141 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.531 second response time [01:17:02] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.09, 6.51, 6.47 [01:18:56] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.26, 6.27, 6.38 [01:22:46] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.23, 6.85, 6.64 [01:23:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.76, 7.48, 7.20 [01:25:08] PROBLEM - mw143 HTTPS on mw143 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10000 milliseconds with 0 bytes received [01:25:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.63, 7.42, 7.21 [01:26:43] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.02, 6.31, 6.50 [01:27:07] PROBLEM - mw143 HTTPS on mw143 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.927 second response time [01:31:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.91, 7.78, 7.30 [01:33:49] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.17, 3.49, 3.95 [01:33:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.83, 7.37, 7.21 [01:39:45] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.34, 3.80, 3.87 [01:39:58] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.26, 6.10, 6.68 [01:40:41] PROBLEM - mw141 HTTPS on mw141 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [01:42:41] PROBLEM - mw141 HTTPS on mw141 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.821 second response time [01:43:39] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 2.16, 3.55, 3.82 [01:45:21] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.80, 7.23, 6.62 [01:45:37] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.28, 4.17, 4.00 [01:45:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.34, 7.22, 7.04 [01:46:55] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.40/maintenance/run.php /srv/mediawiki/1.40/maintenance/importDump.php --wiki=dufanwikiwiki --no-updates --username-prefix=wikia:the-days-union /home/reception/ImportDump/dufanwikiwiki-20240120210552.xml (END - exit=0) [01:46:56] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.40/maintenance/run.php /srv/mediawiki/1.40/maintenance/rebuildall.php --wiki=dufanwikiwiki (START) [01:47:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:47:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:47:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.47, 7.67, 7.22 [01:49:11] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.89, 7.29, 6.78 [01:49:31] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 2.49, 3.87, 3.97 [01:51:08] PROBLEM - mw143 HTTPS on mw143 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10005 milliseconds with 0 bytes received [01:51:22] PROBLEM - mw142 HTTPS on mw142 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [01:51:39] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.92, 4.29, 4.08 [01:51:56] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.1006517112 secs [01:53:07] PROBLEM - mw143 HTTPS on mw143 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.530 second response time [01:53:16] PROBLEM - mw142 HTTPS on mw142 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.521 second response time [01:53:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.02, 7.25, 7.29 [01:55:34] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 1.48, 3.43, 3.83 [01:55:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.53, 8.37, 7.70 [01:56:50] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.52, 6.76, 6.78 [01:57:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.75, 7.59, 7.49 [01:59:26] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 0.37, 1.73, 3.04 [02:08:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.92, 6.71, 6.58 [02:10:43] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.69, 6.72, 6.60 [02:13:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.16, 7.54, 7.34 [02:15:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.73, 6.36, 6.92 [02:27:58] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.49, 6.15, 6.67 [02:31:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.54, 7.25, 7.05 [02:34:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.37, 6.93, 6.53 [02:37:58] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.25, 6.01, 6.55 [02:40:43] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.65, 6.61, 6.56 [02:46:52] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.10, 7.15, 6.87 [02:48:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.16, 7.46, 7.01 [02:50:47] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.79, 7.63, 7.13 [02:54:43] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.59, 6.45, 6.80 [03:00:28] RECOVERY - db142 Backups SQL on db142 is OK: FILE_AGE OK: /var/log/sql-backup.log is 27 seconds old and 0 bytes [03:00:34] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.85, 6.67, 6.73 [03:01:08] PROBLEM - mw143 HTTPS on mw143 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [03:02:31] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.17, 6.16, 6.54 [03:03:07] PROBLEM - mw143 HTTPS on mw143 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.539 second response time [03:04:00] PROBLEM - mon181 Backups Grafana on mon181 is WARNING: FILE_AGE WARNING: /var/log/grafana-backup.log is 864217 seconds old and 93 bytes [03:20:56] PROBLEM - mw131 HTTPS on mw131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [03:21:08] PROBLEM - mw143 HTTPS on mw143 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [03:21:10] PROBLEM - cp34 HTTPS on cp34 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [03:21:21] PROBLEM - mw134 HTTPS on mw134 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10000 milliseconds with 0 bytes received [03:21:22] PROBLEM - mw142 HTTPS on mw142 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [03:21:27] PROBLEM - cp24 HTTPS on cp24 is CRITICAL: HTTP CRITICAL: HTTP/2 503 - 3363 bytes in 0.480 second response time [03:22:55] PROBLEM - mw131 HTTPS on mw131 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.875 second response time [03:23:09] PROBLEM - cp34 HTTPS on cp34 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 4.893 second response time [03:23:10] PROBLEM - mw143 HTTPS on mw143 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 3.268 second response time [03:23:16] PROBLEM - mw142 HTTPS on mw142 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.552 second response time [03:23:19] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 11.68, 8.63, 6.55 [03:23:21] PROBLEM - mw134 HTTPS on mw134 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.586 second response time [03:23:27] PROBLEM - cp24 HTTPS on cp24 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.629 second response time [03:23:28] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.09, 9.66, 7.65 [03:25:19] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 6.30, 7.75, 6.49 [03:25:27] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 7.96, 8.67, 7.51 [03:33:17] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.92, 7.39, 6.71 [03:33:21] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.68, 10.26, 8.47 [03:35:19] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 9.40, 9.59, 8.42 [03:37:06] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.17, 6.42, 6.49 [03:52:23] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.58, 7.34, 6.88 [03:56:19] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.92, 6.27, 6.56 [04:08:03] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.66, 6.78, 6.54 [04:10:01] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.78, 6.58, 6.48 [04:13:17] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.40/maintenance/run.php /srv/mediawiki/1.40/maintenance/rebuildall.php --wiki=dufanwikiwiki (END - exit=0) [04:13:20] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.40/maintenance/run.php /srv/mediawiki/1.40/maintenance/initSiteStats.php --wiki=dufanwikiwiki --active --update (END - exit=0) [04:13:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:13:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:48:12] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.91, 7.15, 6.41 [04:56:03] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.03, 6.58, 6.44 [05:00:26] RECOVERY - db151 Backups SQL mhglobal on db151 is OK: FILE_AGE OK: /var/log/sql-mhglobal-backup-weekly.log is 24 seconds old and 86 bytes [05:22:09] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.79, 2.75, 2.16 [05:26:02] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.05, 3.57, 2.60 [05:29:55] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.99, 3.94, 2.99 [05:33:52] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.55, 3.16, 2.88 [05:35:08] PROBLEM - mw143 HTTPS on mw143 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [05:37:07] PROBLEM - mw143 HTTPS on mw143 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.524 second response time [05:42:03] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.58, 7.33, 6.49 [05:43:58] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.09, 6.52, 6.30 [05:52:40] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [06:00:24] RECOVERY - swiftac111 Backups Swift Account Container on swiftac111 is OK: FILE_AGE OK: /var/log/swift-account-container-backup.log is 22 seconds old and 0 bytes [06:22:21] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp34.miraheze.org - CNAME OK [06:44:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.10, 6.60, 6.26 [06:46:43] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.11, 6.49, 6.27 [07:38:09] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.52, 7.57, 6.50 [07:40:04] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.22, 7.30, 6.52 [07:43:53] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.17, 6.52, 6.41 [07:54:33] PROBLEM - mw141 HTTPS on mw141 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [07:54:47] PROBLEM - cp35 HTTPS on cp35 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [07:56:31] PROBLEM - mw141 HTTPS on mw141 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.550 second response time [07:56:43] PROBLEM - cp35 HTTPS on cp35 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.574 second response time [07:57:27] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.31, 7.31, 6.51 [07:59:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.97, 6.66, 6.36 [08:03:08] PROBLEM - mw143 HTTPS on mw143 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Connection timed out after 10004 milliseconds [08:03:36] PROBLEM - cp24 HTTPS on cp24 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [08:05:07] PROBLEM - mw143 HTTPS on mw143 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.648 second response time [08:05:33] PROBLEM - cp24 HTTPS on cp24 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.676 second response time [08:06:51] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.06, 7.18, 6.60 [08:07:13] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.80, 6.69, 6.33 [08:08:45] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.19, 6.78, 6.51 [08:09:11] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.36, 6.25, 6.20 [08:51:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.12, 6.59, 6.18 [08:53:19] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.37, 6.08, 6.04 [09:03:56] PROBLEM - ns1 NTP time on ns1 is CRITICAL: NTP CRITICAL: Offset 0.5031251311 secs [09:11:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.48, 7.13, 6.54 [09:13:58] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.15, 6.46, 6.37 [09:23:31] PROBLEM - cloud13 Puppet on cloud13 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [09:24:33] PROBLEM - mw141 HTTPS on mw141 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [09:24:47] PROBLEM - cp35 HTTPS on cp35 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [09:25:08] PROBLEM - mw143 HTTPS on mw143 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10001 milliseconds with 0 bytes received [09:25:22] PROBLEM - mw142 HTTPS on mw142 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [09:25:36] PROBLEM - cp24 HTTPS on cp24 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [09:26:01] PROBLEM - mw133 HTTPS on mw133 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [09:26:32] PROBLEM - mw141 HTTPS on mw141 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.655 second response time [09:26:43] PROBLEM - cp35 HTTPS on cp35 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.632 second response time [09:27:07] PROBLEM - mw143 HTTPS on mw143 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.561 second response time [09:27:16] PROBLEM - mw142 HTTPS on mw142 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.535 second response time [09:27:21] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.23, 2.62, 1.42 [09:27:33] PROBLEM - cp24 HTTPS on cp24 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.604 second response time [09:27:58] PROBLEM - mw133 HTTPS on mw133 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.593 second response time [09:29:20] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 1.18, 2.02, 1.35 [09:34:47] PROBLEM - cp35 HTTPS on cp35 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [09:36:43] PROBLEM - cp35 HTTPS on cp35 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.651 second response time [09:39:00] PROBLEM - cp34 HTTPS on cp34 is CRITICAL: HTTP CRITICAL: HTTP/2 502 - 3370 bytes in 0.456 second response time [09:39:09] PROBLEM - mw143 HTTPS on mw143 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [09:39:22] PROBLEM - mw142 HTTPS on mw142 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10000 milliseconds with 0 bytes received [09:39:23] PROBLEM - mw134 HTTPS on mw134 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10001 milliseconds with 0 bytes received [09:39:36] PROBLEM - cp24 HTTPS on cp24 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10000 milliseconds with 0 bytes received [09:41:01] PROBLEM - cp34 HTTPS on cp34 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.570 second response time [09:41:08] PROBLEM - mw143 HTTPS on mw143 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.521 second response time [09:41:16] PROBLEM - mw142 HTTPS on mw142 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.514 second response time [09:41:23] PROBLEM - mw134 HTTPS on mw134 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.558 second response time [09:41:34] PROBLEM - cp24 HTTPS on cp24 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 1.051 second response time [09:51:49] RECOVERY - cloud13 Puppet on cloud13 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [10:06:29] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 5.66, 6.83, 6.23 [10:08:24] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.70, 6.76, 6.27 [10:34:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.20, 6.73, 6.18 [10:36:43] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.23, 6.28, 6.09 [11:37:08] PROBLEM - mw143 HTTPS on mw143 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10000 milliseconds with 0 bytes received [11:38:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.40, 7.97, 6.72 [11:39:07] PROBLEM - mw143 HTTPS on mw143 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.551 second response time [11:40:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.09, 6.98, 6.49 [11:40:47] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.73, 7.16, 6.58 [11:42:43] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.10, 6.67, 6.43 [11:44:43] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.89, 6.62, 6.50 [11:51:26] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 5.58, 3.73, 2.80 [12:00:20] PROBLEM - mw132 HTTPS on mw132 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [12:00:26] PROBLEM - cp25 HTTPS on cp25 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [12:00:27] PROBLEM - mw141 HTTPS on mw141 is CRITICAL: HTTP CRITICAL: HTTP/2 502 - 170 bytes in 4.093 second response time [12:00:38] PROBLEM - cp35 HTTPS on cp35 is CRITICAL: HTTP CRITICAL: HTTP/2 503 - 3363 bytes in 0.226 second response time [12:01:00] PROBLEM - cp34 HTTPS on cp34 is CRITICAL: HTTP CRITICAL: HTTP/2 503 - 3363 bytes in 0.222 second response time [12:02:21] PROBLEM - mw132 HTTPS on mw132 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 2.636 second response time [12:02:26] PROBLEM - mw141 HTTPS on mw141 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.573 second response time [12:02:26] PROBLEM - cp25 HTTPS on cp25 is WARNING: HTTP WARNING: HTTP/2 404 - 3480 bytes in 2.951 second response time [12:02:38] PROBLEM - cp35 HTTPS on cp35 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.913 second response time [12:03:01] PROBLEM - cp34 HTTPS on cp34 is WARNING: HTTP WARNING: HTTP/2 404 - 3480 bytes in 0.589 second response time [12:03:19] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 12.91, 10.38, 7.90 [12:05:19] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 10.08, 9.81, 7.98 [12:05:20] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.90, 10.58, 8.17 [12:05:27] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.79, 10.44, 8.00 [12:07:16] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 8.53, 9.60, 8.09 [12:07:22] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 7.97, 9.54, 7.96 [12:09:21] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.16, 3.47, 2.51 [12:11:21] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.96, 3.24, 2.55 [12:15:21] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.47, 3.83, 2.94 [12:15:27] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [12:15:42] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [12:17:21] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.42, 3.68, 2.99 [12:19:20] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 1.64, 2.97, 2.82 [12:20:47] PROBLEM - cp35 HTTPS on cp35 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [12:22:43] PROBLEM - cp35 HTTPS on cp35 is WARNING: HTTP WARNING: HTTP/2 404 - 3480 bytes in 0.584 second response time [12:30:22] PROBLEM - os131 Current Load on os131 is CRITICAL: LOAD CRITICAL - total load average: 4.04, 3.56, 2.78 [12:30:38] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.29, 9.69, 8.55 [12:32:37] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 13.16, 10.44, 8.93 [12:34:22] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 3.87, 3.65, 3.00 [12:36:26] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 10.33, 9.58, 8.54 [12:36:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.69, 7.11, 6.32 [12:37:46] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 14.74, 11.57, 9.25 [12:38:43] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 4.55, 6.18, 6.08 [12:39:19] PROBLEM - mw131 Current Load on mw131 is CRITICAL: LOAD CRITICAL - total load average: 16.90, 12.15, 9.33 [12:40:22] PROBLEM - os131 Current Load on os131 is CRITICAL: LOAD CRITICAL - total load average: 4.93, 4.15, 3.41 [12:42:26] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 14.17, 12.35, 10.09 [12:44:17] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp34.miraheze.org - CNAME OK [12:44:22] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 3.33, 3.84, 3.46 [12:45:26] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.55, 2.91, 2.53 [12:45:28] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [12:46:23] PROBLEM - os131 Current Load on os131 is CRITICAL: LOAD CRITICAL - total load average: 4.24, 4.01, 3.57 [12:47:24] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.60, 2.85, 2.56 [12:48:22] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 3.06, 3.55, 3.45 [12:48:45] PROBLEM - mw141 Current Load on mw141 is WARNING: LOAD WARNING - total load average: 10.76, 10.29, 8.45 [12:49:29] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.01, 7.10, 6.25 [12:50:22] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 2.33, 3.13, 3.31 [12:51:27] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.11, 7.29, 6.57 [12:51:28] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.35, 6.65, 6.19 [12:52:38] RECOVERY - mw141 Current Load on mw141 is OK: LOAD OK - total load average: 8.59, 10.07, 8.83 [12:53:22] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.16, 6.49, 6.36 [12:53:34] PROBLEM - mw134 HTTPS on mw134 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10000 milliseconds with 0 bytes received [12:55:34] PROBLEM - mw134 HTTPS on mw134 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.579 second response time [13:02:22] PROBLEM - os131 Current Load on os131 is CRITICAL: LOAD CRITICAL - total load average: 4.92, 3.69, 3.33 [13:04:11] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.14, 7.27, 6.52 [13:04:24] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 3.86, 3.69, 3.37 [13:06:08] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.42, 6.98, 6.51 [13:06:22] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 2.56, 3.34, 3.28 [13:06:30] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [13:08:06] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.65, 6.45, 6.36 [13:10:26] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 8.64, 10.33, 11.68 [13:11:30] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [13:15:23] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 7.80, 8.02, 7.03 [13:16:08] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 6.42, 9.30, 11.86 [13:17:18] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 5.80, 7.37, 6.92 [13:18:26] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 4.63, 6.82, 9.57 [13:18:55] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 4.26, 7.21, 11.50 [13:19:12] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.33, 6.58, 6.68 [13:19:19] PROBLEM - mw131 Current Load on mw131 is WARNING: LOAD WARNING - total load average: 4.80, 7.38, 11.20 [13:19:23] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.78, 2.62, 1.79 [13:21:20] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.21, 2.65, 1.91 [13:21:26] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 2.09, 2.51, 3.86 [13:22:04] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 5.47, 6.84, 9.88 [13:22:55] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 5.38, 6.40, 10.19 [13:23:19] RECOVERY - mw131 Current Load on mw131 is OK: LOAD OK - total load average: 6.70, 6.97, 10.13 [13:27:26] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 1.55, 2.09, 3.25 [13:37:49] !log [oa@mwtask181] sudo -u www-data /srv/mediawiki/w/extension/CreateWiki/maintenance/setContainersAccess.php --wiki=woodwikiwiki [13:37:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:38:02] !log [oa@mwtask181] sudo -u www-data /srv/mediawiki/w/extensions/CreateWiki/maintenance/setContainersAccess.php --wiki=woodwikiwiki [13:38:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:39:16] !log [oa@mwtask181] sudo -u www-data /srv/mediawiki/1.40/extensions/CreateWiki/maintenance/setContainersAccess.php --wiki=woodwikiwiki [13:39:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:47:22] PROBLEM - mw142 HTTPS on mw142 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [13:49:16] PROBLEM - mw142 HTTPS on mw142 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.560 second response time [13:50:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.05, 7.15, 6.61 [13:52:47] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.61, 6.58, 6.46 [14:03:20] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.53, 2.60, 1.56 [14:05:20] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 1.87, 2.10, 1.49 [14:07:51] PROBLEM - mw133 HTTPS on mw133 is CRITICAL: HTTP CRITICAL: HTTP/2 502 - 170 bytes in 0.395 second response time [14:08:20] PROBLEM - mw132 HTTPS on mw132 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [14:08:26] PROBLEM - cp25 HTTPS on cp25 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10000 milliseconds with 0 bytes received [14:08:37] PROBLEM - cp35 HTTPS on cp35 is CRITICAL: HTTP CRITICAL: HTTP/2 503 - 3363 bytes in 0.223 second response time [14:09:00] PROBLEM - cp34 HTTPS on cp34 is CRITICAL: HTTP CRITICAL: HTTP/2 503 - 3363 bytes in 0.226 second response time [14:09:51] PROBLEM - mw133 HTTPS on mw133 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.570 second response time [14:10:19] PROBLEM - mw132 HTTPS on mw132 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.572 second response time [14:10:24] PROBLEM - cp25 HTTPS on cp25 is WARNING: HTTP WARNING: HTTP/2 404 - 3480 bytes in 0.574 second response time [14:10:38] PROBLEM - cp35 HTTPS on cp35 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.600 second response time [14:11:01] PROBLEM - cp34 HTTPS on cp34 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.583 second response time [14:16:11] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [14:32:23] PROBLEM - test2.specialchat.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'test2.specialchat.org' expires in 15 day(s) (Tue 06 Feb 2024 02:10:04 PM GMT +0000). [14:32:39] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d23336fac3a9...c1ef9cb02fc9 [14:32:42] [02miraheze/ssl] 07MirahezeSSLBot 03c1ef9cb - Bot: Update SSL cert for test2.specialchat.org [14:44:49] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp34.miraheze.org - CNAME OK [14:51:30] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.12, 7.44, 6.54 [14:53:28] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.12, 6.83, 6.42 [14:55:26] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.89, 5.90, 6.11 [15:01:17] RECOVERY - test2.specialchat.org - LetsEncrypt on sslhost is OK: OK - Certificate 'test2.specialchat.org' will expire on Sat 20 Apr 2024 01:32:28 PM GMT +0000. [15:05:36] PROBLEM - cp24 HTTPS on cp24 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [15:07:33] PROBLEM - cp24 HTTPS on cp24 is WARNING: HTTP WARNING: HTTP/2 404 - 3458 bytes in 0.733 second response time [15:20:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.03, 7.12, 6.57 [15:26:51] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.97, 7.45, 6.86 [15:32:28] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.55, 6.62, 6.19 [15:32:45] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.74, 6.67, 6.71 [15:32:56] PROBLEM - mw131 HTTPS on mw131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [15:34:54] PROBLEM - mw131 HTTPS on mw131 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.517 second response time [15:36:18] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.39, 6.29, 6.16 [15:41:32] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.51, 7.30, 6.95 [15:42:33] PROBLEM - mw141 HTTPS on mw141 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [15:43:30] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.79, 6.95, 6.86 [15:44:32] PROBLEM - mw141 HTTPS on mw141 is WARNING: HTTP WARNING: HTTP/2 404 - 285 bytes in 0.533 second response time [15:45:27] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.08, 6.69, 6.78 [15:49:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.89, 7.24, 6.95 [15:51:19] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.02, 6.99, 6.89 [15:53:17] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.13, 6.68, 6.79 [15:56:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.24, 7.44, 6.63 [15:58:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.18, 7.50, 6.77 [15:59:08] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.59, 6.95, 6.82 [16:00:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 11.01, 8.53, 7.21 [16:01:05] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.79, 7.56, 7.05 [16:03:03] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.87, 7.56, 7.14 [16:06:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 5.69, 7.28, 7.14 [16:08:44] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.52, 7.49, 7.22 [16:08:57] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.71, 6.27, 6.70 [16:10:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 5.92, 6.78, 6.99 [16:12:43] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 4.69, 6.11, 6.72 [16:20:39] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.09, 7.36, 7.02 [16:22:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.12, 7.08, 6.97 [16:26:33] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.01, 6.57, 6.78 [16:31:28] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 9.74, 8.91, 7.48 [16:32:24] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.99, 7.24, 7.04 [16:35:18] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.86, 7.91, 7.36 [16:40:15] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.15, 7.33, 7.11 [16:41:02]