[00:00:46] RECOVERY - os131 Disk Space on os131 is OK: DISK OK - free space: / 22351 MB (11% inode=99%); [00:01:36] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.04, 4.10, 3.99 [00:03:32] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.92, 5.35, 4.18 [00:04:01] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.33, 7.16, 6.79 [00:04:21] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.24, 2.55, 1.33 [00:06:21] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.73, 2.85, 1.58 [00:07:53] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.95, 6.32, 6.56 [00:08:21] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 2.68, 2.87, 1.75 [00:09:19] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.63, 3.50, 3.73 [00:11:29] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.65, 3.79, 3.28 [00:13:10] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.37, 4.39, 4.04 [00:13:25] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.47, 3.59, 3.27 [00:14:16] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query cn.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [00:15:20] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.72, 3.28, 3.19 [00:15:35] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.24, 3.40, 3.90 [00:18:57] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.92, 3.66, 3.95 [00:20:56] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.06, 4.65, 4.27 [00:21:35] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.34, 2.70, 3.40 [00:21:46] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1226408482 secs [00:23:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.06, 11.63, 12.00 [00:24:22] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 5.14, 3.50, 2.60 [00:25:12] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 11.78, 7.79, 5.15 [00:26:22] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 3.31, 3.40, 2.68 [00:27:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.73, 11.38, 11.76 [00:29:05] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 4.95, 6.92, 5.45 [00:29:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.54, 11.19, 11.63 [00:29:45] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.80, 3.50, 3.29 [00:31:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.87, 11.93, 11.86 [00:33:36] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.29, 3.82, 3.46 [00:34:53] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.92, 7.38, 6.04 [00:35:31] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.99, 3.82, 3.50 [00:36:36] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.27, 3.63, 3.48 [00:36:50] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.80, 6.58, 5.92 [00:40:36] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.3357825875 secs [00:41:17] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.09, 3.90, 3.63 [00:41:46] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.08572596312 secs [00:42:34] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.95, 3.20, 3.34 [00:42:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.96, 3.01, 3.90 [00:44:16] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [00:45:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.94, 3.94, 3.71 [00:45:34] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [00:47:09] PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.1071220934 secs [00:47:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.54, 10.85, 11.77 [00:50:56] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.96, 2.71, 3.36 [00:51:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.08, 12.38, 12.12 [00:53:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.50, 4.14, 3.68 [00:55:09] PROBLEM - cp32 NTP time on cp32 is CRITICAL: NTP CRITICAL: Offset 0.6269822717 secs [00:55:29] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.58, 3.68, 3.59 [00:59:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.29, 2.80, 3.29 [00:59:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.09, 6.93, 6.57 [01:01:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 14.00, 9.44, 7.51 [01:02:36] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset -0.02867645025 secs [01:03:12] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.88, 7.46, 6.00 [01:03:25] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.18, 4.43, 3.83 [01:03:39] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.70, 3.69, 3.10 [01:06:14] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.73, 3.06, 3.33 [01:07:34] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.27, 3.64, 3.23 [01:07:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.60, 7.36, 7.19 [01:09:32] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.62, 3.90, 3.36 [01:09:44] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 3.80, 5.99, 6.70 [01:12:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.56, 4.61, 3.94 [01:12:36] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1419392228 secs [01:12:53] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 5.12, 7.71, 7.34 [01:13:33] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.13, 3.68, 3.42 [01:14:16] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.nj.cn.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [01:14:36] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.00247707963 secs [01:15:09] RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset -0.002543717623 secs [01:15:30] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 1.05, 2.85, 3.15 [01:15:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.31, 6.83, 6.78 [01:16:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.85, 3.79, 3.77 [01:16:46] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.88, 5.50, 6.61 [01:17:44] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.23, 6.26, 6.58 [01:19:20] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.16, 3.52, 3.96 [01:22:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 1.26, 2.47, 3.23 [01:27:17] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.58, 4.09, 3.90 [01:27:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 8.00, 7.50, 7.10 [01:31:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.66, 7.52, 7.15 [01:33:15] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.94, 3.85, 3.86 [01:33:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.77, 6.91, 6.97 [01:37:44] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.67, 6.15, 6.64 [01:39:13] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.59, 2.78, 3.40 [01:43:34] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [01:45:21] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 7.38, 10.73, 5.74 [01:45:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.53, 10.46, 11.93 [01:47:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 17.80, 12.88, 12.61 [01:49:21] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 2.79, 6.74, 5.26 [01:51:08] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.36, 3.74, 3.48 [01:53:07] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.09, 4.05, 3.61 [01:55:06] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.56, 3.41, 3.42 [01:57:06] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.38, 3.30, 3.36 [01:58:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.56, 3.34, 3.13 [02:00:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.92, 3.22, 3.11 [02:01:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.09, 10.42, 11.81 [02:03:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.58, 11.46, 11.98 [02:07:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.97, 11.14, 11.84 [02:07:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.97, 3.65, 3.26 [02:09:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.09, 11.88, 12.04 [02:09:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.92, 3.31, 3.17 [02:11:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.53, 11.43, 11.86 [02:12:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.50, 4.11, 3.48 [02:13:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.76, 12.14, 12.07 [02:14:16] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.94, 7.29, 6.46 [02:15:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.13, 11.47, 11.85 [02:16:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.56, 3.90, 3.55 [02:16:12] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.32, 6.80, 6.37 [02:17:35] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.49, 3.60, 3.34 [02:17:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.96, 11.99, 12.01 [02:19:31] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.88, 4.54, 3.72 [02:20:06] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.01, 7.65, 6.88 [02:22:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.86, 3.18, 3.33 [02:23:57] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.34, 6.35, 6.53 [02:29:12] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.46, 6.05, 3.70 [02:31:09] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.83, 4.39, 3.36 [02:31:09] PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.2184063792 secs [02:32:50] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.89, 3.73, 3.34 [02:42:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.66, 3.76, 3.30 [02:42:22] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.95, 7.02, 6.66 [02:42:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.57, 3.49, 3.97 [02:44:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.25, 3.55, 3.28 [02:44:18] RECOVERY - wiki.nj.cn.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.nj.cn.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [02:44:18] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.28, 5.99, 6.32 [02:50:46] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.99, 3.28, 3.70 [02:50:52] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.zulaclub.net All nameservers failed to answer the query. [02:51:03] !log [void@db101] SET GLOBAL read_only=0; [02:51:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:52:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.26, 3.34, 3.29 [02:52:45] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.04, 3.58, 3.75 [02:52:56] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.01, 4.12, 3.85 [02:54:44] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.41, 2.75, 3.43 [02:56:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.80, 3.94, 3.84 [02:57:09] RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset -0.002498149872 secs [02:58:43] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.41, 3.86, 3.69 [03:00:42] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.89, 4.00, 3.78 [03:00:56] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.98, 3.98, 3.80 [03:06:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.90, 3.98, 3.99 [03:08:56] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 3.40, 4.23, 4.11 [03:09:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.91, 10.81, 11.80 [03:10:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.16, 3.49, 3.21 [03:10:39] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.73, 3.04, 3.35 [03:11:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.02, 10.89, 11.68 [03:12:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.91, 3.68, 3.32 [03:14:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.73, 3.66, 3.92 [03:16:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.72, 3.24, 3.23 [03:17:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.80, 10.85, 11.50 [03:18:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.05, 3.45, 3.28 [03:20:11] RECOVERY - wiki.zulaclub.net - reverse DNS on sslhost is OK: SSL OK - wiki.zulaclub.net reverse DNS resolves to cp22.miraheze.org - CNAME OK [03:20:35] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.16, 2.85, 3.08 [03:21:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 15.10, 12.19, 11.86 [03:21:58] PROBLEM - test131 MediaWiki Rendering on test131 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:22:56] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.28, 2.83, 3.40 [03:23:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.75, 11.36, 11.61 [03:23:54] PROBLEM - test131 MediaWiki Rendering on test131 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 302 bytes in 0.592 second response time [03:26:32] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.37, 3.63, 3.37 [03:26:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.06, 3.77, 3.73 [03:29:03] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.34, 2.50, 1.63 [03:30:30] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.04, 3.84, 3.50 [03:30:56] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.73, 2.98, 3.39 [03:31:01] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 2.01, 2.18, 1.61 [03:31:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.64, 11.91, 11.61 [03:32:30] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.30, 3.20, 3.30 [03:32:44] [02miraheze/CreateWiki] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/c3abddf30d93...2e72dc6f023e [03:32:47] [02miraheze/CreateWiki] 07AgentIsai 032e72dc6 - Also promote to interface-admin on creation [03:36:39] [02miraheze/CreateWiki] 07AgentIsai pushed 035 commits to 03support-partial-blocks [+0/-0/±5] 13https://github.com/miraheze/CreateWiki/compare/e39d8a23cfe7^...735b5c60fe69 [03:36:41] [02miraheze/CreateWiki] 07Universal-Omega 03e39d8a2 - Support partial blocks for requesting wikis [03:36:42] [02miraheze/CreateWiki] 07Universal-Omega 03df46a16 - Update extension.json [03:36:44] [02miraheze/CreateWiki] 07Universal-Omega 03c242dd3 - Update SpecialRequestWiki.php [03:36:47] [02miraheze/CreateWiki] ... and 2 more commits. [03:36:48] [02CreateWiki] 07AgentIsai created branch 03support-partial-blocks - 13https://github.com/miraheze/CreateWiki [03:36:59] [02CreateWiki] 07AgentIsai commented on pull request 03#354: Support partial blocks for requesting wikis - 13https://github.com/miraheze/CreateWiki/pull/354#issuecomment-1548928495 [03:37:01] [02CreateWiki] 07AgentIsai reopened pull request 03#354: Support partial blocks for requesting wikis - 13https://github.com/miraheze/CreateWiki/pull/354 [03:37:49] [02CreateWiki] 07Universal-Omega commented on pull request 03#354: Support partial blocks for requesting wikis - 13https://github.com/miraheze/CreateWiki/pull/354#issuecomment-1548928919 [03:37:51] [02CreateWiki] 07AgentIsai edited a comment on pull request 03#354: Support partial blocks for requesting wikis - 13https://github.com/miraheze/CreateWiki/pull/354#issuecomment-1548928495 [03:37:58] [02CreateWiki] 07Universal-Omega commented on pull request 03#354: Support partial blocks for requesting wikis - 13https://github.com/miraheze/CreateWiki/pull/354#issuecomment-1548928996 [03:38:11] [02CreateWiki] 07AgentIsai commented on pull request 03#354: Support partial blocks for requesting wikis - 13https://github.com/miraheze/CreateWiki/pull/354#issuecomment-1548929090 [03:40:43] [02CreateWiki] 07Universal-Omega commented on pull request 03#354: Support partial blocks for requesting wikis - 13https://github.com/miraheze/CreateWiki/pull/354#issuecomment-1548930426 [03:41:01] [02CreateWiki] 07AgentIsai commented on pull request 03#348: Sync i18n with Meta - 13https://github.com/miraheze/CreateWiki/pull/348#issuecomment-1548930601 [03:41:02] miraheze/CreateWiki - AgentIsai the build passed. [03:41:04] [02CreateWiki] 07AgentIsai closed pull request 03#348: Sync i18n with Meta - 13https://github.com/miraheze/CreateWiki/pull/348 [03:42:01] [02CreateWiki] 07Universal-Omega commented on commit 032e72dc6f023e69b66180197b50d46f371c18d861 - 13https://github.com/miraheze/CreateWiki/commit/2e72dc6f023e69b66180197b50d46f371c18d861#commitcomment-113494109 [03:42:08] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 5.28, 3.95, 3.33 [03:42:41] [02CreateWiki] 07Universal-Omega commented on commit 032e72dc6f023e69b66180197b50d46f371c18d861 - 13https://github.com/miraheze/CreateWiki/commit/2e72dc6f023e69b66180197b50d46f371c18d861#commitcomment-113494156 [03:43:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.00, 11.87, 11.96 [03:44:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.18, 3.59, 3.27 [03:45:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.17, 12.06, 12.01 [03:46:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.85, 3.30, 3.20 [03:51:46] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1319347024 secs [03:56:16] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.04, 4.27, 3.66 [03:58:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.45, 3.57, 3.47 [03:59:09] PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset -0.1151724458 secs [04:00:07] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.30, 3.14, 3.33 [04:00:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.73, 3.29, 3.13 [04:01:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.03, 11.36, 11.98 [04:03:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.46, 11.65, 12.00 [04:04:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.74, 3.15, 3.12 [04:07:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.64, 11.20, 11.79 [04:07:46] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.07831305265 secs [04:08:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.76, 10.35, 11.88 [04:10:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.46, 3.30, 3.17 [04:10:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 15.58, 11.60, 12.10 [04:11:09] RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset -0.09434887767 secs [04:12:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.32, 3.35, 3.21 [04:12:40] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.93, 4.45, 3.67 [04:16:31] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.78, 3.79, 3.58 [04:18:26] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.81, 3.14, 3.37 [04:18:54] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1868661046 secs [04:19:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.21, 11.52, 11.42 [04:22:19] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.63, 4.29, 3.77 [04:26:10] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.23, 3.38, 3.53 [04:27:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.79, 11.77, 11.62 [04:28:05] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.95, 2.90, 3.33 [04:29:37] [02mediawiki] 07AgentIsai closed pull request 03#14708: Update 1.40 to latest - 13https://github.com/miraheze/mediawiki/pull/14708 [04:30:36] [02miraheze/mediawiki] 07AgentIsai created branch 03REL1_40_15May 13https://github.com/miraheze/mediawiki/commit/ff7417da694a2ee404be03a057c7d807ba8faafb [04:30:39] [02mediawiki] 07AgentIsai created branch 03REL1_40_15May - 13https://github.com/miraheze/mediawiki [04:31:24] [02mediawiki] 07AgentIsai opened pull request 03#14709: Bump 1.40 to latest - 13https://github.com/miraheze/mediawiki/pull/14709 [04:32:06] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.08, 3.54, 2.92 [04:33:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 15.43, 12.95, 12.08 [04:35:49] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.95, 3.41, 3.37 [04:37:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.32, 11.95, 11.88 [04:37:44] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.71, 3.13, 3.27 [04:38:04] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.55, 3.96, 3.34 [04:39:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.25, 13.10, 12.36 [04:41:58] [02miraheze/mediawiki] 07AgentIsai pushed 031 commit to 03REL1_40_15May [+0/-0/±2] 13https://github.com/miraheze/mediawiki/compare/ff7417da694a...4f321b2a0fd8 [04:42:00] [02miraheze/mediawiki] 07AgentIsai 034f321b2 - Update ImportDump and MirahezeMagic [04:42:03] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.90, 2.87, 3.03 [04:44:14] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.27, 6.99, 6.25 [04:46:10] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.85, 7.01, 6.35 [04:48:06] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.43, 6.56, 6.27 [04:48:54] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.07603433728 secs [04:49:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.47, 11.41, 11.98 [04:55:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.13, 11.38, 11.73 [04:59:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.16, 11.63, 11.84 [05:03:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.51, 11.59, 11.70 [05:07:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.41, 11.48, 11.69 [05:18:54] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset -0.1115698218 secs [05:21:00] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.26, 6.28, 5.94 [05:24:52] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.65, 6.11, 5.97 [05:24:54] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.08520048857 secs [05:29:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 17.79, 12.44, 11.15 [05:30:57] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.18, 3.76, 3.38 [05:38:56] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.00, 2.99, 3.22 [05:40:25] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.93, 3.45, 3.20 [05:41:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.33, 11.60, 11.57 [05:42:20] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.81, 3.95, 3.41 [05:45:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.52, 11.83, 11.64 [05:46:10] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.92, 3.87, 3.50 [05:47:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.44, 10.99, 11.36 [05:49:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.68, 11.80, 11.60 [05:50:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.90, 3.39, 3.39 [05:51:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.59, 11.37, 11.48 [05:52:56] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.04, 6.84, 6.53 [05:53:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 15.07, 12.34, 11.79 [05:54:52] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.52, 6.49, 6.44 [06:08:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.72, 3.46, 3.32 [06:10:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 5.22, 4.18, 3.60 [06:13:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.84, 10.69, 11.71 [06:17:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.63, 3.68, 2.77 [06:19:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.21, 11.22, 11.56 [06:21:35] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.95, 3.98, 3.20 [06:21:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.63, 10.98, 11.46 [06:23:36] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.62, 3.09, 2.97 [06:25:46] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1814672947 secs [06:27:35] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.67, 3.14, 2.97 [06:28:54] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset -0.1065298021 secs [06:29:35] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.73, 2.61, 2.80 [06:29:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.55, 11.70, 11.48 [06:30:54] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.09445399046 secs [06:31:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.79, 11.72, 11.52 [06:35:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.98, 13.00, 12.02 [06:37:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.99, 11.69, 11.68 [06:47:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.84, 11.53, 11.33 [06:55:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.65, 11.50, 11.51 [06:56:36] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1072054207 secs [06:57:46] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09769782424 secs [07:09:41] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.89, 11.76, 11.50 [07:20:36] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.09750643373 secs [07:31:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.49, 3.08, 2.64 [07:33:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.85, 2.64, 2.54 [07:41:46] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.2834919095 secs [07:47:46] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.02415794134 secs [07:48:23] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.2802306414 secs [07:54:23] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset -0.02235350013 secs [07:56:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.09, 3.41, 3.98 [07:57:41] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.10, 10.26, 11.78 [07:57:46] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.2879955769 secs [07:59:46] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.009579688311 secs [08:00:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 6.53, 9.45, 11.70 [08:04:24] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.2076990306 secs [08:05:41] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.07, 8.52, 10.19 [08:05:51] PROBLEM - wiki.joust.ro - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.joust.ro All nameservers failed to answer the query. [08:06:24] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset -0.02545672655 secs [08:08:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.06, 2.96, 3.36 [08:10:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.10, 10.24, 10.64 [08:12:08] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.84, 3.51, 3.50 [08:12:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.97, 9.84, 10.46 [08:14:44] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 11.33, 6.84, 3.67 [08:16:22] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.53, 9.14, 10.02 [08:16:44] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.41, 5.21, 3.48 [08:18:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.19, 3.22, 3.37 [08:19:51] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.25, 7.08, 6.21 [08:21:47] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.22, 6.51, 6.11 [08:27:16] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.13, 10.30, 9.85 [08:29:14] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.53, 10.13, 9.84 [08:31:12] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.31, 9.75, 9.73 [08:35:27] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.79, 10.43, 9.43 [08:37:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 10.05, 10.16, 9.45 [08:38:19] PROBLEM - swiftobject121 Disk Space on swiftobject121 is WARNING: DISK WARNING - free space: / 52284 MB (10% inode=83%); [08:42:56] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.35, 12.54, 10.81 [08:44:53] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.07, 11.66, 10.70 [08:46:51] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.60, 11.80, 10.85 [08:48:49] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.34, 11.08, 10.68 [08:50:28] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [08:50:47] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.03, 12.32, 11.22 [08:51:12] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.44, 10.30, 9.83 [08:52:45] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.91, 11.18, 10.92 [08:55:09] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.84, 10.98, 10.17 [08:57:08] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.69, 10.09, 9.92 [08:58:39] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.37, 11.40, 11.08 [09:00:37] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.51, 11.29, 11.07 [09:03:02] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.60, 11.55, 10.55 [09:04:33] RECOVERY - wiki.joust.ro - reverse DNS on sslhost is OK: SSL OK - wiki.joust.ro reverse DNS resolves to cp22.miraheze.org - CNAME OK [09:05:00] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.68, 11.51, 10.65 [09:06:59] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.01, 11.64, 10.80 [09:10:27] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 15.71, 12.55, 11.61 [09:12:36] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.83, 3.74, 3.39 [09:12:54] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.29, 11.82, 11.25 [09:14:53] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.35, 12.61, 11.60 [09:15:34] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [09:16:15] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.nj.cn.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [09:16:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.96, 3.33, 3.30 [09:20:10] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [09:40:07] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.56, 7.78, 6.56 [09:40:54] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 5.30, 3.93, 3.40 [09:42:03] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.65, 6.80, 6.33 [09:42:49] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.35, 3.70, 3.39 [09:44:09] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [09:48:34] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.07, 3.33, 3.32 [10:07:14] [1/2] I'll help 10 people how to earn $50,000 in just 72hours from the crypto market. [10:07:15] [2/2] Dm me to know How +1 (209) 225‑9985 [10:11:47] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.26, 3.44, 3.31 [10:13:35] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [10:13:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.69, 7.25, 6.23 [10:15:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.18, 3.35, 3.31 [10:15:44] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.12, 6.75, 6.18 [10:18:05] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.09, 10.88, 11.84 [10:19:29] PROBLEM - wiki.litek.top - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.litek.top' expires in 15 day(s) (Thu 01 Jun 2023 10:04:02 GMT +0000). [10:20:03] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 15.35, 12.54, 12.33 [10:20:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.91, 10.00, 11.74 [10:22:01] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.68, 11.59, 12.00 [10:26:07] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/f6886e100fba...69b0cdad852c [10:26:10] [02miraheze/ssl] 07MirahezeSSLBot 0369b0cda - Bot: Update SSL cert for wiki.litek.top [10:26:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 15.72, 11.57, 11.71 [10:27:57] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.18, 12.22, 12.02 [10:28:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.72, 11.79, 11.80 [10:29:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.93, 11.30, 11.73 [10:32:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 15.96, 13.00, 12.21 [10:33:52] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.63, 12.40, 12.04 [10:37:49] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.99, 11.92, 11.98 [10:38:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.71, 4.31, 3.70 [10:39:48] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.14, 12.35, 12.10 [10:42:54] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.2149199247 secs [10:44:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.10, 3.93, 3.78 [10:44:16] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [10:48:59] RECOVERY - wiki.litek.top - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.litek.top' will expire on Mon 14 Aug 2023 09:25:54 GMT +0000. [10:58:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.05, 3.63, 3.55 [11:00:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.68, 3.58, 3.54 [11:02:54] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.009300738573 secs [11:07:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.63, 6.84, 6.13 [11:08:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.87, 3.22, 3.40 [11:09:44] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.39, 6.31, 6.02 [11:12:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.46, 3.47, 3.46 [11:14:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.96, 3.28, 3.39 [11:14:16] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.nj.cn.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [11:27:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.53, 7.62, 6.76 [11:29:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.89, 7.01, 6.64 [11:33:44] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.74, 6.60, 6.58 [11:37:40] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:38:05] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:38:08] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 7.57, 5.38, 4.01 [11:38:21] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.03, 3.24, 1.66 [11:38:44] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.77, 8.91, 5.07 [11:38:46] PROBLEM - mw141 Puppet on mw141 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [11:39:38] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.724 second response time [11:40:06] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 6.774 second response time [11:40:21] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 0.72, 2.31, 1.52 [11:40:44] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.70, 6.23, 4.55 [11:44:16] RECOVERY - wiki.nj.cn.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.nj.cn.eu.org reverse DNS resolves to cp22.miraheze.org - CNAME OK [11:46:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.24, 3.84, 3.84 [11:51:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.15, 7.81, 6.98 [11:55:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.67, 7.71, 7.06 [11:59:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.39, 7.01, 6.97 [12:03:44] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.78, 6.30, 6.74 [12:04:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.95, 3.07, 3.35 [12:06:46] RECOVERY - mw141 Puppet on mw141 is OK: OK: Puppet is currently enabled, last run 26 seconds ago with 0 failures [12:08:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.26, 3.60, 3.49 [12:18:33] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.90, 2.71, 2.03 [12:20:32] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.11, 2.40, 2.00 [12:25:13] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [12:28:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.90, 6.89, 6.51 [12:30:28] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.24, 3.11, 2.37 [12:32:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.02, 2.68, 2.30 [12:32:29] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.28, 6.51, 6.43 [12:38:25] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.98, 3.48, 2.72 [12:38:46] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 11.30, 8.33, 4.78 [12:40:24] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.93, 3.26, 2.73 [12:40:44] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.39, 6.44, 4.53 [12:43:09] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.57, 7.48, 6.86 [12:45:05] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.67, 6.76, 6.67 [12:48:59] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.29, 6.87, 6.77 [12:54:46] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.48, 6.25, 6.58 [13:00:16] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.17, 3.42, 2.84 [13:00:35] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.15, 7.17, 6.91 [13:02:16] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.26, 2.69, 2.65 [13:06:23] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.90, 6.21, 6.68 [13:10:16] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.62, 7.27, 7.00 [13:12:12] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.46, 6.53, 6.76 [13:22:54] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.33, 7.48, 7.04 [13:24:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.57, 7.73, 7.19 [13:26:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.40, 8.02, 7.32 [13:28:41] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.06, 7.34, 7.15 [13:34:10] PROBLEM - uk.religiononfire.mar.in.ua - LetsEncrypt on sslhost is CRITICAL: connect to address uk.religiononfire.mar.in.ua and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [13:34:29] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.23, 6.04, 6.69 [13:58:46] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.45, 6.84, 6.65 [14:02:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [14:02:47] RECOVERY - uk.religiononfire.mar.in.ua - LetsEncrypt on sslhost is OK: OK - Certificate 'uk.religiononfire.mar.in.ua' will expire on Thu 01 Jun 2023 18:41:45 GMT +0000. [14:04:34] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.02, 7.73, 7.07 [14:07:51] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.53, 3.60, 2.81 [14:08:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.12, 3.44, 3.93 [14:08:26] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.84, 7.13, 6.98 [14:09:50] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.77, 2.84, 2.63 [14:10:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.22, 7.70, 7.20 [14:14:13] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.20, 7.38, 7.16 [14:15:34] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.andreijiroh.uk.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [14:20:01] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.04, 7.72, 7.28 [14:20:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.63, 2.85, 3.38 [14:21:56] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.41, 7.25, 7.17 [14:24:04] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [14:24:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.74, 3.60, 3.58 [14:26:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.18, 3.66, 3.10 [14:28:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.10, 3.87, 3.68 [14:28:56] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.02, 4.11, 3.32 [14:29:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.05, 7.80, 7.37 [14:30:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.55, 3.35, 3.51 [14:30:36] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.4662315845 secs [14:30:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.61, 3.58, 3.22 [14:32:56] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.81, 3.13, 3.09 [14:34:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.60, 3.13, 3.39 [14:37:00] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [14:37:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.03, 7.52, 7.45 [14:40:36] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.0006547570229 secs [14:42:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 3.91, 4.51, 3.90 [14:42:12] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:44:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.93, 3.95, 3.77 [14:44:11] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp22.miraheze.org - CNAME OK [14:44:12] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 6.669 second response time [14:47:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.36, 5.08, 3.52 [14:49:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.08, 7.87, 7.39 [14:51:35] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.81, 3.47, 3.24 [14:51:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.97, 7.35, 7.27 [14:52:23] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:53:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.57, 3.69, 3.33 [14:53:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.12, 7.33, 7.25 [14:53:48] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.07, 3.64, 3.30 [14:54:25] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 8.388 second response time [14:55:35] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.74, 3.26, 3.21 [14:55:44] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.59, 3.35, 3.24 [14:55:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.28, 7.11, 7.19 [14:56:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.70, 3.86, 3.64 [14:58:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.82, 3.75, 3.63 [15:01:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.78, 4.00, 3.53 [15:01:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.33, 7.79, 7.30 [15:03:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.60, 6.92, 7.05 [15:05:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [15:06:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.61, 3.14, 3.40 [15:07:44] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.37, 6.13, 6.69 [15:11:35] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.22, 3.47, 3.73 [15:17:35] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.37, 2.58, 3.24 [15:28:29] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.61, 7.57, 7.11 [15:32:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.07, 8.09, 7.38 [15:32:36] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=ozendumhwiki --no-updates --username-prefix=wikia:ozendumh /home/agent/ImportDump/ozendumhwiki-20230516105701.xml (START) [15:32:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:32:44] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.40, 5.32, 3.80 [15:32:54] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.143152833 secs [15:32:55] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=electronictoyswiki --no-updates --username-prefix=wikia:electronic-toys-wiki /home/agent/ImportDump/electronictoyswiki-20230516074240.xml (START) [15:33:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:34:21] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 5.11, 3.71, 2.43 [15:35:23] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=electronictoyswiki --no-updates --username-prefix=wikia:electronic-toys-wiki /home/agent/ImportDump/electronictoyswiki-20230516074240.xml (END - exit=0) [15:35:24] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=electronictoyswiki (START) [15:35:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:35:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:36:12] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.59, 7.58, 7.39 [15:36:21] PROBLEM - os141 Current Load on os141 is WARNING: WARNING - load average: 3.82, 3.70, 2.58 [15:36:44] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=ozendumhwiki --no-updates --username-prefix=wikia:ozendumh /home/agent/ImportDump/ozendumhwiki-20230516105701.xml (END - exit=0) [15:36:44] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 6.03, 6.33, 4.60 [15:36:45] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=ozendumhwiki (START) [15:36:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:36:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:37:43] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=electronictoyswiki (END - exit=0) [15:37:45] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=electronictoyswiki --active --update (END - exit=0) [15:37:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:37:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:38:12] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=ozendumhwiki (END - exit=0) [15:38:13] !log [agent@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=ozendumhwiki --active --update (END - exit=0) [15:38:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:38:21] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 1.04, 2.72, 2.36 [15:38:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:39:18] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.93, 3.52, 3.30 [15:40:07] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.63, 3.91, 3.31 [15:41:54] [02mediawiki] 07wmfgerrit synchronize pull request 03#14709: Bump 1.40 to latest - 13https://github.com/miraheze/mediawiki/pull/14709 [15:43:58] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.22, 3.89, 3.44 [15:45:04] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.81, 3.12, 3.21 [15:45:34] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [15:45:54] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.85, 3.08, 3.19 [15:49:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.07, 7.27, 7.21 [15:52:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 9.28, 4.90, 3.39 [15:53:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.53, 7.80, 7.48 [15:55:13] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [15:55:33] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.18, 3.59, 3.43 [15:57:28] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.00, 2.71, 3.13 [15:58:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.85, 3.49, 3.24 [16:00:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.69, 3.22, 3.17 [16:00:54] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.07733118534 secs [16:03:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.83, 7.85, 7.37 [16:05:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.95, 7.17, 7.17 [16:09:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.42, 3.59, 3.23 [16:11:44] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.73, 6.10, 6.67 [16:12:57] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.55, 3.77, 3.40 [16:14:13] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [16:19:19] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.63, 3.88, 3.24 [16:20:38] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.84, 3.32, 3.35 [16:21:18] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.02, 3.40, 3.13 [16:23:46] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1025405228 secs [16:28:29] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.37, 7.08, 6.77 [16:30:00] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [16:33:08] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.66, 3.88, 3.47 [16:35:04] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.95, 3.47, 3.37 [16:36:59] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.92, 3.31, 3.32 [16:37:03] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.70, 3.22, 3.03 [16:39:01] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 1.93, 2.68, 2.85 [16:40:04] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.28, 6.55, 6.69 [16:47:08] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.25, 3.39, 3.01 [16:48:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.59, 8.05, 7.26 [16:49:07] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.84, 3.23, 3.00 [16:52:08] RECOVERY - biblestrength.net - LetsEncrypt on sslhost is OK: OK - Certificate 'biblestrength.net' will expire on Mon 24 Jul 2023 06:28:17 GMT +0000. [16:52:41] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.04, 7.95, 7.41 [16:53:46] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.07975363731 secs [16:54:39] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [16:58:03] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.09, 3.77, 3.37 [17:00:02] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.93, 3.02, 3.14 [17:04:16] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.52, 7.24, 7.18 [17:06:12] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.86, 7.43, 7.26 [17:10:04] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.27, 6.23, 6.79 [17:10:44] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 12.76, 7.80, 4.00 [17:12:36] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1578312516 secs [17:12:44] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 4.59, 6.90, 4.17 [17:14:44] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.09, 4.82, 3.73 [17:17:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.33, 7.36, 6.95 [17:18:54] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.79, 4.48, 3.71 [17:19:45] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.42, 6.69, 6.77 [17:23:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.48, 7.61, 7.12 [17:25:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.24, 6.98, 6.94 [17:26:07] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:26:52] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.80, 3.45, 3.56 [17:27:44] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.14, 6.57, 6.79 [17:28:05] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 3.929 second response time [17:32:36] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.07158014178 secs [17:32:50] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.07, 3.08, 3.35 [17:35:35] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.07, 3.81, 2.96 [17:39:25] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.18, 3.72, 3.14 [17:41:21] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.78, 3.40, 3.09 [17:51:00] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.24, 3.58, 3.16 [17:51:13] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.39, 6.88, 6.73 [17:53:52] PROBLEM - biblestrength.net - LetsEncrypt on sslhost is CRITICAL: connect to address biblestrength.net and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [17:54:36] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1304847002 secs [17:54:56] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.18, 3.91, 3.39 [17:56:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.79, 3.45, 3.31 [17:57:01] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.39, 8.29, 7.33 [17:58:56] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.28, 2.70, 3.05 [17:58:57] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.84, 7.51, 7.15 [18:00:53] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.30, 7.66, 7.23 [18:02:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [18:04:36] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.09698191285 secs [18:04:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.66, 7.78, 7.43 [18:10:32] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.28, 8.10, 7.61 [18:12:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.14, 3.69, 2.81 [18:12:28] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.31, 7.46, 7.42 [18:18:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.75, 3.77, 3.14 [18:22:07] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.19, 8.40, 7.85 [18:24:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 5.30, 4.29, 3.54 [18:26:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.33, 3.90, 3.49 [18:27:54] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.98, 7.10, 7.55 [18:28:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.05, 3.97, 3.56 [18:30:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.83, 3.88, 3.58 [18:35:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.99, 7.43, 7.42 [18:37:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.92, 6.63, 7.15 [18:38:08] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.16, 3.81, 3.63 [18:39:35] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.80, 6.13, 3.73 [18:40:08] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:41:31] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 5.19, 6.19, 4.06 [18:44:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.83, 3.97, 3.80 [18:44:15] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 9.562 second response time [18:47:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.18, 7.52, 7.30 [18:49:20] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.95, 3.43, 3.02 [18:51:19] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 3.06, 4.10, 3.38 [18:51:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.12, 7.61, 7.42 [18:53:19] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.56, 3.14, 3.11 [18:53:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.35, 7.74, 7.47 [18:54:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.86, 3.01, 3.37 [18:59:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.48, 7.39, 7.53 [19:02:00] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [19:03:37] PROBLEM - uk.religiononfire.mar.in.ua - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'uk.religiononfire.mar.in.ua' expires in 15 day(s) (Thu 01 Jun 2023 18:41:45 GMT +0000). [19:05:08] PROBLEM - en.religiononfire.mar.in.ua - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'en.religiononfire.mar.in.ua' expires in 15 day(s) (Thu 01 Jun 2023 18:42:11 GMT +0000). [19:05:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.39, 7.30, 7.45 [19:07:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.01, 6.51, 7.14 [19:07:59] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 9.03, 4.70, 3.28 [19:09:08] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/69b0cdad852c...484c919ae058 [19:09:11] [02miraheze/ssl] 07MirahezeSSLBot 03484c919 - Bot: Update SSL cert for uk.religiononfire.mar.in.ua [19:09:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.57, 8.26, 7.69 [19:09:55] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.10, 3.78, 3.11 [19:11:50] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.72, 3.09, 2.94 [19:13:10] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.78, 2.93, 2.55 [19:13:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.33, 7.78, 7.66 [19:15:10] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.44, 2.63, 2.48 [19:16:51] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/484c919ae058...fcd32b4e6f1b [19:16:52] [02miraheze/ssl] 07MirahezeSSLBot 03fcd32b4 - Bot: Update SSL cert for en.religiononfire.mar.in.ua [19:19:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.39, 7.68, 7.60 [19:21:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.02, 7.90, 7.72 [19:23:25] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.33, 3.42, 3.09 [19:25:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.87, 7.75, 7.61 [19:29:12] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.65, 2.79, 2.97 [19:31:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.04, 6.94, 7.38 [19:33:47] RECOVERY - en.religiononfire.mar.in.ua - LetsEncrypt on sslhost is OK: OK - Certificate 'en.religiononfire.mar.in.ua' will expire on Mon 14 Aug 2023 18:16:44 GMT +0000. [19:36:56] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.47, 3.93, 3.41 [19:38:56] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.48, 3.30, 3.24 [19:45:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.58, 7.66, 7.31 [19:47:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.93, 7.35, 7.22 [19:49:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.20, 7.37, 7.23 [19:51:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.45, 7.22, 7.19 [19:53:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.32, 7.39, 7.24 [19:55:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.93, 7.10, 7.17 [20:01:37] RECOVERY - uk.religiononfire.mar.in.ua - LetsEncrypt on sslhost is OK: OK - Certificate 'uk.religiononfire.mar.in.ua' will expire on Mon 14 Aug 2023 18:09:01 GMT +0000. [20:03:44] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.25, 6.07, 6.71 [20:05:50] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.43, 4.24, 3.20 [20:07:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.44, 7.88, 7.30 [20:07:49] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.15, 3.43, 3.02 [20:09:49] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.75, 3.85, 3.20 [20:10:44] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 12.43, 7.09, 4.25 [20:11:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.50, 6.67, 7.02 [20:12:23] PROBLEM - os141 Current Load on os141 is CRITICAL: CRITICAL - load average: 4.69, 3.32, 1.87 [20:13:47] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.82, 3.96, 3.40 [20:15:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.65, 7.33, 7.20 [20:15:47] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.97, 3.19, 3.18 [20:16:21] RECOVERY - os141 Current Load on os141 is OK: OK - load average: 0.93, 2.86, 2.11 [20:16:44] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.08, 5.57, 4.70 [20:17:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.92, 7.34, 7.26 [20:19:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.19, 7.63, 7.37 [20:21:44] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.82, 3.85, 3.41 [20:23:43] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.03, 3.48, 3.32 [20:23:48] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 3.83, 4.07, 3.31 [20:25:43] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.61, 3.09, 3.19 [20:27:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.85, 3.60, 3.30 [20:29:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [20:29:35] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.65, 2.95, 3.10 [20:29:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.26, 7.60, 7.68 [20:33:57] I'm having issues every time I try to edit or create a page on my new wiki, the webpage crashes [20:39:27] Ryen: any error in particular? [20:39:36] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.14, 3.49, 3.26 [20:41:35] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.78, 2.82, 3.04 [20:41:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.44, 7.53, 7.36 [20:43:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.01, 7.15, 7.26 [20:45:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.34, 7.37, 7.31 [20:47:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.42, 7.73, 7.47 [20:49:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.15, 8.33, 7.73 [20:53:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.01, 7.67, 7.63 [20:54:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.43, 3.64, 3.25 [20:55:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.71, 3.85, 3.38 [20:56:56] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.05, 3.20, 3.14 [20:57:35] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.01, 3.18, 3.19 [21:04:00] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [21:07:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.87, 8.59, 7.70 [21:09:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.09, 7.11, 7.26 [21:11:08] PROBLEM - phab121 Puppet on phab121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_phabricator-extensions] [21:11:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.98, 7.64, 7.40 [21:13:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.35, 7.18, 7.24 [21:15:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.75, 7.67, 7.40 [21:17:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.71, 7.24, 7.28 [21:19:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.28, 8.61, 7.76 [21:20:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.63, 4.19, 3.40 [21:22:30] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.09, 3.54, 3.24 [21:23:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.75, 7.77, 7.66 [21:24:29] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.19, 2.67, 2.95 [21:25:04] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 9.27, 5.02, 3.49 [21:28:56] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.55, 3.53, 3.22 [21:30:56] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.66, 2.96, 3.05 [21:35:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.95, 8.49, 7.77 [21:39:08] RECOVERY - phab121 Puppet on phab121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [21:39:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.51, 7.93, 7.68 [21:41:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.29, 7.98, 7.72 [21:47:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.72, 7.59, 7.81 [21:47:46] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1231064796 secs [21:49:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.18, 7.98, 7.91 [21:50:19] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.93, 3.52, 2.99 [21:52:44] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 13.07, 8.31, 5.04 [21:53:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.63, 7.58, 7.80 [21:54:17] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.32, 3.21, 3.00 [21:54:44] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.30, 6.52, 4.80 [21:55:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.66, 3.72, 3.15 [21:57:34] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.82, 3.35, 3.08 [21:58:15] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.11, 4.12, 3.42 [21:59:30] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [22:01:46] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.08239644766 secs [22:02:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [22:03:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.22, 7.35, 7.62 [22:05:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.71, 6.67, 7.33 [22:06:12] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.30, 3.81, 3.68 [22:07:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.64, 7.39, 7.50 [22:11:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.69, 7.35, 7.48 [22:12:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.71, 3.44, 2.84 [22:14:07] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.00, 3.21, 2.83 [22:15:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.22, 7.31, 7.39 [22:16:09] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.73, 2.74, 3.29 [22:17:00] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [22:17:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.05, 6.93, 7.25 [22:19:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.81, 7.79, 7.53 [22:21:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.37, 7.08, 7.30 [22:25:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.40, 7.47, 7.33 [22:27:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.11, 7.08, 7.19 [22:29:12] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [22:31:44] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.65, 6.19, 6.79 [22:38:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.66, 7.76, 7.33 [22:40:33] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.93, 7.99, 7.45 [22:42:03] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.66, 3.48, 2.93 [22:43:59] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.19, 3.04, 2.84 [22:44:26] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.57, 7.52, 7.40 [22:47:52] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.93, 3.48, 3.06 [22:49:47] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.32, 3.01, 2.93 [22:54:06] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.57, 6.23, 6.80 [22:57:09] PROBLEM - cp32 NTP time on cp32 is WARNING: NTP WARNING: Offset 0.1680530906 secs [22:58:55] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [22:59:55] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.81, 6.82, 6.88 [23:03:46] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.90, 6.36, 6.73 [23:07:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.34, 6.78, 6.87 [23:09:44] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.84, 6.00, 6.57 [23:10:04] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 3.94, 4.58, 3.77 [23:12:00] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.56, 3.77, 3.56 [23:15:09] RECOVERY - cp32 NTP time on cp32 is OK: NTP OK: Offset 0.0728738904 secs [23:15:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.62, 2.83, 3.25 [23:20:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [23:21:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 7.76, 8.49, 7.44 [23:27:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.11, 7.79, 7.47 [23:29:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.30, 7.81, 7.50 [23:30:44] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 10.36, 6.33, 3.76 [23:31:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 8.00, 7.82, 7.53 [23:36:44] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.94, 5.87, 4.64 [23:39:38] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.68, 4.08, 3.36 [23:41:37] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.18, 3.33, 3.17 [23:45:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.64, 4.00, 3.45 [23:47:35] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.71, 3.82, 3.45 [23:49:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.74, 4.81, 3.82 [23:49:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.89, 7.59, 7.32 [23:51:41] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.87, 3.70, 3.29 [23:51:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.81, 7.11, 7.17 [23:53:35] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.22, 3.74, 3.62 [23:53:36] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.83, 5.21, 3.93 [23:55:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 3.63, 4.09, 3.78 [23:57:27] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.75, 3.63, 3.60 [23:59:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.23, 4.12, 3.77 [23:59:35] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.89, 3.74, 3.75 [23:59:44] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.55, 5.97, 6.66