[00:01:28] RECOVERY - test131 Disk Space on test131 is OK: DISK OK - free space: / 3702 MB (12% inode=79%); [00:03:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.97, 6.56, 3.54 [00:03:15] PROBLEM - es141 Current Load on es141 is WARNING: WARNING - load average: 4.00, 2.62, 1.41 [00:04:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.74, 3.83, 3.69 [00:05:16] PROBLEM - es141 Current Load on es141 is CRITICAL: CRITICAL - load average: 4.44, 3.17, 1.76 [00:06:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.68, 3.70, 3.66 [00:09:15] PROBLEM - es141 Current Load on es141 is WARNING: WARNING - load average: 3.97, 3.79, 2.36 [00:11:14] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.71, 3.75, 3.92 [00:11:15] RECOVERY - es141 Current Load on es141 is OK: OK - load average: 2.62, 3.34, 2.37 [00:12:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.86, 2.62, 3.21 [00:13:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 5.63, 7.35, 5.82 [00:15:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.72, 6.66, 5.77 [00:15:08] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 9.61, 4.77, 4.16 [00:20:58] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.19, 3.73, 3.87 [00:27:47] [02ManageWiki] 07Universal-Omega reviewed pull request 03#390 commit - 13https://github.com/miraheze/ManageWiki/pull/390#discussion_r1152615752 [00:28:44] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.02, 3.28, 3.57 [00:30:15] PROBLEM - db112 Disk Space on db112 is WARNING: DISK WARNING - free space: / 8049 MB (6% inode=98%); [00:32:38] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.24, 2.74, 3.27 [00:36:34] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.44, 4.48, 3.85 [00:39:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.93, 7.01, 5.39 [00:42:15] PROBLEM - db112 Disk Space on db112 is CRITICAL: DISK CRITICAL - free space: / 7969 MB (5% inode=98%); [00:48:52] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [00:50:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 1.18, 3.01, 3.87 [00:53:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 5.76, 7.63, 7.20 [00:54:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.35, 3.49, 3.80 [00:54:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.94, 3.73, 2.96 [00:55:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.56, 6.05, 6.66 [00:56:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.17, 3.16, 3.63 [00:56:50] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.08, 3.58, 3.01 [00:59:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.90, 8.66, 7.73 [01:00:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.43, 3.27, 3.51 [01:00:47] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.74, 2.89, 2.90 [01:01:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 5.99, 7.94, 7.60 [01:02:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.07, 2.68, 3.25 [01:02:37] PROBLEM - cp23 APT on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:04:37] RECOVERY - cp23 APT on cp23 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [01:09:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.09, 4.77, 6.37 [01:11:14] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1916491985 secs [01:13:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.48, 7.73, 7.23 [01:13:11] PROBLEM - es141 Current Load on es141 is CRITICAL: CRITICAL - load average: 4.02, 3.22, 2.85 [01:15:13] RECOVERY - es141 Current Load on es141 is OK: OK - load average: 2.88, 3.27, 2.93 [01:17:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 4.24, 7.24, 7.31 [01:17:45] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [01:19:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.11, 5.08, 6.51 [01:22:26] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.18, 3.42, 2.93 [01:30:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.10, 3.58, 3.41 [01:32:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.81, 3.04, 3.23 [01:33:16] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.10, 4.12, 3.41 [01:35:14] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset -0.04699060321 secs [01:41:03] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.94, 3.70, 3.60 [01:44:56] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.53, 2.74, 3.25 [01:46:12] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.18, 3.48, 3.25 [01:46:52] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [01:48:11] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.30, 3.67, 3.34 [01:52:08] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.59, 3.54, 3.39 [01:54:06] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.42, 4.25, 3.69 [01:56:04] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.47, 3.86, 3.61 [01:58:03] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.05, 4.27, 3.76 [01:58:42] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.53, 2.72, 1.53 [02:00:01] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.02, 3.68, 3.60 [02:01:27] PROBLEM - cp23 APT on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:02:41] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.48, 3.46, 2.14 [02:03:27] RECOVERY - cp23 APT on cp23 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [02:03:48] PROBLEM - es141 Current Load on es141 is CRITICAL: CRITICAL - load average: 4.30, 2.21, 1.18 [02:03:58] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.06, 2.77, 3.25 [02:04:41] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.06, 2.61, 1.99 [02:05:44] RECOVERY - es141 Current Load on es141 is OK: OK - load average: 3.12, 2.46, 1.39 [02:12:16] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.91, 4.25, 3.52 [02:14:12] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.57, 3.93, 3.48 [02:16:09] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.74, 3.35, 3.31 [02:20:05] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.01, 3.43, 3.37 [02:23:58] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.52, 4.16, 3.62 [02:31:45] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.58, 3.93, 3.84 [02:33:41] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.76, 4.56, 4.05 [02:39:31] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.34, 3.93, 3.96 [02:45:21] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.17, 2.41, 3.23 [02:54:12] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.29, 4.47, 3.36 [02:56:39] [02ManageWiki] 07joritochip reviewed pull request 03#390 commit - 13https://github.com/miraheze/ManageWiki/pull/390#discussion_r1152677880 [03:07:26] [02ManageWiki] 07Universal-Omega reviewed pull request 03#390 commit - 13https://github.com/miraheze/ManageWiki/pull/390#discussion_r1152682774 [03:11:58] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.29, 2.88, 3.72 [03:17:53] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.96, 2.65, 3.33 [03:18:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.13, 3.78, 3.10 [03:20:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.63, 3.85, 3.21 [03:22:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.03, 3.28, 3.08 [03:23:47] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.51, 3.85, 3.65 [03:25:46] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.28, 3.85, 3.66 [03:26:15] PROBLEM - db112 Disk Space on db112 is WARNING: DISK WARNING - free space: / 7989 MB (6% inode=98%); [03:26:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.75, 4.07, 3.40 [03:27:44] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.58, 2.99, 3.37 [03:30:15] PROBLEM - db112 Disk Space on db112 is CRITICAL: DISK CRITICAL - free space: / 7968 MB (5% inode=98%); [03:30:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.86, 3.46, 3.29 [03:32:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.03, 3.32, 3.26 [03:37:28] PROBLEM - test131 Disk Space on test131 is WARNING: DISK WARNING - free space: / 3125 MB (10% inode=79%); [03:39:42] All the Miraheze domains that are registered by me have been renewed for another year [04:12:00] miraheze/ManageWiki - joritochip the build has errored. [04:14:32] PROBLEM - gs.sidem.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - gs.sidem.wiki All nameservers failed to answer the query. [04:25:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.61, 4.24, 3.05 [04:29:48] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.94, 3.40, 3.01 [04:30:03] RECOVERY - test131 MediaWiki Rendering on test131 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 2.213 second response time [04:44:19] RECOVERY - gs.sidem.wiki - reverse DNS on sslhost is OK: SSL OK - gs.sidem.wiki reverse DNS resolves to cp23.miraheze.org - CNAME OK [04:48:32] [02mw-config] 07Reception123 closed pull request 03#5166: Add NearbyPages to ManageWikiExtensions - 13https://github.com/miraheze/mw-config/pull/5166 [04:48:33] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/mw-config/compare/0cff04447d9c...6f1bdf0edb67 [04:48:35] [02miraheze/mw-config] 07AgentIsai 036f1bdf0 - Add NearbyPages to ManageWikiExtensions (#5166) [04:48:53] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [04:49:01] !log [reception@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [04:49:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:49:12] !log [reception@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 10s [04:49:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:49:48] miraheze/mw-config - Reception123 the build passed. [04:56:49] !log [@test131] starting deploy of {'config': True} to all [04:56:50] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [04:56:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:56:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:05:26] !log MariaDB [metawiki]> DELETE FROM echo_unread_wikis WHERE euw_user = '127980' AND euw_wiki = 'cosmoswiki'; [05:05:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:46:52] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [06:18:52] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [06:28:41] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.17, 4.39, 2.34 [06:30:41] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.11, 3.10, 2.11 [06:47:45] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [07:00:42] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.51, 2.68, 1.79 [07:01:16] PROBLEM - cp23 APT on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [07:02:42] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 6.01, 3.91, 2.35 [07:03:51] RECOVERY - cp23 APT on cp23 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [07:06:41] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.90, 2.97, 2.40 [07:16:19] PROBLEM - gs.sidem.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - gs.sidem.wiki All nameservers failed to answer the query. [07:23:43] PROBLEM - www.lab612.at - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query lab612.at. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [07:46:06] RECOVERY - gs.sidem.wiki - reverse DNS on sslhost is OK: SSL OK - gs.sidem.wiki reverse DNS resolves to cp23.miraheze.org - CNAME OK [07:52:54] RECOVERY - www.lab612.at - reverse DNS on sslhost is OK: SSL OK - www.lab612.at reverse DNS resolves to cp22.miraheze.org - CNAME OK [08:39:14] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.136186868 secs [08:51:47] PROBLEM - wiki.albrusmusic.ru - reverse DNS on sslhost is WARNING: Timeout: The DNS operation timed out after 5.40685248374939 seconds [09:05:14] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.08925941586 secs [09:21:44] RECOVERY - wiki.albrusmusic.ru - reverse DNS on sslhost is OK: SSL OK - wiki.albrusmusic.ru reverse DNS resolves to cp22.miraheze.org - CNAME OK [09:50:09] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query en.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [10:00:44] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1461144686 secs [10:11:09] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.58, 2.85, 1.98 [10:13:06] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.61, 2.37, 1.91 [10:18:44] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.07786309719 secs [10:18:48] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [10:26:23] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [10:30:23] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 7 = Critical] [10:48:44] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset -0.1104097366 secs [10:50:31] !log swift post miraheze-fictionpediawiki-avatars --read-acl 'mw:media,mw:media,.r:*' --write-acl 'mw:media,mw:media' [10:50:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:50:44] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.09304738045 secs [11:10:35] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/3624630ea74b...cfffdaf93956 [11:10:36] [02miraheze/MirahezeMagic] 07translatewiki 03cfffdaf - Localisation updates from https://translatewiki.net. [11:10:39] [02miraheze/ManageWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/f34f3f22f3ff...7e2f14bfb7ed [11:10:42] [02miraheze/ManageWiki] 07translatewiki 037e2f14b - Localisation updates from https://translatewiki.net. [11:10:44] [02miraheze/landing] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/landing/compare/683c8482068c...c496b3bd9827 [11:10:46] [02miraheze/landing] 07translatewiki 03c496b3b - Localisation updates from https://translatewiki.net. [11:13:30] miraheze/landing - translatewiki the build passed. [11:16:39] miraheze/ManageWiki - translatewiki the build passed. [11:18:09] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [11:21:40] miraheze/MirahezeMagic - translatewiki the build passed. [11:56:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.53, 4.65, 2.84 [11:58:19] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:00:22] RECOVERY - cp33 APT on cp33 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [12:00:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.85, 3.31, 2.72 [12:18:53] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [12:26:28] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:28:30] RECOVERY - cp33 APT on cp33 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [12:42:53] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.1094011068 secs [12:48:20] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:50:25] RECOVERY - cp33 APT on cp33 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [12:59:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.13, 5.29, 2.77 [13:03:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.04, 5.81, 3.66 [13:06:53] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.08750593662 secs [13:16:52] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [13:25:31] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.80, 3.72, 2.62 [13:27:30] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.64, 3.61, 2.74 [13:29:28] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.29, 2.79, 2.54 [13:53:08] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.57, 3.12, 2.45 [13:57:05] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.16, 3.27, 2.75 [13:58:41] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.09, 2.72, 1.82 [14:00:41] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.85, 3.04, 2.05 [14:01:12] PROBLEM - cp23 APT on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [14:02:41] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.44, 2.56, 2.01 [14:03:12] RECOVERY - cp23 APT on cp23 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [14:24:41] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.92, 4.16, 3.10 [14:25:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 8.86, 5.42, 3.06 [14:26:39] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.85, 3.24, 2.89 [14:29:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.82, 5.40, 3.71 [14:52:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.08, 3.37, 2.68 [14:54:23] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.43, 2.67, 2.28 [14:54:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.45, 2.63, 2.49 [14:56:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.25, 2.47, 2.26 [15:24:56] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.75, 3.71, 2.69 [15:26:44] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1035114825 secs [15:26:53] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.61, 3.48, 2.73 [15:28:49] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.95, 4.72, 3.31 [15:30:46] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.30, 3.90, 3.18 [15:36:36] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.80, 3.05, 3.08 [15:39:45] PROBLEM - cp33 APT on cp33 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [15:42:10] RECOVERY - cp33 APT on cp33 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [15:54:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.61, 3.63, 2.68 [15:56:23] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.18, 3.13, 2.62 [15:56:44] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.009499996901 secs [16:18:53] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [16:35:02] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.73, 3.66, 3.24 [16:36:59] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.94, 3.29, 3.14 [16:48:43] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.17, 3.25, 2.84 [16:50:40] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.28, 3.19, 2.87 [16:57:49] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.83, 3.87, 3.14 [16:57:59] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query zulaclub.net. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [16:58:42] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.41, 2.78, 1.84 [17:00:41] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.43, 2.30, 1.78 [17:01:46] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.27, 3.35, 3.13 [17:09:12] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.69, 3.77, 3.09 [17:11:08] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.64, 3.22, 2.96 [17:15:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.22, 10.26, 8.88 [17:16:52] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [17:17:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.64, 9.47, 8.74 [17:18:58] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.00, 4.50, 3.64 [17:20:54] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.13, 3.68, 3.44 [17:28:41] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.38, 3.22, 3.36 [17:55:59] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.zulaclub.net All nameservers failed to answer the query. [18:40:32] PROBLEM - wiki.otir.nl - reverse DNS on sslhost is WARNING: Timeout: The DNS operation timed out after 5.406639575958252 seconds [18:56:01] RECOVERY - wiki.zulaclub.net - reverse DNS on sslhost is OK: SSL OK - wiki.zulaclub.net reverse DNS resolves to cp23.miraheze.org - CNAME OK [18:56:09] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.88, 4.06, 2.87 [18:58:08] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.88, 3.15, 2.68 [19:01:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.82, 6.70, 3.90 [19:05:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 4.81, 6.99, 4.74 [19:07:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 1.29, 5.00, 4.29 [19:09:49] RECOVERY - wiki.otir.nl - reverse DNS on sslhost is OK: SSL OK - wiki.otir.nl reverse DNS resolves to cp23.miraheze.org - CNAME OK [19:36:35] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.10, 3.13, 2.56 [19:38:32] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.23, 2.99, 2.57 [19:50:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.46, 3.64, 3.05 [19:52:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.88, 3.26, 2.97 [20:11:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 11.48, 7.25, 4.28 [20:11:19] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.66, 10.46, 9.30 [20:13:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 3.88, 5.86, 4.14 [20:13:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.34, 9.17, 8.96 [20:20:09] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [20:23:19] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.56, 10.59, 9.63 [20:25:19] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.93, 9.34, 9.30 [20:37:00] PROBLEM - db142 Current Load on db142 is WARNING: WARNING - load average: 7.69, 5.55, 4.04 [20:37:31] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.22, 3.43, 2.92 [20:39:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 2.47, 4.25, 3.74 [20:39:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.98, 3.00, 2.83 [20:48:48] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [20:58:42] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 10.72, 5.04, 2.66 [21:02:41] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 1.37, 3.83, 2.80 [21:04:41] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.14, 2.91, 2.59 [21:18:09] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [21:26:21] [02miraheze/mw-config] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/6f1bdf0edb67...7d27a0449728 [21:26:23] [02miraheze/mw-config] 07MacFan4000 037d27a04 - add OrangeStar access to staffwiki [21:27:25] miraheze/mw-config - MacFan4000 the build passed. [21:27:30] !log [@test131] starting deploy of {'config': True} to all [21:27:31] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [21:27:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:27:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:28:06] !log [@mwtask141] starting deploy of {'config': True} to all [21:28:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:28:14] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 8s [21:28:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:28:46] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 7.08, 5.11, 2.93 [21:28:55] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/invalidateUserSessions.php --wiki=metawiki --user=MacFan4000 (END - exit=0) [21:29:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:30:46] PROBLEM - cp23 APT on cp23 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [21:32:44] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.25, 3.37, 2.74 [21:32:46] RECOVERY - cp23 APT on cp23 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [21:48:52] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [21:53:40] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.94, 4.47, 3.14 [21:58:41] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.81, 3.44, 2.27 [22:00:41] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 2.84, 3.02, 2.25 [22:03:32] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.80, 3.69, 3.57 [22:07:29] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.41, 2.75, 3.22 [22:15:00] PROBLEM - db142 Current Load on db142 is CRITICAL: CRITICAL - load average: 9.89, 5.81, 3.06 [22:17:00] RECOVERY - db142 Current Load on db142 is OK: OK - load average: 4.19, 5.43, 3.28 [22:17:45] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [22:23:02] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.86, 3.47, 2.86 [22:28:52] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.56, 3.71, 3.22 [22:30:49] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.85, 3.16, 3.08 [22:38:39] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.56, 3.93, 3.44 [22:44:29] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.86, 4.05, 3.51 [22:46:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.93, 3.58, 3.41 [22:46:52] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [22:48:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.22, 2.78, 3.14 [22:56:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.55, 4.52, 3.62 [22:58:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.34, 3.68, 3.43 [23:00:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.45, 3.65, 3.42 [23:02:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.83, 3.65, 3.45 [23:04:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.09, 4.44, 3.77 [23:06:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.06, 3.85, 3.64 [23:12:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.18, 3.96, 3.71 [23:18:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.30, 3.85, 3.77 [23:18:52] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query. [23:28:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.94, 2.82, 3.28 [23:29:57] [02ssl] 07Universal-Omega opened pull request 03#644: Remove dcmultiversewiki redirects - 13https://github.com/miraheze/ssl/pull/644 [23:34:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.32, 3.41, 3.32 [23:36:27] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.90, 3.69, 3.44 [23:38:27] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.51, 3.37, 3.36 [23:47:45] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [23:55:15] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.81, 4.64, 3.58 [23:56:27] PROBLEM - dcmultiversewiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address dcmultiversewiki.com and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket