[00:02:28] PROBLEM - cp33 Disk Space on cp33 is WARNING: DISK WARNING - free space: / 7387MiB (9% inode=98%); [00:02:50] PROBLEM - cp23 Current Load on cp23 is CRITICAL: LOAD CRITICAL - total load average: 4.51, 2.78, 1.38 [00:02:55] PROBLEM - cp22 Current Load on cp22 is WARNING: LOAD WARNING - total load average: 3.42, 2.90, 1.47 [00:02:56] PROBLEM - cp32 Disk Space on cp32 is WARNING: DISK WARNING - free space: / 5503MiB (7% inode=98%); [00:03:36] RECOVERY - cp22 Disk Space on cp22 is OK: DISK OK - free space: / 9773MiB (13% inode=98%); [00:04:50] PROBLEM - cp23 Current Load on cp23 is WARNING: LOAD WARNING - total load average: 3.93, 3.05, 1.64 [00:04:55] RECOVERY - cp22 Current Load on cp22 is OK: LOAD OK - total load average: 1.84, 2.46, 1.48 [00:05:14] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.40, 3.82, 3.60 [00:06:28] RECOVERY - cp33 Disk Space on cp33 is OK: DISK OK - free space: / 19261MiB (25% inode=98%); [00:06:49] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.12, 3.33, 2.06 [00:06:50] PROBLEM - cp23 Current Load on cp23 is CRITICAL: LOAD CRITICAL - total load average: 7.67, 4.90, 2.49 [00:07:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.30, 3.66, 3.58 [00:08:49] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.16, 3.34, 2.22 [00:12:55] PROBLEM - cp22 Current Load on cp22 is CRITICAL: LOAD CRITICAL - total load average: 9.33, 6.22, 3.54 [00:13:41] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [00:15:11] PROBLEM - cp23 NTP time on cp23 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:15:12] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 109.228.51.216/cpweb, 2a00:da00:1800:328::1/cpweb [00:15:28] PROBLEM - www.publictestwiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:16:01] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 109.228.51.216/cpweb, 2a00:da00:1800:328::1/cpweb [00:17:14] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.50, 3.05, 3.36 [00:17:31] PROBLEM - cp23 PowerDNS Recursor on cp23 is CRITICAL: CRITICAL - Plugin timed out while executing system call [00:17:36] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.172142297 secs [00:17:54] PROBLEM - cp23 HTTP 4xx/5xx ERROR Rate on cp23 is WARNING: WARNING - NGINX Error Rate is 43% [00:18:14] PROBLEM - cp23 Varnish Backends on cp23 is CRITICAL: 1 backends are down. mw134 [00:19:12] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [00:19:28] RECOVERY - cp23 PowerDNS Recursor on cp23 is OK: DNS OK: 0.204 seconds response time. miraheze.org returns 109.228.51.216,217.174.247.33,2a00:da00:1800:326::1,2a00:da00:1800:328::1 [00:19:50] RECOVERY - cp23 HTTP 4xx/5xx ERROR Rate on cp23 is OK: OK - NGINX Error Rate is 4% [00:20:01] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [00:20:09] RECOVERY - cp23 Varnish Backends on cp23 is OK: All 15 backends are healthy [00:26:55] PROBLEM - cp22 Current Load on cp22 is WARNING: LOAD WARNING - total load average: 0.60, 2.42, 3.96 [00:30:55] RECOVERY - cp22 Current Load on cp22 is OK: LOAD OK - total load average: 0.38, 1.28, 3.14 [00:36:49] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.45, 3.00, 2.72 [00:38:49] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.88, 2.95, 2.73 [00:44:56] PROBLEM - cp23 Current Load on cp23 is WARNING: LOAD WARNING - total load average: 0.23, 0.46, 3.64 [00:45:01] RECOVERY - www.publictestwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'publictestwiki.com' will expire on Thu 05 Oct 2023 01:42:35 GMT +0000. [00:46:52] RECOVERY - cp23 Current Load on cp23 is OK: LOAD OK - total load average: 0.11, 0.35, 3.22 [01:05:15] PROBLEM - phab121 Current Load on phab121 is CRITICAL: CRITICAL - load average: 3.15, 2.08, 1.20 [01:07:16] PROBLEM - cp33 Current Load on cp33 is CRITICAL: LOAD CRITICAL - total load average: 22.31, 14.65, 6.33 [01:07:43] PROBLEM - swiftproxy131 Current Load on swiftproxy131 is CRITICAL: CRITICAL - load average: 12.48, 12.85, 6.48 [01:08:27] PROBLEM - cp32 Current Load on cp32 is CRITICAL: LOAD CRITICAL - total load average: 13.25, 24.60, 12.20 [01:08:50] PROBLEM - cp23 Current Load on cp23 is CRITICAL: LOAD CRITICAL - total load average: 9.63, 22.45, 12.02 [01:08:55] PROBLEM - cp22 Current Load on cp22 is CRITICAL: LOAD CRITICAL - total load average: 24.36, 54.87, 27.64 [01:11:43] RECOVERY - swiftproxy131 Current Load on swiftproxy131 is OK: OK - load average: 0.80, 6.44, 5.32 [01:16:50] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1495488584 secs [01:21:14] PROBLEM - phab121 Current Load on phab121 is WARNING: WARNING - load average: 1.68, 2.00, 1.81 [01:21:16] PROBLEM - cp33 Current Load on cp33 is WARNING: LOAD WARNING - total load average: 1.42, 2.50, 3.73 [01:23:16] PROBLEM - cp33 Current Load on cp33 is CRITICAL: LOAD CRITICAL - total load average: 12.85, 6.27, 4.91 [01:26:50] PROBLEM - cp23 Current Load on cp23 is WARNING: LOAD WARNING - total load average: 0.37, 0.97, 3.97 [01:29:16] PROBLEM - cp33 Current Load on cp33 is WARNING: LOAD WARNING - total load average: 0.62, 2.58, 3.68 [01:30:50] RECOVERY - cp23 Current Load on cp23 is OK: LOAD OK - total load average: 0.16, 0.50, 3.09 [01:31:16] RECOVERY - cp33 Current Load on cp33 is OK: LOAD OK - total load average: 0.58, 1.90, 3.30 [01:33:37] PROBLEM - cp32 Current Load on cp32 is WARNING: LOAD WARNING - total load average: 1.16, 1.77, 4.00 [01:35:15] PROBLEM - phab121 Current Load on phab121 is CRITICAL: CRITICAL - load average: 2.74, 2.15, 1.95 [01:37:37] RECOVERY - cp32 Current Load on cp32 is OK: LOAD OK - total load average: 1.08, 1.34, 3.30 [01:40:55] PROBLEM - cp22 Current Load on cp22 is WARNING: LOAD WARNING - total load average: 0.14, 0.28, 3.72 [01:42:55] RECOVERY - cp22 Current Load on cp22 is OK: LOAD OK - total load average: 0.15, 0.26, 3.29 [01:47:14] PROBLEM - phab121 Current Load on phab121 is WARNING: WARNING - load average: 1.63, 1.91, 1.99 [01:48:02] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.42, 3.45, 3.13 [01:48:22] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 9.17, 6.58, 5.25 [01:50:21] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 5.32, 5.94, 5.17 [01:59:14] RECOVERY - phab121 Current Load on phab121 is OK: OK - load average: 1.30, 1.40, 1.65 [02:05:25] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.08, 3.36, 3.34 [02:11:34] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1011049449 secs [02:17:04] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.69, 3.57, 3.43 [02:20:56] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.81, 3.27, 3.36 [02:32:34] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.42, 3.35, 3.33 [02:36:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.04, 3.37, 3.36 [02:40:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.62, 3.55, 3.43 [02:44:11] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.05, 3.34, 3.38 [02:54:52] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.65, 3.45, 3.39 [03:00:40] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.12, 3.39, 3.39 [03:06:29] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.63, 3.41, 3.37 [03:16:09] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.21, 3.36, 3.40 [04:32:44] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.zulaclub.net All nameservers failed to answer the query. [05:07:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.93, 3.21, 2.58 [05:09:14] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.33, 3.26, 2.68 [05:31:09] RECOVERY - wiki.zulaclub.net - reverse DNS on sslhost is OK: SSL OK - wiki.zulaclub.net reverse DNS resolves to cp23.miraheze.org - CNAME OK [05:50:14] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.34, 3.41, 3.10 [05:52:10] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.22, 3.34, 3.12 [05:56:04] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.12, 3.47, 3.24 [05:59:56] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.46, 3.85, 3.44 [06:10:56] PROBLEM - cp32 Disk Space on cp32 is CRITICAL: DISK CRITICAL - free space: / 4490MiB (5% inode=98%); [06:28:50] PROBLEM - cp23 Current Load on cp23 is CRITICAL: LOAD CRITICAL - total load average: 7.36, 10.56, 5.12 [06:34:50] PROBLEM - cp23 Current Load on cp23 is WARNING: LOAD WARNING - total load average: 0.17, 3.35, 3.55 [06:36:50] RECOVERY - cp23 Current Load on cp23 is OK: LOAD OK - total load average: 0.45, 2.43, 3.19 [07:03:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.00, 3.49, 3.92 [07:09:14] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 1.66, 2.31, 3.26 [07:23:14] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.03, 3.30, 3.08 [07:25:14] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.59, 2.99, 2.99 [07:32:55] PROBLEM - bast141 Current Load on bast141 is CRITICAL: LOAD CRITICAL - total load average: 5.06, 2.84, 1.20 [07:39:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.67, 2.91, 2.70 [07:41:14] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.38, 3.00, 2.75 [07:42:55] RECOVERY - bast141 Current Load on bast141 is OK: LOAD OK - total load average: 0.01, 1.43, 1.62 [07:49:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.87, 3.52, 3.05 [08:05:14] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.87, 3.22, 3.22 [08:16:27] PROBLEM - wiki.kinkpedia.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.kinkpedia.org could not be found [08:19:56] PROBLEM - db121 Backups SQL on db121 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864141 seconds old and 53694 bytes [08:23:07] PROBLEM - cloud13 Puppet on cloud13 is UNKNOWN: NRPE: Unable to read output [08:51:07] RECOVERY - cloud13 Puppet on cloud13 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:09:48] RECOVERY - sparabel.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'sparabel.wiki' will expire on Sat 07 Oct 2023 20:33:09 GMT +0000. [09:25:14] PROBLEM - cp23 NTP time on cp23 is CRITICAL: NTP CRITICAL: Offset 0.5004250109 secs [09:41:44] PROBLEM - sparabel.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address sparabel.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [09:48:50] PROBLEM - cp23 Current Load on cp23 is CRITICAL: LOAD CRITICAL - total load average: 3.03, 5.04, 2.75 [09:50:50] PROBLEM - cp23 Current Load on cp23 is WARNING: LOAD WARNING - total load average: 0.73, 3.52, 2.47 [09:52:50] RECOVERY - cp23 Current Load on cp23 is OK: LOAD OK - total load average: 0.33, 2.45, 2.20 [10:40:18] RECOVERY - sparabel.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'sparabel.wiki' will expire on Sat 07 Oct 2023 20:33:09 GMT +0000. [11:11:44] PROBLEM - sparabel.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address sparabel.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [12:01:50] PROBLEM - test131 Puppet on test131 is UNKNOWN: NRPE: Unable to read output [12:07:49] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.79, 4.03, 3.29 [12:27:50] RECOVERY - test131 Puppet on test131 is OK: OK: Puppet is currently enabled, last run 16 seconds ago with 0 failures [12:34:47] PROBLEM - mw133 Puppet on mw133 is UNKNOWN: NRPE: Unable to read output [12:40:54] !log [@mwtask141] starting deploy of {'config': True} to all [12:40:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:41:03] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 9s [12:41:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:44:59] PROBLEM - bast121 Current Load on bast121 is CRITICAL: LOAD CRITICAL - total load average: 5.67, 2.65, 1.08 [12:45:02] PROBLEM - mw131 Puppet on mw131 is UNKNOWN: NRPE: Unable to read output [12:59:21] PROBLEM - bast121 Current Load on bast121 is WARNING: LOAD WARNING - total load average: 0.00, 1.04, 1.88 [13:00:46] RECOVERY - mw133 Puppet on mw133 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:01:19] RECOVERY - bast121 Current Load on bast121 is OK: LOAD OK - total load average: 0.07, 0.72, 1.66 [13:11:02] RECOVERY - mw131 Puppet on mw131 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:17:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.94, 3.36, 3.97 [13:27:14] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.12, 3.48, 3.68 [13:29:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.45, 3.40, 3.62 [13:41:14] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 1.49, 2.73, 3.26 [14:28:51] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.08416175842 secs [14:32:50] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.1393624842 secs [15:08:08] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.54, 3.22, 2.62 [15:10:04] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.05, 3.18, 2.68 [15:25:35] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 4.00, 3.04, 2.77 [15:31:23] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.27, 3.25, 2.96 [15:35:17] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.73, 3.93, 3.30 [15:37:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.99, 3.89, 3.36 [15:41:14] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.47, 3.14, 3.18 [15:53:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.55, 3.18, 3.04 [16:09:14] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.24, 3.81, 3.48 [16:13:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.65, 3.79, 3.56 [16:19:32] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.56, 3.17, 2.49 [16:21:27] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.11, 3.19, 2.58 [16:23:14] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.25, 3.74, 3.59 [16:25:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.74, 3.66, 3.58 [16:29:14] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.16, 3.80, 3.64 [16:31:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.88, 3.79, 3.66 [16:39:14] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.15, 3.70, 3.63 [16:41:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.54, 3.59, 3.60 [16:42:49] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.97, 3.40, 3.04 [16:44:49] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.79, 3.17, 3.01 [17:05:14] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.69, 3.08, 3.32 [17:11:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.71, 3.54, 3.45 [17:13:14] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.11, 3.29, 3.37 [17:27:16] PROBLEM - cp33 Current Load on cp33 is WARNING: LOAD WARNING - total load average: 1.30, 3.58, 2.31 [17:29:16] RECOVERY - cp33 Current Load on cp33 is OK: LOAD OK - total load average: 0.84, 2.66, 2.13 [17:46:50] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.0995002985 secs [18:09:14] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 5.50, 4.02, 2.96 [18:43:16] PROBLEM - cp33 Current Load on cp33 is CRITICAL: LOAD CRITICAL - total load average: 2.39, 4.15, 2.53 [18:45:16] RECOVERY - cp33 Current Load on cp33 is OK: LOAD OK - total load average: 1.34, 3.16, 2.36 [19:01:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 1.61, 3.15, 3.75 [19:05:14] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.07, 2.60, 3.39 [19:57:14] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.10, 3.67, 3.05 [19:59:14] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.98, 3.80, 3.18 [20:17:14] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.76, 3.24, 3.32 [20:46:50] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.2088076174 secs [21:57:34] PROBLEM - cp22 NTP time on cp22 is CRITICAL: NTP CRITICAL: Offset 0.5001153648 secs [22:07:37] PROBLEM - cp32 Current Load on cp32 is WARNING: LOAD WARNING - total load average: 3.58, 2.48, 1.43 [22:09:37] PROBLEM - cp32 Current Load on cp32 is CRITICAL: LOAD CRITICAL - total load average: 4.37, 3.58, 1.99 [22:11:37] RECOVERY - cp32 Current Load on cp32 is OK: LOAD OK - total load average: 0.95, 2.53, 1.80 [22:11:50] !log [void@phab121] bin/remove destroy @qbsenterprisesupport @thomasl -- Spam only accounts [22:11:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:42:29] [02miraheze/dns] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/dns/commit/47fb339a55fe [22:42:32] [02miraheze/dns] 07paladox 0347fb339 - Add cp24/25/34/35 to dns [22:42:35] [02dns] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/dns [22:42:38] [02dns] 07paladox opened pull request 03#436: Add cp24/25/34/35 to dns - 13https://github.com/miraheze/dns/pull/436 [22:42:40] [02dns] 07paladox closed pull request 03#436: Add cp24/25/34/35 to dns - 13https://github.com/miraheze/dns/pull/436 [22:42:43] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/6bef3b51834b...6c2ad488ef55 [22:42:44] [02miraheze/dns] 07paladox 036c2ad48 - Add cp24/25/34/35 to dns (#436) [22:42:45] [02miraheze/dns] 07paladox deleted branch 03paladox-patch-1 [22:42:47] [02dns] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/dns [22:51:50] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+1/-0/±0] 13https://github.com/miraheze/puppet/commit/7f9f74239d3f [22:51:52] [02miraheze/puppet] 07paladox 037f9f742 - Install cp24/25 [22:51:57] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [22:51:59] [02puppet] 07paladox opened pull request 03#3410: Install cp24/25 - 13https://github.com/miraheze/puppet/pull/3410 [22:52:07] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+1/-0/±0] 13https://github.com/miraheze/puppet/compare/7f9f74239d3f...2db98439a576 [22:52:10] [02miraheze/puppet] 07paladox 032db9843 - Create cp25.yaml [22:52:13] [02puppet] 07paladox synchronize pull request 03#3410: Install cp24/25 - 13https://github.com/miraheze/puppet/pull/3410 [22:52:26] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/2db98439a576...bb52f4b4a01b [22:52:27] [02miraheze/puppet] 07paladox 03bb52f4b - Update site.pp [22:52:29] [02puppet] 07paladox synchronize pull request 03#3410: Install cp24/25 - 13https://github.com/miraheze/puppet/pull/3410 [22:52:33] [02puppet] 07paladox closed pull request 03#3410: Install cp24/25 - 13https://github.com/miraheze/puppet/pull/3410 [22:52:35] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+2/-0/±1] 13https://github.com/miraheze/puppet/compare/3db94a542206...5fd91851ad15 [22:52:38] [02miraheze/puppet] 07paladox 035fd9185 - Install cp24/25 (#3410) [22:52:39] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [22:52:41] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [22:54:27] PROBLEM - cp24 Puppet on cp24 is UNKNOWN: NRPE: Unable to read output [22:54:57] PROBLEM - cp25 Puppet on cp25 is UNKNOWN: NRPE: Unable to read output [22:55:23] PROBLEM - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is CRITICAL: CRITICAL - NGINX Error Rate is 100% [22:55:30] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/4246f4ae3c61...cdd3a6edc78a [22:55:33] [02miraheze/mw-config] 07paladox 03cdd3a6e - Add cp24/25 to wgCdnServers [22:55:46] PROBLEM - cp24 HTTPS on cp24 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - 8191 bytes in 0.016 second response time [22:56:37] miraheze/mw-config - paladox the build passed. [22:56:43] !log [@test131] starting deploy of {'config': True} to all [22:56:44] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [22:56:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:56:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:56:55] RECOVERY - cp25 Puppet on cp25 is OK: OK: Puppet is currently enabled, last run 47 seconds ago with 0 failures [22:57:04] [02miraheze/dns] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/dns/commit/1d890b4d4945 [22:57:05] [02miraheze/dns] 07paladox 031d890b4 - Use cp24/25 for gb [22:57:07] [02dns] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/dns [22:57:09] [02dns] 07paladox opened pull request 03#437: Use cp24/25 for gb - 13https://github.com/miraheze/dns/pull/437 [22:57:42] [02dns] 07paladox closed pull request 03#437: Use cp24/25 for gb - 13https://github.com/miraheze/dns/pull/437 [22:57:43] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/6c2ad488ef55...ddc963867e62 [22:57:45] [02miraheze/dns] 07paladox 03ddc9638 - Use cp24/25 for gb (#437) [22:57:46] [02miraheze/dns] 07paladox deleted branch 03paladox-patch-1 [22:57:48] [02dns] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/dns [22:58:12] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [22:58:16] !log [paladox@mwtask141] DEPLOY ABORTED: Canary check failed for publictestwiki.com@mw131.miraheze.org [22:58:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:58:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:58:26] PROBLEM - cp24 Varnish Backends on cp24 is CRITICAL: 8 backends are down. mw131 mw132 mw141 mw142 mw133 mw134 mw143 mediawiki [23:00:34] PROBLEM - www.thesimswiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.thesimswiki.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:00:34] PROBLEM - wiki.tmyt105.leyhp.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.tmyt105.leyhp.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:00:46] PROBLEM - wiki.exputra.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.exputra.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:01:00] PROBLEM - ecole.science - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - ecole.science reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:01:04] PROBLEM - wiki.globasa.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.globasa.net reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:01:06] PROBLEM - www.project-patterns.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.project-patterns.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:01:07] PROBLEM - apeirology.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - apeirology.com reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:01:07] PROBLEM - wonderfuleveryday.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wonderfuleveryday.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:01:13] PROBLEM - ns1 SSH on ns1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:01:18] RECOVERY - cp24 HTTP 4xx/5xx ERROR Rate on cp24 is OK: OK - NGINX Error Rate is 6% [23:01:19] PROBLEM - nl.xliving.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - nl.xliving.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:01:21] PROBLEM - wiki.valkyrienskies.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.valkyrienskies.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:01:23] PROBLEM - wiki.aoemods.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.aoemods.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:01:26] PROBLEM - www.portalsofphereon.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.portalsofphereon.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:01:30] PROBLEM - ncuwiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - ncuwiki.com reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:01:37] RECOVERY - cp24 HTTPS on cp24 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3733 bytes in 0.019 second response time [23:01:41] PROBLEM - vise.dayid.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - vise.dayid.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:01:45] PROBLEM - project-patterns.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - project-patterns.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:01:49] PROBLEM - wiki.gesamtschule-nordkirchen.de - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gesamtschule-nordkirchen.de reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:02:02] PROBLEM - wiki.newnormal.fans - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.newnormal.fans reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:02:02] PROBLEM - squareenix.miraheze.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - squareenix.miraheze.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:02:09] PROBLEM - wiki.misslessfighters.club - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.misslessfighters.club reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:02:19] RECOVERY - cp24 Varnish Backends on cp24 is OK: All 15 backends are healthy [23:02:24] PROBLEM - grayravens.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - grayravens.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:02:26] PROBLEM - petrawiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - petrawiki.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:02:28] PROBLEM - worldtriggerwiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - worldtriggerwiki.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:02:29] PROBLEM - en.petrawiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.petrawiki.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:02:36] PROBLEM - kalons-reverie.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - kalons-reverie.com reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:02:43] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.zulaclub.net reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:03:15] PROBLEM - sona.pona.la - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - sona.pona.la reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:03:33] PROBLEM - heavyironmodding.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - heavyironmodding.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:03:35] PROBLEM - www.glitchcity.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.glitchcity.wiki reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:03:35] PROBLEM - wiki.theluxuryelevator.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.theluxuryelevator.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:03:44] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 13.21, 7.19, 3.28 [23:03:48] PROBLEM - www.dovearchives.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.dovearchives.wiki reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:03:54] PROBLEM - inourownwords.online - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - inourownwords.online reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:04:01] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [23:04:03] PROBLEM - ns1 ferm_active on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [23:04:04] PROBLEM - commonswiki.miraheze.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - commonswiki.miraheze.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:04:04] PROBLEM - speleo.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - speleo.wiki reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:04:09] PROBLEM - webkinzguide.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - webkinzguide.com reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:04:21] PROBLEM - wiki.coolstation.space - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.coolstation.space reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:04:23] PROBLEM - wiki.jsal.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.jsal.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:04:24] PROBLEM - ns1 Current Load on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [23:04:46] PROBLEM - wiki.zymonic.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.zymonic.com reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:04:51] PROBLEM - wiki.scvo.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.scvo.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:05:00] PROBLEM - wiki.worlduniversityandschool.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.worlduniversityandschool.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:05:08] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [23:05:10] PROBLEM - wiki.puucraft.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.puucraft.net reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:05:13] PROBLEM - www.programming.red - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.programming.red reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:05:19] PROBLEM - wiki.limaru.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.limaru.net reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:05:26] PROBLEM - pubwiki.lab.co.pl - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - pubwiki.lab.co.pl reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:05:28] PROBLEM - wiki.funkey-project.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.funkey-project.com reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:05:28] PROBLEM - wiki.seredrau.xyz - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.seredrau.xyz reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:05:28] PROBLEM - wiki.rosestulipsandliberty.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.rosestulipsandliberty.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:05:29] PROBLEM - wiki.joust.ro - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.joust.ro reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:05:29] PROBLEM - wiki.autocountsoft.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.autocountsoft.com reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:05:32] PROBLEM - guia.cineastas.pt - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - guia.cineastas.pt reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:05:46] PROBLEM - wiki.altilunium.my.id - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.altilunium.my.id reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:05:46] PROBLEM - thenintendowiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - thenintendowiki.com reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:05:57] PROBLEM - wiki.meregos.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.meregos.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:06:03] PROBLEM - wiki.climatechange.ai - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.climatechange.ai reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:06:10] PROBLEM - wiki.anempireofdreams.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.anempireofdreams.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:06:28] PROBLEM - mariopedia.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - mariopedia.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:06:31] PROBLEM - theagonian.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - theagonian.com reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:06:33] PROBLEM - fanonpedia.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - fanonpedia.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:06:36] PROBLEM - wiki.qadrishattari.xyz - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.qadrishattari.xyz reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:06:39] PROBLEM - ethene.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - ethene.wiki reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:07:01] PROBLEM - thepolicyhub.org.uk - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - thepolicyhub.org.uk reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:07:14] PROBLEM - taswin.miraheze.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - taswin.miraheze.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:07:15] PROBLEM - electowiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - electowiki.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:07:18] PROBLEM - crocwiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - crocwiki.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:07:36] PROBLEM - 321nails.crpteam.club - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - 321nails.crpteam.club reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:07:41] PROBLEM - echoes-wiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - echoes-wiki.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:07:45] PROBLEM - data.wikiyri.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - data.wikiyri.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:08:00] PROBLEM - portalsofphereon.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - portalsofphereon.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:08:01] PROBLEM - monarchists.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - monarchists.wiki reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:08:02] PROBLEM - evilgeniuswiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - evilgeniuswiki.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:08:03] PROBLEM - wikiru.wildterra2.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wikiru.wildterra2.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:08:04] PROBLEM - www.lgbtqia.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.lgbtqia.wiki reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:08:04] PROBLEM - wiki.mxlinuxusers.de - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mxlinuxusers.de reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:08:09] RECOVERY - cp24 Puppet on cp24 is OK: OK: Puppet is currently enabled, last run 17 seconds ago with 0 failures [23:08:09] PROBLEM - ns1 conntrack_table_size on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [23:08:16] PROBLEM - wiki.macc.nyc - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.macc.nyc reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:08:17] PROBLEM - wiki.make717.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.make717.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:08:26] PROBLEM - ndgkb.nena.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - ndgkb.nena.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:08:29] PROBLEM - wiki.geoparkcorumbatai.com.br - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.geoparkcorumbatai.com.br reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:08:37] PROBLEM - abeyancewiki.bumblebee-studios.se - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - abeyancewiki.bumblebee-studios.se reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:08:40] PROBLEM - dragonquest2.miraheze.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dragonquest2.miraheze.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:08:41] PROBLEM - yoshi.miraheze.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - yoshi.miraheze.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:08:45] PROBLEM - wiki.fediverse.kr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.fediverse.kr reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:08:48] PROBLEM - dragonquestwiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dragonquestwiki.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:08:49] PROBLEM - metroidpedia.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - metroidpedia.com reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:08:57] PROBLEM - wiki.counterculturelabs.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.counterculturelabs.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:09:14] PROBLEM - o.kaoyu666.cn - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - o.kaoyu666.cn reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:09:14] PROBLEM - rare.miraheze.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - rare.miraheze.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:09:22] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - lcn.zfc.id.lv reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:09:27] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 1.73, 6.80, 4.73 [23:09:39] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/cdd3a6edc78a...ae3ca33565be [23:09:39] PROBLEM - files.petrawiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - files.petrawiki.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:09:40] PROBLEM - icclopedia.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - icclopedia.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:09:41] PROBLEM - wiki.minkyu.kim - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.minkyu.kim reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:09:42] [02miraheze/mw-config] 07paladox 03ae3ca33 - Remove cp22/23 from wgCdnServers [23:09:43] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [23:09:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:09:57] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 13s [23:09:59] PROBLEM - wiki.nevillepedia.eu - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nevillepedia.eu reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:10:03] !log [paladox@mwtask141] starting deploy of {'config': True} to all [23:10:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:10:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:10:11] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 7s [23:10:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:10:24] PROBLEM - ns1 Disk Space on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [23:10:40] miraheze/mw-config - paladox the build passed. [23:11:04] PROBLEM - wiki.mastodon.kr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mastodon.kr reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:11:14] PROBLEM - allthetropes.miraheze.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - allthetropes.miraheze.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:11:15] PROBLEM - wiki.pulsus.cc - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.pulsus.cc reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:11:35] PROBLEM - es.countryhumans.polandball.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - es.countryhumans.polandball.wiki reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:11:36] PROBLEM - wiki.queenscourt.games - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.queenscourt.games reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:11:37] PROBLEM - wiki.beergeeks.co.il - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.beergeeks.co.il reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:11:46] PROBLEM - wiki.litek.top - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.litek.top reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:11:47] PROBLEM - www.winenjoy.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.winenjoy.net reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:11:58] PROBLEM - wiki.ff6worldscollide.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.ff6worldscollide.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:11:59] PROBLEM - wiki.fatcatfablab.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.fatcatfablab.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:12:11] PROBLEM - landofliberos.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - landofliberos.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:12:17] PROBLEM - wiki.bikinibottomdiscord.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.bikinibottomdiscord.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:12:18] PROBLEM - nijigen.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - nijigen.wiki reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:12:19] PROBLEM - wiki.aetherexplorers.science - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.aetherexplorers.science reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:12:22] PROBLEM - recaptime.miraheze.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - recaptime.miraheze.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:12:24] PROBLEM - mh142.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - mh142.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:12:29] PROBLEM - wiki.dwarvesrp.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.dwarvesrp.com reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:12:34] PROBLEM - telepedia-wiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - telepedia-wiki.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:12:41] PROBLEM - wiki.ssij.se - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.ssij.se reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:12:45] PROBLEM - steamdecklinux.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - steamdecklinux.wiki reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:12:54] PROBLEM - podpedia.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - podpedia.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:12:55] PROBLEM - hastursnotebook.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - hastursnotebook.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:12:57] PROBLEM - removededm.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - removededm.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:13:08] PROBLEM - www.istpcomputing.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.istpcomputing.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:13:36] PROBLEM - wmworld.sktz.live - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wmworld.sktz.live reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:13:41] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:13:46] PROBLEM - test2.specialchat.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - test2.specialchat.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:14:11] PROBLEM - rarewarewiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - rarewarewiki.com reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:14:12] PROBLEM - pwiki.drydraytonvillagehall.org.uk - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - pwiki.drydraytonvillagehall.org.uk reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:14:33] PROBLEM - pornwiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - pornwiki.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:14:38] PROBLEM - fanpedia.miraheze.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - fanpedia.miraheze.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:14:51] PROBLEM - finalfantasy.miraheze.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - finalfantasy.miraheze.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:14:53] PROBLEM - isv.orain.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - isv.orain.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:15:03] PROBLEM - index.pdcommunity.ir - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - index.pdcommunity.ir reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:15:06] PROBLEM - txt-dex.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - txt-dex.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:15:07] PROBLEM - kirisame-kissaten.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - kirisame-kissaten.cf reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:15:15] PROBLEM - wiki.luemir.xyz - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.luemir.xyz reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:15:17] PROBLEM - sims.miraheze.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - sims.miraheze.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:15:20] PROBLEM - elsterclopedie.marithaime.nl - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - elsterclopedie.marithaime.nl reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:15:30] PROBLEM - wiki.bloxotel.tk - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.bloxotel.tk reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:15:40] PROBLEM - wikigenius.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wikigenius.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:15:42] PROBLEM - bushcraftwiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - bushcraftwiki.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:15:43] PROBLEM - pt.graalmilitary.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - pt.graalmilitary.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:15:45] PROBLEM - uncyclopediakr.miraheze.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uncyclopediakr.miraheze.org reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:15:47] PROBLEM - aman.awiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - aman.awiki.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:15:50] PROBLEM - wiki.bluevertigo.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.bluevertigo.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:15:58] PROBLEM - wiki.tulpa.info - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.tulpa.info reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:15:58] PROBLEM - dlfm-wiki.top - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dlfm-wiki.top reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:15:59] PROBLEM - royal-wiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - royal-wiki.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:16:00] PROBLEM - grc.repository.archiopedia.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - grc.repository.archiopedia.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:16:03] PROBLEM - guia.esporo.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - guia.esporo.net reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:16:04] PROBLEM - franchise.franchising.org.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - franchise.franchising.org.ua reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:16:04] PROBLEM - itora.bunnies1230.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - itora.bunnies1230.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:16:05] PROBLEM - acgn.sfdev.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - acgn.sfdev.eu.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:16:06] PROBLEM - wiki.alathramc.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.alathramc.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:16:17] PROBLEM - patternarchive.online - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - patternarchive.online reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:16:19] PROBLEM - bluearchive.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - bluearchive.wiki reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:16:24] PROBLEM - psychevos.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - psychevos.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:16:29] PROBLEM - wiki.ooer.ooo - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.ooer.ooo reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:16:33] PROBLEM - allthetropes.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - allthetropes.org reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:16:37] PROBLEM - wiki.closai.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.closai.com reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:16:44] PROBLEM - corru.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - corru.wiki reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:16:46] PROBLEM - smashbroswiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - smashbroswiki.com reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:17:07] PROBLEM - wows.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wows.wiki reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:17:07] PROBLEM - www.fossilfighterswiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.fossilfighterswiki.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:17:22] PROBLEM - zenith.nsmbu.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zenith.nsmbu.net reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:17:23] PROBLEM - wiki.leapit.no - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.leapit.no reverse DNS resolves to vps-a6efa8b4.vps.ovh.net [23:17:23] PROBLEM - elephants.ggreig.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - elephants.ggreig.com reverse DNS resolves to vps-6d4fe34d.vps.ovh.net [23:17:50]