[00:10:17] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.51, 7.34, 6.02 [00:12:16] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.49, 6.70, 5.95 [03:02:34] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 1.07, 2.64, 3.90 [03:06:34] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.08, 3.29, 3.84 [03:11:40] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [03:41:40] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [04:08:34] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.77, 3.77, 4.00 [04:10:34] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 3.89, 3.94, 4.04 [07:11:40] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [07:41:41] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [09:54:37] PROBLEM - db142 Backups SQL on db142 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864233 seconds old and 86709 bytes [11:13:59] [02miraheze/dns] 07paladox pushed 031 commit to 03paladox-patch-1 [+1/-0/±0] 13https://github.com/miraheze/dns/commit/9c0a08ed8f16 [11:14:00] [02miraheze/dns] 07paladox 039c0a08e - Add mirabeta.org to dns [11:14:03] [02dns] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/dns [11:14:04] [02dns] 07paladox opened pull request 03#443: Add mirabeta.org to dns - 13https://github.com/miraheze/dns/pull/443 [11:14:09] [02dns] 07paladox closed pull request 03#443: Add mirabeta.org to dns - 13https://github.com/miraheze/dns/pull/443 [11:14:11] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/dns/compare/4452dab72e78...3cb1286957c1 [11:14:12] [02miraheze/dns] 07paladox 033cb1286 - Add mirabeta.org to dns (#443) [11:14:15] [02miraheze/dns] 07paladox deleted branch 03paladox-patch-1 [11:14:16] [02dns] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/dns [11:18:32] PROBLEM - ns1 SSH on ns1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:20:08] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [11:20:47] PROBLEM - ns1 Current Load on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [11:21:23] PROBLEM - ns1 Disk Space on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [11:21:23] PROBLEM - ns1 conntrack_table_size on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [11:21:42] PROBLEM - ns1 ferm_active on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [11:21:56] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [11:27:35] PROBLEM - ns1 Auth DNS on ns1 is CRITICAL: CRITICAL - Plugin timed out while executing system call [11:27:53] PROBLEM - Host ns1 is DOWN: PING CRITICAL - Packet loss = 100% [11:28:16] ping is up [11:30:34] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 0.87, 2.55, 3.93 [11:31:25] ns1 is v6 only [11:32:34] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.10, 3.26, 4.03 [11:39:37] RECOVERY - Host ns1 is UP: PING OK - Packet loss = 70%, RTA = 103.74 ms [11:39:59] RECOVERY - ns1 Auth DNS on ns1 is OK: DNS OK: 0.235 seconds response time. miraheze.org returns 2001:41d0:801:2000::3a18,2001:41d0:801:2000::5d68,51.195.201.140,51.89.139.24 [11:40:13] RECOVERY - ns1 Current Load on ns1 is OK: LOAD OK - total load average: 0.15, 0.14, 0.14 [11:40:46] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.09547972679 secs [11:40:58] RECOVERY - ns1 ferm_active on ns1 is OK: OK ferm input default policy is set [11:41:12] RECOVERY - ns1 SSH on ns1 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u1 (protocol 2.0) [11:41:15] RECOVERY - ns1 Disk Space on ns1 is OK: DISK OK - free space: / 2863MiB (39% inode=83%); [11:41:15] RECOVERY - ns1 conntrack_table_size on ns1 is OK: OK: nf_conntrack is 0 % full [11:41:29] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [11:43:07] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/1919ef6beab6 [11:43:08] [02miraheze/mw-config] 07paladox 031919ef6 - Switch betaheze to mirabeta [11:43:10] [02mw-config] 07paladox created branch 03paladox-patch-5 - 13https://github.com/miraheze/mw-config [11:43:12] [02mw-config] 07paladox opened pull request 03#5335: Switch betaheze to mirabeta - 13https://github.com/miraheze/mw-config/pull/5335 [11:43:35] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:43:45] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/1919ef6beab6...1c5a0490d07c [11:43:47] [02miraheze/mw-config] 07paladox 031c5a049 - Update MirahezeFunctions.php [11:43:50] [02mw-config] 07paladox synchronize pull request 03#5335: Switch betaheze to mirabeta - 13https://github.com/miraheze/mw-config/pull/5335 [11:44:06] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/1c5a0490d07c...8c356a1433c9 [11:44:08] [02miraheze/mw-config] 07paladox 038c356a1 - Update GlobalCache.php [11:44:09] [02mw-config] 07paladox synchronize pull request 03#5335: Switch betaheze to mirabeta - 13https://github.com/miraheze/mw-config/pull/5335 [11:44:11] miraheze/mw-config - paladox the build passed. [11:44:38] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/8c356a1433c9...95318b616ed5 [11:44:40] [02miraheze/mw-config] 07paladox 0395318b6 - Update LocalWiki.php [11:44:41] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.1080584526 secs [11:44:42] [02mw-config] 07paladox synchronize pull request 03#5335: Switch betaheze to mirabeta - 13https://github.com/miraheze/mw-config/pull/5335 [11:45:00] miraheze/mw-config - paladox the build passed. [11:45:03] [02mw-config] 07paladox closed pull request 03#5335: Switch betaheze to mirabeta - 13https://github.com/miraheze/mw-config/pull/5335 [11:45:05] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±4] 13https://github.com/miraheze/mw-config/compare/e1865070881c...c852eed80e4f [11:45:06] [02miraheze/mw-config] 07paladox 03c852eed - Switch betaheze to mirabeta (#5335) [11:45:06] miraheze/mw-config - paladox the build passed. [11:45:09] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-5 [11:45:12] [02mw-config] 07paladox deleted branch 03paladox-patch-5 - 13https://github.com/miraheze/mw-config [11:45:43] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-13 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/377c433219c0 [11:45:46] miraheze/mw-config - paladox the build passed. [11:45:46] [02miraheze/puppet] 07paladox 03377c433 - Switch betaheze to mirabeta [11:45:49] [02puppet] 07paladox created branch 03paladox-patch-13 - 13https://github.com/miraheze/puppet [11:45:52] [02puppet] 07paladox opened pull request 03#3449: Switch betaheze to mirabeta - 13https://github.com/miraheze/puppet/pull/3449 [11:46:10] miraheze/mw-config - paladox the build passed. [11:46:16] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-13 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/377c433219c0...60dc5e73d62e [11:46:17] [02miraheze/puppet] 07paladox 0360dc5e7 - Update test131.yaml [11:46:19] [02puppet] 07paladox synchronize pull request 03#3449: Switch betaheze to mirabeta - 13https://github.com/miraheze/puppet/pull/3449 [11:46:38] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-13 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/60dc5e73d62e...84b1b40e587f [11:46:39] [02miraheze/puppet] 07paladox 0384b1b40 - Update deploy-mediawiki.py [11:46:40] [02puppet] 07paladox synchronize pull request 03#3449: Switch betaheze to mirabeta - 13https://github.com/miraheze/puppet/pull/3449 [11:46:54] [02puppet] 07paladox closed pull request 03#3449: Switch betaheze to mirabeta - 13https://github.com/miraheze/puppet/pull/3449 [11:46:57] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/puppet/compare/bd5d9236802c...62403d7321b8 [11:46:58] [02miraheze/puppet] 07paladox 0362403d7 - Switch betaheze to mirabeta (#3449) [11:47:01] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-13 [11:47:02] [02puppet] 07paladox deleted branch 03paladox-patch-13 - 13https://github.com/miraheze/puppet [11:47:07] [02puppet] 07paladox closed pull request 03#3446: varnish: Fix mobile - 13https://github.com/miraheze/puppet/pull/3446 [11:47:09] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-12 [11:47:10] [02puppet] 07paladox deleted branch 03paladox-patch-12 - 13https://github.com/miraheze/puppet [11:47:14] [02puppet] 07paladox closed pull request 03#3445: varnish: Remove mobile logic - 13https://github.com/miraheze/puppet/pull/3445 [11:47:16] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [11:47:19] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [11:50:02] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/3cb1286957c1...16315da48451 [11:50:03] [02miraheze/dns] 07paladox 0316315da - Add acme challenge to mirabeta.org [11:53:38] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ssl/commit/cdbd82016f22 [11:53:40] [02miraheze/ssl] 07paladox 03cdbd820 - Add mirabeta.org and remove betaheze.org [11:53:43] [02ssl] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/ssl [11:53:44] [02ssl] 07paladox opened pull request 03#730: Add mirabeta.org and remove betaheze.org - 13https://github.com/miraheze/ssl/pull/730 [11:54:02] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-1/±0] 13https://github.com/miraheze/ssl/compare/cdbd82016f22...06614a8c9d44 [11:54:03] [02miraheze/ssl] 07paladox 0306614a8 - Delete certificates/betaheze.org.crt [11:54:06] [02ssl] 07paladox synchronize pull request 03#730: Add mirabeta.org and remove betaheze.org - 13https://github.com/miraheze/ssl/pull/730 [11:55:05] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+1/-0/±0] 13https://github.com/miraheze/ssl/compare/06614a8c9d44...9e6c24b52ea9 [11:55:07] [02miraheze/ssl] 07paladox 039e6c24b - Create mirabeta.org.crt [11:55:10] [02ssl] 07paladox synchronize pull request 03#730: Add mirabeta.org and remove betaheze.org - 13https://github.com/miraheze/ssl/pull/730 [11:55:27] [02ssl] 07paladox closed pull request 03#730: Add mirabeta.org and remove betaheze.org - 13https://github.com/miraheze/ssl/pull/730 [11:55:29] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+1/-1/±1] 13https://github.com/miraheze/ssl/compare/dad549699d43...253f4cf91ff7 [11:55:31] [02miraheze/ssl] 07paladox 03253f4cf - Add mirabeta.org and remove betaheze.org (#730) [11:55:33] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-1 [11:55:36] [02ssl] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/ssl [11:55:58] [02miraheze/dns] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-1/±0] 13https://github.com/miraheze/dns/commit/6e7bbebc2c87 [11:56:01] [02miraheze/dns] 07paladox 036e7bbeb - Remove betaheze.org from dns [11:56:04] [02dns] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/dns [11:56:07] [02dns] 07paladox opened pull request 03#444: Remove betaheze.org from dns - 13https://github.com/miraheze/dns/pull/444 [11:56:12] [02dns] 07paladox closed pull request 03#444: Remove betaheze.org from dns - 13https://github.com/miraheze/dns/pull/444 [11:56:15] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://github.com/miraheze/dns/compare/16315da48451...438d1fad8080 [11:56:18] [02miraheze/dns] 07paladox 03438d1fa - Remove betaheze.org from dns (#444) [11:56:19] [02miraheze/dns] 07paladox deleted branch 03paladox-patch-1 [11:56:22] [02dns] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/dns [11:56:49] !log [@test131] starting deploy of {'config': True} to all [11:56:50] !log [@test131] DEPLOY ABORTED: Canary check failed for meta.mirabeta.org@localhost [11:56:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:57:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:58:17] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/00994cb67b8e [11:58:18] [02miraheze/puppet] 07paladox 0300994cb - Switch betaheze to mirabeta [11:58:19] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [11:58:22] [02puppet] 07paladox opened pull request 03#3450: Switch betaheze to mirabeta - 13https://github.com/miraheze/puppet/pull/3450 [11:58:31] [02puppet] 07paladox closed pull request 03#3450: Switch betaheze to mirabeta - 13https://github.com/miraheze/puppet/pull/3450 [11:58:33] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/62403d7321b8...95557eeaeb47 [11:58:35] [02miraheze/puppet] 07paladox 0395557ee - Switch betaheze to mirabeta (#3450) [11:58:37] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [11:58:39] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [11:58:56] !log [@mwtask141] starting deploy of {'config': True} to all [11:58:57] PROBLEM - test131 MediaWiki Rendering on test131 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 322 bytes in 0.023 second response time [11:59:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:59:07] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 11s [11:59:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:07:26] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/962ddf02e35b [12:07:27] [02miraheze/mw-config] 07paladox 03962ddf0 - betaheze -> mirabeta [12:07:29] [02mw-config] 07paladox created branch 03paladox-patch-5 - 13https://github.com/miraheze/mw-config [12:07:32] [02mw-config] 07paladox opened pull request 03#5336: betaheze -> mirabeta - 13https://github.com/miraheze/mw-config/pull/5336 [12:07:42] [02mw-config] 07paladox closed pull request 03#5336: betaheze -> mirabeta - 13https://github.com/miraheze/mw-config/pull/5336 [12:07:44] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/c852eed80e4f...d018cd7f21f7 [12:07:46] [02miraheze/mw-config] 07paladox 03d018cd7 - betaheze -> mirabeta (#5336) [12:07:47] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-5 [12:07:48] [02mw-config] 07paladox deleted branch 03paladox-patch-5 - 13https://github.com/miraheze/mw-config [12:08:04] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [12:08:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:08:13] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 9s [12:08:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:08:19] !log [paladox@mwtask141] starting deploy of {'config': True} to all [12:08:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:08:27] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 7s [12:08:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:08:38] miraheze/mw-config - paladox the build passed. [12:08:45] miraheze/mw-config - paladox the build passed. [12:08:49] !log [paladox@test131] starting deploy of {'pull': 'config', 'config': True} to all [12:08:50] !log [paladox@test131] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 1s [12:08:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:08:56] !log [paladox@test131] starting deploy of {'config': True} to all [12:08:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:08:57] !log [paladox@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [12:08:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:09:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:24:59] RECOVERY - test131 MediaWiki Rendering on test131 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 1.786 second response time [13:14:41] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.andreijiroh.uk.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [13:44:08] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [13:47:21] PROBLEM - ns1 NTP time on ns1 is CRITICAL: NTP CRITICAL: Offset 0.5013795793 secs [15:41:29] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/5c0b0b41481f [15:41:31] [02miraheze/puppet] 07paladox 035c0b0b4 - Varnish: clear req.http.X-Subdomain on each request [15:41:33] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [15:41:34] [02puppet] 07paladox opened pull request 03#3451: Varnish: clear req.http.X-Subdomain on each request - 13https://github.com/miraheze/puppet/pull/3451 [15:42:21] [02puppet] 07paladox synchronize pull request 03#3451: Varnish: clear req.http.X-Subdomain on each request - 13https://github.com/miraheze/puppet/pull/3451 [15:42:22] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/5c0b0b41481f...2c1c7178590b [15:42:24] [02miraheze/puppet] 07paladox 032c1c717 - Update default.vcl [15:42:55] [02puppet] 07paladox synchronize pull request 03#3451: Varnish: clear req.http.X-Subdomain on each request - 13https://github.com/miraheze/puppet/pull/3451 [15:42:57] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/2c1c7178590b...c33e555e57f9 [15:43:00] [02miraheze/puppet] 07paladox 03c33e555 - Update default.vcl [15:44:14] [02puppet] 07paladox closed pull request 03#3451: Varnish: clear req.http.X-Subdomain on each request - 13https://github.com/miraheze/puppet/pull/3451 [15:44:15] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/95557eeaeb47...98172e44fa67 [15:44:17] [02miraheze/puppet] 07paladox 0398172e4 - Varnish: clear req.http.X-Subdomain on each request (#3451) [15:44:18] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [15:44:20] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [15:45:35] PROBLEM - www.vanillaanarchy.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.vanillaanarchy.net' expires in 15 day(s) (Thu 02 Nov 2023 15:36:10 GMT +0000). [16:19:59] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [16:23:59] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [16:27:59] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [16:31:59] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [17:24:53] PROBLEM - wiki.theluxuryelevator.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.theluxuryelevator.com' expires in 15 day(s) (Thu 02 Nov 2023 17:20:58 GMT +0000). [17:27:32] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/253f4cf91ff7...ecc344634176 [17:27:34] [02miraheze/ssl] 07MirahezeSSLBot 03ecc3446 - Bot: Update SSL cert for wiki.theluxuryelevator.com [17:53:46] RECOVERY - wiki.theluxuryelevator.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.theluxuryelevator.com' will expire on Mon 15 Jan 2024 16:27:24 GMT +0000. [17:59:54] PROBLEM - eurocom.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'eurocom.wiki' expires in 15 day(s) (Thu 02 Nov 2023 17:29:14 GMT +0000). [18:06:30] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ecc344634176...125ad5f1d6c5 [18:06:33] [02miraheze/ssl] 07MirahezeSSLBot 03125ad5f - Bot: Update SSL cert for eurocom.wiki [18:58:17] RECOVERY - eurocom.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'eurocom.wiki' will expire on Mon 15 Jan 2024 17:06:22 GMT +0000. [20:27:18] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 0.98, 2.48, 4.00 [20:29:16] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.19, 3.05, 4.02 [20:31:16] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.98, 3.34, 4.00 [20:33:16] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 5.62, 3.85, 4.09 [21:30:21] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [21:34:18] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [22:24:52] PROBLEM - fallofsanctuary.com - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query fallofsanctuary.com. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [23:02:21] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 14.83, 7.67, 4.50 [23:14:21] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 2.53, 7.34, 7.41 [23:16:21] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 1.76, 5.41, 6.69 [23:31:53] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.91, 6.21, 5.18 [23:33:53] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.93, 6.43, 5.39 [23:35:53] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.57, 6.16, 5.40 [23:52:52] RECOVERY - fallofsanctuary.com - reverse DNS on sslhost is OK: SSL OK - fallofsanctuary.com reverse DNS resolves to cp25.miraheze.org - NS RECORDS OK