[00:00:23] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is UNKNOWN: UNKNOWN - NGINX Error Rate is UNKNOWN [00:01:18] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is UNKNOWN: UNKNOWN - NGINX Error Rate is UNKNOWN [00:02:23] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is CRITICAL: CRITICAL - NGINX Error Rate is 100% [00:03:18] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is WARNING: WARNING - NGINX Error Rate is 50% [00:03:36] RECOVERY - cp41 Disk Space on cp41 is OK: DISK OK - free space: / 19215MiB (20% inode=98%); [00:04:03] RECOVERY - cp36 Disk Space on cp36 is OK: DISK OK - free space: / 21332MiB (24% inode=98%); [00:05:40] RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 21374MiB (24% inode=98%); [00:07:18] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 60% [00:09:18] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is WARNING: WARNING - NGINX Error Rate is 57% [00:10:05] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03thumb_handler-redirect [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/002162eff452...ae0fff51e1eb [00:10:07] [02miraheze/mw-config] 07Universal-Omega 03ae0fff5 - Add getDefaultServer [00:10:10] [02mw-config] 07Universal-Omega synchronize pull request 03#5531: thumb_handler: redirect default server to primary domain - 13https://github.com/miraheze/mw-config/pull/5531 [00:11:06] miraheze/mw-config - Universal-Omega the build passed. [00:11:18] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 67% [05:32:53] PROBLEM - chobots.wiki - LetsEncrypt on sslhost is CRITICAL: Temporary failure in name resolutionHTTP CRITICAL - Unable to open TCP socket [05:32:53] PROBLEM - os162 NTP time on os162 is WARNING: NTP WARNING: Offset -0.1372536123 secs [05:32:53] PROBLEM - ru.countryhumans.polandball.wiki - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.405 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [05:32:53] PROBLEM - worldsanskrit.net - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [05:32:53] PROBLEM - swiftproxy171 NTP time on swiftproxy171 is WARNING: NTP WARNING: Offset -0.1381970048 secs [05:32:54] PROBLEM - wiki.corgicam.tv - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [05:32:54] PROBLEM - cp51 HTTPS on cp51 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Connection timed out after 10002 milliseconds [05:33:30] PROBLEM - bots171 NTP time on bots171 is CRITICAL: NTP CRITICAL: No response from NTP server [05:33:30] PROBLEM - wiki.thelastdimension.net - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [05:33:30] PROBLEM - prometheus151 NTP time on prometheus151 is CRITICAL: NTP CRITICAL: No response from NTP server [05:33:30] PROBLEM - mem161 NTP time on mem161 is WARNING: NTP WARNING: Offset -0.134616375 secs [05:33:30] PROBLEM - os162 NTP time on os162 is CRITICAL: NTP CRITICAL: No response from NTP server [05:33:31] PROBLEM - wiki.ate42.ru - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.406 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [05:33:31] PROBLEM - mwtask181 NTP time on mwtask181 is WARNING: NTP WARNING: Offset -0.1202816963 secs [05:33:32] PROBLEM - swiftac171 NTP time on swiftac171 is WARNING: NTP WARNING: Offset -0.1180346608 secs [05:33:32] PROBLEM - swiftobject171 NTP time on swiftobject171 is WARNING: NTP WARNING: Offset -0.1330754757 secs [05:33:33] PROBLEM - wiki.insideearth.info - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [05:33:33] PROBLEM - trollpasta.com - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [05:33:34] PROBLEM - wiki.seredrau.xyz - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.405 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [05:33:41] PROBLEM - cp41 HTTPS on cp41 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Connection timed out after 10003 milliseconds [05:33:41] PROBLEM - wiki.s23.org - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [05:33:41] PROBLEM - housing.wiki - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.405 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [05:33:41] PROBLEM - mw182 NTP time on mw182 is WARNING: NTP WARNING: Offset -0.1333507299 secs [05:33:41] PROBLEM - jobchron171 NTP time on jobchron171 is CRITICAL: NTP CRITICAL: No response from NTP server [05:36:04] PROBLEM - mw152 NTP time on mw152 is WARNING: NTP WARNING: Offset -0.1539905071 secs [05:36:04] PROBLEM - mw181 NTP time on mw181 is UNKNOWN: check_ntp_time: Invalid hostname/address - time.cloudflare.comUsage: check_ntp_time -H [-4|-6] [-w ] [-c ] [-v verbose] [-o ] [05:36:04] PROBLEM - mem151 NTP time on mem151 is WARNING: NTP WARNING: Offset -0.1266858578 secs [05:36:04] PROBLEM - db151 NTP time on db151 is WARNING: NTP WARNING: Offset -0.1002260447 secs [05:36:04] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 70%, RTA = 277.00 ms [05:37:18] RECOVERY - bots171 NTP time on bots171 is OK: NTP OK: Offset -0.001085877419 secs [05:37:27] RECOVERY - mem161 NTP time on mem161 is OK: NTP OK: Offset 0.001413315535 secs [05:37:30] RECOVERY - cp41 Nginx Backend for reports171 on cp41 is OK: TCP OK - 0.000 second response time on localhost port 8205 [05:37:31] RECOVERY - os162 NTP time on os162 is OK: NTP OK: Offset 0.001642763615 secs [05:37:32] RECOVERY - prometheus151 NTP time on prometheus151 is OK: NTP OK: Offset -0.005843102932 secs [05:37:33] RECOVERY - cp41 Varnish Backends on cp41 is OK: All 19 backends are healthy [05:37:33] RECOVERY - cp51 Nginx Backend for mon181 on cp51 is OK: TCP OK - 0.002 second response time on localhost port 8201 [05:37:35] RECOVERY - cp41 Puppet on cp41 is OK: OK: Puppet is currently enabled, last run 57 seconds ago with 0 failures [05:37:36] RECOVERY - mwtask181 NTP time on mwtask181 is OK: NTP OK: Offset 0.0003792345524 secs [05:37:36] RECOVERY - swiftac171 NTP time on swiftac171 is OK: NTP OK: Offset 0.001512050629 secs [05:37:38] RECOVERY - jobchron171 NTP time on jobchron171 is OK: NTP OK: Offset 0.0004255771637 secs [05:37:39] RECOVERY - swiftobject171 NTP time on swiftobject171 is OK: NTP OK: Offset 0.0003981590271 secs [05:37:40] RECOVERY - mw182 NTP time on mw182 is OK: NTP OK: Offset 0.00048828125 secs [05:37:40] RECOVERY - cloud16 NTP time on cloud16 is OK: NTP OK: Offset 0.0007432997227 secs [05:37:41] RECOVERY - cp41 HTTPS on cp41 is OK: HTTP OK: HTTP/2 404 - Status line output matched "HTTP/2 404" - 3637 bytes in 0.805 second response time [05:37:43] RECOVERY - mwtask171 NTP time on mwtask171 is OK: NTP OK: Offset 0.0002282857895 secs [05:37:44] RECOVERY - mw152 NTP time on mw152 is OK: NTP OK: Offset 7.531046867e-05 secs [05:37:46] RECOVERY - cloud15 NTP time on cloud15 is OK: NTP OK: Offset -0.001509159803 secs [05:37:48] RECOVERY - cloud18 NTP time on cloud18 is OK: NTP OK: Offset 0.000639051199 secs [05:37:50] RECOVERY - cp41 SSH on cp41 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [05:37:51] RECOVERY - mw161 NTP time on mw161 is OK: NTP OK: Offset 0.002864211798 secs [05:37:52] RECOVERY - mon181 NTP time on mon181 is OK: NTP OK: Offset 0.0003963410854 secs [05:37:57] RECOVERY - mw181 NTP time on mw181 is OK: NTP OK: Offset 0.0004135370255 secs [05:37:58] RECOVERY - db161 NTP time on db161 is OK: NTP OK: Offset 0.001564860344 secs [05:38:00] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 122.62 ms [05:38:00] RECOVERY - mem151 NTP time on mem151 is OK: NTP OK: Offset 0.001953661442 secs [05:38:00] RECOVERY - db151 NTP time on db151 is OK: NTP OK: Offset 0.001480221748 secs [05:38:09] RECOVERY - cp51 Nginx Backend for mwtask181 on cp51 is OK: TCP OK - 0.000 second response time on localhost port 8160 [05:38:12] RECOVERY - cp51 Varnish Backends on cp51 is OK: All 19 backends are healthy [06:01:15] PROBLEM - chobots.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'chobots.wiki' expires in 10 day(s) (Tue 16 Apr 2024 07:51:15 PM GMT +0000). [06:01:18] RECOVERY - ru.countryhumans.polandball.wiki - reverse DNS on sslhost is OK: SSL OK - ru.countryhumans.polandball.wiki reverse DNS resolves to cp51.wikitide.net - CNAME OK [06:01:19] RECOVERY - worldsanskrit.net - reverse DNS on sslhost is OK: SSL OK - worldsanskrit.net reverse DNS resolves to cp37.wikitide.net - NS RECORDS OK [06:01:19] RECOVERY - wiki.corgicam.tv - reverse DNS on sslhost is OK: SSL OK - wiki.corgicam.tv reverse DNS resolves to cp51.wikitide.net - CNAME OK [06:02:31] RECOVERY - cp51 Puppet on cp51 is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures [06:02:46] RECOVERY - wiki.thelastdimension.net - reverse DNS on sslhost is OK: SSL OK - wiki.thelastdimension.net reverse DNS resolves to cp36.wikitide.net - CNAME OK [06:03:02] RECOVERY - wiki.insideearth.info - reverse DNS on sslhost is OK: SSL OK - wiki.insideearth.info reverse DNS resolves to cp36.wikitide.net - CNAME OK [06:03:02] RECOVERY - trollpasta.com - reverse DNS on sslhost is OK: SSL OK - trollpasta.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [06:03:03] RECOVERY - wiki.seredrau.xyz - reverse DNS on sslhost is OK: SSL OK - wiki.seredrau.xyz reverse DNS resolves to cp36.wikitide.net - CNAME OK [06:03:05] RECOVERY - wiki.s23.org - reverse DNS on sslhost is OK: SSL OK - wiki.s23.org reverse DNS resolves to cp37.wikitide.net - CNAME OK [06:03:07] RECOVERY - housing.wiki - reverse DNS on sslhost is OK: SSL OK - housing.wiki reverse DNS resolves to cp37.wikitide.net - CNAME FLAT [06:03:29] RECOVERY - wiki.ate42.ru - reverse DNS on sslhost is OK: SSL OK - wiki.ate42.ru reverse DNS resolves to cp37.wikitide.net - CNAME OK [06:03:49] RECOVERY - allthetropes.miraheze.org - reverse DNS on sslhost is OK: SSL OK - allthetropes.miraheze.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [06:03:50] RECOVERY - www.hoolehistoryheritagesociety.org.uk - LetsEncrypt on sslhost is OK: OK - Certificate 'www.hoolehistoryheritagesociety.org.uk' will expire on Wed 15 May 2024 01:31:01 AM GMT +0000. [06:04:01] RECOVERY - wiki.jeremyboyd.org - reverse DNS on sslhost is OK: SSL OK - wiki.jeremyboyd.org reverse DNS resolves to cp37.wikitide.net - CNAME OK [06:04:02] RECOVERY - www.ngnl.wiki - reverse DNS on sslhost is OK: SSL OK - www.ngnl.wiki reverse DNS resolves to cp36.wikitide.net - CNAME OK [06:04:11] RECOVERY - bluearchive.wiki - reverse DNS on sslhost is OK: SSL OK - bluearchive.wiki reverse DNS resolves to cp37.wikitide.net - CNAME OK [06:04:13] RECOVERY - wiki.omegabuild.uk - reverse DNS on sslhost is OK: SSL OK - wiki.omegabuild.uk reverse DNS resolves to cp37.wikitide.net - CNAME OK [06:04:19] RECOVERY - www.polandballwiki.com - reverse DNS on sslhost is OK: SSL OK - www.polandballwiki.com reverse DNS resolves to cp37.wikitide.net - CNAME OK [06:04:20] RECOVERY - www.hebammenwiki.de - reverse DNS on sslhost is OK: SSL OK - www.hebammenwiki.de reverse DNS resolves to cp36.wikitide.net - CNAME OK [06:04:20] RECOVERY - tssm.wiki - reverse DNS on sslhost is OK: SSL OK - tssm.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [06:04:22] RECOVERY - closinglogosgroup.miraheze.org - reverse DNS on sslhost is OK: SSL OK - closinglogosgroup.miraheze.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [06:04:22] RECOVERY - wiki.zamnhacking.net - reverse DNS on sslhost is OK: SSL OK - wiki.zamnhacking.net reverse DNS resolves to cp37.wikitide.net - CNAME OK [06:04:23] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: SSL OK - lcn.zfc.id.lv reverse DNS resolves to cp37.wikitide.net - CNAME OK [06:04:24] RECOVERY - 321nailswiki.miraheze.org - reverse DNS on sslhost is OK: SSL OK - 321nailswiki.miraheze.org reverse DNS resolves to cp37.wikitide.net - NS RECORDS OK [06:04:25] RECOVERY - steem.wiki - reverse DNS on sslhost is OK: SSL OK - steem.wiki reverse DNS resolves to cp37.wikitide.net - CNAME OK [06:04:29] RECOVERY - www.istpcomputing.com - reverse DNS on sslhost is OK: SSL OK - www.istpcomputing.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [06:04:42] RECOVERY - wiki.theluxuryelevator.com - reverse DNS on sslhost is OK: SSL OK - wiki.theluxuryelevator.com reverse DNS resolves to cp37.wikitide.net - CNAME OK [06:41:43] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mahdiruiz.line.pm All nameservers failed to answer the query. [07:10:54] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:09:10] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK [08:56:12] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [08:57:21] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [08:59:31] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [09:00:03] miraheze/CreateWiki - redbluegreenhat the build has errored. [09:00:27] miraheze/CreateWiki - redbluegreenhat the build has errored. [09:02:44] miraheze/CreateWiki - redbluegreenhat the build has errored. [09:06:16] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [09:09:25] miraheze/CreateWiki - redbluegreenhat the build has errored. [09:15:22] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [09:19:06] miraheze/CreateWiki - redbluegreenhat the build has errored. [09:22:48] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [09:30:45] miraheze/CreateWiki - redbluegreenhat the build passed. [09:33:55] [02CreateWiki] 07redbluegreenhat commented on pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503#issuecomment-2041031001 [09:58:23] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [10:00:19] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [10:01:22] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [10:01:30] miraheze/CreateWiki - redbluegreenhat the build has errored. [10:03:48] miraheze/CreateWiki - redbluegreenhat the build has errored. [10:03:50] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [10:04:34] miraheze/CreateWiki - redbluegreenhat the build has errored. [10:05:15] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [10:07:27] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [10:07:32] miraheze/CreateWiki - redbluegreenhat the build has errored. [10:08:21] miraheze/CreateWiki - redbluegreenhat the build has errored. [10:10:33] miraheze/CreateWiki - redbluegreenhat the build has errored. [10:15:13] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [10:23:09] miraheze/CreateWiki - redbluegreenhat the build passed. [10:25:49] [02CreateWiki] 07redbluegreenhat edited pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [10:26:40] [02CreateWiki] 07redbluegreenhat edited pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [10:27:19] [02CreateWiki] 07redbluegreenhat closed pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [10:27:22] [02miraheze/CreateWiki] 07redbluegreenhat pushed 031 commit to 03master [+0/-0/±5] 13https://github.com/miraheze/CreateWiki/compare/f164d1dfdd6d...001c5a422fea [10:27:25] [02miraheze/CreateWiki] 07redbluegreenhat 03001c5a4 - T12010: Introduce CreateWiki-specific suppression rights and suppression log (#503) [10:27:57] !log [alex@test151] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 [10:27:58] !log [alex@test151] DEPLOY ABORTED: Canary check failed for meta.mirabeta.org@localhost [10:28:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:28:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:28:34] @bluemoon0332 what are you breaking [10:28:56] not my fault for once 😂 [10:29:02] it was already broken when I found it [10:29:04] The laughable think is there's only one test server so that doesn't even stop the rollout [10:29:14] It just stops post build steps [10:29:21] No it doesn't [10:29:27] Cause they all run staging [10:29:34] That stops exactly nothing on beta [10:29:40] @bluemoon0332 [10:29:51] Is it still broken from JobQueue [10:30:01] I don't think that's it [10:30:25] No that's differently broken [10:30:27] wiki straight up doesn't exist right now, see https://meta.mirabeta.org/ [10:30:31] @bluemoon0332 is the ManageWiki cache there [10:30:43] And is cw_wikis populated [10:33:44] I was accidentally querying cw_wikis in mhglobal instead of testglobal, no wonder I was getting nothing [10:34:24] yeah, wiki exists in testglobal's cw_wikis [10:34:43] I think I know what's up though [10:35:16] miraheze/CreateWiki - redbluegreenhat the build passed. [10:35:40] What? [10:36:20] CA is working on being able to choose a subdomain out of multiple domains when requesting a wiki [10:36:51] that means some changes are being done to our entrypoints, seems it has not yet been quite figured out [10:36:53] https://github.com/miraheze/mw-config/commit/e5e5481d6f6f4a54582c8feada49a3dd69fddfac [10:37:21] *a subdomain on one of our domains, not just miraheze.org [10:39:28] !log [alex@test151] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 [10:39:29] !log [alex@test151] finished deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 - SUCCESS in 1s [10:39:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:39:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:39:52] !log [alex@test151] starting deploy of {'l10n': True, 'versions': '1.41'} to test151 [10:40:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:40:14] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [10:41:44] Ah [10:44:02] !log [alex@test151] finished deploy of {'l10n': True, 'versions': '1.41'} to test151 - SUCCESS in 250s [10:44:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:46:37] nice, the new suppression system is working [10:46:40] love to see it [10:48:17] well, I missed some i18n for the Special:Log page for the private log [10:48:28] but other than that everything else is working perfectly [10:54:49] [1/2] Log looks like this btw, it logs suppressions and unsuppressions (there's no private info here, so OK to publish) [10:54:50] [2/2] https://cdn.discordapp.com/attachments/808001911868489748/1226122940349812796/image.png?ex=66239ef9&is=661129f9&hm=bc4515df7031687361757f7b78d5bff1d231e25df704485d9fc1bc215088170a& [10:56:18] Nice [11:17:15] [02CreateWiki] 07redbluegreenhat opened pull request 03#504: T12010: Add missing i18n - 13https://github.com/miraheze/CreateWiki/pull/504 [11:25:58] miraheze/CreateWiki - redbluegreenhat the build passed. [11:26:13] [02CreateWiki] 07redbluegreenhat closed pull request 03#504: T12010: Add missing i18n - 13https://github.com/miraheze/CreateWiki/pull/504 [11:26:16] [02miraheze/CreateWiki] 07redbluegreenhat pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/001c5a422fea...fa9cd74a0ce5 [11:26:18] [02miraheze/CreateWiki] 07redbluegreenhat 03fa9cd74 - T12010: Add missing i18n (#504) [11:26:19] !log [alex@test151] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 [11:26:21] !log [alex@test151] finished deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 - SUCCESS in 1s [11:26:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:26:27] !log [alex@test151] starting deploy of {'l10n': True, 'versions': '1.41'} to test151 [11:26:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:26:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:30:50] !log [alex@test151] finished deploy of {'l10n': True, 'versions': '1.41'} to test151 - SUCCESS in 262s [11:30:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:34:58] miraheze/CreateWiki - redbluegreenhat the build passed. [11:43:33] [02mw-config] 07redbluegreenhat opened pull request 03#5532: T12010: Update user rights - 13https://github.com/miraheze/mw-config/pull/5532 [11:43:45] [02mw-config] 07redbluegreenhat edited pull request 03#5532: T12010: Update user rights - 13https://github.com/miraheze/mw-config/pull/5532 [11:44:25] miraheze/mw-config - redbluegreenhat the build passed. [12:03:23] [02mw-config] 07redbluegreenhat synchronize pull request 03#5532: T12010: Update user rights - 13https://github.com/miraheze/mw-config/pull/5532 [12:04:22] miraheze/mw-config - redbluegreenhat the build passed. [12:08:05] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [12:40:20] PROBLEM - steem.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'steem.wiki' expires in 15 day(s) (Mon 22 Apr 2024 12:37:20 PM GMT +0000). [12:40:34] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/bcef397befd0...981fbbcbd267 [12:40:35] [02miraheze/ssl] 07WikiTideSSLBot 03981fbbc - Bot: Update SSL cert for steem.wiki [12:53:15] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [13:09:39] RECOVERY - steem.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'steem.wiki' will expire on Fri 05 Jul 2024 11:40:26 AM GMT +0000. [13:22:00] RECOVERY - wiki.gab.pt.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.gab.pt.eu.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [14:19:19] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 169.93 ms [14:21:20] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 167.50 ms [14:57:06] PROBLEM - cp41 Disk Space on cp41 is WARNING: DISK WARNING - free space: / 10223MiB (10% inode=98%); [16:28:19] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [17:10:15] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [17:26:57] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp37.wikitide.net - CNAME OK [17:28:02] [02mw-config] 07Universal-Omega synchronize pull request 03#5531: thumb_handler: redirect default server to primary domain - 13https://github.com/miraheze/mw-config/pull/5531 [17:28:05] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03thumb_handler-redirect [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/ae0fff51e1eb...29f7fd7b6647 [17:28:08] [02miraheze/mw-config] 07Universal-Omega 0329f7fd7 - Add ignorePrimary option [17:28:52] [02mw-config] 07Universal-Omega synchronize pull request 03#5531: thumb_handler: redirect default server to primary domain - 13https://github.com/miraheze/mw-config/pull/5531 [17:28:55] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03thumb_handler-redirect [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/29f7fd7b6647...1cac6ca761ac [17:28:56] [02miraheze/mw-config] 07Universal-Omega 031cac6ca - Use ignorePrimary [17:28:59] miraheze/mw-config - Universal-Omega the build passed. [17:29:49] miraheze/mw-config - Universal-Omega the build passed. [17:35:07] [02mw-config] 07Universal-Omega synchronize pull request 03#5531: thumb_handler: redirect default server to primary domain - 13https://github.com/miraheze/mw-config/pull/5531 [17:35:10] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03thumb_handler-redirect [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/1cac6ca761ac...0fec747da45b [17:35:13] [02miraheze/mw-config] 07Universal-Omega 030fec747 - Pass database [17:35:56] miraheze/mw-config - Universal-Omega the build passed. [17:35:59] [02mw-config] 07Universal-Omega synchronize pull request 03#5531: thumb_handler: redirect default server to primary domain - 13https://github.com/miraheze/mw-config/pull/5531 [17:36:02] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03thumb_handler-redirect [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/0fec747da45b...efd8bdb9d60c [17:36:05] [02miraheze/mw-config] 07Universal-Omega 03efd8bdb - Pass database [17:36:51] miraheze/mw-config - Universal-Omega the build passed. [17:39:15] [02mw-config] 07Universal-Omega synchronize pull request 03#5531: thumb_handler: redirect default server to primary domain - 13https://github.com/miraheze/mw-config/pull/5531 [17:39:18] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03thumb_handler-redirect [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/efd8bdb9d60c...5ad0c0f47790 [17:39:19] [02miraheze/mw-config] 07Universal-Omega 035ad0c0f - Fix [17:40:06] miraheze/mw-config - Universal-Omega the build passed. [17:42:44] [02mw-config] 07Universal-Omega synchronize pull request 03#5531: thumb_handler: redirect default server to primary domain - 13https://github.com/miraheze/mw-config/pull/5531 [17:42:47] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03thumb_handler-redirect [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/5ad0c0f47790...9a669e27dc18 [17:42:49] [02miraheze/mw-config] 07Universal-Omega 039a669e2 - Fix [17:43:33] miraheze/mw-config - Universal-Omega the build passed. [17:49:08] [02mw-config] 07Universal-Omega synchronize pull request 03#5531: thumb_handler: redirect default server to primary domain - 13https://github.com/miraheze/mw-config/pull/5531 [17:49:10] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03thumb_handler-redirect [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/9a669e27dc18...0b0908706fce [17:49:12] [02miraheze/mw-config] 07Universal-Omega 030b09087 - Fix [17:50:08] miraheze/mw-config - Universal-Omega the build passed. [17:52:30] [02CreateWiki] 07redbluegreenhat opened pull request 03#505: Drop old code - 13https://github.com/miraheze/CreateWiki/pull/505 [17:52:47] [02CreateWiki] 07redbluegreenhat closed pull request 03#505: Drop old code - 13https://github.com/miraheze/CreateWiki/pull/505 [17:52:49] [02miraheze/CreateWiki] 07redbluegreenhat pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/fa9cd74a0ce5...d4ddc2ff2b96 [17:52:52] [02miraheze/CreateWiki] 07redbluegreenhat 03d4ddc2f - Drop old code (#505) [17:54:01] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/b0c56c1dab1d...55c70176a0e0 [17:54:03] [02miraheze/mw-config] 07Universal-Omega 0355c7017 - Use getRealm directly [17:54:52] miraheze/mw-config - Universal-Omega the build passed. [17:59:00] !log [alex@test151] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 [17:59:01] !log [alex@test151] finished deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 - SUCCESS in 1s [17:59:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:59:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:00:10] miraheze/CreateWiki - redbluegreenhat the build passed. [18:00:40] miraheze/CreateWiki - redbluegreenhat the build passed. [18:02:08] !log [@mwtask181] starting deploy of {'config': True} to all [18:02:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:02:18] !log [@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 9s [18:02:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:03:31] !log [alex@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [18:03:32] !log [alex@test151] DEPLOY ABORTED: Canary check failed for meta.mirabeta.org@localhost [18:03:37] !log [alex@test151] starting deploy of {'pull': 'config', 'config': True, 'force': True} to test151 [18:03:38] !log [alex@test151] finished deploy of {'pull': 'config', 'config': True, 'force': True} to test151 - SUCCESS in 0s [18:03:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:03:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:03:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:04:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:07:00] !log [@mwtask171] starting deploy of {'config': True} to all [18:07:08] !log [@mwtask171] finished deploy of {'config': True} to all - SUCCESS in 8s [18:07:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:07:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:08:27] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK [18:10:46] !log [alex@test151] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/extensions/MirahezeMagic/maintenance/resetWikiCaches.php --wiki=metawikibeta (END - exit=0) [18:10:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:14:49] !log [alex@test151] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 [18:14:50] !log [alex@test151] finished deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 - SUCCESS in 0s [18:14:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:15:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:45:39] [02MatomoAnalytics] 07Universal-Omega synchronize pull request 03#125: MatomoAnalyticsWiki: support for getting most visited pages - 13https://github.com/miraheze/MatomoAnalytics/pull/125 [18:45:41] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03mostvisistedpages [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/97f58153cfe7...9e6546480ff5 [18:45:42] [02miraheze/MatomoAnalytics] 07Universal-Omega 039e65464 - Use getPageTitles [18:49:48] miraheze/MatomoAnalytics - Universal-Omega the build passed. [18:59:04] [02CreateWiki] 07redbluegreenhat opened pull request 03#506: Fix comparison between requested and current suppression level - 13https://github.com/miraheze/CreateWiki/pull/506 [19:07:56] miraheze/CreateWiki - redbluegreenhat the build passed. [19:08:05] [02CreateWiki] 07redbluegreenhat closed pull request 03#506: Fix comparison between requested and current suppression level - 13https://github.com/miraheze/CreateWiki/pull/506 [19:08:07] [02miraheze/CreateWiki] 07redbluegreenhat pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/d4ddc2ff2b96...45b622d624ff [19:08:10] [02miraheze/CreateWiki] 07redbluegreenhat 0345b622d - Fix comparison between requested and current suppression level (#506) [19:08:12] !log [alex@test151] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 [19:08:13] !log [alex@test151] finished deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 - SUCCESS in 1s [19:08:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:08:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:15:52] miraheze/CreateWiki - redbluegreenhat the build passed. [20:27:13] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/c49f19681e7b...b8945c13b643 [20:27:16] [02miraheze/puppet] 07Universal-Omega 03b8945c1 - recursor: remove miraheze.org [21:21:24] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/extensions/CreateWiki/maintenance/setContainersAccess.php --wiki=shatteredisleswiki (END - exit=0) [21:21:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:35:00] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/981fbbcbd267...d4597bbf7662 [21:35:01] [02miraheze/ssl] 07WikiTideSSLBot 03d4597bb - Bot: Update SSL cert for talumn.neptune.wiki [21:36:22] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d4597bbf7662...53f94f4a350c [21:36:25] [02miraheze/ssl] 07WikiTideSSLBot 0353f94f4 - Bot: Update SSL cert for talumn.neptune.wiki [21:37:26] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/53f94f4a350c...38ec13a268ff [21:37:27] [02miraheze/ssl] 07WikiTideSSLBot 0338ec13a - Bot: Update SSL cert for wiki.esnmilanostatale.it [21:41:13] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/38ec13a268ff...0f963b4ab4ce [21:41:15] [02miraheze/ssl] 07WikiTideSSLBot 030f963b4 - Bot: Update SSL cert for pyramidgames.wiki [21:42:01] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/0f963b4ab4ce...578b5e59b80f [21:42:02] [02miraheze/ssl] 07WikiTideSSLBot 03578b5e5 - Bot: Update SSL cert for wiki.fediverse.kr [21:45:36] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/578b5e59b80f...21982fcf96c9 [21:45:37] [02miraheze/ssl] 07WikiTideSSLBot 0321982fc - Bot: Update SSL cert for 100acg.wiki [21:46:28] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/21982fcf96c9...88bfa4909137 [21:46:31] [02miraheze/ssl] 07WikiTideSSLBot 0388bfa49 - Bot: Update SSL cert for wiki.lesunk.com [21:47:21] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/88bfa4909137...af41fec492de [21:47:22] [02miraheze/ssl] 07WikiTideSSLBot 03af41fec - Bot: Update SSL cert for wiki.closai.com [21:49:04] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/af41fec492de...3dc4d93b4e02 [21:49:07] [02miraheze/ssl] 07WikiTideSSLBot 033dc4d93 - Bot: Update SSL cert for wiki.sofurry.games [21:49:58] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3dc4d93b4e02...f0eba66ef0f2 [21:49:59] [02miraheze/ssl] 07WikiTideSSLBot 03f0eba66 - Bot: Update SSL cert for wiki.k2cho.me [21:50:39] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/f0eba66ef0f2...54aec4925a1e [21:50:40] [02miraheze/ssl] 07WikiTideSSLBot 0354aec49 - Bot: Update SSL cert for wiki.gab.pt.eu.org [21:51:14] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/54aec4925a1e...ddbf3c827eab [21:51:16] [02miraheze/ssl] 07WikiTideSSLBot 03ddbf3c8 - Bot: Update SSL cert for en.clockup.wiki [21:51:43] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ddbf3c827eab...c4034674f224 [21:51:44] [02miraheze/ssl] 07WikiTideSSLBot 03c403467 - Bot: Update SSL cert for www.pomologia.org [21:52:03] PROBLEM - cp36 Disk Space on cp36 is WARNING: DISK WARNING - free space: / 9697MiB (10% inode=98%); [21:52:37] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c4034674f224...ae3746854e53 [21:52:40] [02miraheze/ssl] 07WikiTideSSLBot 03ae37468 - Bot: Update SSL cert for wiki.candelabrem.com [21:53:20] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ae3746854e53...75c788828a61 [21:53:21] [02miraheze/ssl] 07WikiTideSSLBot 0375c7888 - Bot: Update SSL cert for themagical.world [21:53:55] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/75c788828a61...5717daffa306 [21:53:58] [02miraheze/ssl] 07WikiTideSSLBot 035717daf - Bot: Update SSL cert for www.moiswiki.net [21:56:20] RECOVERY - talumn.neptune.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'talumn.neptune.wiki' will expire on Fri 05 Jul 2024 08:36:17 PM GMT +0000. [21:56:35] RECOVERY - wiki.esnmilanostatale.it - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.esnmilanostatale.it' will expire on Wed 26 Jun 2024 01:58:31 PM GMT +0000. [21:56:50] RECOVERY - www.pyramidgames.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'pyramidgames.wiki' will expire on Fri 05 Jul 2024 08:41:07 PM GMT +0000. [21:56:54] RECOVERY - pyramidgames.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'pyramidgames.wiki' will expire on Fri 05 Jul 2024 08:41:07 PM GMT +0000. [21:57:00] RECOVERY - wiki.fediverse.kr - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.fediverse.kr' will expire on Fri 05 Jul 2024 08:41:56 PM GMT +0000. [21:57:05] RECOVERY - 100acg.wiki - LetsEncrypt on sslhost is OK: OK - Certificate '100acg.wiki' will expire on Fri 05 Jul 2024 08:45:31 PM GMT +0000. [21:57:11] RECOVERY - wiki.lesunk.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.lesunk.com' will expire on Fri 05 Jul 2024 08:46:23 PM GMT +0000. [21:57:27] RECOVERY - wiki.closai.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.closai.com' will expire on Fri 05 Jul 2024 08:47:16 PM GMT +0000. [22:00:30] RECOVERY - themagical.world - LetsEncrypt on sslhost is OK: OK - Certificate 'themagical.world' will expire on Fri 05 Jul 2024 08:53:14 PM GMT +0000. [22:00:41] RECOVERY - wiki.gab.pt.eu.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.gab.pt.eu.org' will expire on Fri 05 Jul 2024 08:50:33 PM GMT +0000. [22:01:06] RECOVERY - wiki.sofurry.games - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.sofurry.games' will expire on Fri 05 Jul 2024 08:48:58 PM GMT +0000. [22:01:12] RECOVERY - wiki.k2cho.me - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.k2cho.me' will expire on Fri 05 Jul 2024 08:49:51 PM GMT +0000. [22:01:18] RECOVERY - en.clockup.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'en.clockup.wiki' will expire on Fri 05 Jul 2024 08:51:09 PM GMT +0000. [22:01:23] RECOVERY - www.pomologia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'www.pomologia.org' will expire on Fri 05 Jul 2024 08:51:38 PM GMT +0000. [22:01:32] RECOVERY - wiki.candelabrem.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.candelabrem.com' will expire on Fri 05 Jul 2024 08:52:31 PM GMT +0000. [22:01:40] PROBLEM - cp37 Disk Space on cp37 is WARNING: DISK WARNING - free space: / 9689MiB (10% inode=98%); [22:01:49] RECOVERY - www.themagical.world - LetsEncrypt on sslhost is OK: OK - Certificate 'themagical.world' will expire on Fri 05 Jul 2024 08:53:14 PM GMT +0000. [22:02:04] RECOVERY - www.moiswiki.net - LetsEncrypt on sslhost is OK: OK - Certificate 'www.moiswiki.net' will expire on Fri 05 Jul 2024 08:53:50 PM GMT +0000. [22:03:06] PROBLEM - cp51 Puppet on cp51 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [22:05:03] RECOVERY - cp51 Puppet on cp51 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures [22:11:29] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [22:57:16] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/b8945c13b643...fa8f52a9507d [22:57:19] [02miraheze/puppet] 07Universal-Omega 03fa8f52a - phorge: remove miraheze.wiki cert [23:38:51] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK