[01:50:28] <icinga-miraheze>	 PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.mahdiruiz.line.pm. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL
[02:20:12] <icinga-miraheze>	 RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK
[02:34:40] <icinga-miraheze>	 PROBLEM - private.yahyabd.xyz - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.
[03:32:57] <icinga-miraheze>	 PROBLEM - private.yahyabd.xyz - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - private.yahyabd.xyz All nameservers failed to answer the query.
[03:34:12] <icinga-miraheze>	 PROBLEM - wiki.hacklabsr.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.hacklabsr.org could not be found
[03:37:25] <icinga-miraheze>	 [Grafana] !sre FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[04:33:48] <icinga-miraheze>	 PROBLEM - cp51 Puppet on cp51 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[05:02:11] <icinga-miraheze>	 RECOVERY - cp51 Puppet on cp51 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[05:26:19] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 169.39 ms
[05:32:31] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 168.77 ms
[05:40:51] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 171.24 ms
[05:42:54] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 169.75 ms
[05:44:31] <icinga-miraheze>	 PROBLEM - www.clinitheque.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.clinitheque.fr All nameservers failed to answer the query.
[05:47:05] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 175.61 ms
[05:49:08] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 172.44 ms
[06:13:25] <icinga-miraheze>	 RECOVERY - www.clinitheque.fr - reverse DNS on sslhost is OK: SSL OK - www.clinitheque.fr reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK
[07:03:50] <icinga-miraheze>	 PROBLEM - cp51 Puppet on cp51 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[07:16:31] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 167.17 ms
[07:18:34] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 169.57 ms
[07:32:02] <icinga-miraheze>	 PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query.
[07:32:10] <icinga-miraheze>	 RECOVERY - cp51 Puppet on cp51 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[07:37:25] <icinga-miraheze>	 [Grafana] !sre FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[07:55:39] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 169.66 ms
[07:59:44] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 171.22 ms
[08:08:02] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 168.88 ms
[08:10:04] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 169.05 ms
[09:23:43] <MirahezeLSBot>	 !log [@test151] starting deploy of {'config': True} to test151
[09:23:44] <MirahezeLSBot>	 !log [@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s
[09:23:53] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[09:24:02] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[09:30:03] <icinga-miraheze>	 RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp37.wikitide.net - CNAME OK
[09:50:49] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 168.88 ms
[09:52:51] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 172.15 ms
[10:15:36] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 171.45 ms
[10:17:38] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 173.78 ms
[10:40:21] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 176.89 ms
[10:42:23] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 179.71 ms
[10:46:34] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 170.27 ms
[10:48:37] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 172.76 ms
[11:18:34] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 171.32 ms
[11:20:36] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 176.30 ms
[11:37:25] <icinga-miraheze>	 [Grafana] !sre FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[12:24:52] <icinga-miraheze>	 PROBLEM - permanentfuturelab.wiki - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.402 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.
[12:53:32] <icinga-miraheze>	 RECOVERY - permanentfuturelab.wiki - reverse DNS on sslhost is OK: SSL OK - permanentfuturelab.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK
[12:55:17] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 167.56 ms
[13:01:28] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 169.12 ms
[13:13:39] <icinga-miraheze>	 PROBLEM - index.pdcommunity.ir - LetsEncrypt on sslhost is CRITICAL: Temporary failure in name resolutionHTTP CRITICAL - Unable to open TCP socket
[13:23:45] <MirahezeLSBot>	 !log [@test151] starting deploy of {'config': True} to test151
[13:23:46] <MirahezeLSBot>	 !log [@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s
[13:23:54] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[13:24:03] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[13:30:56] <icinga-miraheze>	 PROBLEM - index.pdcommunity.ir - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - index.pdcommunity.ir All nameservers failed to answer the query.
[13:43:16] <icinga-miraheze>	 RECOVERY - index.pdcommunity.ir - LetsEncrypt on sslhost is OK: OK - Certificate 'index.pdcommunity.ir' will expire on Tue 14 May 2024 05:57:39 PM GMT +0000.
[13:48:55] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 171.18 ms
[13:50:58] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 170.14 ms
[13:59:52] <icinga-miraheze>	 RECOVERY - index.pdcommunity.ir - reverse DNS on sslhost is OK: SSL OK - index.pdcommunity.ir reverse DNS resolves to cp37.wikitide.net - CNAME OK
[14:32:25] <icinga-miraheze>	 [Grafana] !sre RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[15:13:28] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 168.28 ms
[15:15:31] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 170.66 ms
[15:28:58] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 170.10 ms
[15:35:07] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 168.33 ms
[15:50:33] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 171.40 ms
[15:53:24] <icinga-miraheze>	 PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL
[15:56:43] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 170.10 ms
[16:03:02] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 167.06 ms
[16:07:08] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 167.95 ms
[16:23:09] <icinga-miraheze>	 PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mahdiruiz.line.pm All nameservers failed to answer the query.
[16:52:56] <icinga-miraheze>	 RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK
[16:53:40] <icinga-miraheze>	 PROBLEM - miraheze.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'miraheze.wiki' expires in 15 day(s) (Fri 10 May 2024 04:35:32 PM GMT +0000).
[17:05:11] <botifico>	 [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/cdcc6f3a9376...03b7ca45c845
[17:05:13] <botifico>	 [02miraheze/puppet] 07Universal-Omega 0303b7ca4 - Remove agent
[17:06:00] <botifico>	 [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/03b7ca45c845...f94f959d56c1
[17:06:01] <botifico>	 [02miraheze/puppet] 07Universal-Omega 03f94f959 - Absent
[17:06:42] <botifico>	 [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/f94f959d56c1...5620e44752c5
[17:06:43] <botifico>	 [02miraheze/puppet] 07Universal-Omega 035620e44 - Remove agent from monitoring
[17:17:59] <icinga-miraheze>	 PROBLEM - puppet181 Puppet on puppet181 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 18 minutes ago with 0 failures
[17:20:02] <botifico>	 [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/afcb40e96115...1b71dbbf03e1
[17:20:03] <botifico>	 [02miraheze/mw-config] 07Universal-Omega 031b71dbb - -agent
[17:20:40] <miraheze-github>	 miraheze/mw-config - Universal-Omega the build passed.
[17:23:32] <MirahezeLSBot>	 !log [@test151] starting deploy of {'config': True} to test151
[17:23:33] <MirahezeLSBot>	 !log [@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s
[17:23:41] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:23:50] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:32:40] <MirahezeLSBot>	 !log [@mwtask181] starting deploy of {'config': True} to all
[17:32:49] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:32:52] <MirahezeLSBot>	 !log [@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 12s
[17:33:01] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:36:59] <MirahezeLSBot>	 !log [@mwtask171] starting deploy of {'config': True} to all
[17:37:08] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:37:09] <MirahezeLSBot>	 !log [@mwtask171] finished deploy of {'config': True} to all - SUCCESS in 9s
[17:37:18] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:40:12] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 167.45 ms
[17:42:15] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 171.67 ms
[18:03:52] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 171.12 ms
[18:05:54] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 171.27 ms
[18:38:28] <botifico>	 [02mw-config] 07koreirozzu opened pull request 03#5544: [T11927] Set wgRestrictDisplayTitle to false for swsewiki - 13https://github.com/miraheze/mw-config/pull/5544
[18:49:55] <botifico>	 [02mw-config] 07koreirozzu closed pull request 03#5544: [T11927] Set wgRestrictDisplayTitle to false for swsewiki - 13https://github.com/miraheze/mw-config/pull/5544
[18:49:56] <botifico>	 [02mw-config] 07koreirozzu commented on pull request 03#5544: [T11927] Set wgRestrictDisplayTitle to false for swsewiki - 13https://github.com/miraheze/mw-config/pull/5544#issuecomment-2075611356
[19:11:27] <icinga-miraheze>	 PROBLEM - wiki.andreijiroh.uk.eu.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.andreijiroh.uk.eu.org' expires in 15 day(s) (Fri 10 May 2024 07:08:08 PM GMT +0000).
[19:11:48] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/86b38a71635e...5645e973fb1b
[19:11:51] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot 035645e97 - Bot: Update SSL cert for wiki.andreijiroh.uk.eu.org
[19:12:20] <icinga-miraheze>	 [Grafana] !sre FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[19:22:13] <MirahezeRelay>	 <pixldev> 🫡
[19:32:53] <icinga-miraheze>	 PROBLEM - cp51 Puppet on cp51 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx]
[19:37:46] <icinga-miraheze>	 PROBLEM - cp41 Puppet on cp41 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx]
[19:50:25] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 171.56 ms
[19:51:01] <icinga-miraheze>	 PROBLEM - antiguabarbudacalypso.com - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.
[19:52:28] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 168.58 ms
[20:00:54] <icinga-miraheze>	 RECOVERY - cp51 Puppet on cp51 is OK: OK: Puppet is currently enabled, last run 42 seconds ago with 0 failures
[20:05:47] <icinga-miraheze>	 RECOVERY - cp41 Puppet on cp41 is OK: OK: Puppet is currently enabled, last run 47 seconds ago with 0 failures
[20:11:23] <icinga-miraheze>	 RECOVERY - wiki.andreijiroh.uk.eu.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.andreijiroh.uk.eu.org' will expire on Tue 23 Jul 2024 06:11:42 PM GMT +0000.
[20:20:12] <icinga-miraheze>	 RECOVERY - antiguabarbudacalypso.com - reverse DNS on sslhost is OK: SSL OK - antiguabarbudacalypso.com reverse DNS resolves to cp37.wikitide.net - NS RECORDS OK
[20:28:24] <icinga-miraheze>	 PROBLEM - mockgovernments.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - mockgovernments.com All nameservers failed to answer the query.
[20:58:12] <icinga-miraheze>	 RECOVERY - mockgovernments.com - reverse DNS on sslhost is OK: SSL OK - mockgovernments.com reverse DNS resolves to cp37.wikitide.net - NS RECORDS OK
[21:55:36] <icinga-miraheze>	 PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 166.89 ms
[21:57:39] <icinga-miraheze>	 RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 167.69 ms
[22:05:59] <icinga-miraheze>	 PROBLEM - wiki.col6.de - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.col6.de' expires in 15 day(s) (Fri 10 May 2024 09:53:00 PM GMT +0000).
[22:06:19] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/5645e973fb1b...0a7e843c2908
[22:06:20] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot 030a7e843 - Bot: Update SSL cert for wiki.col6.de
[22:07:44] <MirahezeLSBot>	 !log [macfan@test151] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php extensions/CentralAuth/fixStuckGlobalRename.pgp --wiki=cwtestwikibeta --logwiki=metawikibeta --ignorestatus AgentTest Vanished user a3543a2b8247ebabfa32061f92a44669 (END - exit=256)
[22:07:53] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[22:08:14] <MirahezeLSBot>	 !log [macfan@test151] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=cwtestwikibeta --logwiki=metawikibeta --ignorestatus AgentTest Vanished user a3543a2b8247ebabfa32061f92a44669 (END - exit=256)
[22:08:23] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[22:09:23] <MirahezeLSBot>	 !log [macfan@test151] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=cwtestwikibeta --logwiki=metawikibeta --ignorestatus AgentTest Vanished user a3543a2b8247ebabfa32061f92a44669 (END - exit=256)
[22:09:32] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[22:09:36] <MirahezeRelay>	 <pixldev> :xsob:
[22:09:52] <MirahezeLSBot>	 !log [macfan@test151] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=cwtestwikibeta --logwiki=metawikibeta --ignorestatus AgentTest Vanished_user_a3543a2b8247ebabfa32061f92a44669 (END - exit=0)
[22:10:01] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[22:10:19] <MirahezeLSBot>	 !log [macfan@test151] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=frozenplumtesttestwikibeta --logwiki=metawikibeta --ignorestatus AgentTest Vanished_user_a3543a2b8247ebabfa32061f92a44669 (END - exit=65280)
[22:10:29] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[22:11:06] <MirahezeLSBot>	 !log [macfan@test151] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=frozenplumtestwikibeta --logwiki=metawikibeta --ignorestatus AgentTest Vanished_user_a3543a2b8247ebabfa32061f92a44669 (END - exit=0)
[22:11:16] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[22:11:45] <MirahezeLSBot>	 !log [macfan@test151] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=loginwikibeta --logwiki=metawikibeta --ignorestatus AgentTest Vanished_user_a3543a2b8247ebabfa32061f92a44669 (END - exit=0)
[22:11:54] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[22:11:59] <MirahezeLSBot>	 !log [macfan@test151] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=loginwikibeta --logwiki=testwikibeta --ignorestatus AgentTest Vanished_user_a3543a2b8247ebabfa32061f92a44669 (END - exit=256)
[22:12:08] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[22:12:38] <MirahezeLSBot>	 !log [macfan@test151] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=metawikibeta --logwiki=metawikibeta --ignorestatus AgentTest Vanished_user_a3543a2b8247ebabfa32061f92a44669 (END - exit=0)
[22:12:48] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[22:12:54] <MirahezeLSBot>	 !log [macfan@test151] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=metawikibeta --logwiki=testwikibeta --ignorestatus AgentTest Vanished_user_a3543a2b8247ebabfa32061f92a44669 (END - exit=256)
[22:13:03] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[22:15:21] <MirahezeRelay>	 <pixldev> am I missing something or is this the same script over and over
[22:15:31] <MirahezeLSBot>	 !log [macfan@test151] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=testwikibeta --logwiki=metawikibeta --ignorestatus AgentTest Vanished_user_a3543a2b8247ebabfa32061f92a44669 (END - exit=0)
[22:15:40] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[22:16:10] <MirahezeLSBot>	 !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=testcwwikiwiki --logwiki=metawiki --ignorestatus Agent_Isai MirahezeGDPR_7d0d9955a879b561967e37f13c9e30c8 (END - exit=0)
[22:16:20] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[22:20:47] <icinga-miraheze>	 PROBLEM - journeytheword.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - journeytheword.wiki All nameservers failed to answer the query.
[22:30:10] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/0a7e843c2908...a9d75161a6a7
[22:30:11] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot 03a9d7516 - Bot: Add SSL cert for eosdev.xyz
[22:30:59] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/a9d75161a6a7...b0d7fef3f13a
[22:31:02] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot 03b0d7fef - Bot: Add SSL cert for wiki.simplifiedcoding.org
[22:31:53] <icinga-miraheze>	 PROBLEM - wiki.manos.pt - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.manos.pt' expires in 15 day(s) (Fri 10 May 2024 10:20:07 PM GMT +0000).
[22:32:04] <icinga-miraheze>	 PROBLEM - wiki.lewisgill.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.lewisgill.com' expires in 15 day(s) (Fri 10 May 2024 10:15:58 PM GMT +0000).
[22:32:14] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/b0d7fef3f13a...f2dd535e1b23
[22:32:17] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot 03f2dd535 - Bot: Update SSL cert for wiki.manos.pt
[22:32:25] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/f2dd535e1b23...7cb6fdc5d130
[22:32:26] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot 037cb6fdc - Bot: Update SSL cert for wiki.lewisgill.com
[22:32:29] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/7cb6fdc5d130...0f32463365d8
[22:32:32] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot 030f32463 - Bot: Add SSL cert for msmhacks.uk
[22:32:56] <icinga-miraheze>	 PROBLEM - cp51 Puppet on cp51 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx]
[22:35:15] <icinga-miraheze>	 RECOVERY - wiki.col6.de - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.col6.de' will expire on Tue 23 Jul 2024 09:06:12 PM GMT +0000.
[22:40:34] <icinga-miraheze>	 PROBLEM - zomvilles.pixiemeta.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'zomvilles.pixiemeta.com' expires in 15 day(s) (Fri 10 May 2024 10:27:24 PM GMT +0000).
[22:40:55] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/0f32463365d8...ebbcfa4cf8ce
[22:40:58] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot 03ebbcfa4 - Bot: Update SSL cert for zomvilles.pixiemeta.com
[22:49:58] <icinga-miraheze>	 RECOVERY - journeytheword.wiki - reverse DNS on sslhost is OK: SSL OK - journeytheword.wiki reverse DNS resolves to cp37.wikitide.net - NS RECORDS OK
[23:00:37] <icinga-miraheze>	 RECOVERY - wiki.lewisgill.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.lewisgill.com' will expire on Tue 23 Jul 2024 09:32:19 PM GMT +0000.
[23:00:48] <icinga-miraheze>	 RECOVERY - wiki.manos.pt - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.manos.pt' will expire on Tue 23 Jul 2024 09:32:08 PM GMT +0000.
[23:00:54] <icinga-miraheze>	 RECOVERY - cp51 Puppet on cp51 is OK: OK: Puppet is currently enabled, last run 30 seconds ago with 0 failures
[23:09:04] <icinga-miraheze>	 PROBLEM - wiki.sheepservermc.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.sheepservermc.net' expires in 15 day(s) (Fri 10 May 2024 10:40:41 PM GMT +0000).
[23:09:24] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ebbcfa4cf8ce...cb3993700ca1
[23:09:27] <botifico>	 [02miraheze/ssl] 07WikiTideSSLBot 03cb39937 - Bot: Update SSL cert for wiki.sheepservermc.net
[23:09:46] <icinga-miraheze>	 RECOVERY - zomvilles.pixiemeta.com - LetsEncrypt on sslhost is OK: OK - Certificate 'zomvilles.pixiemeta.com' will expire on Tue 23 Jul 2024 09:40:49 PM GMT +0000.
[23:12:20] <icinga-miraheze>	 [Grafana] !sre FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[23:21:20] <MirahezeRelay>	 <pixldev> Can a Phab admin delete Agent’s Herald rule on phorge
[23:21:26] <MirahezeRelay>	 <pixldev> seems respectful
[23:22:00] <MirahezeRelay>	 <cosmicalpha> Not right now I think it needs shell to do and I can't
[23:23:05] <MirahezeRelay>	 <pixldev, replying to cosmicalpha> huh
[23:23:06] <MirahezeRelay>	 <pixldev> oh well
[23:23:11] <MirahezeRelay>	 <pixldev> it not a big deal
[23:23:17] <MirahezeRelay>	 <cosmicalpha> phab/phorge is odd
[23:27:04] <botifico>	 [02dns] 07MacFan4000 opened pull request 03#513: add irad.wiki zone - 13https://github.com/miraheze/dns/pull/513
[23:27:47] <MirahezeRelay>	 <pixldev, replying to cosmicalpha> eh
[23:33:53] <icinga-miraheze>	 PROBLEM - cp51 Puppet on cp51 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx]
[23:40:41] <botifico>	 [02dns] 07Universal-Omega closed pull request 03#513: add irad.wiki zone - 13https://github.com/miraheze/dns/pull/513
[23:40:43] <botifico>	 [02miraheze/dns] 07Universal-Omega pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/dns/compare/0abe288d624d...26d9a812003e
[23:40:46] <botifico>	 [02miraheze/dns] 07MacFan4000 0326d9a81 - add irad.wiki zone (#513)
[23:54:11] <botifico>	 [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/5620e44752c5...958dabeb1abe
[23:54:14] <botifico>	 [02miraheze/puppet] 07Universal-Omega 03958dabe - mediawiki: add ajax.cloudflare.com to CSP