[00:06:47] <icinga-miraheze>	 RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 57114MiB (12% inode=99%);
[00:08:30] <icinga-miraheze>	 RECOVERY - cp38 Disk Space on cp38 is OK: DISK OK - free space: / 57642MiB (12% inode=99%);
[01:02:25] <icinga-miraheze>	 PROBLEM - wiki.ithy.com - Cloudflare on sslhost is CRITICAL: Name or service not knownHTTP CRITICAL - Unable to open TCP socket
[01:30:59] <icinga-miraheze>	 RECOVERY - wiki.ithy.com - Cloudflare on sslhost is OK: OK - Certificate 'wiki.ithy.com' will expire on Tue 17 Jun 2025 09:53:49 PM GMT +0000.
[01:55:15] <miraheze-github>	 miraheze/mw-config - Aeywoo the build passed.
[01:56:05] <miraheze-github>	 miraheze/mw-config - Aeywoo the build passed.
[03:01:30] <icinga-miraheze>	 RECOVERY - mon181 Backups Grafana on mon181 is OK: FILE_AGE OK: /var/log/grafana-backup/grafana-backup/grafana-backup.log is 67 seconds old and 941 bytes
[03:32:00] <MacFanBot>	 [02mw-config] 07AgentIsai merged 07Aeywoo's pull request #5898: Undeploy Skin:Evelution per T13471#270359 (07miraheze:03main...07Aeywoo:03patch-6) 13https://github.com/miraheze/mw-config/pull/5898
[03:32:00] <MacFanBot>	 [02mw-config] 07AgentIsai pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/1e292e3e0b2b2876a4df1c7f5516f9b09afe93c0
[03:32:00] <MacFanBot>	 02mw-config/03main 07Aeywoo 031e292e3 T13471: Undeploy Evelution…
[03:32:42] <MacFanBot>	 [02mw-config] 07AgentIsai merged 07Aeywoo's pull request #5900: Adding footer links for the STOPXWiki (07miraheze:03main...07Aeywoo:03patch-5) 13https://github.com/miraheze/mw-config/pull/5900
[03:32:44] <MacFanBot>	 [02mw-config] 07AgentIsai pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/e3efc3867e1c72b9cfdd3a61cdea45513144c067
[03:32:45] <MacFanBot>	 02mw-config/03main 07Aeywoo 03e3efc38 T13440: Add footer links for the STOPXWiki (#5900)…
[03:32:50] <miraheze-github>	 miraheze/mw-config - AgentIsai the build passed.
[03:33:35] <miraheze-github>	 miraheze/mw-config - AgentIsai the build passed.
[06:21:53] <icinga-miraheze>	 [Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1744348880000&orgId=1&to=1744352513355
[06:26:53] <icinga-miraheze>	 [Grafana] RESOLVED: MediaWiki JobQueue is stalled https://grafana.wikitide.net/d/GtxbP1Xnk?from=1744349180000&orgId=1&to=1744352780000
[06:30:49] <RhinosF1>	 That looks technically true
[06:32:56] <RhinosF1>	 BlankEclair: what's a notificationGettingStarted/KeepGoingJob
[06:33:03] <RhinosF1>	 There's a 100 jobs in the queue
[06:33:13] <RhinosF1>	 That genuinely don't look to be moving
[09:00:47] <icinga-miraheze>	 PROBLEM - cp37 Disk Space on cp37 is WARNING: DISK WARNING - free space: / 49878MiB (10% inode=99%);
[09:35:01] <icinga-miraheze>	 RECOVERY - www.allthetropes.org - Cloudflare on sslhost is OK: OK - Certificate 'www.allthetropes.org' will expire on Sat 24 May 2025 03:40:56 PM GMT +0000.
[09:56:30] <icinga-miraheze>	 PROBLEM - cp38 Disk Space on cp38 is WARNING: DISK WARNING - free space: / 49933MiB (10% inode=99%);
[10:27:18] <MirahezeLSBot>	 !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/importImages.php --wiki=mothermotherwiki --search-recursively /home/reception/mothermother --summary=Imported from https://mothermother.fandom.com (START)
[10:27:20] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[10:27:42] <MirahezeLSBot>	 !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/importImages.php --wiki=mothermotherwiki --search-recursively /home/reception/mothermother --summary=Imported from https://mothermother.fandom.com (END - exit=0)
[10:27:45] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[10:28:04] <icinga-miraheze>	 PROBLEM - db151 Backups SQL on db151 is WARNING: FILE_AGE WARNING: /var/log/db-backups/db-backups/db-backups.log is 1382552 seconds old and 789936 bytes
[12:20:23] <MacFanBot>	 [02dns] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/dns/commit/4f6567097e4e17df71a1f4ec40306433a1681142
[12:20:23] <MacFanBot>	 02dns/03main 07paladox 034f65670 Increase tcp/udp threads to 4
[12:21:17] <miraheze-github>	 miraheze/dns - paladox the build passed.
[12:55:03] <MacFanBot>	 [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/fd0d004a5f09a828ba6445803dcc553be66c5c89
[12:55:03] <MacFanBot>	 02puppet/03main 07paladox 03fd0d004 Update puppetlabs apt key
[12:57:52] <MacFanBot>	 [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/7c8ac5be2e6dc475a4d42e6f099e4bd8ad75c619
[12:57:52] <MacFanBot>	 02puppet/03main 07paladox 037c8ac5b Fix key name
[12:58:01] <icinga-miraheze>	 PROBLEM - bast181 Puppet on bast181 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.asc]
[12:58:06] <icinga-miraheze>	 PROBLEM - reports171 Puppet on reports171 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.asc]
[12:58:33] <icinga-miraheze>	 PROBLEM - puppet181 Puppet on puppet181 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.asc]
[12:58:40] <icinga-miraheze>	 PROBLEM - db161 Puppet on db161 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.asc]
[12:59:29] <icinga-miraheze>	 PROBLEM - db171 Puppet on db171 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.asc]
[12:59:39] <icinga-miraheze>	 PROBLEM - swiftobject181 Puppet on swiftobject181 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.asc]
[12:59:46] <icinga-miraheze>	 PROBLEM - mw182 Puppet on mw182 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.asc]
[13:00:04] <icinga-miraheze>	 PROBLEM - mem201 Puppet on mem201 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.asc]
[13:00:29] <icinga-miraheze>	 PROBLEM - mw202 Puppet on mw202 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.asc]
[13:00:34] <icinga-miraheze>	 RECOVERY - puppet181 Puppet on puppet181 is OK: OK: Puppet is currently enabled, last run 0 seconds ago with 0 failures
[13:00:41] <icinga-miraheze>	 RECOVERY - db161 Puppet on db161 is OK: OK: Puppet is currently enabled, last run 12 seconds ago with 0 failures
[13:01:15] <icinga-miraheze>	 PROBLEM - rdb151 Puppet on rdb151 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.asc]
[13:01:23] <icinga-miraheze>	 PROBLEM - mem161 Puppet on mem161 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.asc]
[13:01:28] <icinga-miraheze>	 RECOVERY - db171 Puppet on db171 is OK: OK: Puppet is currently enabled, last run 58 seconds ago with 0 failures
[13:01:52] <icinga-miraheze>	 RECOVERY - mem161 Puppet on mem161 is OK: OK: Puppet is currently enabled, last run 12 seconds ago with 0 failures
[13:02:02] <icinga-miraheze>	 RECOVERY - bast181 Puppet on bast181 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[13:02:04] <icinga-miraheze>	 RECOVERY - mem201 Puppet on mem201 is OK: OK: Puppet is currently enabled, last run 25 seconds ago with 0 failures
[13:02:20] <icinga-miraheze>	 RECOVERY - mw202 Puppet on mw202 is OK: OK: Puppet is currently enabled, last run 24 seconds ago with 0 failures
[13:02:23] <icinga-miraheze>	 RECOVERY - mw182 Puppet on mw182 is OK: OK: Puppet is currently enabled, last run 27 seconds ago with 0 failures
[14:09:30] <MirahezeRelay>	 <benjamincurry04103> [1/3] I’ll teach 10 people to earn $30k or more within 72 hours but you will pay me 10% of your profit when you receive it. Note only interested people should apply, drop a message let's get started by asking (HOW) 
[14:09:30] <MirahezeRelay>	 <benjamincurry04103> [2/3] Telegram 
[14:09:30] <MirahezeRelay>	 <benjamincurry04103> [3/3] @Official_Benjamincurry00
[15:28:13] <MacFanBot>	 [02puppet] 07paladox created 03paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/puppet/commit/11b0d95f48d7
[15:28:13] <MacFanBot>	 02puppet/03paladox-patch-1 07paladox 0311b0d95 cache::haproxy: set http-keep-alive in `listen tls`
[15:28:17] <MacFanBot>	 [02puppet] 07paladox opened pull request #4320: cache::haproxy: set http-keep-alive in `listen tls` (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4320
[15:28:23] <MacFanBot>	 [02puppet] 07coderabbitai[bot] commented on pull request #4320: --- […] 13https://github.com/miraheze/puppet/pull/4320#issuecomment-2797235221
[15:28:38] <MacFanBot>	 [02puppet] 07paladox merged pull request #4320: cache::haproxy: set http-keep-alive in `listen tls` (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4320
[15:28:40] <MacFanBot>	 [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/c23656d45720c0f63f28d41bddf3083b78834b2f
[15:28:42] <MacFanBot>	 [02puppet] 07paladox 04deleted 03paladox-patch-1 at 0311b0d95 13https://api.github.com/repos/miraheze/puppet/commit/11b0d95
[15:28:43] <MacFanBot>	 02puppet/03main 07paladox 03c23656d cache::haproxy: set http-keep-alive in `listen tls` (#4320)
[15:56:34] <icinga-miraheze>	 PROBLEM - duofur.rosebyte.xyz - Cloudflare on sslhost is CRITICAL: Name or service not knownHTTP CRITICAL - Unable to open TCP socket
[16:26:13] <icinga-miraheze>	 PROBLEM - matomo151 Current Load on matomo151 is CRITICAL: LOAD CRITICAL - total load average: 8.06, 6.89, 5.76
[16:28:13] <icinga-miraheze>	 PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 7.38, 7.07, 5.97
[16:30:13] <icinga-miraheze>	 RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 6.09, 6.67, 5.95
[16:46:53] <icinga-miraheze>	 PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 7.81, 6.86, 6.41
[16:50:43] <icinga-miraheze>	 RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 6.04, 6.59, 6.42
[17:02:14] <icinga-miraheze>	 PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 7.70, 7.09, 6.74
[17:06:13] <icinga-miraheze>	 RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 6.29, 6.45, 6.54
[17:28:13] <icinga-miraheze>	 PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 7.28, 7.00, 6.68
[17:30:13] <icinga-miraheze>	 RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 5.80, 6.79, 6.66
[17:34:24] <MacFanBot>	 [02MirahezeMagic] 07Reception123 created 03Reception123-patch-3 (+1 new commit) 13https://github.com/miraheze/MirahezeMagic/commit/0475be301a52
[17:34:24] <MacFanBot>	 02MirahezeMagic/03Reception123-patch-3 07Reception123 030475be3 send rangeblocked logged out users to account creation form T13466
[17:34:29] <MacFanBot>	 [02MirahezeMagic] 07Reception123 opened pull request #566: send rangeblocked logged out users to account creation form T13466 (03main...03Reception123-patch-3) 13https://github.com/miraheze/MirahezeMagic/pull/566
[17:34:35] <MacFanBot>	 [02MirahezeMagic] 07coderabbitai[bot] commented on pull request #566: --- […] 13https://github.com/miraheze/MirahezeMagic/pull/566#issuecomment-2797610327
[17:35:36] <MacFanBot>	 [02MirahezeMagic] 07Reception123 pushed 1 new commit to 03Reception123-patch-3 13https://github.com/miraheze/MirahezeMagic/commit/542cc98ded9954371c7505854c7ab1b62299ebe8
[17:35:36] <MacFanBot>	 02MirahezeMagic/03Reception123-patch-3 07Reception123 03542cc98 wrong file
[17:36:23] <MacFanBot>	 [02MirahezeMagic] 07Reception123 pushed 1 new commit to 03Reception123-patch-3 13https://github.com/miraheze/MirahezeMagic/commit/c6e3ca977f41238b1bfc969a72f257abc4519bb3
[17:36:23] <MacFanBot>	 02MirahezeMagic/03Reception123-patch-3 07Reception123 03c6e3ca9 Update en.json
[17:36:30] <MacFanBot>	 [02MirahezeMagic] 07Reception123 pushed 1 new commit to 03Reception123-patch-3 13https://github.com/miraheze/MirahezeMagic/commit/bc412519f7141cd8c76d6666fa3e18c170a55054
[17:36:30] <MacFanBot>	 02MirahezeMagic/03Reception123-patch-3 07Reception123 03bc41251 Update Main.php
[17:42:10] <miraheze-github>	 miraheze/MirahezeMagic - Reception123 the build has errored.
[17:50:08] <MacFanBot>	 [02MirahezeMagic] 07Reception123 pushed 1 new commit to 03main 13https://github.com/miraheze/MirahezeMagic/commit/41ccdf4df7626045e6d597fa0750fe5511fc24e0
[17:50:08] <MacFanBot>	 02MirahezeMagic/03main 07Reception123 0341ccdf4 send rangeblocked logged out users to account creation form T13466 (#566)
[17:50:10] <MirahezeLSBot>	 !log [reception@mwtask181] starting deploy of {'l10n': True, 'versions': '1.43', 'upgrade_extensions': 'MirahezeMagic'} to all
[17:50:10] <MacFanBot>	 [02MirahezeMagic] 07Reception123 merged pull request #566: send rangeblocked logged out users to account creation form T13466 (03main...03Reception123-patch-3) 13https://github.com/miraheze/MirahezeMagic/pull/566
[17:50:12] <MacFanBot>	 [02MirahezeMagic] 07Reception123 04deleted 03Reception123-patch-3 at 03bc41251 13https://api.github.com/repos/miraheze/MirahezeMagic/commit/bc41251
[17:50:12] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:55:27] <MacFanBot>	 [02ssl] 07Reception123 pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/690b99cffaf5fa33f2d2f04ff4bace4b9906e686
[17:55:28] <MacFanBot>	 02ssl/03main 07Reception123 03690b99c ultiblocks to ampmod redirect per T13493
[17:56:05] <miraheze-github>	 miraheze/MirahezeMagic - Reception123 the build has errored.
[17:57:54] <MirahezeLSBot>	 !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=ultiblockswiki --new=ampmodwiki --user=Reception123 (END - exit=0)
[17:57:56] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:58:24] <MirahezeLSBot>	 !log [reception@mwtask181] finished deploy of {'l10n': True, 'versions': '1.43', 'upgrade_extensions': 'MirahezeMagic'} to all - SUCCESS in 493s
[17:58:26] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:58:54] <MacFanBot>	 [02ManageWiki] 07Universal-Omega pushed 1 new commit to 03types 13https://github.com/miraheze/ManageWiki/commit/5a9dbebc02a5eaf22ae34adcd0128565f1dce61c
[17:58:54] <MacFanBot>	 02ManageWiki/03types 07CosmicAlpha 035a9dbeb Some cleanup + type hint callbacks
[18:01:28] <miraheze-github>	 miraheze/ManageWiki - Universal-Omega the build has errored.
[18:02:54] <MacFanBot>	 [02ManageWiki] 07Universal-Omega pushed 1 new commit to 03types 13https://github.com/miraheze/ManageWiki/commit/88e76638da7c6c48d7c3ec34efbdc51b070f11ff
[18:02:54] <MacFanBot>	 02ManageWiki/03types 07CosmicAlpha 0388e7663 Fix
[18:03:10] <MirahezeLSBot>	 !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=untitledfurrygamewiki --new=twistingruinwiki --user=Reception123 (END - exit=0)
[18:03:13] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[18:05:11] <MirahezeLSBot>	 !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php CreateWiki:DeleteWiki --wiki=loginwiki --delete --deletewiki paneidoversewiki (END - exit=0)
[18:05:14] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[18:05:50] <MirahezeRelay>	 <reception123> !log deleted and dropped paneidoversewiki
[18:05:53] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[18:07:51] <miraheze-github>	 miraheze/ManageWiki - Universal-Omega the build passed.
[18:13:40] <icinga-miraheze>	 PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 16.50, 8.88, 5.34
[18:39:39] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.27, 19.65, 16.15
[18:41:36] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 12.15, 16.94, 15.59
[18:43:24] <icinga-miraheze>	 PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 1.31, 7.62, 11.53
[18:45:24] <icinga-miraheze>	 RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 0.63, 5.26, 10.19
[18:54:27] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.21, 18.44, 16.67
[18:56:27] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 17.09, 17.94, 16.71
[19:12:06] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.02, 20.12, 18.00
[19:14:03] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 18.00, 19.08, 17.86
[19:31:07] <icinga-miraheze>	 PROBLEM - www.astrowiki.fr - Cloudflare on sslhost is CRITICAL: Name or service not knownHTTP CRITICAL - Unable to open TCP socket
[19:36:10] <icinga-miraheze>	 RECOVERY - echoverse.wiki - Cloudflare on sslhost is OK: OK - Certificate 'echoverse.wiki' will expire on Fri 27 Jun 2025 03:12:21 AM GMT +0000.