[00:10:56] <icinga-miraheze>	 RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 54730MiB (12% inode=99%);
[00:21:30] <MacFanBot>	 [02mw-config] 07BlankEclair opened pull request #5778: T13058: Add $wgExportAllowListContributors to ManageWiki (07miraheze:03master...07BlankEclair:03wgExportAllowListContributors) 13https://github.com/miraheze/mw-config/pull/5778
[00:22:21] <miraheze-github>	 miraheze/mw-config - BlankEclair the build has errored.
[00:25:54] <MirahezeRelay>	 <pixldev> Hi claire
[00:27:07] <BlankEclair>	 meow
[00:29:27] <BlankEclair>	 > [07/01/2025 11:10] <icinga-miraheze> RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 54730MiB (12% inode=99%);
[00:29:30] <BlankEclair>	 > inode=99%
[00:29:33] <BlankEclair>	 > DISK OK
[00:29:35] <BlankEclair>	 yeah sure
[00:30:44] <BlankEclair>	 why do i keep finding security vulnerabilities :((
[00:33:45] <MirahezeRelay>	 <pixldev> What is it now
[00:33:59] <BlankEclair>	 a secret (for now)
[00:34:11] <MirahezeRelay>	 <pixldev> WikiFourm
[02:50:25] <icinga-miraheze>	 PROBLEM - edapedia.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'edapedia.net' expires in 14 day(s) (Wed 22 Jan 2025 02:47:33 AM GMT +0000).
[02:50:38] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/0e2be65f16189e63b21d8db0d2d8b3ae92932939
[02:50:38] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 030e2be65 Bot: Update SSL cert for edapedia.net
[03:19:20] <icinga-miraheze>	 RECOVERY - edapedia.net - LetsEncrypt on sslhost is OK: OK - Certificate 'edapedia.net' will expire on Mon 07 Apr 2025 01:52:00 AM GMT +0000.
[03:25:26] <icinga-miraheze>	 PROBLEM - corru.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'corru.wiki' expires in 14 day(s) (Wed 22 Jan 2025 02:58:47 AM GMT +0000).
[03:25:38] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/45f806949f9537023b3644009f2eae8d158de8d5
[03:25:39] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 0345f8069 Bot: Update SSL cert for corru.wiki
[03:54:52] <icinga-miraheze>	 RECOVERY - corru.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'corru.wiki' will expire on Mon 07 Apr 2025 02:27:02 AM GMT +0000.
[03:58:56] <icinga-miraheze>	 PROBLEM - cp37 Disk Space on cp37 is WARNING: DISK WARNING - free space: / 49855MiB (10% inode=99%);
[11:29:51] <icinga-miraheze>	 PROBLEM - wikigenius.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wikigenius.org' expires in 14 day(s) (Wed 22 Jan 2025 11:09:01 AM GMT +0000).
[12:16:37] <icinga-miraheze>	 PROBLEM - wiki.walkscape.app - Cloudflare on sslhost is CRITICAL: No address associated with hostnameHTTP CRITICAL - Unable to open TCP socket
[14:13:52] <icinga-miraheze>	 RECOVERY - steem.wiki - Cloudflare on sslhost is OK: OK - Certificate 'steem.wiki' will expire on Wed 26 Mar 2025 07:34:06 PM GMT +0000.
[14:23:34] <icinga-miraheze>	 PROBLEM - lostmediawiki.ru - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'lostmediawiki.ru' expires in 14 day(s) (Wed 22 Jan 2025 02:19:38 PM GMT +0000).
[14:23:45] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/0151088fe47b38ba011cbce389abd97039065722
[14:23:45] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 030151088 Bot: Update SSL cert for lostmediawiki.ru
[14:50:36] <icinga-miraheze>	 PROBLEM - otherkin.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'otherkin.wiki' expires in 14 day(s) (Wed 22 Jan 2025 02:41:57 PM GMT +0000).
[14:50:47] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/3d155ad4a29380a88f817f969bc65d64eebf990f
[14:50:47] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 033d155ad Bot: Update SSL cert for otherkin.wiki
[14:52:37] <icinga-miraheze>	 RECOVERY - lostmediawiki.ru - LetsEncrypt on sslhost is OK: OK - Certificate 'lostmediawiki.ru' will expire on Mon 07 Apr 2025 01:25:09 PM GMT +0000.
[15:05:53] <icinga-miraheze>	 PROBLEM - beidipedia.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'beidipedia.com' expires in 14 day(s) (Wed 22 Jan 2025 02:50:31 PM GMT +0000).
[15:06:04] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/e7fed346ed441fd9dd5afeb358436d34f699bb72
[15:06:05] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 03e7fed34 Bot: Update SSL cert for beidipedia.com
[15:19:42] <icinga-miraheze>	 RECOVERY - otherkin.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'otherkin.wiki' will expire on Mon 07 Apr 2025 01:52:12 PM GMT +0000.
[15:35:36] <icinga-miraheze>	 RECOVERY - beidipedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'beidipedia.com' will expire on Mon 07 Apr 2025 02:07:29 PM GMT +0000.
[17:33:54] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/59b0cba5cb03d3bfd3861695daf57bbdd3bfbfae
[17:33:54] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 0359b0cba Bot: Update SSL cert for wikigenius.org
[17:57:38] <icinga-miraheze>	 RECOVERY - wikigenius.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wikigenius.org' will expire on Mon 07 Apr 2025 04:35:18 PM GMT +0000.
[17:59:12] <MacFanBot>	 [02ssl] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/2adadbd384be6cf41fc1cf262a3b7f080ea024c8
[17:59:12] <MacFanBot>	 02ssl/03master 07MacFan4000 032adadbd Update certs.yaml
[17:59:45] <MacFanBot>	 [02ssl] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/ca36474ae5163f1ec7f9af78254502dcd8cda8d9
[17:59:45] <MacFanBot>	 02ssl/03master 07MacFan4000 03ca36474 Delete certificates/wiki.zergmafia.top.crt
[18:02:20] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki job queue has more than 2000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[18:14:12] <MacFanBot>	 [02ssl] 07MacFan4000 created 03MacFan4000-patch-1 (+1 new commit) 13https://github.com/miraheze/ssl/commit/59c6c38a2efb
[18:14:12] <MacFanBot>	 02ssl/03MacFan4000-patch-1 07MacFan4000 0359c6c38 Update certs.yaml
[18:14:28] <MacFanBot>	 [02ssl] 07MacFan4000 opened pull request #817: remove expired/non-pointed domains (03master...03MacFan4000-patch-1) 13https://github.com/miraheze/ssl/pull/817
[18:15:16] <MacFanBot>	 [02ssl] 07MacFan4000 pushed 1 new commit to 03MacFan4000-patch-1 13https://github.com/miraheze/ssl/commit/7f494667ffca67e17071dfa0df56a3e72fe448ce
[18:15:17] <MacFanBot>	 02ssl/03MacFan4000-patch-1 07MacFan4000 037f49466 Delete certificates/chobots.wiki.crt
[18:15:34] <MacFanBot>	 [02ssl] 07MacFan4000 pushed 1 new commit to 03MacFan4000-patch-1 13https://github.com/miraheze/ssl/commit/f158fb51b3e06affcb9105a67ce62bf55a73b140
[18:15:34] <MacFanBot>	 02ssl/03MacFan4000-patch-1 07MacFan4000 03f158fb5 Delete certificates/quadball.wiki.crt
[18:15:37] <MacFanBot>	 [02ssl] 07coderabbitai[bot] commented on pull request #817: --- […] 13https://github.com/miraheze/ssl/pull/817#issuecomment-2575945798
[18:17:45] <MacFanBot>	 [02ssl] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/e11647140b1da334dd956d600abec4cab64b488b
[18:17:46] <MacFanBot>	 02ssl/03master 07MacFan4000 03e116471 remove expired/non-pointed domains (#817)
[18:17:49] <MacFanBot>	 [02ssl] 07MacFan4000 merged pull request #817: remove expired/non-pointed domains (03master...03MacFan4000-patch-1) 13https://github.com/miraheze/ssl/pull/817
[18:17:50] <MacFanBot>	 [02ssl] 07MacFan4000 04deleted 03MacFan4000-patch-1 at 03f158fb5 13https://github.com/miraheze/ssl/commit/f158fb5
[18:18:45] <MacFanBot>	 [02dns] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/dns/commit/1cebf957ed37bb4794294e312a5965cdb66cb02f
[18:18:45] <MacFanBot>	 02dns/03master 07MacFan4000 031cebf95 Delete zones/quadball.wiki
[18:19:40] <miraheze-github>	 miraheze/dns - MacFan4000 the build passed.
[18:27:20] <icinga-miraheze>	 [Grafana] RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[19:02:20] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki job queue has more than 2000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[19:18:54] <MirahezeLSBot>	 !log [paladox@mwtask181] starting deploy of {'upgrade_world': True, 'versions': '1.42', 'upgrade_extensions': 'all', 'upgrade_skins': 'all'} to all
[19:18:55] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:22:20] <icinga-miraheze>	 [Grafana] RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[19:22:42] <MirahezeLSBot>	 !log [paladox@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output.
[19:22:43] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:33:49] <MirahezeLSBot>	 !log [paladox@mwtask181] starting deploy of {'upgrade_world': True, 'versions': '1.42', 'upgrade_extensions': 'all', 'upgrade_skins': 'all'} to all
[19:33:50] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:33:57] <MirahezeLSBot>	 !log [paladox@mwtask181] starting deploy of {'upgrade_world': True, 'versions': '1.42', 'upgrade_extensions': 'all', 'upgrade_skins': 'all'} to mwtask181
[19:33:59] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:42:58] <MirahezeLSBot>	 !log [paladox@mwtask181] finished deploy of {'upgrade_world': True, 'versions': '1.42', 'upgrade_extensions': 'all', 'upgrade_skins': 'all'} to mwtask181 - SUCCESS in 540s
[19:42:59] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:49:51] <MirahezeLSBot>	 !log [paladox@mwtask181] starting deploy of {'upgrade_vendor': True, 'versions': '1.42'} to mwtask181
[19:49:52] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:50:13] <MirahezeLSBot>	 !log [paladox@mwtask181] finished deploy of {'upgrade_vendor': True, 'versions': '1.42'} to mwtask181 - SUCCESS in 22s
[19:50:15] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:55:49] <MirahezeLSBot>	 !log [paladox@mwtask181] starting deploy of {'upgrade_world': True, 'upgrade_vendor': True, 'versions': '1.42', 'upgrade_extensions': 'all', 'upgrade_skins': 'all'} to mwtask181
[19:55:50] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:56:02] <MirahezeLSBot>	 !log [paladox@mwtask181] starting deploy of {'upgrade_world': True, 'upgrade_vendor': True, 'versions': '1.42', 'upgrade_extensions': 'all', 'upgrade_skins': 'all'} to all
[19:56:04] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[20:23:34] <MirahezeLSBot>	 !log [paladox@mwtask181] finished deploy of {'upgrade_world': True, 'upgrade_vendor': True, 'versions': '1.42', 'upgrade_extensions': 'all', 'upgrade_skins': 'all'} to all - SUCCESS in 1652s
[20:23:36] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[20:24:51] <icinga-miraheze>	 PROBLEM - grayzonewarfare.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'grayzonewarfare.wiki' expires in 14 day(s) (Wed 22 Jan 2025 08:13:00 PM GMT +0000).
[20:25:03] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/0d6c6b41c4ce2a644e77c2b8e7614e2ff2d3ba32
[20:25:03] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 030d6c6b4 Bot: Update SSL cert for grayzonewarfare.wiki
[20:26:34] <MirahezeLSBot>	 !log [paladox@mwtask181] starting deploy of {'versions': ['1.42', '1.43'], 'upgrade_extensions': 'mw-snapblocks'} to all
[20:26:35] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[20:27:16] <MirahezeLSBot>	 !log [paladox@mwtask181] finished deploy of {'versions': ['1.42', '1.43'], 'upgrade_extensions': 'mw-snapblocks'} to all - SUCCESS in 41s
[20:27:17] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[20:36:17] <icinga-miraheze>	 PROBLEM - osrwiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'osrwiki.org' expires in 14 day(s) (Wed 22 Jan 2025 08:21:57 PM GMT +0000).
[20:36:28] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/5f91f4e1aa0a5f24601f672c503d9c2e1120f3e8
[20:36:28] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 035f91f4e Bot: Update SSL cert for osrwiki.org
[20:54:10] <icinga-miraheze>	 RECOVERY - grayzonewarfare.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'grayzonewarfare.wiki' will expire on Mon 07 Apr 2025 07:26:26 PM GMT +0000.
[21:06:12] <icinga-miraheze>	 RECOVERY - osrwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'osrwiki.org' will expire on Mon 07 Apr 2025 07:37:52 PM GMT +0000.
[22:16:02] <icinga-miraheze>	 PROBLEM - cp37 Disk Space on cp37 is CRITICAL: DISK CRITICAL - free space: / 27204MiB (5% inode=99%);
[22:35:16] <icinga-miraheze>	 PROBLEM - polcompball.net - LetsEncrypt on sslhost is CRITICAL: No address associated with hostnameHTTP CRITICAL - Unable to open TCP socket
[23:23:50] <MirahezeRelay>	 <pixldev, replying to icinga-miraheze> Good to know
[23:33:16] <icinga-miraheze>	 RECOVERY - polcompball.net - LetsEncrypt on sslhost is OK: OK - Certificate 'polcompball.net' will expire on Wed 12 Feb 2025 01:32:10 AM GMT +0000.