[00:03:45] <MirahezeLSBot>	 !log [void@puppet181] renewed wikitide.net wildcard cert
[00:03:47] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[00:06:23] <icinga-miraheze>	 RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 57749MiB (12% inode=99%);
[00:07:00] <icinga-miraheze>	 RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 57351MiB (12% inode=99%);
[00:07:04] <icinga-miraheze>	 RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 57034MiB (12% inode=99%);
[00:08:40] <icinga-miraheze>	 RECOVERY - graylog161 graylog tls port 12210 ssl cert check on graylog161 is OK: OK - Certificate 'wikitide.net' will expire on Tue Oct 14 23:03:38 2025 +0000.TCP OK - 0.056 second response time on localhost port 12210
[01:14:17] <miraheze-github>	 miraheze/dns - The-Voidwalker the build passed.
[01:40:12] <icinga-miraheze>	 PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is CRITICAL: CRITICAL - NGINX Error Rate is 62%
[01:42:12] <icinga-miraheze>	 RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 32%
[02:02:05] <MirahezeRelay>	 <orduin> !log restart swift-proxy.service on swiftproxy*
[02:02:08] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[02:24:33] <icinga-miraheze>	 RECOVERY - swift-lb.wikitide.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wikitide.net' will expire on Wed 15 Oct 2025 12:16:28 AM GMT +0000.
[02:24:55] <MirahezeRelay>	 <cosmicalpha> !log restart nginx on swiftproxy161
[02:24:58] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[02:26:53] <MacFanBot>	 [02puppet] 07Universal-Omega created 03revert-4427-paladox-patch-11 (+1 new commit) 13https://github.com/miraheze/puppet/commit/8098c06662cf
[02:26:53] <MacFanBot>	 02puppet/03revert-4427-paladox-patch-11 07CosmicAlpha 038098c06 Revert "phorge: Fix path to phorge-backup.log for monitoring (#4427)"…
[02:26:57] <MacFanBot>	 [02puppet] 07Universal-Omega opened pull request #4450: Revert "phorge: Fix path to phorge-backup.log for monitoring" (03main...03revert-4427-paladox-patch-11) 13https://github.com/miraheze/puppet/pull/4450
[02:27:03] <MacFanBot>	 [02puppet] 07coderabbitai[bot] commented on pull request #4450: <details open="true"> […] 13https://github.com/miraheze/puppet/pull/4450#issuecomment-3082223683
[02:27:05] <MacFanBot>	 [02puppet] 07Universal-Omega merged pull request #4450: Revert "phorge: Fix path to phorge-backup.log for monitoring" (03main...03revert-4427-paladox-patch-11) 13https://github.com/miraheze/puppet/pull/4450
[02:27:07] <MacFanBot>	 [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/881abfe1ecee8f7354e8435b5724c465d19de7e4
[02:27:09] <MacFanBot>	 [02puppet] 07Universal-Omega 04deleted 03revert-4427-paladox-patch-11 at 038098c06 13https://api.github.com/repos/miraheze/puppet/commit/8098c06
[02:27:11] <MacFanBot>	 02puppet/03main 07CosmicAlpha 03881abfe Revert "phorge: Fix path to phorge-backup.log for monitoring" (#4450)…
[02:27:23] <MacFanBot>	 [02puppet] 07Universal-Omega created 03revert-4426-paladox-patch-11 (+1 new commit) 13https://github.com/miraheze/puppet/commit/5ef58f624417
[02:27:23] <MacFanBot>	 02puppet/03revert-4426-paladox-patch-11 07CosmicAlpha 035ef58f6 Revert "puppetdb: fix location to puppetdb-backup puppetdb.log for monitoring…"…
[02:27:27] <MacFanBot>	 [02puppet] 07Universal-Omega opened pull request #4451: Revert "puppetdb: fix location to puppetdb-backup puppetdb.log for monitoring" (03main...03revert-4426-paladox-patch-11) 13https://github.com/miraheze/puppet/pull/4451
[02:27:33] <MacFanBot>	 [02puppet] 07coderabbitai[bot] commented on pull request #4451: <details open="true"> […] 13https://github.com/miraheze/puppet/pull/4451#issuecomment-3082224483
[02:27:35] <MacFanBot>	 [02puppet] 07Universal-Omega merged pull request #4451: Revert "puppetdb: fix location to puppetdb-backup puppetdb.log for monitoring" (03main...03revert-4426-paladox-patch-11) 13https://github.com/miraheze/puppet/pull/4451
[02:27:35] <MacFanBot>	 [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/5c910da398ff6a5447a0580b6a6b31a359a2744b
[02:27:37] <MacFanBot>	 02puppet/03main 07CosmicAlpha 035c910da Revert "puppetdb: fix location to puppetdb-backup puppetdb.log for monitoring" (#4451)…
[02:27:39] <MacFanBot>	 [02puppet] 07Universal-Omega 04deleted 03revert-4426-paladox-patch-11 at 035ef58f6 13https://api.github.com/repos/miraheze/puppet/commit/5ef58f6
[02:31:24] <MacFanBot>	 [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/7826cf0e1f2e6d8d5c4b1608b2e2c531e3fb4c7e
[02:31:24] <MacFanBot>	 02puppet/03main 07CosmicAlpha 037826cf0 Update mattermost to 10.10.1
[02:33:54] <MirahezeRelay>	 <cosmicalpha> !log restart nginx on mattermost1
[02:33:57] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[02:39:40] <MirahezeRelay>	 <cosmicalpha> !log restart nginx on swiftproxy171
[02:39:43] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[02:42:55] <icinga-miraheze>	 RECOVERY - puppet181 Backups PuppetDB on puppet181 is OK: FILE_AGE OK: /var/log/puppetdb-backup/puppetdb.log is 178954 seconds old and 395 bytes
[02:43:58] <icinga-miraheze>	 RECOVERY - phorge171 Backups Phorge Static on phorge171 is OK: FILE_AGE OK: /var/log/phorge-backup/phorge-backup.log is 172920 seconds old and 382 bytes
[02:44:47] <icinga-miraheze>	 PROBLEM - matomo151 Check unit status of matomo-archiver-3 on matomo151 is CRITICAL: CRITICAL: Status of the systemd unit matomo-archiver-3
[02:47:52] <MacFanBot>	 [02mw-config] 07Universal-Omega left a file comment in pull request #6026 032c1cb3a: This whole block can probably be simplified as `$wgDefaultUserOptions['usecodemirror'] = (int)$wmgCodeMirrorEnableDefault`. 13https://github.com/miraheze/mw-config/pull/6026#discussion_r2212036929
[02:53:08] <MacFanBot>	 [02mw-config] 07Universal-Omega left a review on pull request #6026: We may also be able to add a new section just for default user options, and use a checkmatrix type that can support all those that need into. And under or above it support those with strings to add these configs more extensively. 13https://github.com/miraheze/mw-config/pull/6026#pullrequestreview-3027534494
[03:28:18] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega created 03readonly (+1 new commit) 13https://github.com/miraheze/MachineTranslation/commit/e7b01a8c47db
[03:28:18] <MacFanBot>	 02MachineTranslation/03readonly 07CosmicAlpha 03e7b01a8 Use property promotion and readonly properties
[03:28:26] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega opened pull request #17: Use property promotion and readonly properties (03main...03readonly) 13https://github.com/miraheze/MachineTranslation/pull/17
[03:28:32] <MacFanBot>	 [02MachineTranslation] 07coderabbitai[bot] commented on pull request #17: <details open="true"> […] 13https://github.com/miraheze/MachineTranslation/pull/17#issuecomment-3082324077
[03:28:44] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega pushed 1 new commit to 03readonly 13https://github.com/miraheze/MachineTranslation/commit/4aa7bd2f308f546c98afcd34c6f99b4f26091804
[03:28:44] <MacFanBot>	 02MachineTranslation/03readonly 07CosmicAlpha 034aa7bd2 Fix
[03:30:01] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega pushed 1 new commit to 03readonly 13https://github.com/miraheze/MachineTranslation/commit/1a2f64e4c4ca9e5841c10d25c9020b45fe7305c5
[03:30:01] <MacFanBot>	 02MachineTranslation/03readonly 07CosmicAlpha 031a2f64e Update
[03:31:16] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega pushed 1 new commit to 03readonly 13https://github.com/miraheze/MachineTranslation/commit/2da8eb8dbac06ac9b0d01a4f2fbfafc61f0b9ed3
[03:31:17] <MacFanBot>	 02MachineTranslation/03readonly 07CosmicAlpha 032da8eb8 Update
[03:32:02] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega pushed 1 new commit to 03readonly 13https://github.com/miraheze/MachineTranslation/commit/f83fb999f7b5fcb8fe6c89c077dbd03d66fd8087
[03:32:02] <MacFanBot>	 02MachineTranslation/03readonly 07CosmicAlpha 03f83fb99 Update
[03:34:56] <miraheze-github>	 miraheze/MachineTranslation - Universal-Omega the build has errored.
[03:35:48] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega created 03phan (+1 new commit) 13https://github.com/miraheze/MachineTranslation/commit/e394e47b81ad
[03:35:48] <MacFanBot>	 02MachineTranslation/03phan 07CosmicAlpha 03e394e47 Update phan config
[03:35:55] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega opened pull request #18: Update phan config (03main...03phan) 13https://github.com/miraheze/MachineTranslation/pull/18
[03:36:01] <MacFanBot>	 [02MachineTranslation] 07coderabbitai[bot] commented on pull request #18: <details open="true"> […] 13https://github.com/miraheze/MachineTranslation/pull/18#issuecomment-3082335250
[03:36:42] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega pushed 1 new commit to 03phan 13https://github.com/miraheze/MachineTranslation/commit/af0c7e32e9858a4d7040335c1e6829b334a9b2ab
[03:36:42] <MacFanBot>	 02MachineTranslation/03phan 07CosmicAlpha 03af0c7e3 Add
[03:37:32] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega pushed 1 new commit to 03readonly 13https://github.com/miraheze/MachineTranslation/commit/cb72de51a0149437848605298b01252b2d254149
[03:37:32] <MacFanBot>	 02MachineTranslation/03readonly 07CosmicAlpha 03cb72de5 Bump
[03:38:17] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega closed pull request #11: Validate ServiceConfig (03main...03validate) 13https://github.com/miraheze/MachineTranslation/pull/11
[03:38:21] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega 04deleted 03validate at 0380771dc 13https://api.github.com/repos/miraheze/MachineTranslation/commit/80771dc
[03:39:28] <miraheze-github>	 miraheze/MachineTranslation - Universal-Omega the build has errored.
[03:41:58] <miraheze-github>	 miraheze/MachineTranslation - Universal-Omega the build passed.
[03:43:58] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega pushed 1 new commit to 03phan 13https://github.com/miraheze/MachineTranslation/commit/b4a19297eddbc6712a5e79883bd15fc3c4f598c8
[03:43:58] <MacFanBot>	 02MachineTranslation/03phan 07CosmicAlpha 03b4a1929 Update
[03:45:37] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega merged pull request #17: Use property promotion and readonly properties (03main...03readonly) 13https://github.com/miraheze/MachineTranslation/pull/17
[03:45:38] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/MachineTranslation/commit/74dc835557a5038d8fa4377337ac2b84f4dc5770
[03:45:38] <MacFanBot>	 02MachineTranslation/03main 07CosmicAlpha 0374dc835 Use property promotion and readonly properties
[03:45:38] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega 04deleted 03readonly at 03cb72de5 13https://api.github.com/repos/miraheze/MachineTranslation/commit/cb72de5
[03:45:56] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega pushed 1 new commit to 03phan 13https://github.com/miraheze/MachineTranslation/commit/b418e2dd7fbcaea9151c8f55510a8bf167e2e518
[03:45:56] <MacFanBot>	 02MachineTranslation/03phan 07CosmicAlpha 03b418e2d Merge branch 'main' into phan
[03:48:35] <miraheze-github>	 miraheze/MachineTranslation - Universal-Omega the build has errored.
[03:50:03] <miraheze-github>	 miraheze/MachineTranslation - Universal-Omega the build passed.
[03:54:05] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega pushed 1 new commit to 03phan 13https://github.com/miraheze/MachineTranslation/commit/0af3f4d5e2ef4c3f7985309d803e0b595a485fee
[03:54:05] <MacFanBot>	 02MachineTranslation/03phan 07CosmicAlpha 030af3f4d Update
[03:56:37] <miraheze-github>	 miraheze/MachineTranslation - Universal-Omega the build has errored.
[04:07:38] <MacFanBot>	 [02MachineTranslation] 07Universal-Omega pushed 1 new commit to 03phan 13https://github.com/miraheze/MachineTranslation/commit/3a6bf6f366267498146238797129403ac7de7228
[04:07:38] <MacFanBot>	 02MachineTranslation/03phan 07CosmicAlpha 033a6bf6f Update
[04:10:25] <miraheze-github>	 miraheze/MachineTranslation - Universal-Omega the build has errored.
[07:09:18] <MirahezeLSBot>	 !log [somerandomdeveloper@test151] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php ManageWiki:ToggleExtension --wiki=exttestwikibeta --name=ratepage --disable (END - exit=0)
[07:09:21] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[08:02:47] <icinga-miraheze>	 RECOVERY - matomo151 Check unit status of matomo-archiver-3 on matomo151 is OK: OK: Status of the systemd unit matomo-archiver-3
[09:59:04] <icinga-miraheze>	 PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 49830MiB (10% inode=99%);
[11:52:23] <icinga-miraheze>	 PROBLEM - cp191 Disk Space on cp191 is WARNING: DISK WARNING - free space: / 49928MiB (10% inode=99%);
[12:07:00] <icinga-miraheze>	 PROBLEM - cp171 Disk Space on cp171 is WARNING: DISK WARNING - free space: / 49853MiB (10% inode=99%);
[12:16:42] <miraheze-github>	 miraheze/GlobalNewFiles - translatewiki the build has errored.
[12:16:51] <miraheze-github>	 miraheze/DataDump - translatewiki the build passed.
[12:16:57] <miraheze-github_>	 miraheze/CreateWiki - translatewiki the build has errored.
[12:17:04] <miraheze-github>	 miraheze/IncidentReporting - translatewiki the build passed.
[12:17:11] <miraheze-github_>	 miraheze/ManageWiki - translatewiki the build has errored.
[12:17:59] <miraheze-github>	 miraheze/ImportDump - translatewiki the build has errored.
[12:20:31] <miraheze-github>	 miraheze/MirahezeMagic - translatewiki the build has errored.
[12:20:41] <miraheze-github>	 miraheze/RemovePII - translatewiki the build passed.
[12:27:46] <MirahezeLSBot>	 !log [somerandomdeveloper@test151] starting deploy of {'versions': '1.44', 'upgrade_extensions': 'LanguageSelector'} to test151
[12:27:47] <MirahezeLSBot>	 !log [somerandomdeveloper@test151] finished deploy of {'versions': '1.44', 'upgrade_extensions': 'LanguageSelector'} to test151 - SUCCESS in 1s
[12:27:49] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[12:27:53] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[12:34:28] <miraheze-github>	 miraheze/DiscordNotifications - translatewiki the build passed.