[00:01:17] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/97d82b5aa168f1c2293ceebc38be91368ebca832 [00:01:17] 02statichelp/03main 07WikiTideBot 0397d82b5 Bot: Auto-update Tech namespace pages 2025-09-11 00:01:15 [00:08:01] RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 55862MiB (12% inode=99%); [00:08:44] RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 55822MiB (12% inode=99%); [00:09:16] RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 56395MiB (12% inode=99%); [00:36:15] !log [somerandomdeveloper@test151] starting deploy of {'versions': ['1.44', '1.45'], 'upgrade_extensions': 'RobloxAPI'} to test151 [00:36:16] !log [somerandomdeveloper@test151] finished deploy of {'versions': ['1.44', '1.45'], 'upgrade_extensions': 'RobloxAPI'} to test151 - SUCCESS in 1s [00:36:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:36:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:36:42] !log [somerandomdeveloper@test151] starting deploy of {'l10n': True, 'versions': '1.44'} to test151 [00:36:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:37:44] !log [somerandomdeveloper@mwtask181] starting deploy of {'l10n': True, 'versions': '1.44', 'upgrade_extensions': 'RobloxAPI'} to all [00:37:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:41:34] !log [somerandomdeveloper@test151] finished deploy of {'l10n': True, 'versions': '1.44'} to test151 - SUCCESS in 292s [00:41:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:45:19] !log [somerandomdeveloper@mwtask181] finished deploy of {'l10n': True, 'versions': '1.44', 'upgrade_extensions': 'RobloxAPI'} to all - SUCCESS in 454s [00:45:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:47:38] [02mw-config] 07SomeMWDev created 03robloxapi-caching (+1 new commit) 13https://github.com/miraheze/mw-config/commit/ee0e6a5bddc4 [00:47:38] 02mw-config/03robloxapi-caching 07SomeRandomDeveloper 03ee0e6a5 Increase RobloxAPI caching expiries… [00:48:08] !log [somerandomdeveloper@test151] starting deploy of {'config': True} to test151 [00:48:09] !log [somerandomdeveloper@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [00:48:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:48:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:49:38] [02mw-config] 07SomeMWDev 04force-pushed 03robloxapi-caching from 03ee0e6a5 to 03151c1ff 13https://api.github.com/repos/miraheze/mw-config/commits/robloxapi-caching [00:49:38] 02mw-config/03robloxapi-caching 07SomeRandomDeveloper 03151c1ff Increase RobloxAPI caching expiries… [00:50:05] !log [somerandomdeveloper@test151] starting deploy of {'config': True} to test151 [00:50:06] !log [somerandomdeveloper@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [00:50:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:50:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:50:47] !log [universalomega@mwtask181] starting deploy of {'versions': '1.44', 'upgrade_extensions': 'MirahezeMagic'} to all [00:50:49] !log [universalomega@mwtask181] DEPLOY ABORTED: Canary check failed for publictestwiki.com@localhost [00:50:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:50:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:51:06] !log [universalomega@mwtask181] starting deploy of {'versions': '1.44', 'upgrade_extensions': 'MirahezeMagic'} to all [00:51:07] !log [universalomega@mwtask181] finished deploy of {'versions': '1.44', 'upgrade_extensions': 'MirahezeMagic'} to all - SUCCESS in 0s [00:51:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:51:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:51:16] !log [universalomega@mwtask181] starting deploy of {'force_upgrade': True, 'versions': '1.44', 'upgrade_extensions': 'MirahezeMagic'} to all [00:51:19] !log [universalomega@mwtask181] DEPLOY ABORTED: Canary check failed for publictestwiki.com@mw151 [00:51:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:51:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:51:29] !log [universalomega@mwtask181] starting deploy of {'force': True, 'force_upgrade': True, 'versions': '1.44', 'upgrade_extensions': 'MirahezeMagic'} to all [00:51:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:51:33] [02mw-config] 07SomeMWDev opened pull request #6099: Increase RobloxAPI caching expiries (03main...03robloxapi-caching) 13https://github.com/miraheze/mw-config/pull/6099 [00:51:54] !log [universalomega@mwtask181] finished deploy of {'force': True, 'force_upgrade': True, 'versions': '1.44', 'upgrade_extensions': 'MirahezeMagic'} to all - SUCCESS in 25s [00:51:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:52:39] miraheze/mw-config - SomeMWDev the build has errored. [00:54:42] [02mw-config] 07SomeMWDev 04force-pushed 03robloxapi-caching from 03151c1ff to 0369520bb 13https://api.github.com/repos/miraheze/mw-config/commits/robloxapi-caching [00:54:44] 02mw-config/03robloxapi-caching 07SomeRandomDeveloper 0369520bb Increase RobloxAPI caching expiries… [00:55:39] miraheze/mw-config - SomeMWDev the build passed. [00:56:06] [02mw-config] 07SomeMWDev merged pull request #6099: Increase RobloxAPI caching expiries (03main...03robloxapi-caching) 13https://github.com/miraheze/mw-config/pull/6099 [00:56:06] [02mw-config] 07SomeMWDev pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/b89b7bc240b417b352591fbb9a10f16052089ccc [00:56:07] 02mw-config/03main 07SomeRandomDeveloper 03b89b7bc Increase RobloxAPI caching expiries (#6099)… [00:56:07] [02mw-config] 07SomeMWDev 04deleted 03robloxapi-caching at 0369520bb 13https://api.github.com/repos/miraheze/mw-config/commit/69520bb [00:56:34] !log [somerandomdeveloper@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [00:56:35] !log [somerandomdeveloper@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 0s [00:56:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:56:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:56:57] !log [somerandomdeveloper@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [00:57:00] miraheze/mw-config - SomeMWDev the build passed. [00:57:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:57:22] !log [somerandomdeveloper@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 24s [00:57:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:04:33] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.95, 20.58, 13.48 [02:06:33] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 8.50, 16.21, 12.77 [02:11:17] !log [universalomega@mwtask181] starting deploy of {'versions': '1.44', 'upgrade_extensions': 'DynamicPageList4'} to all [02:11:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:11:42] !log [universalomega@mwtask181] finished deploy of {'versions': '1.44', 'upgrade_extensions': 'DynamicPageList4'} to all - SUCCESS in 25s [02:11:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:14:20] [02mw-config] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/0c55fdec6f611a94c117b2185fd67e7730186a4e [02:14:20] 02mw-config/03main 07CosmicAlpha 030c55fde Set default maxCategoryCount for DPL4 to 10 [02:14:36] !log [universalomega@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [02:14:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:14:58] !log [universalomega@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 22s [02:15:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:18:32] miraheze/mw-config - Universal-Omega the build passed. [02:27:57] [02mediawiki-repos] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/mediawiki-repos/commit/d91705e168e1522496f1a20ff767f06e156a2fcc [02:27:57] 02mediawiki-repos/03main 07CosmicAlpha 03d91705e Install CookieConsent [05:56:51] [02mw-config] 07Universal-Omega created 03CookieConsent (+1 new commit) 13https://github.com/miraheze/mw-config/commit/0192c392e907 [05:56:51] 02mw-config/03CookieConsent 07CosmicAlpha 030192c39 Replace CookieWarning with CookieConsent [05:56:58] [02mw-config] 07Universal-Omega opened pull request #6100: Replace CookieWarning with CookieConsent (03main...03CookieConsent) 13https://github.com/miraheze/mw-config/pull/6100 [05:57:51] miraheze/mw-config - Universal-Omega the build passed. [06:02:18] [02mw-config] 07Universal-Omega pushed 1 new commit to 03CookieConsent 13https://github.com/miraheze/mw-config/commit/6db68526541a9560963f48a5cc457588c0aa8dd3 [06:02:20] 02mw-config/03CookieConsent 07CosmicAlpha 036db6852 Add wgCookieConsentCategories [06:03:14] miraheze/mw-config - Universal-Omega the build passed. [06:04:00] [02mw-config] 07Universal-Omega merged 07dependabot[bot]'s pull request #6094: Update justinrainbow/json-schema requirement from 6.5.1 to 6.5.2 (03main...03dependabot/composer/justinrainbow/json-schema-6.5.2) 13https://github.com/miraheze/mw-config/pull/6094 [06:04:00] [02mw-config] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/fcc2f4cc14b2f89350ce351f122100a1955d3954 [06:04:00] 02mw-config/03main 07dependabot[bot] 03fcc2f4c Update justinrainbow/json-schema requirement from 6.5.1 to 6.5.2 (#6094)… [06:04:01] [02mw-config] 07Universal-Omega 04deleted 03dependabot/composer/justinrainbow/json-schema-6.5.2 at 039c29f9e 13https://api.github.com/repos/miraheze/mw-config/commit/9c29f9e [06:05:00] miraheze/mw-config - Universal-Omega the build passed. [06:05:00] RECOVERY - test151 Check unit status of mediawiki_job_purge-checkuser on test151 is OK: OK: Status of the systemd unit mediawiki_job_purge-checkuser [06:09:00] PROBLEM - test151 Check unit status of mediawiki_job_purge-checkuser on test151 is CRITICAL: CRITICAL: Status of the systemd unit mediawiki_job_purge-checkuser [06:44:08] PROBLEM - mwtask161 Current Load on mwtask161 is WARNING: LOAD WARNING - total load average: 22.10, 21.08, 16.99 [06:46:44] PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 49842MiB (10% inode=99%); [06:51:46] RECOVERY - mwtask161 Current Load on mwtask161 is OK: LOAD OK - total load average: 19.43, 20.28, 18.27 [07:00:01] PROBLEM - cp171 Disk Space on cp171 is WARNING: DISK WARNING - free space: / 49857MiB (10% inode=99%); [07:00:33] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.07, 15.63, 9.69 [07:02:33] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 26.38, 19.19, 11.70 [07:08:33] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.57, 22.94, 15.88 [07:16:33] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 17.25, 19.35, 17.15 [07:19:16] PROBLEM - cp191 Disk Space on cp191 is WARNING: DISK WARNING - free space: / 49933MiB (10% inode=99%); [07:30:33] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.32, 21.17, 19.01 [07:34:33] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.29, 23.29, 20.46 [07:38:33] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.59, 24.12, 21.42 [07:50:34] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 17.30, 22.33, 22.64 [07:51:08] [02mw-config] 07Universal-Omega 04force-pushed 03CookieConsent from 036db6852 to 036363373 13https://api.github.com/repos/miraheze/mw-config/commits/CookieConsent [07:51:08] 02mw-config/03CookieConsent 07CosmicAlpha 033fe4f42 Replace CookieWarning with CookieConsent [07:51:09] 02mw-config/03CookieConsent 07CosmicAlpha 036363373 Add wgCookieConsentCategories [07:52:05] miraheze/mw-config - Universal-Omega the build passed. [07:54:33] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.59, 22.37, 22.42 [07:56:34] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 20.27, 21.87, 22.26 [08:04:33] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 14.78, 18.16, 20.30 [08:52:29] [1/3] I’ll teach 10 people to earn $30k or more within 72 hours but you will pay me 10% of your profit when you receive it. Note only interested people should apply, drop a message let's get started by asking (HOW) [08:52:29] [2/3] Telegram ✍️ [08:52:29] [3/3] @Official_Benjamincurry00 [08:56:29] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 24.91, 10.07, 4.11 [08:58:27] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 7.53, 9.33, 4.63 [09:37:20] [Grafana] FIRING: The mediawiki job queue has more than 100000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?from=1757579810000&orgId=1&to=1757583440013 [10:24:33] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.68, 19.19, 15.72 [10:28:33] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 18.70, 19.71, 16.82 [10:34:33] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.28, 20.98, 18.32 [10:42:33] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 15.78, 19.46, 18.91 [11:10:04] PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push [11:16:04] RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push [12:13:56] [02PDFEmbed] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/PDFEmbed/commit/19e63544e17a674750c0b9dff4bb9b3bb38425e0 [12:13:56] 02PDFEmbed/03main 07translatewiki.net 0319e6354 Localisation updates from https://translatewiki.net. [12:13:56] [02MatomoAnalytics] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/MatomoAnalytics/commit/68762681f6a827a77c27fa3dfacbaf89655b8710 [12:13:56] 02MatomoAnalytics/03main 07translatewiki.net 036876268 Localisation updates from https://translatewiki.net. [12:18:48] miraheze/MatomoAnalytics - translatewiki the build passed. [12:18:58] miraheze/PDFEmbed - translatewiki the build passed. [13:37:20] [Grafana] FIRING: The mediawiki job queue has more than 100000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?from=1757579810000&orgId=1&to=1757597840044[Grafana] FIRING: The mediawiki JobQueue backlog is increasing by more than 100 jobs a minute over an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1757594060000&orgId=1&to=1757597840044 [13:38:51] 1.26 million GUP jobs D: [13:45:40] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/21270a45f2082fd00854a80e0f734afea220d9e2 [13:45:40] 02ssl/03main 07WikiTideBot 0321270a4 Bot: Auto-update domain lists [13:57:20] [Grafana] FIRING: The mediawiki job queue has more than 100000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?from=1757579810000&orgId=1&to=1757599040047[Grafana] RESOLVED: MediaWiki Jobs are rapidly increasing https://grafana.wikitide.net/d/GtxbP1Xnk?from=1757595260000&orgId=1&to=1757598860000 [14:03:33] [02mw-config] 07dependabot[bot] created 03dependabot/composer/phpunit/phpunit-12.3.10 (+1 new commit) 13https://github.com/miraheze/mw-config/commit/2a00fd04d473 [14:03:33] 02mw-config/03dependabot/composer/phpunit/phpunit-12.3.10 07dependabot[bot] 032a00fd0 Update phpunit/phpunit requirement from 12.3.8 to 12.3.10… [14:03:34] [02mw-config] 07dependabot[bot] added the label 'dependencies' to pull request #6101 (Update phpunit/phpunit requirement from 12.3.8 to 12.3.10) 13https://github.com/miraheze/mw-config/pull/6101 [14:03:36] [02mw-config] 07dependabot[bot] added the label 'php' to pull request #6101 (Update phpunit/phpunit requirement from 12.3.8 to 12.3.10) 13https://github.com/miraheze/mw-config/pull/6101 [14:03:38] [02mw-config] 07dependabot[bot] opened pull request #6101: Update phpunit/phpunit requirement from 12.3.8 to 12.3.10 (03main...03dependabot/composer/phpunit/phpunit-12.3.10) 13https://github.com/miraheze/mw-config/pull/6101 [14:03:40] [02mw-config] 07dependabot[bot] added the label 'dependencies' to pull request #6101 (Update phpunit/phpunit requirement from 12.3.8 to 12.3.10) 13https://github.com/miraheze/mw-config/pull/6101 [14:03:42] [02mw-config] 07dependabot[bot] added the label 'php' to pull request #6101 (Update phpunit/phpunit requirement from 12.3.8 to 12.3.10) 13https://github.com/miraheze/mw-config/pull/6101 [14:04:29] miraheze/mw-config - dependabot[bot] the build passed. [14:09:58] [02puppet] 07paladox created 03paladox-patch-3 (+1 new commit) 13https://github.com/miraheze/puppet/commit/26e3db742fa4 [14:09:58] 02puppet/03paladox-patch-3 07paladox 0326e3db7 changeprop201: increase LocalGlobalUserPageCacheUpdateJob concurrency to 25 [14:10:04] [02puppet] 07paladox opened pull request #4518: changeprop201: increase LocalGlobalUserPageCacheUpdateJob concurrenc… (03main...03paladox-patch-3) 13https://github.com/miraheze/puppet/pull/4518 [14:10:12] [02puppet] 07coderabbitai[bot] commented on pull request #4518:

[…] 13https://github.com/miraheze/puppet/pull/4518#issuecomment-3280967591 [14:10:25] [02puppet] 07paladox retitled PR #4518: " changeprop201: increase LocalGlobalUserPageCacheUpdateJob concurrenc…" ➜ " changeprop201: increase LocalGlobalUserPageCacheUpdateJob concurrency to 25" 13https://github.com/miraheze/puppet/pull/4518 [14:10:32] [02puppet] 07paladox merged pull request #4518: changeprop201: increase LocalGlobalUserPageCacheUpdateJob concurrency to 25 (03main...03paladox-patch-3) 13https://github.com/miraheze/puppet/pull/4518 [14:10:32] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/e26d0a0c9adf27957e58f788be367543e1b2ec58 [14:10:33] 02puppet/03main 07paladox 03e26d0a0 changeprop201: increase LocalGlobalUserPageCacheUpdateJob concurrency to 25 (#4518) [14:10:34] [02puppet] 07paladox 04deleted 03paladox-patch-3 at 0326e3db7 13https://api.github.com/repos/miraheze/puppet/commit/26e3db7 [14:12:20] [Grafana] FIRING: The mediawiki job queue has more than 100000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?from=1757579810000&orgId=1&to=1757599940049[Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1757596100000&orgId=1&to=1757599940049[Grafana] RESOLVED: MediaWiki Jobs are rapidly increasing https [14:12:20] .wikitide.net/d/GtxbP1Xnk?from=1757595260000&orgId=1&to=1757598860000 [14:28:33] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.07, 18.59, 14.78 [14:38:33] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 19.30, 20.15, 17.67 [15:01:04] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.85, 19.88, 18.20 [15:04:58] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 18.40, 19.75, 18.60 [15:10:47] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 20.59, 20.96, 19.49 [15:12:44] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 18.45, 20.36, 19.46 [15:32:33] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.71, 21.55, 19.76 [15:34:34] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 20.37, 21.59, 20.04 [15:36:33] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 15.62, 19.30, 19.39 [15:46:34] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.86, 21.66, 20.21 [15:48:34] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.12, 22.49, 20.70 [15:52:34] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.41, 22.91, 21.32 [15:56:33] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.44, 23.53, 21.92 [15:57:45] [02MirahezeMagic] 07Universal-Omega 04force-pushed 03mw-hooks from 03b148613 to 03b4ff10c 13https://api.github.com/repos/miraheze/MirahezeMagic/commits/mw-hooks [15:57:45] 02MirahezeMagic/03mw-hooks 07CosmicAlpha 037c9779c Move ManageWiki hooks from MirahezeFunctions [15:57:45] 02MirahezeMagic/03mw-hooks 07CosmicAlpha 03db00a7c Add [15:57:45] 02MirahezeMagic/03mw-hooks 07CosmicAlpha 03f0415a7 Update [15:57:46] 02MirahezeMagic/03mw-hooks 07CosmicAlpha 037cecd52 Fix using primary domain [15:57:47] 02MirahezeMagic/03mw-hooks 07CosmicAlpha 033659789 Add use [15:59:14] [02MirahezeMagic] 07Universal-Omega pushed 1 new commit to 03mw-hooks 13https://github.com/miraheze/MirahezeMagic/commit/bbbb58c558c5629b4ee9fad38466e77ba164c627 [15:59:14] 02MirahezeMagic/03mw-hooks 07CosmicAlpha 03bbbb58c Add use [16:00:11] [02MirahezeMagic] 07Universal-Omega pushed 1 new commit to 03rw-ui 13https://github.com/miraheze/MirahezeMagic/commit/526c517e6c35a7a1efdb85505ff32776a3346c6f [16:00:11] 02MirahezeMagic/03rw-ui 07CosmicAlpha 03526c517 Merge branch 'main' into rw-ui [16:02:47] [02MirahezeMagic] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/MirahezeMagic/commit/f53eae69bccd633007144b53c1468cc5782536fd [16:02:47] 02MirahezeMagic/03main 07CosmicAlpha 03f53eae6 phan: set enable_class_alias_support to false [16:05:59] [02MirahezeMagic] 07Universal-Omega 04force-pushed 03mw-hooks from 03bbbb58c to 03c5bbe76 13https://api.github.com/repos/miraheze/MirahezeMagic/commits/mw-hooks [16:05:59] 02MirahezeMagic/03mw-hooks 07CosmicAlpha 03c806173 Move ManageWiki hooks from MirahezeFunctions [16:05:59] 02MirahezeMagic/03mw-hooks 07CosmicAlpha 03761db38 Add [16:06:00] 02MirahezeMagic/03mw-hooks 07CosmicAlpha 03fa092f6 Update [16:06:02] 02MirahezeMagic/03mw-hooks 07CosmicAlpha 036d4d783 Fix using primary domain [16:06:02] 02MirahezeMagic/03mw-hooks 07CosmicAlpha 0399f3a8d Add use [16:06:23] [02MirahezeMagic] 07Universal-Omega pushed 1 new commit to 03rw-ui 13https://github.com/miraheze/MirahezeMagic/commit/055ebf9bfd403fc749beb5a161a11ac317fcc968 [16:06:23] 02MirahezeMagic/03rw-ui 07CosmicAlpha 03055ebf9 Merge branch 'main' into rw-ui [16:14:43] miraheze/MirahezeMagic - Universal-Omega the build passed. [16:18:28] miraheze/MirahezeMagic - Universal-Omega the build passed. [16:18:46] miraheze/MirahezeMagic - Universal-Omega the build passed. [16:23:52] [02mw-config] 07Universal-Omega created 03license (+1 new commit) 13https://github.com/miraheze/mw-config/commit/8fe04a4fbada [16:23:52] 02mw-config/03license 07CosmicAlpha 038fe04a4 Fix license icons on beta [16:23:56] [02mw-config] 07Universal-Omega opened pull request #6102: Fix license icons on beta (03main...03license) 13https://github.com/miraheze/mw-config/pull/6102 [16:24:49] miraheze/mw-config - Universal-Omega the build passed. [16:43:07] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php /srv/mediawiki/1.44/maintenance/importDump.php --wiki=midnighthorrorswiki dump.xml --no-updates (START) [16:43:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:44:08] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php /srv/mediawiki/1.44/maintenance/importDump.php --wiki=midnighthorrorswiki dump.xml --no-updates (END - exit=0) [16:44:09] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php /srv/mediawiki/1.44/maintenance/rebuildall.php --wiki=midnighthorrorswiki (START) [16:44:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:44:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:46:33] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.01, 23.74, 23.99 [16:52:33] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.28, 24.35, 24.14 [17:12:34] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.19, 23.54, 23.96 [17:18:34] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.50, 24.14, 24.01 [17:20:34] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.67, 23.99, 23.98 [17:22:34] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.54, 24.29, 24.10 [17:35:11] PROBLEM - mw152 APT on mw152 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [17:36:48] PROBLEM - mw182 APT on mw182 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [17:37:19] PROBLEM - mw183 APT on mw183 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [17:38:04] PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push [17:41:21] PROBLEM - mw162 APT on mw162 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [17:41:27] PROBLEM - mw203 APT on mw203 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [17:41:35] PROBLEM - mw202 APT on mw202 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [17:41:39] PROBLEM - mwtask171 APT on mwtask171 is CRITICAL: APT CRITICAL: 150 packages available for upgrade (9 critical updates). [17:42:08] PROBLEM - mw172 APT on mw172 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [17:42:18] PROBLEM - mw161 APT on mw161 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [17:43:26] PROBLEM - mwtask161 APT on mwtask161 is CRITICAL: APT CRITICAL: 150 packages available for upgrade (9 critical updates). [17:43:29] PROBLEM - mwtask151 APT on mwtask151 is CRITICAL: APT CRITICAL: 150 packages available for upgrade (9 critical updates). [17:45:45] PROBLEM - mw173 APT on mw173 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [17:48:26] PROBLEM - mw193 APT on mw193 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [17:51:27] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php /srv/mediawiki/1.44/maintenance/rebuildall.php --wiki=midnighthorrorswiki (END - exit=0) [17:51:29] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php /srv/mediawiki/1.44/maintenance/initSiteStats.php --wiki=midnighthorrorswiki --update (END - exit=0) [17:51:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:51:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:51:42] PROBLEM - mw191 APT on mw191 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [17:51:57] PROBLEM - mw171 APT on mw171 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [17:53:12] PROBLEM - mw153 APT on mw153 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [17:56:05] RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push [17:58:07] PROBLEM - mw192 APT on mw192 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [17:58:29] PROBLEM - mw151 APT on mw151 is CRITICAL: APT CRITICAL: 148 packages available for upgrade (9 critical updates). [17:59:18] PROBLEM - mw163 APT on mw163 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [18:02:11] PROBLEM - mw181 APT on mw181 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [18:03:35] PROBLEM - mwtask181 APT on mwtask181 is CRITICAL: APT CRITICAL: 150 packages available for upgrade (9 critical updates). [18:04:14] PROBLEM - test151 APT on test151 is CRITICAL: APT CRITICAL: 135 packages available for upgrade (9 critical updates). [18:05:19] PROBLEM - mw201 APT on mw201 is CRITICAL: APT CRITICAL: 146 packages available for upgrade (9 critical updates). [18:12:20] [Grafana] FIRING: The mediawiki job queue has more than 100000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?from=1757579810000&orgId=1&to=1757614340081[Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1757596100000&orgId=1&to=1757614340081 [18:26:34] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 18.10, 22.73, 23.90 [18:50:34] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.87, 23.16, 22.42 [19:01:11] PROBLEM - mattermost1 APT on mattermost1 is CRITICAL: APT CRITICAL: 120 packages available for upgrade (1 critical updates). [19:01:58] PROBLEM - os191 APT on os191 is CRITICAL: APT CRITICAL: 113 packages available for upgrade (1 critical updates). [19:02:32] PROBLEM - ns2 APT on ns2 is CRITICAL: APT CRITICAL: 118 packages available for upgrade (1 critical updates). [19:02:52] PROBLEM - reports171 APT on reports171 is CRITICAL: APT CRITICAL: 117 packages available for upgrade (1 critical updates). [19:03:00] PROBLEM - swiftobject161 APT on swiftobject161 is CRITICAL: APT CRITICAL: 77 packages available for upgrade (1 critical updates). [19:03:32] PROBLEM - swiftobject151 APT on swiftobject151 is CRITICAL: APT CRITICAL: 77 packages available for upgrade (1 critical updates). [19:03:35] PROBLEM - os161 APT on os161 is CRITICAL: APT CRITICAL: 113 packages available for upgrade (1 critical updates). [19:03:40] PROBLEM - db151 APT on db151 is CRITICAL: APT CRITICAL: 131 packages available for upgrade (1 critical updates). [19:04:48] PROBLEM - cloud20 APT on cloud20 is CRITICAL: APT CRITICAL: 192 packages available for upgrade (1 critical updates). [19:04:55] PROBLEM - bast181 APT on bast181 is CRITICAL: APT CRITICAL: 113 packages available for upgrade (1 critical updates). [19:05:37] PROBLEM - ldap171 APT on ldap171 is CRITICAL: APT CRITICAL: 113 packages available for upgrade (1 critical updates). [19:07:01] PROBLEM - mon181 APT on mon181 is CRITICAL: APT CRITICAL: 83 packages available for upgrade (1 critical updates). [19:07:44] PROBLEM - prometheus151 APT on prometheus151 is CRITICAL: APT CRITICAL: 121 packages available for upgrade (1 critical updates). [19:08:26] PROBLEM - matomo151 APT on matomo151 is CRITICAL: APT CRITICAL: 76 packages available for upgrade (1 critical updates). [19:09:05] PROBLEM - mem201 APT on mem201 is CRITICAL: APT CRITICAL: 121 packages available for upgrade (1 critical updates). [19:09:45] PROBLEM - swiftproxy161 APT on swiftproxy161 is CRITICAL: APT CRITICAL: 130 packages available for upgrade (1 critical updates). [19:10:09] PROBLEM - cp201 APT on cp201 is CRITICAL: APT CRITICAL: 84 packages available for upgrade (1 critical updates). [19:11:17] PROBLEM - swiftproxy171 APT on swiftproxy171 is CRITICAL: APT CRITICAL: 130 packages available for upgrade (1 critical updates). [19:11:23] PROBLEM - eventgate181 APT on eventgate181 is CRITICAL: APT CRITICAL: 123 packages available for upgrade (1 critical updates). [19:11:28] PROBLEM - db172 APT on db172 is CRITICAL: APT CRITICAL: 96 packages available for upgrade (1 critical updates). [19:12:05] PROBLEM - bast161 APT on bast161 is CRITICAL: APT CRITICAL: 113 packages available for upgrade (1 critical updates). [19:13:24] PROBLEM - swiftobject191 APT on swiftobject191 is CRITICAL: APT CRITICAL: 77 packages available for upgrade (1 critical updates). [19:13:26] PROBLEM - swiftobject201 APT on swiftobject201 is CRITICAL: APT CRITICAL: 77 packages available for upgrade (1 critical updates). [19:13:27] PROBLEM - cp171 APT on cp171 is CRITICAL: APT CRITICAL: 114 packages available for upgrade (1 critical updates). [19:13:38] PROBLEM - mem161 APT on mem161 is CRITICAL: APT CRITICAL: 121 packages available for upgrade (1 critical updates). [19:14:32] PROBLEM - swiftac171 APT on swiftac171 is CRITICAL: APT CRITICAL: 128 packages available for upgrade (1 critical updates). [19:14:41] PROBLEM - puppet181 APT on puppet181 is CRITICAL: APT CRITICAL: 88 packages available for upgrade (1 critical updates). [19:16:01] PROBLEM - swiftobject181 APT on swiftobject181 is CRITICAL: APT CRITICAL: 77 packages available for upgrade (1 critical updates). [19:16:24] PROBLEM - db161 APT on db161 is CRITICAL: APT CRITICAL: 131 packages available for upgrade (1 critical updates). [19:16:30] PROBLEM - phorge171 APT on phorge171 is CRITICAL: APT CRITICAL: 78 packages available for upgrade (1 critical updates). [19:16:31] PROBLEM - cloud15 APT on cloud15 is CRITICAL: APT CRITICAL: 191 packages available for upgrade (1 critical updates). [19:18:15] PROBLEM - cloud16 APT on cloud16 is CRITICAL: APT CRITICAL: 190 packages available for upgrade (1 critical updates). [19:18:21] PROBLEM - os201 APT on os201 is CRITICAL: APT CRITICAL: 85 packages available for upgrade (1 critical updates). [19:18:34] PROBLEM - mem151 APT on mem151 is CRITICAL: APT CRITICAL: 121 packages available for upgrade (1 critical updates). [19:19:08] PROBLEM - ns1 APT on ns1 is CRITICAL: APT CRITICAL: 119 packages available for upgrade (1 critical updates). [19:19:44] PROBLEM - os151 APT on os151 is CRITICAL: APT CRITICAL: 115 packages available for upgrade (1 critical updates). [19:20:05] PROBLEM - kafka181 APT on kafka181 is CRITICAL: APT CRITICAL: 121 packages available for upgrade (1 critical updates). [19:21:29] PROBLEM - db171 APT on db171 is CRITICAL: APT CRITICAL: 123 packages available for upgrade (1 critical updates). [19:22:34] PROBLEM - cloud17 APT on cloud17 is CRITICAL: APT CRITICAL: 190 packages available for upgrade (1 critical updates). [19:23:32] PROBLEM - cloud19 APT on cloud19 is CRITICAL: APT CRITICAL: 192 packages available for upgrade (1 critical updates). [19:23:38] PROBLEM - db182 APT on db182 is CRITICAL: APT CRITICAL: 86 packages available for upgrade (1 critical updates). [19:24:08] PROBLEM - cloud18 APT on cloud18 is CRITICAL: APT CRITICAL: 190 packages available for upgrade (1 critical updates). [19:25:45] PROBLEM - rdb151 APT on rdb151 is CRITICAL: APT CRITICAL: 121 packages available for upgrade (1 critical updates). [19:25:50] PROBLEM - bots171 APT on bots171 is CRITICAL: APT CRITICAL: 119 packages available for upgrade (1 critical updates). [19:26:31] PROBLEM - mem191 APT on mem191 is CRITICAL: APT CRITICAL: 113 packages available for upgrade (1 critical updates). [19:26:40] PROBLEM - swiftobject171 APT on swiftobject171 is CRITICAL: APT CRITICAL: 77 packages available for upgrade (1 critical updates). [19:26:57] PROBLEM - cp191 APT on cp191 is CRITICAL: APT CRITICAL: 114 packages available for upgrade (1 critical updates). [19:27:33] PROBLEM - cp161 APT on cp161 is CRITICAL: APT CRITICAL: 123 packages available for upgrade (1 critical updates). [19:27:51] PROBLEM - os162 APT on os162 is CRITICAL: APT CRITICAL: 115 packages available for upgrade (1 critical updates). [19:28:51] PROBLEM - changeprop201 APT on changeprop201 is CRITICAL: APT CRITICAL: 124 packages available for upgrade (1 critical updates). [19:29:29] PROBLEM - graylog161 APT on graylog161 is CRITICAL: APT CRITICAL: 119 packages available for upgrade (1 critical updates). [19:29:51] PROBLEM - os202 APT on os202 is CRITICAL: APT CRITICAL: 76 packages available for upgrade (1 critical updates). [19:30:18] PROBLEM - db181 APT on db181 is CRITICAL: APT CRITICAL: 123 packages available for upgrade (1 critical updates). [19:42:33] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.37, 23.48, 24.00 [19:44:33] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.18, 23.74, 24.03 [19:45:39] PROBLEM - os151 APT on os151 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [19:46:34] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.27, 23.64, 23.96 [19:47:41] PROBLEM - os151 APT on os151 is CRITICAL: APT CRITICAL: 115 packages available for upgrade (1 critical updates). [19:48:33] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.12, 23.95, 24.04 [19:50:33] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.14, 23.76, 23.97 [19:56:33] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.27, 23.65, 23.80 [19:58:28] PROBLEM - ns2 APT on ns2 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [20:00:29] PROBLEM - ns2 APT on ns2 is CRITICAL: APT CRITICAL: 118 packages available for upgrade (1 critical updates). [20:03:02] PROBLEM - mw152 APT on mw152 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [20:03:14] PROBLEM - matomo151 Disk Space on matomo151 is WARNING: DISK WARNING - free space: / 1967MiB (10% inode=89%); [20:05:05] PROBLEM - mw152 APT on mw152 is CRITICAL: APT CRITICAL: 147 packages available for upgrade (10 critical updates). [20:48:04] PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push [20:56:04] RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push [21:02:17] [02RequestCustomDomain] 07Universal-Omega created 031.44 from 03main (+0 new commit) 13https://github.com/miraheze/RequestCustomDomain/compare/1.44 [21:04:13] [02RequestCustomDomain] 07Universal-Omega pushed 1 new commit to 031.44 13https://github.com/miraheze/RequestCustomDomain/commit/bfa29a8a2637167dcaaddbd8f4d3deea44eeb9e9 [21:04:13] 02RequestCustomDomain/031.44 07CosmicAlpha 03bfa29a8 Update for MediaWiki 1.44 [21:04:29] [02RequestCustomDomain] 07Universal-Omega opened pull request #96: Update for MediaWiki 1.44 (03main...031.44) 13https://github.com/miraheze/RequestCustomDomain/pull/96 [21:04:37] [02RequestCustomDomain] 07coderabbitai[bot] commented on pull request #96:

[…] 13https://github.com/miraheze/RequestCustomDomain/pull/96#issuecomment-3282632572 [21:08:31] miraheze/RequestCustomDomain - Universal-Omega the build has errored. [21:08:33] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 17.13, 21.88, 23.57 [21:12:20] [Grafana] FIRING: The mediawiki job queue has more than 100000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?from=1757579810000&orgId=1&to=1757625140105[Grafana] RESOLVED: MediaWiki JobQueue is stalled https://grafana.wikitide.net/d/GtxbP1Xnk?from=1757621360000&orgId=1&to=1757624960000 [21:12:33] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.41, 22.01, 23.07 [21:18:22] [02RequestCustomDomain] 07Universal-Omega pushed 1 new commit to 031.44 13https://github.com/miraheze/RequestCustomDomain/commit/b28de1a181dce83f2c1ab48b399aff33457f9520 [21:18:23] 02RequestCustomDomain/031.44 07CosmicAlpha 03b28de1a Update [21:22:33] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 19.10, 22.76, 23.48 [21:28:33] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.87, 23.08, 23.25 [21:30:33] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.57, 23.28, 23.32 [21:33:24] miraheze/RequestCustomDomain - Universal-Omega the build has errored. [21:39:22] [02ci-scripts] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/ci-scripts/commit/ec55cc537f9e7efa6ef5a83e7a2817edbd063fe6 [21:39:22] 02ci-scripts/03main 07CosmicAlpha 03ec55cc5 Exclude GrowthExperiments on all repos [21:39:42] [02RequestCustomDomain] 07Universal-Omega merged pull request #96: Update for MediaWiki 1.44 (03main...031.44) 13https://github.com/miraheze/RequestCustomDomain/pull/96 [21:39:42] [02RequestCustomDomain] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/RequestCustomDomain/commit/2930bcfdc4e59f50d685f0339a67811fe2acb982 [21:39:44] 02RequestCustomDomain/03main 07CosmicAlpha 032930bcf Update for MediaWiki 1.44 [21:39:46] [02RequestCustomDomain] 07Universal-Omega 04deleted 031.44 at 03b28de1a 13https://api.github.com/repos/miraheze/RequestCustomDomain/commit/b28de1a [21:48:33] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.50, 22.47, 22.39 [21:50:34] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.55, 22.93, 22.58 [21:58:44] miraheze/RequestCustomDomain - Universal-Omega the build passed. [22:02:23] RECOVERY - matomo151 Disk Space on matomo151 is OK: DISK OK - free space: / 6368MiB (35% inode=89%); [22:10:33] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 17.06, 19.00, 20.11 [22:18:33] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 19.95, 20.79, 20.42 [22:20:33] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 15.10, 18.80, 19.75 [22:29:23] miraheze/ImportDump - Universal-Omega the build passed. [22:43:19] [02mw-config] 07Universal-Omega merged 07dependabot[bot]'s pull request #6101: Update phpunit/phpunit requirement from 12.3.8 to 12.3.10 (03main...03dependabot/composer/phpunit/phpunit-12.3.10) 13https://github.com/miraheze/mw-config/pull/6101 [22:43:19] [02mw-config] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/c8de7a9a69bc17f668e32c618e6ce41d1652f268 [22:43:19] 02mw-config/03main 07dependabot[bot] 03c8de7a9 Update phpunit/phpunit requirement from 12.3.8 to 12.3.10 (#6101)… [22:43:20] [02mw-config] 07Universal-Omega 04deleted 03dependabot/composer/phpunit/phpunit-12.3.10 at 032a00fd0 13https://api.github.com/repos/miraheze/mw-config/commit/2a00fd0 [22:43:56] [02mw-config] 07Universal-Omega merged pull request #6102: Fix license icons on beta (03main...03license) 13https://github.com/miraheze/mw-config/pull/6102 [22:43:56] [02mw-config] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/b3270e39fd10b06d395fb1d6ece053fd23f968f4 [22:43:56] 02mw-config/03main 07CosmicAlpha 03b3270e3 Fix license icons on beta [22:43:57] [02mw-config] 07Universal-Omega 04deleted 03license at 038fe04a4 13https://api.github.com/repos/miraheze/mw-config/commit/8fe04a4 [22:44:10] miraheze/mw-config - Universal-Omega the build passed. [22:44:51] miraheze/mw-config - Universal-Omega the build passed. [22:45:14] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.47, 21.69, 19.65 [23:02:20] [Grafana] FIRING: The mediawiki job queue has more than 100000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?from=1757579810000&orgId=1&to=1757631740121[Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1757628020000&orgId=1&to=1757631740121 [23:10:50] [02python-functions] 07dependabot[bot] created 03dependabot/pip/dot-github/mypy-1.18.1 (+1 new commit) 13https://github.com/miraheze/python-functions/commit/84bac3d51c92 [23:10:50] 02python-functions/03dependabot/pip/dot-github/mypy-1.18.1 07dependabot[bot] 0384bac3d build(deps): bump mypy from 1.17.1 to 1.18.1 in /.github… [23:10:51] [02python-functions] 07dependabot[bot] added the label 'dependencies' to pull request #181 (build(deps): bump mypy from 1.17.1 to 1.18.1 in /.github) 13https://github.com/miraheze/python-functions/pull/181 [23:10:51] [02python-functions] 07dependabot[bot] added the label 'python' to pull request #181 (build(deps): bump mypy from 1.17.1 to 1.18.1 in /.github) 13https://github.com/miraheze/python-functions/pull/181 [23:10:53] [02python-functions] 07dependabot[bot] opened pull request #181: build(deps): bump mypy from 1.17.1 to 1.18.1 in /.github (03main...03dependabot/pip/dot-github/mypy-1.18.1) 13https://github.com/miraheze/python-functions/pull/181 [23:10:55] [02python-functions] 07dependabot[bot] added the label 'dependencies' to pull request #181 (build(deps): bump mypy from 1.17.1 to 1.18.1 in /.github) 13https://github.com/miraheze/python-functions/pull/181 [23:10:57] [02python-functions] 07dependabot[bot] added the label 'python' to pull request #181 (build(deps): bump mypy from 1.17.1 to 1.18.1 in /.github) 13https://github.com/miraheze/python-functions/pull/181 [23:11:00] [02python-functions] 07coderabbitai[bot] commented on pull request #181: --- […] 13https://github.com/miraheze/python-functions/pull/181#issuecomment-3282921501 [23:16:33] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 19.29, 23.02, 23.72 [23:17:20] [Grafana] FIRING: The mediawiki job queue has more than 100000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?from=1757579810000&orgId=1&to=1757632640124[Grafana] RESOLVED: MediaWiki JobQueue is stalled https://grafana.wikitide.net/d/GtxbP1Xnk?from=1757628980000&orgId=1&to=1757632580000 [23:17:27] [02python-functions] 07github-merge-queue[bot] created 03gh-readonly-queue/main/pr-181-448c3ded24b7062ae1b0683ac0c90fba974f501f (+1 new commit) 13https://github.com/miraheze/python-functions/commit/5bdde6e637d8 [23:17:29] 02python-functions/03gh-readonly-queue/main/pr-181-448c3ded24b7062ae1b0683ac0c90fba974f501f 07dependabot[bot] 035bdde6e build(deps): bump mypy from 1.17.1 to 1.18.1 in /.github (#181)… [23:17:32] miraheze/python-functions - dependabot[bot] the build passed. [23:23:53] [02python-functions] 07github-merge-queue[bot] pushed 1 new commit to 03main 13https://github.com/miraheze/python-functions/commit/5bdde6e637d86e506edf3501096563adf0f2a579 [23:23:53] 02python-functions/03main 07dependabot[bot] 035bdde6e build(deps): bump mypy from 1.17.1 to 1.18.1 in /.github (#181)… [23:23:53] [02python-functions] 07github-merge-queue[bot] 04deleted 03gh-readonly-queue/main/pr-181-448c3ded24b7062ae1b0683ac0c90fba974f501f at 035bdde6e 13https://api.github.com/repos/miraheze/python-functions/commit/5bdde6e [23:23:54] [02python-functions] 07github-actions[bot] merged 07dependabot[bot]'s pull request #181: build(deps): bump mypy from 1.17.1 to 1.18.1 in /.github (03main...03dependabot/pip/dot-github/mypy-1.18.1) 13https://github.com/miraheze/python-functions/pull/181 [23:23:56] [02python-functions] 07github-actions[bot] 04deleted 03dependabot/pip/dot-github/mypy-1.18.1 at 0384bac3d 13https://api.github.com/repos/miraheze/python-functions/commit/84bac3d [23:27:28] miraheze/python-functions - github-merge-queue[bot] the build passed. [23:53:52] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 10.56, 8.33, 4.26 [23:54:53] !log [void@puppet181] Upgraded packages on cloud15: libcups2 and libjson-xs-perl [23:54:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:55:06] !log [void@puppet181] Upgraded packages on db171: libcups2 [23:55:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:55:18] !log [void@puppet181] Upgraded packages on changeprop201: libcups2 [23:55:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:55:31] !log [void@puppet181] Upgraded packages on bots171: libcups2 [23:55:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:55:43] !log [void@puppet181] Upgraded packages on cp161: libcups2 [23:55:46] RECOVERY - db171 APT on db171 is OK: APT OK: 122 packages available for upgrade (0 critical updates). [23:55:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:55:52] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 2.43, 5.86, 3.85 [23:55:56] !log [void@puppet181] Upgraded packages on db151: libcups2 [23:56:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:56:01] RECOVERY - bots171 APT on bots171 is OK: APT OK: 118 packages available for upgrade (0 critical updates). [23:56:09] !log [void@puppet181] Upgraded packages on db172: libcups2 [23:56:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:56:21] !log [void@puppet181] Upgraded packages on cloud16: libcups2 and libjson-xs-perl [23:56:23] RECOVERY - db151 APT on db151 is OK: APT OK: 130 packages available for upgrade (0 critical updates). [23:56:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:56:33] !log [void@puppet181] Upgraded packages on matomo151: libcups2 [23:56:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:56:45] !log [void@puppet181] Upgraded packages on cloud19: libcups2 and libjson-xs-perl [23:56:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:56:55] RECOVERY - changeprop201 APT on changeprop201 is OK: APT OK: 123 packages available for upgrade (0 critical updates). [23:56:58] !log [void@puppet181] Upgraded packages on cloud17: libcups2 and libjson-xs-perl [23:57:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:57:11]