[00:01:55] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 17.81, 16.50, 20.02 [00:01:59] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/e3055fa8df93e36c4cba6b90fdfb2427cef6d555 [00:01:59] 02statichelp/03main 07WikiTideBot 03e3055fa Bot: Auto-update Tech namespace pages 2025-09-17 00:01:57 [00:05:02] PROBLEM - cp171 Disk Space on cp171 is CRITICAL: DISK CRITICAL - free space: / 27114MiB (5% inode=99%); [00:05:26] PROBLEM - cp201 Disk Space on cp201 is CRITICAL: DISK CRITICAL - free space: / 27083MiB (5% inode=99%); [00:08:16] PROBLEM - cp191 Disk Space on cp191 is CRITICAL: DISK CRITICAL - free space: / 26902MiB (5% inode=99%); [00:08:54] RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 55774MiB (12% inode=99%); [00:09:24] RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 55549MiB (12% inode=99%); [00:10:15] RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 56185MiB (12% inode=99%); [01:19:56] !log [universalomega@mwtask181] starting deploy of {'versions': '1.44', 'upgrade_extensions': 'MirahezeMagic'} to all [01:20:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:20:21] !log [universalomega@mwtask181] finished deploy of {'versions': '1.44', 'upgrade_extensions': 'MirahezeMagic'} to all - SUCCESS in 25s [01:20:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:21:00] !log [universalomega@mwtask181] starting deploy of {'l10n': True, 'versions': '1.44'} to all [01:21:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:28:53] !log [universalomega@mwtask181] finished deploy of {'l10n': True, 'versions': '1.44'} to all - SUCCESS in 472s [01:28:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:32:18] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/752cc37a7cce390415691d6a8c773a6c9f97832a [04:32:18] 02puppet/03main 07CosmicAlpha 03752cc37 Disable MediaModeration job for now [04:41:28] PROBLEM - matomo151 Check unit status of matomo-archiver-3 on matomo151 is CRITICAL: CRITICAL: Status of the systemd unit matomo-archiver-3 [05:57:48] PROBLEM - db182 Disk Space on db182 is WARNING: DISK WARNING - free space: / 47736MiB (10% inode=99%); [06:01:04] PROBLEM - cp171 Disk Space on cp171 is WARNING: DISK WARNING - free space: / 49870MiB (10% inode=99%); [06:15:48] PROBLEM - db182 Disk Space on db182 is CRITICAL: DISK CRITICAL - free space: / 27079MiB (5% inode=99%); [06:25:13] PROBLEM - cp191 Disk Space on cp191 is WARNING: DISK WARNING - free space: / 49943MiB (10% inode=99%); [06:37:48] RECOVERY - db182 Disk Space on db182 is OK: DISK OK - free space: / 127153MiB (27% inode=99%); [08:01:46] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.91, 16.87, 9.75 [08:02:51] RECOVERY - matomo151 Check unit status of matomo-archiver-3 on matomo151 is OK: OK: Status of the systemd unit matomo-archiver-3 [08:07:46] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.76, 22.13, 14.52 [08:09:46] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.11, 22.82, 15.70 [08:13:46] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.55, 23.24, 17.51 [08:15:46] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.91, 23.76, 18.40 [08:23:46] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.27, 23.50, 20.58 [08:27:46] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.41, 23.33, 21.07 [08:28:56] [Grafana] FIRING: The mediawiki JobQueue backlog is increasing by more than 100 jobs a minute over an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1758094100000&orgId=1&to=1758097736680 [08:33:56] [Grafana] RESOLVED: MediaWiki Jobs are rapidly increasing https://grafana.wikitide.net/d/GtxbP1Xnk?from=1758094340000&orgId=1&to=1758097940000 [08:55:46] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 20.77, 23.23, 23.98 [09:01:46] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 27.62, 24.26, 24.02 [09:03:46] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.86, 23.84, 23.89 [09:13:46] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.37, 22.30, 22.77 [09:15:46] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.67, 22.48, 22.79 [09:19:46] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.40, 23.45, 23.11 [09:21:46] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.01, 23.16, 23.07 [09:24:11] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 8.03, 11.70, 9.76 [09:25:46] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.15, 24.10, 23.41 [09:26:10] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 5.95, 9.72, 9.26 [09:27:46] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.27, 23.23, 23.20 [09:31:41] PROBLEM - mattermost1 Puppet on mattermost1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/usr/lib/nagios/plugins/check_conntrack] [09:31:46] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.21, 22.51, 22.80 [09:32:25] PROBLEM - puppet181 Check unit status of git_pull_puppet on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit git_pull_puppet [09:37:46] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.43, 23.28, 23.35 [09:40:25] RECOVERY - puppet181 Check unit status of git_pull_puppet on puppet181 is OK: OK: Status of the systemd unit git_pull_puppet [09:55:46] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 17.54, 18.89, 20.21 [09:59:42] RECOVERY - mattermost1 Puppet on mattermost1 is OK: OK: Puppet is currently enabled, last run 28 seconds ago with 0 failures [10:26:17] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.83, 21.71, 19.87 [10:30:12] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.69, 23.09, 20.84 [10:34:07] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.98, 23.52, 21.56 [10:38:02] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.88, 23.59, 21.96 [10:41:57] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.02, 23.82, 22.47 [10:45:51] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.48, 23.92, 22.79 [11:01:46] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.64, 23.35, 23.63 [11:03:46] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.48, 23.91, 23.81 [11:09:46] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.08, 23.32, 23.67 [11:39:46] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 19.41, 18.80, 20.25 [11:41:07] [02mw-config] 07SomeMWDev created 03disable-semanticdrilldown (+1 new commit) 13https://github.com/miraheze/mw-config/commit/8300fa58b7bb [11:41:07] 02mw-config/03disable-semanticdrilldown 07SomeRandomDeveloper 038300fa5 Globally disable SemanticDrilldown… [11:41:19] [02mw-config] 07SomeMWDev opened pull request #6111: Globally disable SemanticDrilldown (03main...03disable-semanticdrilldown) 13https://github.com/miraheze/mw-config/pull/6111 [11:42:12] miraheze/mw-config - SomeMWDev the build passed. [11:42:21] [02mw-config] 07SomeMWDev merged pull request #6111: Globally disable SemanticDrilldown (03main...03disable-semanticdrilldown) 13https://github.com/miraheze/mw-config/pull/6111 [11:42:22] [02mw-config] 07SomeMWDev pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/909f45f9cf6ad56d072cae98613cc2b8ed8673d1 [11:42:25] [02mw-config] 07SomeMWDev 04deleted 03disable-semanticdrilldown at 038300fa5 13https://api.github.com/repos/miraheze/mw-config/commit/8300fa5 [11:42:26] 02mw-config/03main 07SomeRandomDeveloper 03909f45f Globally disable SemanticDrilldown (#6111)… [11:42:38] MirahezeLSBot seems to be dead again [11:43:21] miraheze/mw-config - SomeMWDev the build passed. [12:27:13] [02mw-config] 07pskyechology pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/0a8ba83505557bd796375c9fc8fb45255adc63a9 [12:27:14] 02mw-config/03main 07Skye 030a8ba83 Fix SMW footer icon [12:28:07] miraheze/mw-config - pskyechology the build passed. [13:21:14] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 6.64, 10.78, 6.72 [13:23:13] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 1.81, 7.60, 6.03 [15:05:11] !log test [15:05:17] right it be dead [15:05:19] that's not good [15:05:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:30:51] [02landing] 07Universal-Omega created 03rm-unused-imgs (+1 new commit) 13https://github.com/miraheze/landing/commit/d3f2852aea3a [17:30:51] 02landing/03rm-unused-imgs 07CosmicAlpha 03d3f2852 Remove unused images [17:30:56] [02landing] 07Universal-Omega opened pull request #168: Remove unused images (03main...03rm-unused-imgs) 13https://github.com/miraheze/landing/pull/168 [17:31:03] [02landing] 07coderabbitai[bot] commented on pull request #168:

[…] 13https://github.com/miraheze/landing/pull/168#issuecomment-3303947960 [17:31:13] [02landing] 07Universal-Omega pushed 1 new commit to 03rm-unused-imgs 13https://github.com/miraheze/landing/commit/c70cd9d7d6402006eccab1a22690b433e9cbec04 [17:31:13] 02landing/03rm-unused-imgs 07CosmicAlpha 03c70cd9d Remove [17:31:24] [02landing] 07Universal-Omega pushed 1 new commit to 03rm-unused-imgs 13https://github.com/miraheze/landing/commit/08523eea95416e8c52cd124208a444ebf576962f [17:31:24] 02landing/03rm-unused-imgs 07CosmicAlpha 0308523ee Remove [17:31:49] miraheze/landing - Universal-Omega the build passed. [17:32:13] miraheze/landing - Universal-Omega the build passed. [17:32:27] [02landing] 07Universal-Omega pushed 1 new commit to 03rm-unused-imgs 13https://github.com/miraheze/landing/commit/745bdda096dd1c1d7e59931eb300ec2bb949d435 [17:32:27] 02landing/03rm-unused-imgs 07CosmicAlpha 03745bdda Remove [17:33:23] miraheze/landing - Universal-Omega the build passed. [17:38:52] [02landing] 07Universal-Omega created 03svg from 03main (+0 new commit) 13https://github.com/miraheze/landing/compare/svg [17:41:06] [02landing] 07Universal-Omega pushed 1 new commit to 03svg 13https://github.com/miraheze/landing/commit/55500ece9ac11891b2b865c63f2f4d1ab163b24b [17:41:06] 02landing/03svg 07CosmicAlpha 0355500ec Use svg for WikiTide Logo [17:41:07] [02landing] 07Universal-Omega opened pull request #169: Use svg for WikiTide Logo (03main...03svg) 13https://github.com/miraheze/landing/pull/169 [17:41:07] [02landing] 07coderabbitai[bot] commented on pull request #169:

[…] 13https://github.com/miraheze/landing/pull/169#issuecomment-3303978618 [17:41:42] miraheze/landing - Universal-Omega the build passed. [17:43:51] [02landing] 07Universal-Omega pushed 1 new commit to 03svg 13https://github.com/miraheze/landing/commit/3a403869641ff78d036ca78d4137751b5fc1b2d2 [17:43:51] 02landing/03svg 07CosmicAlpha 033a40386 Update size [17:44:42] miraheze/landing - Universal-Omega the build passed. [17:55:20] [02landing] 07Universal-Omega pushed 1 new commit to 03svg 13https://github.com/miraheze/landing/commit/87db5f5c6f14abb130ab5bcadc4f47c6691d6327 [17:55:21] 02landing/03svg 07CosmicAlpha 0387db5f5 Add svgo [17:56:11] miraheze/landing - Universal-Omega the build passed. [17:57:52] [02landing] 07Universal-Omega retitled PR #169: "Use svg for WikiTide Logo" ➜ "Use svg for WikiTide Logo and add svgo" 13https://github.com/miraheze/landing/pull/169 [17:59:40] [02landing] 07Universal-Omega pushed 1 new commit to 03svg 13https://github.com/miraheze/landing/commit/4ed558bdf7c103bf1cf121a337bc11ad3a67f334 [17:59:40] 02landing/03svg 07CosmicAlpha 034ed558b Fix [18:00:37] miraheze/landing - Universal-Omega the build passed. [18:05:34] [02landing] 07Universal-Omega merged pull request #168: Remove unused images (03main...03rm-unused-imgs) 13https://github.com/miraheze/landing/pull/168 [18:05:36] [02landing] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/landing/commit/ad450f32092f3e24763e864d275dca2421863d37 [18:05:38] [02landing] 07Universal-Omega 04deleted 03rm-unused-imgs at 03745bdda 13https://api.github.com/repos/miraheze/landing/commit/745bdda [18:05:39] 02landing/03main 07CosmicAlpha 03ad450f3 Remove unused images [18:05:56] [02landing] 07Universal-Omega merged pull request #169: Use svg for WikiTide Logo and add svgo (03main...03svg) 13https://github.com/miraheze/landing/pull/169 [18:05:57] [02landing] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/landing/commit/8f307d2d2476580df73f3ca5fdf7c00975c17daa [18:05:57] 02landing/03main 07CosmicAlpha 038f307d2 Use svg for WikiTide Logo and add svgo [18:05:58] [02landing] 07Universal-Omega 04deleted 03svg at 034ed558b 13https://api.github.com/repos/miraheze/landing/commit/4ed558b [18:06:32] miraheze/landing - Universal-Omega the build passed. [18:06:51] miraheze/landing - Universal-Omega the build passed. [18:14:45] [02mw-config] 07c-oreills opened pull request #6112: T14285: add NS_MAIN to wgMFNamespacesWithoutCollapsibleSections for twistwoodtaleswiki (07miraheze:03main...07c-oreills:03main) 13https://github.com/miraheze/mw-config/pull/6112 [18:19:40] PROBLEM - matomo151 Disk Space on matomo151 is WARNING: DISK WARNING - free space: / 1969MiB (10% inode=89%); [18:30:48] [02landing] 07Universal-Omega created 03svg (+1 new commit) 13https://github.com/miraheze/landing/commit/92cdabde89bd [18:30:48] 02landing/03svg 07CosmicAlpha 0392cdabd Use some SVGs instead of webp [18:31:20] [02landing] 07Universal-Omega opened pull request #170: Use some SVGs instead of webp (03main...03svg) 13https://github.com/miraheze/landing/pull/170 [18:31:27] [02landing] 07coderabbitai[bot] commented on pull request #170:

[…] 13https://github.com/miraheze/landing/pull/170#issuecomment-3304129365 [18:32:04] miraheze/landing - Universal-Omega the build passed. [18:41:24] [02landing] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/landing/commit/323042d3770a606264f5ac9fb517d952e9f7cb09 [18:41:24] 02landing/03main 07CosmicAlpha 03323042d Add img-fluid style back [18:42:16] miraheze/landing - Universal-Omega the build passed. [18:46:37] [02puppet] 07dependabot[bot] created 03dependabot/bundler/modules/graylog/rexml-3.4.2 (+1 new commit) 13https://github.com/miraheze/puppet/commit/1ff69c931066 [18:46:39] 02puppet/03dependabot/bundler/modules/graylog/rexml-3.4.2 07dependabot[bot] 031ff69c9 build(deps-dev): bump rexml from 3.3.9 to 3.4.2 in /modules/graylog… [18:46:41] [02puppet] 07dependabot[bot] added the label 'dependencies' to pull request #4526 (build(deps-dev): bump rexml from 3.3.9 to 3.4.2 in /modules/graylog) 13https://github.com/miraheze/puppet/pull/4526 [18:46:43] [02puppet] 07dependabot[bot] opened pull request #4526: build(deps-dev): bump rexml from 3.3.9 to 3.4.2 in /modules/graylog (03main...03dependabot/bundler/modules/graylog/rexml-3.4.2) 13https://github.com/miraheze/puppet/pull/4526 [18:46:45] [02puppet] 07dependabot[bot] added the label 'ruby' to pull request #4526 (build(deps-dev): bump rexml from 3.3.9 to 3.4.2 in /modules/graylog) 13https://github.com/miraheze/puppet/pull/4526 [18:46:47] [02puppet] 07coderabbitai[bot] commented on pull request #4526: --- […] 13https://github.com/miraheze/puppet/pull/4526#issuecomment-3304173088 [18:46:51] [02landing] 07Universal-Omega merged pull request #170: Use some SVGs instead of webp (03main...03svg) 13https://github.com/miraheze/landing/pull/170 [18:46:51] [02landing] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/landing/commit/df575c58f1b687db14106802afae0b0fe41e5db3 [18:46:51] 02landing/03main 07CosmicAlpha 03df575c5 Use some SVGs instead of webp [18:46:52] [02landing] 07Universal-Omega 04deleted 03svg at 0392cdabd 13https://api.github.com/repos/miraheze/landing/commit/92cdabd [18:47:35] miraheze/landing - Universal-Omega the build passed. [18:49:01] [02landing] 07Universal-Omega created 03revert-170-svg (+1 new commit) 13https://github.com/miraheze/landing/commit/ef16e82985e8 [18:49:01] 02landing/03revert-170-svg 07CosmicAlpha 03ef16e82 Revert "Use some SVGs instead of webp"… [18:49:12] [02landing] 07Universal-Omega opened pull request #171: Revert "Use some SVGs instead of webp" (03main...03revert-170-svg) 13https://github.com/miraheze/landing/pull/171 [18:49:22] [02landing] 07coderabbitai[bot] commented on pull request #171:

[…] 13https://github.com/miraheze/landing/pull/171#issuecomment-3304180753 [18:49:58] [02landing] 07Universal-Omega pushed 1 new commit to 03revert-170-svg 13https://github.com/miraheze/landing/commit/b5137c554db8a868686c0b46987064d3e1fe318d [18:49:58] 02landing/03revert-170-svg 07CosmicAlpha 03b5137c5 - [18:50:04] miraheze/landing - Universal-Omega the build passed. [18:50:25] [02landing] 07Universal-Omega merged pull request #171: Revert "Use some SVGs instead of webp" (03main...03revert-170-svg) 13https://github.com/miraheze/landing/pull/171 [18:50:25] [02landing] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/landing/commit/62f6eb4a5e475b9447c73dcf4c41d45ab1fa4e8d [18:50:26] 02landing/03main 07CosmicAlpha 0362f6eb4 Revert "Use some SVGs instead of webp"… [18:50:28] [02landing] 07Universal-Omega 04deleted 03revert-170-svg at 03b5137c5 13https://api.github.com/repos/miraheze/landing/commit/b5137c5 [18:50:32] [02landing] 07Universal-Omega created 03revert-171-revert-170-svg (+1 new commit) 13https://github.com/miraheze/landing/commit/4c72ad856809 [18:50:32] 02landing/03revert-171-revert-170-svg 07CosmicAlpha 034c72ad8 Revert "Revert "Use some SVGs instead of webp""… [18:50:33] [02landing] 07Universal-Omega opened pull request #172: Revert "Revert "Use some SVGs instead of webp"" (03main...03revert-171-revert-170-svg) 13https://github.com/miraheze/landing/pull/172 [18:50:40] [02landing] 07coderabbitai[bot] commented on pull request #172:

[…] 13https://github.com/miraheze/landing/pull/172#issuecomment-3304184616 [18:50:50] miraheze/landing - Universal-Omega the build passed. [18:51:10] miraheze/landing - Universal-Omega the build passed. [19:18:01] PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push [19:23:08] [02mw-config] 07c-oreills opened pull request #6113: T14285: Add wgMFNamespacesWithoutCollapsibleSections to ManageWikiNamespaces.php (07miraheze:03main...07c-oreills:03wgMFNamespacesWithoutCollapsibleSectionsManageWikiNamespaces) 13https://github.com/miraheze/mw-config/pull/6113 [19:26:01] RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push [19:34:06] [02mw-config] 07c-oreills left a file comment in pull request #6113 038461b06: I'm unsure if (a) this is the right format or (b) it's good practice to override the default like this rather than just allow the `mobilefrontend` extension to set the default. Advice very much appreciated! 13https://github.com/miraheze/mw-config/pull/6113#discussion_r2356555188 [19:39:32] miraheze/mw-config - c-oreills the build passed.