[00:01:15] <MacFanBot>	 [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/efdb9058bff4d779eca1e2293dd12aade720ad78
[00:01:15] <MacFanBot>	 02statichelp/03main 07WikiTideBot 03efdb905 Bot: Auto-update Tech namespace pages 2025-09-03 00:01:12
[00:01:15] <MirahezeRelay>	 <abaddriverlol> [1/2] @posix_memalign I've managed to enable WikibaseRepository on exttestwikibeta by temporarily commenting out some lines in ManageWikiExtensions; I still have to investigate what caused this, but you should be able to test wikibase and related extensions now
[00:01:15] <MirahezeRelay>	 <abaddriverlol> [2/2] https://cdn.discordapp.com/attachments/808001911868489748/1412588213926629446/image.png?ex=68b8d6ca&is=68b7854a&hm=2a4ffe447d755e0b72d86d0bc49fd93fb2461494de556db4c4b0eff80f842f9a&
[00:03:53] <MirahezeLSBot>	 !log [somerandomdeveloper@test151] starting deploy of {'config': True} to test151
[00:03:54] <MirahezeLSBot>	 !log [somerandomdeveloper@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s
[00:03:56] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[00:04:00] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[00:05:02] <MirahezeRelay>	 <abaddriverlol> I think I've found the cause, `wbt_type` was dropped in <https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/1110810>, so it probably repeatedly tried to run the script to create the table, but that resulted in errors since the table won't be created anymore
[00:05:34] <MirahezeLSBot>	 !log [somerandomdeveloper@test151] starting deploy of {'config': True} to test151
[00:05:34] <MirahezeLSBot>	 !log [somerandomdeveloper@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s
[00:05:37] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[00:05:40] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[00:06:26] <icinga-miraheze>	 RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 57263MiB (12% inode=99%);
[00:06:52] <icinga-miraheze>	 RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 57544MiB (12% inode=99%);
[00:07:37] <icinga-miraheze>	 RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 56813MiB (12% inode=99%);
[00:08:33] <MacFanBot>	 [02mw-config] 07SomeMWDev created 03wikibase-1.44 (+1 new commit) 13https://github.com/miraheze/mw-config/commit/062f50ccdf31
[00:08:34] <MacFanBot>	 02mw-config/03wikibase-1.44 07SomeRandomDeveloper 03062f50c Remove wbt_type from WikibaseRepository SQL…
[00:09:31] <MacFanBot>	 [02mw-config] 07SomeMWDev opened pull request #6079: Remove wbt_type from WikibaseRepository SQL (03main...03wikibase-1.44) 13https://github.com/miraheze/mw-config/pull/6079
[00:10:26] <miraheze-github>	 miraheze/mw-config - SomeMWDev the build passed.
[00:21:34] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.39, 5.53, 3.49
[00:25:34] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is CRITICAL: LOAD CRITICAL - total load average: 8.11, 6.79, 4.47
[00:29:34] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.73, 7.36, 5.24
[00:31:34] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 4.79, 6.72, 5.29
[00:58:01] <MirahezeLSBot>	 !log [blankeclair@mwtask181] Starting import for lbcwiki (XML: None; Images: images/) (START)
[00:58:02] <MirahezeLSBot>	 !log [blankeclair@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php importImages --wiki=lbcwiki --sleep=1 '--comment=Importing images from http://wiki.lbc.radawriting.com/ ([[phorge:T14215|T14215]])' --search-recursively -- images/ (START)
[00:58:05] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[00:58:08] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:03:47] <MirahezeLSBot>	 !log [blankeclair@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php importImages --wiki=lbcwiki --sleep=1 '--comment=Importing images from http://wiki.lbc.radawriting.com/ ([[phorge:T14215|T14215]])' --search-recursively -- images/ (END - exit=0)
[01:03:48] <MirahezeLSBot>	 !log [blankeclair@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php initSiteStats --wiki=lbcwiki --update (START)
[01:03:49] <MirahezeLSBot>	 !log [blankeclair@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php initSiteStats --wiki=lbcwiki --update (END - exit=0)
[01:03:50] <MirahezeLSBot>	 !log [blankeclair@mwtask181] Finished import for lbcwiki (XML: None; Images: images/) (END - exit=0)
[01:03:50] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:03:54] <BlankEclair>	 huh, already?
[01:03:54] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:03:58] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:04:01] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:06:15] <BlankEclair>	 > Found: 291
[01:06:25] <BlankEclair>	 > 1302 files from `find images/ -type f | wc -l`
[01:29:34] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is CRITICAL: LOAD CRITICAL - total load average: 8.38, 5.21, 3.21
[01:31:34] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.59, 5.91, 3.71
[01:33:34] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 2.27, 4.56, 3.48
[02:23:34] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.90, 6.25, 4.42
[02:27:34] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is CRITICAL: LOAD CRITICAL - total load average: 8.21, 7.17, 5.21
[02:29:34] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 3.71, 6.07, 5.06
[02:31:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.09, 21.13, 13.64
[02:39:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.66, 23.47, 18.01
[02:53:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.45, 23.63, 20.88
[03:07:20] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki job queue has more than 100000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?from=1756865210000&orgId=1&to=1756868840014
[03:51:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.13, 22.53, 23.75
[04:02:20] <icinga-miraheze>	 [Grafana] RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?from=1756865210000&orgId=1&to=1756872110000
[04:03:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.37, 23.12, 23.14
[04:05:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.57, 22.84, 23.06
[04:21:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.39, 23.03, 22.41
[04:31:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.22, 23.99, 23.38
[04:33:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 23.46, 24.08, 23.51
[04:41:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.76, 23.57, 23.58
[04:45:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.65, 23.89, 23.70
[05:04:21] <MirahezeLSBot>	 !log [blankeclair@mwtask181] Starting import for lbcwiki (XML: None; Images: images/) (START)
[05:04:22] <MirahezeLSBot>	 !log [blankeclair@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php importImages --wiki=lbcwiki --sleep=1 '--comment=Importing images from http://wiki.lbc.radawriting.com/ ([[phorge:T14215|T14215]])' --search-recursively -- images/ (START)
[05:04:24] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[05:04:28] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[05:05:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 11.87, 20.71, 22.98
[05:09:32] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 9.75, 14.16, 19.73
[05:35:25] <MirahezeLSBot>	 !log [blankeclair@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php importImages --wiki=lbcwiki --sleep=1 '--comment=Importing images from http://wiki.lbc.radawriting.com/ ([[phorge:T14215|T14215]])' --search-recursively -- images/ (END - exit=0)
[05:35:26] <MirahezeLSBot>	 !log [blankeclair@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php initSiteStats --wiki=lbcwiki --update (START)
[05:35:28] <MirahezeLSBot>	 !log [blankeclair@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php initSiteStats --wiki=lbcwiki --update (END - exit=0)
[05:35:28] <MirahezeLSBot>	 !log [blankeclair@mwtask181] Finished import for lbcwiki (XML: None; Images: images/) (END - exit=0)
[05:35:29] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[05:35:32] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[05:35:35] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[05:35:39] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[06:11:01] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.53, 6.93, 5.32
[06:18:51] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 4.85, 6.73, 6.01
[06:28:38] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.59, 6.81, 6.10
[06:32:33] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 5.79, 6.39, 6.10
[06:55:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.15, 17.58, 11.49
[06:55:37] <MacFanBot>	 [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/d7241e7f7a1c5edbe7df989e2e815202a51f791e
[06:55:37] <MacFanBot>	 02ssl/03main 07WikiTideBot 03d7241e7 Bot: Auto-update domain lists
[06:58:00] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is CRITICAL: LOAD CRITICAL - total load average: 8.14, 5.98, 4.65
[06:59:58] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.44, 6.29, 4.92
[07:01:56] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 6.67, 6.20, 5.04
[07:05:49] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 6.95, 6.57, 5.45
[07:09:44] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 5.97, 6.68, 5.79
[07:16:34] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.31, 6.76, 6.10
[07:20:29] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 2.03, 5.16, 5.68
[07:26:20] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is CRITICAL: LOAD CRITICAL - total load average: 8.11, 6.66, 6.11
[07:28:18] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.39, 6.88, 6.26
[07:31:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.05, 23.04, 21.38
[07:34:11] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 3.31, 5.76, 6.10
[07:35:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.61, 23.79, 22.10
[07:45:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.86, 24.07, 23.00
[07:55:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.36, 23.84, 23.45
[07:59:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.05, 23.77, 23.51
[08:01:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.77, 23.64, 23.49
[08:07:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.28, 23.30, 23.34
[08:07:34] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.52, 6.37, 4.86
[08:09:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.34, 23.14, 23.30
[08:11:34] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 6.40, 6.25, 5.14
[08:21:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.32, 23.27, 23.21
[08:23:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.25, 23.27, 23.23
[08:31:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.57, 23.63, 23.37
[08:49:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 14.96, 21.14, 23.36
[09:03:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.46, 21.66, 21.89
[09:05:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 20.05, 21.45, 21.83
[09:12:34] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.22, 6.18, 4.64
[09:13:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 26.39, 22.47, 21.82
[09:16:29] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 2.17, 5.02, 4.60
[09:17:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 12.92, 20.46, 21.40
[09:19:32] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 7.20, 15.73, 19.55
[09:32:13] <MirahezeRelay>	 <posix_memalign, replying to abaddriverlol> [1/2] I was thinking of making https://wikibase.mirabeta.org the repository, but exttest works too (it'll probably be fine if the client and repository are the same wikis?). 
[09:32:13] <MirahezeRelay>	 <posix_memalign, replying to abaddriverlol> [2/2] I got hit by an error the moment I tried to create a new item. Error 48 on [P554](https://issue-tracker.miraheze.org/P554).
[10:00:32] <icinga-miraheze>	 PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is WARNING: WARNING - NGINX Error Rate is 48%
[10:02:32] <icinga-miraheze>	 RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 21%
[10:05:37] <icinga-miraheze>	 PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 49833MiB (10% inode=99%);
[10:12:17] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.99, 6.91, 5.01
[10:14:14] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is CRITICAL: LOAD CRITICAL - total load average: 8.12, 7.35, 5.40
[10:16:12] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.75, 7.38, 5.64
[10:20:08] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 5.97, 6.72, 5.79
[10:32:26] <icinga-miraheze>	 PROBLEM - cp171 Disk Space on cp171 is WARNING: DISK WARNING - free space: / 49882MiB (10% inode=99%);
[10:57:34] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.99, 5.62, 4.06
[11:02:52] <icinga-miraheze>	 PROBLEM - cp191 Disk Space on cp191 is WARNING: DISK WARNING - free space: / 49925MiB (10% inode=99%);
[11:17:34] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 4.20, 6.52, 6.26
[11:27:34] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.70, 5.68, 5.41
[11:31:34] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 4.48, 5.93, 5.64
[11:49:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.16, 18.86, 14.18
[11:51:32] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 13.92, 17.26, 14.19
[12:10:08] <MacFanBot>	 [02puppet] 07paladox created 03paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/puppet/commit/984a1be4f807
[12:10:08] <MacFanBot>	 02puppet/03paladox-patch-1 07paladox 03984a1be opensearch: increase heap to 4g
[12:10:12] <MacFanBot>	 [02puppet] 07paladox opened pull request #4495: opensearch: increase heap to 4g (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4495
[12:10:18] <MacFanBot>	 [02puppet] 07coderabbitai[bot] commented on pull request #4495: <details> […] 13https://github.com/miraheze/puppet/pull/4495#issuecomment-3248985460
[12:12:00] <MacFanBot>	 [02puppet] 07paladox merged pull request #4495: opensearch: increase heap to 4g (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4495
[12:12:00] <MacFanBot>	 [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/e36cb4543430db7e2e955e9c56ea2dab90fad5bf
[12:12:00] <MacFanBot>	 02puppet/03main 07paladox 03e36cb45 opensearch: increase heap to 4g (#4495)
[12:12:01] <MacFanBot>	 [02puppet] 07paladox 04deleted 03paladox-patch-1 at 03984a1be 13https://api.github.com/repos/miraheze/puppet/commit/984a1be
[12:22:31] <MacFanBot>	 [02mw-config] 07paladox created 03paladox-patch-4 (+1 new commit) 13https://github.com/miraheze/mw-config/commit/d672b40c07fe
[12:22:31] <MacFanBot>	 02mw-config/03paladox-patch-4 07paladox 03d672b40 Set'merge.scheduler.max_thread_count' to 1 in wgCirrusSearchExtraIndexSettings…
[12:22:42] <MacFanBot>	 [02mw-config] 07paladox opened pull request #6080: Set'merge.scheduler.max_thread_count' to 1 in wgCirrusSearchExtraIndexSettings (03main...03paladox-patch-4) 13https://github.com/miraheze/mw-config/pull/6080
[12:23:35] <miraheze-github>	 miraheze/mw-config - paladox the build passed.
[12:33:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 20.94, 17.35, 13.65
[12:33:45] <MacFanBot>	 [02mw-config] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/c1d24bf93905f37de542fe2662725c4e41a8af7e
[12:33:47] <MacFanBot>	 02mw-config/03main 07paladox 03c1d24bf Set'merge.scheduler.max_thread_count' to 1 in wgCirrusSearchExtraIndexSettings (#6080)…
[12:33:49] <MacFanBot>	 [02mw-config] 07paladox merged pull request #6080: Set'merge.scheduler.max_thread_count' to 1 in wgCirrusSearchExtraIndexSettings (03main...03paladox-patch-4) 13https://github.com/miraheze/mw-config/pull/6080
[12:33:51] <MacFanBot>	 [02mw-config] 07paladox 04deleted 03paladox-patch-4 at 03d672b40 13https://api.github.com/repos/miraheze/mw-config/commit/d672b40
[12:34:08] <MirahezeLSBot>	 !log [paladox@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all
[12:34:10] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[12:34:34] <MirahezeLSBot>	 !log [paladox@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 26s
[12:34:37] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[12:34:42] <miraheze-github>	 miraheze/mw-config - paladox the build passed.
[12:35:32] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 15.31, 16.28, 13.69
[12:50:17] <MacFanBot>	 [02mw-config] 07paladox created 03paladox-patch-4 (+1 new commit) 13https://github.com/miraheze/mw-config/commit/1648f1a5eff3
[12:50:17] <MacFanBot>	 02mw-config/03paladox-patch-4 07paladox 031648f1a Set some CirrusSearch configs
[12:50:22] <MacFanBot>	 [02mw-config] 07paladox opened pull request #6081: Set some CirrusSearch configs (03main...03paladox-patch-4) 13https://github.com/miraheze/mw-config/pull/6081
[12:51:23] <miraheze-github>	 miraheze/mw-config - paladox the build passed.
[12:51:33] <MacFanBot>	 [02mw-config] 07paladox merged pull request #6081: Set some CirrusSearch configs (03main...03paladox-patch-4) 13https://github.com/miraheze/mw-config/pull/6081
[12:51:35] <MacFanBot>	 [02mw-config] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/4bf0e2d20219f607f90348f56bcab051ec7e468e
[12:51:36] <MirahezeLSBot>	 !log [paladox@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all
[12:51:37] <MacFanBot>	 [02mw-config] 07paladox 04deleted 03paladox-patch-4 at 031648f1a 13https://api.github.com/repos/miraheze/mw-config/commit/1648f1a
[12:51:39] <MacFanBot>	 02mw-config/03main 07paladox 034bf0e2d Set some CirrusSearch configs (#6081)…
[12:51:39] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[12:52:01] <MirahezeLSBot>	 !log [paladox@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 25s
[12:52:05] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[12:52:05] <MirahezeLSBot>	 !log [paladox@test151] starting deploy of {'pull': 'config', 'config': True} to test151
[12:52:06] <MirahezeLSBot>	 !log [paladox@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 1s
[12:52:07] <MirahezeLSBot>	 !log [paladox@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all
[12:52:09] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[12:52:11] <MirahezeLSBot>	 !log [paladox@test151] starting deploy of {'pull': 'config', 'config': True} to test151
[12:52:12] <MirahezeLSBot>	 !log [paladox@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 0s
[12:52:12] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[12:52:15] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[12:52:18] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[12:52:21] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[12:52:27] <miraheze-github>	 miraheze/mw-config - paladox the build passed.
[12:52:31] <MirahezeLSBot>	 !log [paladox@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 24s
[12:52:35] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[13:03:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.96, 20.93, 17.74
[13:05:32] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 8.03, 16.03, 16.34
[13:10:43] <MacFanBot>	 [02puppet] 07paladox created 03paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/puppet/commit/2e09c4671f2e
[13:10:43] <MacFanBot>	 02puppet/03paladox-patch-1 07paladox 032e09c46 opensearch: Set file_rolling_type to rollingFile…
[13:10:46] <MacFanBot>	 [02puppet] 07paladox opened pull request #4496: opensearch: Set file_rolling_type to rollingFile (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4496
[13:10:52] <MacFanBot>	 [02puppet] 07coderabbitai[bot] commented on pull request #4496: <details> […] 13https://github.com/miraheze/puppet/pull/4496#issuecomment-3249197301
[13:11:02] <MacFanBot>	 [02puppet] 07paladox merged pull request #4496: opensearch: Set file_rolling_type to rollingFile (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4496
[13:11:02] <MacFanBot>	 [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/32f120cc32b5ba261b5b3e68636d404bfa56eae5
[13:11:02] <MacFanBot>	 02puppet/03main 07paladox 0332f120c opensearch: Set file_rolling_type to rollingFile (#4496)…
[13:11:04] <MacFanBot>	 [02puppet] 07paladox 04deleted 03paladox-patch-1 at 032e09c46 13https://api.github.com/repos/miraheze/puppet/commit/2e09c46
[13:14:02] <icinga-miraheze>	 PROBLEM - os162 Puppet on os162 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[13:14:17] <icinga-miraheze>	 PROBLEM - os161 Puppet on os161 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[13:14:29] <icinga-miraheze>	 PROBLEM - os151 Puppet on os151 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[13:14:31] <icinga-miraheze>	 PROBLEM - os201 Puppet on os201 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[13:15:11] <icinga-miraheze>	 PROBLEM - os202 Puppet on os202 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[13:15:35] <icinga-miraheze>	 PROBLEM - os191 Puppet on os191 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[13:18:32] <MacFanBot>	 [02puppet] 07paladox created 03paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/puppet/commit/500951baed2d
[13:18:32] <MacFanBot>	 02puppet/03paladox-patch-1 07paladox 03500951b opensearch: Add support for keeping a maximum number of log files under dailyRollingFile…
[13:18:42] <MacFanBot>	 [02puppet] 07paladox opened pull request #4497: opensearch: Add support for keeping a maximum number of log files under dailyRollingFile (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4497
[13:18:48] <MacFanBot>	 [02puppet] 07coderabbitai[bot] commented on pull request #4497: <details> […] 13https://github.com/miraheze/puppet/pull/4497#issuecomment-3249228331
[13:19:12] <MacFanBot>	 [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/9484b75e3bf8a92edf9293e817ba7f728015cc8a
[13:19:12] <MacFanBot>	 02puppet/03paladox-patch-1 07paladox 039484b75 Update opensearch.pp
[13:19:19] <MacFanBot>	 [02puppet] 07paladox merged pull request #4497: opensearch: Add support for keeping a maximum number of log files under dailyRollingFile (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4497
[13:19:20] <MacFanBot>	 [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/7a5066fb2ea12e6ef78c837ef49adedf136b8aac
[13:19:22] <MacFanBot>	 02puppet/03main 07paladox 037a5066f opensearch: Add support for keeping a maximum number of log files under dailyRollingFile (#4497)…
[13:19:23] <MacFanBot>	 [02puppet] 07paladox 04deleted 03paladox-patch-1 at 039484b75 13https://api.github.com/repos/miraheze/puppet/commit/9484b75
[13:20:02] <icinga-miraheze>	 RECOVERY - os162 Puppet on os162 is OK: OK: Puppet is currently enabled, last run 7 seconds ago with 0 failures
[13:20:20] <icinga-miraheze>	 RECOVERY - os161 Puppet on os161 is OK: OK: Puppet is currently enabled, last run 27 seconds ago with 0 failures
[13:20:28] <icinga-miraheze>	 RECOVERY - os151 Puppet on os151 is OK: OK: Puppet is currently enabled, last run 33 seconds ago with 0 failures
[13:20:34] <icinga-miraheze>	 RECOVERY - os201 Puppet on os201 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures
[13:21:13] <icinga-miraheze>	 RECOVERY - os202 Puppet on os202 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[13:21:36] <icinga-miraheze>	 RECOVERY - os191 Puppet on os191 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[13:36:01] <MacFanBot>	 [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/001a42992074d94675f47ac3b654895122211ab3
[13:36:01] <MacFanBot>	 02puppet/03main 07paladox 03001a429 Update log4j2.properties.erb
[13:37:31] <MacFanBot>	 [02puppet] 07paladox created 03paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/puppet/commit/7c832ef8aaeb
[13:37:31] <MacFanBot>	 02puppet/03paladox-patch-1 07paladox 037c832ef opensearch: fix setting log4j2.properties
[13:37:36] <MacFanBot>	 [02puppet] 07paladox opened pull request #4498: opensearch: fix setting log4j2.properties (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4498
[13:37:43] <MacFanBot>	 [02puppet] 07coderabbitai[bot] commented on pull request #4498: <details> […] 13https://github.com/miraheze/puppet/pull/4498#issuecomment-3249298819
[13:37:46] <MacFanBot>	 [02puppet] 07paladox merged pull request #4498: opensearch: fix setting log4j2.properties (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4498
[13:37:47] <MacFanBot>	 [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/1e17cb138c8b1a83cdb189a910d17afec401780d
[13:37:48] <MacFanBot>	 02puppet/03main 07paladox 031e17cb1 opensearch: fix setting log4j2.properties (#4498)
[13:37:50] <MacFanBot>	 [02puppet] 07paladox 04deleted 03paladox-patch-1 at 037c832ef 13https://api.github.com/repos/miraheze/puppet/commit/7c832ef
[16:33:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.10, 16.47, 10.05
[16:35:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.88, 19.06, 11.80
[16:37:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.42, 20.74, 13.29
[16:39:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.55, 21.71, 14.56
[16:42:49] <MacFanBot>	 [02puppet] 07Universal-Omega created 03os-2.19.3 (+1 new commit) 13https://github.com/miraheze/puppet/commit/ac3051e1620b
[16:42:49] <MacFanBot>	 02puppet/03os-2.19.3 07CosmicAlpha 03ac3051e opensearch: upgrade to 2.19.3
[16:42:54] <MacFanBot>	 [02puppet] 07Universal-Omega opened pull request #4499: opensearch: upgrade to 2.19.3 (03main...03os-2.19.3) 13https://github.com/miraheze/puppet/pull/4499
[16:43:00] <MacFanBot>	 [02puppet] 07coderabbitai[bot] commented on pull request #4499: <details> […] 13https://github.com/miraheze/puppet/pull/4499#issuecomment-3249989710
[16:43:30] <MacFanBot>	 [02puppet] 07github-actions[bot] pushed 1 new commit to 03os-2.19.3 13https://github.com/miraheze/puppet/commit/1765213ececc285b21afd75424c23c79a8c22098
[16:43:30] <MacFanBot>	 02puppet/03os-2.19.3 07github-actions 031765213 CI: lint puppet code to standards…
[16:45:59] <MacFanBot>	 [02puppet] 07Universal-Omega created 03matomo-5.4.0 (+1 new commit) 13https://github.com/miraheze/puppet/commit/89993f271681
[16:45:59] <MacFanBot>	 02puppet/03matomo-5.4.0 07CosmicAlpha 0389993f2 matomo: upgrade to 5.4.0
[16:46:05] <MacFanBot>	 [02puppet] 07Universal-Omega opened pull request #4500: matomo: upgrade to 5.4.0 (03main...03matomo-5.4.0) 13https://github.com/miraheze/puppet/pull/4500
[16:46:12] <MacFanBot>	 [02puppet] 07coderabbitai[bot] commented on pull request #4500: <details> […] 13https://github.com/miraheze/puppet/pull/4500#issuecomment-3249998861
[16:46:44] <MacFanBot>	 [02puppet] 07github-actions[bot] pushed 1 new commit to 03matomo-5.4.0 13https://github.com/miraheze/puppet/commit/dd13fb8aef7c80880db2d45ce761c2e7cc09336f
[16:46:44] <MacFanBot>	 02puppet/03matomo-5.4.0 07github-actions 03dd13fb8 CI: lint puppet code to standards…
[16:47:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.87, 23.76, 18.40
[17:00:56] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki JobQueue backlog is increasing by more than 100 jobs a minute over an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1756915220000&orgId=1&to=1756918856681
[17:01:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.64, 23.94, 21.89
[17:03:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.25, 24.07, 22.19
[17:05:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.05, 23.70, 22.29
[17:07:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.22, 23.79, 22.49
[17:11:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.27, 23.74, 22.80
[17:15:56] <icinga-miraheze>	 [Grafana] RESOLVED: MediaWiki Jobs are rapidly increasing https://grafana.wikitide.net/d/GtxbP1Xnk?from=1756915880000&orgId=1&to=1756919480000
[17:20:57] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki job queue has more than 100000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?from=1756916210000&orgId=1&to=1756920057513[Grafana] RESOLVED: MediaWiki Jobs are rapidly increasing https://grafana.wikitide.net/d/GtxbP1Xnk?from=1756915880000&orgId=1&to=1756919480000
[17:21:49] <icinga-miraheze>	 PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 6.91, 6.18, 4.76
[17:23:34] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.71, 6.33, 4.71
[17:27:49] <icinga-miraheze>	 PROBLEM - os202 Current Load on os202 is CRITICAL: LOAD CRITICAL - total load average: 8.19, 7.43, 5.79
[17:29:34] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 3.24, 5.50, 5.00
[17:29:49] <icinga-miraheze>	 RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 3.87, 5.92, 5.42
[17:31:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.52, 23.65, 23.30
[17:39:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.22, 23.58, 23.56
[18:03:34] <icinga-miraheze>	 PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.47, 6.22, 5.03
[18:03:49] <icinga-miraheze>	 PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.33, 6.65, 5.44
[18:05:49] <icinga-miraheze>	 RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 6.53, 6.68, 5.60
[18:05:57] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki job queue has more than 100000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?from=1756916210000&orgId=1&to=1756922757520[Grafana] FIRING: The mediawiki JobQueue backlog is increasing by more than 100 jobs a minute over an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1756919120000&orgId=1&to=1756922757520
[18:09:34] <icinga-miraheze>	 RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 6.49, 6.44, 5.51
[18:10:57] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki job queue has more than 100000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?from=1756916210000&orgId=1&to=1756923057520[Grafana] RESOLVED: MediaWiki Jobs are rapidly increasing https://grafana.wikitide.net/d/GtxbP1Xnk?from=1756919300000&orgId=1&to=1756922900000
[18:21:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.58, 22.89, 21.95
[18:39:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.31, 23.59, 23.58
[18:47:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.18, 22.83, 23.01
[19:15:15] <MacFanBot>	 [02mw-config] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/816f1b77d8b7c858cc98a813ef301df0f0f4a0a4
[19:15:15] <MacFanBot>	 02mw-config/03main 07CosmicAlpha 03816f1b7 Remove check for srewiki
[19:16:09] <miraheze-github>	 miraheze/mw-config - Universal-Omega the build passed.
[19:51:16] <MacFanBot>	 [02puppet] 07Universal-Omega created 03ossettings (+1 new commit) 13https://github.com/miraheze/puppet/commit/3a0e72c8ccfb
[19:51:16] <MacFanBot>	 02puppet/03ossettings 07CosmicAlpha 033a0e72c opensearch: replace deprecated settings
[19:51:22] <MacFanBot>	 [02puppet] 07Universal-Omega opened pull request #4501: opensearch: replace deprecated settings (03main...03ossettings) 13https://github.com/miraheze/puppet/pull/4501
[19:51:26] <MacFanBot>	 [02puppet] 07Universal-Omega un-readied pull request #4501: opensearch: replace deprecated settings (03main...03ossettings) 13https://github.com/miraheze/puppet/pull/4501
[19:51:28] <MacFanBot>	 [02puppet] 07coderabbitai[bot] commented on pull request #4501: <details> […] 13https://github.com/miraheze/puppet/pull/4501#issuecomment-3250562023
[19:51:57] <MacFanBot>	 [02puppet] 07github-actions[bot] pushed 1 new commit to 03ossettings 13https://github.com/miraheze/puppet/commit/1ce525b22fa1583c69e7a3eb09be189cf038d838
[19:51:58] <MacFanBot>	 02puppet/03ossettings 07github-actions 031ce525b CI: lint puppet code to standards…
[20:07:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 20.52, 22.74, 23.93
[20:17:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.18, 23.20, 23.54
[20:21:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.65, 23.05, 23.48
[20:24:15] <MacFanBot>	 [02puppet] 07Universal-Omega pushed 1 new commit to 03ossettings 13https://github.com/miraheze/puppet/commit/9309885f3605fde0efa118e728ff2b3f84fc2377
[20:24:15] <MacFanBot>	 02puppet/03ossettings 07CosmicAlpha 039309885 Update
[20:25:32] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.42, 23.06, 23.33
[20:26:07] <MacFanBot>	 [02puppet] 07Universal-Omega readied pull request #4501: opensearch: replace deprecated settings (03main...03ossettings) 13https://github.com/miraheze/puppet/pull/4501
[20:45:57] <icinga-miraheze>	 [Grafana] RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?from=1756916210000&orgId=1&to=1756932110000
[21:11:40] <MirahezeLSBot>	 !log [somerandomdeveloper@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/moveBatch.php --wiki=glitchcitywiki --r '[[phab:T14222|Requested]]' /home/somerandomdeveloper/movenamespace.txt (END - exit=0)
[21:11:43] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[21:27:20] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki job queue has more than 100000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?from=1756931210000&orgId=1&to=1756934840014
[21:33:43] <MirahezeRelay>	 <abaddriverlol> [1/2] !log sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/edit.php --wiki=rosettacodewiki -s 'Fix [[phab:T14209|SMW issue]]' -u 'SomeRandomDeveloper_(Miraheze)' Catego
[21:33:43] <MirahezeRelay>	 <abaddriverlol> [2/2] ry:Português_Puro < /home/somerandomdeveloper/T14209.txt
[21:34:05] <MirahezeRelay>	 <abaddriverlol> RIP log bot
[21:35:05] <MirahezeRelay>	 <bluemoon0332> RIP, thank you for your service
[21:39:04] <icinga-miraheze>	 PROBLEM - mwtask181 Disk Space on mwtask181 is WARNING: DISK WARNING - free space: / 15103MiB (6% inode=93%);
[21:41:04] <icinga-miraheze>	 PROBLEM - mwtask181 Disk Space on mwtask181 is CRITICAL: DISK CRITICAL - free space: / 9399MiB (4% inode=93%);
[21:41:18] <MirahezeRelay>	 <abaddriverlol> uhh what
[21:41:21] <MirahezeRelay>	 <abaddriverlol> 6gb in 2mins?
[21:41:30] <MirahezeRelay>	 <bluemoon0332> me when I eat whole GBs in seconds
[21:41:44] <MirahezeRelay>	 <bluemoon0332> someone is downloading a dump I guess
[21:41:56] <MirahezeRelay>	 <abaddriverlol> probably
[21:44:24] <MirahezeRelay>	 <bluemoon0332> @abaddriverlol may not hurt to check what processes are running in mwtask to confirm that, just in case
[21:44:27] <MacFan4000>	 logbot is still connected
[21:44:32] <MacFan4000>	 !log test
[21:44:35] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[21:45:00] <MirahezeRelay>	 <abaddriverlol> !log sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/edit.php --wiki=rosettacodewiki -s 'Fix [[phab:T14209|SMW issue]]' -u 'SomeRandomDeveloper_(Miraheze)' Category:Português_Puro < /home/somerandomdeveloper/T14209.txt
[21:45:04] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[21:45:05] <MirahezeRelay>	 <abaddriverlol> ???
[21:45:06] <MirahezeRelay>	 <abaddriverlol> ah
[21:45:07] <MirahezeRelay>	 <abaddriverlol> thanks
[21:46:12] <MacFan4000>	 i think it may be because your first attempt came through as two lines meaning it had [1/2] at the beginning
[21:46:35] <MirahezeRelay>	 <abaddriverlol> oh yeah
[21:46:45] <MirahezeRelay>	 <abaddriverlol> I copied it from termius and it had a newline for some reason
[21:47:50] <MirahezeRelay>	 <abaddriverlol, replying to bluemoon0332> yeah i think there is some image stuff going on
[21:49:09] <MirahezeLSBot>	 !log [somerandomdeveloper@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php MirahezeMagic:AssignImportedEdits --wiki=skibiditoiletwiki --import-prefix=fandom:skibidi-toilet --from=Piano1Forte2_Bot --no-run (END - exit=256)
[21:49:12] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[21:50:08] <MirahezeRelay>	 <cosmicalpha, replying to abaddriverlol> might be ia backups?
[21:50:08] <MirahezeLSBot>	 !log [somerandomdeveloper@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php MirahezeMagic:AssignImportedEdits --wiki=skibiditoiletwiki --import-prefix=fandom:skibidi-toilet --from='Piano1Forte2_Bot' --no-run (END - exit=256)
[21:50:11] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[21:50:19] <MirahezeRelay>	 <abaddriverlol, replying to MirahezeLSBot> [1/2] > Invalid 'from' user.
[21:50:19] <MirahezeRelay>	 <abaddriverlol, replying to MirahezeLSBot> [2/2] what
[21:50:28] <MirahezeRelay>	 <abaddriverlol, replying to cosmicalpha> gzip was using 96% cpu according to htop
[21:50:52] <MirahezeRelay>	 <cosmicalpha> its likely backups I think.
[21:52:23] <MirahezeLSBot>	 !log [somerandomdeveloper@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php MirahezeMagic:AssignImportedEdits --wiki=skibiditoiletwiki --import-prefix=fandom:skibidi-toilet --from='Piano1Forte2 Bot' --no-run (END - exit=0)
[21:52:26] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[21:53:25] <MirahezeLSBot>	 !log [somerandomdeveloper@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php MirahezeMagic:AssignImportedEdits --wiki=skibiditoiletwiki --import-prefix=fandom:skibidi-toilet --from='Piano1Forte2 Bot' (END - exit=0)
[21:53:28] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[21:59:04] <icinga-miraheze>	 RECOVERY - mwtask181 Disk Space on mwtask181 is OK: DISK OK - free space: / 29724MiB (13% inode=93%);
[21:59:36] <MacFan4000>	 I just deleted the remaining image dumps from last months failures which freed up 20 GB
[22:03:44] <MirahezeRelay>	 <abaddriverlol> nice
[22:06:38] <MacFanBot>	 [02ssl] 07SomeMWDev created 03T14213 (+1 new commit) 13https://github.com/miraheze/ssl/commit/ffbbb5b04d50
[22:06:39] <MacFanBot>	 02ssl/03T14213 07SomeRandomDeveloper 03ffbbb5b T14213: Add redirect for levelsharesquarewiki
[22:06:57] <MacFanBot>	 [02ssl] 07SomeMWDev opened pull request #883: T14213: Add redirect for levelsharesquarewiki (03main...03T14213) 13https://github.com/miraheze/ssl/pull/883
[22:09:11] <MacFanBot>	 [02ssl] 07SomeMWDev pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/6bbcc080a8e00534964322e93f29efff82df0d37
[22:09:11] <MacFanBot>	 02ssl/03main 07SomeRandomDeveloper 036bbcc08 T14213: Add redirect for levelsharesquarewiki (#883)
[22:09:11] <MacFanBot>	 [02ssl] 07SomeMWDev merged pull request #883: T14213: Add redirect for levelsharesquarewiki (03main...03T14213) 13https://github.com/miraheze/ssl/pull/883
[22:09:13] <MacFanBot>	 [02ssl] 07SomeMWDev 04deleted 03T14213 at 03ffbbb5b 13https://api.github.com/repos/miraheze/ssl/commit/ffbbb5b
[22:09:48] <MacFanBot>	 [02ssl] 07coderabbitai[bot] commented on pull request #883: Adds a new top-level redirect mapping named levelsharesquarewiki in redirects.yaml, pointing levelsharesquare.miraheze.org to wiki.levelsharesquare.com with specified SSL certificate (miraheze-origin-cert), CA (Cloudflare), and HSTS policy (strict). […] 13https://github.com/miraheze/ssl/pull/883#issuecomment-3250971605
[22:13:34] <MacFanBot>	 [02ssl] 07SomeMWDev created 03T14221 (+1 new commit) 13https://github.com/miraheze/ssl/commit/fd3b669f51cf
[22:13:34] <MacFanBot>	 02ssl/03T14221 07SomeRandomDeveloper 03fd3b669 T14221: Redirect apstverse.miraheze.org to apstrackt.miraheze.org
[22:13:45] <MacFanBot>	 [02ssl] 07SomeMWDev opened pull request #884: T14221: Redirect apstverse.miraheze.org to apstrackt.miraheze.org (03main...03T14221) 13https://github.com/miraheze/ssl/pull/884
[22:14:33] <MacFanBot>	 [02ssl] 07SomeMWDev merged pull request #884: T14221: Redirect apstverse.miraheze.org to apstrackt.miraheze.org (03main...03T14221) 13https://github.com/miraheze/ssl/pull/884
[22:14:34] <MacFanBot>	 [02ssl] 07SomeMWDev pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/f78760a6eaf38f5b9dbf54ec1e275e61e5497612
[22:14:34] <MacFanBot>	 02ssl/03main 07SomeRandomDeveloper 03f78760a T14221: Redirect apstverse.miraheze.org to apstrackt.miraheze.org (#884)
[22:14:35] <MacFanBot>	 [02ssl] 07SomeMWDev 04deleted 03T14221 at 03fd3b669 13https://api.github.com/repos/miraheze/ssl/commit/fd3b669
[22:15:27] <MacFanBot>	 [02ssl] 07coderabbitai[bot] commented on pull request #884: Added a new top-level redirect entry in redirects.yaml mapping apstverse.miraheze.org to apstrackt.miraheze.org with specified SSL and HSTS settings. No other entries were changed. […] 13https://github.com/miraheze/ssl/pull/884#issuecomment-3250985505
[22:24:22] <MacFanBot>	 [02MirahezeMagic] 07Universal-Omega created 03AssignImportedEdits-cleanup (+1 new commit) 13https://github.com/miraheze/MirahezeMagic/commit/9043b8120f30
[22:24:22] <MacFanBot>	 02MirahezeMagic/03AssignImportedEdits-cleanup 07CosmicAlpha 039043b81 AssignImportedEdits: cleanup
[22:24:27] <MacFanBot>	 [02MirahezeMagic] 07Universal-Omega opened pull request #591: AssignImportedEdits: cleanup (03main...03AssignImportedEdits-cleanup) 13https://github.com/miraheze/MirahezeMagic/pull/591
[22:24:33] <MacFanBot>	 [02MirahezeMagic] 07coderabbitai[bot] commented on pull request #591: <details> […] 13https://github.com/miraheze/MirahezeMagic/pull/591#issuecomment-3251001915
[22:26:23] <MacFanBot>	 [02MirahezeMagic] 07github-actions[bot] pushed 1 new commit to 03AssignImportedEdits-cleanup 13https://github.com/miraheze/MirahezeMagic/commit/ac1bff60a34d2ae7efef2b47bedecbdb1d6b7c1a
[22:26:23] <MacFanBot>	 02MirahezeMagic/03AssignImportedEdits-cleanup 07github-actions 03ac1bff6 CI: lint code to MediaWiki standards…
[22:28:09] <miraheze-github>	 miraheze/MirahezeMagic - Universal-Omega the build has errored.
[22:28:18] <MacFanBot>	 [02MirahezeMagic] 07Universal-Omega pushed 1 new commit to 03AssignImportedEdits-cleanup 13https://github.com/miraheze/MirahezeMagic/commit/6410d10b545ec96a1cd50b9fbc58bb5012f1fd47
[22:28:19] <MacFanBot>	 02MirahezeMagic/03AssignImportedEdits-cleanup 07CosmicAlpha 036410d10 Update
[22:40:26] <miraheze-github>	 miraheze/MirahezeMagic - Universal-Omega the build passed.
[22:55:03] <MacFanBot>	 [02puppet] 07Universal-Omega pushed 1 new commit to 03ossettings 13https://github.com/miraheze/puppet/commit/e21c36480ef7dfe8d5ad8620276082b68a8c58e9
[22:55:03] <MacFanBot>	 02puppet/03ossettings 07CosmicAlpha 03e21c364 Update
[23:02:20] <icinga-miraheze>	 [Grafana] RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?from=1756931210000&orgId=1&to=1756940510000
[23:42:05] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 15.88, 20.96, 23.31
[23:46:05] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.41, 22.41, 23.30
[23:48:05] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 17.09, 20.40, 22.46
[23:58:05] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.67, 20.91, 21.23