[00:42:53] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-codfw, 06SRE: Q1:rack/setup/install mc-misc200[12] - https://phabricator.wikimedia.org/T372800#10866312 (10Jhancock.wm) 05Open→03Resolved this looks like it's a server failure. the server won't boot. I'm gonna open a troubleshooting ticket rather than reopen the instal...
[10:26:40] <nemo-yiannis>	 👋 I have a pending patch to be deployed: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1151177 can i go ahead ?
[13:05:43] <wikibugs>	 06serviceops, 10function-orchestrator, 10Abstract Wikipedia team (25Q4 (Apr–Jun)), 07OKR-Work: Enable memcached in the orchestrator - https://phabricator.wikimedia.org/T391986#10867458 (10jijiki) 05Open→03Resolved
[13:06:48] <wikibugs>	 06serviceops, 10function-orchestrator, 10Abstract Wikipedia team (25Q4 (Apr–Jun)), 07OKR-Work: Enable memcached in the orchestrator - https://phabricator.wikimedia.org/T391986#10867461 (10jijiki)
[13:10:35] <wikibugs>	 06serviceops: low rate of mw-memcached errors - https://phabricator.wikimedia.org/T371881#10867465 (10jijiki) A chunk of the memcached errors we are observing are due to  T374366, which we have known for a long time and attempted to mitigate (to no avail yet). After chatting with @Muehlenhoff and @JMeybohm, we w...
[14:43:49] <wikibugs>	 06serviceops: PHP Fatal error:  Uncaught MediaWiki\Config\ConfigException: Failed to load configuration from etcd: lost lock - https://phabricator.wikimedia.org/T392197#10867802 (10MSantos)
[15:12:17] <wikibugs>	 06serviceops, 06Security-Team, 13Patch-For-Review, 07SecTeam-Processed: Migrate Security-Team jobs to mw-cron - https://phabricator.wikimedia.org/T388531#10867895 (10Krinkle) >>! In T388531#10865923, @Reedy wrote: > Do we log the commands run anywhere? As if that's the actual solution, to just explicitly s...
[17:51:23] <wikibugs>	 06serviceops, 10MW-on-K8s, 13Patch-For-Review: Allow using helper scripts inside of mwscript-k8s - https://phabricator.wikimedia.org/T378479#10868600 (10Scott_French) All three followup items from T378479#10842481 have now been completed.
[19:22:01] <wikibugs>	 06serviceops, 06Security-Team, 10MW-1.45-notes (1.45.0-wmf.2; 2025-05-20), 13Patch-For-Review, 07SecTeam-Processed: Migrate Security-Team jobs to mw-cron - https://phabricator.wikimedia.org/T388531#10868918 (10Scott_French) One point of note while trying to get up to speed on what was learned from the la...