[11:02:31] 06serviceops, 06Growth-Team, 10GrowthExperiments, 10MW-on-K8s, 07Wikimedia-production-error: growthexperiments-fixlinkrecommendationdata-dryrun 'link-recommendation' is not a link recommendation task type error - https://phabricator.wikimedia.org/T395249#10874891 (10Michael) >>! In T395249#10858980, @Cl... [11:11:15] 06serviceops, 10MW-on-K8s: Members of restricted cannot use --dblist - https://phabricator.wikimedia.org/T395521#10874927 (10Clement_Goubert) There seems to be something wrong with the command line parsing for members of `restricted`: ` cgoubert@deploy1003:/srv/deployment-charts/helmfile.d/services/mw-script$... [11:17:04] This issue is *weird* ^ [11:22:27] 06serviceops, 06Growth-Team, 10GrowthExperiments, 10MW-on-K8s, 07Wikimedia-production-error: growthexperiments-fixlinkrecommendationdata-dryrun 'link-recommendation' is not a link recommendation task type error - https://phabricator.wikimedia.org/T395249#10874954 (10Clement_Goubert) The run for `akwiki`... [11:25:49] 06serviceops, 06Security-Team, 10MW-1.45-notes (1.45.0-wmf.2; 2025-05-20), 13Patch-For-Review, 07SecTeam-Processed: Migrate Security-Team jobs to mw-cron - https://phabricator.wikimedia.org/T388531#10874987 (10Clement_Goubert) Re: font, I probably checked a local copy of the script that wasn't up to date... [11:36:24] 06serviceops, 06Security-Team, 10MW-1.45-notes (1.45.0-wmf.2; 2025-05-20), 13Patch-For-Review, 07SecTeam-Processed: Migrate Security-Team jobs to mw-cron - https://phabricator.wikimedia.org/T388531#10875066 (10Clement_Goubert) ` cgoubert@deploy1003:/srv/deployment-charts/helmfile.d/services/mw-cron$ kube... [12:05:40] 06serviceops, 06Security-Team, 10MW-1.45-notes (1.45.0-wmf.2; 2025-05-20), 13Patch-For-Review, 07SecTeam-Processed: Migrate Security-Team jobs to mw-cron - https://phabricator.wikimedia.org/T388531#10875133 (10Clement_Goubert) Apparently I didn't kill it before it deleted all captchas, leading to the con... [12:08:50] 06serviceops, 10MediaWiki-Core-AuthManager, 10MediaWiki-extensions-CentralAuth, 06MediaWiki-Platform-Team, and 4 others: Gradually isolate mediawiki authentication code and infrastructure - https://phabricator.wikimedia.org/T391784#10875156 (10Tgr) I think "Gradually isolate mediawiki authentication code a... [13:29:44] 06serviceops, 10DiscussionTools, 10MediaWiki-REST-API, 06MW-Interfaces-Team, and 3 others: Reroute RESTbase Endpoints for Linter Errors to MediaWiki REST Endpoints - https://phabricator.wikimedia.org/T384216#10875428 (10HCoplin-WMF) [13:55:25] 06serviceops, 10Prod-Kubernetes, 13Patch-For-Review: PodSecurityPolicies will be deprecated with Kubernetes 1.21 - https://phabricator.wikimedia.org/T273507#10875536 (10JMeybohm) Nice! Now this is stalled by the k8s upgrade on the wikikube production clusters ({T341984}) after which we can use VAPs for Media... [14:25:03] 06serviceops, 10CXServer, 06Language and Product Localization, 10RESTBase Sunsetting, and 2 others: Decommission cxserver endpoints from RESTBase - https://phabricator.wikimedia.org/T372753#10875708 (10Nikerabbit) [14:28:29] 06serviceops, 10MediaWiki-Core-AuthManager, 10MediaWiki-extensions-CentralAuth, 06Security-Team, and 4 others: Gradually isolate mediawiki authentication code and infrastructure - https://phabricator.wikimedia.org/T391784#10875728 (10JTweed-WMF) [14:40:37] 06serviceops, 10MediaWiki-Core-AuthManager, 10MediaWiki-extensions-CentralAuth, 06Security-Team, and 4 others: Gradually isolate mediawiki authentication code and infrastructure - https://phabricator.wikimedia.org/T391784#10875799 (10Ladsgroup) >>! In T391784#10875156, @Tgr wrote: > I think "Gradually isol... [15:55:10] 06serviceops, 10Wikidata, 10Wikidata-Query-Service: wikidata-updatequeryservicelag cron job failing - https://phabricator.wikimedia.org/T395814#10876051 (10Clement_Goubert) >>! In T395814#10875953, @Lucas_Werkmeister_WMDE wrote: >> The job has never been flaky before as far as I’m aware. > > @Clement_Gouber... [18:10:25] 06serviceops, 10decommission-hardware, 13Patch-For-Review: decommission mw135[8-9], mw136[4-6], mw137[2-3], mw140[0-4], mw1406, mw14[11-13] - https://phabricator.wikimedia.org/T383227#10876919 (10ops-monitoring-bot) cookbooks.sre.hosts.decommission executed by jasmine@cumin1002 for hosts: `wikikube-worker[10... [18:47:09] 06serviceops, 10decommission-hardware, 13Patch-For-Review: decommission mw135[8-9], mw136[4-6], mw137[2-3], mw140[0-4], mw1406, mw14[11-13] - https://phabricator.wikimedia.org/T383227#10877023 (10ops-monitoring-bot) cookbooks.sre.hosts.decommission executed by jasmine@cumin1002 for hosts: `wikikube-worker102... [22:45:23] 06serviceops, 10MW-on-K8s: Members of restricted cannot use --dblist - https://phabricator.wikimedia.org/T395521#10877806 (10Scott_French) Many thanks to @jrbs for the discussion out of band. As @Clement_Goubert notes in T395521#10874927, the configuration of `job/mw-script.eqiad.k2ozj7tm` looks quite odd: 1.... [22:48:04] 06serviceops, 10MW-on-K8s: Members of restricted cannot use --dblist - https://phabricator.wikimedia.org/T395521#10877809 (10jrbs) >>! In T395521#10877806, @Scott_French wrote: > @jrbs - Could you try deleting `~/.cache/helm` and then retrying a simple test like `mwscript-k8s --comment="testing" --dblist="s1"... [23:32:56] 06serviceops, 10MW-on-K8s, 13Patch-For-Review: Members of restricted cannot use --dblist - https://phabricator.wikimedia.org/T395521#10877883 (10Scott_French) Amazing - thank you very for confirming, @jrbs. I believe https://gerrit.wikimedia.org/r/1152854 should fix this, and should not pose a significant us...