[08:08:11] <wikibugs>	 10serviceops, 10Infrastructure-Foundations, 10Puppet-Core, 10SRE, and 5 others: Migrate roles to puppet7 - https://phabricator.wikimedia.org/T349619#9562410 (10MoritzMuehlenhoff)
[12:13:02] <wikibugs>	 10serviceops, 10Data-Engineering, 10MW-on-K8s, 10SRE, 10Patch-For-Review: Migrate mw-page-content-change-enrich to mw-api-int - https://phabricator.wikimedia.org/T357785#9563142 (10BTullis) I'm happy for this change to go ahead. I'll keep an eye on the [[https://grafana-rw.wikimedia.org/d/K9x0c4aVk/flink...
[12:31:52] <wikibugs>	 10serviceops, 10MW-on-K8s, 10SRE, 10Traffic, and 2 others: Migrate internal traffic to k8s - https://phabricator.wikimedia.org/T333120#9563252 (10Clement_Goubert)
[12:33:12] <wikibugs>	 10serviceops, 10Data-Engineering, 10MW-on-K8s, 10SRE: Migrate mw-page-content-change-enrich to mw-api-int - https://phabricator.wikimedia.org/T357785#9563249 (10Clement_Goubert) 05In progress→03Resolved I can confirm that mw-page-content-enrich now requests from mw-api-int (blue) and not the appserver...
[14:23:50] <wikibugs>	 10serviceops, 10ops-codfw: Degraded RAID on mw2442 - https://phabricator.wikimedia.org/T357380#9563553 (10Jhancock.wm) SR185570210 requested replacement disk from dell
[14:40:10] <wikibugs>	 10serviceops, 10SRE, 10ops-codfw: Issues reimaging servers in codfw - https://phabricator.wikimedia.org/T358001#9563665 (10Jhancock.wm) @hnowlan I've replaced the network cable on both of these. These are both connected to a 1G switch so there is no SFP to replace in this case.   If this does not fix the iss...
[14:51:25] <wikibugs>	 10serviceops, 10Content-Transform-Team, 10MW-on-K8s, 10SRE, and 2 others: Create parsoid mediawiki deployment - https://phabricator.wikimedia.org/T357392#9563708 (10akosiaris) 05Open→03In progress p:05Triage→03Medium
[14:51:33] <wikibugs>	 10serviceops, 10MW-on-K8s, 10SRE, 10Traffic, 10Release-Engineering-Team (Seen): Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536#9563710 (10akosiaris)
[15:48:38] <claime>	 hnowlan: all right if I merge and deploy this then? https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1004735 :p
[15:49:18] <hnowlan>	 claime: go for it 
[15:53:08] <claime>	 Deployed to staging, looks like it still works, at least the example req that's in wikitech
[15:56:31] <claime>	 eqiad done
[15:57:26] <claime>	 huh the rate_limit cluster is serving 504s
[15:59:37] <claime>	 yeah it's broken, rolling back
[16:00:46] <claime>	 https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1005475
[16:02:59] <hnowlan>	 weird, that shouldn't be affected 
[16:03:43] <claime>	 hnowlan: images wouldn't display on api.w.o as well
[16:03:49] <claime>	 so there are some shenanigans
[16:05:55] <hnowlan>	 very odd - I can pick it up if you fancy 
[16:18:48] <claime>	 sure yeah
[16:22:43] <wikibugs>	 10serviceops, 10MW-on-K8s, 10SRE, 10Scap, 10Release-Engineering-Team (Now this 🫠): Find a way to address canary releases directly - https://phabricator.wikimedia.org/T358117#9564123 (10Clement_Goubert)
[16:24:09] <wikibugs>	 10serviceops, 10MW-on-K8s, 10SRE, 10Scap, 10Release-Engineering-Team (Now this 🫠): Find a way to address canary releases directly - https://phabricator.wikimedia.org/T358117#9564137 (10Clement_Goubert) p:05Triage→03Medium
[16:29:31] <wikibugs>	 10serviceops, 10MW-on-K8s, 10SRE, 10Scap, 10Release-Engineering-Team (Now this 🫠): Find a way to address canary releases directly - https://phabricator.wikimedia.org/T358117#9564163 (10Clement_Goubert)
[16:54:56] <wikibugs>	 10serviceops, 10MW-on-K8s: Migrate remaining internal MW API traffic to k8s - https://phabricator.wikimedia.org/T357907#9564316 (10Clement_Goubert) 05Open→03In progress p:05Triage→03Medium
[16:55:27] <wikibugs>	 10serviceops, 10MW-on-K8s, 10SRE, 10Traffic, and 2 others: Migrate internal traffic to k8s - https://phabricator.wikimedia.org/T333120#9564307 (10Clement_Goubert)
[20:07:45] <wikibugs>	 10serviceops, 10MW-on-K8s, 10SRE, 10Scap, and 2 others: Scap should check errors coming from mw-on-k8s canaries during deployments - https://phabricator.wikimedia.org/T357402#9565070 (10CodeReviewBot) dancy opened https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/219  Check bare metal and mw-...
[23:37:12] <wikibugs>	 10serviceops, 10Shellbox: Code in Shellbox specific to WMF production - https://phabricator.wikimedia.org/T357949#9565748 (10tstarling) >>! In T357949#9565402, @bd808 wrote: > The .pipeline files are used to create container images for running in Wikimedia production or anywhere else that may choose to run She...
[23:47:14] <wikibugs>	 10serviceops, 10Shellbox: Code in Shellbox specific to WMF production - https://phabricator.wikimedia.org/T357949#9565763 (10bd808) >>! In T357949#9565748, @tstarling wrote: > You're saying that anyone anywhere who wants to run Shellbox in a container, whether they are using MediaWiki or not, should use the ex...