[08:03:40] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411688 (10ops-monitoring-bot) depool host wikikube-worker[2070-2071].codfw.wmnet by jelto@cumin1002 with reason: Reimage node to bookworm + containerd [08:04:52] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411689 (10ops-monitoring-bot) Cookbook cookbooks.sre.k8s.pool-depool-node started by jelto@cumin1002 depool for host wikikube-worker[2070-2071].codfw.w... [08:10:53] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411700 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1002 for host wikikube-worker2071.codfw.wmnet with OS book... [08:10:55] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411701 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1002 for host wikikube-worker2070.codfw.wmnet with OS book... [08:51:27] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411733 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1002 for host wikikube-worker2070.codfw.wmnet with OS bookworm... [08:55:03] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411738 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1002 for host wikikube-worker2071.codfw.wmnet with OS bookworm... [08:55:33] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411739 (10ops-monitoring-bot) pool host wikikube-worker2070.codfw.wmnet by jelto@cumin1002 with reason: None [08:55:34] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411740 (10ops-monitoring-bot) Cookbook cookbooks.sre.k8s.pool-depool-node started by jelto@cumin1002 pool for host wikikube-worker2070.codfw.wmnet comp... [08:57:54] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411752 (10ops-monitoring-bot) depool host wikikube-worker[2068-2069].codfw.wmnet by jelto@cumin1002 with reason: Reimage node to bookworm + containerd [08:59:05] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411753 (10ops-monitoring-bot) Cookbook cookbooks.sre.k8s.pool-depool-node started by jelto@cumin1002 depool for host wikikube-worker[2068-2069].codfw.w... [08:59:51] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411754 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1002 for host wikikube-worker2069.codfw.wmnet with OS book... [08:59:53] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411755 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1002 for host wikikube-worker2068.codfw.wmnet with OS book... [09:11:32] o/ trying to migrate some cirrus operations to mwscript-k8s and facing connectivity issues between mw and elastic, checked envoy & egress of the deployed mwscript release and I see nothing wrong there, do you have suggestions on where to look? [09:13:19] wondering if there's a race issue, i.e. since the mw container entry point is the script itself how do we make sure that the envoy container is ready before that mwscript entry point is started? [09:16:16] o/ [09:16:31] do you have a specific patch? (ignorant about it but I can help in case) [09:19:06] elukey: no patch but a simple failure running e.g. mwscript-k8s --follow -- extensions/CirrusSearch/maintenance/Metastore.php --wiki testwiki --cluster codfw --show-index-version [09:20:13] <_joe_> dcausse: wirte a task; I assume the problem might be the php container starting executing before the tls proxy is up [09:20:26] _joe_: ok will do [09:30:13] <_joe_> thanks :) [09:37:46] 06serviceops: Mediawiki maint scripts using service proxied by the tls proxy might fail when running with mwscript-k8s - https://phabricator.wikimedia.org/T382398 (10dcausse) 03NEW [09:40:34] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411850 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1002 for host wikikube-worker2069.codfw.wmnet with OS bookworm... [09:43:52] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411854 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1002 for host wikikube-worker2068.codfw.wmnet with OS bookworm... [09:44:13] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411856 (10ops-monitoring-bot) pool host wikikube-worker2068.codfw.wmnet by jelto@cumin1002 with reason: None [09:44:17] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411857 (10ops-monitoring-bot) Cookbook cookbooks.sre.k8s.pool-depool-node started by jelto@cumin1002 pool for host wikikube-worker2068.codfw.wmnet comp... [09:52:59] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-eqiad to containerd - https://phabricator.wikimedia.org/T377876#10411880 (10JMeybohm) [09:53:13] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-eqiad to containerd - https://phabricator.wikimedia.org/T377876#10411881 (10JMeybohm) [09:54:43] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411885 (10ops-monitoring-bot) pool host wikikube-worker2069.codfw.wmnet by jelto@cumin1002 with reason: None [09:54:46] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411886 (10ops-monitoring-bot) Cookbook cookbooks.sre.k8s.pool-depool-node started by jelto@cumin1002 pool for host wikikube-worker2069.codfw.wmnet comp... [09:55:01] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411891 (10ops-monitoring-bot) pool host wikikube-worker2071.codfw.wmnet by jelto@cumin1002 with reason: None [09:55:03] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411892 (10ops-monitoring-bot) Cookbook cookbooks.sre.k8s.pool-depool-node started by jelto@cumin1002 pool for host wikikube-worker2071.codfw.wmnet comp... [09:57:12] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411893 (10ops-monitoring-bot) depool host kubernetes[2036-2039].codfw.wmnet by jelto@cumin1002 with reason: Renaming nodes [09:59:30] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411908 (10ops-monitoring-bot) Cookbook cookbooks.sre.k8s.pool-depool-node started by jelto@cumin1002 depool for host kubernetes[2036-2039].codfw.wmnet... [10:04:54] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-eqiad to containerd - https://phabricator.wikimedia.org/T377876#10411921 (10ops-monitoring-bot) Started reimage of nodes in wikikube-eqiad cluster: * wikikube-worker[1007,1021,1080,1287].eqiad.wmnet [10:04:55] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10411922 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.rename started by jelto@cumin1002 from kubernetes2036 to wikikube-worker2047 completed: - ku... [10:20:01] 06serviceops, 06DC-Ops, 06Infrastructure-Foundations, 10netops, and 4 others: Reimage wikikube-worker1290 in eqiad as a replacement for wikikube-ctrl1001 - https://phabricator.wikimedia.org/T379790#10411941 (10JMeybohm) [10:20:09] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-eqiad to containerd - https://phabricator.wikimedia.org/T377876#10411943 (10JMeybohm) [10:20:15] 06serviceops, 06DC-Ops, 06Infrastructure-Foundations, 10netops, and 4 others: Reimage wikikube-worker1290 in eqiad as a replacement for wikikube-ctrl1001 - https://phabricator.wikimedia.org/T379790#10411942 (10JMeybohm) [10:23:42] 06serviceops, 10ChangeProp, 06Content-Transform-Team, 10MediaWiki-Core-HTTP-Cache, and 3 others: Reduce the number of resource_change and resource_purge events emitted due to template changes - https://phabricator.wikimedia.org/T369898#10411969 (10Joe) >>! In T369898#10409773, @daniel wrote: >>>! In T36989... [10:56:04] 06serviceops, 10Add-Link, 10Growth-Structured-Tasks, 06Growth-Team, and 2 others: Architecture Conversation: linkrecommendations service - how to handle user-initiated requests? - https://phabricator.wikimedia.org/T382404 (10Michael) 03NEW [10:58:02] 06serviceops, 10Add-Link, 10Growth-Structured-Tasks, 06Growth-Team, and 2 others: Architecture Conversation: linkrecommendations service - how to handle user-initiated requests? - https://phabricator.wikimedia.org/T382404#10412028 (10Michael) (This is technically not a "service-deployment-request" because... [11:09:06] 06serviceops, 06Content-Transform-Team, 10WMDE-TechWish-Maintenance, 07Epic, 10Maps (Kartotherian): Allow Kartotherian to use a local HTTP proxy - https://phabricator.wikimedia.org/T381257#10412060 (10elukey) 05Open→03Resolved a:03elukey [11:16:05] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412098 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.rename started by jelto@cumin1002 from kubernetes2036 to wikikube-work... [11:23:47] 06serviceops, 10MW-on-K8s: Allow members of restricted to run maintenance scripts - https://phabricator.wikimedia.org/T378429#10412116 (10jrbs) Just in case it's helpful for more use cases: Trust and Safety routinely uses maintenance scripts for various support stuff but also for removing illegal content when... [11:26:17] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412123 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.rename started by jelto@cumin1002 from kubernetes2037 to wikikube-work... [11:34:41] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412134 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.rename started by jelto@cumin1002 from kubernetes2038 to wikikube-work... [11:41:14] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412137 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.rename started by jelto@cumin1002 from kubernetes2039 to wikikube-work... [11:46:13] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412140 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1002 for host wikikube-worker2189.co... [11:46:14] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412141 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1002 for host wikikube-worker2188.co... [12:02:14] 06serviceops, 10Add-Link, 10Growth-Structured-Tasks, 06Growth-Team, and 2 others: Architecture Conversation: linkrecommendations service - how to handle user-initiated requests? - https://phabricator.wikimedia.org/T382404#10412181 (10akosiaris) Thanks for this writeup. Couple of comments below. * If not a... [12:38:17] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412271 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1002 for host wikikube-worker2188.codfw.... [12:41:31] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412286 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1002 for host wikikube-worker2189.codfw.... [12:45:20] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412289 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1002 for host wikikube-worker2190.co... [12:45:21] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412290 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1002 for host wikikube-worker2191.co... [13:24:26] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-eqiad to containerd - https://phabricator.wikimedia.org/T377876#10412363 (10ops-monitoring-bot) Started reimage of nodes in wikikube-eqiad cluster: * wikikube-worker[1280-1284].eqiad.wmnet [13:26:41] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Migrate wikikube-eqiad to containerd - https://phabricator.wikimedia.org/T377876#10412374 (10ops-monitoring-bot) Started reimage of nodes in wikikube-eqiad cluster: * wikikube-worker[1291-1295].eqiad.wmnet [13:39:16] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412411 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1002 for host wikikube-worker2191.codfw.... [14:03:25] hnowlan: great job with the videoscalers :) [14:03:42] is mercurious something that we'll use more broadly? First time that I hear it [14:04:18] elukey: we're definitely considering it, no specific plans right now but it has a lot of potential [14:06:09] very nice yes [14:09:58] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412500 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1002 for host wikikube-worker2190.codfw.... [14:10:20] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412501 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1002 for host wikikube-worker2190.co... [14:12:46] indeed, great work! was video scaling the last work load(s) to run on the baremetal job runners, are these good to go in January? [14:20:25] moritzm: yep, I'll be reclaiming them first thing after I come back [14:21:03] 06serviceops, 10MW-on-K8s, 10TimedMediaHandler, 10MediaWiki-Platform-Team (Radar), and 3 others: Create maintenance script to execute jobs provided in json format from standard input - https://phabricator.wikimedia.org/T369048#10412542 (10Krinkle) I noticed that the new EventBus/runSingleJob.php script... [14:22:50] excellent :-) [15:13:58] 06serviceops, 06MediaWiki-Platform-Team, 07Epic: Migrate Wikimedia production from PHP 8.1 to PHP 8.3 - https://phabricator.wikimedia.org/T360995#10412702 (10Tgr) [15:17:37] 06serviceops, 10MW-on-K8s, 10TimedMediaHandler, 13Patch-For-Review, 07Video: Add logic to scap to create videoscaling Job each release - https://phabricator.wikimedia.org/T371700#10412753 (10hnowlan) 05Open→03Resolved a:03hnowlan A few successful scaps have rolled out new images as expected. [15:20:30] 06serviceops, 06collaboration-services, 06DC-Ops, 10ops-codfw, and 2 others: Comm Error: backplane 0 when reimaging wikikube-worker2190 - https://phabricator.wikimedia.org/T382420 (10Jelto) 03NEW [15:30:41] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412805 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1002 for host wikikube-worker2190.codfw.... [15:38:34] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412843 (10ops-monitoring-bot) pool host wikikube-worker[2188-2189,2191].codfw.wmnet by jelto@cumin1002 with reason: None [15:38:37] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412844 (10ops-monitoring-bot) Cookbook cookbooks.sre.k8s.pool-depool-node started by jelto@cumin1002 pool for host wikikube-worke... [15:39:46] 06serviceops, 06DC-Ops, 10ops-codfw, 10Prod-Kubernetes, 07Kubernetes: Relabel codfw kubernetes nodes - https://phabricator.wikimedia.org/T382422 (10Jelto) 03NEW [15:42:22] There's a bunch of pending admin_ng changes related to external services (postgresql, cas-idp-test) and kartotherian [15:43:11] claime: they often don't get rolled out to all DCs, I would just deploy [15:43:18] to all k8s clusters rather [15:43:25] I'll apply them in a few minutes if there's no objections [15:43:27] yeah [15:45:01] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412899 (10ops-monitoring-bot) depool host wikikube-worker[2065,2067].codfw.wmnet by jelto@cumin1002 with reason: Reimage node to... [15:46:14] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412901 (10ops-monitoring-bot) Cookbook cookbooks.sre.k8s.pool-depool-node started by jelto@cumin1002 depool for host wikikube-wor... [15:47:09] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412903 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1002 for host wikikube-worker2065.co... [15:47:11] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10412904 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1002 for host wikikube-worker2067.co... [15:59:20] 06serviceops, 06DC-Ops, 10ops-codfw, 10Prod-Kubernetes, 07Kubernetes: Relabel codfw kubernetes nodes - https://phabricator.wikimedia.org/T382422#10412942 (10Jhancock.wm) a:03Jhancock.wm [15:59:37] 06serviceops, 06collaboration-services, 06DC-Ops, 10ops-codfw, and 3 others: Comm Error: backplane 0 when reimaging wikikube-worker2190 - https://phabricator.wikimedia.org/T382420#10412944 (10Jhancock.wm) a:03Jhancock.wm [16:29:52] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10413021 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1002 for host wikikube-worker2067.codfw.... [16:34:38] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10413037 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1002 for host wikikube-worker2065.codfw.... [16:37:22] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10413038 (10ops-monitoring-bot) pool host wikikube-worker[2065,2067].codfw.wmnet by jelto@cumin1002 with reason: None [16:37:23] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10413039 (10ops-monitoring-bot) Cookbook cookbooks.sre.k8s.pool-depool-node started by jelto@cumin1002 pool for host wikikube-worke... [16:39:54] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10413041 (10ops-monitoring-bot) depool host wikikube-worker[2063-2064].codfw.wmnet by jelto@cumin1002 with reason: Reimage node to... [16:41:03] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10413046 (10ops-monitoring-bot) Cookbook cookbooks.sre.k8s.pool-depool-node started by jelto@cumin1002 depool for host wikikube-wor... [16:42:00] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10413061 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1002 for host wikikube-worker2063.co... [16:42:09] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10413062 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1002 for host wikikube-worker2064.co... [17:02:41] 06serviceops, 10API Platform (RESTBase Deprecation Roadmap): Migrate node-based services in production to node16 - https://phabricator.wikimedia.org/T308371#10413110 (10Jdforrester-WMF) [17:24:45] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10413194 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1002 for host wikikube-worker2063.codfw.... [17:28:58] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10413207 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1002 for host wikikube-worker2064.codfw.... [17:49:19] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10413348 (10ops-monitoring-bot) pool host wikikube-worker[2063-2064].codfw.wmnet by jelto@cumin1002 with reason: None [17:49:20] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate wikikube-codfw to containerd - https://phabricator.wikimedia.org/T377877#10413349 (10ops-monitoring-bot) Cookbook cookbooks.sre.k8s.pool-depool-node started by jelto@cumin1002 pool for host wikikube-worke... [18:23:01] 06serviceops, 06Abstract Wikipedia team, 10function-evaluator, 13Patch-For-Review: Have SRE provide a production-ready Rust image upstream - https://phabricator.wikimedia.org/T380807#10413489 (10Jdforrester-WMF) >>! In T380807#10405269, @akosiaris wrote: > What are the support requirements for such an imag... [18:24:22] 06serviceops, 06Abstract Wikipedia team, 10function-evaluator, 13Patch-For-Review: Have SRE provide a production-ready Rust image upstream - https://phabricator.wikimedia.org/T380807#10413501 (10Jdforrester-WMF) [18:57:17] 06serviceops, 06collaboration-services, 06DC-Ops, 10ops-codfw, and 3 others: Comm Error: backplane 0 when reimaging wikikube-worker2190 - https://phabricator.wikimedia.org/T382420#10413692 (10Jhancock.wm) 05Open→03Resolved reseated all cables connected to the backplane and the connection on the mai... [19:16:11] 06serviceops, 06MW-Interfaces-Team, 10RESTBase Sunsetting, 10MW-1.44-notes (1.44.0-wmf.4; 2024-11-19), 07User-notice: Switchover plan from RESTbase to REST Gateway for rest_v1/page/html and rest_v1/page/title endpoints - https://phabricator.wikimedia.org/T374683#10413803 (10HCoplin-WMF) [19:43:05] 06serviceops, 06DC-Ops, 10ops-codfw, 10Prod-Kubernetes, and 2 others: Relabel codfw kubernetes nodes - https://phabricator.wikimedia.org/T382422#10413878 (10Jhancock.wm) 05Open→03Resolved