[08:53:33] good morning o/ [08:56:08] hey Aiko o/ [09:09:22] 06Machine-Learning-Team, 06serviceops, 07Kubernetes: Allow Kubernetes workers to be deployed on Bookworm - https://phabricator.wikimedia.org/T365253#9916776 (10JMeybohm) >>! In T365253#9909637, @elukey wrote: > I have built and uploaded the new dragonfly packages to bookworm-wikimedia, and updated the ml-sta... [09:10:48] 06Machine-Learning-Team, 06serviceops, 07Kubernetes: Allow Kubernetes workers to be deployed on Bookworm - https://phabricator.wikimedia.org/T365253#9916784 (10elukey) 05Open→03Resolved [09:11:05] hello folks! In theory we should be ok in using bookworm for new k8s workers [09:11:10] everything is in place [09:11:55] (even for prod I mean) [09:15:05] Excellent, thank you! [11:27:38] heads up: I am deploying a change for security Contexts (https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1026954) in staging, starting with the experimental NS and then proceeding alphabetically. Beyond services restarting, there should be no disruption. [12:30:19] 06Machine-Learning-Team: Have problem with migrating to LiftWing from ores - https://phabricator.wikimedia.org/T364089#9917377 (10AgnesAbah) @kostajh thanks for your contributions i will be waiting for @achou and @isarantopoulos response [13:43:46] 06Machine-Learning-Team, 06DC-Ops, 10ops-codfw, 06SRE: Q3:rack/setup/install ml-staging2003 - https://phabricator.wikimedia.org/T357415#9917648 (10Jhancock.wm) 05Open→03Resolved a:03Jhancock.wm [14:17:20] 06Machine-Learning-Team: Roll out kserve-inference securityContext change to ML isvcs - https://phabricator.wikimedia.org/T368273 (10klausman) 03NEW [14:33:23] Morning all [14:34:57] heyo Chris! [16:10:00] 06Machine-Learning-Team, 06DC-Ops, 10ops-codfw, 06SRE: hw troubleshooting: memory errors during boot for ml-serve2001.codfw.wmnet - https://phabricator.wikimedia.org/T366670#9918333 (10Jhancock.wm) 05Open→03Resolved a:03Jhancock.wm [16:18:35] 06Machine-Learning-Team, 06DC-Ops, 10ops-codfw, 06SRE: hw troubleshooting: memory errors for ml-serve2007.codfw.wmnet - https://phabricator.wikimedia.org/T366688#9918409 (10Jhancock.wm) @klausman when would you like to schedule this one? I am free Wednesday from 8 to 11 and Thursday/Friday from 8 to 12 cen... [16:21:58] 06Machine-Learning-Team, 06DC-Ops, 10ops-codfw, 06SRE: hw troubleshooting: memory errors for ml-serve2007.codfw.wmnet - https://phabricator.wikimedia.org/T366688#9918431 (10klausman) All of those work for me, with a preference for Thursday, so I'll drain&power-off the machine Thu before 8am your time (1300...