[13:43:41] hey folks [13:44:27] the docker report job for ml-staging-codfw is reporting errors due to the deprecated mirrors.wikimedia.org due to [13:44:28] Building debmonitor report for docker-registry.discovery.wmnet/coredns:1.11.3-1 [13:46:51] ah ok there is a pending upgrade for admin_ng [13:50:39] interestingly, nothing stands out in prod clusters [13:51:30] klausman, dpogorzelski o/ - on ml-staging-codfw it seems that there is an image for coredns that should be refreshed with a minor upgrade, ok if I apply it? [14:12:05] I wonder thy I did not catch that during my many coredns deploys the last days [14:30:13] jayme: even you have limits! :D [14:31:08] elukey: but that clearly means I messed up. The image should be docker-registry.discovery.wmnet/coredns1.11:1.11.3-2 now [14:31:17] (version in image name) [14:34:23] yeah there is a pending update for that, it is just a matter of doing an admin_ng apply [14:34:37] not really messed up :D [14:34:43] half messed up then :) [14:35:03] I was looping over kube-environments...probably mis grepped the staging part [14:38:15] okok so I assume it is safe to sync, will do it now [14:39:38] I cannot make it to the SIG meeting, but I wanted to gauge how we felt about kicking the conversation of applying external-services admin_ng diff [14:39:39] s [14:40:59] what sparked this thought again was the loss of krb1002, replaced by krb1003 a couple of days later. Apps were re-deployed with config pointing to the new host, but the admin_ng/external-services hadn't been applied, so that broke krb access to these apps until I applied said diff [14:41:36] in the ~2.5 years running that external-services home-grown "framework", have we ever be saved once by *not* having applied a network policy rule? [14:43:04] for explicitness' sake: I'm not thinking about full admin_ng auto apply, simply the external-services party [14:43:05] *part [14:43:16] I don't think it ever failed us. [14:43:27] ofc it will immediately when we enable auto apply :D [14:44:22] aand deployed [14:45:02] <3 [17:15:42] cluster maintainers please take a look at the meeting notes regarding the outstading k8s component upgrades: https://www.mediawiki.org/wiki/Kubernetes_SIG/Meetings/2026-08-25 [17:16:36] I wanted to talk about how to communicate those so applying them to your clusters is not forgotten. But since we had a limited number of participants we did not do so