[00:53:03] <wikibugs>	 10netops, 06Infrastructure-Foundations, 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 07Kubernetes: Calico IPv4 block exhaustion on dse-k8s cluster, blocking new node provisioning - https://phabricator.wikimedia.org/T429773#12052339 (10BTullis)
[00:53:58] <wikibugs>	 10netops, 06Infrastructure-Foundations, 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 07Kubernetes: Calico IPv4 block exhaustion on dse-k8s cluster, blocking new node provisioning - https://phabricator.wikimedia.org/T429773#12052340 (10BTullis)
[07:31:33] <wikibugs>	 06Traffic, 10Lift-Wing, 06ServiceOps new, 10ServiceOps-SharedInfra, and 2 others: Host Qwen 3.6-27B as an inference service - https://phabricator.wikimedia.org/T425680#12052786 (10kevinbazira) >>! In T425680#12051237, @gkyziridis wrote: > Tagging @kevinbazira if he has any thoughts on that.  Must you use A...
[09:58:30] <klausman>	 \o I'd like to do the IPIP update for our staging k8s today. Could I get a review for https://gerrit.wikimedia.org/r/c/operations/puppet/+/1305397 ? (and an okay for the implicit pybal restart of the associated cookbook)
[10:02:00] <fabfur>	 ack, are you following https://wikitech.wikimedia.org/wiki/Kubernetes/Clusters/IPIP#Kubernetes_API_(control_plane) ? 
[10:02:07] <klausman>	 Yep
[10:02:16] <klausman>	 The dry run went cleanly
[10:02:37] <klausman>	 I can laso do a PCC run, if desired
[10:03:09] <fabfur>	 +1 for me (given also it's a staging service so I think not super critical :) )
[10:03:18] <klausman>	 merci!
[10:03:40] <klausman>	 will run sre.loadbalancer.migrate-service-ipip now
[10:03:47] <fabfur>	 ok
[10:13:48] <klausman>	 pybal restarting
[10:14:30] <klausman>	 and complete
[10:14:53] <fabfur>	 ack, tnx
[10:26:19] <wikibugs>	 06Traffic, 10Lift-Wing, 06ServiceOps new, 10ServiceOps-SharedInfra, and 2 others: Host Qwen 3.6-27B as an inference service - https://phabricator.wikimedia.org/T425680#12053418 (10gkyziridis) Thnx for your comment @kevinbazira, I tried the above as well but I am having errors: `lang=Shell (Worker_TP0 pid=2...
[10:30:54] <klausman>	 fabfur can I bother you with the reviews for the worker nodes on staging being switched to IPIP as well? Luca has already done the more complex ones, but 2-3 of them remain
[10:31:18] <klausman>	 (after lunch, that is)
[10:32:00] <fabfur>	 sure
[10:35:18] <klausman>	 https://gerrit.wikimedia.org/r/c/operations/puppet/+/1305623 is the first one. Take your time, I'll be out for lunch for a bit
[15:02:47] <wikibugs>	 10netops, 06Infrastructure-Foundations: codfw: upgrade routers (2026) - https://phabricator.wikimedia.org/T417871#12054668 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=2186f648-4746-4ad9-8401-8db932f31c67) set by pt1979@cumin2003 for 2:00:00 on 2 host(s) and their services with reason: j...