[00:03:23] 10netops, 06Infrastructure-Foundations: codfw: upgrade routers (2026) - https://phabricator.wikimedia.org/T417871#11812465 (10Papaul) Planing on doing the mr router on April 16th at 10am CT [04:54:48] 10Mail, 06Infrastructure-Foundations, 10Phabricator: Mails do not arrive any more - https://phabricator.wikimedia.org/T423055#11812591 (10Aklapper) 05Open→03Stalled You could check/search on https://phabricator.wikimedia.org/mail/. (I have not experienced a problem with my Phabricator mail notifications... [06:06:53] 10Mail, 06Infrastructure-Foundations, 10Phabricator: Mails do not arrive any more - https://phabricator.wikimedia.org/T423055#11812685 (10IKhitron) >>! In T423055#11812591, @Aklapper wrote: > You could check/search on https://phabricator.wikimedia.org/mail/. > (I have not experienced a problem with my Phabri... [06:10:58] 10Mail, 06Infrastructure-Foundations, 10Phabricator: Mails do not arrive any more - https://phabricator.wikimedia.org/T423055#11812686 (10Aklapper) 05Stalled→03Invalid Glad it works again for you! [06:13:38] 10Mail, 06Infrastructure-Foundations, 10Phabricator: Mails do not arrive any more - https://phabricator.wikimedia.org/T423055#11812689 (10IKhitron) Thank you. [12:13:25] 10netops, 06Infrastructure-Foundations, 10observability, 10Prod-Kubernetes, and 5 others: Increase visibility of kubernetes network status - https://phabricator.wikimedia.org/T356877#11814024 (10JMeybohm) >>! In T356877#11808865, @cmooney wrote: > Broadly the patch submitted looked good to me, though I see... [12:58:08] 10netops, 06Infrastructure-Foundations, 10observability, 10Prod-Kubernetes, and 5 others: Increase visibility of kubernetes network status - https://phabricator.wikimedia.org/T356877#11814131 (10cmooney) >>! In T356877#11814024, @JMeybohm wrote: > We can also run the prometheus bird exporter as a sidecar t... [14:24:27] moritzm are you still around? I'm having an issue with reprepro, ref https://phabricator.wikimedia.org/P90344 . Not urgent but just wanna make sure I didn't break anything [14:28:49] 10Mail, 06collaboration-services, 06Infrastructure-Foundations, 10VPS-project-Phabricator: @wikimedia.org email addresses don't seem to be receiving emails sent by the test Phabricator instance - https://phabricator.wikimedia.org/T422559#11814640 (10LSobanski) p:05Triage→03Low [14:36:45] 10CFSSL-PKI, 06Infrastructure-Foundations: nobody really needs or wants 521 - https://phabricator.wikimedia.org/T338822#11814701 (10LSobanski) 05Open→03Declined [14:46:19] 10SRE-tools, 06Infrastructure-Foundations, 10Puppet-Core, 06SRE: First puppet run after reimage slow (connection timeout) - https://phabricator.wikimedia.org/T262609#11814782 (10LSobanski) 05Open→03Resolved a:03LSobanski Should have been addressed with upgrade to Puppet 7, please reopen if you st... [15:53:03] https://gerrit.wikimedia.org/r/c/operations/puppet-lint/wmf_styleguide-check/+/1270486 :) [17:26:25] 10netops, 06Infrastructure-Foundations, 06SRE, 07Incident Severity 3: Row C traffic outage Nov 11 2025 - https://phabricator.wikimedia.org/T409800#11815922 (10MLechvien-WMF) [22:27:51] 10SRE-tools, 06Infrastructure-Foundations, 13Patch-For-Review: Cookbook for rack depool - https://phabricator.wikimedia.org/T327300#11817242 (10Ladsgroup) Databases now have a centralized depool and repool cookbook that encapsulates all the different ways you need to depool and repool db hosts (for different...