[00:15:59] 10netops, 10Infrastructure-Foundations, 10SRE: Rancid on netmon1003 unable to login to network devices - https://phabricator.wikimedia.org/T314936 (10andrea.denisse) [00:16:49] 10netops, 10Infrastructure-Foundations, 10SRE: Rancid on netmon1003 unable to login to network devices - https://phabricator.wikimedia.org/T314936 (10andrea.denisse) 05In progress→03Resolved [00:17:20] 10netops, 10Infrastructure-Foundations, 10SRE: Rancid on netmon1003 unable to login to network devices - https://phabricator.wikimedia.org/T314936 (10andrea.denisse) Fixed in [[ https://gerrit.wikimedia.org/r/822196 | 822196 ]]. [00:52:38] 10Puppet, 10Infrastructure-Foundations, 10MobileFrontend (Tracking), 10User-Jdlrobson: Mobile site does not automatically redirect to desktop version (and not possible to use browser "use desktop view") - https://phabricator.wikimedia.org/T60425 (10Iniquity) >>! In T60425#5179124, @MaxSem wrote: > Why do y... [08:56:16] 10netops, 10Infrastructure-Foundations, 10SRE: Rancid on netmon1003 unable to login to network devices - https://phabricator.wikimedia.org/T314936 (10cmooney) 05Resolved→03Open @andrea.denisse Hey, does you patch correct the other problem I observed above? With the prompt for accepting the host key cau... [10:41:56] 10netops, 10Infrastructure-Foundations, 10Traffic: Increased latency between eqiad<-->eqsin beginning on August 15th ~08:30:00 - https://phabricator.wikimedia.org/T315429 (10Vgutierrez) [13:26:02] 10Puppet, 10Beta-Cluster-Infrastructure, 10Infrastructure-Foundations, 10SRE, and 3 others: Evaluation Error on deployment-cache-text06 puppet run - https://phabricator.wikimedia.org/T315351 (10RhinosF1) [13:37:59] 10netops, 10Infrastructure-Foundations, 10SRE: Rancid on netmon1003 unable to login to network devices - https://phabricator.wikimedia.org/T314936 (10andrea.denisse) @cmooney Thanks for the heads-up, I missed that part, my bad. [14:00:04] 10netops, 10Infrastructure-Foundations, 10Observability-Alerting, 10SRE: Enable OSPF Icinga check for EVPN based switches - https://phabricator.wikimedia.org/T315053 (10lmata) [14:07:34] 10netops, 10Cloud-Services, 10Infrastructure-Foundations, 10SRE: Allow jumbo frames between cloud hosts in production realm - https://phabricator.wikimedia.org/T315446 (10cmooney) p:05Triage→03Medium [14:28:19] 10Puppet, 10Beta-Cluster-Infrastructure, 10Infrastructure-Foundations, 10SRE, and 3 others: Evaluation Error on deployment-cache-text06 puppet run - https://phabricator.wikimedia.org/T315351 (10Zabe) 05Open→03Resolved Some cherry-picks made by ori made puppet run again, see T315394 for follow-up. [15:11:49] cdanis: do yuo mind if i delet yuo from sretest i wont to test an issue with the admin module and zsh (https://gerrit.wikimedia.org/r/c/operations/puppet/+/824130) [15:13:04] jbond: go for it [15:13:24] thanks [15:29:28] 10netops, 10Cloud-Services, 10Infrastructure-Foundations, 10SRE: Allow jumbo frames between cloud hosts in production realm - https://phabricator.wikimedia.org/T315446 (10cmooney) Ok so looking at this a bit closer it seems the ommision was just that the MTU wasn't set high on cloudsw1-c8, on its links to... [15:59:01] 10netops, 10Cloud-Services, 10Infrastructure-Foundations, 10SRE: Allow jumbo frames between cloud hosts in production realm - https://phabricator.wikimedia.org/T315446 (10dcaro) [16:57:16] 10netops, 10Cloud-Services, 10Infrastructure-Foundations, 10SRE: Allow jumbo frames between cloud hosts in production realm - https://phabricator.wikimedia.org/T315446 (10cmooney) 05Open→03Resolved Ok gonna close this one as the cloud team have confirmed things are now working for them. Apologies for... [18:51:53] 10netops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: eqiad: upgrade row C and D uplinks from 4x10G to 1x40G - https://phabricator.wikimedia.org/T313463 (10Jclark-ctr) [18:56:24] 10netops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad, 10Sustainability (Incident Followup): eqiad row C switch fabric recabling - https://phabricator.wikimedia.org/T313384 (10Jclark-ctr) [21:53:56] 10Mail, 10Infrastructure-Foundations, 10SRE: Add xcollazo@wikimedia.org to the analytics-alerts mailing list - https://phabricator.wikimedia.org/T315486 (10Dzahn)