[07:13:49] 10netops, 10Infrastructure-Foundations, 10observability: Grafana posting to http://wpt-graphite.wmftest.org:8080/ - https://phabricator.wikimedia.org/T307445 (10ayounsi) @jbond i think this can be closed? [07:32:56] 10netops, 10Infrastructure-Foundations, 10SRE: DHCPd: update config to log more info - https://phabricator.wikimedia.org/T309524 (10ayounsi) [07:43:03] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, and 2 others: decommission atlas-esams - https://phabricator.wikimedia.org/T307026 (10ayounsi) [07:44:17] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, and 2 others: decommission atlas-esams - https://phabricator.wikimedia.org/T307026 (10ayounsi) a:05ayounsi→03RobH Thanks Faidon. I ended up emailing them as the dashboard seems limited. @robh: we can proceed with the decom of that box, following... [08:02:34] 10netbox, 10netops, 10Infrastructure-Foundations, 10SRE: Netbox Juniper report - https://phabricator.wikimedia.org/T306238 (10ayounsi) @jbond, I was wondering if the upgrade went as expected. And if there was a timeline for OIDC. No urgency at all, even if it's 1, 6 or more months, just doing some planning :) [08:10:00] 10netops, 10Infrastructure-Foundations, 10SRE, 10Traffic: drmrs: primary software task - https://phabricator.wikimedia.org/T282788 (10ayounsi) [08:11:03] 10netops, 10Infrastructure-Foundations, 10SRE, 10Traffic, 10Patch-For-Review: drmrs: initial geodns configuration - https://phabricator.wikimedia.org/T304089 (10ayounsi) 05Open→03Resolved a:03ayounsi I think everything here is done, and follow up is in {T311472} Feel free to re-open if needed. [08:18:08] 10netops, 10Infrastructure-Foundations, 10SRE: Validate new (anycast) IPv6 /48 announcement being accepted by transits - https://phabricator.wikimedia.org/T301900 (10ayounsi) @cmooney I had a quick look at NTT/Telia/Lumen looking glass and the prefix seems to be accepted properly. Longer term, this could ma... [08:18:43] 10netbox, 10netops, 10Infrastructure-Foundations, 10SRE: Improve Netbox import script to avoid port-number collisions in JunOS - https://phabricator.wikimedia.org/T301392 (10ayounsi) [08:24:34] 10netbox, 10netops, 10Infrastructure-Foundations, 10SRE: Represent sub-interface and bridge device assocations in Netbox - https://phabricator.wikimedia.org/T296832 (10ayounsi) [08:31:36] 10SRE-tools, 10netops, 10Infrastructure-Foundations, 10serviceops: Support services VIPs with not marked as VIP in Netbox - https://phabricator.wikimedia.org/T295793 (10ayounsi) Pinging @jelto for gitlab (not sure if the issue is still present or relevant) and @akosiaris for lists1001 (I can confirm the mi... [08:38:34] 10SRE-tools, 10DC-Ops, 10Infrastructure-Foundations: sre.hosts.reimage: wait reboot time timeout on aqs nodes - https://phabricator.wikimedia.org/T307260 (10Volans) 05Open→03Resolved This should be resolved. Feel free to reopen it in case it's not. [08:48:06] 10netops, 10Infrastructure-Foundations, 10SRE: Packet Drops on Eqiad ASW -> CR uplinks - https://phabricator.wikimedia.org/T291627 (10ayounsi) With {T304712} will give us the possibility to move to 40G uplinks (instead of 4x10G) for some rows: C as of now, and D once {T308331} is done. This could be a good t... [08:53:19] 10SRE-tools, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Allow idrac ftp fetching of firmware updates (either to existing ftp or new solution) - https://phabricator.wikimedia.org/T283771 (10ayounsi) [08:53:26] 10SRE-tools, 10Infrastructure-Foundations, 10Observability-Alerting, 10SRE Observability (FY2022/2023-Q1), 10User-fgiunchedi: Spicerack: add support for Alertmanager - https://phabricator.wikimedia.org/T293209 (10Volans) >>! In T293209#8043558, @fgiunchedi wrote: > I don't know offhand how to best achiev... [11:06:29] 10CAS-SSO, 10Infrastructure-Foundations, 10SRE: Enable OIDC - https://phabricator.wikimedia.org/T311999 (10MoritzMuehlenhoff) [11:06:45] 10CAS-SSO, 10Infrastructure-Foundations, 10SRE: Enable OIDC in CAS - https://phabricator.wikimedia.org/T311999 (10MoritzMuehlenhoff) p:05Triage→03Medium [11:08:32] 10netbox, 10netops, 10Infrastructure-Foundations, 10SRE: Netbox Juniper report - https://phabricator.wikimedia.org/T306238 (10MoritzMuehlenhoff) >>! In T306238#8047863, @ayounsi wrote: > @jbond, I was wondering if the upgrade went as expected. And if there was a timeline for OIDC. > No urgency at all, even... [11:18:50] https://github.com/netbox-community/netbox/issues/8511#event-6920557719 yay [12:19:58] is that to help with replacing our cross-connects spreadsheet? [12:25:46] yep exactly [12:38:59] nice :) [12:40:32] paravoid: still WIP, but we could even have an output like the spreadsheet https://netbox.wikimedia.org/circuits/circuits/?export=cross-connects [12:40:50] nice! [14:43:22] slyngs: https://wikitech.wikimedia.org/wiki/Splunk_On-Call#Starting_your_shift [14:46:22] XioNoX: Aargh, confusing, because I had my weeks switched, my name isn't on the rotation business hours :-) [14:46:51] yeah, you need to find the name of the person you're an overide of [14:48:08] Yes, and I was on Nairobi times, hence the weird hours [14:48:21] Perfect, fixed. Thank you [15:09:21] 10SRE-tools, 10netops, 10Infrastructure-Foundations, 10serviceops: Support services VIPs with not marked as VIP in Netbox - https://phabricator.wikimedia.org/T295793 (10akosiaris) >>! In T295793#8047985, @ayounsi wrote: > Pinging @jelto for gitlab (not sure if the issue is still present or relevant) and @a... [20:50:19] 10SRE-tools, 10netops, 10Infrastructure-Foundations, 10Wikimedia-Mailing-lists, 10serviceops: Support services VIPs with not marked as VIP in Netbox - https://phabricator.wikimedia.org/T295793 (10Legoktm) Any IP (mis)configuration most likely predates Amir's and my involvement with mailman, we never touc...