[06:58:19] 10netbox, 10Infrastructure-Foundations, 10Patch-For-Review: Netbox: define strategy to track standard server configurations - https://phabricator.wikimedia.org/T284614 (10Volans) >>! In T284614#7280751, @ayounsi wrote: > FYI, Netbox is alerting with: >> mismatched device models: PowerEdge R740xd2 - ConfigJ 2... [07:10:41] 10SRE-tools, 10Infrastructure-Foundations, 10SRE, 10Spicerack: Spicerack downtime methods fail when the admin reason includes an apostrophe - https://phabricator.wikimedia.org/T288558 (10Volans) a:05RLazarus→03Volans Ack, I'll make a new release in the next few days, claiming the task. [07:26:57] 10Puppet, 10puppet-compiler, 10Infrastructure-Foundations, 10SRE, and 2 others: replace all puppet crons with systemd timers - https://phabricator.wikimedia.org/T273673 (10Ladsgroup) [07:27:22] 10Puppet, 10Infrastructure-Foundations, 10Wikidata, 10wdwb-tech, 10User-Ladsgroup: Migrate wikibase-dispatch-changes crons to systemd timers - https://phabricator.wikimedia.org/T288175 (10Ladsgroup) 05Open→03Resolved It's done \o/ [07:34:24] 10netbox, 10Infrastructure-Foundations: Import row information into Netbox for Ganeti instances - https://phabricator.wikimedia.org/T262446 (10ayounsi) This is done for eqiad in netbox-next: https://netbox-next.wikimedia.org/virtualization/clusters/?q=&site=eqiad [08:51:22] 10netops, 10Infrastructure-Foundations, 10SRE: Cloud IPv6 subnets - https://phabricator.wikimedia.org/T187929 (10ayounsi) @cmooney That looks cleaner indeed. @faidon We're moving away from 172.16/12 IPs being able to reach the Wikis, which means VM traffic needs to be NATed and looses useful troubleshooting... [10:00:12] https://blog.ipspace.net/2021/05/worth-reading-azure-switch-failure-analysis.html [10:00:37] "A data center switch has a 2% chance of failing in 3 months (= less than 10% per year)" [10:27:55] 10Puppet, 10puppet-compiler, 10Infrastructure-Foundations, 10Patch-For-Review: puppet-facts-export sometimes fails with 'trusted' fact not found - https://phabricator.wikimedia.org/T289335 (10jbond) a:03jbond I had a quick look at the code and there is a potential race condition. We first get a list of... [14:44:34] 10Puppet, 10Infrastructure-Foundations, 10MW-on-K8s, 10Kubernetes, 10Patch-For-Review: Add a fact holding the type of a disk (spinning/ssd) - https://phabricator.wikimedia.org/T288509 (10JMeybohm) a:03JMeybohm [20:12:54] hey IF -- I was just chatting with Arnold, our new SRE who isn't in ldap/ops yet, about whether we could get him access to Puppetboard -- the answer is no, because Puppetboard fronts data from the private puppet repo, right? so there's no way we could grant access to ldap/nda or anything, he'll just have to wait until he gets root later on in his onboarding? [20:36:17] rzl: I'll leave this to j.bond and mo.ritz but my 2 cents are that IIRC we've already scaled down the data that puppetboard can show because we were not happy with it exposing secrets. nda might be too much but maybe we could have a dedicated ldap group for SREs not in ldap/ops... just my thoughts [20:40:00] ah okay -- yeah that would make sense, would love to set up that kind of group structure if we're going to keep this onboarding structure (which is seeming promising so far)