[06:12:33] 10netops, 10Infrastructure-Foundations, 10Prod-Kubernetes, 10SRE: Agree strategy for Kubernetes BGP peering to top-of-rack switches - https://phabricator.wikimedia.org/T306649 (10JMeybohm) >>! In T306649#7931058, @akosiaris wrote: >> Regarding the "fake nodes": I think that could be done with adding the le... [06:29:04] 10netops, 10Infrastructure-Foundations, 10Prod-Kubernetes, 10SRE: Agree strategy for Kubernetes BGP peering to top-of-rack switches - https://phabricator.wikimedia.org/T306649 (10ayounsi) > Plus, they are VMs and we have the same problem we have with the kask dedicated nodes (also VMs). Netbox doesn't have... [08:19:46] 10netops, 10Infrastructure-Foundations, 10observability: Grafana posting to http://wpt-graphite.wmftest.org:8080/ - https://phabricator.wikimedia.org/T307445 (10Marostegui) [08:20:30] 10netops, 10Cloud-VPS, 10Infrastructure-Foundations, 10SRE, 10cloud-services-team (Kanban): Move cloud vps ns-recursor IPs to host/row-independent addressing - https://phabricator.wikimedia.org/T307357 (10Marostegui) p:05Triage→03Medium [09:01:40] 10netops, 10Infrastructure-Foundations, 10Prod-Kubernetes, 10SRE: Agree strategy for Kubernetes BGP peering to top-of-rack switches - https://phabricator.wikimedia.org/T306649 (10akosiaris) >>! In T306649#7933266, @JMeybohm wrote: >>>! In T306649#7931058, @akosiaris wrote: >>> Regarding the "fake nodes": I... [09:16:59] volans: I'll go ahead and merge your log backup change that ok? [09:19:42] topranks: go ahead thanks [09:19:48] I was having a hard time ssh-ing to puppetmaster1001 [09:20:02] so I was chaning the bastion to use [09:20:12] and the phone ringed while I was doing that :D [09:20:14] ok np [09:20:16] :) [09:20:21] I hope not a network issue! [09:20:49] the same I told you some week ago, it's most likely on my provider side, I'll try to find some time to digg at some point [09:21:04] nothing that points to WMF-related issues for that matters [10:48:19] moritzm: what kind of review is expected on changes like https://gerrit.wikimedia.org/r/c/operations/puppet/+/792579 ? [10:48:50] a +1 one :-P [10:58:21] Depends, how much you want to drill down. I checked that all commits are from people with a @wikimedia.org address, so basically just a sanity check that all comment lines are in the appropriate format is really needed [10:59:59] ok, will do! [11:09:34] thx! [11:57:17] hi, I have fastnetmon-related change at https://gerrit.wikimedia.org/r/c/operations/puppet/+/792569 and seeking reviewers [11:58:56] godog: +1 [11:59:45] jbond: cheers! appreciate it [11:59:56] np [13:59:23] 10netops, 10Infrastructure-Foundations, 10Prod-Kubernetes, 10SRE: Agree strategy for Kubernetes BGP peering to top-of-rack switches - https://phabricator.wikimedia.org/T306649 (10elukey) I merged two changes for the ml-serve-eqiad cluster, and now the concerns expressed in T306649#7881940 should be gone:... [14:30:20] 10netops, 10Infrastructure-Foundations, 10Prod-Kubernetes, 10SRE: Agree strategy for Kubernetes BGP peering to top-of-rack switches - https://phabricator.wikimedia.org/T306649 (10akosiaris) >>! In T306649#7934722, @elukey wrote: > I merged two changes for the ml-serve-eqiad cluster, and now the concerns ex... [17:16:29] 10Mail, 10Infrastructure-Foundations, 10SRE, 10Znuny, 10fundraising-tech-ops: move donation,donate, donations (otrs, wikimania) exim aliases from SRE to ITS - https://phabricator.wikimedia.org/T297915 (10bcampbell) Hey @Dzahn my apologies for the delay. I just completed the first two steps: - ITS conf... [21:05:15] 10Mail, 10Infrastructure-Foundations, 10SRE, 10Znuny, 10fundraising-tech-ops: move donation,donate, donations (otrs, wikimania) exim aliases from SRE to ITS - https://phabricator.wikimedia.org/T297915 (10Dzahn) 05Stalled→03Open [21:05:21] 10Mail, 10Infrastructure-Foundations, 10SRE, 10Epic: Move most (all?) exim personal aliases to WMF ITS - https://phabricator.wikimedia.org/T122144 (10Dzahn) [21:05:31] 10Mail, 10Infrastructure-Foundations, 10SRE, 10Epic: Move most (all?) exim personal aliases to WMF ITS - https://phabricator.wikimedia.org/T122144 (10Dzahn) [21:05:41] 10Mail, 10Infrastructure-Foundations, 10SRE, 10Znuny, 10fundraising-tech-ops: move donation,donate, donations (otrs, wikimania) exim aliases from SRE to ITS - https://phabricator.wikimedia.org/T297915 (10Dzahn) 05Open→03In progress [21:27:55] 10Mail, 10Infrastructure-Foundations, 10SRE, 10Znuny, 10fundraising-tech-ops: move donation,donate, donations (otrs, wikimania) exim aliases from SRE to ITS - https://phabricator.wikimedia.org/T297915 (10Dzahn) Hi @bcampbell I removed the donate@ alias from the mail servers right now. I can confirm it n... [21:28:55] 10Mail, 10Infrastructure-Foundations, 10SRE, 10Znuny, 10fundraising-tech-ops: move donation,donate, donations (otrs, wikimania) exim aliases from SRE to ITS - https://phabricator.wikimedia.org/T297915 (10Dzahn) How about "donation@" as opposed to "donate@". Is that an alias for fundraising for for donat... [21:46:58] 10Mail, 10Infrastructure-Foundations, 10SRE, 10Znuny, 10fundraising-tech-ops: move donation,donate, donations (otrs, wikimania) exim aliases from SRE to ITS - https://phabricator.wikimedia.org/T297915 (10bcampbell) I've sent test mail from a couple different addresses, one internal and one external, and... [21:54:19] 10SRE-tools, 10DNS, 10Infrastructure-Foundations, 10Traffic: DNS repo: add Jenkins job to ensure there are no duplicates - https://phabricator.wikimedia.org/T155761 (10Volans) I've a local patch that I'm testing to perform the validation of the whole dataset (manual + netbox). The preliminary results are b... [22:00:56] 10SRE-tools, 10Discovery, 10Discovery-Search, 10Infrastructure-Foundations, 10IPv6: Some elastic hosts do not have IPv6 DNS entries - https://phabricator.wikimedia.org/T271143 (10bking) a:03bking [22:01:13] 10SRE-tools, 10Discovery, 10Discovery-Search, 10Infrastructure-Foundations, 10IPv6: Some elastic hosts do not have IPv6 DNS records - https://phabricator.wikimedia.org/T271143 (10bking) [22:22:50] 10SRE-tools, 10Discovery, 10Discovery-Search, 10Infrastructure-Foundations, 10IPv6: Some elastic hosts do not have IPv6 DNS records - https://phabricator.wikimedia.org/T271143 (10bking) For clarity, client-side IPv6 connectivity to search functions in wikipedia, wikicommons, etc does not require the Elas... [22:23:48] 10Mail, 10Infrastructure-Foundations, 10SRE, 10Znuny, 10fundraising-tech-ops: move donation,donate, donations (otrs, wikimania) exim aliases from SRE to ITS - https://phabricator.wikimedia.org/T297915 (10Dzahn) The following aliases have all been removed on the SRE side now: donation@ donations@ donate@... [22:24:33] 10SRE-tools, 10Discovery, 10Discovery-Search, 10Infrastructure-Foundations, 10IPv6: Some elastic hosts do not have IPv6 DNS records - https://phabricator.wikimedia.org/T271143 (10bking) 05Open→03Resolved [22:24:35] 10netbox, 10Infrastructure-Foundations, 10IPv6, 10User-jbond: Some clusters do not have DNS for IPv6 addresses (TRACKING TASK) - https://phabricator.wikimedia.org/T253173 (10bking) [22:25:05] 10Mail, 10Infrastructure-Foundations, 10SRE, 10Znuny, 10fundraising-tech-ops: move donation,donate, donations (otrs, wikimania) exim aliases from SRE to ITS - https://phabricator.wikimedia.org/T297915 (10Dzahn) 05In progress→03Resolved [22:25:11] 10Mail, 10Infrastructure-Foundations, 10SRE, 10Epic: Move most (all?) exim personal aliases to WMF ITS - https://phabricator.wikimedia.org/T122144 (10Dzahn)