[09:26:12] 10netbox, 10netops, 10Infrastructure-Foundations, 10SRE: Represent sub-interface and bridge device assocations in Netbox - https://phabricator.wikimedia.org/T296832 (10cmooney) The updated PuppetDB -> Netbox import script has now been merged, and I've run it against all servers in Netbox in state 'active'... [10:14:34] Is there a way in puppet to get a list of hosts with a role applied? What I'm trying to do is add all gitlab-runner hosts to an allow list on doc hosts. Or is there a better way of doing it? [10:36:01] yes there are helper functions, sorry on mobile so I can't grep eoghan [10:36:23] depending if you need to include your own host or not you might need a different syntax [10:36:32] I'll keep digging, thanks (: [10:36:35] because of the first time puppet rubs [10:36:55] eoghan: look at the cumin master profile IIRC [10:37:11] 👍 Thanks! [10:37:16] is something like query_hosts or similar [10:42:42] `wmflib::role::hosts` looks like it, thanks volans|off ! https://gerrit.wikimedia.org/g/operations/puppet/+/7a1269486f181c6ce5c8b310735f63f3262a1c68/modules/profile/manifests/cumin/master.pp#21 [10:46:34] looos like it [10:46:38] yw [10:58:25] hey is anybody using sretest1001 (or 1002)? [10:58:45] I'm gonna apply some changes to the DHCP relay config on the core routers and I want to test a reimage after make sure things work as expected [11:11:09] 10netops, 10Infrastructure-Foundations, 10SRE, 10cloud-services-team: Configure cloudsw1-b1-codfw and migrate cloud hosts in codfw B1 to it - https://phabricator.wikimedia.org/T327919 (10cmooney) [11:11:19] 10netops, 10Cloud-VPS, 10Infrastructure-Foundations, 10SRE, 10cloud-services-team: Move cloud vps ns-recursor IPs to host/row-independent addressing - https://phabricator.wikimedia.org/T307357 (10cmooney) [11:12:07] 10netops, 10Cloud-VPS, 10Infrastructure-Foundations, 10SRE, and 2 others: cloudservices[2004/2005]-dev & cloudweb2002-dev: connect them to cloudsw so they can have cloud-private vlan - https://phabricator.wikimedia.org/T336587 (10cmooney) 05Open→03Resolved Couple of niggles getting this going on the ho... [11:14:48] topranks: I sretest1002 for bookworm install tests, but can be reimaged any time [11:15:27] moritzm: ok cool sounds good, I'll merge my changes and try a reimage in that case [11:15:30] thanks! [12:12:15] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bookworm [12:17:32] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bookworm executed with errors: - sretest1002... [12:19:22] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bookworm [12:24:15] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bookworm executed with errors: - sretest1002... [12:24:39] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bookworm [12:44:13] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bookworm executed with errors: - sretest1002... [12:44:35] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bookworm [12:51:09] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bookworm executed with errors: - sretest1002... [12:51:30] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bookworm [12:56:53] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bookworm executed with errors: - sretest1002... [12:57:09] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bookworm [13:02:49] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bookworm executed with errors: - sretest1002... [13:30:22] sorry for the spam, having issues with reimage of sretest1002 following the CR router changes, still debugging [14:17:25] Hello. Is there a way to downgrade the version of a package hosted by reprepro? I accidentally uploaded version 0.15 of a package and I want to replace it with version 0.0.15. [14:18:39] Do I simply remove the old package first, or is there a switch to permit downgrading? [14:28:51] btullis: I would assume removing then readding would work [14:29:54] does it throw an error when you try to import the older version? [14:30:18] * jhathaway admits not knowing a lot about reprepro [14:31:25] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bookworm executed with errors: - sretest1002... [14:31:48] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bookworm [14:38:08] jhathaway: Thanks, yeah it just says it won't be used. I think I'll go ahead and delete the old, then re-add. I have the old artifact, in case I need it. [14:38:13] https://www.irccloud.com/pastebin/evRymLQc/ [14:38:41] sounds good [14:39:05] let me know if it works and i'll add something to the wikitech article [14:41:36] btullis: if it helps, that's what I have done in the past as well [14:41:39] remove and re-add [14:41:52] Great, thanks both. Will do. [14:55:11] moritzm: not sure if you're still around but sretest1002 is not happy :( [14:55:31] my testing was with the dhcp-relay part of the process, so the very first step, which failed over and over till I worked it out [14:55:54] once I did it loaded into debain-installer etc. as you'd expect, but it hit some issue with the disk layout [14:56:08] it came up with a manual prompt, I figured given it was test server I'd hit enter for "continue" [14:56:47] TL;DR it didn't get through the debain installer and I'm not 100% sure what the current status is [14:57:29] I was trying to reimage to bookworm, perhaps that's where the issue is I should try bullseye instead? [14:58:02] jhathaway: For the record, it worked like a charm. No warnings or unexpected things (so far) [14:58:06] https://www.irccloud.com/pastebin/eDs8G963/ [14:58:15] btullis: great thanks! [15:18:29] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10cmooney) >>! In T320508#8488549, @ayounsi wrote: > Marking this task dependent on DHCP option 97 to reduce the risk of DHCP oddities related to Option 82. Ironic I hadn't... [15:19:14] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10cmooney) 05Open→03Resolved [15:20:25] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Consolidate Automation Templates for DC Switches - https://phabricator.wikimedia.org/T312635 (10cmooney) [15:20:48] 10netops, 10Infrastructure-Foundations, 10SRE: Allow managing drmrs DHCP settings with Homer - https://phabricator.wikimedia.org/T328737 (10cmooney) 05Open→03Resolved Complete now after merging above patch. [15:25:39] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bookworm executed with errors: - sretest1002... [15:32:40] topranks: the d-i installer for bookworm should work in general, but maybe there was a breaking change, if you mostly want to test some things related to DHCP, best to use bullseye [15:32:56] canl have a closer look at what's wrong with bookworm tomorrow [15:35:26] moritzm: thanks [15:35:47] ultimately for me it didn't matter if the reimage completed or not, as long as the dhcp exchange worked [15:37:39] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bullseye [15:37:52] for now I'll try buster and see if it likes that better rather than leaving it in a broken state [15:52:10] installer seemed to work ok with bullseye [16:10:08] 10netops, 10Infrastructure-Foundations, 10SRE: Core routers: replace bootp with dhcp-relay - https://phabricator.wikimedia.org/T320508 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by cmooney@cumin1001 for host sretest1002.eqiad.wmnet with OS bullseye completed: - sretest1002 (**PASS**)... [16:53:05] moritzm: is there more recent than https://github.com/lfit/itpol/blob/master/linux-workstation-security.md for best practices for a new laptop? [17:25:53] some of what's in the document doesn't make sense (like using rkhunter), but it's quite okay. In general, "use Debian, keep installing sec updates, use FDE, apply common sense" provides a good starting point [17:26:16] we don't have specifically written up documentation TTBOMK (apart of https://office.wikimedia.org/wiki/Staff_computer_equipment) [17:35:34] thx! [18:19:42] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Update network SSH keys to ssh-ed25519 - https://phabricator.wikimedia.org/T336769 (10ayounsi) [18:50:57] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Migrate row E/F network aggregation to dedicated Spine switches - https://phabricator.wikimedia.org/T322937 (10cmooney) lvs1020 is currently the "secondary" lvs in eqiad, so I'd propose we start with trying to do that one if we can. It's c... [18:55:21] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Q2:(Need By: TBD) Rows E/F network racking task - https://phabricator.wikimedia.org/T292095 (10cmooney) @Jclark-ctr hey. It's taken a bit of time to line this up, hit a few bumps in the road with the Juniper config. As detailed in T3... [19:01:43] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Q2:(Need By: TBD) Rows E/F network racking task - https://phabricator.wikimedia.org/T292095 (10Jclark-ctr) @cmooney i am available tomorrow if you would like to address it that quickly. otherwise monday [19:54:53] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Q2:(Need By: TBD) Rows E/F network racking task - https://phabricator.wikimedia.org/T292095 (10cmooney) @Jclark-ctr thanks yeah I just had a word with @ssingh and I think tomorrow if probably possible. What time suits you to be on site?