[00:26:39] 10SRE-tools, 10netops, 10Infrastructure-Foundations, 10SRE: Setup zero touch provisioning (ZTP) for network devices - https://phabricator.wikimedia.org/T336485 (10Papaul) @Volans I checked on the dhcp server in the dhcp config file there is an option that is missing ' for configuration we have now is : `... [05:40:02] 10Puppet, 10Cloud-VPS, 10cloud-services-team: puppet package versioning on Bookworm for cloud-vps - https://phabricator.wikimedia.org/T338195 (10Dzahn) [06:16:24] 10netops, 10Infrastructure-Foundations: Peering: prefer primary IXP for direcly connected networks - https://phabricator.wikimedia.org/T338201 (10ayounsi) p:05Triage→03High [08:43:19] ok to reboot the netbox DB hosts? [08:43:31] or is anyone currently working on Netbox? [08:43:35] ok for me [08:47:39] k, I'll start with those shortly [08:47:55] thx [09:00:06] netbox DB hosts are rebooted and I also used the "window" to deploy the latest Postgres security updates, Netbox all still working fine for me [09:04:04] nice! [09:12:00] 10CAS-SSO, 10Infrastructure-Foundations: IDP: add policy to release given name - https://phabricator.wikimedia.org/T338214 (10jbond) p:05Triage→03Medium [09:20:49] 10SRE-tools, 10netops, 10Infrastructure-Foundations, 10SRE: Setup zero touch provisioning (ZTP) for network devices - https://phabricator.wikimedia.org/T336485 (10Volans) @Papaul that's set already using option 66 that is a standard option, see `tftp-server-name` in `automation.conf`. And the previous comm... [09:55:36] 10CAS-SSO, 10Infrastructure-Foundations, 10SRE, 10serviceops-collab, and 4 others: migrate gitlab away from the CAS protocol - https://phabricator.wikimedia.org/T320390 (10jbond) > Here are logs from my login to the admin page: https://logstash.wikimedia.org/goto/61b6d364170814f8682de1275d89d767 Perhaps i... [12:43:32] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Users management on SONiC - https://phabricator.wikimedia.org/T338028 (10MoritzMuehlenhoff) It would appear to me that given that we have the full freedom of a Debian-based OS here (after all next to the FLOSS aspect the major perk of SONiC... [12:48:55] 10Puppet, 10Cloud-VPS, 10cloud-services-team: puppet package versioning on Bookworm for cloud-vps - https://phabricator.wikimedia.org/T338195 (10Andrew) Thanks @Dzahn ! The challenge is to encode that in cloud-init yaml (which may or may not be possible) [13:04:43] (SystemdUnitFailed) firing: httpbb_hourly_appserver.service Failed on cumin1001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [13:15:52] 10SRE-tools, 10netops, 10Infrastructure-Foundations, 10SRE: Setup zero touch provisioning (ZTP) for network devices - https://phabricator.wikimedia.org/T336485 (10Papaul) @Volans ok i didn't look in the automation.conf file for the option 66 thanks [13:19:43] (SystemdUnitFailed) resolved: httpbb_hourly_appserver.service Failed on cumin1001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [13:35:14] 10netops, 10Infrastructure-Foundations, 10SRE: Migrate row E/F network aggregation to dedicated Spine switches - https://phabricator.wikimedia.org/T322937 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=cc1782a9-2e71-42dd-977e-e3f886320173) set by cmooney@cumin1001 for 0:30:00 on 3 host(s... [13:38:46] 10netbox, 10Infrastructure-Foundations: Netbox: get rid of WMF Production Patches - https://phabricator.wikimedia.org/T310717 (10ayounsi) > Switch swagger to non-public mode More recent Netbox export Swagger through "spectacular", this patch will not apply on 3.5.x for example. A (not tested) possible fix is... [14:06:33] 10netops, 10Infrastructure-Foundations, 10SRE: Migrate row E/F network aggregation to dedicated Spine switches - https://phabricator.wikimedia.org/T322937 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=ff119eb0-a98a-450e-a695-debe4cc1b997) set by cmooney@cumin1001 for 0:30:00 on 3 host(s... [14:24:04] 10netbox, 10Infrastructure-Foundations: Netbox: get rid of WMF Production Patches - https://phabricator.wikimedia.org/T310717 (10ayounsi) > Add a passthrough configuration system https://github.com/wikimedia/operations-puppet/blob/production/modules/netbox/templates/configuration.py.erb#L186 is what we curre... [17:32:22] 10Puppet, 10Cloud-VPS, 10cloud-services-team: puppet package versioning on Bookworm for cloud-vps - https://phabricator.wikimedia.org/T338195 (10Andrew) I now have the proper version installing via cloud-init () but now when puppet is invoked it says: ` root@buildvm-c88281bc-7bb0-4a46-9f97-c9b59ba3b845:~#... [17:34:02] 10Puppet, 10Cloud-VPS, 10cloud-services-team: puppet package versioning on Bookworm for cloud-vps - https://phabricator.wikimedia.org/T338195 (10Andrew) > > I think this is a missing dependency in the package. Indeed, installing 'ruby-sorted-set' fixes things. So the puppet package should have a dependenc... [20:28:50] 10Puppet, 10Cloud-VPS, 10cloud-services-team: puppet package versioning on Bookworm for cloud-vps - https://phabricator.wikimedia.org/T338195 (10bd808) > 3. Upgrade puppetmasters to version 7, which should be backwards-compatible with existing clients. That's part of {T330490} [20:49:04] 10Puppet, 10Release-Engineering-Team: Puppet git::clone probably does not need `umask` parameter - https://phabricator.wikimedia.org/T338277 (10hashar)