[06:13:36] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Q2:(Need By: TBD) replace mr1-eqiad - https://phabricator.wikimedia.org/T294474 (10ayounsi) We synced up on IRC. The SCS ports was not configured, imho that's something DCops should do. Once done, looks like the device is stuck in a... [07:27:59] 10SRE-tools, 10Infrastructure-Foundations: Cumin should group similar SSH errors - https://phabricator.wikimedia.org/T306490 (10Majavah) [08:04:14] I'll be rebooting the netbox* hosts in half an hour, let me know if that interferes with any maintenance [08:05:29] moritzm: please make sure nobody is doing any kind of re-imaging, but lgtm [08:09:52] fwiw decommissioning too [08:18:31] 10SRE-tools, 10Infrastructure-Foundations: Cumin should group similar SSH errors - https://phabricator.wikimedia.org/T306490 (10jcrespo) Please note that cumin does group already identical content output. E.g.: ` # cumin 'P:cumin::target%cluster = backup' 'uname -v' 24 hosts will be targeted: backup[2001-2007... [08:18:57] ack, I'll also ping the -sre channel [08:23:14] 10SRE-tools, 10Infrastructure-Foundations: Cumin should group similar SSH errors - https://phabricator.wikimedia.org/T306490 (10Volans) > Would it be possible to group the similar SSH errors where the only difference is the target hostname? Not currently, as the underlying library that does the grouping does... [14:21:21] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-ulsfo: (Need By: TBD) rack/setup/install new mr1-ulsfo - https://phabricator.wikimedia.org/T294314 (10ayounsi) [14:21:44] 10SRE-tools, 10netbox, 10Infrastructure-Foundations, 10Patch-For-Review: Move device attributes - https://phabricator.wikimedia.org/T259166 (10ayounsi) 05Resolved→03Open This revealed a limitation in Netbox. After investigation the short version is that Netbox caches the device connected to a cable for... [15:41:45] 10SRE-tools, 10netbox, 10Infrastructure-Foundations, 10Patch-For-Review: Move device attributes - https://phabricator.wikimedia.org/T259166 (10ayounsi) I fixed the prod ones with in nbshell: `lang=python source_device = Device.objects.get(name="mr1-ulsfo-old") destination_device = Device.objects.get(name=... [16:03:43] 10SRE-tools, 10netbox, 10Infrastructure-Foundations, 10Patch-For-Review: Move device attributes - https://phabricator.wikimedia.org/T259166 (10ayounsi) 05Open→03Resolved [16:03:46] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-ulsfo: (Need By: TBD) rack/setup/install new mr1-ulsfo - https://phabricator.wikimedia.org/T294314 (10ayounsi) [16:21:47] 10SRE-tools, 10netops, 10Infrastructure-Foundations, 10Spicerack: Spicerack: add network devices support - https://phabricator.wikimedia.org/T306552 (10ayounsi) p:05Triage→03Medium [16:33:55] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-ulsfo: (Need By: TBD) rack/setup/install new mr1-ulsfo - https://phabricator.wikimedia.org/T294314 (10RobH) 05Open→03Resolved fixed the serial/power by just removing from old and duplicating on new, as it used all the old cables. [16:34:02] 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-ulsfo: (Need By: TBD) rack/setup/install new mr1-ulsfo - https://phabricator.wikimedia.org/T294314 (10RobH)