[08:30:11] 10SRE-tools, 10Infrastructure-Foundations: Decommissioning two hosts end up with: Failed to wipe swraid - https://phabricator.wikimedia.org/T311593 (10Marostegui) [08:39:47] 10SRE-tools, 10Infrastructure-Foundations: Decommissioning two hosts end up with: Failed to wipe swraid - https://phabricator.wikimedia.org/T311593 (10MoritzMuehlenhoff) Maybe we need run "swapoff -a" prior to the wipefs call? [08:51:06] 10SRE-tools, 10Infrastructure-Foundations: Decommissioning two hosts end up with: Failed to wipe swraid - https://phabricator.wikimedia.org/T311593 (10Marostegui) I have a few more hosts to decommission. I can try to do so, but we'd not know whether it helped or it would have just worked without it too :) Up t... [09:03:36] 10SRE-tools, 10Infrastructure-Foundations: Decommissioning two hosts end up with: Failed to wipe swraid - https://phabricator.wikimedia.org/T311593 (10MoritzMuehlenhoff) >>! In T311593#8036155, @Marostegui wrote: > I have a few more hosts to decommission. I can try to do so, but we'd not know whether it helped... [09:23:43] 10SRE-tools, 10Infrastructure-Foundations: Decommissioning two hosts end up with: Failed to wipe swraid - https://phabricator.wikimedia.org/T311593 (10Marostegui) Wilco! [10:44:44] 10Puppet, 10Beta-Cluster-Infrastructure, 10Infrastructure-Foundations, 10Product-Infrastructure-Team-Backlog, 10VPS-Projects: Puppet failures on deployment-docker-changeprop01, deployment-docker-cpjobqueue01, deployment-push-notifications01, deployment-docker-mob... - https://phabricator.wikimedia.org/T259812 [13:12:28] 10SRE-tools, 10Infrastructure-Foundations: Decommissioning two hosts end up with: Failed to wipe swraid - https://phabricator.wikimedia.org/T311593 (10Marostegui) I ran swapoff -a on db2081 and it went fine. Could be coincidence or it could've been the fix. Hard to know. However, I guess it doesn't hurt to inc... [14:06:26] 10CAS-SSO, 10Infrastructure-Foundations, 10SRE: Enable webauthn in CAS to replace U2F - https://phabricator.wikimedia.org/T311236 (10MoritzMuehlenhoff) >>! In T311236#8023441, @MoritzMuehlenhoff wrote: >>>! In T311236#8023420, @jbond wrote: >>>but that bails out with a bean error related to the fasterxml par... [17:38:59] 10netops, 10Infrastructure-Foundations, 10SRE, 10Traffic: Upgrade to Bird 2 - https://phabricator.wikimedia.org/T310574 (10ssingh) ` ===== NODE GROUP ===== (40) authdns[1001,2001].wikimedia.org,c... [17:46:13] 10CAS-SSO, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Update CAS to 6.5 - https://phabricator.wikimedia.org/T311235 (10ssingh) p:05Triage→03Medium [20:59:20] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Configure cloudsw1-e4-eqiad and cloudsw1-f4-eqiad - https://phabricator.wikimedia.org/T304936 (10Cmjohnson) [21:43:56] 10netops, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Configure cloudsw1-e4-eqiad and cloudsw1-f4-eqiad - https://phabricator.wikimedia.org/T304936 (10Cmjohnson)