[08:52:31] 10Puppet, 10Infrastructure-Foundations, 10GitLab (Infrastructure), 10Patch-For-Review, and 3 others: Puppetise gitlab-ansible playbook - https://phabricator.wikimedia.org/T283076 (10Jelto) I'm going to disable puppet on production GitLab (gitlab1001) soon for around two hours to test https://gerrit.wikimed... [08:58:05] 10netops, 10Infrastructure-Foundations, 10serviceops: TCP retransmissions in eqiad and codfw - https://phabricator.wikimedia.org/T291385 (10cmooney) I'm gonna resolve this task for now rather than have it sitting around. The strong suspicion is still that the drops on our core switching fabric are responsib... [08:58:23] 10netops, 10Infrastructure-Foundations, 10serviceops: TCP retransmissions in eqiad and codfw - https://phabricator.wikimedia.org/T291385 (10cmooney) 05In progress→03Resolved [09:51:16] 10Puppet, 10Infrastructure-Foundations, 10GitLab (Infrastructure), 10Patch-For-Review, and 3 others: Puppetise gitlab-ansible playbook - https://phabricator.wikimedia.org/T283076 (10Jelto) GitLab on the replica looks fine and change of the uid/gid was successful. I used the following steps: ` sudo /usr/bi... [10:02:39] 10CAS-SSO, 10Infrastructure-Foundations, 10Observability-Metrics, 10SRE, 10User-jbond: Thanos and Grafana lose the session after an hour - https://phabricator.wikimedia.org/T268233 (10MoritzMuehlenhoff) Thanks Timo This quarter we're drafting our plans for requirements in terms of 2FA (and the implied tr... [10:34:35] 10netbox, 10netops, 10Infrastructure-Foundations, 10SRE: Stage drmrs in Netbox - https://phabricator.wikimedia.org/T283594 (10ayounsi) All infra cables needed for remote hands have been created in Netbox. Most IPs allocated as well. [11:36:11] XioNoX, topranks: when running the makevm cookboo, netbox presents me various drmrs devices, okay to go ahead and merge these along? [11:37:52] moritzm: yep, I forgot to run the dns cookbook [11:38:01] ack, thx [12:41:34] 10Puppet, 10Infrastructure-Foundations, 10GitLab (Infrastructure), 10Patch-For-Review, and 3 others: Puppetise gitlab-ansible playbook - https://phabricator.wikimedia.org/T283076 (10Jelto) I re-enabled puppet on `gitlab1001` and uid/gid change and git user configuration was successful. ` jelto@gitlab1001:... [13:58:25] 10Puppet, 10Cloud-VPS, 10Infrastructure-Foundations, 10Patch-For-Review, and 3 others: Refactor puppet:base module to reduce unneeded shared code paths - https://phabricator.wikimedia.org/T289661 (10dcaro) [20:59:42] 10Puppet, 10Infrastructure-Foundations, 10GitLab (Infrastructure), 10Patch-For-Review, and 3 others: Puppetise gitlab-ansible playbook - https://phabricator.wikimedia.org/T283076 (10brennen) Looks good from my end - looks like there's some ongoing work with restore scripts, but feel free to resolve once th... [21:04:31] 10Puppet, 10Infrastructure-Foundations, 10GitLab (Infrastructure), 10Patch-For-Review, and 3 others: Puppetise gitlab-ansible playbook - https://phabricator.wikimedia.org/T283076 (10Dzahn) That's right. The restore script works when started manually but it does not work, and unfortunately breaks things, wh...