[05:56:40] <_joe_> Amir1 / Emperor I've filed https://phabricator.wikimedia.org/T408062 for the work you should be doing for WE5. Please file subtasks as you see fit and let's talk about anything that is unclear. I set the priority to "high" to indicate how this is priority one work for the org. [05:57:05] <_joe_> that doesn't mean you need to drop what you're doing right now, but please plan to have some time for this next week [07:15:22] federico3: just wanted to be fully transparent in case it failed [07:19:34] ok, thanks [07:20:37] I have added a first version of nft-compatible transferpy to cumin1002 [07:20:58] please don't use it for anything you cannot lose time with, it is not yet production ready [07:21:21] specially needs more work for error handling [07:21:31] but for small things can be already tested [07:41:08] https://github.com/deuxfleurs-org/garage?tab=readme-ov-file interesting [07:48:10] yes, it was one of the alternatives to test as a replacement [08:26:14] (for media backups) [11:45:49] any volunteer for https://gerrit.wikimedia.org/r/c/operations/puppet/+/1198290 ? It's really run-the-mill, similar to previous changes [11:49:35] did es2057 transfer endeed succesfult? [11:49:42] *fully [11:53:28] My guess is yes, and that it didn't affect transfers at all, because I was transferring to cumin1003, and you weren't touching that except for running the command, federico3 ? [11:54:26] yes it went well [11:54:44] so it turns in the end I had removed something the right way [11:54:50] just I was super careful [11:54:59] thinking it could had affected you [11:55:14] indeed I was running the command from 1003 but transferring between the 2 dbs [11:55:30] that was my mistake, I got confused, too many hosts :-D [11:55:34] so all good in the end [11:55:41] the change also looks good to me [11:55:59] but I wonder if you may want the review of someone more up to date with mw config [11:56:13] no worries, all good (and in the worst case I could restart the xfer) [11:56:25] the only question is if site.pp needs update? [11:57:34] It's a runbook-driven change similar to previous ones, pretty safe [11:58:26] ah, site.pp looks good [11:58:55] somtimes I add a #WIP or #in setup or something and then forget to remove it [11:59:32] is it already on prometheus (zarcillo)? [11:59:58] I can see it is, so it looks all good [12:00:15] yep https://grafana-rw.wikimedia.org/d/000000273/mysql?forceLogin=true&from=now-12h&orgId=1&refresh=1m&timezone=utc&to=now&var-job=%24__all&var-port=9104&var-server=es2057 [12:06:15] thanks [13:13:48] FIRING: PuppetFailure: Puppet has failed on ms-be2078:9100 - https://puppetboard.wikimedia.org/nodes?status=failed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetFailure [13:26:35] that's not currently in production, but elukey & papaul are working on it to look at UEFI things [13:28:07] Emperor: o/ the host is reimaged with Trixie, wasn't able to repro (not sure if you saw the task's updates) [13:30:25] elukey: presumably the reimage fails because of missing key packages from trixie ( T407513 ) meaning the puppet run that's part of the reimage cookbook always fails? [13:30:26] T407513: Key packages missing from trixie-wikimedia - https://phabricator.wikimedia.org/T407513 [13:34:16] Emperor: I am talking about https://phabricator.wikimedia.org/T404356 [13:34:25] I cannot repro that issue :D