[09:35:58] quick fix: https://gerrit.wikimedia.org/r/c/operations/puppet/+/730435 but open for alternatives [09:39:49] jynus: ack [09:41:01] it is sadly a shortcoming of mixing debian packages and puppet dependencies :-( [09:41:26] yeah, sigh [09:42:09] not sure if worth doing it properly, as I envision this done better as part of a new db framework [10:16:09] jynus: i had a go at improving things: https://gerrit.wikimedia.org/r/c/operations/puppet/+/730443 [10:16:27] thanks! [10:17:13] oh, you changed you code, I don't think that was wrong [10:17:17] *your [10:17:26] I wasn't very happy about mine in reality [10:18:34] sadly I cannot vote +0.5: I think the change is ok, but you should decide what is better! [10:19:35] if you end up merging that, I will revert my other patch- but I was mostly thinking on improving only the cumin parts [10:23:59] i reckon it's simplest to work around the debian/puppet dependency issue in one place, so that other users don't have to care [10:24:07] pcc looks sane, so i'll go ahead and merge this [10:24:58] +1 [10:25:59] jynus: thanks for giving me a heads up on this issue, i wouldn't have noticed it otherwise [12:17:28] Emperor: since eqiad new hw is essentially in service, I was thinking next week we can put back traffic fully to eqiad and finish ms-be2045 and the new codfw hw, then put traffic back to both codfw and eqiad [13:10:39] sobanski: look, we have some docs on packaging orchestrator! https://wikitech.wikimedia.org/wiki/Orchestrator#Operations [13:14:31] Nice! I linked to it from the Runbooks page for databases. [13:15:14] (i'm working on the orch upgrade, and i'll be damned if i'm going to figure this all out once and _not_ document it) [13:25:17] buf, for wikivoyage, non of the visible (latest or older version) files are missing, but only 45% of the deleted ones are on swift [13:25:23] *enwikivoyage [13:26:36] there are 491 I cannot find on swift, unles they are on a random container [13:27:55] mmm, I think I found a new type of corruption/inconsistency on the db [13:28:06] the upload time says 2008 [13:28:19] but the wiki was created on wikimedia in 2012 [13:28:42] so maybe it was imported from externally, but private files from before 2012 were lost [13:29:35] so many edge cases :-(