[07:54:20] godog: good morning! following up from last week, I patched our already-patched prometheus .deb to include the openstack SD fix I need: https://people.wikimedia.org/~taavi/prometheus_2.24.1_wmf2/ [07:54:35] could you build that and include in apt.wm.o? [08:26:58] taavi: hi! oh sweet, thank you! yes I'll build/upload tomorrow (making a note) [09:00:23] godog: quick question, did the patch to alertmanager get deployed too? (the one allowing to choose the default alertmanager instances when creating a new silence) [09:02:43] dcaro: your patch to karma ? I was expecting a new upstream release but that didn't happen yet afaics ? but yeah we can patch locally [09:02:45] it was actually karma [09:02:46] yes [09:03:39] yeah upstream hasn't done releases in the last few months, I'm guessing he's busy [09:03:42] godog: iirc there was a release upstream, but not on debian right? (and the upstream one was on top of changes that required a new alertmanager) [09:04:03] karma v0.113 includes the patch I think [09:04:09] ah yeah you are right dcaro, it was tied to a new alertmanager [09:04:29] * dcaro is starting to remember things too xd [09:05:24] we'll be upgrading alert hosts to bookworm this quarter or the next, that will get us to alertmanager 0.25 though [09:05:52] 👍 [09:07:51] godog: do you have a task I can link to? [09:07:57] (so I don't forget once that is done xd) [09:10:43] dcaro: yes! https://phabricator.wikimedia.org/T333615 [09:10:46] making a note there [09:10:54] awesome, thanks! [09:27:48] !log tools deploying calico-0.0.6-20230710081103-dcbbe692, just a rebuild (T341084) [09:27:52] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [09:27:52] T341084: [toolforge] Move all the components to the gitlab ci/cd flow - https://phabricator.wikimedia.org/T341084 [11:00:32] !log paws removed paws-db-backup-1 T340054 [11:00:35] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL [11:00:36] T340054: Is paws-db-backup-1 used? - https://phabricator.wikimedia.org/T340054 [11:18:48] !log paws shut down systems associated with T341457 should only be redirect page. [11:18:52] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL [11:18:52] T341457: Remove old paws hosts - https://phabricator.wikimedia.org/T341457 [13:01:14] !log tools deploy envvars-api 0.0.22 (T341462) [13:01:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [13:01:18] T341462: [api-gateway] nginx: don't fail to start if any backend is not there - https://phabricator.wikimedia.org/T341462 [17:57:20] !log tools.lexeme-forms deployed 42679bb5dc (l10n updates: yue) [17:57:25] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL [19:12:18] is this pageviews dumps https://dumps.wikimedia.org/other/pageview_complete/ in somewhere in toolforge? the dumps in "/public/dumps/pageviews" is different, is one file per hour and don't have the page ids [19:13:53] `/mnt/nfs/dumps-clouddumps1002.wikimedia.org/other/pageview_complete/` seems to exist but I don’t know how stable it is [19:15:26] (alternatively, `ls "$(realpath /public/dumps/pageviews)"/../pageview_complete/` at least avoids hard-coding the clouddumps1002 name) [19:17:30] that is what I was looking for, thank you! [19:20:56] danilo: `/public/dumps/public/other/pageview_complete/` would be the best path to use [20:14:40] I will use it, thanks [20:34:22] !log tools.stewardbots Deploy 2146af6, 2aed494 (T341003) [20:34:27] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [20:34:27] T341003: webservice python3.7 is deprecated - https://phabricator.wikimedia.org/T341003 [20:39:35] !log tools freeing up disk space usage on tools docker-registry with `taavi@tools-docker-registry-05:~$ sudo sudo -u docker-registry docker-registry garbage-collect /etc/docker/registry/config.yml --delete-untagged` [20:39:38] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [20:57:53] toolforge seems to be a bit slow. not quite in the usual "someone is hammering NFS" way though [23:23:51] !log tools.docker-registry Update upstream code to their v2.5.0 [23:23:53] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.docker-registry/SAL