[11:55:42] Is anyone around who can tell me whether https://cxserver.wikimedia.org/v1 is routed through restbase, or if it gets routed to https://cxserver.discovery.wmnet directly from ATS? [12:08:42] duesen: seems to be direct, https://gerrit.wikimedia.org/g/operations/puppet/+/production/hieradata/common/profile/trafficserver/backend.yaml#33 [12:09:03] taavi: thank you! [12:14:27] I can't see much mention of it in restbase config, I think taavi is correct [13:10:18] <_joe_> cxserver is definitely not going via rb, yes [13:10:27] <_joe_> duesen: you can tell by a few things, but mostly [13:10:42] <_joe_> anything that goes via restbase has an /api/rest_v1 prefix [13:17:22] hi, I have a Puppet question! I need a new Unix system user with a sudo rule for humans to use it, I went to create both directly in the manifest which has all the logic so everything is in the same `.pp` file which looks great. Is that any sane https://gerrit.wikimedia.org/r/c/operations/puppet/+/927975/3/modules/profile/manifests/local_dev/docker_publish.pp :) [13:17:56] that is using `sudo::group` which is apparently not that much used [13:20:02] the alternative is to move the logic to modules/admin/data/data.yaml then I don't need a fixed uid for the user and I kind of dislike having the sudo rule in a different file. It sounds easier to have everything at the same place [13:27:07] <_joe_> didn't you just answer yourself? [13:27:09] <_joe_> :) [13:27:51] <_joe_> hashar: that needs to be in admin/data [13:28:27] <_joe_> else IIRC sudo::group clashes with it [13:28:37] I guess so yeah [13:28:47] but I can keep the system user in the manifest? [13:29:28] I so liked the pattern of having everything next to each other :-] [14:00:47] moritzm: Can we talk about https://gerrit.wikimedia.org/r/c/operations/puppet/+/927795 briefly? [14:01:32] I think my patch is correct, inasmuch as the purge option is still an option, and the current code breaks if it isn't set. [14:15:56] yeah, fair enough, +1d [14:19:29] <_joe_> volans: serviceops has been discussing the best way to properly stop a LAMP pod, I told them you have some prior art on the topic [14:19:52] :-P [14:19:59] art is the right word indeed [14:20:03] <_joe_> volans: wanna hear something funny? [14:20:16] <_joe_> we did some sorcery using envoy's draining capabilities [14:20:28] <_joe_> and then found out we can just sleep N seconds [14:20:34] ahahahah [14:20:35] <_joe_> and to all practical purposes the result is the same [14:20:55] <_joe_> so claime was asking if we want to keep the draining code as an option [14:21:09] <_joe_> _joe_> call the option prestop_volans [14:21:16] GRRRR :D [14:21:24] :D [14:21:26] <_joe_> :* [14:22:05] I'm happy we get to keep my bash monstruosity to call the draining api and then watch for connections tbh [14:22:22] It won't be used, but it'll be there [14:22:35] thanks moritzm :) [14:25:51] yw, good luck with the rest of the bookworm setup :-) [14:28:47] I think that was the last piece! [14:29:26] well, that + waiting for release [14:44:26] _joe_, claime: job queue wait time is > 30 minutes again [15:04:04] <_joe_> duesen: let's see how it goes over a few days, but also [15:04:15] <_joe_> if we knew what's causing those jumps [15:04:19] <_joe_> it would help [15:04:50] is there a ticket I can read to get context about the change? [15:05:09] (about what the job does) [15:26:54] jynus: https://phabricator.wikimedia.org/T320534 [15:27:03] thanks [15:27:45] I understand now [15:27:49] jynus: basically, the job renders a page with parsoid and writes the result to parser cache. The job is scheduled on edit, and also when viewing a page that has stale cached html due to a template change. [15:28:43] _joe_: I'm not sure how to interpret the graph - are the jumps specific to this job? Or could they be caused by a totally different job clogging the pipes? [15:29:39] There are some pages that are slow to parse, and there are also spikes of edits, but given the fact that we can render more than 40 pages at once, these sudden spikes are quite odd. It's as if all rendering was blocked for a while. [15:38:28] and it is now on all wikis? [15:47:21] fyi: I'm going to begin the eqiad portion of the sessionstore/cassandra upgrade in the next 15-45 mins /cc rzl cwhite claime [15:47:34] if I don't hear any objections of course... [15:47:49] ack [15:48:00] thanks! good luck! [15:48:10] 🚀 [16:05:58] Ok, I'm de-pooling eqiad! [18:14:20] Alright, all done, re-pooling sessionstore/eqiad