[10:28:11] errand+lunch [12:59:58] o/ [13:45:56] having bouncer issues ATM :( [13:46:39] OK, I think they are fixed [15:01:49] \o [15:02:29] workout, back in ~40 [15:08:34] are the gitlab project keys ip restricted in some way? I'm putting together a ticket to get trusted runners for the jvm projects to do releases, but mildly surprised ci ships the token to wmcs runners, but the token only grants read access even though the ui says it can write [15:08:41] (or i got the access controls wrong :P) [15:18:28] o/ [15:18:31] no clue :/ [15:18:35] oh, it's the trusted branches stuff. The token has to be a "maintainer" and not a developer to push to a protected tag [15:23:48] in theory, thats ci publishing hebmorph: https://gitlab.wikimedia.org/repos/search-platform/HebMorph/-/packages [15:24:02] i imagine now extra tediousness to include all the random package registrys :P but thankfully this only needed in one place [15:25:49] * ebernhardson also realized now i actually never got opensearch-analysis-hebrew to pass tests...only assemble a loadable package :P [15:26:03] something funky with the security manager preventing it from loading the dictionaries during integration testing... [15:26:21] (which is separately amazing, because the integration test, afaict, is assert(true) [15:29:34] :) [15:30:36] that gitlab package is nice indeed but some proxy might be great to have a single place to lookup all of them [15:32:40] turns out you can get them at the project level too, from /repos/search-platform [15:32:53] but there is no deduplication, so don't share names [15:34:51] perhaps the parent pom is a place to list them all? or a "bom" to import if repositories are "importable" [15:37:45] re "from /repos/search-platform" you mean there can be an artifact repo at this level and written from all individual projects? [15:38:08] dcausse: i mean everything published to the projects shows up in a meta-repo at https://gitlab.wikimedia.org/groups/repos/search-platform/-/packages [15:38:25] oh [15:38:40] can this be included as a repository in a pom.xml? [15:38:42] dcausse: so we could point maven repo to that, instead of individual projects. But it doesn't have any smarts, simple merge of everything [15:38:58] but only for reading, publishing always goes to the repo [15:39:55] i haven't tested yet, but will soon. Currently testing the hebrew plugin from it locally installed, but it's on my list [15:40:40] sure [15:47:47] ebernhardson: re the plugin: https://gitlab.wikimedia.org/repos/search-platform/opensearch-analysis-hebrew/-/commit/6cbd08d033306de7be92c10f11673fccd4c3fddd [15:48:07] I think I simply removed the integration test :) [15:48:08] back [15:48:27] dcausse: oh, i guess that works too :) I was about to try and run the old version to see how it loaded the dictionaries [15:56:46] I once had a quick chat with the maintainer and he told that the paid version has a custom class so I would not be surprised that the test is passing with this custom class instead of the one loading hspell [16:15:49] yea looks to work fine, set the maven repo to https://gitlab.wikimedia.org/api/v4/groups/308/-/packages/maven [16:17:47] nice [16:21:56] you should even be able to use the group iD 186 (https://gitlab.wikimedia.org/repos) which should regroup everything published on Gitlab. [16:22:30] That's what the latest parent pom used: https://gitlab.wikimedia.org/repos/maven/wmf-jvm-parent-pom/-/blob/main/pom.xml?ref_type=heads#L201 [16:22:37] nifty [16:22:39] so if you upgrade to it, you'll get that for gree :) [16:22:44] s/gree/free/ [16:23:15] But that latest version is meant to publish artifacts to Gitlab, so you might have to check that part as well. Not sure if you want to tackle that right now. [16:23:57] well, these things are all gradle so far :P But i'll be onto maven soon [16:31:41] an airflow task marked "Scheduled" but with no attempts in 4 days is suspicious [16:32:05] "Not scheduling since there are 0 open slots in pool sequential and require 1 pool slots" [16:32:07] hm... [16:32:21] is something running? [16:32:35] otherwise, is the pool full? [16:32:38] seems like something is not "fair" [16:33:14] looking at other "big" jobs to see [16:35:07] feature_selection-20180215-query_explorer-dbn-20180215-query_explorer-pruned_mrmr is running since 2024-10-26 [16:35:48] ahh, so something got stuck. That isn't even a silly kafka task [16:36:24] approxQuantile at QuantileDiscretizer.scala:225 [16:36:50] yea seeing the same, it's been processing the same tasks for a long long time [16:37:18] umm, i guess can kill this and let it try again? [16:37:58] killed [16:38:27] thanks [16:39:09] looked at the stack of the sole executor running, found nothing, seemed like a deadlock [16:40:16] can see in the pool view, sorting by start date, lots of pending things. Probably going to take time to catch up [16:40:41] it started running a glent one now, so it did rotate instead of directly retrying same [16:41:27] so it's fair that's nice [16:42:17] well... I mean I don't know if glent requested the slot before the job I was checking but at least it's giving room for others [16:54:34] I wonder if I can abuse our partitionning definition for input table (table_name/col1=val1/col2=val2) with a column that's not a partition [16:55:16] dcausse: yea, thats expected. It's really just a static k=v match [16:56:32] yes... tempted to doing that instead of rethinking the whole sparql analysis jobs with that new "subgraph" column added in T376134 [16:56:42] dcausse: thats totally reasonable [16:57:07] T376134: WDQS query logging should be separated per subgraph - https://phabricator.wikimedia.org/T376134 [17:26:50] * ebernhardson realizes all over build.gradle it's spelled analisis [17:29:03] heading out, I'm off tomorrow, have a nice rest of the week [17:29:58] enjoy! [18:12:17] hmm, getting the to publish to gitlab from gerrit part now. left wondering which repo we would publish to [18:13:07] which would influence the secret i imagine, unless jenkins is using a different token from the per-repo access tokens i've been using [18:25:59] back