[04:29:50] (03PS2) 10Andrew Bogott: Added test_worker.py [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/725496 [04:30:46] (03CR) 10jerkins-bot: [V: 04-1] Added test_worker.py [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/725496 (owner: 10Andrew Bogott) [04:31:47] (03CR) 10Andrew Bogott: "recheck" [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/725496 (owner: 10Andrew Bogott) [04:36:18] (03CR) 10Andrew Bogott: [C: 03+2] Added test_worker.py [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/725496 (owner: 10Andrew Bogott) [04:40:31] (03Merged) 10jenkins-bot: Added test_worker.py [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/725496 (owner: 10Andrew Bogott) [05:53:37] 10Analytics-Radar, 10SecTeam-Processed, 10Security, 10User-Elukey: Establish a process to periodically review and approve access for hadoop/hue users - https://phabricator.wikimedia.org/T121136 (10elukey) 05Open→03Declined I think that we can safely close this task for the moment. webrequest data (and... [07:06:43] (03CR) 10Joal: [C: 03+1] "Happy to merge this, but I don't confident enough in how exclusions are reorganized - @guillaume would you mind giving your approval - I'l" [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/730282 (owner: 10ODimitrijevic) [13:39:30] (03PS1) 10Jgiannelos: maps.tile-change: Use batched tiles per event instead of single tile [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/730803 (https://phabricator.wikimedia.org/T293366) [13:45:50] FYI; I'm installing apache security updates on the various analytics web tools now, there might a small glitch when Apache restarts upon the update [13:47:42] all done [14:02:02] Thank you moritzm [14:34:35] (03CR) 10Ottomata: "Perhaps this is actually a new schema, since you are now representing multiple tiles? maps/tiles_change?" [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/730803 (https://phabricator.wikimedia.org/T293366) (owner: 10Jgiannelos) [14:39:01] Took a look at https://phabricator.wikimedia.org/T292879 and I wonder if it's just that user JAnstee is missing a presto permission, maybe needs kerberos enabled? [14:39:27] Hm ok there's another piece of information on the ticket: [14:39:28] > According to https://phabricator.wikimedia.org/T266249, she's already in the analytics-privatedata-users group. [14:40:20] Maybe it has to do with capitalization: presto sees JAnstee but in our system she is janstee ? [14:45:17] yeah i think that is the problem [14:45:18] https://superset.wikimedia.org/users/show/254 [14:45:29] not sure how that happens though... [14:45:32] i sign in as Ottomata [14:45:34] and my username is otto [14:46:23] you can manually edit the user record [14:46:23] https://superset.wikimedia.org/users/edit/254 [14:46:36] but hmm, maybe LDAP has the shell username wrong? [14:47:42] nope, looks right [14:47:44] uid: janstee [14:47:44] cn: JAnstee [14:48:02] razzi: try editing her user record User Name to janstee ? [14:48:04] dunno why it is rong [14:56:16] +1 I'd do the same test [16:04:11] (03PS1) 10Jgiannelos: maps: Schema for batched tile changes [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/730841 (https://phabricator.wikimedia.org/T293366) [16:06:15] (03Abandoned) 10Jgiannelos: maps.tile-change: Use batched tiles per event instead of single tile [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/730803 (https://phabricator.wikimedia.org/T293366) (owner: 10Jgiannelos) [16:10:07] (03CR) 10Jgiannelos: "Follow up patch with the new schema here: https://gerrit.wikimedia.org/r/c/schemas/event/primary/+/730841" [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/730803 (https://phabricator.wikimedia.org/T293366) (owner: 10Jgiannelos) [16:13:19] (03CR) 10Ottomata: "Nit on naming, but up to you." [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/730841 (https://phabricator.wikimedia.org/T293366) (owner: 10Jgiannelos) [16:22:54] (03PS2) 10Jgiannelos: maps: Schema for batched tile changes [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/730841 (https://phabricator.wikimedia.org/T293366) [16:23:07] (03CR) 10Jgiannelos: maps: Schema for batched tile changes (031 comment) [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/730841 (https://phabricator.wikimedia.org/T293366) (owner: 10Jgiannelos) [17:43:14] 10Analytics: Conda's CPPFLAGS may not be correct when pip installing a package that needs c/cpp compilation - https://phabricator.wikimedia.org/T292699 (10elukey) No real need, I think it is fine to wait if anybody wants to get experience with Debian packaging etc.. I was wondering if we could also follow up wi... [18:06:34] 10Analytics, 10Data-Engineering, 10SRE Observability (FY2021/2022-Q2), 10User-fgiunchedi: Migrate analytics cluster alerts from Icinga to AlertManager - https://phabricator.wikimedia.org/T293399 (10BTullis) [18:48:21] (03PS1) 10GoranSMilovanovic: GT291170 [analytics/wmde/WD/WikidataAdHocAnalytics] - 10https://gerrit.wikimedia.org/r/730877 [18:48:32] (03CR) 10GoranSMilovanovic: [V: 03+2 C: 03+2] GT291170 [analytics/wmde/WD/WikidataAdHocAnalytics] - 10https://gerrit.wikimedia.org/r/730877 (owner: 10GoranSMilovanovic) [18:49:20] (03PS1) 10GoranSMilovanovic: T291170 [analytics/wmde/WD/WikidataAdHocAnalytics] - 10https://gerrit.wikimedia.org/r/730880 [18:49:30] (03CR) 10GoranSMilovanovic: [V: 03+2 C: 03+2] T291170 [analytics/wmde/WD/WikidataAdHocAnalytics] - 10https://gerrit.wikimedia.org/r/730880 (owner: 10GoranSMilovanovic) [19:08:47] 10Analytics-Radar, 10Event-Platform, 10WMF-JobQueue, 10Wikibase change dispatching scripts to jobs, and 2 others: Queuing jobs is extremely slow - https://phabricator.wikimedia.org/T292048 (10Legoktm) >>! In T292048#7399106, @Ladsgroup wrote: > We ran into this issue again today, we introduced a new job, l... [19:42:00] 10Analytics-Radar, 10Event-Platform, 10WMF-JobQueue, 10Wikibase change dispatching scripts to jobs, and 2 others: Queuing jobs is extremely slow - https://phabricator.wikimedia.org/T292048 (10Ottomata) > Increasing the replicas also reduced the save time p75 Wow, let's add more more replicas. I can see t... [19:43:54] 10Analytics-Radar, 10Event-Platform, 10WMF-JobQueue, 10Wikibase change dispatching scripts to jobs, and 2 others: Queuing jobs is extremely slow - https://phabricator.wikimedia.org/T292048 (10Ottomata) > Or maybe we should remove a tcp connection from the equation. @Joe you mean skip the eventgate middle-... [19:44:34] 10Analytics-Radar, 10Event-Platform, 10WMF-JobQueue, 10Wikibase change dispatching scripts to jobs, and 2 others: Queuing jobs is extremely slow - https://phabricator.wikimedia.org/T292048 (10Ottomata) > Any reason we can't/shouldn't run an eventgate sidecar along with MW in k8s? That doesn't remove the TC... [20:53:08] 10Analytics-Radar, 10Event-Platform, 10WMF-JobQueue, 10Wikibase change dispatching scripts to jobs, and 2 others: Queuing jobs is extremely slow - https://phabricator.wikimedia.org/T292048 (10matmarex) [21:03:40] (03CR) 10Cicalese: [C: 03+1] Update PHP and version queries [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/730389 (owner: 10Tim Starling)