[00:59:23] (03PS4) 10BrandonXLF: Use one_or_none to handle non-existent queries [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/682030 (https://phabricator.wikimedia.org/T280915) [01:00:05] (03PS5) 10BrandonXLF: Use one_or_none to handle non-existent queries [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/682030 (https://phabricator.wikimedia.org/T280915) [04:46:30] 10Analytics-Radar, 10Product-Analytics (Kanban): [REQUEST] Investigate decrease in New Registered Users - https://phabricator.wikimedia.org/T289799 (10Iflorez) Hi all, I am looking at counts from [[ https://meta.wikimedia.org/wiki/Schema:ServerSideAccountCreation | ServerSideAccountCreation ]] and comparing t... [08:53:21] 10Analytics, 10Analytics-Wikistats: Wikimedia Statistics - Horizontal (time) axis wrongly formatted when the option "Monthly" is choosen - https://phabricator.wikimedia.org/T290551 (10Larske) [09:23:22] 'morning data friends [09:23:44] I'm trying to deploy a test airflow dag to the instance created at https://wikitech.wikimedia.org/wiki/Analytics/Systems/Airflow#platform-eng [09:24:11] but looks like /srv/airflow-platform_eng/dags is not group writable [09:24:32] ls -lhd /srv/airflow-platform_eng/dags drwxr-xr-x 2 analytics-platform-eng analytics-platform-eng 4.0K Jul 29 00:33 /srv/airflow-platform_eng/dags [09:25:36] should I use the service user to copy files at that location? If so, how do I login / impersonate that user? [09:26:34] Holler if you'd rather that I capture this in a ticket. I saw that the OP at https://phabricator.wikimedia.org/T284225 is closed, and I did not want to reopen / pollute that comment thread. [09:27:33] cc / clarakosi [09:39:13] gmodena: o/ IIRC the Search team deploys the dag via scap (so not a direct copy), not sure what is the procedure for the new instances.. In theory with `sudo -u analytics-platform-eng $command` you should be able to impersonate the service user and copy/chmod(if needed) files in the dag dir [09:40:19] the main issue in direct copy is not tracking what's in there, but for a test it should be fine [09:49:43] elukey ack. Thank! I wanted to check with responsible adult before starting to mess around with sudo :) [09:51:04] and this is indeed just a test; it is not meant to promote direct copy of files madness. [09:53:29] gmodena: it is fine then to just copy in my opinion, make sure that permissions are ok after that (owner/perms etc..) [10:08:06] elukey works like a charm [10:10:25] good :) [10:53:11] (03PS1) 10GoranSMilovanovic: logging [analytics/wmde/WD/WikidataAnalytics] - 10https://gerrit.wikimedia.org/r/719480 [10:53:23] (03CR) 10GoranSMilovanovic: [V: 03+2 C: 03+2] logging [analytics/wmde/WD/WikidataAnalytics] - 10https://gerrit.wikimedia.org/r/719480 (owner: 10GoranSMilovanovic) [11:59:39] (03PS2) 10Joal: Add num-partitions param to mw-history checkers [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/719290 (https://phabricator.wikimedia.org/T290469) [12:03:41] (03PS3) 10Joal: Add num-partitions param to mw-history checkers [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/719290 (https://phabricator.wikimedia.org/T290469) [12:12:04] (03PS3) 10Joal: Grow mediawiki-history oozie jobs resources [analytics/refinery] - 10https://gerrit.wikimedia.org/r/719111 (https://phabricator.wikimedia.org/T290469) [12:12:48] (03CR) 10Joal: "To be merged/deployed when https://gerrit.wikimedia.org/r/c/analytics/refinery/source/+/719290/ has been released (potential jar version c" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/719111 (https://phabricator.wikimedia.org/T290469) (owner: 10Joal) [14:26:55] 10Analytics, 10Cassandra, 10Data-Engineering, 10Data-Engineering-Kanban, 10Epic: Cassandra3 migration for Analytics AQS - https://phabricator.wikimedia.org/T249755 (10BTullis) The transfer operation of the snapshots completed successfully. I have done some additional verification to make sure that the s... [14:39:37] (03PS1) 10GoranSMilovanovic: minor [analytics/wmde/WD/WikidataAnalytics] - 10https://gerrit.wikimedia.org/r/719529 [14:39:50] (03CR) 10GoranSMilovanovic: [V: 03+2 C: 03+2] minor [analytics/wmde/WD/WikidataAnalytics] - 10https://gerrit.wikimedia.org/r/719529 (owner: 10GoranSMilovanovic) [15:19:15] 10Analytics, 10SRE, 10SRE-Access-Requests: Requesting access to analytics-privatedata-users group for Abban Dunne - https://phabricator.wikimedia.org/T289775 (10akosiaris) @odimitrijevic Hi! Your approval is required on this task. [15:30:04] 10Analytics, 10MobileFrontend, 10Performance-Team (Radar), 10Technical-Debt: Figure out XAnalytics stuff - https://phabricator.wikimedia.org/T190381 (10Jdlrobson) [15:39:58] 10Quarry: add stop status to quarry - https://phabricator.wikimedia.org/T289349 (10mdipietro) a:03mdipietro [15:43:25] 10Analytics, 10Patch-For-Review: Decide whether to migrate from Presto to Trino - https://phabricator.wikimedia.org/T266640 (10nshahquinn-wmf) [16:33:55] btullis: o/ [16:34:14] Yep? [16:34:25] are doing we doing the SRE sync? I saw in the meet's preview that the team is in it [16:34:54] (I didn't want to jump in another meeting by mistake, if it was say standup or similar) [16:35:18] Yes, we were just taking a 5 min break after standup. Will join now. [16:35:29] I think we try the tardis. [16:35:29] ah super, joining [16:35:45] we can also skip if there is nothing to discuss [16:37:11] I'm happy either way. [17:07:07] (03CR) 10Andrew Bogott: "recheck" [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/716558 (owner: 10Andrew Bogott) [17:18:41] (03PS1) 10Michael DiPietro: add stop status [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/719567 (https://phabricator.wikimedia.org/T289349) [17:22:29] (03CR) 10jerkins-bot: [V: 04-1] add stop status [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/719567 (https://phabricator.wikimedia.org/T289349) (owner: 10Michael DiPietro) [17:24:35] (03CR) 10Michael DiPietro: add stop status (031 comment) [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/719567 (https://phabricator.wikimedia.org/T289349) (owner: 10Michael DiPietro) [17:26:21] (03PS2) 10Michael DiPietro: add stop status [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/719567 (https://phabricator.wikimedia.org/T289349) [17:30:42] (03PS3) 10Michael DiPietro: add stop status [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/719567 (https://phabricator.wikimedia.org/T289349) [18:19:57] urbanecm: Hi - I have an answer to your riddle from yesterday :) [18:20:04] joal: great! [18:20:08] what is it? 🙂 [18:21:50] urbanecm: it's related to the fact that the 'legacy' pageview format (the one used for pageviews) is not using the same convention for date-in-filename that the one pageview-complete uses [18:23:07] urbanecm: in pageviews, pageviews-20200907-000000.gz contains data UP TO 2020-09-07T00:00 - therefore what we currently would put in 2020-09-06T23:00 [18:24:03] If you wish to find exact match for your pages, you need to use pageview shifted from an hour: 20200907-010000 -> 20200908-000000 (included) [18:24:09] urbanecm: --^ [18:24:17] urbanecm: Does it make sense? [18:24:23] joal: ah. That makes sense. Thanks for the explanation. [18:24:50] your welcome urbanecm - I had to dig a bit in the data to make sure that was the correct assumption :) [18:26:52] 🙂 [18:27:03] Ok team - gone for tonight see you tomorrow