[08:36:33] Good morning. I will be continuing the refinery deploy that I didn't quite get finished yesterday. [08:47:54] !log deploying refinery with refinery source version 0.2.40 [08:47:56] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [09:49:54] !log deploying refinery to hdfs [09:49:56] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [10:06:11] !log deploying airflow analytics_test instance [10:06:15] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [10:06:38] !log deploying airflow analytics instance [10:06:39] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [10:15:25] 06Data-Engineering, 10Data-Platform (Data Platform Ops Week Working Group), 10Data-Platform-SRE (2024.05.06 - 2024.05.26): Migrate data-engineering-alerts email list from Mailman to a Google Group - https://phabricator.wikimedia.org/T364632#9798919 (10BTullis) 05Openā†’03Resolved I believe that this is... [10:39:56] 14Analytics, 06Data-Engineering, 06DBA, 10Event-Platform: Eventually Consistent MediaWiki State Change Events - https://phabricator.wikimedia.org/T120242#9799052 (10akosiaris) >>! In T120242#9792008, @Ottomata wrote: > > IMO this is also the level of data inconsistency we should aim for in externalized st... [13:10:28] (03PS1) 10Joal: Add wikidatawiki grouped-wikis file [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1031897 (https://phabricator.wikimedia.org/T364045) [15:24:46] 10Data-Engineering (Q4 2024 April 1st - June 30th), 10Event-Platform, 10GitLab (Pipeline Services MigrationšŸ¤): Migrate Data Engineering Pipelinelib repos to GitLab - https://phabricator.wikimedia.org/T344730#9800601 (10Ottomata) Nice! [15:26:43] 07Analytics-Data-Problem, 06Data-Platform, 06Movement-Insights: Unique devices per country spikes on wikifunctions - https://phabricator.wikimedia.org/T364872#9800609 (10VirginiaPoundstone) Removing the Data Products tag until the preliminary data quality review is conducted. If any implementation requiremen... [15:52:51] 14Analytics, 10AQS2.0, 06Tech-Docs-Team, 10Data Products (Epics Timeline), and 2 others: AQS 2.0 user documentation - https://phabricator.wikimedia.org/T288664#9800827 (10apaskulin) [15:57:11] 14Analytics, 10AQS2.0, 06Tech-Docs-Team, 10Data Products (Epics Timeline), and 2 others: AQS 2.0 user documentation - https://phabricator.wikimedia.org/T288664#9800854 (10apaskulin) [16:12:29] 06Data-Engineering: Add job to create Wikidata partition to wmf.mediawiki_wikitext_history - https://phabricator.wikimedia.org/T363451#9800967 (10VirginiaPoundstone) [17:28:37] 14Analytics, 06Data-Engineering, 06DBA, 10Event-Platform: Eventually Consistent MediaWiki State Change Events - https://phabricator.wikimedia.org/T120242#9801336 (10Ottomata) > justify that opinion with use cases Fair point! I will, or fail trying! :) (I probably should have left that "IMO" out of that l... [19:02:59] (03PS1) 10Joal: Update script importing XML dumps onto HDFS [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1032018 (https://phabricator.wikimedia.org/T364045) [19:09:47] (03PS1) 10Joal: Update MediawikiXMLDumpsConverter [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1032020 (https://phabricator.wikimedia.org/T364045) [19:41:32] (03PS2) 10Snwachukwu: Medaiwkihistory: typesafe access to compliance value. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1031433 (owner: 10Gmodena) [20:01:43] (03CR) 10Snwachukwu: [C:03+2] Medaiwkihistory: typesafe access to compliance value. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1031433 (owner: 10Gmodena) [20:21:09] (03Merged) 10jenkins-bot: Medaiwkihistory: typesafe access to compliance value. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1031433 (owner: 10Gmodena) [20:48:54] (03CR) 10Aqu: [C:03+1] "Looks good !" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1032020 (https://phabricator.wikimedia.org/T364045) (owner: 10Joal) [20:53:22] 07Analytics-Data-Problem, 06Data-Platform, 06Movement-Insights: Unique devices per country spikes on wikifunctions - https://phabricator.wikimedia.org/T364872#9801941 (10Mayakp.wiki) @MNeisler : Can you provide impact of this issue. I would like to know - proportion of wikifunctions unique devices to overall... [21:01:16] (03CR) 10Aqu: [C:03+1] "Looks OK!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1032018 (https://phabricator.wikimedia.org/T364045) (owner: 10Joal) [21:01:44] (03CR) 10Aqu: [C:03+1] Add wikidatawiki grouped-wikis file [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1031897 (https://phabricator.wikimedia.org/T364045) (owner: 10Joal)