[03:52:19] (03CR) 10TChin: [V:03+2 C:03+2] Add refinery-source jars for v0.2.50 to artifacts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1073499 (owner: 10Maven-release-user) [03:56:37] !log Deployed refinery-source using jenkins [03:56:39] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [08:47:16] 10Data-Engineering (Q1 2024 July 1st - September 30th), 06Data-Platform-SRE: Obtain SRE resources needed to test the HDFS synchronizer service - https://phabricator.wikimedia.org/T371994#10156210 (10Gehel) Code for this application is at https://gitlab.wikimedia.org/repos/data-engineering/hdfs-synchronizer/-/m... [08:55:25] (03CR) 10Gehel: [C:04-1] "recheck" [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1062711 (https://phabricator.wikimedia.org/T360219) (owner: 10Gehel) [09:24:45] 06Data-Engineering: Move more of refine_hive_hourly dag logic into RefineConfiguration - https://phabricator.wikimedia.org/T375064 (10Antoine_Quhen) 03NEW [09:33:50] 06Data-Engineering: Move more of refine_hive_hourly dag logic into RefineConfiguration - https://phabricator.wikimedia.org/T375064#10156347 (10Antoine_Quhen) [09:33:53] 10Data-Engineering (Q1 2024 July 1st - September 30th), 13Patch-For-Review: [Refine refactoring] Refine jobs should be scheduled by Airflow: implementation - https://phabricator.wikimedia.org/T356762#10156348 (10Antoine_Quhen) [09:35:09] (03CR) 10David Martin: Add mediawiki.product_metrics.wikilambda_api data stream to the sanitization allowlist (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1064438 (https://phabricator.wikimedia.org/T372987) (owner: 10David Martin) [09:38:59] (03CR) 10David Martin: "Hi @dandreescu@wikimedia.org - I responded on the other patch with a proposed resolution. However, no resolution is needed here. All of " [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1064435 (https://phabricator.wikimedia.org/T372986) (owner: 10David Martin) [10:47:43] (03CR) 10David Martin: Add mediawiki.product_metrics.wikilambda_api data stream to the sanitization allowlist (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1064438 (https://phabricator.wikimedia.org/T372987) (owner: 10David Martin) [11:23:05] !log Deploying refinery [11:23:06] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:45:40] 10Data-Engineering (Q1 2024 July 1st - September 30th), 06Data-Platform-SRE: Obtain SRE resources needed to test the HDFS synchronizer service - https://phabricator.wikimedia.org/T371994#10156642 (10BTullis) I'd be happy to work on this with you @amastilovic Many of the steps above are things that you can dri... [11:46:50] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Data-Platform-SRE (2024.09.06 - 2024.09.27): Obtain SRE resources needed to test the HDFS synchronizer service - https://phabricator.wikimedia.org/T371994#10156650 (10BTullis) [11:47:57] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Data-Platform-SRE (2024.09.06 - 2024.09.27): Deploy the HDFS synchronizer service to the dse-k8s cluster - https://phabricator.wikimedia.org/T371994#10156654 (10BTullis) [12:09:45] !log Deployed refinery using scap, then deployed onto hdfs [12:09:48] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:39:37] 06Data-Engineering, 10Dumps 2.0 (Kanban Board): [Iceberg Migration] Implement mechanism for automatic Iceberg table maintenance - https://phabricator.wikimedia.org/T338065#10157106 (10xcollazo) 05In progress→03Resolved [13:41:34] 06Data-Engineering, 10Dumps 2.0: [Iceberg Migration] Extend Iceberg table maintenance mechanism to support data rewrite - https://phabricator.wikimedia.org/T373694#10157113 (10xcollazo) While working on T369868, we figured that we will indeed need this mechanism for #dumps_2.0, as the new approach of 3 writes... [13:41:36] 06Data-Engineering, 10Dumps 2.0: [Iceberg Migration] Extend Iceberg table maintenance mechanism to support data rewrite - https://phabricator.wikimedia.org/T373694#10157118 (10xcollazo) [13:43:05] 06Data-Engineering, 10Dumps 2.0: [Iceberg Migration] Extend Iceberg table maintenance mechanism to support data rewrite - https://phabricator.wikimedia.org/T373694#10157125 (10xcollazo) [14:15:40] 06Data-Engineering, 10Data-Engineering-Dashiki, 10Data Products (Epics Timeline), 07Epic: Public dashboard process - https://phabricator.wikimedia.org/T361214#10157278 (10AndrewTavis_WMDE) Note from the Data Usage at WMF Interview that I just participated in: @Milimetric suggested that [Dashiki](https://wi... [14:27:13] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Event-Platform, 13Patch-For-Review: Migrate Event Platform Schema Respositories to Gitlab - https://phabricator.wikimedia.org/T366836#10157337 (10Snwachukwu) @gmodena , We have the mediawiki group and wmde group (recently added by developer-experienc... [14:38:29] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Event-Platform, 13Patch-For-Review: Migrate Event Platform Schema Respositories to Gitlab - https://phabricator.wikimedia.org/T366836#10157398 (10Snwachukwu) I have MRs for contribution policy drafts. Please help review @gmodena 1. [[ https://gitlab.... [15:09:31] 06Data-Engineering, 06Data Products, 10MediaWiki-extensions-EventLogging, 10Temporary accounts: Prepare EventLogging for temp accounts - https://phabricator.wikimedia.org/T374812#10157538 (10VirginiaPoundstone) @Ahoelzl and @lbowmaker this work is part of DPE's temp account work. Please add this to work pl... [15:20:43] (03PS1) 10Gerrit maintenance bot: Add gor.wikiquote to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1073826 (https://phabricator.wikimedia.org/T375095) [15:21:31] (03PS1) 10Gerrit maintenance bot: Add nr.wikipedia to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1073829 (https://phabricator.wikimedia.org/T375102) [15:48:21] 06Data-Engineering: [Maintenance] Add a deletion job for `hdfs_usage` data - https://phabricator.wikimedia.org/T348774#10157864 (10JAllemandou) Ping @lbowmaker , we should prioritize this if we can :) [15:53:25] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to stat1007 for cyndywikime - https://phabricator.wikimedia.org/T375060#10157879 (10Vgutierrez) [15:54:09] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to stat1007 for cyndywikime - https://phabricator.wikimedia.org/T375060#10157890 (10Vgutierrez) SSH key has been confirmed out of band [15:55:34] 06Data-Engineering, 10CirrusSearch, 10Structured Data Engineering, 06Structured-Data-Backlog, 03Discovery-Search (Current work): Migrate image recommendation to use page_weighted_tags_changed stream - https://phabricator.wikimedia.org/T372912#10157895 (10pfischer) 05Open→03In progress a:03pfischer