[07:05:40] (03CR) 10Joal: [C:03+1] "LGTM! thanks @kcvelaga@wikimedia.org :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1306491 (https://phabricator.wikimedia.org/T430020) (owner: 10KCVelaga) [08:34:54] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): dbt-jobs backfill: all base models for moderator actions - https://phabricator.wikimedia.org/T429995#12084379 (10JMonton-WMF) The backfill is running. I run a `--full-refresh` for the first month: `bash start_date=2020-07-01 sudo -u analytics /opt/conda-an... [09:45:11] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Wikimedia Enterprise, 10Wikimedia Enterprise - Content Integrity, 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 07Essential-Work: Implement an Airflow operator for moving data from point A to B - https://phabricator.wikimedia.org/T405360#12084705 (... [09:45:32] 06Data-Engineering, 06Data-Platform-SRE (2026-07-03 - 2026-07-31): Enable Ceph S3 locations for Hive Metastore tables - https://phabricator.wikimedia.org/T425673#12084716 (10Gehel) [09:45:37] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-07-03 - 2026-07-31): Optimize enqueueing of refine_webrequest_hourly pipeline - https://phabricator.wikimedia.org/T419050#12084720 (10Gehel) [09:45:55] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 07Essential-Work: Carry out end-user testing of spark on kubernetes - https://phabricator.wikimedia.org/T412925#12084728 (10Gehel) [09:46:03] 06Data-Engineering, 06Data-Engineering-Radar, 06cloud-services-team, 06Data-Persistence, and 3 others: Create wiki replicas views for globaljsonlinks tables - https://phabricator.wikimedia.org/T387419#12084730 (10Gehel) [09:46:16] 06Data-Engineering, 06cloud-services-team, 06Data-Persistence, 10Data-Services, and 3 others: Set up x1 replication to Wiki Replicas - https://phabricator.wikimedia.org/T395881#12084732 (10Gehel) [09:46:26] 06Data-Engineering, 06Data-Engineering-Radar, 06Privacy Engineering, 06Security-Team, and 2 others: Privacy review of x1 tables in preparation of adding them to wikireplicas - https://phabricator.wikimedia.org/T415219#12084738 (10Gehel) [09:48:39] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE (2026-07-03 - 2026-07-31): implement script to move data from P&T data lake to FR Tech data lake - https://phabricator.wikimedia.org/T425133#12084785 (10Gehel) [09:48:45] 06Data-Engineering, 10Test Kitchen, 06Data-Platform-SRE (2026-07-03 - 2026-07-31): Airflow instance for Experiment Platform - https://phabricator.wikimedia.org/T416709#12084787 (10Gehel) [09:53:41] 06Data-Engineering, 06Data-Platform-SRE (2026-07-03 - 2026-07-31): Task Tries and Logs for Airflow DAGs sometimes unavailable - https://phabricator.wikimedia.org/T419162#12084879 (10Gehel) [09:53:55] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 13Patch-For-Review: Add a presto query logger - https://phabricator.wikimedia.org/T269832#12084881 (10Gehel) [09:54:05] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data Pipelines, 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 07Essential-Work: Airflow dynamic task mapping logs mix up when, on rerun, an id is mapped to a different map_index_template - https://phabricator.wikimedia.org/T408802#12084886 (10Geh... [09:54:36] 06Data-Engineering, 10BetaFeatures, 06cloud-services-team, 10Data-Services, and 5 others: Create view for betafeatures_user_counts table in wiki replicas - https://phabricator.wikimedia.org/T402145#12084890 (10Gehel) [09:55:22] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 07Essential-Work: Move the dumps_v1 DAGs from the Airflow test_k8s instance to the main instance - https://phabricator.wikimedia.org/T404084#12084927 (10Gehel) [09:55:47] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 07Essential-Work: Superset "track job" button leads to broken URL - https://phabricator.wikimedia.org/T410149#12084940 (10Gehel) [09:56:55] 06Data-Engineering, 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 07Essential-Work: ERROR AsyncEventQueue: Listener DatahubSparkListener threw an exception - https://phabricator.wikimedia.org/T400207#12084971 (10Gehel) [09:57:11] 06Data-Engineering, 06Data-Platform-SRE (2026-07-03 - 2026-07-31): Delete orphan EventLogging topic `eventlogging_HomepageModulet` - https://phabricator.wikimedia.org/T429017#12084982 (10Gehel) [10:29:58] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: EPIC: DYK Archive - https://phabricator.wikimedia.org/T430319#12085138 (10Snwachukwu) [10:30:42] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: Write Design Doc - https://phabricator.wikimedia.org/T431102 (10Snwachukwu) 03NEW [10:33:14] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: Backfill DYK Archive HTML for latest HTML in event.mediawiki_page_html_content_change_v1 - https://phabricator.wikimedia.org/T431103 (10Snwachukwu) 03NEW [10:34:26] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: Write Design Doc - https://phabricator.wikimedia.org/T431102#12085196 (10Snwachukwu) [10:34:27] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: EPIC: DYK Archive - https://phabricator.wikimedia.org/T430319#12085197 (10Snwachukwu) [10:34:28] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: Backfill DYK Archive HTML for latest HTML in event.mediawiki_page_html_content_change_v1 - https://phabricator.wikimedia.org/T431103#12085195 (10Snwachukwu) [10:50:29] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027: Example visualization for IR2 - https://phabricator.wikimedia.org/T426763#12085234 (10Snwachukwu) [11:01:15] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027: IR1: Example visualization of intervention - https://phabricator.wikimedia.org/T425926#12085271 (10Snwachukwu) [11:03:43] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Secret management on airflow for the automated transfer of (public) datasets from stats infra --> WME AWS - https://phabricator.wikimedia.org/T415208#12085275 (10Snwachukwu) Not sure of the status. I'll sync with @Htriedman [12:12:13] 06Data-Engineering, 06Data-Platform-SRE, 06SRE, 07Sustainability (Incident Followup): The webrequest_sampled_live data pipeline and its query tools have become mission-critical and require re-engineering for resilience - https://phabricator.wikimedia.org/T431112 (10BTullis) 03NEW [14:07:31] 06Data-Engineering, 06cloud-services-team, 10Data-Services, 06Privacy Engineering, and 2 others: Add global_edit_count to wikireplicas - https://phabricator.wikimedia.org/T344108#12085846 (10EMill-WMF) @Soda Thanks for pushing here - your analysis is right, and we are fine with exposing this value without... [14:18:59] 06Data-Engineering, 06cloud-services-team, 10Data-Services, 06DBA, and 3 others: Add global_edit_count to wikireplicas - https://phabricator.wikimedia.org/T344108#12085858 (10taavi) Similar to T402145, this table is currently marked as `visibility: private` in the catalog, and so isn't replicated to the Wi... [14:20:03] 06Data-Engineering, 06SRE, 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 07Sustainability (Incident Followup): The webrequest_sampled_live data pipeline and its query tools have become mission-critical and require re-engineering for resilience - https://phabricator.wikimedia.org/T431112#12085861 (10BTullis... [14:30:03] !log Test Kitchen experiment (poll 120410) - adds: none; removes: logged-out-retention-round15; fields: none - TK tips at https://w.wiki/_cvdP [14:30:05] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:53:10] 06Data-Engineering, 06cloud-services-team, 10Data-Services, 06DBA, and 3 others: Add global_edit_count to wikireplicas - https://phabricator.wikimedia.org/T344108#12085972 (10Marostegui) >>! In T344108#12085858, @taavi wrote: > Similar to T402145, this table is currently marked as `visibility: private` in... [14:53:35] 06Data-Engineering, 06cloud-services-team, 10Data-Services, 06DBA, and 3 others: Add global_edit_count to wikireplicas - https://phabricator.wikimedia.org/T344108#12085974 (10Marostegui) >>! In T344108#12085972, @Marostegui wrote: >>>! In T344108#12085858, @taavi wrote: >> Similar to T402145, this table is... [14:55:26] 06Data-Engineering, 06cloud-services-team, 10Data-Services, 06DBA, and 3 others: Add global_edit_count to wikireplicas - https://phabricator.wikimedia.org/T344108#12085978 (10taavi) >>! In T344108#12085973, @Marostegui wrote: > This would need to be signed off by Security. That's T344108#12085846, unless... [14:58:20] 06Data-Engineering, 06cloud-services-team, 10Data-Services, 06DBA, and 3 others: Add global_edit_count to wikireplicas - https://phabricator.wikimedia.org/T344108#12085999 (10Marostegui) Ah right! Missed that! THanks [15:16:02] 06Data-Engineering, 06SRE, 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 07Sustainability (Incident Followup): The webrequest_sampled_live data pipeline and its query tools have become mission-critical and require re-engineering for resilience - https://phabricator.wikimedia.org/T431112#12086035 (10elukey... [16:26:07] (03PS2) 10Aqu: Add GSC site/url impression Hive tables [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1305857 (https://phabricator.wikimedia.org/T427457) [18:46:18] 06Data-Engineering, 06SRE, 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 07Sustainability (Incident Followup): The webrequest_sampled_live data pipeline and its query tools have become mission-critical and require re-engineering for resilience - https://phabricator.wikimedia.org/T431112#12086387 (10LSobans...