[07:52:05] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content, 10Dumps-Generation, 13Patch-For-Review: Decommission dumps job `download_enterprise_htmldumps` - https://phabricator.wikimedia.org/T390556#10815036 (10brouberol) Which directories exactly? [10:03:10] 06Data-Engineering, 06Data-Engineering-Radar, 10Observability-Logging, 06Traffic: Clean-up varnishkafka webrequest leftovers in Hadoop-world - https://phabricator.wikimedia.org/T394011 (10JAllemandou) 03NEW [10:03:55] 06Data-Engineering, 06Data-Engineering-Radar, 10Observability-Logging, 06Traffic: Shutdown varnishkafka instances - https://phabricator.wikimedia.org/T393772#10815510 (10JAllemandou) Hey folks, I'm sorry I was OoO last week and missed the timeline. I'm happy for VK to be shut-down when you wish. we don't g... [10:15:22] (03CR) 10Joal: [C:03+2] "LGTM! Merge" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1138675 (https://phabricator.wikimedia.org/T392468) (owner: 10Majavah) [10:19:00] 10Data-Engineering (Q4 2025 April 1st - June 30th), 06cloud-services-team, 10Cloud-VPS, 07IPv6, 13Patch-For-Review: Add new WMCS IP ranges to analytics - https://phabricator.wikimedia.org/T392468#10815574 (10JAllemandou) a:03JAllemandou [10:20:46] 10Data-Engineering (Q4 2025 April 1st - June 30th), 06cloud-services-team, 10Cloud-VPS, 07IPv6, 13Patch-For-Review: Add new WMCS IP ranges to analytics - https://phabricator.wikimedia.org/T392468#10815577 (10JAllemandou) [10:26:54] (03Merged) 10jenkins-bot: refinery-core: iputils: Update Cloud VPS IP space [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1138675 (https://phabricator.wikimedia.org/T392468) (owner: 10Majavah) [10:45:44] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content, 10Dumps-Generation, 13Patch-For-Review: Decommission dumps job `download_enterprise_htmldumps` - https://phabricator.wikimedia.org/T390556#10815692 (10Mitar) These: * https://dumps.wikimedia.org/other/enterprise_html/runs/2025040... [11:05:18] (03PS1) 10Lucas Werkmeister (WMDE): Add -w 1 flag to WikimediaGraphite nc calls [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1145144 [11:26:02] 10Data-Engineering (Q4 2025 April 1st - June 30th), 06Traffic: Clean-up varnishkafka webrequest leftovers in Hadoop-world - https://phabricator.wikimedia.org/T394011#10815806 (10JAllemandou) [11:26:12] 10Data-Engineering (Q4 2025 April 1st - June 30th), 06Traffic: Clean-up varnishkafka webrequest leftovers in Hadoop-world - https://phabricator.wikimedia.org/T394011#10815807 (10JAllemandou) a:05Fabfur→03JAllemandou [11:31:28] (03PS1) 10Joal: Remove deprecated_webrequest HQL code [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1145164 (https://phabricator.wikimedia.org/T394011) [11:34:47] 10Data-Engineering (Q4 2025 April 1st - June 30th), 06Data-Platform-SRE: Remove `analytics` instance folder in airflow repo - https://phabricator.wikimedia.org/T394015 (10JAllemandou) 03NEW [11:34:56] 10Data-Engineering (Q4 2025 April 1st - June 30th), 06Data-Platform-SRE: Remove `analytics` instance folder in airflow repo - https://phabricator.wikimedia.org/T394015#10815832 (10JAllemandou) a:03JAllemandou [11:36:18] 10Data-Engineering (Q4 2025 April 1st - June 30th), 06Data-Platform-SRE: Remove `analytics` instance folder in airflow repo - https://phabricator.wikimedia.org/T394015#10815842 (10JAllemandou) >>! In T391075#10815786, @CodeReviewBot wrote: > joal opened https://gitlab.wikimedia.org/repos/data-engineering/airfl... [11:38:21] (03CR) 10Filippo Giunchedi: [C:03+1] Add -w 1 flag to WikimediaGraphite nc calls [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1145144 (owner: 10Lucas Werkmeister (WMDE)) [11:49:34] (03CR) 10Filippo Giunchedi: [C:03+1] "LMK when this is deployed and I'll go ahead with I2fcf4a767" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1145144 (owner: 10Lucas Werkmeister (WMDE)) [12:14:28] 10Data-Engineering-Roadmap, 06Data-Platform-SRE, 10Dumps-Generation, 10MW-on-K8s, and 4 others: WE 5.4 KR - Hypothesis 5.4.4 - Q3 FY24/25 - Migrate current-generation dumps to run on kubernetes - https://phabricator.wikimedia.org/T352650#10816009 (10BTullis) [14:18:02] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content, 13Patch-For-Review: Implement alerting for wmf_content.mediawiki_content_history_v1 - https://phabricator.wikimedia.org/T384962#10816612 (10xcollazo) Manually killed zombie yarn apps via: ` analytics@an-launcher1002:/home/xcollazo$... [14:24:38] (03CR) 10Xcollazo: [C:03+1] Remove wbt tables from sqoop utility [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1139419 (https://phabricator.wikimedia.org/T391006) (owner: 10Joal) [14:30:12] 06Data-Engineering, 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 10Event-Platform: PageMovedEvent should carry article redirect data - https://phabricator.wikimedia.org/T394049#10816730 (10gmodena) [14:30:32] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 10Event-Platform: PageMovedEvent should carry article redirect data - https://phabricator.wikimedia.org/T394049#10816735 (10gmodena) [14:33:01] 14Analytics-Radar, 06Data-Engineering, 06Data-Engineering-Icebox, 10Cite, and 3 others: Remove Reference Previews tracking metrics - https://phabricator.wikimedia.org/T242127#10816773 (10awight) [14:35:23] (03CR) 10Xcollazo: [C:03+1] Remove deprecated_webrequest HQL code [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1145164 (https://phabricator.wikimedia.org/T394011) (owner: 10Joal) [15:59:03] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content, 10Dumps-Generation, 13Patch-For-Review: Decommission dumps job `download_enterprise_htmldumps` - https://phabricator.wikimedia.org/T390556#10817260 (10brouberol) All done! [15:59:55] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content, 10Dumps-Generation, 13Patch-For-Review: Decommission dumps job `download_enterprise_htmldumps` - https://phabricator.wikimedia.org/T390556#10817266 (10amastilovic) Thank you! [16:00:25] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content, 10Dumps-Generation, 13Patch-For-Review: Decommission dumps job `download_enterprise_htmldumps` - https://phabricator.wikimedia.org/T390556#10817267 (10amastilovic) 05Open→03Resolved [16:27:55] 10Data-Engineering (Q4 2025 April 1st - June 30th), 06Data-Platform-SRE: Provide tooling to instantiate ad-hoc temporary Airflow DEV environments - https://phabricator.wikimedia.org/T393521#10817442 (10Ahoelzl) [16:28:36] 10Data-Engineering (Q4 2025 April 1st - June 30th), 06Data-Platform-SRE: Provide tooling to instantiate ad-hoc temporary Airflow DEV environments - https://phabricator.wikimedia.org/T393521#10817444 (10Ahoelzl) SRE tracking ticket https://phabricator.wikimedia.org/T393998 [16:37:00] 10Data-Engineering (Q4 2025 April 1st - June 30th): Migrate Gobblin to Airflow - https://phabricator.wikimedia.org/T390249#10817488 (10amastilovic) [16:37:16] 10Data-Engineering (Q4 2025 April 1st - June 30th): Migrate Gobblin to Airflow - https://phabricator.wikimedia.org/T390249#10817489 (10amastilovic) 05In progress→03Resolved [16:40:31] 10Data-Engineering (Q4 2025 April 1st - June 30th): Establish what data must be backed up before the Hadoop 3 upgrade - https://phabricator.wikimedia.org/T394071 (10Ahoelzl) 03NEW [16:42:34] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE HAProxy Migration: [HAProxy Migration ]MAY-2025 Delete `webrequest_deprecated` data and DAGs - https://phabricator.wikimedia.org/T391003#10817553 (10Ahoelzl) a:03JAllemandou [16:42:50] 10Data-Engineering (Q4 2025 April 1st - June 30th), 07Essential-Work: 8 new wikis missing from mediawiki_history - https://phabricator.wikimedia.org/T368788#10817555 (10Ahoelzl) a:03mforns [17:11:51] (03PS1) 10Aqu: Tentative to fix refinery-job-lite CI [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1145270 [17:37:28] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content, 10Dumps-Generation, 13Patch-For-Review: Decommission dumps job `download_enterprise_htmldumps` - https://phabricator.wikimedia.org/T390556#10817880 (10xcollazo) @amastilovic do we still need to merge https://gerrit.wikimedia.o... [17:41:02] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 10Event-Platform: PagedMovedEvent should carry reason data - https://phabricator.wikimedia.org/T394046#10817910 (10Ahoelzl) [17:44:40] 06Data-Engineering-Radar, 10Observability-Logging, 06Traffic: Shutdown varnishkafka instances - https://phabricator.wikimedia.org/T393772#10817926 (10Ahoelzl) [17:45:38] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Data-Platform-SRE (2025.05.02 - 2025.05.23): Enable lock transaction management on the Hive Metastore - https://phabricator.wikimedia.org/T386854#10817929 (10xcollazo) [17:47:46] 06Data-Engineering, 06Machine-Learning-Team, 07Essential-Work: Make the revert risk predictions datasets available for analysis - https://phabricator.wikimedia.org/T388453#10817939 (10Ahoelzl) @JAllemandou can you please evaluate the effort? [17:49:36] 06Data-Engineering, 06Data-Platform-SRE, 10Technical-blog-posts: Write a blog post about the recent Airflow migration to Kubernetes - https://phabricator.wikimedia.org/T393603#10817954 (10Ahoelzl) [17:50:48] 06Data-Engineering, 06Data-Platform-SRE, 10Technical-blog-posts: Write a blog post about the recent Airflow migration to Kubernetes - https://phabricator.wikimedia.org/T393603#10817959 (10VirginiaPoundstone) @BTullis great idea to write a post for the tech blog! What support do you need to advance this? [17:53:14] 06Data-Engineering, 10Cassandra, 13Patch-For-Review: Audit and update AQS Cassandra roles & grants - https://phabricator.wikimedia.org/T313877#10817966 (10Ahoelzl) @Eevans should we expedite this? [17:53:57] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Cassandra, 13Patch-For-Review: Audit and update AQS Cassandra roles & grants - https://phabricator.wikimedia.org/T313877#10817969 (10Ahoelzl) [17:53:59] 06Data-Engineering-Radar, 10Observability-Logging, 06Traffic: Shutdown varnishkafka instances - https://phabricator.wikimedia.org/T393772#10817970 (10Fabfur) Sorry the first date has been postponed to tomorrow (May 13th) [17:55:21] 06Data-Engineering, 10Cassandra: Cassandra multi-tenant access configuration - https://phabricator.wikimedia.org/T318407#10817976 (10Ahoelzl) @mforns can you please evaluate? [17:55:35] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Cassandra: Cassandra multi-tenant access configuration - https://phabricator.wikimedia.org/T318407#10817978 (10Ahoelzl) [17:56:21] 14Data-Engineering (Q3 2025 January 1st - March 31th), 10DPE-Mediawiki-Content, 07Essential-Work, 13Patch-For-Review: DAG failing due to failure to acquire lock on wmf_data_ops.data_quality_metrics table - https://phabricator.wikimedia.org/T386114#10817980 (10xcollazo) 05Resolved→03Open This is happeni... [17:57:47] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content, 10Dumps-Generation, 13Patch-For-Review: Decommission dumps job `download_enterprise_htmldumps` - https://phabricator.wikimedia.org/T390556#10817986 (10Mitar) Thank you. [18:10:50] 14Data-Engineering (Q3 2025 January 1st - March 31th), 10DPE-Mediawiki-Content, 07Essential-Work, 13Patch-For-Review: DAG failing due to failure to acquire lock on wmf_data_ops.data_quality_metrics table - https://phabricator.wikimedia.org/T386114#10818016 (10xcollazo) >>! In T386114#10817980, @xcollazo wr... [18:12:30] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 10Event-Platform: PagedMovedEvent should carry reason data - https://phabricator.wikimedia.org/T394046#10818020 (10Ahoelzl) @gmodena I assumed this should be addressed for Q4, so move it. [18:26:53] 14Data-Engineering (Q3 2025 January 1st - March 31th), 10DPE-Mediawiki-Content, 07Essential-Work, 13Patch-For-Review: DAG failing due to failure to acquire lock on wmf_data_ops.data_quality_metrics table - https://phabricator.wikimedia.org/T386114#10818045 (10xcollazo) 05Open→03Resolved [19:33:17] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Data-Platform-SRE (2025.05.02 - 2025.05.23): Enable lock transaction management on the Hive Metastore - https://phabricator.wikimedia.org/T386854#10818249 (10xcollazo) Config looks good: ` xcollazo@an-test-client1002:/etc/hive/conf$ cat hive-site.xml | grep... [19:50:49] Starting build #35 for job analytics-refinery-maven-release [20:09:53] Project analytics-refinery-maven-release build #35: 04STILL FAILING in 19 min: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release/35/ [20:18:54] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 10Data-Platform-SRE (2025.05.02 - 2025.05.23): Requesting access to for  - https://phabricator.wikimedia.org/T393066#10818363 (10SCampos-WMF) Thank you, @BTullis just confirming that everything is work... [20:19:07] (03CR) 10Aqu: [C:03+2] Tentative to fix refinery-job-lite CI [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1145270 (owner: 10Aqu) [20:30:24] (03Merged) 10jenkins-bot: Tentative to fix refinery-job-lite CI [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1145270 (owner: 10Aqu) [20:43:23] Starting build #36 for job analytics-refinery-maven-release [20:50:51] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 10Event-Platform: PageMovedEvent should carry article redirect data - https://phabricator.wikimedia.org/T394049#10818491 (10daniel) I'd suggest adding the following method to PageMovedEvent: `getRedirectPage... [20:51:37] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 10Event-Platform: PagedMovedEvent should carry reason data - https://phabricator.wikimedia.org/T394046#10818493 (10daniel) This can follow the example of PageDeletedEvent. [21:03:32] Yippee, build fixed! [21:03:32] Project analytics-refinery-maven-release build #36: 09FIXED in 20 min: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release/36/ [23:07:37] 06Data-Engineering, 06Data-Engineering-Radar, 06Growth-Team, 10GrowthExperiments, and 5 others: mw.track: support for histogram metrics - https://phabricator.wikimedia.org/T383563#10818898 (10colewhite)