[00:42:16] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06SRE Observability: Finalize and/or standardize metrics and logging - https://phabricator.wikimedia.org/T438383#12414410 (10AKhatun_WMF) [08:36:14] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05acl*data-engineering, 06Data-Platform-SRE (2026-10-09 - 2026-10-30), 07Essential-Work, 13Patch-For-Review: Upgrade airflow-main to 3.3.x - https://phabricator.wikimedia.org/T435480#12415116 (10Gehel) [08:36:17] 06Data-Engineering, 06Data-Platform-SRE (2026-10-09 - 2026-10-30): Conda-Analytics environments are prone to dependency conflicts and installation errors - https://phabricator.wikimedia.org/T423067#12415118 (10Gehel) [08:36:31] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Platform, 06Data-Platform-SRE (2026-10-09 - 2026-10-30), 07Epic, 07Essential-Work: [Tech evaluation] Table management options for the S3 data lake - https://phabricator.wikimedia.org/T437472#12415131 (10Gehel) [08:36:44] 06Data-Engineering, 06Data-Engineering-Radar, 06Privacy Engineering, 06Security-Team, and 3 others: Privacy review of x1 tables in preparation of adding them to wikireplicas - https://phabricator.wikimedia.org/T415219#12415135 (10Gehel) [08:36:50] 06Data-Engineering, 06cloud-services-team, 06Data-Persistence, 10Data-Services, and 4 others: Set up x1 replication to Wiki Replicas - https://phabricator.wikimedia.org/T395881#12415133 (10Gehel) [08:37:10] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Wikimedia Enterprise, 10Wikimedia Enterprise - Content Integrity, 06Data-Platform-SRE (2026-10-09 - 2026-10-30), and 2 others: Implement an Airflow operator for moving data from point A to B - https://phabricator.wikimedia.org/T405360#12415139 (10... [08:37:22] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Platform, 06Data-Platform-SRE (2026-10-09 - 2026-10-30): Debmonitor checks failing for docker-registry.discovery.wmnet/repos/data-engineering/eventstreams:v0.19.0 - https://phabricator.wikimedia.org/T432824#12415147 (10Gehel) [08:37:39] 06Data-Engineering, 06Data-Platform-SRE (2026-10-09 - 2026-10-30): Enable Ceph S3 locations for Hive Metastore tables - https://phabricator.wikimedia.org/T425673#12415151 (10Gehel) [08:37:53] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-10-09 - 2026-10-30): Optimize enqueueing of refine_webrequest_hourly pipeline - https://phabricator.wikimedia.org/T419050#12415159 (10Gehel) [08:38:03] 06Data-Engineering, 06Data-Engineering-Radar, 06cloud-services-team, 06Data-Persistence, and 4 others: Create wiki replicas views for globaljsonlinks tables - https://phabricator.wikimedia.org/T387419#12415161 (10Gehel) [08:38:22] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-10-09 - 2026-10-30), 07Essential-Work: [Infra] Carry out end-user testing of spark on kubernetes - https://phabricator.wikimedia.org/T412925#12415165 (10Gehel) [08:40:34] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-10-09 - 2026-10-30), 07Essential-Work: [Tech evaluation] Table catalog options for the S3 data lake - https://phabricator.wikimedia.org/T435604#12415223 (10Gehel) [08:40:40] 06Data-Engineering, 06Data-Platform-SRE (2026-10-09 - 2026-10-30): Enable STS and IAM on the codfw RADOS Gateway and register the OIDC identity providers - https://phabricator.wikimedia.org/T435590#12415225 (10Gehel) [08:41:04] 14Analytics-Radar, 06Data-Engineering, 10Kafka-Infrastructure, 06serviceops-radar, and 5 others: Configuration Management for Kafka settings - https://phabricator.wikimedia.org/T276088#12415231 (10Gehel) [08:43:53] 06Data-Engineering, 06Data-Platform-SRE (2026-10-09 - 2026-10-30), 07Essential-Work: Remove the spark_shuffle_3_3 and spark_shuffle_3_4 YARN aux services - https://phabricator.wikimedia.org/T440229#12415289 (10Gehel) [08:46:35] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE (2026-10-09 - 2026-10-30), 10Event-Platform: Enable kubernetes.operator.savepoint.dispose-on-delete by default - https://phabricator.wikimedia.org/T432455#12415352 (10Gehel) [08:46:41] 06Data-Engineering, 06Data-Platform-SRE (2026-10-09 - 2026-10-30): Task Tries and Logs for Airflow DAGs sometimes unavailable - https://phabricator.wikimedia.org/T419162#12415354 (10Gehel) [08:47:35] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE (2026-10-09 - 2026-10-30), 07Essential-Work: Superset "track job" button leads to broken URL - https://phabricator.wikimedia.org/T410149#12415382 (10Gehel) [08:47:58] 06Data-Engineering, 06Data-Platform-SRE (2026-10-09 - 2026-10-30), 07Essential-Work: ERROR AsyncEventQueue: Listener DatahubSparkListener threw an exception - https://phabricator.wikimedia.org/T400207#12415392 (10Gehel) [08:48:54] 06Data-Engineering, 10BetaFeatures, 06cloud-services-team, 10Data-Services, and 6 others: Create view for betafeatures_user_counts table in wiki replicas - https://phabricator.wikimedia.org/T402145#12415401 (10Gehel) [08:49:21] 06Data-Engineering, 06Data-Platform-SRE (2026-10-09 - 2026-10-30), 07Essential-Work, 13Patch-For-Review: [Infra] Add a presto query logger - https://phabricator.wikimedia.org/T269832#12415414 (10Gehel) [08:55:12] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Define and document the earliest start_date for dbt DAGs - https://phabricator.wikimedia.org/T438276#12415471 (10GGoncalves-WMF) 05Open→03Resolved [09:35:35] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Platform, 06Data-Platform-SRE (2026-10-09 - 2026-10-30), 13Patch-For-Review: Debmonitor checks failing for docker-registry.discovery.wmnet/repos/data-engineering/eventstreams:v0.19.0 - https://phabricator.wikimedia.org/T432824#12415562 (1... [11:14:36] 06Data-Engineering, 06Data-Engineering-Radar, 06ServiceOps, 10ServiceOps-SharedInfra, 13Patch-For-Review: Create namespace for LAC lambdas - https://phabricator.wikimedia.org/T439879#12415865 (10MLechvien-WMF) [11:16:20] 06Data-Engineering, 06Data-Engineering-Radar, 06ServiceOps, 10ServiceOps-SharedInfra: Standard helm chart for simple service-utils nodejs apps - https://phabricator.wikimedia.org/T428174#12415880 (10MLechvien-WMF) →14Duplicate dup:03T436785 [13:40:35] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05acl*data-engineering, 07Essential-Work: Refactor create_dagrun tests for Airflow 3 - https://phabricator.wikimedia.org/T440709 (10atsuko) 03NEW [13:45:47] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05acl*data-engineering, 06Data-Platform-SRE (2026-10-09 - 2026-10-30), 07Essential-Work, 13Patch-For-Review: Upgrade airflow-main to 3.3.x - https://phabricator.wikimedia.org/T435480#12416302 (10atsuko) @Ahoelzl there are lots of tests that need... [13:47:07] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05acl*data-engineering, 06Data-Platform-SRE (2026-10-09 - 2026-10-30), 07Essential-Work, 13Patch-For-Review: Upgrade airflow-main to 3.3.x - https://phabricator.wikimedia.org/T435480#12416307 (10atsuko) [15:28:11] 14Data-Engineering (Q3 2025 January 1st - March 31th), 06Infrastructure-Foundations, 10netops: Update `netflow` retention strategy in Druid (too much data) - https://phabricator.wikimedia.org/T387839#12416665 (10dr0ptp4kt) The replication factor was restored to 2 during a call. In case we hit a situation lik... [15:28:29] 14Data-Engineering (Q3 2025 January 1st - March 31th), 06Infrastructure-Foundations, 10netops: Update `netflow` retention strategy in Druid (too much data) - https://phabricator.wikimedia.org/T387839#12416667 (10dr0ptp4kt) 05Open→03Resolved [15:30:41] 10Data-Engineering-Roadmap, 07Epic: dbt automated initial backfill - https://phabricator.wikimedia.org/T438322#12416676 (10amastilovic) @JAllemandou @GGoncalves-WMF can we further discuss this please, since I believe there are some unresolved issues with this solution. Namely: > Models should run in both inc... [16:09:45] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Set catchup=True as default in dbt scheduler YAML configuration - https://phabricator.wikimedia.org/T438275#12416868 (10amastilovic) [16:31:32] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: User and team Hive database cleanup tracking for Fall 2026 - https://phabricator.wikimedia.org/T437733#12417005 (10xcollazo) >>! In T437733#12406879, @KCVelaga_WMF wrote: > @xcollazo per our discussion, the following staging and int... [16:33:54] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Implement default expiration for staging and intermediate dbt models - https://phabricator.wikimedia.org/T439690#12417019 (10xcollazo) Something we noticed elsewhere and wanted to bring to attention: >>! In T437733#12417005, @xcollazo wrote: > ... > >... [16:41:54] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: User and team Hive database cleanup tracking for Fall 2026 - https://phabricator.wikimedia.org/T437733#12417080 (10xcollazo) [16:54:50] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-API-Platform-Team, 10Event-Platform, 13Patch-For-Review: mediawiki.page_change.v1 - adapt page change kind model to MediaWiki's PageUpdateCauses. - https://phabricator.wikimedia.org/T430588#12417109 (10xcollazo) I have to ask: do we //r... [17:31:22] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733 (10Superb_Owl) 03NEW [17:32:11] (03CR) 10Xcollazo: [C:03+1] "Acknowledged" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1344687 (https://phabricator.wikimedia.org/T297231) (owner: 10Joal) [17:48:31] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12417336 (10Superb_Owl) [17:49:47] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12417338 (10Superb_Owl) [17:51:07] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12417354 (10Superb_Owl) [17:51:36] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12417355 (10Superb_Owl) [18:15:33] 14Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027: DE3.2 - Logged-in Wikipedia reader 2nd week retention on web - https://phabricator.wikimedia.org/T424708#12417510 (10JAnstee_WMF) [18:16:44] 14Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027: DE3.2 - Logged-in Wikipedia reader 2nd week retention on web - https://phabricator.wikimedia.org/T424708#12417511 (10JAnstee_WMF) Status update: Implementation sign-off complete. [18:29:48] 14Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027, 13Patch-For-Review: DE3.1 - Logged-out Wikipedia 21-day retention on mobile web - https://phabricator.wikimedia.org/T424706#12417540 (10JAnstee_WMF) Status update: Implementation sign-off complete. Final implementation request... [18:41:32] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Delay MediaWiki File Exports by 7 days - https://phabricator.wikimedia.org/T440743 (10xcollazo) 03NEW [18:41:41] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Delay MediaWiki File Exports by 7 days - https://phabricator.wikimedia.org/T440743#12417577 (10xcollazo) [18:41:43] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: XML Dumps v1 sunset - https://phabricator.wikimedia.org/T437518#12417578 (10xcollazo) [18:57:49] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Include v2 dumps into mirrors rsync configuration - https://phabricator.wikimedia.org/T437463#12417637 (10xcollazo) **Update: changes merged** - https://gerrit.wikimedia.org/r/1346752 (2026-10-07): the distribution hosts keep the 6... [19:02:50] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Apply Iceberg table maintenance to all dbt marts - https://phabricator.wikimedia.org/T439692#12417647 (10amastilovic) Thanks for the comments, @xcollazo and @JAllemandou ! > For my understanding then, do all dbt tables follow the exact same partitioning... [19:49:08] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12417900 (10Aklapper) [19:51:06] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12417927 (10Aklapper) @Superb_Owl: Thanks for filing this. What do you expect as outcomes from this ticket? (For future reference, please use the Bug Report template fo... [19:53:20] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12417942 (10Aklapper) > How else does a country of 121k people be second only to the U.S. in using English Wikivoyage in August of 2026 What "country of 121k people"?... [19:56:37] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12417982 (10Superb_Owl) [19:57:47] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12417983 (10Superb_Owl) [19:59:03] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12417999 (10Superb_Owl) >>! In T440733#12417942, @Aklapper wrote: >> How else does a country of 121k people be second only to the U.S. in using English Wikivoyage in Au... [20:00:03] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12418001 (10Superb_Owl) [20:04:33] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12418009 (10Superb_Owl) >>! In T440733#12417900, @Aklapper wrote: > @Superb_Owl: Thanks for filing this. > What do you expect as outcomes from this ticket? (For future... [20:05:28] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12418010 (10Superb_Owl) [20:06:58] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12418019 (10Superb_Owl) [20:14:30] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12418095 (10Superb_Owl) [20:21:05] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12418144 (10Superb_Owl) [20:25:45] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12418150 (10Superb_Owl) [20:54:24] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Delay MediaWiki File Exports by 7 days - https://phabricator.wikimedia.org/T440743#12418255 (10xcollazo) **Chosen mechanism** Both export DAGs now use `DelayedMonthlyTimetable` (`wmf_airflow_common/plugins/delayed_timetables.py`),... [20:54:39] 07Analytics-Data-Problem, 10Pageviews-Anomaly: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12418257 (10Superb_Owl) [20:55:47] 07Analytics-Data-Problem, 10Pageviews-Anomaly, 06Wikimedia Enterprise: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12418274 (10Superb_Owl) [20:57:37] 07Analytics-Data-Problem, 10Pageviews-Anomaly, 06Wikimedia Enterprise: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12418277 (10Superb_Owl) [21:44:08] 07Analytics-Data-Problem, 10Pageviews-Anomaly, 06Wikimedia Enterprise: Wikivoyage + Wikinews 'user' spikes after late 2024 - https://phabricator.wikimedia.org/T440733#12418355 (10Superb_Owl) [23:42:01] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Apply Iceberg table maintenance to all dbt marts - https://phabricator.wikimedia.org/T439692#12418574 (10amastilovic) Just thought of something important: > Just a reminder: the config can be global, but the DAG (or DAGs) must be declared on the Airflow...