[00:41:07] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting Kerberos identity for rsilvola - https://phabricator.wikimedia.org/T442695#12493127 (10Dzahn) @Rsilvola Please look out for an email about the creation of your Kerberos principal. It will give you a temp. password and exp... [00:41:14] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting Kerberos identity for rsilvola - https://phabricator.wikimedia.org/T442695#12493129 (10Dzahn) 05Open→03Resolved a:03Dzahn [01:31:35] 06Data-Engineering: Data Warehouse: Build a dbt mart and Superset dashboard for interactive query performance - https://phabricator.wikimedia.org/T442967#12493234 (10Aklapper) [01:31:36] 06Data-Engineering: Data Warehouse: Privacy and retention review for query logs - https://phabricator.wikimedia.org/T442965#12493236 (10Aklapper) [01:31:38] 06Data-Engineering: presto: Add a query-completion event listener that sends events to Event Platform - https://phabricator.wikimedia.org/T442966#12493235 (10Aklapper) [01:31:39] 06Data-Engineering: superset: Enable Superset application metrics (StatsdStatsLogger) - https://phabricator.wikimedia.org/T442964#12493237 (10Aklapper) [01:31:41] 06Data-Engineering: presto: Export per-resource-group metrics to Prometheus - https://phabricator.wikimedia.org/T442961#12493239 (10Aklapper) [01:31:42] 06Data-Engineering: superset: Use separate Presto connections for dashboards and SQL Lab - https://phabricator.wikimedia.org/T442963#12493238 (10Aklapper) [01:31:43] 06Data-Engineering: Data Warehouse: Measure interactive query performance for Superset and Growthbook - https://phabricator.wikimedia.org/T442959#12493240 (10Aklapper) [01:31:46] 06Data-Engineering: superset: Analyze the existing Superset event log and SQL Lab query history - https://phabricator.wikimedia.org/T442960#12493241 (10Aklapper) [01:47:22] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 10Commons-Impact-Metrics, 10Commons-Impact-Metrics-Requests: Update Commons Impact Metrics allow-list for September 2026 - https://phabricator.wikimedia.org/T439050#12493266 (10GFontenelle_WMF) 05Open→03Resolved [02:25:08] 06Data-Engineering, 10Commons-Impact-Metrics, 10Commons-Impact-Metrics-Requests: Update Commons Impact Metrics allow-list for October 2026 - https://phabricator.wikimedia.org/T442978 (10GFontenelle_WMF) 03NEW [02:55:23] 06Data-Engineering, 06Movement-Insights (FY26-27 H1), 13Patch-For-Review: Update Data Lake–to–Postgres data mirroring - https://phabricator.wikimedia.org/T442925#12493326 (10nshahquinn-wmf) a:03nshahquinn-wmf [08:00:36] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)): KAPOW initial backfill - https://phabricator.wikimedia.org/T442233#12493707 (10mforns) [09:13:51] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 13Patch-For-Review: mediawiki.page_change.v1 SLO - Develop the freshness metric - https://phabricator.wikimedia.org/T440968#12494012 (10APizzata-WMF) [09:15:41] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 13Patch-For-Review: mediawiki.page_change.v1 SLO - Develop the freshness metric - https://phabricator.wikimedia.org/T440968#12494019 (10APizzata-WMF) Should we rename the Prometheus metric to a test name so to avoid clouding the values for this test... [10:27:11] 06Data-Engineering, 06Data-Platform-SRE (2026-10-30 - 2026-11-20), 07Essential-Work, 13Patch-For-Review: [Infra] Add a presto query logger - https://phabricator.wikimedia.org/T269832#12494320 (10Gehel) [10:27:26] 06Data-Engineering, 06Data-Platform-SRE (2026-10-30 - 2026-11-20), 07Essential-Work: ERROR AsyncEventQueue: Listener DatahubSparkListener threw an exception - https://phabricator.wikimedia.org/T400207#12494332 (10Gehel) [10:27:32] 06Data-Engineering, 10BetaFeatures, 06cloud-services-team, 10Data-Services, and 6 others: Create view for betafeatures_user_counts table in wiki replicas - https://phabricator.wikimedia.org/T402145#12494326 (10Gehel) [10:27:50] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE (2026-10-30 - 2026-11-20), 07Essential-Work: Superset "track job" button leads to broken URL - https://phabricator.wikimedia.org/T410149#12494340 (10Gehel) [10:28:47] 06Data-Engineering, 06Data-Platform-SRE (2026-10-30 - 2026-11-20): Task Tries and Logs for Airflow DAGs sometimes unavailable - https://phabricator.wikimedia.org/T419162#12494366 (10Gehel) [10:28:53] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE (2026-10-30 - 2026-11-20), 10Event-Platform: Enable kubernetes.operator.savepoint.dispose-on-delete by default - https://phabricator.wikimedia.org/T432455#12494368 (10Gehel) [10:30:24] 06Data-Engineering, 06Data-Platform-SRE (2026-10-30 - 2026-11-20), 07Essential-Work: Remove the spark_shuffle_3_3 and spark_shuffle_3_4 YARN aux services - https://phabricator.wikimedia.org/T440229#12494422 (10Gehel) [10:33:31] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 06Data-Platform-SRE (2026-10-30 - 2026-11-20), 07Essential-Work: [Tech evaluation] Table catalog options for the S3 data lake - https://phabricator.wikimedia.org/T435604#12494476 (10Gehel) [10:34:25] 06Data-Engineering, 06Data-Platform-SRE (2026-10-30 - 2026-11-20): Enable STS and IAM on the codfw RADOS Gateway and register the OIDC identity providers - https://phabricator.wikimedia.org/T435590#12494501 (10Gehel) [10:42:36] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 07Essential-Work: Build an HDFS growth dashboard based on wmf.hdfs_usage - https://phabricator.wikimedia.org/T438056#12494537 (10Antoine_Quhen) P97029 looks good. I've checked the fields from the original fsimage we are not adding to the table yet.... [11:32:54] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 10Data-Platform, 06Data-Platform-SRE (2026-10-30 - 2026-11-20), 07Epic, 07Essential-Work: [Tech evaluation] Table management options for the S3 data lake - https://phabricator.wikimedia.org/T437472#12494683 (10Gehel) [11:33:31] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 06Data-Platform-SRE (2026-10-30 - 2026-11-20), 07Essential-Work, 13Patch-For-Review: [Infra] Carry out end-user testing of spark on kubernetes - https://phabricator.wikimedia.org/T412925#12494699 (10Gehel) [11:33:37] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05acl*data-engineering, 06Data-Platform-SRE (2026-10-30 - 2026-11-20), 07Essential-Work, 13Patch-For-Review: Upgrade airflow-main to 3.3.x - https://phabricator.wikimedia.org/T435480#12494701 (10Gehel) [11:33:55] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 06Wikimedia Enterprise, 10Wikimedia Enterprise - Content Integrity, 06Data-Platform-SRE (2026-10-30 - 2026-11-20), and 2 others: Implement an Airflow operator for moving data from point A to B - https://phabricator.wikimedia.org/T405360#12494703 (... [11:34:13] 06Data-Engineering, 06Data-Engineering-Radar, 06cloud-services-team, 06Data-Persistence, and 4 others: Create wiki replicas views for globaljsonlinks tables - https://phabricator.wikimedia.org/T387419#12494713 (10Gehel) [11:34:23] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-10-30 - 2026-11-20): Optimize enqueueing of refine_webrequest_hourly pipeline - https://phabricator.wikimedia.org/T419050#12494715 (10Gehel) [11:34:53] 06Data-Engineering, 06Data-Engineering-Radar, 06Privacy Engineering, 06Security-Team, and 3 others: Privacy review of x1 tables in preparation of adding them to wikireplicas - https://phabricator.wikimedia.org/T415219#12494727 (10Gehel) [11:35:11] 06Data-Engineering, 06cloud-services-team, 06Data-Persistence, 10Data-Services, and 4 others: Set up x1 replication to Wiki Replicas - https://phabricator.wikimedia.org/T395881#12494729 (10Gehel) [11:35:26] 06Data-Engineering, 06Data-Platform-SRE (2026-10-30 - 2026-11-20): Conda-Analytics environments are prone to dependency conflicts and installation errors - https://phabricator.wikimedia.org/T423067#12494737 (10Gehel) [13:59:30] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 13Patch-For-Review: mediawiki.page_change.v1 SLO - Develop the freshness metric - https://phabricator.wikimedia.org/T440968#12495289 (10Ottomata) Naw, I wouldn't bother. Let's merge! [14:34:44] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting Kerberos identity for rsilvola - https://phabricator.wikimedia.org/T442695#12495400 (10Rsilvola) Done. Thanks! [15:10:20] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 07Essential-Work: Build an HDFS growth dashboard based on wmf.hdfs_usage - https://phabricator.wikimedia.org/T438056#12495634 (10xcollazo) >>! In T438056#12494537, @Antoine_Quhen wrote: > P97029 looks good. > I've checked the fields from the origina... [15:43:49] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)): mediawiki.page_change.v1 SLO - Develop the freshness metric - https://phabricator.wikimedia.org/T440968#12495774 (10APizzata-WMF) Merged and [[ https://gitlab.wikimedia.org/repos/data-engineering/eventgate-wikimedia/-/jobs/1021035 | released ]]. Unfo... [15:53:35] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 10Event-Platform: eventlogging_EditAttemptStep sometimes misses dt field - https://phabricator.wikimedia.org/T442089#12495825 (10AKhatun_WMF) ` spark-sql (default)> SELECT year, month, day, count(*) FROM event.editattemptstep WHERE year = 2026 AND m... [16:23:57] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE: Define intake workflow for WMCS ingress logs - https://phabricator.wikimedia.org/T441140#12495976 (10Volans) >>! In T441140#12451039, @Ottomata wrote: >> By any chance do you have already in mind how a simpler auth could look like?... [16:49:42] FIRING: PageviewStatsBaselineStale: Trending baseline pageview.stats.v1 is stale in eqiad - https://wikitech.wikimedia.org/wiki/MediaWiki_Event_Enrichment/Pageview_Trending_Relative - https://grafana.wikimedia.org/d/relative-trending-pipeline/relative-trending-pipeline - https://alerts.wikimedia.org/?q=alertname%3DPageviewStatsBaselineStale [16:55:26] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to internal data platform for cfoster - https://phabricator.wikimedia.org/T442648#12496181 (10cfoster) @Dzahn bumping this - would love to get started on some data analysis today or early nex... [17:09:32] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Blunderbuss: Have analytics_test artifacts go to analytics_test HDFS cluster - https://phabricator.wikimedia.org/T442190#12496263 (10amastilovic) Task completed [17:20:52] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 10Event-Platform, 13Patch-For-Review: eventlogging_EditAttemptStep sometimes misses dt field - https://phabricator.wikimedia.org/T442089#12496329 (10AKhatun_WMF) Digging done. It looks a race condition causing dt to be not set. Why its specific to... [17:25:25] (03PS1) 10Xcollazo: refinery-job: Remove MediawikiXMLDumpsConverter [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1360951 (https://phabricator.wikimedia.org/T438056) [17:25:27] (03PS1) 10Xcollazo: refinery-job-35: Move HdfsXMLFsImageConverter to Spark 3.5 and Iceberg [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1360952 (https://phabricator.wikimedia.org/T438056) [17:30:52] (03CR) 10Xcollazo: "Note that removing this code makes 1360952 easier to follow since over there we move some library classes to the `refinery-spark-35` packa" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1360951 (https://phabricator.wikimedia.org/T438056) (owner: 10Xcollazo) [17:51:21] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to internal data platform for cfoster - https://phabricator.wikimedia.org/T442648#12496420 (10Dzahn) Hi @cfoster sorry, busy days over here. Here you go! Please keep an eye out for an em... [18:17:23] (03PS1) 10Xcollazo: hdfs: Add wmf_data_ops.hdfs_usage_v2 table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1360963 (https://phabricator.wikimedia.org/T438056) [18:31:59] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Allow variable override of memory config for druid_load_editattemptstep DAG - https://phabricator.wikimedia.org/T442924#12496532 (10amastilovic) Task completed, although the problem it tried to fix was found elsewhere. [19:11:07] 06Data-Engineering: airflow-dags: Replace getpass.getuser() with the Airflow app owner - https://phabricator.wikimedia.org/T443056 (10xcollazo) 03NEW [19:12:06] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): dbt scheduler: Implement support for "backfill scheduling groups" - https://phabricator.wikimedia.org/T442623#12496757 (10amastilovic) [19:25:14] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 07Essential-Work, 13Patch-For-Review: Build an HDFS growth dashboard based on wmf.hdfs_usage - https://phabricator.wikimedia.org/T438056#12496790 (10xcollazo) **Cleanup done (2026-10-29)** - Deleted 203 old XML fsimage files from `/wmf/data/raw/h... [19:28:28] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 07Essential-Work, 13Patch-For-Review: Build an HDFS growth dashboard based on wmf.hdfs_usage - https://phabricator.wikimedia.org/T438056#12496795 (10xcollazo) More one time cleanup: ` xcollazo@an-launcher1003:~$ sudo -u analytics kerberos-run-comm... [19:49:03] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 13Patch-For-Review: User and team Hive database cleanup tracking for Fall 2026 - https://phabricator.wikimedia.org/T437733#12496843 (10xcollazo) 05Open→03Resolved [19:49:29] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 13Patch-For-Review: mediawiki-content-pipelines: Add a Spark test harness - https://phabricator.wikimedia.org/T441886#12496845 (10xcollazo) 05In progress→03Resolved [20:49:42] FIRING: PageviewStatsBaselineStale: Trending baseline pageview.stats.v1 is stale in eqiad - https://wikitech.wikimedia.org/wiki/MediaWiki_Event_Enrichment/Pageview_Trending_Relative - https://grafana.wikimedia.org/d/relative-trending-pipeline/relative-trending-pipeline - https://alerts.wikimedia.org/?q=alertname%3DPageviewStatsBaselineStale [20:56:26] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Movement-Insights (FY26-27 H1), 13Patch-For-Review: dbt_scheduled: Migrate movement-insights and page-trending jobs from legacy DAGs to scheduling groups - https://phabricator.wikimedia.org/T441959#12497151 (10Mayakp.wiki) p:05Triage→03Medium [20:57:02] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Movement-Insights (FY26-27 H1): dbt_scheduled: Migrate movement-insights account-registration jobs from dbt_monthly to scheduling groups - https://phabricator.wikimedia.org/T441957#12497163 (10Mayakp.wiki) p:05Triage→03Medium [20:57:37] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Movement-Insights (FY26-27 H1), 13Patch-For-Review: dbt_scheduled: Migrate movement-insights and page-trending jobs from legacy DAGs to scheduling groups - https://phabricator.wikimedia.org/T441959#12497167 (10Mayakp.wiki) 05Open→03In progress [22:20:39] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 06SRE Observability: Finalize and/or standardize metrics and logging - https://phabricator.wikimedia.org/T438383#12497484 (10AKhatun_WMF) [23:10:16] 06Data-Engineering, 06Movement-Insights (FY26-27 H1), 13Patch-For-Review: dbt_scheduled: Migrate movement-insights and page-trending jobs from legacy DAGs to scheduling groups - https://phabricator.wikimedia.org/T441959#12497682 (10Ahoelzl) [23:10:22] 06Data-Engineering: dbt_scheduled: Migrate product-analytics jobs from dbt_daily to scheduling groups - https://phabricator.wikimedia.org/T441958#12497683 (10Ahoelzl) [23:11:32] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 07OKR-Work (WE1 FY2025-26): [Spike] Adding access_method metadata to moderator action event streams - https://phabricator.wikimedia.org/T419019#12497684 (10Ahoelzl) [23:11:40] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)): Do not set WMF-Last-Access cookie when Sec-Fetch-Dest is not 'document' - https://phabricator.wikimedia.org/T403897#12497685 (10Ahoelzl) [23:13:53] 06Data-Engineering, 06Movement-Insights (FY26-27 H1): dbt_scheduled: Migrate movement-insights account-registration jobs from dbt_monthly to scheduling groups - https://phabricator.wikimedia.org/T441957#12497688 (10Ahoelzl) [23:14:07] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 05acl*data-engineering, 07Essential-Work: Refactor create_dagrun tests for Airflow 3 - https://phabricator.wikimedia.org/T440709#12497690 (10Ahoelzl) [23:14:42] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 10Data-Persistence-Design-Review: [Data Persistence Design Review] Thanks - https://phabricator.wikimedia.org/T440474#12497697 (10Ahoelzl) a:03Snwachukwu [23:16:01] 06Data-Engineering: Device a efficient way to handle wiki removal from sqoop wikis list - https://phabricator.wikimedia.org/T433009#12497701 (10Ahoelzl) a:05Snwachukwu→03None [23:16:32] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)): Fix mediarequest_top_files OOM - https://phabricator.wikimedia.org/T430737#12497703 (10Ahoelzl) [23:16:48] 06Data-Engineering, 06Product Safety and Integrity, 06Product-Analytics (Kanban): Add mediawiki_product_metrics_incident_reporting_system_interaction to the sanitization allowlist - https://phabricator.wikimedia.org/T384650#12497705 (10Ahoelzl) [23:18:46] 06Data-Engineering, 13Patch-For-Review: Improve the quality of Sqoop error logging - https://phabricator.wikimedia.org/T422412#12497711 (10Ahoelzl) a:05amastilovic→03None [23:19:17] 06Data-Engineering: Migrate etl producing wmf_readership.google_sitedata_daily_aggregate to dbt - https://phabricator.wikimedia.org/T439364#12497713 (10Ahoelzl) a:05Antoine_Quhen→03None [23:19:31] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)): Implement default expiration for staging and intermediate dbt models - https://phabricator.wikimedia.org/T439690#12497715 (10Ahoelzl) [23:21:18] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)): Add airflow-dags test to check if all dbt_scheduled YAML dependencies exist in datasets.yaml - https://phabricator.wikimedia.org/T439631#12497717 (10Ahoelzl) [23:21:35] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 13Patch-For-Review: Apply Iceberg table maintenance to all dbt marts - https://phabricator.wikimedia.org/T439692#12497718 (10Ahoelzl) [23:21:58] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 06Movement-Insights (FY26-27 H1): dbt-jobs backfill: constructive_activation_and_retention_monthly - https://phabricator.wikimedia.org/T440251#12497719 (10Ahoelzl) [23:22:29] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 05acl*data-engineering, 06Data-Platform-SRE (2026-10-30 - 2026-11-20), 07Essential-Work, 13Patch-For-Review: Upgrade airflow-main to 3.3.x - https://phabricator.wikimedia.org/T435480#12497720 (10Ahoelzl) [23:22:34] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 10DPE-MediaWiki-Incremental-History: Incremental MediaWiki History v2 - data requests - https://phabricator.wikimedia.org/T437051#12497724 (10Ahoelzl) [23:50:33] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 13Patch-For-Review: Develop the event.mediawiki_page_change_v1 completeness metric - https://phabricator.wikimedia.org/T440963#12497784 (10Ahoelzl) [23:59:50] 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)): mediawiki.page_change.v1 SLO - Develop the freshness metric - https://phabricator.wikimedia.org/T440968#12497794 (10Ottomata) Hm, I'm not sure about auth to the site itself, but I just added you to the deployment-prep (beta) project. Try again maybe?