[00:23:28] 06Data-Engineering, 06ServiceOps new: Standard helm chart for simple service-utils nodejs apps - https://phabricator.wikimedia.org/T428174#11987292 (10Ottomata) > Do you have a sense of when the new use case ("headless visual editor") might come to exist? They are hoping mid Q1 FY2026-2027 (but I think that m... [01:57:02] (03PS4) 10Xcollazo: Add event_user_is_cross_wiki, page_is_deleted, revision_is_deleted_by_page_deletion, user_central_id [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1297743 (https://phabricator.wikimedia.org/T425730) [08:03:51] 06Data-Engineering, 10EventStreams, 06Data-Platform-SRE (2026-04-24 - 2026-05-15), 07Incident Severity 3, 07Wikimedia-Incident: 502/503 for mediawiki.page_change.v1 stream - https://phabricator.wikimedia.org/T427839#11987684 (10Gehel) [09:27:19] 06Data-Engineering, 10Observability-Logging, 06SRE, 10Wikimedia-Logstash, and 3 others: Produce ECS formatted logstash logs to Event Platform, allowing them to be queried in the WMF Data Lake with SQL - https://phabricator.wikimedia.org/T291645#11987903 (10BTullis) 05Open→03Resolved I think that we... [09:34:15] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform, 13Patch-For-Review: Add "wiki_id" to Page View Stream - https://phabricator.wikimedia.org/T427925#11987921 (10JMonton-WMF) I'm ok with any approach, but I'd like to understand properly the cons, pros and concerns. > Data duplication... [10:27:31] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 13Patch-For-Review: Implement MERGE INTO writers for mediawiki_history_incremental_v1 - https://phabricator.wikimedia.org/T425729#11988061 (10xcollazo) Added four fields to both `MWHistoryDeltaWriter` and `MWHistorySn... [10:27:36] (03PS5) 10Xcollazo: Add event_user_is_cross_wiki, page_is_deleted, revision_is_deleted_by_page_deletion, user_central_id [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1297743 (https://phabricator.wikimedia.org/T425729) [10:33:57] 06Data-Engineering: Remove datahub lineage for mediawiki_history_reduced - https://phabricator.wikimedia.org/T428242 (10JAllemandou) 03NEW [11:07:33] !log Test Kitchen edge-unique experiments (poll 88585) - adds: none; removes: growthexperiments-editattempt-anonwarning; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [11:07:34] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:07:51] (03CR) 10Xcollazo: Add DDL for mediawiki_history_incremental_v1 Iceberg table (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1287959 (https://phabricator.wikimedia.org/T425729) (owner: 10Xcollazo) [11:32:53] (03PS6) 10Xcollazo: Add event_user_is_cross_wiki, page_is_deleted, revision_is_deleted_by_page_deletion, user_central_id [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1297743 (https://phabricator.wikimedia.org/T425729) [12:08:35] 06Data-Engineering: MediaWiki history dumps - wikidatawiki dump has an extra empty column in the second position - https://phabricator.wikimedia.org/T428251 (10Bamyers99) 03NEW [12:19:57] (03CR) 10Joal: [C:03+1] "LGTM!" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1297743 (https://phabricator.wikimedia.org/T425729) (owner: 10Xcollazo) [12:26:24] (03CR) 10A-pizzata: [C:03+1] Add event_user_is_cross_wiki, page_is_deleted, revision_is_deleted_by_page_deletion, user_central_id [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1297743 (https://phabricator.wikimedia.org/T425729) (owner: 10Xcollazo) [12:50:21] 06Data-Engineering, 06Data-Platform-SRE (2026-04-24 - 2026-05-15): Enable Ceph S3 locations for Hive Metastore tables - https://phabricator.wikimedia.org/T425673#11988591 (10BTullis) [12:57:15] 06Data-Engineering, 06Data-Engineering-Icebox: [Spike] Define technology roadmap around Airflow / k8s / ceph - https://phabricator.wikimedia.org/T361509#11988644 (10BTullis) [13:15:15] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop il_to column from imagelinks table in wmf production - https://phabricator.wikimedia.org/T419635#11988745 (10FCeratto-WMF) [13:24:26] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Wikimedia Enterprise, 10Wikimedia Enterprise - Content Integrity, 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 07Essential-Work: Implement an Airflow operator for moving data from point A to B - https://phabricator.wikimedia.org/T405360#11988799 (... [13:25:00] 06Data-Engineering, 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 07Essential-Work: ERROR AsyncEventQueue: Listener DatahubSparkListener threw an exception - https://phabricator.wikimedia.org/T400207#11988815 (10Gehel) [13:25:10] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 06Data-Platform-SRE (2026-06-05 - 2026-06-26): A more recent Spark + Iceberg will make Incremental MediaWiki History much more efficient - https://phabricator.wikimedia.org/T424381#11988817 (10Gehel) [13:26:26] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 07Essential-Work: Superset "track job" button leads to broken URL - https://phabricator.wikimedia.org/T410149#11988847 (10Gehel) [13:26:44] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 07Essential-Work: Move the dumps_v1 DAGs from the Airflow test_k8s instance to the main instance - https://phabricator.wikimedia.org/T404084#11988859 (10Gehel) [13:27:06] 06Data-Engineering, 10BetaFeatures, 06cloud-services-team, 10Data-Services, and 2 others: Create view for betafeatures_user_counts table in wiki replicas - https://phabricator.wikimedia.org/T402145#11988863 (10Gehel) [13:27:49] 06Data-Engineering, 10Dumps-Generation, 10Wikidata, 06Data-Platform-SRE (2026-06-05 - 2026-06-26): Wikidata full .json.gz dumps not published since 20250625 - https://phabricator.wikimedia.org/T412428#11988883 (10Gehel) [13:27:55] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Data Pipelines, 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 07Essential-Work: Airflow dynamic task mapping logs mix up when, on rerun, an id is mapped to a different map_index_template - https://phabricator.wikimedia.org/T408802#11988887 (10Gehel) [13:30:17] 06Data-Engineering, 10Technical-blog-posts, 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 07Essential-Work: Write a blog post about the recent Airflow migration to Kubernetes - https://phabricator.wikimedia.org/T393603#11988923 (10Gehel) [13:30:33] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE (2026-06-05 - 2026-06-26): Task Tries and Logs for Airflow DAGs sometimes unavailable - https://phabricator.wikimedia.org/T419162#11988928 (10Gehel) [13:31:27] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 13Patch-For-Review: Support for Java 21 and Flink 2 - https://phabricator.wikimedia.org/T412978#11988944 (10Gehel) [13:31:33] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE (2026-06-05 - 2026-06-26): implement script to move data from P&T data lake to FR Tech data lake - https://phabricator.wikimedia.org/T425133#11988946 (10Gehel) [13:31:51] 06Data-Engineering, 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 07Epic, 13Patch-For-Review: Upgrade Spark to a version with long term Iceberg support - https://phabricator.wikimedia.org/T338057#11988954 (10Gehel) [13:32:09] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 13Patch-For-Review: Presto cluster improvements for concurrency and workload - https://phabricator.wikimedia.org/T424112#11988962 (10Gehel) [13:32:44] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 07Essential-Work: Do performance testing of a big Hadoop Table hosted by Ceph - https://phabricator.wikimedia.org/T381416#11988974 (10Gehel) [13:33:30] 06Data-Engineering, 10Test Kitchen, 06Data-Platform-SRE (2026-06-05 - 2026-06-26): Airflow instance for Experiment Platform - https://phabricator.wikimedia.org/T416709#11988990 (10Gehel) [13:33:36] 06Data-Engineering, 06Data-Platform-SRE (2026-06-05 - 2026-06-26): Enable Ceph S3 locations for Hive Metastore tables - https://phabricator.wikimedia.org/T425673#11988996 (10Gehel) [13:34:09] 06Data-Engineering, 06Data-Engineering-Radar, 06Privacy Engineering, 06Security-Team, and 2 others: Privacy review of x1 tables in preparation of adding them to wikireplicas - https://phabricator.wikimedia.org/T415219#11989008 (10Gehel) [13:34:31] 06Data-Engineering, 06cloud-services-team, 06Data-Persistence, 10Data-Services, and 3 others: Set up x1 replication to Wiki Replicas - https://phabricator.wikimedia.org/T395881#11989014 (10Gehel) [13:34:41] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE (2026-06-05 - 2026-06-26): Optimize enqueueing of refine_webrequest_hourly pipeline - https://phabricator.wikimedia.org/T419050#11989018 (10Gehel) [13:34:47] 06Data-Engineering, 06Data-Engineering-Radar, 06cloud-services-team, 06Data-Persistence, and 3 others: Create wiki replicas views for globaljsonlinks tables - https://phabricator.wikimedia.org/T387419#11989016 (10Gehel) [13:34:57] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 07Essential-Work: Carry out end-user testing of spark on kubernetes - https://phabricator.wikimedia.org/T412925#11989020 (10Gehel) [13:35:11] 06Data-Engineering, 10Observability-Logging, 06SRE, 10Wikimedia-Logstash, and 3 others: Produce ECS formatted logstash logs to Event Platform, allowing them to be queried in the WMF Data Lake with SQL - https://phabricator.wikimedia.org/T291645#11989026 (10Gehel) [13:42:22] 06Data-Engineering: Remove datahub lineage for mediawiki_history_reduced - https://phabricator.wikimedia.org/T428242#11989073 (10Ottomata) Ah sorry I missed the alert email from Tuesday! FWIW I also see this in the error logs, in case it isn't datahub. ` 26/06/02 22:47:50 WARN TaskSetManager: Lost task 1942.0 i... [14:02:32] (03CR) 10Xcollazo: [C:03+2] Add event_user_is_cross_wiki, page_is_deleted, revision_is_deleted_by_page_deletion, user_central_id [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1297743 (https://phabricator.wikimedia.org/T425729) (owner: 10Xcollazo) [14:04:29] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform, 13Patch-For-Review: Relative Trending - Flink app for page_view - https://phabricator.wikimedia.org/T425624#11989145 (10JMonton-WMF) A test with real data (Kafka Jumbo), with 10 replicas, is working without any issues with the current t... [14:17:10] 06Data-Engineering: Update regular expressions for bot UserAgents - https://phabricator.wikimedia.org/T428267 (10mforns) 03NEW [14:17:15] (03Merged) 10jenkins-bot: Add event_user_is_cross_wiki, page_is_deleted, revision_is_deleted_by_page_deletion, user_central_id [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1297743 (https://phabricator.wikimedia.org/T425729) (owner: 10Xcollazo) [14:26:53] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform, 13Patch-For-Review: Relative Trending - Flink app for page_view - https://phabricator.wikimedia.org/T425624#11989223 (10JMonton-WMF) Starting from "earliest", similar to a situation where we'd need to backfill data, with 10 Task Manager... [14:48:03] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 06Data-Platform-SRE (2026-06-05 - 2026-06-26): A more recent Spark + Iceberg will make Incremental MediaWiki History much more efficient - https://phabricator.wikimedia.org/T424381#11989294 (10xcollazo) 05Open→... [15:09:58] (03PS8) 10Xcollazo: Add DDL for mediawiki_history_incremental_v1 Iceberg table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1287959 (https://phabricator.wikimedia.org/T425729) [15:10:13] (03CR) 10Xcollazo: Add DDL for mediawiki_history_incremental_v1 Iceberg table (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1287959 (https://phabricator.wikimedia.org/T425729) (owner: 10Xcollazo) [15:16:42] 06Data-Engineering, 06ServiceOps new, 10ServiceOps-SharedInfra: Standard helm chart for simple service-utils nodejs apps - https://phabricator.wikimedia.org/T428174#11989471 (10Scott_French) p:05Triage→03Medium Great, thank you both, then! While we won't have time to work on this much at all in what rem... [15:17:01] 06Data-Engineering, 10ServiceOps-SharedInfra, 06ServiceOps new (Next quarter): Standard helm chart for simple service-utils nodejs apps - https://phabricator.wikimedia.org/T428174#11989478 (10Scott_French) [15:18:51] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History: Extend MWHistorySnapshotMerger to reconcile page and user event rows - https://phabricator.wikimedia.org/T427328#11989488 (10xcollazo) 05Open→03Resolved [15:19:09] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History: Spike: drop all 90-day windows in MWHistoryDeltaWriter and replace with full-table revert detection - https://phabricator.wikimedia.org/T427314#11989490 (10xcollazo) 05Open→03Resolved [15:19:28] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 13Patch-For-Review: Implement MERGE INTO writers for mediawiki_history_incremental_v1 - https://phabricator.wikimedia.org/T425729#11989493 (10xcollazo) 05In progress→03Resolved [15:19:39] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Movement-Insights: interwiki imports and its effects on revision data - https://phabricator.wikimedia.org/T425735#11989495 (10xcollazo) 05In progress→03Resolved [15:28:20] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 07Epic, 13Patch-For-Review: Incremental MediaWiki History Phase I - https://phabricator.wikimedia.org/T424350#11989533 (10xcollazo) [15:28:46] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 07Epic, 13Patch-For-Review: Incremental MediaWiki History Phase I - https://phabricator.wikimedia.org/T424350#11989537 (10xcollazo) [15:29:09] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History: Iceberg 1.6.1 bug makes SELECTs fail due to vectorized read path being the default - https://phabricator.wikimedia.org/T426801#11989538 (10xcollazo) [15:29:12] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 07Epic, 13Patch-For-Review: Incremental MediaWiki History Phase I - https://phabricator.wikimedia.org/T424350#11989540 (10xcollazo) [15:29:25] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 07Epic, 13Patch-For-Review: Incremental MediaWiki History Phase I - https://phabricator.wikimedia.org/T424350#11989543 (10xcollazo) [15:29:37] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 07Epic, 13Patch-For-Review: Incremental MediaWiki History Phase I - https://phabricator.wikimedia.org/T424350#11989546 (10xcollazo) [15:47:30] 06Data-Engineering, 10Cassandra: Move commons impact metrics to analytics keyspace - https://phabricator.wikimedia.org/T428276 (10Eevans) 03NEW [15:47:41] 06Data-Engineering, 10Cassandra: Move commons impact metrics to analytics keyspace - https://phabricator.wikimedia.org/T428276#11989621 (10Eevans) p:05Triage→03Low [16:10:46] (03PS1) 10Gerrit maintenance bot: Add mag.wikipedia to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1298321 (https://phabricator.wikimedia.org/T428279) [16:29:06] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History: Use Iceberg branches (i.e WAP) to write atomically - https://phabricator.wikimedia.org/T428288 (10xcollazo) 03NEW [17:28:31] (03PS1) 10Xcollazo: Add Iceberg WAP branching to MWHistoryDeltaWriter and MWHistorySnapshotMerger [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298333 (https://phabricator.wikimedia.org/T428288) [17:47:50] 06Data-Engineering, 06Data-Platform-SRE, 10FR-Tech-Analytics, 07Epic: Allow egress from airflow workers to fr-tech minio - https://phabricator.wikimedia.org/T428294 (10AStein-WMF) 03NEW [17:48:22] 06Data-Engineering, 06Data-Platform-SRE, 10FR-Tech-Analytics, 07Epic: Allow egress from airflow workers to fr-tech minio - https://phabricator.wikimedia.org/T428294#11990033 (10AStein-WMF) a:03BTullis [18:06:43] (03PS2) 10Xcollazo: Add Iceberg WAP branching to MWHistoryDeltaWriter and MWHistorySnapshotMerger [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298333 (https://phabricator.wikimedia.org/T428288) [18:08:13] 06Data-Engineering: generate mediawiki_history_reduced spark job failing - 2026-06 - https://phabricator.wikimedia.org/T428242#11990044 (10Ottomata) [19:12:09] (03PS3) 10Xcollazo: Add Iceberg WAP branching to MWHistoryDeltaWriter and MWHistorySnapshotMerger [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298333 (https://phabricator.wikimedia.org/T428288) [19:24:39] (03PS4) 10Xcollazo: Add Iceberg WAP branching to MWHistoryDeltaWriter and MWHistorySnapshotMerger [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298333 (https://phabricator.wikimedia.org/T428288) [19:42:44] (03PS9) 10Xcollazo: Add DDL for mediawiki_history_incremental_v1 Iceberg table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1287959 (https://phabricator.wikimedia.org/T425729)