[08:06:43] (03CR) 10Joal: [C:03+1] "LGTM!" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298378 (https://phabricator.wikimedia.org/T428288) (owner: 10Xcollazo) [08:48:53] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 10Event-Platform, 13Patch-For-Review: mediawiki.page_change.v1 events - add wikidata id for pages - https://phabricator.wikimedia.org/T428176#11992603 (10Pablo) Yeah, one would expect Wikidata page links to remain v... [09:21:21] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform, 13Patch-For-Review: Add "wiki_id" to Page View Stream - https://phabricator.wikimedia.org/T427925#11992815 (10JAllemandou) I understand the technical concern about loading API data in the flink app. I don't have a good solution for this... [09:30:59] 06Data-Engineering, 13Patch-For-Review: generate mediawiki_history_reduced spark job failing - 2026-06 - https://phabricator.wikimedia.org/T428242#11992848 (10JAllemandou) Job succeeded with the memory bump - Thanks @otto :) [09:59:40] (03CR) 10Joal: [V:03+2 C:03+2] "Merging for deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1287959 (https://phabricator.wikimedia.org/T425729) (owner: 10Xcollazo) [10:17:16] (03CR) 10Joal: [C:03+2] "Merging for deploy" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1297115 (https://phabricator.wikimedia.org/T266374) (owner: 10Joal) [10:22:57] (03CR) 10A-pizzata: "comment on reusing variable" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298333 (https://phabricator.wikimedia.org/T428288) (owner: 10Xcollazo) [10:25:47] (03PS1) 10Joal: Update changelog.md for v0.3.17 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298724 [10:33:07] (03Merged) 10jenkins-bot: Fix MWH revert algorithm [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1297115 (https://phabricator.wikimedia.org/T266374) (owner: 10Joal) [10:38:33] (03CR) 10A-pizzata: [C:03+1] Fix control_map timestamps: use ISO-8601 UTC format (yyyy-MM-dd'T'HH:mm:ss.SSS'Z') uniformly [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298378 (https://phabricator.wikimedia.org/T428288) (owner: 10Xcollazo) [11:26:49] (03PS2) 10Joal: Update changelog.md for v0.3.17 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298724 [11:27:05] (03CR) 10Joal: [V:03+2 C:03+2] "Merging for deploy" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298724 (owner: 10Joal) [11:33:02] Starting build #63 for job analytics-refinery-maven-release [11:36:07] 06Data-Engineering, 10Data-Platform, 06Data-Platform-SRE, 06Wikimedia Enterprise: Provide auth-less access to Enterprise APIs from WMF Analytics cluster - https://phabricator.wikimedia.org/T403298#11993577 (10awight) Just a data point, the https://enterprise.wikimedia.com/blog/enhanced-free-api/ updated fr... [11:59:55] Project analytics-refinery-maven-release build #63: 09SUCCESS in 26 min: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release/63/ [12:24:11] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE: Provide a scheduled data download service from Google Cloud Storage - https://phabricator.wikimedia.org/T427457#11993782 (10Antoine_Quhen) a:03Antoine_Quhen We've done some research to size the GCS > HDFS download. **1/ Is the load... [12:42:10] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Product-Analytics, 06Product Safety and Integrity (Sprint Iris (May 25 - Jun 12)): Backfill via AirFlow for time_to_revert_bad_faith_edits - https://phabricator.wikimedia.org/T425526#11993855 (10Tchanders) 05Open→03Declined I don't think we nee... [13:22:35] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Create API and User-Agent compliance related tables under wmf_traffic - https://phabricator.wikimedia.org/T427840#11994125 (10JAllemandou) Tables created with ownership to `analytics`. [13:28:03] (03CR) 10Xcollazo: Add Iceberg WAP branching to MWHistoryDeltaWriter and MWHistorySnapshotMerger (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298333 (https://phabricator.wikimedia.org/T428288) (owner: 10Xcollazo) [14:00:13] (03PS5) 10Xcollazo: Add Iceberg WAP branching to MWHistoryDeltaWriter and MWHistorySnapshotMerger [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298333 (https://phabricator.wikimedia.org/T428288) [14:10:27] (03PS2) 10Mforns: [WIP] Implement weekly unique devices metric [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1285415 (owner: 10Milimetric) [14:14:41] (03CR) 10A-pizzata: [C:03+1] "LGTM!" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298333 (https://phabricator.wikimedia.org/T428288) (owner: 10Xcollazo) [14:22:33] (03CR) 10Xcollazo: [C:03+2] Add Iceberg WAP branching to MWHistoryDeltaWriter and MWHistorySnapshotMerger [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298333 (https://phabricator.wikimedia.org/T428288) (owner: 10Xcollazo) [14:23:52] (03CR) 10AKhatun: [C:03+1] Fix control_map timestamps: use ISO-8601 UTC format (yyyy-MM-dd'T'HH:mm:ss.SSS'Z') uniformly [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298378 (https://phabricator.wikimedia.org/T428288) (owner: 10Xcollazo) [14:26:23] (03PS2) 10Xcollazo: Fix control_map timestamps: use ISO-8601 UTC format (yyyy-MM-dd'T'HH:mm:ss.SSS'Z') uniformly [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298378 (https://phabricator.wikimedia.org/T428288) [14:39:48] (03Merged) 10jenkins-bot: Add Iceberg WAP branching to MWHistoryDeltaWriter and MWHistorySnapshotMerger [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298333 (https://phabricator.wikimedia.org/T428288) (owner: 10Xcollazo) [15:05:52] (03CR) 10Xcollazo: [C:03+2] Fix control_map timestamps: use ISO-8601 UTC format (yyyy-MM-dd'T'HH:mm:ss.SSS'Z') uniformly [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298378 (https://phabricator.wikimedia.org/T428288) (owner: 10Xcollazo) [15:21:04] (03Merged) 10jenkins-bot: Fix control_map timestamps: use ISO-8601 UTC format (yyyy-MM-dd'T'HH:mm:ss.SSS'Z') uniformly [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298378 (https://phabricator.wikimedia.org/T428288) (owner: 10Xcollazo) [15:34:59] (03CR) 10Joal: [V:03+2 C:03+2] "Merging for later deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1298321 (https://phabricator.wikimedia.org/T428279) (owner: 10Gerrit maintenance bot) [16:05:15] FIRING: HdfsRpcQueueLength: RPC queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue/latency - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [16:10:15] RESOLVED: HdfsRpcQueueLength: RPC queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue/latency - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [17:16:05] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027, 13Patch-For-Review: DE3.1 - Logged-out reader 21-day retention on web - https://phabricator.wikimedia.org/T424706#11995389 (10Milimetric) [17:19:56] 06Data-Engineering, 10BetaFeatures, 06cloud-services-team, 10Data-Services, and 4 others: Create view for betafeatures_user_counts table in wiki replicas - https://phabricator.wikimedia.org/T402145#11995400 (10SD0001) >>! In https://gerrit.wikimedia.org/r/c/operations/puppet/+/1298329/comments/9648a202_f27... [17:27:21] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 10Event-Platform, 13Patch-For-Review: mediawiki.page_change.v1 - add revision.editor.first_edit_dt field - https://phabricator.wikimedia.org/T425029#11995437 (10tchin) Since this requires bumping the page change sch... [17:43:05] !log Test Kitchen edge-unique experiments (poll 102639) - adds: we-1-8-account-creation-form-v2; removes: none; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [17:43:07] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [18:06:48] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 10Event-Platform: Create mediawiki.user_change event stream - https://phabricator.wikimedia.org/T423952#11995704 (10xcollazo) We have done multiple test runs ingesting data from `event.mediawiki_user_change_dev0`, and... [19:06:37] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Data-Engineering-Wikistats, 10Wikidata: Wikidata unique devices statistics are obviously wrong - https://phabricator.wikimedia.org/T420210#11995920 (10Snwachukwu) a:03Snwachukwu [19:07:03] (03PS3) 10Mforns: [WIP] Implement weekly unique devices metric [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1285415 (owner: 10Milimetric) [19:50:03] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History: Add row_update_dt watermark column to mediawiki_history_incremental_v1 - https://phabricator.wikimedia.org/T428503 (10xcollazo) 03NEW [20:26:16] (03PS1) 10Xcollazo: Add row_update_dt incremental watermark to MWHistoryDeltaWriter [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298899 (https://phabricator.wikimedia.org/T428503) [20:34:23] (03PS1) 10Xcollazo: Add row_update_dt column to mediawiki_history_incremental_v1 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1298901 (https://phabricator.wikimedia.org/T428503) [20:34:45] (03PS2) 10Xcollazo: Add row_update_dt column to mediawiki_history_incremental_v1 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1298901 (https://phabricator.wikimedia.org/T428503) [20:45:31] (03CR) 10CI reject: [V:04-1] Add row_update_dt incremental watermark to MWHistoryDeltaWriter [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298899 (https://phabricator.wikimedia.org/T428503) (owner: 10Xcollazo) [20:54:16] (03CR) 10Xcollazo: "recheck" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1298899 (https://phabricator.wikimedia.org/T428503) (owner: 10Xcollazo) [21:06:53] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE, 06Traffic, 10Wikidata, 10WMDE Analytics: Airflow processes to import dump logs and generate monthly metrics - https://phabricator.wikimedia.org/T403159#11996368 (10Ahoelzl) a:03Ahoelzl [21:07:07] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Ingest JSON access logs for dumps.wikimedia.org in the data lake - https://phabricator.wikimedia.org/T425128#11996370 (10Ahoelzl) p:05Triage→03High a:03Ahoelzl [21:15:18] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Traffic, 13Patch-For-Review: Surge in webrequest validation check - https://phabricator.wikimedia.org/T422030#11996409 (10Ahoelzl) a:05xcollazo→03Ahoelzl [21:22:05] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE (2026-06-05 - 2026-06-26): implement script to move data from P&T data lake to FR Tech data lake - https://phabricator.wikimedia.org/T425133#11996417 (10Ahoelzl) a:05amastilovic→03Antoine_Quhen [21:23:00] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE: Provide a scheduled data download service from Google Cloud Storage - https://phabricator.wikimedia.org/T427457#11996419 (10Ahoelzl) [21:23:01] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Wikimedia Enterprise, 10Wikimedia Enterprise - Content Integrity, 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 07Essential-Work: Implement an Airflow operator for moving data from point A to B - https://phabricator.wikimedia.org/T405360#11996418 (... [21:23:43] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Wikimedia Enterprise, 10Wikimedia Enterprise - Content Integrity, 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 07Essential-Work: Implement an Airflow operator for moving data from point A to B - https://phabricator.wikimedia.org/T405360#11996423 (... [21:35:37] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 07Essential-Work: Inconsistent wiki list: grouped_wikis.csv extended *after* some sqoop jobs have already started - https://phabricator.wikimedia.org/T425385#11996469 (10Ahoelzl) a:05Ahoelzl→03Snwachukwu [21:36:34] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 07Essential-Work: Inconsistent wiki list: grouped_wikis.csv extended *after* some sqoop jobs have already started - https://phabricator.wikimedia.org/T425385#11996471 (10Ahoelzl) @Snwachukwu can you ensure the list update gets scheduled monthly ahead of ti... [21:43:35] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 07Epic: Provide configurable, scheduled data platform data import / export capabilities - https://phabricator.wikimedia.org/T428511 (10Ahoelzl) 03NEW [21:43:57] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 07Epic: Provide configurable, scheduled data platform data import / export capabilities - https://phabricator.wikimedia.org/T428511#11996508 (10Ahoelzl) [21:43:59] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Wikimedia Enterprise, 10Wikimedia Enterprise - Content Integrity, 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 07Essential-Work: Implement an Airflow operator for moving data from point A to B - https://phabricator.wikimedia.org/T405360#11996507 (... [21:44:42] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 07Epic: Provide configurable, scheduled data platform data import / export capabilities - https://phabricator.wikimedia.org/T428511#11996510 (10Ahoelzl) [21:44:43] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Load Google Search Console data into the Data Lake - https://phabricator.wikimedia.org/T420996#11996509 (10Ahoelzl) [21:45:44] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 07Epic: Provide configurable, scheduled data platform data import / export capabilities - https://phabricator.wikimedia.org/T428511#11996514 (10Ahoelzl) [21:46:52] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 07Epic: Provide configurable, scheduled data platform data import / export capabilities - https://phabricator.wikimedia.org/T428511#11996517 (10Ahoelzl) [21:46:55] 06Data-Engineering, 06Data-Platform-SRE, 10FR-Tech-Analytics, 07Epic: Allow egress from airflow workers to fr-tech minio - https://phabricator.wikimedia.org/T428294#11996516 (10Ahoelzl) [21:49:17] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Wikimedia Enterprise: WME Pageviews DAG for HDFS to S3 Transfer - https://phabricator.wikimedia.org/T426017#11996519 (10Ahoelzl) a:05Ahoelzl→03Antoine_Quhen [21:49:19] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 07Epic: Provide configurable, scheduled data platform data import / export capabilities - https://phabricator.wikimedia.org/T428511#11996522 (10Ahoelzl) [21:49:20] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Wikimedia Enterprise: WME Pageviews DAG for HDFS to S3 Transfer - https://phabricator.wikimedia.org/T426017#11996521 (10Ahoelzl) [21:58:35] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting access to for - https://phabricator.wikimedia.org/T427553#11996540 (10RLazarus) Hi @APDube-WMF! I see you provided an SSH key on the task, but if Superset access is all you need, we won't actually need it. I'll set you up wi... [22:29:08] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): WE5.3.3b: Contributor Count Per Page [Attribution API] - https://phabricator.wikimedia.org/T426316#11996655 (10AKhatun_WMF)