[01:14:06] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [01:19:06] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [03:06:15] 06Data-Engineering, 06Data-Platform-SRE, 07Epic, 13Patch-For-Review: Upgrade Spark to a version with long term Iceberg support - https://phabricator.wikimedia.org/T338057#12068648 (10nshahquinn-wmf) @BTullis thanks for the update! What's the plan for upgrading the other packages in the environment and maki... [06:29:18] 06Data-Engineering: Setup and populate initial version of user_agents_info table - https://phabricator.wikimedia.org/T430020#12068842 (10KCVelaga_WMF) [07:22:42] (03PS1) 10KCVelaga: DDL for wmf_traffic.user_agents_info table. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1306491 (https://phabricator.wikimedia.org/T430020) [07:43:30] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to "analytics-privatedata" for mona_thierse - https://phabricator.wikimedia.org/T430304#12068956 (10fgiunchedi) @Milimetric @Ahoelzl @Ottomata I'm seeking `analytics-privatedata-users` approval for Mona, an former WMDE... [07:46:35] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Document editor counts table and APIs - https://phabricator.wikimedia.org/T429863#12068974 (10GGoncalves-WMF) Right, I'm totally fine with translating our designs to Wikitech once completed and making that the source of truth. I only prefer Google docs for g... [08:18:40] 06Data-Engineering, 10Event-Platform: [EventGate] Add configurable UA denylist - https://phabricator.wikimedia.org/T429898#12069151 (10GGoncalves-WMF) If the goal is to reject events from self-declared bots, it's worth also keeping an eye on T430020, which is attempting to provide one canonical table for those... [08:28:45] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 13Patch-For-Review: Right-size Spark resource config using History Server data - https://phabricator.wikimedia.org/T428966#12069193 (10APizzata-WMF) Here are the results after the addition of the `page_id` in the orde... [08:33:50] 06Data-Engineering: dbt standardized workflow and repository structure - https://phabricator.wikimedia.org/T430622 (10GGoncalves-WMF) 03NEW [08:34:08] 06Data-Engineering: dbt standardized workflow and repository structure - https://phabricator.wikimedia.org/T430622#12069246 (10GGoncalves-WMF) [08:34:12] 10Data-Engineering-Roadmap, 07Epic, 07OKR-Work (WE1 FY2025-26): dbt DPE work - https://phabricator.wikimedia.org/T416679#12069247 (10GGoncalves-WMF) [08:39:40] 06Data-Engineering: dbt solated model orchestration - https://phabricator.wikimedia.org/T430625 (10GGoncalves-WMF) 03NEW [08:39:54] 06Data-Engineering: dbt isolated model orchestration - https://phabricator.wikimedia.org/T430625#12069297 (10GGoncalves-WMF) [08:40:11] 06Data-Engineering: dbt isolated model orchestration - https://phabricator.wikimedia.org/T430625#12069299 (10GGoncalves-WMF) [08:40:13] 10Data-Engineering-Roadmap, 07Epic, 07OKR-Work (WE1 FY2025-26): dbt DPE work - https://phabricator.wikimedia.org/T416679#12069300 (10GGoncalves-WMF) [09:09:44] 06Data-Engineering: dbt isolated model orchestration - https://phabricator.wikimedia.org/T430625#12069380 (10amastilovic) [09:09:48] 06Data-Engineering, 13Patch-For-Review: dbt: per-group DAG generation + a dedicated dbt Airflow instance - https://phabricator.wikimedia.org/T429439#12069379 (10amastilovic) [09:17:08] 06Data-Engineering, 10Test Kitchen, 10Event-Platform: [EventGate] Reject events from inactive instruments and experiments - https://phabricator.wikimedia.org/T430541#12069480 (10phuedx) > Need to decide whether to reject silently or emit an error event. My initial impression is that this behaviour should be... [09:27:58] 14Analytics-Radar, 06Data-Engineering, 06Data-Platform-SRE, 10Kafka-Infrastructure, and 3 others: Configuration Management for Kafka settings - https://phabricator.wikimedia.org/T276088#12069508 (10elukey) 05Stalled→03In progress p:05Low→03Medium a:03RKemper [09:41:12] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027: Define process and owners for manual QA of experiment data - https://phabricator.wikimedia.org/T430634 (10Miriam) 03NEW [09:41:57] 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 10Test Kitchen, 07Essential-Work, 13Patch-For-Review: Remove mw.eventLog.id - https://phabricator.wikimedia.org/T408179#12069628 (10phuedx) After a little investigation I discovered that the reference to `mw.eventLog.id.getSessionId()` in the Wikis... [09:44:11] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027: Define process and owners for manual QA of experiment data - https://phabricator.wikimedia.org/T430634#12069656 (10Miriam) [09:47:09] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027: Revisiting DE 3.1 and 3.3 web metric targets at the end of Q1 - https://phabricator.wikimedia.org/T430636 (10Miriam) 03NEW [09:47:42] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027: Revisiting DE 3.1 and 3.2 web metric targets at the end of Q1 - https://phabricator.wikimedia.org/T430636#12069678 (10Miriam) [09:48:04] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027: Revisiting DE 3.1 and 3.2 web metric targets at the end of Q1 - https://phabricator.wikimedia.org/T430636#12069681 (10Miriam) [09:48:05] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027: DE3.2 - Logged-in Wikipedia reader 2nd week retention on web - https://phabricator.wikimedia.org/T424708#12069682 (10Miriam) [10:21:25] 06Data-Engineering, 13Patch-For-Review: Setup and populate initial version of user_agents_info table - https://phabricator.wikimedia.org/T430020#12069846 (10KCVelaga_WMF) I just had a chat with @Pablo - reviewing the Cloudflare's taxonomy. There are additional pieces of information in the taxonomy JSON that... [10:29:59] (03PS2) 10A-pizzata: Distribute and sort snapshot rows on write [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1305674 (https://phabricator.wikimedia.org/T428966) [10:30:40] (03CR) 10A-pizzata: Distribute and sort snapshot rows on write (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1305674 (https://phabricator.wikimedia.org/T428966) (owner: 10A-pizzata) [10:45:52] (03CR) 10Joal: [C:03+2] "LGTM! Merging" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1305674 (https://phabricator.wikimedia.org/T428966) (owner: 10A-pizzata) [11:00:04] (03Merged) 10jenkins-bot: Distribute and sort snapshot rows on write [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1305674 (https://phabricator.wikimedia.org/T428966) (owner: 10A-pizzata) [11:00:06] !log Test Kitchen experiment (poll 93446) - adds: none; removes: none; fields: cite-footnote-content-interaction-experiment - TK tips at https://w.wiki/_cvdP [11:00:09] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:06:09] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to "analytics-privatedata" for mona_thierse - https://phabricator.wikimedia.org/T430304#12070030 (10Milimetric) Approved (NOTE: I initially thought there was no NDA on file, but see from the comments it's now available,... [12:03:32] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to "analytics-privatedata" for mona_thierse - https://phabricator.wikimedia.org/T430304#12070229 (10fgiunchedi) >>! In T430304#12070030, @Milimetric wrote: > Approved (NOTE: I initially thought there was no NDA on file,... [12:05:05] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to "analytics-privatedata" for mona_thierse - https://phabricator.wikimedia.org/T430304#12070243 (10fgiunchedi) [12:16:40] (03PS1) 10A-pizzata: Update changelog.md to release v0.3.21 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1306676 [12:42:55] 06Data-Engineering, 13Patch-For-Review: dbt: per-group DAG generation + a dedicated dbt Airflow instance - https://phabricator.wikimedia.org/T429439#12070405 (10GGoncalves-WMF) Thanks! A couple of questions and comments: > Only three fixed cadences. There is no way to express schedules like "every Tuesday" or... [12:56:36] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Dbt backfill: editor_month and base_account_registration - https://phabricator.wikimedia.org/T430602#12070457 (10amastilovic) [12:58:34] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Dbt backfill: editor_month and base_account_registration - https://phabricator.wikimedia.org/T430602#12070474 (10amastilovic) I've backfilled (one run) `base_account_registration model` and am in the process of backfilling editor_month from 2001-01 (currentl... [13:08:30] 06Data-Engineering, 13Patch-For-Review: Setup and populate initial version of user_agents_info table - https://phabricator.wikimedia.org/T430020#12070515 (10GGoncalves-WMF) That would mean we'd have to regularly import the JSON to be able to exact-match individual UAs against it, rather than extracting informa... [13:21:03] 06Data-Engineering, 13Patch-For-Review: Setup and populate initial version of user_agents_info table - https://phabricator.wikimedia.org/T430020#12070559 (10KCVelaga_WMF) >>! In T430020#12070515, @GGoncalves-WMF wrote: > That would mean we'd have to regularly import the JSON to be able to exact-match individua... [13:23:30] (03PS1) 10KCVelaga: DDL for wmf_traffic.user_agents_info table. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1306491 (https://phabricator.wikimedia.org/T430020) [13:26:09] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History: Right-size Spark resource config using History Server data - https://phabricator.wikimedia.org/T428966#12070596 (10APizzata-WMF) Decision taken from this spike: - merged the change [[https://gerrit.wikimedia.org/r/c/an... [13:39:24] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to "analytics-privatedata" for mona_thierse - https://phabricator.wikimedia.org/T430304#12070658 (10Monrac5) >>! In T430304#12068953, @fgiunchedi wrote: > Thank you all! > > @Monrac5 we'd need to verify your ssh public... [13:42:25] 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 10Test Kitchen, 07Essential-Work, 05MW-1.47-notes (1.47.0-wmf.10; 2026-07-07): Remove mw.eventLog.id - https://phabricator.wikimedia.org/T408179#12070679 (10phuedx) [13:42:57] 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 10Test Kitchen, 07Essential-Work, 05MW-1.47-notes (1.47.0-wmf.10; 2026-07-07): Remove mw.eventLog.id - https://phabricator.wikimedia.org/T408179#12070687 (10phuedx) [14:30:10] !log Test Kitchen experiment (poll 94698) - adds: none; removes: we-1-8-account-creation-form-v2; fields: none - TK tips at https://w.wiki/_cvdP [14:30:13] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:48:36] (03CR) 10Joal: [C:03+2] "LGTM!" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1306676 (owner: 10A-pizzata) [15:02:30] (03Merged) 10jenkins-bot: Update changelog.md to release v0.3.21 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1306676 (owner: 10A-pizzata) [15:14:01] Starting build #68 for job analytics-refinery-maven-release [15:42:02] Project analytics-refinery-maven-release build #68: 09SUCCESS in 28 min: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release/68/ [16:16:04] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05FY2025-26 KR 5.1, 06MediaWiki-Platform-Team (Kanban Board), 07OKR-Work, 13Patch-For-Review: redioscope: periodically publish top clients to the data lake - https://phabricator.wikimedia.org/T424823#12071910 (10Ahoelzl) [16:21:22] 06Data-Engineering: please create wmf_frtech database - https://phabricator.wikimedia.org/T430694#12071985 (10JAllemandou) The table got created with name `wmf_fr_tech` to mimic the usersname `analytics-fr-tech`. The base folder for data `/wmf/data/wmf_fr_tech` has also been created. Both the base hive folder fo... [16:21:37] 06Data-Engineering: please create wmf_frtech database - https://phabricator.wikimedia.org/T430694#12071989 (10JAllemandou) 05Open→03Resolved a:03JAllemandou [16:26:04] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform, 13Patch-For-Review: Schema and Stream for "webrequest.page_view" - https://phabricator.wikimedia.org/T426091#12072015 (10Ahoelzl) 05Open→03Resolved [16:26:09] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027: DE3.1 - Logged-out Wikipedia 21-day retention on mobile web - https://phabricator.wikimedia.org/T424706#12072021 (10Ahoelzl) 05Open→03Resolved [16:26:10] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027: DE3.2 - Logged-in Wikipedia reader 2nd week retention on web - https://phabricator.wikimedia.org/T424708#12072020 (10Ahoelzl) 05Open→03Resolved [16:26:13] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027: Heartbeat - Active UWERs - https://phabricator.wikimedia.org/T426420#12072022 (10Ahoelzl) 05Open→03Resolved [16:26:15] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): WE5.3.3b: Contributor Count Per Page [Attribution API] - https://phabricator.wikimedia.org/T426316#12072025 (10Ahoelzl) 05Open→03Resolved [16:26:16] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Document editor counts table and APIs - https://phabricator.wikimedia.org/T429863#12072023 (10Ahoelzl) 05Open→03Resolved [16:26:18] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Test Kitchen, 07Essential-Work: Implement/enforce 90 day data retention policy in derived Iceberg tables - https://phabricator.wikimedia.org/T429548#12072026 (10Ahoelzl) 05Open→03Resolved [16:26:19] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Add Cassandra loading capability in dbt dags - https://phabricator.wikimedia.org/T429862#12072027 (10Ahoelzl) 05Open→03Resolved [16:26:28] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Enable Airflow DAG trigger config dialog by default - https://phabricator.wikimedia.org/T428872#12072035 (10Ahoelzl) 05Open→03Resolved [16:26:35] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform, 13Patch-For-Review: Relative Trending - Milestone 3 - Stream & Schema - https://phabricator.wikimedia.org/T429588#12072036 (10Ahoelzl) 05Open→03Resolved [16:26:37] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform: mediawiki.page_html_content_change.v1 stream content_uri field uses localhost instead of wiki hostname - https://phabricator.wikimedia.org/T427598#12072038 (10Ahoelzl) 05Open→03Resolved [16:26:39] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 13Patch-For-Review: Ingest wmf_mediawiki tables to datahub - https://phabricator.wikimedia.org/T429931#12072039 (10Ahoelzl) 05Open→03Resolved [16:26:48] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Fix inconsistent use of wmf_log in dbt-jobs - https://phabricator.wikimedia.org/T429771#12072042 (10Ahoelzl) 05Open→03Resolved [16:26:54] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform: HTML Pipeline - Issue with max size messages - https://phabricator.wikimedia.org/T425336#12072050 (10Ahoelzl) 05Open→03Resolved [16:26:56] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 10Event-Platform: Event schemas - mediawiki user entity should be wiki aware - https://phabricator.wikimedia.org/T426198#12072048 (10Ahoelzl) 05Open→03Resolved [16:27:05] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform: Relative Trending - Milestone 2 - Base line metrics - https://phabricator.wikimedia.org/T428721#12072052 (10Ahoelzl) 05Open→03Resolved [16:27:06] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform: Relative Trending - Milestone 2 - Build baseline table - https://phabricator.wikimedia.org/T428724#12072051 (10Ahoelzl) 05Open→03Resolved [16:27:16] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History: Add row_update_dt watermark column to mediawiki_history_incremental_v1 - https://phabricator.wikimedia.org/T428503#12072064 (10Ahoelzl) 05Open→03Resolved [16:27:23] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform, 13Patch-For-Review: Relative Trending - Milestone 2 - Load baseline into Kafka - https://phabricator.wikimedia.org/T428725#12072060 (10Ahoelzl) 05Open→03Resolved [16:27:27] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-Mediawiki-Content, 13Patch-For-Review: Missing/inconsistent page_redirect_target field for redirects in Mediawiki content current v1 dumps - https://phabricator.wikimedia.org/T400632#12072061 (10Ahoelzl) 05Open→03Resolved [16:27:35] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 07Epic, 13Patch-For-Review: Bug Fix: generate page-create events and make revisions event type `create` only - https://phabricator.wikimedia.org/T429570#12072066 (10Ahoelzl) 05Open→03Resolved [16:27:41] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-Mediawiki-Content: mw_content_reconcile_mw_content_history_monthly failed on rerun - https://phabricator.wikimedia.org/T428999#12072069 (10Ahoelzl) 05Open→03Resolved [16:27:45] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-Mediawiki-Content: fix CI error for mediawiki-content-pipelines - https://phabricator.wikimedia.org/T429574#12072071 (10Ahoelzl) 05Open→03Resolved [16:27:57] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History: Bug fix: populate event_user_groups_historical for revisions and pages - https://phabricator.wikimedia.org/T428928#12072068 (10Ahoelzl) 05Open→03Resolved [16:28:01] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 13Patch-For-Review: generate mediawiki_history_reduced spark job failing - 2026-06 - https://phabricator.wikimedia.org/T428242#12072072 (10Ahoelzl) 05Open→03Resolved [16:28:05] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Display DbtSkeinOperator Skein resource config in task notes - https://phabricator.wikimedia.org/T428889#12072078 (10Ahoelzl) 05Open→03Resolved [16:28:09] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform, 13Patch-For-Review: Streaming HTML & Edit Types - productionization checklist - https://phabricator.wikimedia.org/T423920#12072075 (10Ahoelzl) 05Open→03Resolved [16:28:17] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027: Health - Unique devices - https://phabricator.wikimedia.org/T424750#12072088 (10Ahoelzl) 05Open→03Resolved [16:28:21] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform: Relative Trending - Milestone 2 - Create schema - https://phabricator.wikimedia.org/T428723#12072089 (10Ahoelzl) 05Open→03Resolved [16:28:25] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Dumps-Generation: Data missing from en.wiktionary.org February 2026 "MediaWiki Content File Exports" compared to "XML Database dump" - https://phabricator.wikimedia.org/T417596#12072090 (10Ahoelzl) 05Open→03Resolved [16:28:29] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Blunderbuss doesn't replace the whole destination folder in HDFS - https://phabricator.wikimedia.org/T423573#12072092 (10Ahoelzl) 05Open→03Resolved [16:28:33] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Consolidate dbt-jobs sqlfluff versions in GitLab CI/CD and Conda package on stat machines - https://phabricator.wikimedia.org/T427693#12072093 (10Ahoelzl) 05Open→03Resolved [16:28:38] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06SRE, 10Event-Platform: Flink Page View: Create K8s resources - https://phabricator.wikimedia.org/T426425#12072096 (10Ahoelzl) 05Open→03Resolved [16:28:44] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE (2026-06-05 - 2026-06-26), 13Patch-For-Review: Support for Java 21 and Flink 2 - https://phabricator.wikimedia.org/T412978#12072094 (10Ahoelzl) 05In progress→03Resolved [16:28:50] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): FY-2026-2027 metrics dbt jobs failure due to exit code 143. - https://phabricator.wikimedia.org/T427222#12072098 (10Ahoelzl) 05Open→03Resolved [16:28:54] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform: Schema and Stream for "webrequest_frontend_text" - https://phabricator.wikimedia.org/T426092#12072101 (10Ahoelzl) 05Open→03Resolved [16:28:58] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 10Event-Platform, 13Patch-For-Review: EventBus - consider schema versions when serializing entities - https://phabricator.wikimedia.org/T424767#12072099 (10Ahoelzl) 05Open→03Resolved [16:29:02] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform: Relative Trending - Design document - https://phabricator.wikimedia.org/T425421#12072103 (10Ahoelzl) 05Open→03Resolved [16:29:06] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform: Flink page_view: Create docker build - https://phabricator.wikimedia.org/T426419#12072107 (10Ahoelzl) 05Open→03Resolved [16:29:10] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10AQS2.0: AQS pageviews/v3/top_pages_per_editor not returning data - https://phabricator.wikimedia.org/T426426#12072109 (10Ahoelzl) 05Open→03Resolved [16:29:14] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Fix FY-2026-2027 dbt dags failure - https://phabricator.wikimedia.org/T427616#12072111 (10Ahoelzl) 05Open→03Resolved [16:29:18] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Fix mediarequest_top_files Dag Failures - https://phabricator.wikimedia.org/T426983#12072110 (10Ahoelzl) 05Open→03Resolved [16:30:41] 06Data-Engineering, 10Test Kitchen, 10Event-Platform: [EventGate] Record metrics for instruments and experiments - https://phabricator.wikimedia.org/T430322#12072119 (10JVanderhoop-WMF) This would enable TK users to self-serve in answering the question "are events coming in for my experiment/instrument?" [16:37:59] 06Data-Engineering, 06Data-Platform-SRE: Enable Superset feature flag for Alerts - https://phabricator.wikimedia.org/T430682#12072172 (10GGoncalves-WMF) [16:40:59] 06Data-Engineering, 10Test Kitchen, 10Event-Platform: [EventGate] Reject events from inactive instruments and experiments - https://phabricator.wikimedia.org/T430541#12072192 (10JVanderhoop-WMF) p:05Triage→03Medium [16:42:29] 06Data-Engineering, 10Test Kitchen, 10Event-Platform: [EventGate] Reject events from inactive instruments and experiments - https://phabricator.wikimedia.org/T430541#12072214 (10JVanderhoop-WMF) If we are concerned about data governance, we could also make it a requirement that the instrument be in Test Kitc... [16:46:37] 06Data-Engineering, 10Event-Platform: [EventGate] Add configurable UA denylist - https://phabricator.wikimedia.org/T429898#12072232 (10JVanderhoop-WMF) We may not want to blanket deny bots, but this one in particular is really impacting our SLO, which does suggest a targeted approach could work well. For bot... [16:47:07] 06Data-Engineering, 10Test Kitchen, 10Event-Platform: [EventGate] Add configurable UA denylist - https://phabricator.wikimedia.org/T429898#12072233 (10JVanderhoop-WMF) [16:47:35] 06Data-Engineering, 10Test Kitchen, 10Event-Platform: [EventGate] Add configurable UA denylist - https://phabricator.wikimedia.org/T429898#12072240 (10JVanderhoop-WMF) p:05Triage→03High [16:49:06] 06Data-Engineering, 10Test Kitchen, 10Event-Platform: [EventGate] Add configurable UA denylist - https://phabricator.wikimedia.org/T429898#12072251 (10JVanderhoop-WMF) @Ahoelzl @tchin -- this is impacting our SLO so we will tackle this and submit a patch for your review. [17:13:27] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to "analytics-privatedata" for mona_thierse - https://phabricator.wikimedia.org/T430304#12072406 (10KFrancis) Hi all, the NDA is complete. Thanks! [17:28:55] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform: mediawiki.page_change.v1 event - Add revision revert details - https://phabricator.wikimedia.org/T423583#12072527 (10Ottomata) Schema is merged, eventgates restarted. So far so good. Since we bumped schema versions for {T421237}, there... [17:56:51] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform: mediawiki.page_change.v1 event - Add revision revert details - https://phabricator.wikimedia.org/T423583#12072658 (10Ottomata) @tchin and I ran the following ALTERs for these tables: `lang=sql --- feature counts change alters ALTER... [18:05:13] 06Data-Engineering, 10ChangeProp, 10EventStreams, 06MediaWiki-Engineering, and 15 others: Migrate node-based services in production to node22 - https://phabricator.wikimedia.org/T393434#12072725 (10Sfaci) [18:06:05] 06Data-Engineering, 10ChangeProp, 10EventStreams, 06MediaWiki-Engineering, and 13 others: Migrate node-based services in production to node22 - https://phabricator.wikimedia.org/T393434#12072742 (10Sfaci) [18:54:06] 06Data-Engineering, 06Data-Platform-SRE, 06Java-Scala-Standardization, 07Essential-Work, 13Patch-For-Review: Migrate existing Java packages to deploying to Gitlab, including new version of parent pom, validation that all dependencies are available, and v... - https://phabricator.wikimedia.org/T367405#12072983 [19:06:02] 06Data-Engineering, 06Data-Platform-SRE, 06Java-Scala-Standardization, 07Essential-Work, 13Patch-For-Review: Migrate existing Java packages to deploying to Gitlab, including new version of parent pom, validation that all dependencies are available, and v... - https://phabricator.wikimedia.org/T367405#12073033 [19:31:56] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 10Event-Platform, 13Patch-For-Review: mediawiki.page_change.v1 event - Add user first_registration_dt field - https://phabricator.wikimedia.org/T426998#12073229 (10Ottomata) Or, perhaps instead, we can set a more ex... [19:32:57] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 10Event-Platform, 13Patch-For-Review: mediawiki.page_change.v1 event - Add user first_registration_dt field - https://phabricator.wikimedia.org/T426998#12073238 (10Ottomata) > That's when the user's central/global a... [19:34:45] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 10Event-Platform, 13Patch-For-Review: mediawiki.page_change.v1 event - Add user first_registration_dt field - https://phabricator.wikimedia.org/T426998#12073248 (10Ottomata) 05Open→03Declined Being bold and... [19:37:35] 06Data-Engineering, 06MW-Interfaces-Team, 10Event-Platform: mediawiki.page_change.v1 - adapt page change kind model to MediaWiki's PageUpdateCauses. - https://phabricator.wikimedia.org/T430588#12073276 (10Ottomata) We should probably do this along with {T409464}, as the modeling choices may influence each ot... [19:39:06] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 10Event-Platform: Incremental MWH - MediaWiki event data source improvements - https://phabricator.wikimedia.org/T423935#12073280 (10Ottomata) [19:42:46] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 10Event-Platform: Incremental MWH - MediaWiki event data source improvements - https://phabricator.wikimedia.org/T423935#12073285 (10Ottomata) [19:59:36] 06Data-Engineering: Fix mediarequest_top_files OOM - https://phabricator.wikimedia.org/T430737 (10AKhatun_WMF) 03NEW [20:08:37] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 10Event-Platform: EventBus - user entity schema should differentiate between explicit and implicit user groups - https://phabricator.wikimedia.org/T425360#12073421 (10Ottomata) 05Open→03Declined Declining this... [21:05:27] 06Data-Engineering: sanitization re-run request: event_sanitized.mediawiki_page_html_feature_counts_change_v1 - https://phabricator.wikimedia.org/T430752 (10CMyrick-WMF) 03NEW [21:24:42] 06Data-Engineering: Fix mediarequest_top_files OOM - https://phabricator.wikimedia.org/T430737#12073839 (10AKhatun_WMF) The skew comes entirely from GROUPING SETS in step 1, combined with the PARTITION BY in step 2. GROUPING SETS means "compute this aggregation at several levels of detail at once, then stack al... [22:00:13] 06Data-Engineering: sanitization re-run request: event_sanitized.mediawiki_page_html_feature_counts_change_v1 - https://phabricator.wikimedia.org/T430752#12073889 (10AKhatun_WMF) The sanitization code was deployed around ~18th May. So it may not have data for the period before that, but the way sanitization work... [22:08:13] 06Data-Engineering: sanitization re-run request: event_sanitized.mediawiki_page_html_feature_counts_change_v1 - https://phabricator.wikimedia.org/T430752#12073931 (10AKhatun_WMF) More precisely, the delayed job runs daily at 05:00 and processes a fixed 24-hour window: since=1104h (46 days) to until=1080h (45 day... [22:46:59] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 10Event-Platform: mediawiki.page_change.v1 event - Add user first_registration_dt field - https://phabricator.wikimedia.org/T426998#12073989 (10nshahquinn-wmf) @Ottomata sorry for the delayed response! Following u... [22:53:48] !log Deploying Refinery at 4e7a2b32 for changes: pageview allowlist 1305158 (+min.wikiquote) 1305162 (+bol.wikipedia), 1305156 (+isv.wikipedia); 1305980 (pv allowlist -api.wikimedia, sqoop +isvwiki); sqoop 1295064 (+globalimagelinks) 1295069 (+filerevision) [22:53:50] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log