[00:04:34] 06Data-Engineering, 06Data-Engineering-Icebox, 10Data-release: Wikipedia Clickstream dataset. Programmatic Access - https://phabricator.wikimedia.org/T134231#12362273 (10Soda) //@fnegri directed me here during a recent Toolforge Discord call.// We've recently had some flakiness with this (the wmcloud/toolfo... [02:37:09] 14Analytics, 06Data-Engineering, 10Data-Engineering-Wikistats: Wikimedia Statistics page-views-by-country no-data - https://phabricator.wikimedia.org/T426935#12362486 (10nshahquinn-wmf) 05Open→03Declined @Skif.res you are correct that the [pageviews-per-country](https://stats.wikimedia.org/#/all-proj... [02:48:43] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05Metrics-Sprint-2026-2027: Non-incremental active UWERs job has incorrect Airflow schedule and dependencies - https://phabricator.wikimedia.org/T438584#12362489 (10nshahquinn-wmf) @amastilovic here's my idea of how to approach it: - for each monthly r... [02:57:02] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05Metrics-Sprint-2026-2027: Non-incremental active UWERs job has incorrect Airflow schedule and dependencies - https://phabricator.wikimedia.org/T438584#12362490 (10nshahquinn-wmf) @amastilovic your ideas would probably work too, but here's the approac... [04:20:06] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Cloud-VPS (Debian Bullseye Deprecation), 10Event-Platform: Upgrade (replace) beta|deployment-prep event platform VPS instances - https://phabricator.wikimedia.org/T429497#12362535 (10tchin) Created `deployment-schema04` to replace `deployment-schem... [06:04:03] (03CR) 10Joal: [C:03+1] "Super neat optimization! great stuff :)" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1344777 (https://phabricator.wikimedia.org/T438245) (owner: 10Xcollazo) [06:41:01] (03CR) 10Joal: Add spark operational metrics code (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1344687 (https://phabricator.wikimedia.org/T297231) (owner: 10Joal) [07:19:34] 06Data-Engineering, 06Data-Engineering-Icebox, 10Data-release: Wikipedia Clickstream dataset. Programmatic Access - https://phabricator.wikimedia.org/T134231#12362682 (10JAllemandou) Bringing this to @Ahoelzl and @Ottomata for attention / prioritization. [07:53:07] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Traffic, 06Data-Platform-SRE (2026-09-18 - 2026-10-09): Provide a scheduled data download service from Google Cloud Storage - https://phabricator.wikimedia.org/T427457#12362749 (10Gehel) [09:30:38] (03PS1) 10Aqu: HiveToJdbc: Add optional target_table param [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1345051 (https://phabricator.wikimedia.org/T436999) [09:41:27] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic, 10Event-Platform: Relative Trending - https://phabricator.wikimedia.org/T425418#12363119 (10JMonton-WMF) I'm moving this epic to "Done" as the Relative Trending has been released to production and the Hypothesis closed. There is still an im... [10:33:40] (03PS6) 10Joal: Add spark operational metrics code [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1344687 (https://phabricator.wikimedia.org/T297231) [12:36:02] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Observability-Metrics: Finalize and/or standardize metrics - https://phabricator.wikimedia.org/T438383#12363804 (10fgiunchedi) >>! In T438383#12360074, @Ottomata wrote: > @Eevans @Clement_Goubert @fgiunchedi etc., do we have standard http (and maybe... [13:25:32] 06Data-Engineering, 10Data-release: Wikipedia Clickstream dataset. Programmatic Access - https://phabricator.wikimedia.org/T134231#12364107 (10Ottomata) [13:25:54] 06Data-Engineering, 10Data-release: Wikipedia Clickstream dataset. Programmatic Access - https://phabricator.wikimedia.org/T134231#12364109 (10Ottomata) Moved to incoming for grooming [13:30:05] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Observability-Metrics: Finalize and/or standardize metrics - https://phabricator.wikimedia.org/T438383#12364123 (10Ottomata) Hm, whatever they are, nodejs [[ https://gitlab.wikimedia.org/repos/data-engineering/service-utils#express-integration | serv... [13:33:16] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10AQS2.0, 10Data-Platform, 06Growth-Team, 07OKR-Work (DE1 FY2026-27): DE1.3.2: Implement "Sliding 30 Day Top-K Pages for Editor" - https://phabricator.wikimedia.org/T437710#12364152 (10Snwachukwu) Sliding 30 day top pages per editor is now available [13:35:31] (03CR) 10Ottomata: Add spark operational metrics code (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1344687 (https://phabricator.wikimedia.org/T297231) (owner: 10Joal) [13:42:23] 06Data-Engineering, 10Data-Platform, 06Data-Platform-SRE, 06ServiceOps: [ServiceOps] Confirm services (eventgate-(main|logging-external|analytics(-external) in healthy state - https://phabricator.wikimedia.org/T439188#12364183 (10JMeybohm) I don't see anything in particular being off with the eventgates, b... [13:51:05] (03PS1) 10Aqu: Declare new table Google Sitedata Aggregate [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1345123 (https://phabricator.wikimedia.org/T436999) [13:58:15] 06Data-Engineering, 10Data-Platform, 06Data-Platform-SRE, 06ServiceOps: [ServiceOps] Confirm services (eventgate-(main|logging-external|analytics(-external) in healthy state - https://phabricator.wikimedia.org/T439188#12364227 (10Ottomata) We haven't received any alerts or complaints. Unless we do I'd ass... [14:01:09] (03PS7) 10Joal: Add spark operational metrics code [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1344687 (https://phabricator.wikimedia.org/T297231) [14:06:12] (03CR) 10Joal: Add spark operational metrics code (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1344687 (https://phabricator.wikimedia.org/T297231) (owner: 10Joal) [14:13:20] (03CR) 10CI reject: [V:04-1] Add spark operational metrics code [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1344687 (https://phabricator.wikimedia.org/T297231) (owner: 10Joal) [14:20:41] 06Data-Engineering, 10Data-Platform, 06Data-Platform-SRE, 06ServiceOps: [ServiceOps] Confirm services (eventgate-(main|logging-external|analytics(-external) in healthy state - https://phabricator.wikimedia.org/T439188#12364349 (10atsuko) Checked the health to my ability (no obvious breakages in k8s pods, n... [14:22:48] 06Data-Engineering, 10Data-Platform, 06Data-Platform-SRE, 06ServiceOps: [ServiceOps] Confirm services (eventgate-(main|logging-external|analytics(-external) in healthy state - https://phabricator.wikimedia.org/T439188#12364353 (10MLechvien-WMF) 05Open→03Resolved Thanks all! We're good to close this [14:28:14] (03CR) 10Xcollazo: [C:03+2] "And there are more coming, we just need to wait for the [profiled runs](https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1344777 (https://phabricator.wikimedia.org/T438245) (owner: 10Xcollazo) [14:29:30] 06Data-Engineering, 10Data-Platform, 06Data-Platform-SRE, 06ServiceOps: [ServiceOps] Confirm services (eventgate-(main|logging-external|analytics(-external) in healthy state - https://phabricator.wikimedia.org/T439188#12364365 (10jasmine_) [14:30:04] 06Data-Engineering, 10Data-Platform, 06Data-Platform-SRE, 06ServiceOps: [ServiceOps] Confirm services (eventgate-(main|logging-external|analytics(-external) in healthy state - https://phabricator.wikimedia.org/T439188#12364368 (10jasmine_) [14:33:51] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Productionize the updates to the wmf_traffic.browser_info table - https://phabricator.wikimedia.org/T431317#12364403 (10mforns) It is the same API. I took it from them. Yes, we could find a way to export from one another. For KAPOW, the data needs to be... [14:35:31] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): gitlab-mcp-server: Add read-only MCP server for GitLab merge requests and comments - https://phabricator.wikimedia.org/T438508#12364410 (10xcollazo) [14:35:41] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Cloud-VPS (Debian Bullseye Deprecation), 10Event-Platform: Upgrade (replace) beta|deployment-prep event platform VPS instances - https://phabricator.wikimedia.org/T429497#12364411 (10tchin) Created `deployment-eventgate05`, accessible at `https://i... [14:41:30] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10AQS2.0, 10Data-Platform, 06Growth-Team, 07OKR-Work (DE1 FY2026-27): DE1.3.2: Implement "Sliding 30 Day Top-K Pages for Editor" - https://phabricator.wikimedia.org/T437710#12364438 (10Snwachukwu) I used the wrong ticket number in my patch. here... [14:45:53] (03Merged) 10jenkins-bot: refinery-job: Sort the XML export partitions in Catalyst [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1344777 (https://phabricator.wikimedia.org/T438245) (owner: 10Xcollazo) [15:06:24] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE (2026-09-18 - 2026-10-09), 07Essential-Work: Matomo: Enable “AIAgents” core plugin - https://phabricator.wikimedia.org/T438862#12364526 (10bking) 05Open→03In progress a:03bking [15:28:15] 06Data-Engineering, 10Data-Platform, 06ServiceOps, 06Data-Platform-SRE (2026-09-18 - 2026-10-09): [ServiceOps] Confirm services (eventgate-(main|logging-external|analytics(-external) in healthy state - https://phabricator.wikimedia.org/T439188#12364668 (10BTullis) [15:33:02] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: User and team Hive database cleanup tracking for Sept 2026 - https://phabricator.wikimedia.org/T437733#12364685 (10xcollazo) [15:34:56] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: User and team Hive database cleanup tracking for Sept 2026 - https://phabricator.wikimedia.org/T437733#12364697 (10xcollazo) @APizzata-WMF thanks for taking care of `/user/hive/warehouse/apizzata.db`. It does look like there is anot... [15:43:49] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: User and team Hive database cleanup tracking for Sept 2026 - https://phabricator.wikimedia.org/T437733#12364731 (10xcollazo) @amastilovic it looks like many other dbt models are generating lots of files under `/user/hive/warehouse/w... [18:59:06] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [19:04:06] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [19:21:27] (03PS1) 10Mforns: fix usage comment of pageview_hourly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1345207 [19:21:28] (03PS1) 10Mforns: Rename compute_pageview_hourly.hql [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1345208 [19:22:26] (03Abandoned) 10Mforns: fix usage comment of pageview_hourly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1345207 (owner: 10Mforns) [19:22:40] (03Abandoned) 10Mforns: Rename compute_pageview_hourly.hql [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1345208 (owner: 10Mforns) [19:26:14] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Include v2 dumps into mirrors rsync configuration - https://phabricator.wikimedia.org/T437463#12365307 (10xcollazo) 05Open→03In progress [19:27:57] (03PS8) 10Mforns: Switch pageview_actor consumers to use webrequest_v2 instead. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1318283 (https://phabricator.wikimedia.org/T425668) [19:29:58] (03PS9) 10Mforns: Switch pageview_actor consumers to use webrequest_v2 instead. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1318283 (https://phabricator.wikimedia.org/T425668) [19:31:49] (03CR) 10Mforns: [V:03+2] "I just realized that the pageview_hourly query still had the old pageview_actor name." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1318283 (https://phabricator.wikimedia.org/T425668) (owner: 10Mforns) [19:41:47] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Include v2 dumps into mirrors rsync configuration - https://phabricator.wikimedia.org/T437463#12365334 (10xcollazo) **Summary so far** Happy to report that to accomplish this, we do not need to touch the v1 dumps code or the v2 code. The mirror rsync mo... [19:52:28] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History, 10Event-Platform: mediawiki.page_change.v1 - include dummy revisions created by page protection changes - https://phabricator.wikimedia.org/T439278 (10Ottomata) 03NEW [19:53:03] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History, 10Event-Platform: mediawiki.page_change.v1 - include dummy revisions created by page protection changes - https://phabricator.wikimedia.org/T439278#12365384 (10Ottomata) [20:01:22] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History, 10Event-Platform: mediawiki.page_change.v1 - include dummy revisions created by page protection changes - https://phabricator.wikimedia.org/T439278#12365389 (10Ottomata) [20:14:04] 06Data-Engineering, 06MediaWiki-API-Platform-Team, 10Event-Platform, 13Patch-For-Review: mediawiki.page_change.v1 - adapt page change kind model to MediaWiki's PageUpdateCauses. - https://phabricator.wikimedia.org/T430588#12365417 (10Ottomata) Changing the semantics of `page_change_kind` would be backwards... [20:15:32] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-API-Platform-Team, 10Event-Platform, 13Patch-For-Review: mediawiki.page_change.v1 - adapt page change kind model to MediaWiki's PageUpdateCauses. - https://phabricator.wikimedia.org/T430588#12365419 (10Ottomata) [20:15:38] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-API-Platform-Team, 10Event-Platform, 13Patch-For-Review: mediawiki.page_change.v1 - adapt page change kind model to MediaWiki's PageUpdateCauses. - https://phabricator.wikimedia.org/T430588#12365420 (10Ottomata) a:03Ottomata [20:25:43] (03PS8) 10Joal: Add spark operational metrics derived from event logs [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1344687 (https://phabricator.wikimedia.org/T297231) [20:32:01] (03PS1) 10Aqu: Add weekly change_tag raw table creation HQL [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1345215 (https://phabricator.wikimedia.org/T437961) [20:37:11] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05Metrics-Sprint-2026-2027, 13Patch-For-Review: Calculate active UWERs on a weekly cadence - https://phabricator.wikimedia.org/T437961#12365448 (10Antoine_Quhen) Like actor, change_tag has no timestamp and is not append-only (tags can be deleted), so... [20:42:11] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05Metrics-Sprint-2026-2027, 13Patch-For-Review: Calculate active UWERs on a weekly cadence - https://phabricator.wikimedia.org/T437961#12365452 (10Ottomata) Or, please just wait for {T434143} asked in T437961#12329516. [20:46:31] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Implement ArticleFeatureCounts Application - https://phabricator.wikimedia.org/T437000#12365458 (10AKhatun_WMF) Per security review some retry/timeout inconsistencies were surfaced. I have now updated the code to rely on envoy retri... [20:49:58] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05Metrics-Sprint-2026-2027, 13Patch-For-Review: Backfill and update schedule of gcp_gsc_export.max_date_sitedata_daily_aggregate - https://phabricator.wikimedia.org/T436999#12365463 (10nshahquinn-wmf) @Antoine_Quhen thanks for working on this! Are yo... [21:13:06] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [21:28:06] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [21:48:49] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): dbt-jobs backfill: new reverted-edit models - https://phabricator.wikimedia.org/T438793#12365634 (10CMyrick-WMF) 05Open→03Resolved [21:52:22] 06Data-Engineering, 06Data-Platform-SRE (2026-09-18 - 2026-10-09), 13Patch-For-Review: Enable STS and IAM on the codfw RADOS Gateway and register the OIDC identity providers - https://phabricator.wikimedia.org/T435590#12365639 (10andrea.denisse) Hi, I had to revert #1345151 because it was taking puppet-merge... [22:22:45] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05Metrics-Sprint-2026-2027, 13Patch-For-Review: Backfill and update schedule of gcp_gsc_export.max_date_sitedata_daily_aggregate - https://phabricator.wikimedia.org/T436999#12365729 (10Ahoelzl) @nshahquinn-wmf this ticket is only concerned with autom...