[08:03:25] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10SRE-SLO: page_change SLO windows - https://phabricator.wikimedia.org/T438054#12348692 (10APizzata-WMF) a:05APizzata-WMF→03None [08:05:46] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Review SLIS image suggestion pipeline - https://phabricator.wikimedia.org/T415195#12348703 (10APizzata-WMF) 05Open→03Resolved [08:38:35] 06Data-Engineering, 10Beta-Cluster-Infrastructure, 10Event-Platform: Beta: Old code in jobrunner06 - https://phabricator.wikimedia.org/T438818 (10JMonton-WMF) 03NEW [08:39:01] 06Data-Engineering, 10Beta-Cluster-Infrastructure, 10Event-Platform: Beta: Old code in jobrunner06 - https://phabricator.wikimedia.org/T438818#12348824 (10JMonton-WMF) I'll try to create a patch that forces the jobrunner06 to be updated too, and find who owns or can review this. [08:45:44] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE (2026-09-18 - 2026-10-09), 07Essential-Work: Implement brownout logic for dumps v1 on dumps web interface - https://phabricator.wikimedia.org/T437461#12348844 (10BTullis) a:03BTullis [08:45:55] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE (2026-09-18 - 2026-10-09), 07Essential-Work: Implement brownout logic for dumps v1 on dumps web interface - https://phabricator.wikimedia.org/T437461#12348845 (10BTullis) p:05Triage→03High [09:23:07] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10AQS2.0, 10Data-Platform, 06Growth-Team, and 3 others: Make the AQS endpoints more usable for the ImpactModule extension - https://phabricator.wikimedia.org/T433983#12349050 (10Urbanecm_WMF) [09:31:44] (03CR) 10Joal: "Code looks good, comments and commit message could be better IMO :)" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1343615 (https://phabricator.wikimedia.org/T437454) (owner: 10Xcollazo) [09:49:16] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Beta-Cluster-Infrastructure, 10Event-Platform: Beta: Old code in jobrunner06 - https://phabricator.wikimedia.org/T438818#12349210 (10JMonton-WMF) [09:51:34] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform: Relative Trending: Documentation - https://phabricator.wikimedia.org/T438832 (10JMonton-WMF) 03NEW [10:52:37] (03PS1) 10A-pizzata: Add the mediawiki_private_actor_weekly table. It stores each weekly snapshot in a day partition under tables/weekly. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1343956 (https://phabricator.wikimedia.org/T437961) [13:24:07] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10GitLab: Trusted-runner access for developer-activity-pipelines (publish a container image) - https://phabricator.wikimedia.org/T438042#12350068 (10Antoine_Quhen) 05Open→03Resolved [13:57:47] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Movement-Insights: WE3.3 Account Creation metrics dashboard showing lower numbers than expected - https://phabricator.wikimedia.org/T438385#12350219 (10Ahoelzl) p:05Triage→03High a:03amastilovic [14:21:03] (03PS2) 10Xcollazo: refinery-job-35: Make XML Export 7z recompressor do more in task [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1343615 (https://phabricator.wikimedia.org/T437454) [14:22:42] (03CR) 10Xcollazo: refinery-job-35: Make XML Export 7z recompressor do more in task (033 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1343615 (https://phabricator.wikimedia.org/T437454) (owner: 10Xcollazo) [14:38:57] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Implement alerts and create runbooks - https://phabricator.wikimedia.org/T438384#12350508 (10AKhatun_WMF) [14:39:20] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Implement alerts and create runbooks - https://phabricator.wikimedia.org/T438384#12350510 (10AKhatun_WMF) [14:48:25] 06Data-Engineering, 06Data-Platform-SRE: Matomo: Enable “AIAgents” core plugin - https://phabricator.wikimedia.org/T438862 (10SCampos-WMF) 03NEW [14:59:14] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Productionize the updates to the wmf_traffic.browser_info table - https://phabricator.wikimedia.org/T431317#12350725 (10GGoncalves-WMF) I'm not totally sure it's the same API, but SRE are doing something similar to this at the edge (in a private repo) an... [15:02:35] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: data lake: git commit history - https://phabricator.wikimedia.org/T433696#12350745 (10Antoine_Quhen) PreNote: the schema is already defined and event-compatible on our side. Technical direction we're currently advancing: **1/ Coll... [15:09:13] 06Data-Engineering, 06Data-Platform-SRE: Matomo: Enable “AIAgents” core plugin - https://phabricator.wikimedia.org/T438862#12350767 (10BTullis) Hi @SCampos-WMF - Unfortunately, our Matomo instance is only running version `5.3.1` at the moment: {F104322723} This is a technical limitation due to the way that Mat... [15:31:24] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10SRE-SLO: page_change SLO windows - https://phabricator.wikimedia.org/T438054#12350943 (10elukey) @APizzata-WMF I think that Sloth's base alert will complain that no metrics is present during that time, so the ideal state would be those daily metrics... [15:37:44] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Sqoop globalimagelinks and filerevision - https://phabricator.wikimedia.org/T427532#12350958 (10mforns) The MR has been merged. The two datasets should be query ready every month moving forward. [16:28:35] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform, 07Wikimedia-production-error: "User::loadFromSession called before the end of Setup.php" when EventBus is serializing user groups - https://phabricator.wikimedia.org/T436345#12351254 (10Dreamy_Jazz) 05Open→03Resolved This wa... [16:28:59] (03CR) 10Joal: [C:03+1] "LGTM ! thank you for the update" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1343615 (https://phabricator.wikimedia.org/T437454) (owner: 10Xcollazo) [16:40:40] (03PS3) 10Astein: This MR updates the banner_activity_minutely hql used in fr tech's airflow dag. the hql creates/ appends to a table that filters webrequests for banner impressions. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1343586 [16:48:48] (03CR) 10Joal: [C:03+1] "LGTM" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1343586 (owner: 10Astein) [17:23:56] 06Data-Engineering, 06Data-Platform-SRE (2026-09-18 - 2026-10-09): Enable STS and IAM on the codfw RADOS Gateway and register the OIDC identity providers - https://phabricator.wikimedia.org/T435590#12351613 (10BTullis) p:05Triage→03High a:03BTullis [17:24:35] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10SRE-SLO: page_change SLO windows - https://phabricator.wikimedia.org/T438054#12351615 (10APizzata-WMF) Hey @elukey, thanks for the reply. Unfortunately we can compute these metrics only once a month, but I get your point. Please let me know if anyth... [17:52:24] (03CR) 10Xcollazo: [C:03+2] refinery-job-35: Make XML Export 7z recompressor do more in task (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1343615 (https://phabricator.wikimedia.org/T437454) (owner: 10Xcollazo) [18:02:30] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Movement-Insights: WE3.3 Account Creation metrics dashboard showing lower numbers than expected - https://phabricator.wikimedia.org/T438385#12351830 (10amastilovic) I've re-run the `2026-08` job and here are the results of the verification query: `... [18:07:17] (03Merged) 10jenkins-bot: refinery-job-35: Make XML Export 7z recompressor do more in task [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1343615 (https://phabricator.wikimedia.org/T437454) (owner: 10Xcollazo) [18:18:09] 06Data-Engineering, 06Data-Platform-SRE: Matomo: Enable “AIAgents” core plugin - https://phabricator.wikimedia.org/T438862#12351883 (10SCampos-WMF) Thank you @BTullis. Noted on the dependency, and I'm okay to wait until we migrate Matomo frontend to Kubernetes! [18:46:46] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): brand health tracker data ingestion - https://phabricator.wikimedia.org/T425347#12351961 (10Ahoelzl) a:05APizzata-WMF→03Ahoelzl [18:47:16] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Movement-Insights: WE3.3 Account Creation metrics dashboard showing lower numbers than expected - https://phabricator.wikimedia.org/T438385#12351973 (10amastilovic) By the way, since this particular run (`2026-08`) was already in Airflow, all you nee... [18:49:35] 06Data-Engineering: Migrate Sqoop jobs to Airflow - https://phabricator.wikimedia.org/T409514#12351976 (10Ahoelzl) a:05amastilovic→03None [18:50:19] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): [SPIKE] Evaluate Astronomer Cosmos for dbt scheduling & external dependency handling in Airflow - https://phabricator.wikimedia.org/T425038#12351990 (10Ahoelzl) 05Open→03Declined We basically ended up shipping our own dbt / Airflow integration an... [18:51:45] 06Data-Engineering, 07Epic: Provide configurable, scheduled data platform data import / export capabilities - https://phabricator.wikimedia.org/T428511#12351999 (10Ahoelzl) a:05Antoine_Quhen→03None [18:53:07] 10Data-Engineering-Roadmap, 07Epic: Provide configurable, scheduled data platform data import / export capabilities - https://phabricator.wikimedia.org/T428511#12352004 (10Ahoelzl) [18:53:23] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Wikimedia Enterprise: PageViews S3 Data Transfer MR [Enterprise] - https://phabricator.wikimedia.org/T425543#12352007 (10Ahoelzl) [18:53:27] 06Data-Engineering, 07Epic, 07OKR-Work, 13Patch-For-Review: SDS 1.3.2 [EPIC] Automated alerting for changes in automated traffic behavior - https://phabricator.wikimedia.org/T407235#12352008 (10Ahoelzl) [18:53:52] 06Data-Engineering, 06Wikimedia Enterprise: PageViews S3 Data Transfer MR [Enterprise] - https://phabricator.wikimedia.org/T425543#12352010 (10Ahoelzl) a:05Antoine_Quhen→03None [18:59:09] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Wikimedia Enterprise: WME Pageviews DAG for HDFS to S3 Transfer - https://phabricator.wikimedia.org/T426017#12352053 (10Ahoelzl) This has been implemented a while ago, correct? I see the Airflow DAG: https://airflow-platform-eng.wikimedia.org/dags/pa... [18:59:57] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Load Google Search Console social account impressions and clicks data into Data Lake - https://phabricator.wikimedia.org/T436710#12352055 (10Ahoelzl) [19:13:34] (03PS1) 10Aqu: Update Changelog for 0.3.27 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1344060 [19:17:18] (03CR) 10Aqu: [C:03+2] This MR updates the banner_activity_minutely hql used in fr tech's airflow dag. the hql creates/ appends to a table that filters webrequests [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1343586 (owner: 10Astein) [19:17:22] (03CR) 10Aqu: [V:03+2 C:03+2] This MR updates the banner_activity_minutely hql used in fr tech's airflow dag. the hql creates/ appends to a table that filters webrequests [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1343586 (owner: 10Astein) [19:17:41] (03CR) 10Aqu: [V:03+2 C:03+2] This MR updates the banner_activity_minutely hql used in fr tech's airflow dag. the hql creates/ appends to a table that filters webrequests (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1343586 (owner: 10Astein) [19:40:32] (03CR) 10Aqu: [C:03+2] Update Changelog for 0.3.27 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1344060 (owner: 10Aqu) [19:51:59] (03Merged) 10jenkins-bot: Update Changelog for 0.3.27 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1344060 (owner: 10Aqu) [19:54:26] Starting build #74 for job analytics-refinery-maven-release [19:57:44] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Beta-Cluster-Infrastructure, 10Event-Platform: Beta: Old code in jobrunner06 - https://phabricator.wikimedia.org/T438818#12352392 (10Southparkfan) →14Duplicate dup:03T438656 [20:05:42] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06RoadToWiki, 05Metrics-Sprint-2026-2027, 13Patch-For-Review: Mirror datasets used on the key metrics dashboard to Postgres - https://phabricator.wikimedia.org/T438586#12352416 (10amastilovic) a:05Antoine_Quhen→03amastilovic [20:20:55] Project analytics-refinery-maven-release build #74: 09SUCCESS in 26 min: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release/74/ [20:22:17] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Implement 7z compression for v2 history dumps - https://phabricator.wikimedia.org/T437454#12352456 (10xcollazo) **Update: the task instance count is back to the level before the 7z work** The first iteration of the new DAG made abo... [23:21:46] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06RoadToWiki, 05Metrics-Sprint-2026-2027, 13Patch-For-Review: Mirror datasets used on the key metrics dashboard to Postgres - https://phabricator.wikimedia.org/T438586#12353053 (10amastilovic) I've scheduled the Datalake-2-Postgres transfers in Air... [23:37:27] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05Metrics-Sprint-2026-2027: Non-incremental active UWERs job has incorrect Airflow schedule and dependencies - https://phabricator.wikimedia.org/T438584#12353086 (10Ahoelzl) [23:52:49] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05Metrics-Sprint-2026-2027: Backfill and update schedule of gcp_gsc_export.max_date_sitedata_daily_aggregate - https://phabricator.wikimedia.org/T436999#12353102 (10Ahoelzl) The currently manually updated table is `gcp_gsc_export.sitedata_daily_aggrega...