[02:26:04] 06Data-Engineering, 10Data Pipelines, 06Data-Platform-SRE (2026-08-28 - 2026-09-18): Airflow 3: remove Amazon provider warnings from CLI startup - https://phabricator.wikimedia.org/T437981#12325796 (10RKemper) It works! [04:32:41] (03PS1) 10Snwachukwu: Add sliding 30-day top pages per editor load HQL [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1342105 (https://phabricator.wikimedia.org/T437230) [09:05:57] (03CR) 10Joal: [C:03+1] "From" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1335154 (https://phabricator.wikimedia.org/T431476) (owner: 10Ottomata) [09:42:03] 06Data-Engineering, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work: [Tech evaluation] Table catalog options for the S3 data lake - https://phabricator.wikimedia.org/T435604#12326722 (10Gehel) [09:42:08] 06Data-Engineering, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work: Allow unauthenticated access to the dse-k8s OIDC discovery and JWKS endpoints - https://phabricator.wikimedia.org/T435591#12326725 (10Gehel) [09:43:51] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work, 05Metrics-Sprint-2026-2027: Metrics Superset dashboard access for all WMF - https://phabricator.wikimedia.org/T431697#12326751 (10Gehel) [09:45:39] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work: Implement brownout logic for dumps v1 on dumps web interface - https://phabricator.wikimedia.org/T437461#12326785 (10Gehel) [09:45:57] 06Data-Engineering, 10Data Pipelines, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work: Airflow 3: remove Amazon provider warnings from CLI startup - https://phabricator.wikimedia.org/T437981#12326787 (10Gehel) [09:46:23] 06Data-Engineering, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work, 13Patch-For-Review: [Infra] Add a presto query logger - https://phabricator.wikimedia.org/T269832#12326791 (10Gehel) [09:46:46] 14Analytics-Radar, 06Data-Engineering, 10Kafka-Infrastructure, 06serviceops-radar, and 5 others: Configuration Management for Kafka settings - https://phabricator.wikimedia.org/T276088#12326792 (10Gehel) [09:50:02] 06Data-Engineering, 10Data Pipelines, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work: Airflow 3: remove Amazon provider warnings from CLI startup - https://phabricator.wikimedia.org/T437981#12326879 (10Gehel) 05Open→03Resolved [11:14:06] 06Data-Engineering, 10Data Pipelines, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work: Airflow 3: remove Amazon provider warnings from CLI startup - https://phabricator.wikimedia.org/T437981#12327309 (10atsuko) Will need to publish new image tag `docker-registry.wikimedia.org/repos/data... [12:38:02] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10SRE-SLO: page_change SLO windows - https://phabricator.wikimedia.org/T438054#12327721 (10APizzata-WMF) Following further discussions with the team, we are designing an alternative approach for the Monthly Completeness SLO: on a monthly basis, all dai... [12:56:34] (03CR) 10Milimetric: sqoop-mediawiki-tables - add a shard override (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1335154 (https://phabricator.wikimedia.org/T431476) (owner: 10Ottomata) [13:06:05] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History: Quality verification for mediawiki_history_incremental_v1 using Iceberg time travel - https://phabricator.wikimedia.org/T425734#12327859 (10xcollazo) >>! In T425734#12323720, @xcollazo wrote: >>>! In T425734#1232369... [13:35:47] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to analytics-privatedata-users level 3 for derenrich - https://phabricator.wikimedia.org/T437668#12328019 (10HSwan-WMF) @hnowlan Approved. Thank you! [13:44:20] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History: Quality verification for mediawiki_history_incremental_v1 using Iceberg time travel - https://phabricator.wikimedia.org/T425734#12328097 (10xcollazo) Wait... I forgot to clear the DagProperties, so all the re-runs j... [13:49:15] 06Data-Engineering, 06Data-Engineering-Radar, 10Research-engineering, 13Patch-For-Review: mwparserfromhtml errors - https://phabricator.wikimedia.org/T437669#12328119 (10Isaac) 05Open→03Resolved Merged and thanks again for filing the bugs / helping with review! Now at 2.1.2: https://pypi.org/projec... [13:52:42] (03CR) 10Mforns: [V:03+2] Switch pageview_actor consumers to use webrequest_v2 instead. (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1318283 (https://phabricator.wikimedia.org/T425668) (owner: 10Mforns) [13:53:16] (03CR) 10Mforns: [V:03+2] "This is ready to be merged, but let's wait until the KAPOW cutover date." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1318283 (https://phabricator.wikimedia.org/T425668) (owner: 10Mforns) [15:02:22] 10Data-Engineering-Roadmap, 07Epic: Epic: dbt Backlog - https://phabricator.wikimedia.org/T437619#12328879 (10Ahoelzl) [15:02:33] 10Data-Engineering-Roadmap, 07Epic: Epic: dbt Roadmap - https://phabricator.wikimedia.org/T437618#12328881 (10Ahoelzl) [15:02:49] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Essential-Work: Build an HDFS growth dashboard based on wmf.hdfs_usage - https://phabricator.wikimedia.org/T438056#12328883 (10Ahoelzl) [15:02:58] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Essential-Work: Build an HDFS growth dashboard based on wmf.hdfs_usage - https://phabricator.wikimedia.org/T438056#12328884 (10Ahoelzl) a:03Ahoelzl [15:03:06] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Essential-Work: Build an HDFS growth dashboard based on wmf.hdfs_usage - https://phabricator.wikimedia.org/T438056#12328886 (10Ahoelzl) p:05Triage→03High [15:04:41] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10GitLab, 13Patch-For-Review: Trusted-runner access for developer-activity-pipelines (publish a container image) - https://phabricator.wikimedia.org/T438042#12328905 (10Ahoelzl) p:05Triage→03High a:03Antoine_Quhen [15:07:56] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to analytics-privatedata-users level 3 for derenrich - https://phabricator.wikimedia.org/T437668#12328931 (10Ahoelzl) Approved. [15:08:07] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to analytics-privatedata-users level 3 for derenrich - https://phabricator.wikimedia.org/T437668#12328933 (10Ahoelzl) [15:20:25] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work, 05Metrics-Sprint-2026-2027: Metrics Superset dashboard access for all WMF - https://phabricator.wikimedia.org/T431697#12329005 (10brouberol) [15:20:55] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Product-Analytics: Create a new database for WMF traffic related metrics - https://phabricator.wikimedia.org/T436678#12329011 (10Ahoelzl) [15:21:42] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Product-Analytics: Create a new database for WMF traffic related metrics - https://phabricator.wikimedia.org/T436678#12329014 (10Milimetric) 05Open→03Resolved a:03Milimetric Decided: `wmf_traffic` for now. NOTE: longer conversation around... [15:30:43] 06Data-Engineering, 10MediaWiki-Change-tagging, 06Product Safety and Integrity, 05DE4.1 Platform capacity: Remove private tags from mediawiki-history public dumps - https://phabricator.wikimedia.org/T431524#12329079 (10Milimetric) [15:50:25] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History, 10Event-Platform: EventBus - user entity schema should differentiate between explicit and implicit user groups - https://phabricator.wikimedia.org/T425360#12329226 (10Ottomata) 05Declined→03Open Discussed in the context of {T432025}, and we want to... [16:03:35] 06Data-Engineering, 10MediaWiki-Change-tagging, 06Product Safety and Integrity, 05DE4.1 Platform capacity: Update Sqoop to get `change_tag` and `change_tag_def` from production replicas - https://phabricator.wikimedia.org/T431523#12329347 (10Milimetric) 05Open→03Declined I just looked this up and i... [16:04:19] 06Data-Engineering, 10MediaWiki-Change-tagging, 06Product Safety and Integrity, 05DE4.1 Platform capacity: Remove private tags from mediawiki-history public dumps - https://phabricator.wikimedia.org/T431524#12329358 (10Milimetric) 05Open→03Declined referenced task declined so declining this one too [16:06:25] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Platform, 10EditCheck, 10VisualEditor, and 3 others: Implement sampling requirements for Headless VE technical pilot - https://phabricator.wikimedia.org/T433024#12329379 (10ppelberg) [16:17:54] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Calculate active UWERs on a weekly cadence - https://phabricator.wikimedia.org/T437961#12329516 (10Ottomata) > - base_moderator_actions_suppression > -- uses the mediawiki_revision_visibility_change event stream, so can be made incremental with minimal c... [16:50:38] 06Data-Engineering, 06Data-Engineering-Radar, 06MediaWiki-API-Platform-Team (MediaWiki-Interfaces-Sprints), 07OKR-Work: REST API vNext: Create dedicated REST shared-schema repository - https://phabricator.wikimedia.org/T432799#12329737 (10AGhirelli-WMF) [16:51:20] 06Data-Engineering, 06Data-Engineering-Radar, 06MediaWiki-API-Platform-Team (MediaWiki-Interfaces-Sprints), 07OKR-Work: REST API vNext: Create dedicated REST shared-schema repository - https://phabricator.wikimedia.org/T432799#12329745 (10AGhirelli-WMF) [16:51:59] 06Data-Engineering, 06Data-Engineering-Radar, 06MediaWiki-API-Platform-Team (MediaWiki-Interfaces-Sprints), 07OKR-Work: REST API vNext: Create dedicated REST shared-schema repository - https://phabricator.wikimedia.org/T432799#12329751 (10AGhirelli-WMF) [18:08:57] (03PS1) 10Xcollazo: refinery-job-35: Add 7z recompressor for the v2 history dumps [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1342321 (https://phabricator.wikimedia.org/T437454) [18:40:01] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Implement 7z compression for v2 history dumps - https://phabricator.wikimedia.org/T437454#12330391 (10xcollazo) Two patches are ready for review. Below some notes on the implementation: **Design: recompress the bz2 output, do not e... [18:45:45] (03CR) 10Mforns: Add sliding 30-day top pages per editor load HQL (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1342105 (https://phabricator.wikimedia.org/T437230) (owner: 10Snwachukwu) [19:20:09] (03PS3) 10Milimetric: sqoop-mediawiki-tables - add a shard override [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1335154 (https://phabricator.wikimedia.org/T431476) (owner: 10Ottomata) [19:40:06] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [19:44:14] (03CR) 10Milimetric: "Tested and found all these to work:" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1335154 (https://phabricator.wikimedia.org/T431476) (owner: 10Ottomata) [19:45:55] (03PS4) 10Milimetric: sqoop-mediawiki-tables - add a shard override [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1335154 (https://phabricator.wikimedia.org/T431476) (owner: 10Ottomata) [19:46:09] (03CR) 10Milimetric: [C:03+2] sqoop-mediawiki-tables - add a shard override [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1335154 (https://phabricator.wikimedia.org/T431476) (owner: 10Ottomata) [19:47:48] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Commons, 06Data-Persistence, 07OKR-Work, 13Patch-For-Review: Data Lake - sqoop of commons link tables needs to adapt to move to dedicated MariaDB cluster - https://phabricator.wikimedia.org/T431476#12330623 (10Milimetric) k, I tested my patch i... [19:55:06] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [20:03:49] (03CR) 10Snwachukwu: Add sliding 30-day top pages per editor load HQL (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1342105 (https://phabricator.wikimedia.org/T437230) (owner: 10Snwachukwu) [20:40:20] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): refinery-spark: Distribute HdfsFileFingerprintWriter instead of looping in the driver - https://phabricator.wikimedia.org/T438244 (10xcollazo) 03NEW [20:42:49] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): MediawikiDumper: Reduce the memory that the huge wiki exports reserve - https://phabricator.wikimedia.org/T438245 (10xcollazo) 03NEW [21:13:44] 06Data-Engineering, 06Data-Platform-SRE: Matomo: Enable “for kept data above, also keep segmented reports” option - https://phabricator.wikimedia.org/T438249 (10SCampos-WMF) 03NEW [21:17:28] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): refinery-spark: Distribute HdfsFileFingerprintWriter instead of looping in the driver - https://phabricator.wikimedia.org/T438244#12331029 (10xcollazo) 05Open→03In progress a:03xcollazo [21:43:06] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [22:13:06] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly