[03:25:13] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Product-Analytics, 10Event-Platform, 13Patch-For-Review: Enable Event Platform instruments to opt out of collecting User-Agent data - https://phabricator.wikimedia.org/T382173#10495639 (10Ottomata) Merged and triggered a release. Version 1.8.0 shoul... [07:38:48] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add normalization columns to categorylinks table - https://phabricator.wikimedia.org/T384592#10495686 (10Marostegui) [07:40:43] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add normalization columns to categorylinks table - https://phabricator.wikimedia.org/T384592#10495689 (10Marostegui) [08:20:06] 06Data-Engineering, 06Data-Platform-SRE: Unable to trigger dag with config - https://phabricator.wikimedia.org/T384805 (10dcausse) 03NEW [08:38:08] 06Data-Engineering, 10Data-Platform-SRE (2025.01.11 - 2025.01.31), 13Patch-For-Review: Unable to trigger dag with config - https://phabricator.wikimedia.org/T384805#10495744 (10brouberol) 05Open→03In progress a:03brouberol [09:58:01] (03PS1) 10Filippo Giunchedi: Pick up tls_sess and nocookies for webrequest_sampled [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1114340 (https://phabricator.wikimedia.org/T383900) [09:59:47] (03CR) 10Filippo Giunchedi: "We added these two new fields to webrequest_sampled in Ib1b2b09755fd and this is the followup change on the druid/analytics side. I have n" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1114340 (https://phabricator.wikimedia.org/T383900) (owner: 10Filippo Giunchedi) [10:34:47] 06Data-Engineering, 06Traffic, 13Patch-For-Review: Rollout haproxykafka on all hosts - https://phabricator.wikimedia.org/T378578#10496276 (10Fabfur) [12:33:04] (03CR) 10Elukey: [C:03+1] Pick up tls_sess and nocookies for webrequest_sampled [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1114340 (https://phabricator.wikimedia.org/T383900) (owner: 10Filippo Giunchedi) [12:43:16] (03CR) 10CDanis: [C:03+1] "thanks!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1114340 (https://phabricator.wikimedia.org/T383900) (owner: 10Filippo Giunchedi) [12:58:12] 14Analytics-Radar, 06Data-Engineering, 06Data-Engineering-Icebox, 10observability, and 3 others: Retire udp2log: onboard its producers and consumers to the logging pipeline - https://phabricator.wikimedia.org/T205856#10496789 (10fgiunchedi) Something that occurred to me while looking at {T366710}: with mw-... [13:49:21] (03CR) 10Filippo Giunchedi: "Thank you for the reviews! What are the deployment steps post-merge (if any) ?" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1114340 (https://phabricator.wikimedia.org/T383900) (owner: 10Filippo Giunchedi) [14:12:13] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Product-Analytics, 13Patch-For-Review: [SPIKE] Experiment with approaches for a incremental updates of MediaWiki data in the Data Lake - https://phabricator.wikimedia.org/T370354#10497066 (10Ottomata) @nozeroday asked > are there some internal documen... [14:22:38] (03CR) 10Elukey: [C:03+1] "I think that DE will need to update the Airflow jobs, but can't think about anything else!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1114340 (https://phabricator.wikimedia.org/T383900) (owner: 10Filippo Giunchedi) [14:56:25] 06Data-Engineering, 06Data-Engineering-Radar, 06Product-Analytics, 10Wmfdata-Python: Retrieve host & port info when connecting to MariaDB replicas on the cluster - https://phabricator.wikimedia.org/T340472#10497263 (10mpopov) Either from Refinery or from https://gitlab.wikimedia.org/repos/data-engineering/... [15:19:46] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Product-Analytics, 10Event-Platform, 13Patch-For-Review: Enable Event Platform instruments to opt out of collecting User-Agent data - https://phabricator.wikimedia.org/T382173#10497375 (10mpopov) @Ottomata: Will existing streams be affected? Or will... [15:29:32] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Product-Analytics, 10Event-Platform, 13Patch-For-Review: Enable Event Platform instruments to opt out of collecting User-Agent data - https://phabricator.wikimedia.org/T382173#10497402 (10Ottomata) > Or will UA info collection be "on" by default and... [19:03:20] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Experimentation Lab, 10Dumps 2.0 (Kanban Board), 13Patch-For-Review: Dashboard and alerting of data quality metrics for wmf_content.mediawiki_content_history_v1 - https://phabricator.wikimedia.org/T357684#10498451 (10tchin) [[ https://superset.wikime... [19:17:38] 06Data-Engineering, 10Dumps 2.0, 10Data-Platform-SRE (2025.01.11 - 2025.01.31), 13Patch-For-Review: Upgrade Spark to a version with long term Iceberg support, and with fixes to support Dumps 2.0 - https://phabricator.wikimedia.org/T338057#10498488 (10Ahoelzl) [19:46:18] 10Data-Engineering (Q3 2024 January 1st - March 31th), 07Essential-Work: Analyze Dumps Usage Through Apache Logs - https://phabricator.wikimedia.org/T383175#10498566 (10JEbe-WMF) {F58292463} [20:26:06] 06Data-Engineering, 10Observability-Logging, 06SRE, 13Patch-For-Review: Add x-analytics nocookie=1 and x-tls-sess to webrequest-sampled-live stream - https://phabricator.wikimedia.org/T383900#10498690 (10RLazarus) [20:52:23] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Product-Analytics, 10Event-Platform, 13Patch-For-Review: Enable Event Platform instruments to opt out of collecting User-Agent data - https://phabricator.wikimedia.org/T382173#10498817 (10Ottomata) I deployed the latest eventgate-wikimedia version v1... [21:11:46] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 07Epic: DomainEvents - Broadcasting and receiving cross-process events - https://phabricator.wikimedia.org/T379935#10498864 (10Ottomata) [21:16:18] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team: DomainEvents - [Hypothesis] WE5.2.6 Event Broadcasting Discovery & Design - https://phabricator.wikimedia.org/T384874 (10Ottomata) 03NEW [21:21:12] 06Data-Engineering, 10Dumps-Generation: Wikimedia Downloads not complete - https://phabricator.wikimedia.org/T383030#10498893 (10ValterVB) Again same problem: "2025-01-23 20:35:30 itwiki: Dump complete" but some dump aren't ready: example "waiting All pages, current versions only." [21:46:36] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Dumps 2.0 (Kanban Board), 13Patch-For-Review: Enable HA for the mw-content-history-reconcile-enrich flink application - https://phabricator.wikimedia.org/T375176#10498969 (10xcollazo) 05Open→03Resolved [23:56:14] 06Data-Engineering, 10Data-Platform-SRE (2025.01.11 - 2025.01.31), 07Essential-Work, 13Patch-For-Review: Update canary_events DAG to use an internal domain and/or the service mesh to obtain its eventstream config - https://phabricator.wikimedia.org/T384329#10499263 (10BTullis) I have tried adding the servi...