[10:14:56] 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Add rc_source_name_timestamp index to recentchanges table in wmf production - https://phabricator.wikimedia.org/T400854#11093481 (10Ladsgroup) [10:15:13] 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Add rc_source_name_timestamp index to recentchanges table in wmf production - https://phabricator.wikimedia.org/T400854#11093482 (10Ladsgroup) 05Open→03Resolved [12:26:18] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 13Patch-For-Review: [Refine Refactoring] Refine jobs should be scheduled by Airflow: deployment - https://phabricator.wikimedia.org/T369845#11093835 (10Antoine_Quhen) Following the migration, we noticed that some partition paths were changed compared t... [15:20:32] 06Data-Engineering, 10CirrusSearch, 10DPE-Mediawiki-Content, 10Discovery-Search (2025.07.25 - 2025.08.15): Source the CirrusSearch index dumps from hadoop instead of a MW maintenance script - https://phabricator.wikimedia.org/T366248#11094650 (10pfischer) [15:25:12] 06Data-Engineering, 10CirrusSearch, 10DPE-Mediawiki-Content, 10Discovery-Search (2025.07.25 - 2025.08.15): Source the CirrusSearch index dumps from hadoop instead of a MW maintenance script - https://phabricator.wikimedia.org/T366248#11094663 (10pfischer) [16:07:44] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Manage druid `webrequest_sampled_live` data size - https://phabricator.wikimedia.org/T398236#11094849 (10JAllemandou) Hi @CDanis - I've been looking at data sizes (no nice chart :( ) and have not seen significant change when I restarted the job. I'm sur... [16:08:44] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Manage druid `webrequest_sampled_live` data size - https://phabricator.wikimedia.org/T398236#11094853 (10CDanis) Is that something we could hack up a Prometheus exporter for real quick? What's the method of fetching the sizes? [16:29:28] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Manage druid `webrequest_sampled_live` data size - https://phabricator.wikimedia.org/T398236#11094947 (10JAllemandou) Actually we have an exporter, and it's just that the chart was never made /facepalm. Give me a minute, I'll update the dashboard. [16:32:08] (03PS4) 10Mforns: Improve the automated traffic detection pipeline [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1163417 [16:32:50] !log roll restart druid coordinator service for the analytics cluster [16:32:52] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:47:13] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Manage druid `webrequest_sampled_live` data size - https://phabricator.wikimedia.org/T398236#11095073 (10JAllemandou) Here is a link: https://grafana.wikimedia.org/goto/B6yvQzXHR?orgId=1 The funny thing is that the dates don't match: the datasize starts... [16:47:52] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 10EventStreams, 10Event-Platform: EventStreams: duplicate events from double compute (wdqs/rdf) streams - https://phabricator.wikimedia.org/T396564#11095076 (10tchin) @dcausse fyi I just deployed eventstreams with your patch [16:50:55] (03CR) 10Joal: [V:03+2 C:03+2] "Merging for next deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1179270 (https://phabricator.wikimedia.org/T402134) (owner: 10Gerrit maintenance bot) [18:17:08] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): airflow-dags: Move Dockerfile linter to standard linter image and stage - https://phabricator.wikimedia.org/T402204 (10amastilovic) 03NEW [19:01:47] 06Data-Engineering, 10DPE-Mediawiki-Content, 07Epic: Production-level file export (aka dump) of MW Content in XML - https://phabricator.wikimedia.org/T384382#11095511 (10xcollazo) [19:35:12] 06Data-Engineering, 06Data-Engineering-Icebox, 10Wmfdata-Python, 10Movement-Insights (FY25-26 H1), 13Patch-For-Review: Support importing Pandas datasets into the Data Lake using Wmfdata - https://phabricator.wikimedia.org/T273196#11095668 (10nshahquinn-wmf) 05In progress→03Resolved [22:00:51] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): airflow-dags: Move Dockerfile linter to standard linter image and stage - https://phabricator.wikimedia.org/T402204#11096248 (10amastilovic) https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/1609 [22:25:18] 06Data-Engineering, 06Data-Platform-SRE, 07Epic: Upgrade Hadoop to version 3.3.6 and Hive to version 4.0.1 - https://phabricator.wikimedia.org/T379385#11096331 (10nshahquinn-wmf)