[04:07:44] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to analytics-privatedata-users for Seanleong-WMDE - https://phabricator.wikimedia.org/T376034#10201472 (10seanleong-WMDE) Hi @kamila, yup, I have read the user responsibilities. Thank you @Dzahn and @KFrancis. [04:59:31] 06Data-Engineering, 06Data Products, 06Growth-Team, 10MediaWiki-extensions-EventLogging, and 3 others: [EPIC] Deprecate EventLogging::logEvent() - https://phabricator.wikimedia.org/T318263#10201493 (10phuedx) [04:59:33] 06Data-Engineering, 06Data Products, 10MediaWiki-extensions-EventLogging, 07Epic: [EPIC] Deprecate mw.eventLog.logEvent() - https://phabricator.wikimedia.org/T317874#10201494 (10phuedx) [04:59:42] 06Data-Engineering, 06Data Products, 10MediaWiki-extensions-EventLogging, 07Epic: [EPIC] Deprecate EventLogging::schemaValidate() - https://phabricator.wikimedia.org/T317793#10201495 (10phuedx) [05:18:00] 14Analytics, 06Data Products, 07Epic: Event Platform Client Libraries - https://phabricator.wikimedia.org/T228175#10201510 (10phuedx) [06:27:06] 06Data-Engineering, 10Dumps 2.0, 10Event-Platform: [SPIKE] how can we support Spark producer/consumers in Event Platform - https://phabricator.wikimedia.org/T374341#10201536 (10gmodena) > So far I wrote that code in Java, but it feels a bit cumbersome when using Spark API that has been written in Scala. Woul... [08:32:23] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Data-Platform-SRE (2024.09.28 - 2024.10.18), 10Event-Platform, 13Patch-For-Review: Migrate Event Platform Schema Respositories to Gitlab - https://phabricator.wikimedia.org/T366836#10201737 (10Gehel) p:05Triage→03High [08:35:04] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Data-Platform-SRE (2024.09.28 - 2024.10.18), 13Patch-For-Review: Update druid config to automatically drop unused segments - https://phabricator.wikimedia.org/T376118#10201769 (10Gehel) p:05Triage→03High [09:22:21] 06Data-Engineering, 10Cassandra, 10Data Pipelines, 10Data-Platform-SRE (2024.09.28 - 2024.10.18), 13Patch-For-Review: Create puppet resource for adding/updating/deleting secrets or other small files on HDFS - https://phabricator.wikimedia.org/T323692#10201850 (10BTullis) [12:58:45] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to analytics-privatedata-users for Seanleong-WMDE - https://phabricator.wikimedia.org/T376034#10202329 (10kamila) 05Open→03In progress Thanks for confirming @seanleong-WMDE, I will finalise this as soon as the group... [14:16:30] 06Data-Engineering, 10Data-Platform-SRE (2024.09.28 - 2024.10.18): Design a suitable DAG deployment method - https://phabricator.wikimedia.org/T368033#10202598 (10Ottomata) > However, we lose the DAG isolation per instance, as each pod would be able to read (and execute) all DAGs. Technically this is true, an... [16:16:28] 10Data-Engineering (Q1 2024 July 1st - September 30th): [Refine Refactoring] Configure and deploy all Refine data sets for parallel production processing and testing - https://phabricator.wikimedia.org/T361501#10202989 (10Ahoelzl) Closing, duplicate of https://phabricator.wikimedia.org/T369845 [16:17:56] 10Data-Engineering (Q1 2024 July 1st - September 30th): [Refine Refactoring] Configure and deploy all Refine data sets for parallel production processing and testing - https://phabricator.wikimedia.org/T361501#10202997 (10Ahoelzl) 05Open→03Invalid [16:22:32] 10Data-Engineering (Q1 2024 July 1st - September 30th): [Refine Refactoring] Define and implement a automated testing / comparison tool for config store configured datasets - https://phabricator.wikimedia.org/T361502#10203005 (10Ahoelzl) a:03Antoine_Quhen [16:38:24] 06Data-Engineering, 03Discovery-Search (Current work): Datahub - ingest Hive discovery database - https://phabricator.wikimedia.org/T374118#10203076 (10EBernhardson) Unfortunately still not finding the tables / columns via search in datahub. If it is a caching issue it's lasting longer than I would have expected. [17:57:48] 06Data-Engineering, 06Movement-Insights: Temporarily Extend Retention Window for webrequest tables - https://phabricator.wikimedia.org/T375943#10203344 (10Ahoelzl) a:03JAllemandou [21:52:51] 06Data-Engineering, 03Discovery-Search (Current work): Datahub - ingest Hive discovery database - https://phabricator.wikimedia.org/T374118#10204045 (10BTullis) This sounds like a DataHub problem to me. Maybe there is an issue with the indexing. Shall we make a ticket to investigate that? If you tag it with #d... [22:30:27] 06Data-Engineering, 10Event-Platform, 13Patch-For-Review, 10Web Team Essential Work 2024 (Migrate to new Event Platform), 10Web-Team-Backlog (FY2024-25 Q2 Sprint 1): Delete redundant mobile- and desktopwebuiactions event in WikimediaEvents - https://phabricator.wikimedia.org/T376065#10204089 (10Jdlrobson...