[07:54:10] 06Data-Engineering, 06Data Products, 06DBA, 10GlobalBlocking, and 2 others: Apply a default to gbw_address in the global_block_whitelist table - https://phabricator.wikimedia.org/T375184#10166541 (10ABran-WMF) p:05Triage→03Medium [10:28:29] 06Data-Engineering, 10Cassandra, 10Data Pipelines, 10Data-Platform-SRE (2024.09.06 - 2024.09.27), 13Patch-For-Review: Create puppet defined type for adding/updating/deleting secrets or other small files on HDFS - https://phabricator.wikimedia.org/T323692#10166949 (10BTullis) I believe that [Add an hdfs_f... [14:04:54] !log Deployed latest DAGs to analytics Airflow instance. T369868. [14:04:57] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:04:57] T369868: Improve handling of delete, restore, and merge from incremental update - https://phabricator.wikimedia.org/T369868 [14:11:49] (03CR) 10Aqu: [C:03+1] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1073214 (https://phabricator.wikimedia.org/T368788) (owner: 10Joal) [14:20:58] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Event-Platform, 13Patch-For-Review: Migrate Event Platform Schema Respositories to Gitlab - https://phabricator.wikimedia.org/T366836#10167666 (10Ottomata) [14:36:40] 06Data-Engineering, 06Data Products, 06DBA, 10GlobalBlocking, and 2 others: Apply a default to gbw_address in the global_block_whitelist table - https://phabricator.wikimedia.org/T375184#10167712 (10Dreamy_Jazz) [14:44:57] 10Data-Engineering (Q1 2024 July 1st - September 30th): [gobblin-wmf] Update parent POM to a more recent version, and make sure that the verify maven goal passes - https://phabricator.wikimedia.org/T375390 (10amastilovic) 03NEW [14:45:45] 10Data-Engineering (Q1 2024 July 1st - September 30th): [gobblin-wmf] Update parent POM to a more recent version, and make sure that the verify maven goal passes - https://phabricator.wikimedia.org/T375390#10167760 (10amastilovic) Project migrated to GitLab: https://gitlab.wikimedia.org/repos/data-engineering/go... [14:47:05] 10Data-Engineering (Q1 2024 July 1st - September 30th): [gobblin-wmf] Update parent POM to a more recent version, and make sure that the verify maven goal passes - https://phabricator.wikimedia.org/T375390#10167766 (10amastilovic) [15:01:28] 06Data-Engineering, 06Data-Platform-SRE, 06serviceops, 06SRE, 10Event-Platform: DRY kafka broker declaration in helmfiles - https://phabricator.wikimedia.org/T253058#10167841 (10brouberol) `cirrus-streaming-updater` is replacing the list of brokers by the external services service name: https://gerrit.wi... [15:14:45] 06Data-Engineering, 06DBA, 10GlobalBlocking, 07Schema-change-in-production, 10Trust and Safety Product Sprint (Sprint Beatboxing (Sept 16-27)): Apply a default to gbw_address in the global_block_whitelist table - https://phabricator.wikimedia.org/T375184#10167870 (10VirginiaPoundstone) [15:19:51] 06Data-Engineering, 06DBA, 10GlobalBlocking, 07Schema-change-in-production, 10Trust and Safety Product Sprint (Sprint Beatboxing (Sept 16-27)): Apply a default to gbw_address in the global_block_whitelist table - https://phabricator.wikimedia.org/T375184#10167911 (10Milimetric) (we don't think this will... [15:27:32] (03CR) 10Milimetric: [V:03+2 C:03+2] "Thanks!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1064438 (https://phabricator.wikimedia.org/T372987) (owner: 10David Martin) [15:57:42] 07Analytics-Data-Problem, 06Data-Engineering, 06Data Products, 10Pageviews-API: Missed pageview data over API - https://phabricator.wikimedia.org/T370108#10168160 (10VirginiaPoundstone) @Mayakp.wiki how would something like ideally sit in a data governance workflow? What should happen to this task next? [16:58:00] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to stat1007 for cyndywikime - https://phabricator.wikimedia.org/T375060#10168452 (10Ottomata) Approved [16:59:47] 06Data-Engineering, 10Event-Platform: [Event Platform] We should alert on EventBus performance degradation. - https://phabricator.wikimedia.org/T375197#10168460 (10Ottomata) [17:08:39] 06Data-Engineering, 10Dumps 2.0, 10Event-Platform: [SPIKE] how can we support Spark producer/consumers in Event Platform - https://phabricator.wikimedia.org/T374341#10168477 (10Ottomata) I would try to keep eventgate produce throughput under 1000 per second. Both eventgate and kafka are horizontally scalabl... [17:14:12] 06Data-Engineering, 10CirrusSearch, 10Structured Data Engineering, 06Structured-Data-Backlog, 03Discovery-Search (Current work): Migrate image recommendation to use page_weighted_tags_changed stream - https://phabricator.wikimedia.org/T372912#10168490 (10Ottomata) > Are we OK to produce from spark to ka... [17:15:44] 06Data-Engineering, 10CirrusSearch, 10Structured Data Engineering, 06Structured-Data-Backlog, 03Discovery-Search (Current work): Migrate image recommendation to use page_weighted_tags_changed stream - https://phabricator.wikimedia.org/T372912#10168495 (10Ottomata) > This could be solved by introducing an... [18:03:15] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Event-Platform: [NEEDS GROOMING] We should improve the code health of gobblin-wmf - https://phabricator.wikimedia.org/T370368#10168754 (10amastilovic) [18:07:15] 06Data-Engineering, 03Discovery-Search (Current work): Datahub - ingest Hive discovery database - https://phabricator.wikimedia.org/T374118#10168762 (10EBernhardson) Once todays daily run fires (approx Sept 24 00:00 UTC) can verify everything shows up as expected and complete this task. [18:14:32] 06Data-Engineering, 10Wikidata, 03Discovery-Search (Current work), 10Event-Platform: Configure https://stream.wikimedia.org to expose rdf-streaming-updater.mutation - https://phabricator.wikimedia.org/T374921#10168778 (10Ottomata) [18:30:10] 06Data-Engineering, 10MediaWiki-extensions-WikimediaEvents, 10Observability-Metrics, 10Event-Platform, and 3 others: Add Prometheus support to statsd.js via mw.track() - https://phabricator.wikimedia.org/T355837#10168849 (10Ottomata) > If it matters that we take it out of the query string, sendBeacon does... [18:31:54] 06Data-Engineering, 06Data-Platform, 06Product-Analytics, 10Temporary accounts, 10Event-Platform: Ensure performer attributes in schemas clarify if the user is a temporary account - https://phabricator.wikimedia.org/T374940#10168866 (10Ottomata) [19:47:06] (03PS1) 10Gerrit maintenance bot: Add tdd.wikipedia to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1075073 (https://phabricator.wikimedia.org/T375424) [19:52:20] (03PS1) 10Gerrit maintenance bot: Add shn.wikinews to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1075075 (https://phabricator.wikimedia.org/T375433) [20:10:03] 06Data-Engineering, 06DBA, 10GlobalBlocking, 07Schema-change-in-production, 10Trust and Safety Product Sprint (Sprint Beatboxing (Sept 16-27)): Apply a default to gbw_address in the global_block_whitelist table - https://phabricator.wikimedia.org/T375184#10169242 (10Ladsgroup) I'm going to try to do this... [20:36:28] !log drained dse-k8s-worker1001 ready for reimage for T365283 [20:36:31] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [20:36:31] T365283: Reimage dse-k8s-worker100[1-4] to correct the partman recipe and enable additional local storage - https://phabricator.wikimedia.org/T365283