[05:19:33] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop deprecated abuse filter fields on wmf wikis - https://phabricator.wikimedia.org/T367781#9989066 (10Marostegui) [05:36:17] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop deprecated abuse filter fields on wmf wikis - https://phabricator.wikimedia.org/T367781#9989085 (10Marostegui) @ABran-WMF I've switched s7 master [06:32:47] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop deprecated abuse filter fields on wmf wikis - https://phabricator.wikimedia.org/T367781#9989114 (10ABran-WMF) thanks! will roll the change there as well [08:08:21] 06Data-Engineering, 06Data Products, 10Pageviews-API: Missed pageview data over API - https://phabricator.wikimedia.org/T370108#9989208 (10Dusan_Krehel) [08:37:59] 06Data-Engineering, 06Data Products, 06DBA, 07Schema-change-in-production: Cleanup revision table schema - https://phabricator.wikimedia.org/T367856#9989297 (10Marostegui) [08:45:07] !log stopping mariadb section 1-8 on clouddb1021 for T368518 [08:45:10] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [08:45:11] T368518: decommission clouddb1021 - https://phabricator.wikimedia.org/T368518 [08:51:35] (03PS3) 10Gmodena: Bump wikimedia-event-utilities version to 1.3.6 [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1054652 (https://phabricator.wikimedia.org/T370199) (owner: 10Ottomata) [08:56:04] (03CR) 10Btullis: [C:03+1] "Looks good to me. I have also had to do this with conjars in the past." [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1054652 (https://phabricator.wikimedia.org/T370199) (owner: 10Ottomata) [08:59:15] (03CR) 10CI reject: [V:04-1] Bump wikimedia-event-utilities version to 1.3.6 [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1054652 (https://phabricator.wikimedia.org/T370199) (owner: 10Ottomata) [09:41:05] 06Data-Engineering, 10LDAP-Access-Requests, 06SRE, 10SRE-Access-Requests: LDAP access to the analytics-privatedata-users group for Quiddity - https://phabricator.wikimedia.org/T370091#9989407 (10Clement_Goubert) a:05KStineRowe_WMF→03Clement_Goubert [09:41:20] 06Data-Engineering, 10LDAP-Access-Requests, 06SRE, 10SRE-Access-Requests: LDAP access to the analytics-privatedata-users group for Quiddity - https://phabricator.wikimedia.org/T370091#9989408 (10Clement_Goubert) [09:57:45] (03PS4) 10Gmodena: Bump wikimedia-event-utilities version to 1.3.6 [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1054652 (https://phabricator.wikimedia.org/T370199) (owner: 10Ottomata) [09:57:53] 06Data-Engineering, 06Data Products, 06DBA, 07Schema-change-in-production: Cleanup revision table schema - https://phabricator.wikimedia.org/T367856#9989452 (10Marostegui) [09:58:48] (03PS5) 10Gmodena: Bump wikimedia-event-utilities version to 1.3.6 [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1054652 (https://phabricator.wikimedia.org/T370199) (owner: 10Ottomata) [10:00:26] 06Data-Engineering, 06Data Products, 10Pageviews-API: Missed pageview data over API - https://phabricator.wikimedia.org/T370108#9989461 (10Dusan_Krehel) [10:00:49] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop deprecated abuse filter fields on wmf wikis - https://phabricator.wikimedia.org/T367781#9989462 (10ABran-WMF) [10:05:45] (03CR) 10CI reject: [V:04-1] Bump wikimedia-event-utilities version to 1.3.6 [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1054652 (https://phabricator.wikimedia.org/T370199) (owner: 10Ottomata) [10:56:41] 06Data-Engineering, 10LDAP-Access-Requests, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: LDAP access to the analytics-privatedata-users group for Quiddity - https://phabricator.wikimedia.org/T370091#9989556 (10Clement_Goubert) 05In progress→03Resolved I have merged the access change, puppet... [11:12:44] 10Data-Engineering (Q1 2024 July 1st - September 30th), 13Patch-For-Review: Add instrumentation for actor signatures - https://phabricator.wikimedia.org/T362783#9989609 (10gmodena) [11:13:46] 10Data-Engineering (Q1 2024 July 1st - September 30th), 13Patch-For-Review: Add instrumentation for actor signatures - https://phabricator.wikimedia.org/T362783#9989610 (10gmodena) DQ job and airflow dag have been implemented. Deployment requires a new release of refinery-source and a dag deployment on analytics. [11:14:59] 10Data-Engineering (Q1 2024 July 1st - September 30th), 13Patch-For-Review: Add host level instrumentation on webrequest - https://phabricator.wikimedia.org/T362785#9989614 (10gmodena) [11:16:37] 10Data-Engineering (Q1 2024 July 1st - September 30th), 13Patch-For-Review: Add host level instrumentation on webrequest - https://phabricator.wikimedia.org/T362785#9989616 (10gmodena) DQ job and airflow dag have been updated. Deployment requires a new release of refinery-source and a dag deployment on analytics. [12:16:02] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Event-Platform: Evaluate ESC and explore an alternative design. - https://phabricator.wikimedia.org/T365005#9989812 (10gmodena) After re-scoping both Config Store and MPIC, we decided to not move forward with refactoring ESC at this stage. A summary of... [13:47:08] (03PS6) 10Gmodena: Bump wikimedia-event-utilities version to 1.3.6 [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1054652 (https://phabricator.wikimedia.org/T370199) (owner: 10Ottomata) [13:58:31] (03CR) 10Ottomata: [C:03+1] Bump wikimedia-event-utilities version to 1.3.6 [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1054652 (https://phabricator.wikimedia.org/T370199) (owner: 10Ottomata) [13:59:15] (03CR) 10Gmodena: [C:03+2] Bump wikimedia-event-utilities version to 1.3.6 [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1054652 (https://phabricator.wikimedia.org/T370199) (owner: 10Ottomata) [14:06:14] (03Merged) 10jenkins-bot: Bump wikimedia-event-utilities version to 1.3.6 [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1054652 (https://phabricator.wikimedia.org/T370199) (owner: 10Ottomata) [14:14:34] Starting build #1 for job analytics-gobblin-wmf-maven-release [14:23:15] Project analytics-gobblin-wmf-maven-release build #1: 09SUCCESS in 8 min 40 sec: https://integration.wikimedia.org/ci/job/analytics-gobblin-wmf-maven-release/1/ [14:45:51] (03PS21) 10Aqu: Refactor Refine to be triggerd by Airflow [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1016808 (https://phabricator.wikimedia.org/T356762) [14:47:55] (03CR) 10Aqu: "I've added changes following last review." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1016808 (https://phabricator.wikimedia.org/T356762) (owner: 10Aqu) [14:48:16] 14Analytics-Radar, 06Data-Engineering, 06Data-Platform-SRE, 06serviceops-radar, and 2 others: Configuration Management for Kafka settings - https://phabricator.wikimedia.org/T276088#9990559 (10bking) FYI, there is [[ https://github.com/StephenSorriaux/ansible-kafka-admin | an ansible library ]] that claims... [15:06:43] (03CR) 10CI reject: [V:04-1] Refactor Refine to be triggerd by Airflow [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1016808 (https://phabricator.wikimedia.org/T356762) (owner: 10Aqu) [15:16:55] (03PS1) 10Gmodena: artifacts: add gobblin-wmf 1.0.2 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1054898 (https://phabricator.wikimedia.org/T370199) [15:20:38] (03PS2) 10Gmodena: artifacts: add gobblin-wmf 1.0.2 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1054898 (https://phabricator.wikimedia.org/T370199) [15:35:05] (03PS3) 10Ottomata: artifacts: add gobblin-wmf 1.0.2 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1054898 (https://phabricator.wikimedia.org/T370199) (owner: 10Gmodena) [15:35:17] 06Data-Engineering, 06DBA: dbstore1008:3317 (s7) crashed - https://phabricator.wikimedia.org/T370122#9990765 (10BTullis) @Marostegui - is there a chance that you might have missed some of the grants after re-cloning s7 yesterday. We have had a sqoop failure with a message about access being denied for the... [15:35:44] (03PS4) 10Ottomata: artifacts: add gobblin-wmf 1.0.2 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1054898 (https://phabricator.wikimedia.org/T370199) (owner: 10Gmodena) [15:41:06] (03CR) 10Ottomata: [C:03+2] artifacts: add gobblin-wmf 1.0.2 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1054898 (https://phabricator.wikimedia.org/T370199) (owner: 10Gmodena) [15:41:15] (03CR) 10Ottomata: [V:03+2 C:03+2] artifacts: add gobblin-wmf 1.0.2 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1054898 (https://phabricator.wikimedia.org/T370199) (owner: 10Gmodena) [15:42:56] 06Data-Engineering, 06DBA: dbstore1008:3317 (s7) crashed - https://phabricator.wikimedia.org/T370122#9990778 (10Marostegui) @btullis most likely, as I forgot this host has special grants yeah. If you can apply them, that'd be good. If not, I can do it first thing tomorrow morning. Thanks for the heads up [15:44:23] !log deploying refinery to pick up bump to gobblin wmf [15:44:24] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:47:26] 06Data-Engineering, 06DBA: dbstore1008:3317 (s7) crashed - https://phabricator.wikimedia.org/T370122#9990810 (10BTullis) >>! In T370122#9990778, @Marostegui wrote: > @btullis most likely, as I forgot this host has special grants yeah. If you can apply them, that'd be good. If not, I can do it first thing t... [15:47:39] (03PS1) 10KCVelaga: Add WMF data pipelines (git submodule) & scripts for regular runs [analytics/wmf-product/jobs] - 10https://gerrit.wikimedia.org/r/1054903 (https://phabricator.wikimedia.org/T362612) [15:48:29] 06Data-Engineering, 06DBA: dbstore1008:3317 (s7) crashed - https://phabricator.wikimedia.org/T370122#9990811 (10Marostegui) Thank you [15:49:15] (03PS1) 10Ottomata: Fix location of gobblin-wmf-core-jar-with-dependencies.jar symlink [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1054904 (https://phabricator.wikimedia.org/T370199) [15:49:49] (03CR) 10Ottomata: [V:03+2 C:03+2] Fix location of gobblin-wmf-core-jar-with-dependencies.jar symlink [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1054904 (https://phabricator.wikimedia.org/T370199) (owner: 10Ottomata) [16:37:53] 06Data-Engineering, 06Data Products, 10Metrics Platform Backlog: [MPIC] Analyse risk of potential performance issues with static approach to stream configuration - https://phabricator.wikimedia.org/T366627#9991078 (10VirginiaPoundstone) @Ottomata mid August? First Product team will use MPIC early October and... [16:42:39] 06Data-Engineering, 06Data Products, 10Metrics Platform Backlog: [MPIC] Analyse risk of potential performance issues with static approach to stream configuration - https://phabricator.wikimedia.org/T366627#9991110 (10Ottomata) Great thank you. I will try to get this done by then. [16:48:06] 06Data-Engineering, 10MW-on-K8s: gobblin-wmf: bump event-utilities dependency to unblock MW on K8s migration. - https://phabricator.wikimedia.org/T370199#9991181 (10Ottomata) [16:49:21] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10MW-on-K8s: gobblin-wmf: bump event-utilities dependency to unblock MW on K8s migration. - https://phabricator.wikimedia.org/T370199#9991186 (10Ottomata) [16:59:34] !log deploying airflow-dags to pick up https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/762 for T367949 [16:59:37] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:59:37] T367949: Spin down api_appserver and appserver clusters - https://phabricator.wikimedia.org/T367949 [17:56:46] 06Data-Engineering-Radar, 10MediaWiki-extensions-EventLogging, 10QuickSurveys, 06WMDE-TechWish: QuickSurveys should show an error when response is blocked - https://phabricator.wikimedia.org/T256463#9991818 (10Jdlrobson) [18:16:47] 06Data-Engineering, 07Epic: All things DataHub - https://phabricator.wikimedia.org/T369756#9991957 (10lbowmaker) [18:16:48] 06Data-Engineering, 10Data Pipelines, 10Data-Catalog: Spike: Integrate Spark with DataHub - https://phabricator.wikimedia.org/T306896#9991956 (10lbowmaker) [18:20:22] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Data Pipelines, 10Data-Catalog: Spike: Integrate Spark with DataHub - https://phabricator.wikimedia.org/T306896#9992016 (10lbowmaker) [18:20:37] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Data Pipelines, 10Data-Catalog: Spike: Integrate Spark with DataHub - https://phabricator.wikimedia.org/T306896#9992041 (10lbowmaker) [18:20:40] 10Data-Engineering (Q1 2024 July 1st - September 30th): [SPIKE] Define process to build out lineage in DataHub - https://phabricator.wikimedia.org/T369758#9992042 (10lbowmaker) [18:20:44] 06Data-Engineering, 07Epic: All things DataHub - https://phabricator.wikimedia.org/T369756#9992043 (10lbowmaker) [18:21:27] 06Data-Engineering, 07Epic: All things DataHub - https://phabricator.wikimedia.org/T369756#9992051 (10lbowmaker) [18:31:08] 06Data-Engineering, 06Structured-Data-Backlog: DagProperties don't automatically update Airflow variables - https://phabricator.wikimedia.org/T348963#9992150 (10Ottomata) Alternative idea: When DagProperties populates the Airflow Variable, make it fill in the property values with some dummy `"__DEFAULT__"` va... [18:32:03] 06Data-Engineering, 06Structured-Data-Backlog: DagProperties don't automatically update Airflow variables - https://phabricator.wikimedia.org/T348963#9992159 (10Ottomata) >>if the incoming DagProperties no longer has a given property, then the DAG can't be parsed because the given property can't be overridden... [18:38:50] 06Data-Engineering, 06Structured-Data-Backlog: DagProperties don't automatically update Airflow variables - https://phabricator.wikimedia.org/T348963#9992197 (10Ottomata) > the Variable is created by default and populated with all original values was also intentional, to reduce the work of having to create the... [18:42:55] 14Data-Engineering (Q4 2024 April 1st - June 30th), 10Event-Platform, 07Spike: [SPIKE] Can we express Event Platform configs in Datasets Config? - https://phabricator.wikimedia.org/T361017#9992207 (10Ottomata) > Having Airflow job configuration defined in a place where it can be changed without having to... [19:32:14] 06Data-Engineering, 10CirrusSearch, 03Discovery-Search (Current work), 13Patch-For-Review: [Search Update Pipeline] Source streams for private wikis - https://phabricator.wikimedia.org/T346046#9992405 (10EBernhardson) I've been looking over the related code and pondering what all could potentially go wrong... [19:34:42] 06Data-Engineering, 06Product-Analytics, 07Epic: [SPIKE] Experiment with approaches for a incremental updates of MediaWiki data in the Data Lake - https://phabricator.wikimedia.org/T370354 (10lbowmaker) 03NEW [19:35:38] 06Data-Engineering, 10Event-Platform: Implement stream of HTML content on mw.page_change event - https://phabricator.wikimedia.org/T360794#9992432 (10Isaac) Hey @lbowmaker -- I wanted to check in on the status of this. For the article quality model (T360455), I would like to run a batch job that builds a distr... [20:25:48] 06Data-Engineering, 10CirrusSearch, 03Discovery-Search (Current work), 13Patch-For-Review: [Search Update Pipeline] Source streams for private wikis - https://phabricator.wikimedia.org/T346046#9992583 (10EBernhardson) [20:44:37] 06Data-Engineering, 10CirrusSearch, 03Discovery-Search (Current work), 13Patch-For-Review: [Search Update Pipeline] Source streams for private wikis - https://phabricator.wikimedia.org/T346046#9992629 (10Ottomata) Chatted with @EBernhardson in IRC. Conclusion is that the keys in EventBusStreamNamesMap s... [20:47:26] 06Data-Engineering, 10CirrusSearch, 03Discovery-Search (Current work), 13Patch-For-Review: [Search Update Pipeline] Source streams for private wikis - https://phabricator.wikimedia.org/T346046#9992635 (10Ottomata) Couple of WIP patches up for discussion. - [Use StreamConfigs to determine if an event shoul... [21:22:38] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Event-Platform: [NEEDS GROOMING] We should improve the code health of gobblin-wmf - https://phabricator.wikimedia.org/T370368 (10gmodena) 03NEW