[00:08:09] 06Data-Engineering: [SPIKE] Learn and document how to use Flink-CDC from MediaWiki MariaDB locally - https://phabricator.wikimedia.org/T373144#10086686 (10Ottomata) [01:08:52] 06Data-Engineering: [SPIKE] Learn and document how to use Flink-CDC from MediaWiki MariaDB locally - https://phabricator.wikimedia.org/T373144#10086748 (10Ottomata) [01:09:38] 06Data-Engineering: [SPIKE] Learn and document how to use Flink-CDC from MediaWiki MariaDB locally - https://phabricator.wikimedia.org/T373144#10086749 (10Ottomata) [11:18:59] 10Quarry: Update cluster to 1.26 - https://phabricator.wikimedia.org/T373093#10087457 (10github-toolforge-bot) vivian-rook opened https://github.com/toolforge/quarry/pull/67 [13:05:56] (03CR) 10Milimetric: [V:03+2 C:03+2] Don't select cuc_actiontext from cu_changes for sqoop [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1057221 (https://phabricator.wikimedia.org/T371099) (owner: 10Dreamy Jazz) [13:06:07] 06Data-Engineering, 10Data Products (Data products Sprint 18), 13Patch-For-Review: No longer use removed cuc_actiontext column in analytics/refinery - https://phabricator.wikimedia.org/T371099#10087667 (10Milimetric) this got lost in the sprint move and almost broke a bunch of stuff, but I had a nightmare ab... [13:07:02] !log deploying refinery for simple sqoop changes [13:07:04] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:09:55] 06Data-Engineering, 06serviceops-radar, 10Event-Platform: Improve eventgates health check/readiness probe - https://phabricator.wikimedia.org/T373192 (10JMeybohm) 03NEW [14:28:20] 06Data-Engineering, 06serviceops-radar, 10Event-Platform: Improve eventgates health check/readiness probe - https://phabricator.wikimedia.org/T373192#10088011 (10CDanis) @Ottomata wrote: > If eventgate cannot produce to Kafka, it is broken. In this case, Kafka overloaded was maybe the problem, but in other c... [14:36:08] (03CR) 10KCVelaga: [C:03+1] "Looks good." [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/1061096 (https://phabricator.wikimedia.org/T369687) (owner: 10Mforns) [14:51:08] 10Data-Engineering (Q1 2024 July 1st - September 30th), 06Data-Platform: 7 new wikis missing from mediawiki_history - https://phabricator.wikimedia.org/T368788#10088098 (10CMyrick-WMF) [14:51:48] 10Data-Engineering (Q1 2024 July 1st - September 30th), 06Data-Platform: 7 new wikis missing from mediawiki_history - https://phabricator.wikimedia.org/T368788#10088100 (10CMyrick-WMF) Now that there is a July mediawiki_history snapshot, I've updated this ticket to include the missing wiki that was created in... [14:52:09] 10Data-Engineering (Q1 2024 July 1st - September 30th), 06Data-Platform: 8 new wikis missing from mediawiki_history - https://phabricator.wikimedia.org/T368788#10088101 (10CMyrick-WMF) [15:04:15] 06Data-Engineering, 06Data-Platform, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Change page.page_links_updated to fixed-length timestamp in wmf wikis - https://phabricator.wikimedia.org/T371742#10088129 (10VirginiaPoundstone) [16:37:36] 06Data-Engineering, 10Data Products (Data products Sprint 18), 13Patch-For-Review: No longer use removed cuc_actiontext column in analytics/refinery - https://phabricator.wikimedia.org/T371099#10088409 (10Milimetric) I have deployed this so it should be picked up by the job that starts on Sep. 1st. I have n... [18:55:40] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Dumps 2.0 (Kanban Board): Implement Airflow Dataset class for RestExternalTaskSensor - https://phabricator.wikimedia.org/T372647#10088767 (10xcollazo) >>! In T372647#10086538, @Ottomata wrote: > Or, just implement `IcebergDataset`, and have `get_sensor_... [19:09:32] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Dumps 2.0 (Kanban Board): Implement Airflow Dataset class for RestExternalTaskSensor - https://phabricator.wikimedia.org/T372647#10088797 (10amastilovic) > Maybe just change the name of the file to data_dependencies.yaml and the module to DataDependency... [20:28:05] (03CR) 10Mforns: [C:03+2] Add MP fragment schema for translation workflows [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/1061096 (https://phabricator.wikimedia.org/T369687) (owner: 10Mforns) [20:28:35] (03Merged) 10jenkins-bot: Add MP fragment schema for translation workflows [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/1061096 (https://phabricator.wikimedia.org/T369687) (owner: 10Mforns)