[07:45:17] (03PS1) 10Kosta Harlan: ip_reputation: Define properties on items in the tunnels array [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/1038678 (https://phabricator.wikimedia.org/T354597) [07:48:27] (03CR) 10STran: [C:03+2] ip_reputation: Define properties on items in the tunnels array [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/1038678 (https://phabricator.wikimedia.org/T354597) (owner: 10Kosta Harlan) [07:48:57] (03Merged) 10jenkins-bot: ip_reputation: Define properties on items in the tunnels array [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/1038678 (https://phabricator.wikimedia.org/T354597) (owner: 10Kosta Harlan) [08:08:37] karapace1002 has depleted disk space, /var/log/syslog and /var/log/messages have filled the disk with errors like Jun 4 07:46:20 karapace1002 python[668930]: aiohttp.access #011MainThread#011INFO #0110.000616s - "GET /schemas/ids/2?fetchMaxId=false HTTP/1.1" 404 "Java/11.0.22" response=377b request_body=-b [08:14:12] yep, seen, thanks! the disk is filling every couple of hours, after I truncate these messages [08:14:37] I've silenced the alert for now, as I'm working on deprecating the hosts, in T363461 [08:14:37] T363461: Remove the need for karapace by using the schema registry built into DataHub - https://phabricator.wikimedia.org/T363461 [08:18:10] joal: say I wanted to release a configuration change to a datahub ingestion airflow job. Could I do that anytime, or would you prefer it to be on the release train schedue? [08:18:12] *schedule [08:27:20] brouberol: ack [08:31:11] Good morning brouberol - you can dpeloy airflow anytime [08:31:24] morning! Thanks, duly noted [08:31:56] the train is here to help people not to have to deploy refinery-source and refinery too often (they're long to deploy...) but airflow is fast and easy - deploy at will [08:32:30] Just one thing to consider: if when deploying you deploy more than your change, please let other deployed patches owners know [11:37:50] 06Data-Engineering, 10Data Pipelines, 10Data-Platform-SRE (2024.05.27 - 2024.06.16): Upgrade Airflow to 2.9.1 - https://phabricator.wikimedia.org/T365449#9859043 (10Stevemunene) a:03Stevemunene [11:47:47] brouberol: Heya - would you have a minute to merge https://gerrit.wikimedia.org/r/c/operations/puppet/+/1036614 please ? [11:51:59] joal: looking [11:54:26] approved, and I'll +2 + puppet-merge it [11:55:04] all done! [12:30:46] !log delete WikiKube datahub release T361185 [12:30:50] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [12:30:51] T361185: Move datahub to dse-k8s cluster - https://phabricator.wikimedia.org/T361185 [12:57:28] Thank you brouberol :) [13:00:54] 06Data-Engineering, 10Event-Platform: Event Platform schemas should not support type changes to structs as array element or map value types - https://phabricator.wikimedia.org/T366487#9859356 (10Ottomata) The code for this will be similar to {T366562} Whoever works on these could probably do both at the same... [13:08:30] yw :) [14:01:51] 06Data-Engineering, 10[DEPRECATED] wdwb-tech, 10Citoid, 06Content-Transform-Team-WIP, and 10 others: Migrate node-based services in production to node18 - https://phabricator.wikimedia.org/T349118#9859778 (10Jdforrester-WMF) [14:24:10] 06Data-Engineering, 10Event-Platform: Event Platform schemas should not support type changes to structs as array element or map value types - https://phabricator.wikimedia.org/T366487#9859869 (10Ottomata) [14:24:15] 06Data-Engineering, 10Event-Platform: Event Platform schemas should not support type changes to structs as array element or map value types - https://phabricator.wikimedia.org/T366487#9859870 (10Ottomata) [15:10:32] 06Data-Engineering, 06Data Products, 10Dumps 2.0, 10Event-Platform: Rename columns and/or table to abide by the data modeling guidelines - https://phabricator.wikimedia.org/T366542#9860157 (10Ottomata) [15:20:36] 06Data-Engineering, 10Beta-Cluster-Infrastructure, 10Event-Platform: cirrusSearchCheckerJob JobQueueErrors (Could not enqueue jobs) on Beta Cluster - https://phabricator.wikimedia.org/T322491#9860212 (10colewhite) @Jdforrester-WMF @Legoktm Adding the result to the response key has led to `mapper_parsing_exce... [15:24:43] 06Data-Engineering, 10Event-Platform: Migrate Data Engineering NodeJS repos to GitLab - https://phabricator.wikimedia.org/T366611 (10Ottomata) 03NEW [15:25:11] 10Data-Engineering (Q4 2024 April 1st - June 30th), 10Event-Platform, 10GitLab (Pipeline Services MigrationšŸ¤), 13Patch-For-Review: Migrate Data Engineering Pipelinelib repos to GitLab - https://phabricator.wikimedia.org/T344730#9860265 (10Ottomata) [15:26:05] 10Data-Engineering (Q4 2024 April 1st - June 30th), 10Event-Platform, 10GitLab (Pipeline Services MigrationšŸ¤), 13Patch-For-Review: Migrate Data Engineering Pipelinelib repos to GitLab - https://phabricator.wikimedia.org/T344730#9860272 (10Ottomata) [15:26:22] 10Data-Engineering (Q4 2024 April 1st - June 30th), 10Event-Platform, 10GitLab (Pipeline Services MigrationšŸ¤), 13Patch-For-Review: Migrate Data Engineering Pipelinelib repos to GitLab - https://phabricator.wikimedia.org/T344730#9860270 (10Ottomata) I have removed the non PipelineLib repos from this task.... [15:28:15] 06Data-Engineering: Publish Data Engineering maintained NodeJS packages to GitLab and use them in depender code - https://phabricator.wikimedia.org/T366612 (10Ottomata) 03NEW [15:28:54] 06Data-Engineering, 10Event-Platform: Migrate Data Engineering NodeJS repos to GitLab - https://phabricator.wikimedia.org/T366611#9860296 (10Ottomata) [15:32:35] 06Data-Engineering, 10Event-Platform: Migrate Data Engineering NodeJS repos to GitLab - https://phabricator.wikimedia.org/T366611#9860327 (10Ottomata) [15:33:11] 06Data-Engineering: Publish Data Engineering maintained NodeJS packages to GitLab and use them in depender code - https://phabricator.wikimedia.org/T366612#9860329 (10Ottomata) [15:37:01] 10Data-Engineering (Q4 2024 April 1st - June 30th), 10Event-Platform, 10GitLab (Pipeline Services MigrationšŸ¤), 13Patch-For-Review: Migrate Data Engineering Pipelinelib repos to GitLab - https://phabricator.wikimedia.org/T344730#9860349 (10Ottomata) [15:38:09] 06Data-Engineering: [Epic] Migrate Data Engineering maintained NodeJS repositories to GitLab - https://phabricator.wikimedia.org/T366614 (10Ottomata) 03NEW [15:38:30] 06Data-Engineering: [Epic] Migrate Data Engineering maintained NodeJS repositories to GitLab - https://phabricator.wikimedia.org/T366614#9860366 (10Ottomata) [15:38:34] 06Data-Engineering, 10Event-Platform: Migrate Data Engineering NodeJS repos to GitLab - https://phabricator.wikimedia.org/T366611#9860367 (10Ottomata) [15:38:39] 10Data-Engineering (Q4 2024 April 1st - June 30th), 10Event-Platform, 10GitLab (Pipeline Services MigrationšŸ¤), 13Patch-For-Review: Migrate Data Engineering Pipelinelib repos to GitLab - https://phabricator.wikimedia.org/T344730#9860368 (10Ottomata) [15:38:57] 06Data-Engineering: Create gitlab ci npm publish pipeline and job in workflow_utils gitlab_ci_templates - https://phabricator.wikimedia.org/T366537#9860370 (10Ottomata) [15:38:58] 06Data-Engineering, 10Event-Platform: Migrate Data Engineering NodeJS repos to GitLab - https://phabricator.wikimedia.org/T366611#9860369 (10Ottomata) [15:39:19] 06Data-Engineering, 10Event-Platform: Migrate Data Engineering NodeJS repos to GitLab - https://phabricator.wikimedia.org/T366611#9860371 (10Ottomata) a:03Snwachukwu [15:40:21] 06Data-Engineering, 10Event-Platform: Migrate Data Engineering NodeJS library repos to GitLab - https://phabricator.wikimedia.org/T366611#9860372 (10Ottomata) [15:42:35] 06Data-Engineering, 10Event-Platform: Migrate Data Engineering NodeJS library repos to GitLab - https://phabricator.wikimedia.org/T366611#9860380 (10Ottomata) [15:53:02] 10Data-Engineering (Q4 2024 April 1st - June 30th), 10Event-Platform, 10GitLab (Pipeline Services MigrationšŸ¤), 13Patch-For-Review: Migrate Data Engineering Pipelinelib repos to GitLab - https://phabricator.wikimedia.org/T344730#9860412 (10Ottomata) [15:58:32] 10Quarry, 10PAWS: update github action - https://phabricator.wikimedia.org/T348873#9860452 (10taavi) [16:02:51] 06Data-Engineering, 10Beta-Cluster-Infrastructure, 10Event-Platform: cirrusSearchCheckerJob JobQueueErrors (Could not enqueue jobs) on Beta Cluster - https://phabricator.wikimedia.org/T322491#9860507 (10Jdforrester-WMF) >>! In T322491#9860211, @colewhite wrote: > @Jdforrester-WMF @Legoktm @Ottomata Adding th... [16:11:33] 06Data-Engineering, 06Data Products, 10Dumps 2.0, 10Event-Platform: Rename columns and/or table to abide by the data modeling guidelines - https://phabricator.wikimedia.org/T366542#9860601 (10Ottomata) > decided to stick closely with the MediaWiki db field names E.g. we went with rev_id and rev_dt instead... [16:16:18] 06Data-Engineering, 10Data Products (Data Products Sprint 14), 10Web-Team-Backlog (FY2023-24 Q4 Sprint 5): Follow-Up Ticket for QA: Validate Sample Rate Adjustments - https://phabricator.wikimedia.org/T365489#9860633 (10SToyofuku-WMF) a:03SToyofuku-WMF [16:27:53] 06Data-Engineering, 06Data-Platform-SRE, 06Infrastructure-Foundations, 10Event-Platform: > ~1 request/second to intake-logging.wikimedia.org times out at the traffic/service interface - https://phabricator.wikimedia.org/T264021#9860683 (10CDanis) 05Openā†’03Resolved a:03CDanis I think you are right... [16:40:59] 06Data-Engineering, 10Beta-Cluster-Infrastructure, 10Event-Platform: cirrusSearchCheckerJob JobQueueErrors (Could not enqueue jobs) on Beta Cluster - https://phabricator.wikimedia.org/T322491#9860748 (10colewhite) >>! In T322491#9860507, @Jdforrester-WMF wrote: > Ah, is 'response' a reserved key? `response`... [17:42:28] 06Data-Engineering, 06Data Products, 10Metrics Platform Backlog: [MPIC] Analyse risk of potential performance issues with static approach to stream configuration - https://phabricator.wikimedia.org/T366627 (10WDoranWMF) 03NEW p:05Triageā†’03High [17:51:46] 06Data-Engineering, 10Cassandra, 06Data Products: DELETE mechanism for Cassanda Analytics datasets - https://phabricator.wikimedia.org/T366631 (10xcollazo) 03NEW [18:08:13] 06Data-Engineering, 10Beta-Cluster-Infrastructure, 10Event-Platform, 13Patch-For-Review: cirrusSearchCheckerJob JobQueueErrors (Could not enqueue jobs) on Beta Cluster - https://phabricator.wikimedia.org/T322491#9861131 (10Ottomata) Related: {T363587} [18:39:42] 06Data-Engineering, 10Cassandra, 06Data Products: DELETE mechanism for Cassanda Analytics datasets - https://phabricator.wikimedia.org/T366631#9861232 (10Eevans) Just for posterity sake: Some of the Commons Impact Metrics tables already accommodate doing range deletes. Those that do not could be made to do... [18:44:01] 06Data-Engineering, 06Data Products, 10Metrics Platform Backlog: [MPIC] Analyse risk of potential performance issues with static approach to stream configuration - https://phabricator.wikimedia.org/T366627#9861247 (10Ottomata) I think the metric we need here mostly is: how will fewer tables affect dashboard... [18:44:02] 06Data-Engineering, 06Data Products, 10Metrics Platform Backlog: [MPIC] Analyse risk of potential performance issues with static approach to stream configuration - https://phabricator.wikimedia.org/T366627#9861248 (10Ottomata) [18:44:04] 10Data-Engineering (Q4 2024 April 1st - June 30th): [Datasets Config][Spike] Understand and document the details and conflicts between Datasets Config, Refine refactor, Dynamic EventStreamConfig, and Metrics Platform Instrumentation Configurator - https://phabricator.wikimedia.org/T361853#9861249 (10Ottomata) [21:28:57] 10Data-Engineering (Q4 2024 April 1st - June 30th): Move reportupdater reports away from their local filesystem locations - https://phabricator.wikimedia.org/T365382#9861849 (10amastilovic) 05Openā†’03Resolved This ticket has been resolved, the tasks from the ticket definition have been performed on `an-la... [23:19:43] 10Data-Engineering (Q4 2024 April 1st - June 30th): Implement automatic sync of refinery HQL files to HDFS - https://phabricator.wikimedia.org/T365659#9862120 (10amastilovic)