[00:01:37] 06Data-Engineering, 10Event-Platform: [Epic] Migrate Data Engineering maintained NodeJS repositories to GitLab - https://phabricator.wikimedia.org/T366614#9862176 (10Ottomata) [00:02:59] 06Data-Engineering, 10Event-Platform: Migrate Data Engineering NodeJS library repos to GitLab - https://phabricator.wikimedia.org/T366611#9862178 (10Ottomata) [07:09:46] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting permissions for analytics-privatedata-users (with kerberos) for Mareike Heuer - https://phabricator.wikimedia.org/T364715#9862426 (10MoritzMuehlenhoff) >>! In T364715#9840315, @colewhite wrote: > Added Data Engineering tag fo... [07:25:15] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting permissions for analytics-privatedata-users (with kerberos) for Mareike Heuer - https://phabricator.wikimedia.org/T364715#9862490 (10Reedy) [08:21:41] headsup, I'm going to upgrade mariadb on an-test-coord1001, which will induce some small downtime [08:21:51] but only for the test services [10:04:58] 10Data-Engineering (Q4 2024 April 1st - June 30th), 07Spike: [Status Store] [SPIKE] Investigate and document approach for Iceberg Sensors - https://phabricator.wikimedia.org/T360922#9862956 (10Antoine_Quhen) a:03amastilovic [10:11:53] 10Data-Engineering (Q4 2024 April 1st - June 30th), 07Spike: [Status Store] [SPIKE] Investigate and document approach for Iceberg Sensors - https://phabricator.wikimedia.org/T360922#9862979 (10Antoine_Quhen) We have been working with @amastilovic on a Wikitech page to describe the reasoning behind choosing Air... [10:16:52] !log switching stat100[4-7] into insetup::buster role for T353785 [10:16:55] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [10:16:55] T353785: Decom EOL stats servers stat100[4-7] - https://phabricator.wikimedia.org/T353785 [11:25:30] btullis: o/ there are some timers on clouddumps1001/2 that are trying to rsync from stat1007 and now failing, namely dumps-fetch-wikistats_1.service and dumps-fetch-media-contestwinners.service [11:30:19] taavi: Thanks. I thought I had absented them all. Please feel free to silence temporarily and I will remove the timers later. [11:30:40] thanks, do you want me to create a task? [11:39:26] taavi: Ben is OOO for the rest of the day, but yes please, and I will have a look in his absence [11:41:49] brouberol: thanks, https://phabricator.wikimedia.org/T366694 [11:45:33] ack, on it [11:56:36] all done taavi [12:33:01] 06Data-Engineering, 06Data Products, 10Metrics Platform Backlog: [MPIC] Analyse risk of potential performance issues with static approach to stream configuration - https://phabricator.wikimedia.org/T366627#9863494 (10phuedx) >>! In T366627#9861246, @Ottomata wrote: > All that is to say: we can make some gues... [15:54:50] 06Data-Engineering, 06Data Products, 10Metrics Platform Backlog: [MPIC] Analyse risk of potential performance issues with static approach to stream configuration - https://phabricator.wikimedia.org/T366627#9864329 (10Ottomata) > Are there any query performance optimisations that we couldn't implement because... [15:54:52] 06Data-Engineering, 10Cassandra, 06Data Products: DELETE mechanism for Cassanda Analytics datasets - https://phabricator.wikimedia.org/T366631#9864330 (10xcollazo) > It is not a change that can be made once data is in place though. In other words, now seems like an ideal time; Doing this later would require... [16:12:56] 06Data-Engineering, 06Data Products: Public DataHub - https://phabricator.wikimedia.org/T366720 (10Milimetric) 03NEW [16:33:53] 06Data-Engineering, 06Data Products: Public DataHub - https://phabricator.wikimedia.org/T366720#9864569 (10Milimetric) [17:25:21] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting permissions for analytics-privatedata-users (with kerberos) for Mareike Heuer - https://phabricator.wikimedia.org/T364715#9864928 (10colewhite) The correct link to the docs for setting up kerberos: https://wikitech.wikimedia.o... [17:30:57] !log re-enabling XML dumps on snapshot10[10-13] [17:30:59] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [20:07:01] 10Data-Engineering (Q4 2024 April 1st - June 30th), 07Spike: [Developer Experience] [SPIKE] Investigate process to automate deployment of folders and artifacts to HDFS - https://phabricator.wikimedia.org/T360968#9865599 (10Ahoelzl) a:03amastilovic [20:08:51] 10Data-Engineering (Q4 2024 April 1st - June 30th), 07Spike: [Developer Experience] [SPIKE] Investigate process to automate deployment of folders and artifacts to HDFS - https://phabricator.wikimedia.org/T360968#9865602 (10Ahoelzl) Design work document: https://docs.google.com/document/d/1gytt1rzO5wO1IWmmrGKdf... [20:10:27] 10Data-Engineering (Q4 2024 April 1st - June 30th), 07Spike: [Developer Experience] [SPIKE] Investigate process to automate deployment of folders and artifacts to HDFS - https://phabricator.wikimedia.org/T360968#9865611 (10Ottomata) Q: Have we discussed these ideas with Release Engineering folks? They are cur... [20:36:25] 06Data-Engineering, 10Data Products (Data Products Sprint 14), 10Web-Team-Backlog (FY2023-24 Q4 Sprint 5): Follow-Up Ticket for QA: Validate Sample Rate Adjustments - https://phabricator.wikimedia.org/T365489#9865705 (10KSarabia-WMF) @SToyofuku-WMF Bringing this conversation to Phab in the spirit of our ea... [20:56:11] 06Data-Engineering, 10Data Products (Data Products Sprint 14), 10Web-Team-Backlog (FY2023-24 Q4 Sprint 5): Follow-Up Ticket for QA: Validate Sample Rate Adjustments - https://phabricator.wikimedia.org/T365489#9865726 (10SToyofuku-WMF) a:05SToyofuku-WMF→03None Ah, gotcha! If there's cache impact here, th... [21:19:37] 06Data-Engineering, 10Data-Engineering-Dashiki, 10Data Products (Data Products Sprint 14): publish an announcement about dashiki migratoin - https://phabricator.wikimedia.org/T366755 (10VirginiaPoundstone) 03NEW [21:20:24] 06Data-Engineering, 10Data-Engineering-Dashiki, 10Data Products (Data Products Sprint 14): publish an announcement about dashiki migratoin - https://phabricator.wikimedia.org/T366755#9865770 (10VirginiaPoundstone) [21:29:10] 06Data-Engineering, 10Data-Engineering-Dashiki, 10Data Products (Data Products Sprint 14): publish an announcement about dashiki migration - https://phabricator.wikimedia.org/T366755#9865794 (10VirginiaPoundstone)