[05:02:32] 06Data-Engineering, 06Data Products, 06DBA, 07Schema-change-in-production: Cleanup revision table schema - https://phabricator.wikimedia.org/T367856#9959574 (10Marostegui) [05:32:24] 06Data-Engineering, 10MediaWiki-General, 10Event-Platform, 10MediaWiki-Platform-Team (Radar): Create legacy EventLogging proxy HTTP intake (for MediaWikiPingback) endpoint to EventGate - https://phabricator.wikimedia.org/T353817#9959609 (10Joe) Personal opinion, we should never ever turn off `DirectorySlas... [08:16:39] hello folks! [08:18:15] I'd need to rollout a new version of the krb packages across most of the DE ndoes [08:18:18] *nodes [08:18:32] so far I tested it on stat1010 and I can knit and dfs ls on hadoop [08:19:46] I've also rolled it out to an-worker108* [08:19:58] if you want to add more canary nodes lemme know [08:20:20] I'd say that we can wait until this afternoon and if nothing pops up, we can roll it out everywhere [08:20:30] (last but not least I'll take care of the krb* nodes as well) [08:25:45] elukey: Thanks so much. I'd say feel free to proceed as you wish, we don't need any more canaries. [08:26:43] btullis: super thanks, I'll probably ping you before the krb masters since my knowledge is a bit rusty :) [08:27:07] I'll be rebooting two journal nodes sequentially this morning. an-worker1080 and 1090, just in case we collide. [08:27:33] elukey: Sure thing. Feel free to ping whenever you like. [08:44:48] 06Data-Engineering, 10Dumps-Generation, 06SRE, 10Data Products (Data Products Sprint 16), and 2 others: Dumps generation without prefetch cause disruption to the production environment - https://phabricator.wikimedia.org/T368098#9959951 (10elukey) Folks today I found snapshot1017 with puppet disable for mo... [08:46:19] 06Data-Engineering, 10Dumps-Generation, 06SRE, 10Data Products (Data Products Sprint 16), and 2 others: Dumps generation without prefetch cause disruption to the production environment - https://phabricator.wikimedia.org/T368098#9959956 (10Marostegui) I don't think we should be leaving a host with puppet d... [10:12:46] 06Data-Engineering, 07Kubernetes: Migrate dse cluster off of Pod Security Policies - https://phabricator.wikimedia.org/T369492 (10JMeybohm) 03NEW [10:21:06] !log rolling out conda-analytics version 0.0.35 for T369240 and T369210 [10:21:10] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [10:21:10] T369240: NEW BUG REPORT Update conda-analytics package specifications - https://phabricator.wikimedia.org/T369240 [10:21:11] T369210: NEW BUG REPORT conda-analytics-clone creates environments named with a trailing brace - https://phabricator.wikimedia.org/T369210 [10:24:33] 06Data-Engineering, 06Data Products, 06DBA, 07Schema-change-in-production: Cleanup revision table schema - https://phabricator.wikimedia.org/T367856#9960311 (10Marostegui) [11:34:40] 06Data-Engineering, 06Data Products, 06DBA, 07Schema-change-in-production: Cleanup revision table schema - https://phabricator.wikimedia.org/T367856#9960515 (10Marostegui) [13:25:41] btullis: all nodes got the krb5 update, krb2002 too.. afaics no service restart is needed, so I'll wait a bit and then proceed with 1001 too [13:26:07] elukey: Great, many thanks. [14:12:03] 14Analytics: Requesting Kerberos access for xiaoxiao - https://phabricator.wikimedia.org/T369517 (10XiaoXiao-WMF) 03NEW [14:47:00] 06Data-Engineering, 06Data Products, 10Metrics Platform Backlog: [MPIC] Analyse risk of potential performance issues with static approach to stream configuration - https://phabricator.wikimedia.org/T366627#9961361 (10Ottomata) We discussed in Data Eng Sync meeting today. We will eventually support custom ta... [14:52:33] krb nodes upgraded, lemme know if you see anything weird [14:58:26] 14Analytics, 10AQS2.0, 06Tech-Docs-Team, 10Data Products (Epics Timeline), and 2 others: AQS 2.0 user documentation - https://phabricator.wikimedia.org/T288664#9961400 (10KBach) [15:04:29] 06Data-Engineering, 10Dumps-Generation, 06SRE, 10Data Products (Data Products Sprint 16), and 2 others: Dumps generation without prefetch cause disruption to the production environment - https://phabricator.wikimedia.org/T368098#9961423 (10xcollazo) >>! In T368098#9959951, @elukey wrote: > Folks today I fo... [15:12:46] 06Data-Engineering, 10Dumps-Generation, 06SRE, 10Data Products (Data Products Sprint 16), and 2 others: Dumps generation without prefetch cause disruption to the production environment - https://phabricator.wikimedia.org/T368098#9961448 (10xcollazo) >>! In T368098#9953045, @Ladsgroup wrote: >>>! In T368098... [15:14:02] 06Data-Engineering, 03Discovery-Search (Current work), 07Wikimedia-production-error: '.event.pageViewId' should be string, '.event.subTest' should be string, '.event.searchSessionId' should be string - https://phabricator.wikimedia.org/T286814#9961453 (10pfischer) 05Open→03In progress a:03pfischer [15:17:42] 06Data-Engineering, 10CirrusSearch, 06Data Products, 06Discovery-Search, and 3 others: Error: Call to a member function getPageAsLinkTarget() on null - https://phabricator.wikimedia.org/T368543#9961473 (10Gehel) [15:17:53] 10Data-Engineering (Q4 2024 April 1st - June 30th), 06Release-Engineering-Team, 07Spike: [Developer Experience] [SPIKE] Investigate process to automate deployment of folders and artifacts to HDFS - https://phabricator.wikimedia.org/T360968#9961474 (10Ottomata) Good q, I'm not certain. @amastilovic ? The tr... [15:18:44] 06Data-Engineering, 10CirrusSearch, 06Data Products, 10MediaWiki-extensions-EventLogging, and 2 others: Error: Call to a member function getPageAsLinkTarget() on null - https://phabricator.wikimedia.org/T368543#9961478 (10Gehel) This is about event logging, removing #discovery-search [15:22:09] 10Data-Engineering (Q4 2024 April 1st - June 30th), 10Event-Platform, 10GitLab (Pipeline Services Migration🐤), 13Patch-For-Review: Migrate Data Engineering Pipelinelib repos to GitLab - https://phabricator.wikimedia.org/T344730#9961512 (10Snwachukwu) 05Open→03Resolved [15:22:38] 06Data-Engineering, 10MediaWiki-General, 10Event-Platform, 10MediaWiki-Platform-Team (Radar): Create legacy EventLogging proxy HTTP intake (for MediaWikiPingback) endpoint to EventGate - https://phabricator.wikimedia.org/T353817#9961518 (10Ottomata) Great, I can do that. Thank you! [16:35:04] 06Data-Engineering, 10MediaWiki-General, 10Event-Platform, 10MediaWiki-Platform-Team (Radar): Create legacy EventLogging proxy HTTP intake (for MediaWikiPingback) endpoint to EventGate - https://phabricator.wikimedia.org/T353817#9961942 (10Ottomata) > Move docroot/mediawiki.org/beacon/event/index.php to w/... [16:36:07] 14Analytics: Requesting Kerberos access for xiaoxiao - https://phabricator.wikimedia.org/T369517#9961949 (10Aklapper) [16:39:20] 06Data-Engineering: Requesting Kerberos access for xiaoxiao - https://phabricator.wikimedia.org/T369517#9961968 (10Aklapper) [17:36:10] 06Data-Engineering, 10MediaWiki-General, 10Event-Platform, 10MediaWiki-Platform-Team (Radar), 13Patch-For-Review: Create legacy EventLogging proxy HTTP intake (for MediaWikiPingback) endpoint to EventGate - https://phabricator.wikimedia.org/T353817#9962222 (10phuedx) >>! In T353817#9961942, @Ottomata wro... [17:41:21] 06Data-Engineering, 10MediaWiki-General, 10Event-Platform, 10MediaWiki-Platform-Team (Radar), 13Patch-For-Review: Create legacy EventLogging proxy HTTP intake (for MediaWikiPingback) endpoint to EventGate - https://phabricator.wikimedia.org/T353817#9962321 (10Ottomata) Indeed! I should have said: "in c... [17:45:32] 06Data-Engineering: Requesting Kerberos access for xiaoxiao - https://phabricator.wikimedia.org/T369517#9962338 (10Ottomata) Since the request is kerberos access, I'm assuming the real request is for [[ https://wikitech.wikimedia.org/wiki/Data_Platform/Data_access#All_of_the_above | full Data Platform access ]]... [18:30:50] 06Data-Engineering, 06Product-Analytics, 10Data-Platform-SRE (2024.07.08 - 2024.07.28): Publishing conda environments with WMF Data Workflow Utils is broken - https://phabricator.wikimedia.org/T367848#9962520 (10Gehel) [18:34:04] 06Data-Engineering, 10Data Pipelines, 06Release-Engineering-Team, 10Data-Platform-SRE (2024.07.08 - 2024.07.28), 13Patch-For-Review: Upgrade Airflow to 2.9.2 - https://phabricator.wikimedia.org/T365449#9962539 (10Gehel) [18:35:29] 06Data-Engineering, 10Cassandra, 10Data Pipelines, 10Data-Platform-SRE (2024.07.08 - 2024.07.28): Create puppet defined type for adding/updating/deleting secrets or other small files on HDFS - https://phabricator.wikimedia.org/T323692#9962557 (10Gehel) [18:56:02] 06Data-Engineering, 10MediaWiki-General, 10Event-Platform, 10MediaWiki-Platform-Team (Radar), 13Patch-For-Review: Create legacy EventLogging proxy HTTP intake (for MediaWikiPingback) endpoint to EventGate - https://phabricator.wikimedia.org/T353817#9962622 (10Krinkle) >>! In T353817#9961942, @Ottomata wr... [19:38:13] 06Data-Engineering, 10MediaWiki-General, 10Event-Platform, 10MediaWiki-Platform-Team (Radar), 13Patch-For-Review: Create legacy EventLogging proxy HTTP intake (for MediaWikiPingback) endpoint to EventGate - https://phabricator.wikimedia.org/T353817#9962760 (10Ottomata) > If SRE don't want this as standal... [20:14:45] 10Quarry: Deduplicate config load - https://phabricator.wikimedia.org/T349135#9962915 (10SD0001) 05Open→03Resolved a:03SD0001 [20:17:25] 10Quarry, 07good first task, 07Regression: Bad resultset number case is not handled - https://phabricator.wikimedia.org/T218470#9962923 (10SD0001) [20:17:48] 10Quarry: Error in web instances. - https://phabricator.wikimedia.org/T362157#9962921 (10SD0001) →14Duplicate dup:03T218470 [21:24:29] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: add approvers to analytics-research-admins - https://phabricator.wikimedia.org/T368435#9963118 (10Dzahn) Also thanks @Volans for the details and suggesting to add docs to Wikitech [21:27:07] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: add approvers to analytics-research-admins - https://phabricator.wikimedia.org/T368435#9963110 (10Dzahn) 05Open→03Resolved a:03Dzahn Thank you @Miriam! You and Xiao Xiao have been added to the code base. So far this isn't happe... [23:19:27] 14Analytics, 10AQS2.0, 06Tech-Docs-Team, 10Data Products (Epics Timeline), and 2 others: AQS 2.0 user documentation - https://phabricator.wikimedia.org/T288664#9963476 (10apaskulin) [23:34:10] 10Data-Engineering (Q4 2024 April 1st - June 30th), 06Release-Engineering-Team, 07Spike: [Developer Experience] [SPIKE] Investigate process to automate deployment of folders and artifacts to HDFS - https://phabricator.wikimedia.org/T360968#9963593 (10amastilovic) I've considered the option of pulling from th...