[07:59:58] 10serviceops, 10Wikimedia Enterprise, 10Wikimedia-Performance-recommendation, 10affects-Kiwix-and-openZIM: large amount of traffic to the action=parse API from MWOffliner - https://phabricator.wikimedia.org/T324866 (10vadim-kovalenko) @cscott , could you explain distinguishes between `parsoid=1` and `usepa... [10:12:43] 10serviceops, 10DC-Ops, 10SRE, 10ops-codfw: Q1:rack/setup/install kubernetes20[25-54] - https://phabricator.wikimedia.org/T342534 (10Volans) >>! In T342534#9137952, @Papaul wrote: > @jbond @Volans on 2027 - 2029 > puppet is failing with > ` > ----- OUTPUT of 'run-puppet-agent --quiet' -----... [10:31:42] 10serviceops, 10Data-Persistence, 10Performance-Team, 10SRE, 10Datacenter-Switchover: September 2023 Datacenter Switchover - https://phabricator.wikimedia.org/T345263 (10kamila) [10:32:36] 10serviceops, 10Foundational Technology Requests, 10Prod-Kubernetes, 10Kubernetes: Kubernetes v1.23 use PKI for service-account signing (instead of cergen) - https://phabricator.wikimedia.org/T329826 (10JMeybohm) [10:32:38] 10serviceops: Allow for multiple confd instances in puppet - https://phabricator.wikimedia.org/T341669 (10JMeybohm) 05Open→03Resolved Thanks to @jbond's refactor this is now resolved (again). [10:34:11] 10serviceops, 10Content-Transform-Team-WIP, 10Mobile-Content-Service, 10RESTBase Sunsetting, and 3 others: Setup allowed list for MCS decom - https://phabricator.wikimedia.org/T340036 (10MSantos) [10:34:25] 10serviceops, 10Content-Transform-Team-WIP, 10Mobile-Content-Service, 10RESTBase Sunsetting, and 3 others: Setup allowed list for MCS decom - https://phabricator.wikimedia.org/T340036 (10MSantos) 05Open→03Resolved [10:35:11] 10serviceops, 10Prod-Kubernetes, 10observability, 10Kubernetes: Increase visibility of container/pod ressource exhaustion - https://phabricator.wikimedia.org/T266216 (10kamila) [10:35:19] 10serviceops, 10Kubernetes, 10User-jijiki: Deploy kube-state-metrics - https://phabricator.wikimedia.org/T264625 (10kamila) 05In progress→03Stalled on the back burner while I'm busy freaking out about the DC switchover (T345263) [11:03:43] 10serviceops, 10DC-Ops, 10SRE, 10ops-codfw, 10Patch-For-Review: Q1:rack/setup/install kubernetes20[25-54] - https://phabricator.wikimedia.org/T342534 (10JMeybohm) >>! In T342534#9137952, @Papaul wrote: > [...] > so 2025 and 2026 nodes had 2 roles, insetup and kubernetes::worker roles That was overlook... [11:11:07] 10serviceops, 10DC-Ops, 10SRE, 10ops-codfw: Q1:rack/setup/install kubernetes20[25-54] - https://phabricator.wikimedia.org/T342534 (10jbond) FYi i just checked kubernetes2025 (via install-console), kubernetes2027 and kubernetes2029 and puppet seems to be running well now [11:34:14] 10serviceops: [Fallback Task] Upgrade the MediaWiki servers to ICU 67 - https://phabricator.wikimedia.org/T345561 (10akosiaris) [11:34:41] 10serviceops: [Fallback Task] Upgrade the MediaWiki servers to ICU 67 - https://phabricator.wikimedia.org/T345561 (10akosiaris) [13:21:10] 10serviceops, 10Parsoid (Tracking), 10Patch-For-Review: Upgrade nodejs on testreduce1001 - https://phabricator.wikimedia.org/T345220 (10MoritzMuehlenhoff) >>! In T345220#9137002, @ssastry wrote: > It may just be carry over from the times when we ran both kinds of tests on scandium ... but that has changed si... [14:29:47] 10serviceops, 10SRE, 10CommRel-Specialists-Support (Jul-Sep-2023), 10Datacenter-Switchover: CommRel support for September 2023 Datacenter Switchover - https://phabricator.wikimedia.org/T345265 (10Trizek-WMF) >>! In T345265#9134920, @kamila wrote: > Thank you @Trizek-WMF ! The message looks good. Maybe I'd... [14:32:35] 10serviceops, 10SRE, 10CommRel-Specialists-Support (Jul-Sep-2023), 10Datacenter-Switchover: CommRel support for September 2023 Datacenter Switchover - https://phabricator.wikimedia.org/T345265 (10kamila) Thank you @Trizek-WMF, sounds good! I will ping you regarding translations :-) [15:26:19] 10serviceops, 10SRE, 10CommRel-Specialists-Support (Jul-Sep-2023), 10Datacenter-Switchover: CommRel support for September 2023 Datacenter Switchover - https://phabricator.wikimedia.org/T345265 (10Trizek-WMF) [15:40:33] 10serviceops, 10CirrusSearch, 10MediaWiki-Configuration, 10MediaWiki-Platform-Team, 10Discovery-Search (Current work): Provide a method for internal services to run api requests in a private context - https://phabricator.wikimedia.org/T345185 (10Gehel) [15:44:29] 10serviceops, 10SRE, 10CommRel-Specialists-Support (Jul-Sep-2023), 10Datacenter-Switchover: CommRel support for September 2023 Datacenter Switchover - https://phabricator.wikimedia.org/T345265 (10Trizek-WMF) [17:02:38] 10serviceops, 10MediaWiki-General, 10MediaWiki-libs-Stats, 10SRE, and 5 others: MediaWiki Prometheus support - https://phabricator.wikimedia.org/T240685 (10Krinkle) [18:02:50] 10serviceops, 10SRE, 10Datacenter-Switchover: Switchover cookbooks live test - https://phabricator.wikimedia.org/T345588 (10kamila) [19:56:01] 10serviceops, 10CirrusSearch, 10MediaWiki-Configuration, 10MediaWiki-Platform-Team, 10Discovery-Search (Current work): Provide a method for internal services to run api requests in a private context - https://phabricator.wikimedia.org/T345185 (10Tgr) I would maybe create a small authentication extension...