[10:23:35] 06serviceops, 10ChangeProp, 10Page Content Service, 13Patch-For-Review: Investigate significant changeprop backlogs after PCS migration - https://phabricator.wikimedia.org/T397072#10978379 (10hnowlan) T398243 is probably related to this issue [10:24:23] 06serviceops, 10ChangeProp, 10Page Content Service, 13Patch-For-Review: Investigate significant changeprop backlogs after PCS migration - https://phabricator.wikimedia.org/T397072#10978382 (10hnowlan) p:05Triage→03High [12:04:47] 06serviceops, 06Infrastructure-Foundations, 10netops: lsw1-a8-codfw: fpc0 PFE Statistics received unknown trigger (type Semaphore, id 0) - https://phabricator.wikimedia.org/T398433#10978630 (10akosiaris) [12:30:36] 06serviceops, 06Infrastructure-Foundations, 10netops: lsw1-a8-codfw: fpc0 PFE Statistics received unknown trigger (type Semaphore, id 0) - https://phabricator.wikimedia.org/T398433#10978750 (10ayounsi) 05Open→03Resolved a:03ayounsi {F63349871} Much better. Thanks for the depool, you can repool th... [12:34:55] 06serviceops, 06Infrastructure-Foundations, 10netops: lsw1-a8-codfw: fpc0 PFE Statistics received unknown trigger (type Semaphore, id 0) - https://phabricator.wikimedia.org/T398433#10978765 (10ops-monitoring-bot) Cookbook cookbooks.sre.k8s.pool-depool-node started by akosiaris@cumin1003 pool for host wik... [12:35:36] 06serviceops, 06Infrastructure-Foundations, 10netops: lsw1-a8-codfw: fpc0 PFE Statistics received unknown trigger (type Semaphore, id 0) - https://phabricator.wikimedia.org/T398433#10978766 (10ops-monitoring-bot) Cookbook cookbooks.sre.k8s.pool-depool-node started by akosiaris@cumin1003 pool for host wik... [12:46:00] 06serviceops, 06Infrastructure-Foundations, 10netops: lsw1-a8-codfw: fpc0 PFE Statistics received unknown trigger (type Semaphore, id 0) - https://phabricator.wikimedia.org/T398433#10978804 (10akosiaris) wikikube workers repooled. [12:47:32] 06serviceops, 06Infrastructure-Foundations, 10netops: lsw1-a8-codfw: fpc0 PFE Statistics received unknown trigger (type Semaphore, id 0) - https://phabricator.wikimedia.org/T398433#10978810 (10Ladsgroup) db2146 is also repooling [13:29:11] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: hw troubleshooting: Backplane error for wikikube-worker1069.eqiad.wmnet - https://phabricator.wikimedia.org/T397829#10978939 (10Jclark-ctr) Replaced Motherboard /backplane and / backplane cable Server is back up and can be repooled @Clement_Goubert [13:29:18] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: hw troubleshooting: Backplane error for wikikube-worker1069.eqiad.wmnet - https://phabricator.wikimedia.org/T397829#10978941 (10Jclark-ctr) 05Open→03Resolved [13:48:14] 06serviceops, 10Prod-Kubernetes, 06Traffic, 07Kubernetes, 13Patch-For-Review: Handling inbound IPIP traffic on low traffic LVS k8s based realservers - https://phabricator.wikimedia.org/T352956#10979047 (10akosiaris) All kubernetes clusters are now configured to use MTU 1460. This will take some time (wee... [13:48:19] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: hw troubleshooting: Backplane error for wikikube-worker1069.eqiad.wmnet - https://phabricator.wikimedia.org/T397829#10979051 (10Clement_Goubert) Host repooled, thanks! [13:49:07] 06serviceops, 10Prod-Kubernetes, 06Traffic, 07Kubernetes, 13Patch-For-Review: Handling inbound IPIP traffic on low traffic LVS k8s based realservers - https://phabricator.wikimedia.org/T352956#10979052 (10Vgutierrez) awesome news, thanks @akosiaris [15:10:24] 06serviceops, 10Page Content Service, 10RESTBase Sunsetting, 07Code-Health-Objective, and 2 others: Move PCS endpoints behind API Gateway - https://phabricator.wikimedia.org/T264670#10979692 (10Jgiannelos) 05Open→03Resolved a:03Jgiannelos PCS is not serving any traffic via RB. [17:06:49] 06serviceops, 06MediaWiki-Platform-Team, 07Epic: Migrate Wikimedia production from PHP 8.1 to PHP 8.3 - https://phabricator.wikimedia.org/T360995#10980160 (10Jdforrester-WMF) [17:06:50] 06serviceops, 13Patch-For-Review: Prepare PHP 8.3 production images - https://phabricator.wikimedia.org/T398246#10980161 (10Jdforrester-WMF) [17:07:02] 06serviceops, 06MediaWiki-Platform-Team, 07Epic: Migrate Wikimedia production from PHP 8.1 to PHP 8.3 - https://phabricator.wikimedia.org/T360995#10980162 (10Jdforrester-WMF) [18:23:11] 06serviceops, 10MW-on-K8s, 10Data-Platform-SRE (2025.07.05 - 2025.07.25), 10Discovery-Search (2025.06.13 - 2025.07.04), 10MW-1.45-notes (1.45.0-wmf.8; 2025-07-01): Investigate EQIAD daily completion suggester rebuild failure - https://phabricator.wikimedia.org/T395465#10980340 (10EBernhardson) This looks...