[08:32:49] 10serviceops, 10Prod-Kubernetes, 10Kubernetes: Limit the concurrency of envoy in service mesh - https://phabricator.wikimedia.org/T354532 (10JMeybohm) Another thing we saw a significant change with different limits/concurrency in is `envoy_cluster_upstream_cx_connect_ms` (Connection establishment) - all whil... [12:47:45] 10serviceops, 10Thumbor: Error accessing File:KlimtDieJungfrau.jpg after it was moved to the Main Page on enwiki - https://phabricator.wikimedia.org/T354858 (10hnowlan) [12:49:35] 10serviceops, 10Thumbor: Error accessing File:KlimtDieJungfrau.jpg after it was moved to the Main Page on enwiki - https://phabricator.wikimedia.org/T354858 (10hnowlan) Currently the 429 errors are because there have been too many failures for this image, and that throttling is masking the real error source -... [12:50:24] 10serviceops, 10Thumbor: Error accessing File:KlimtDieJungfrau.jpg after it was moved to the Main Page on enwiki - https://phabricator.wikimedia.org/T354858 (10hnowlan) Did the image previously have valid thumbnails *before* it was moved to the front page? [13:35:00] 10serviceops, 10SRE, 10Data-Platform-SRE (2023/24 Q3 Milestone 1), 10Discovery-Search (Current work): SUP: Partition update_pipeline kafka topic - https://phabricator.wikimedia.org/T354064 (10Gehel) 05In progress→03Resolved [13:52:19] 10serviceops, 10Content-Transform-Team-WIP, 10RESTBase Sunsetting, 10Wikifeeds, 10Epic: Switchover outgoing wikifeeds parsoid requests - https://phabricator.wikimedia.org/T347027 (10Jgiannelos) 05Open→03Resolved [13:57:40] 10serviceops, 10Prod-Kubernetes, 10Data-Platform-SRE (2023/24 Q3 Milestone 1), 10Kubernetes, 10Patch-For-Review: Improve how we address outside k8s infrastructure from within charts (e.g. network policies) - https://phabricator.wikimedia.org/T331894 (10brouberol) I hijacked the kubestaging `mw-debug` for... [14:36:04] 10serviceops, 10Thumbor: Error accessing File:KlimtDieJungfrau.jpg after it was moved to the Main Page on enwiki - https://phabricator.wikimedia.org/T354858 (10A_smart_kitten) @hnowlan I believe it did - I briefly saw a thumbnail myself at one point, and onwiki it was reported that "there were no issues when [... [14:43:58] 10serviceops, 10Infrastructure-Foundations, 10Puppet-Core, 10SRE, and 5 others: Migrate roles to puppet7 - https://phabricator.wikimedia.org/T349619 (10Gehel) [17:13:05] 10serviceops, 10MediaWiki-General, 10MediaWiki-libs-Stats, 10SRE, and 4 others: MediaWiki Prometheus support - https://phabricator.wikimedia.org/T240685 (10lmata) [17:15:33] 10serviceops, 10Prod-Kubernetes: PodSecurityPolicies will be deprecated with Kubernetes 1.21 - https://phabricator.wikimedia.org/T273507 (10JMeybohm) a:03JMeybohm >>! In T273507#9305083, @elukey wrote: > I would personally try to spend some time in understanding the ValidationAdmissionPolicy feature before s... [17:39:12] 10serviceops, 10MediaWiki-General, 10MediaWiki-libs-Stats, 10SRE, and 4 others: MediaWiki Prometheus support - https://phabricator.wikimedia.org/T240685 (10lmata) [22:16:51] 10serviceops, 10API Platform, 10CirrusSearch, 10MediaWiki-Configuration, 10Discovery-Search (Current work): Create new NetworkSession mediawiki extension - https://phabricator.wikimedia.org/T354976 (10EBernhardson)