[08:32:57] 10serviceops, 10Prod-Kubernetes, 10Kubernetes: Reserve resources for system daemons on kubernetes nodes - https://phabricator.wikimedia.org/T277876 (10JMeybohm) 05Open→03Stalled To complete this (e.g. remove the cap of 2 CPUs) we need to add more hardware as of: * {T343996} * {T343993} * {T341265} * {T... [08:33:00] 10serviceops, 10Prod-Kubernetes, 10Shared-Data-Infrastructure, 10Kubernetes: Update Kubernetes clusters to >1.25 - https://phabricator.wikimedia.org/T341984 (10JMeybohm) [08:33:05] 10serviceops, 10Foundational Technology Requests, 10Prod-Kubernetes, 10Shared-Data-Infrastructure, and 2 others: Update Kubernetes clusters to v1.23 - https://phabricator.wikimedia.org/T307943 (10JMeybohm) [08:33:10] 10serviceops, 10MW-on-K8s, 10SRE, 10Traffic, 10Release-Engineering-Team (Seen): Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10JMeybohm) [10:24:00] 10serviceops, 10SRE, 10ops-codfw: Decommission thumbor200[3456] - https://phabricator.wikimedia.org/T344597 (10hnowlan) [10:25:47] 10serviceops, 10SRE, 10ops-eqiad: Decommission thumbor100[1256] - https://phabricator.wikimedia.org/T344598 (10hnowlan) [10:46:32] 10serviceops, 10MW-on-K8s, 10Observability-Logging, 10SRE: Keep calculating latencies for MediaWiki requests in the WikiKube environment - https://phabricator.wikimedia.org/T276095 (10kamila) a:03kamila [11:02:32] 10serviceops, 10SRE, 10ops-eqiad: Decommission thumbor100[1256] - https://phabricator.wikimedia.org/T344598 (10ops-monitoring-bot) cookbooks.sre.hosts.decommission executed by hnowlan@cumin1001 for hosts: `thumbor[1001-1002,1005-1006].eqiad.wmnet` - thumbor1001.eqiad.wmnet (**PASS**) - Downtimed host on Ic... [11:04:06] 10serviceops, 10SRE, 10ops-eqiad: Decommission thumbor100[1256] - https://phabricator.wikimedia.org/T344598 (10hnowlan) [11:35:37] 10serviceops, 10SRE, 10ops-codfw: Decommission thumbor200[3456] - https://phabricator.wikimedia.org/T344597 (10ops-monitoring-bot) cookbooks.sre.hosts.decommission executed by hnowlan@cumin1001 for hosts: `thumbor[2003-2006].codfw.wmnet` - thumbor2003.codfw.wmnet (**PASS**) - Downtimed host on Icinga/Alert... [14:05:19] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: mw-on-k8s app container CPU throttling at low average load - https://phabricator.wikimedia.org/T342748 (10Clement_Goubert) All deployments of mw-on-k8s are now using: - Autocomputed CPU requests, no limits - Autocomputed Memory requests and limits Only exceptio... [14:39:42] 10serviceops, 10MW-on-K8s, 10Wikidata, 10Wikidata-Termbox, and 2 others: Migrate termbox to mw-api-int - https://phabricator.wikimedia.org/T334064 (10Clement_Goubert) a:05Clement_Goubert→03Joe [14:55:20] 10serviceops, 10Data Products, 10RESTbase Sunsetting, 10Code-Health-Objective, 10Patch-For-Review: Route to new AQS Knowledge Gaps endpoint - https://phabricator.wikimedia.org/T342213 (10fkaelin) Thanks for the clarification @hnowlan I see the deployment have been done by @BTullis, should this data be... [15:12:15] 10serviceops, 10Data-Platform-SRE, 10Discovery-Search (Current work): Requesting permission to use kafka-main cluster to transport CirrusSearch updates - https://phabricator.wikimedia.org/T341625 (10Gehel) [15:34:42] 10serviceops, 10SRE, 10ops-codfw: Decommission thumbor200[3456] - https://phabricator.wikimedia.org/T344597 (10Jhancock.wm) @hnowlan Wasn't sure so wanted to reach out. Is it safe to proceed with decommissioning these servers? [15:47:01] 10serviceops, 10Content-Transform-Team-WIP, 10Mobile-Content-Service, 10RESTbase Sunsetting, and 3 others: Setup allowed list for MCS decom - https://phabricator.wikimedia.org/T340036 (10MSantos) [16:20:17] 10serviceops, 10SRE, 10ops-codfw: Decommission thumbor200[3456] - https://phabricator.wikimedia.org/T344597 (10hnowlan) >>! In T344597#9106635, @Jhancock.wm wrote: > @hnowlan Wasn't sure so wanted to reach out. Is it safe to proceed with decommissioning these servers? Apologies - I should be much clearer i... [16:21:47] 10serviceops, 10SRE, 10ops-codfw: Decommission thumbor200[34] - https://phabricator.wikimedia.org/T344597 (10hnowlan) [16:22:30] 10serviceops, 10SRE, 10ops-eqiad: Decommission thumbor100[12] - https://phabricator.wikimedia.org/T344598 (10hnowlan)