[06:05:00] 10serviceops, 10Data-Platform-SRE, 10Discovery-Search (Current work): Requesting permission to use kafka-main cluster to transport CirrusSearch updates - https://phabricator.wikimedia.org/T341625 (10Joe) >>! In T341625#9091018, @EBernhardson wrote: >>>! In T341625#9086139, @Joe wrote: >> I am uneasy suggesti... [06:28:03] 10serviceops, 10docker-pkg, 10Release Pipeline (Blubber): Fix how we keep docker-pkg based images up to date - https://phabricator.wikimedia.org/T344478 (10Joe) [06:28:15] 10serviceops, 10docker-pkg, 10Release Pipeline (Blubber): Fix how we keep docker-pkg based images up to date - https://phabricator.wikimedia.org/T344478 (10Joe) p:05Triage→03Medium [10:09:37] 10serviceops, 10collaboration-services, 10GitLab (CI & Job Runners): Standardize Debian package builds on GitLab CI - https://phabricator.wikimedia.org/T304491 (10Jelto) Picking up the discussion from last weeks meeting and your mail @MatthewVernon : I was able to include your [wmf-debci](https://gitlab.wik... [10:10:55] effie: I downtimed the flapping check on maps2006 - not sure why it's the only instance of the replication lag check that's failing though :/ [10:11:22] that is alright, it is the least of our problems with that part of our infra [10:17:38] 10serviceops, 10MW-on-K8s, 10Prod-Kubernetes, 10Kubernetes: Allow more flexibility in ResourceQuota and LimitRanger config - https://phabricator.wikimedia.org/T343978 (10JMeybohm) 05Open→03Resolved [10:17:40] 10serviceops, 10MW-on-K8s: mw-on-k8s app container CPU throttling at low average load - https://phabricator.wikimedia.org/T342748 (10JMeybohm) [10:18:33] 10serviceops, 10collaboration-services, 10GitLab (CI & Job Runners): Standardize Debian package builds on GitLab CI - https://phabricator.wikimedia.org/T304491 (10MatthewVernon) [I'm in the process of tweaking the branch-naming convention for that - I'm instead moving to extracting the build suite from the c... [10:20:24] 10serviceops, 10MW-on-K8s: mw-on-k8s app container CPU throttling at low average load - https://phabricator.wikimedia.org/T342748 (10Clement_Goubert) Limitless php containers deployed today on mw-api-int. Next week we will reintroduce memory limits, and extend the deployment of limitless CPU for all mw-on-k8s... [13:04:06] 10serviceops, 10Prod-Kubernetes, 10Kubernetes, 10Patch-For-Review: Reserve resources for system daemons on kubernetes nodes - https://phabricator.wikimedia.org/T277876 (10JMeybohm) >>! In T277876#9099477, @Clement_Goubert wrote: > Just to clarify, the above patch will not lead to evictions (unless we have... [13:13:44] 10serviceops, 10Data-Platform-SRE, 10Discovery-Search (Current work): Requesting permission to use kafka-main cluster to transport CirrusSearch updates - https://phabricator.wikimedia.org/T341625 (10pfischer) @Joe, thank you for your feedback! > I'm not sure I fully grasp the model, do you have any diagram... [13:28:08] 10serviceops, 10Prod-Kubernetes, 10Kubernetes, 10Patch-For-Review: Reserve resources for system daemons on kubernetes nodes - https://phabricator.wikimedia.org/T277876 (10Clement_Goubert) Considering there's no reservation for system resources at the moment, I feel like that would be a better solution than... [14:16:32] 10serviceops, 10Prod-Kubernetes, 10Kubernetes, 10Patch-For-Review: Reserve resources for system daemons on kubernetes nodes - https://phabricator.wikimedia.org/T277876 (10JMeybohm) >>! In T277876#9101677, @Clement_Goubert wrote: > Considering there's no reservation for system resources at the moment, I fee... [15:13:58] 10serviceops, 10Content-Transform-Team-WIP, 10Mobile-Content-Service, 10RESTbase Sunsetting, and 3 others: Setup allowed list for MCS decom - https://phabricator.wikimedia.org/T340036 (10MSantos) @Kelson the allowed list policy will end by the end of September. I was assuming you already got access to it,... [15:33:35] 10serviceops, 10Prod-Kubernetes, 10Kubernetes, 10Patch-For-Review: Remove allow-pod-to-pod GlobalNetworkPolicy - https://phabricator.wikimedia.org/T344177 (10JMeybohm) [15:36:39] 10serviceops, 10Prod-Kubernetes, 10Kubernetes, 10Patch-For-Review: Remove GlobalNetworkPolicies - https://phabricator.wikimedia.org/T344177 (10JMeybohm) [19:24:56] 10serviceops, 10SRE, 10MediaWiki-Platform-Team (Radar): Evaluate using igbinary for MW php-apcu at WMF - https://phabricator.wikimedia.org/T225074 (10Krinkle) [19:25:07] 10serviceops, 10SRE, 10Performance-Team (Radar): Evaluate using igbinary for MW php-apcu at WMF - https://phabricator.wikimedia.org/T225074 (10Krinkle) [19:28:51] 10serviceops, 10DC-Ops, 10SRE, 10ops-codfw: Q1:rack/setup/install kubernetes20[25-54] - https://phabricator.wikimedia.org/T342534 (10Jhancock.wm) [19:31:34] 10serviceops, 10WMF-JobQueue, 10Unstewarded-production-error, 10User-brennen, 10Wikimedia-production-error: Could not enqueue jobs: "Unable to deliver all events: 503: Service Unavailable" - https://phabricator.wikimedia.org/T249745 (10Krinkle) [23:34:49] 10serviceops, 10Data-Platform-SRE, 10Discovery-Search (Current work): Requesting permission to use kafka-main cluster to transport CirrusSearch updates - https://phabricator.wikimedia.org/T341625 (10EBernhardson) >>! In T341625#9100951, @Joe wrote: > I have different numbers from [[ https://grafana.wikimedia...