[08:23:10] 10serviceops, 10CFSSL-PKI, 10Infrastructure-Foundations, 10Machine-Learning-Team, 10Patch-For-Review: Extend cfssl-issuer to return the Root CA certificate - https://phabricator.wikimedia.org/T299906 (10JMeybohm) 05Open→03Resolved Updated cfssl-issuer is deployed to all clusters where it is currently... [08:41:47] 10serviceops, 10Release-Engineering-Team, 10Scap: Deploy Scap version 4.2.1 - https://phabricator.wikimedia.org/T300058 (10JMeybohm) `scap pull` and restbase dummy deploy seemed fine. [10:24:11] 10serviceops, 10Prod-Kubernetes, 10Kubernetes: Run helm test after deploy - https://phabricator.wikimedia.org/T276949 (10Jelto) I think the behavior changed quite a bit between `helm2` and `helm3`. So the described behavior above is a bit outdated. Deletion of test resources in `helm3` can be controlled us... [13:29:16] 10serviceops, 10Prod-Kubernetes, 10Kubernetes, 10Patch-For-Review: kube-apiserver need to reach webhooks running inside of the cluster - https://phabricator.wikimedia.org/T290967 (10ops-monitoring-bot) Host rebooted by jayme@cumin1001 with reason: cgroup_enable=memory after docker install [13:43:35] 10serviceops, 10Prod-Kubernetes, 10Kubernetes, 10Patch-For-Review: kube-apiserver need to reach webhooks running inside of the cluster - https://phabricator.wikimedia.org/T290967 (10ops-monitoring-bot) Host rebooted by jayme@cumin1001 with reason: cgroup_enable=memory after docker install [14:11:16] 10serviceops, 10Prod-Kubernetes, 10Kubernetes, 10Patch-For-Review: kube-apiserver need to reach webhooks running inside of the cluster - https://phabricator.wikimedia.org/T290967 (10JMeybohm) [14:49:26] https://tech.olx.com/demystifying-istio-circuit-breaking-27a69cac2ce4 looks nice [14:49:57] I'll add some basic settings to protect the mw-api from your dear ml friends :D [14:50:20] the defaults seem a little too much [15:37:38] 10serviceops, 10Release-Engineering-Team, 10Scap: Deploy Scap version 4.2.1 - https://phabricator.wikimedia.org/T300058 (10dancy) [15:39:11] do we have a suggested per-pod max parallel requests for services like mediawiki-api? [15:39:36] in my case there will be 2/4 pods for egress gw [16:26:07] <_joe_> elukey: "don't make us notice us" [16:26:15] <_joe_> :P [16:26:36] ahahaha okok [16:26:37] <_joe_> meaning, as long as the total amount of requests to the mw api is less than 5% of total [16:27:32] <_joe_> elukey: btw, we will have the possibility to spin up dedicated mw clusters for internal api calls soon [16:28:06] ah nice! [17:47:34] 10serviceops, 10SRE, 10Traffic-Icebox: Use Envoy instead of nginx for TLS termination on Appservers - https://phabricator.wikimedia.org/T240576 (10RLazarus) 05Open→03Resolved a:03RLazarus Good news! This is long since done, tidying it up. [19:45:11] 10serviceops, 10SRE, 10Patch-For-Review: Remove mediawiki::packages::fonts from non thumbor servers - https://phabricator.wikimedia.org/T294378 (10Dzahn) purged from all of parsoid (wtp* and parse*) and the rest of eqiad (mw14*)