[04:37:10] 10serviceops, 10RESTBase: PCS caching and pregeneration when restbase is decommissioned - https://phabricator.wikimedia.org/T319365 (10Joe) [05:34:56] 10serviceops, 10RESTBase: PCS caching and pregeneration when restbase is decommissioned - https://phabricator.wikimedia.org/T319365 (10Joe) Summarizing my results: **Edge cache** From turnilo I extracted the hourly webrequests for `/api/rest_v1/page/(mobile-html|summary|media-list|mobile-html-offline-resource... [07:09:35] 10serviceops, 10SRE: Move Kafka main to the new intermediate PKI CA - https://phabricator.wikimedia.org/T319372 (10elukey) [08:55:34] claime: I remember you talking about getting a horizon project for serviceops, did you already get that? I'm thinking about creating a protoon envitonment for k8s refactoring/update stuff [10:19:40] 10serviceops, 10SRE, 10Thumbor, 10Thumbor Migration, and 2 others: Migrate thumbor to Kubernetes - https://phabricator.wikimedia.org/T233196 (10hnowlan) [10:30:37] jayme: I did [10:31:21] jayme: It's called appservers though, and we may need to update quota depending on how many vms you need [10:31:36] I'll give you access [10:31:43] claime: ❤️ [10:31:47] was about to ask :) [10:32:43] I think I need at least 3 plus protoon related stuff (which is 1 I guess) [10:36:30] Ok, we'll be a bit short on quota, I'll ping wmcs for an upgrade [10:39:50] thank you! [10:40:33] np :) [10:42:04] 10serviceops, 10Product-Infrastructure-Team-Backlog, 10SRE, 10WMDE-TechWish-Maintenance, and 3 others: New Service Request geoshapes - https://phabricator.wikimedia.org/T274388 (10awight) [11:37:59] 10serviceops, 10Thumbor Migration, 10Platform Team Workboards (Platform Engineering Reliability): Define an SLO for Thumbor - https://phabricator.wikimedia.org/T319399 (10hnowlan) [11:40:52] jayme: quota upped, you're good to go [11:41:51] Hmm, only the vcpus got upped, not the instances [11:41:57] checking back in with wmcs [11:57:14] jayme: you should be able to at least start deploying your pontoon puppetmaster + 2VM while we wait for instance count upgrade [12:09:46] jayme: instance quota increased, you're good to go [12:10:44] I can't remember if you've deployed pontoon stacks already, tell me if you want/need a hand [12:16:41] claime: thanks, I probably won't get to it before tomorrow anyways. Looks like wmcs did not bump memory, though. With 2GB instances there is still only room for 3 [12:17:07] Ah. Circling back [12:30:15] jayme: if you need to, you can check out my sandbox branch for the sops-appservers stack https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/sandbox/cgoubert/pontoon-sops-appservers/ [12:32:02] cool, thanks [13:07:46] all good. [13:22:45] 10serviceops: Envoy can't connect to servers using TLS 1.3 (but can serve TLS 1.3 to clients) - https://phabricator.wikimedia.org/T246083 (10bking) a:03bking [13:23:20] hey inflatador, 'sup? [13:24:31] claime nothing too exciting...yet >:) [13:25:38] how 'bout you? [13:28:27] bit of a rushed day, had to take my dog to the vet this morning because he got his paw stepped on by a horse. It's broken, but he's gonna be fine [13:28:37] awww [13:28:52] He's currently stoned out of his gourd on his bed. He's ok. :p [13:28:52] glad to hear he's OK though [13:30:14] I'm glad he is too :D [15:07:53] 10serviceops: Envoy can't connect to servers using TLS 1.3 (but can serve TLS 1.3 to clients) - https://phabricator.wikimedia.org/T246083 (10bking) Before we get started, some quick info: - [[ https://github.com/envoyproxy/envoy/issues/9300 | Possibly relevant github thread ]] - [[ https://github.com/envoyproxy... [15:12:43] 10serviceops, 10DC-Ops, 10SRE, 10ops-codfw: Q2:rack/setup/install webperf2005.codfw.wmnet - https://phabricator.wikimedia.org/T319428 (10RobH) [15:13:45] 10serviceops, 10DC-Ops, 10SRE, 10ops-codfw: Q2:rack/setup/install webperf2005.codfw.wmnet - https://phabricator.wikimedia.org/T319428 (10RobH) [15:13:50] 10serviceops, 10DC-Ops, 10SRE, 10ops-codfw: Q2:rack/setup/install webperf2005.codfw.wmnet - https://phabricator.wikimedia.org/T319428 (10RobH) [15:15:29] 10serviceops, 10SRE: service implementation tracking: webperf2005.codfw.wmnet - https://phabricator.wikimedia.org/T319429 (10RobH) [15:28:54] 10serviceops, 10Performance-Team: Migrate WMF production from PHP 7.4 to PHP 8.1 - https://phabricator.wikimedia.org/T319432 (10Krinkle) [15:33:12] _joe_: mind if I put in https://gerrit.wikimedia.org/r/838843 today, to clean up the hourly httpbb run? [15:33:56] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad: Q2:rack/setup/install webperf1005.eqiad.wmnet - https://phabricator.wikimedia.org/T319433 (10RobH) [15:34:02] <_joe_> rzl: I'm in a meeting with chatty people, but I planned to merge clement's version of that [15:34:16] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad: Q2:rack/setup/install webperf1005.eqiad.wmnet - https://phabricator.wikimedia.org/T319433 (10RobH) [15:34:20] oh! even better 👍 [15:34:50] <_joe_> https://gerrit.wikimedia.org/r/c/operations/puppet/+/838112 [15:34:57] <_joe_> you can merge it too :P [15:35:36] 10serviceops, 10SRE: service implementation tracking: webperf1005.eqiad.wmnet - https://phabricator.wikimedia.org/T319434 (10RobH) [15:37:41] sure, doing [15:41:49] fixed, thanks claime! ✅ [15:42:09] actually now that we're multi-dc, I think I'll add another hourly httpbb run, from cumin2002 to a codfw canary appserver [15:42:18] yw rzl [15:42:23] probably mostly redundant, but if it ever isn't, we'll want to know [15:55:17] agreed [15:58:34] quick workout, back in ~40 [16:36:19] back [17:03:57] My spark and spark-operator production images are ready for anyone's lovely eyes to review, in case you should feel so inclined. [17:03:57] https://gerrit.wikimedia.org/r/c/operations/docker-images/production-images/+/838151 and https://gerrit.wikimedia.org/r/c/operations/docker-images/production-images/+/838858 [17:04:18] I have de-coupled them as previously advised. [17:36:23] 10serviceops, 10Release-Engineering-Team, 10docker-pkg: docker-pkg / docker downloads all versions of parent image upon building - https://phabricator.wikimedia.org/T310458 (10hashar) I had to build some new images for https://gerrit.wikimedia.org/r/c/integration/config/+/814154/ which I have done on `contin... [18:21:18] 10serviceops, 10Release-Engineering-Team, 10docker-pkg: docker-pkg / docker downloads all versions of parent image upon building - https://phabricator.wikimedia.org/T310458 (10Clement_Goubert) \o/ [18:47:38] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad, 10serviceops-collab: Q2:rack/setup/install webperf1005.eqiad.wmnet - https://phabricator.wikimedia.org/T319433 (10Dzahn) [18:51:40] 10serviceops, 10RESTBase: PCS caching and pregeneration when restbase is decommissioned - https://phabricator.wikimedia.org/T319365 (10daniel) [18:52:08] 10serviceops, 10RESTBase: PCS caching and pregeneration when restbase is decommissioned - https://phabricator.wikimedia.org/T319365 (10daniel) [21:41:59] 10serviceops, 10serviceops-collab, 10Patch-For-Review: Review DNS TTLs for ServiceOps-Collab owned services - https://phabricator.wikimedia.org/T315319 (10Dzahn) 05Open→03In progress