[00:03:11] 10serviceops, 10PHP 7.4 support, 10Patch-For-Review, 10Platform Team Workboards (Clinic Duty Team): Rename articles and users to prepare for PHP 7.3 unicode changes - https://phabricator.wikimedia.org/T292552 (10Izno) Consider {T292824} in the context of this task. [03:49:42] 10serviceops, 10PHP 7.4 support, 10Patch-For-Review, 10Platform Team Workboards (Clinic Duty Team): Rename articles and users to prepare for PHP 7.3 unicode changes - https://phabricator.wikimedia.org/T292552 (10tstarling) I added the Eszett override to the proposed Gerrit change. Really, all the ligature... [08:04:06] 10serviceops, 10Security-Team, 10serviceops-collab, 10GitLab (CI & Job Runners), and 3 others: Setup GitLab Runner in trusted environment - https://phabricator.wikimedia.org/T295481 (10Jelto) [08:30:36] 10serviceops, 10Observability-Tracing, 10Epic: Package OpenTelemetry Collector as a .deb - https://phabricator.wikimedia.org/T320551 (10Clement_Goubert) a:03Clement_Goubert [09:39:17] 10serviceops, 10Thumbor, 10TimedMediaHandler: Fix missing thumbnail for MPEG (.mpg) videos - https://phabricator.wikimedia.org/T244570 (10valerio.bozzolan) [09:39:33] 10serviceops, 10Thumbor, 10TimedMediaHandler: Fix missing thumbnail for MPEG (.mpg) videos - https://phabricator.wikimedia.org/T244570 (10valerio.bozzolan) Can somebody change the subtask to "Bug"? I do not have enough permissions. Thank you! [10:16:21] 10serviceops, 10Thumbor, 10TimedMediaHandler: Fix missing thumbnail for MPEG (.mpg) videos - https://phabricator.wikimedia.org/T244570 (10TheDJ) [10:20:00] 10serviceops, 10Thumbor, 10TimedMediaHandler: Fix missing thumbnail for MPEG (.mpg) videos - https://phabricator.wikimedia.org/T244570 (10TheDJ) [10:20:02] 10serviceops, 10SRE, 10Thumbor, 10User-jijiki: Upgrade Thumbor to Bullseye - https://phabricator.wikimedia.org/T216815 (10TheDJ) [10:20:32] 10serviceops, 10Thumbor, 10TimedMediaHandler: Fix missing thumbnail for MPEG (.mpg) videos - https://phabricator.wikimedia.org/T244570 (10TheDJ) [10:20:37] 10serviceops, 10SRE, 10Thumbor, 10User-jijiki: Upgrade Thumbor to Bullseye - https://phabricator.wikimedia.org/T216815 (10TheDJ) [10:20:48] 10serviceops, 10Thumbor, 10TimedMediaHandler: Fix missing thumbnail for MPEG (.mpg) videos - https://phabricator.wikimedia.org/T244570 (10TheDJ) [10:20:50] 10serviceops, 10SRE, 10Thumbor, 10User-jijiki: Upgrade Thumbor to Bullseye - https://phabricator.wikimedia.org/T216815 (10TheDJ) [10:49:43] 10serviceops, 10Observability-Tracing, 10Epic: Package OpenTelemetry Collector as a .deb - https://phabricator.wikimedia.org/T320551 (10Clement_Goubert) There are three distributions of `otelcol` to chose from, two "standard" and one custom: - `otelcol-core`: smallest distribution, mostly supports pure `... [11:18:08] going to move kubetcd1005 to a new Ganeti node in a bit (which involves switching to DRBD for a bit, so latencies will be elevated for a while) [13:44:22] claime: hello! we want to deploy https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/738578 but only to some staging clusters now. we'll wait til monday to do any prod ones [13:44:29] for staging, we don't need to depool, right? [13:44:40] we can just do um, helmfile destroy (?) and then apply? [13:44:52] (are there docs on how to destroy the deployment?) [13:51:43] i hope so, we are going try [13:53:40] ottomata: staging doesn't have alarms afaik, idk if a simple helmfile apply would work or not [13:58:28] gr8, we just did eventgate-analytics staging, wtih destroy, all was well! [13:58:48] we have to destroy because this is a big chart and resource name change, (just like the eventstreams one you just did) [13:59:03] ok, so it looks good in staging! [13:59:20] claime: can I get some of your time on Monday to do the production deploys (with depooling?) [14:00:33] ottomata: sure, do you want to be present when I do? (just for scheduling purposes) [14:01:13] yes, i can be here. what timezone are you? [14:01:43] GMT+2, so around now would be perfect if you can swing it [14:03:37] 10serviceops, 10Data Engineering Planning, 10SRE, 10Event-Platform Value Stream (Sprint 02), 10Patch-For-Review: eventgate chart should use common_templates - https://phabricator.wikimedia.org/T303543 (10gmodena) The patch has been reviewed and merged. @Ottomata and @dcausse helped me out with a deplo... [14:04:45] ottomata: Or a bit earlier if you have a meeting [14:05:11] It takes a while because of the TTL and the time for clients to switchover [14:06:39] woiw oof i have meetings monday. sheesh. we can do it though. i can be avail 50 mintues before now. :15 after the last hour [14:06:59] claime: if you like you could start the depool and run the deployments, and i can just be avail and arond. [14:07:06] start the depool before that* [14:07:17] Yep, that's good too [14:08:25] great [14:09:47] I'll schedule the first switchover and deployment around 1300GMT, my meeting is at 1500GMT, so we have plenty of leeway and a good bit of overlap [14:10:22] ty [14:11:13] actually, i have sre privs...so i'd love to know you do the depool/repool and follow along so I could document it and maybe next time I could do it? (I may have done it before but its been a LONG time) [14:17:53] ottomata: I moved the invite a bit later so it fits in your schedule on monday, but feel free to move it around if you'd rather do it another day/at another time [15:07:31] 10serviceops, 10Data Engineering Planning, 10SRE, 10Event-Platform Value Stream (Sprint 02), 10Patch-For-Review: eventgate chart should use common_templates - https://phabricator.wikimedia.org/T303543 (10Ottomata) @Clement_Goubert and I will deploy the rest on Monday. [16:03:29] workout, back in ~30 [16:05:20] 10serviceops, 10Observability-Tracing, 10Epic: Package OpenTelemetry Collector as a .deb - https://phabricator.wikimedia.org/T320551 (10CDanis) Thanks Clement! I think basing on otelcol-contrib is probably the easiest/best thing to do for v0. Sound good to you? [16:46:29] back [17:32:13] 10serviceops, 10SRE, 10observability, 10Maps (Kartotherian): Get Kartotherian SLO metrics into Prometheus - https://phabricator.wikimedia.org/T320748 (10RLazarus) p:05Triage→03Medium [17:37:16] lunch, back in ~45 [17:47:18] 10serviceops, 10Maps, 10SRE, 10observability: SLO dashboards with N latency targets - https://phabricator.wikimedia.org/T320749 (10RLazarus) [17:47:33] 10serviceops, 10Maps, 10SRE, 10observability: SLO dashboards with N latency targets - https://phabricator.wikimedia.org/T320749 (10RLazarus) p:05Triage→03Medium [17:57:56] 10serviceops, 10Maps, 10SRE, 10observability: SLO dashboards with N latency targets - https://phabricator.wikimedia.org/T320749 (10herron) @RLazarus thanks for putting a task together. Totally makes sense and is something I've wanted to do for some time as well. Yes I'm game, especially now that more SLO... [18:18:23] 10serviceops, 10Maps, 10SRE, 10observability: SLO dashboards with N latency targets - https://phabricator.wikimedia.org/T320749 (10RLazarus) Oh, one more angle to think about! There are two Maps services we're writing SLOs for. For Kartotherian, we're just planning a request latency target at the 50th and... [18:30:07] back [20:34:11] 10serviceops, 10Data-Engineering, 10SRE, 10Event-Platform Value Stream (Sprint 02), 10Patch-For-Review: eventstreams chart should use latest common_templates - https://phabricator.wikimedia.org/T310721 (10JArguello-WMF) 05Open→03Resolved [20:34:17] 10serviceops, 10SRE, 10Patch-For-Review, 10good first task: Upgrade all deployment charts to use the latest version of common_templates - https://phabricator.wikimedia.org/T292390 (10JArguello-WMF) [20:36:46] 10serviceops, 10Performance-Team: Migrate WMF production from PHP 7.4 to PHP 8.1 - https://phabricator.wikimedia.org/T319432 (10Jdforrester-WMF) 8.1 not 8.0 or 8.2? > Making CI passing wmf-deployed MediaWiki extensions on PHP 8.0 and PHP 8.1. The wmf-quibble jobs are rather expensive. Do we really need to ru... [20:40:04] 10serviceops, 10Performance-Team: Migrate WMF production from PHP 7.4 to PHP 8.1 - https://phabricator.wikimedia.org/T319432 (10Jdforrester-WMF) [21:11:00] 10serviceops, 10serviceops-collab, 10GitLab (Administration, Settings & Policy): Configure a default cleanup policy for GitLab package registry - https://phabricator.wikimedia.org/T315877 (10brennen)