[02:54:20] 10serviceops, 10Growth-Team, 10Growth-Team-Filtering, 10MW-on-K8s, 10Notifications: Broken (empty) cross-wiki notification when using $wgLocalHTTPProxy (e.g. on Kubernetes) - https://phabricator.wikimedia.org/T223413 (10Tgr) I still don't get how a fix to the networking of k8s could have fixed a bug that... [03:01:25] 10serviceops, 10MW-on-K8s, 10Observability-Logging, 10Developer Productivity, 10MediaWiki-Platform-Team (Radar): php-fpm logs from Kubernetes lack 'message' and 'normalized_message' - https://phabricator.wikimedia.org/T350430 (10Krinkle) @Clement_Goubert Those prefixes are a recognisable part of error me... [03:01:31] 10serviceops, 10MW-on-K8s, 10Observability-Logging, 10Developer Productivity, 10MediaWiki-Platform-Team (Radar): php-fpm logs from Kubernetes lack 'message' and 'normalized_message' - https://phabricator.wikimedia.org/T350430 (10Krinkle) 05Resolved→03Open [07:02:03] 10serviceops, 10CX-cxserver, 10RESTBase Sunsetting, 10Language-Team (Language-2023-October-December), 10Patch-For-Review: Make cxserver call parsoid endpoints on MediaWiki, instead of going through RESTbase - https://phabricator.wikimedia.org/T344982 (10daniel) [08:30:57] 10serviceops, 10Data-Engineering, 10Event-Platform, 10Patch-For-Review: Upgrade change propagation to nodejs18 - https://phabricator.wikimedia.org/T348950 (10elukey) [08:34:25] 10serviceops, 10Data-Engineering, 10Event-Platform, 10Patch-For-Review: Upgrade change propagation to nodejs18 - https://phabricator.wikimedia.org/T348950 (10elukey) All changeprop instances (except beta) run nodejs 18 and the last version of node-rdkafka. As explained above, there was an increase in cpu u... [08:34:55] hnowlan: o/ shall we upgrade changeprop in beta? IIRC it should be a hiera change in horizon right? [09:16:26] I see https://wikitech.wikimedia.org/wiki/Changeprop#To_deployment-prep [09:18:41] ah lovely I don't /srv/deployment-charts on deploy03 [09:28:09] 10serviceops, 10CX-cxserver, 10Citoid, 10Content-Transform-Team-WIP, and 9 others: Migrate node-based services in production to node18 - https://phabricator.wikimedia.org/T349118 (10elukey) [09:41:05] 10serviceops, 10Data-Engineering, 10Discovery-Search (Current work), 10Event-Platform, 10Patch-For-Review: Improve the flink-app chart to provide more useful defaults - https://phabricator.wikimedia.org/T346315 (10Gehel) 05Open→03Resolved [10:04:28] 10serviceops, 10Discovery-Search: Enable mediawiki.cirrussearch.page_rerender.v1 on all public wikis - https://phabricator.wikimedia.org/T351503 (10dcausse) [10:31:01] elukey: sorry missed this - I actually can't remember where it used to get built [10:31:28] I think Tim added those simpler build steps but [10:44:55] hi, I tried deploying this with `helmfile -e eqiad -i apply` but it looks like helmfile thinks there are no changes, am I missing something? does the chart version need to be bumped? https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/975061 [10:48:29] kostajh: you have that job enabled in values-eqiad.yaml [10:48:39] which overrides values.yaml [10:48:41] jayme: ah. thanks. [11:29:14] 10serviceops, 10MW-on-K8s, 10Observability-Logging, 10Developer Productivity, 10MediaWiki-Platform-Team (Radar): php-fpm logs from Kubernetes lack 'message' and 'normalized_message' - https://phabricator.wikimedia.org/T350430 (10Clement_Goubert) @Krinkle I didn't know that, I thought they were just level... [12:04:44] 10serviceops, 10SRE: Rebuild PHP 7.4 packages for Bullseye - https://phabricator.wikimedia.org/T350767 (10MoritzMuehlenhoff) p:05Triage→03Medium [13:44:06] 10serviceops, 10SRE: Rebuild PHP 7.4 packages for Bullseye - https://phabricator.wikimedia.org/T350767 (10MoritzMuehlenhoff) [14:39:07] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10JMeybohm) [16:11:42] 10serviceops, 10MW-on-K8s, 10Observability-Logging, 10Developer Productivity, 10MediaWiki-Platform-Team (Radar): php-fpm logs from Kubernetes lack 'message' and 'normalized_message' - https://phabricator.wikimedia.org/T350430 (10Krinkle) LGTM! [19:55:00] 10serviceops, 10Phabricator, 10Release-Engineering-Team: move "releng-secrets" git repo away from Phabricator (stop pushing over ssh to phab) - https://phabricator.wikimedia.org/T301170 (10Aklapper) What's described in the task title actually got resolved in {T351321} instead. [19:56:15] 10serviceops, 10Phabricator, 10Release-Engineering-Team: Change from ssh to pushing via https for "releng-secrets" git repo - https://phabricator.wikimedia.org/T301170 (10Aklapper) [19:56:44] 10serviceops, 10Phabricator, 10Release-Engineering-Team: Change from ssh to pushing via https for "releng-secrets" git repo - https://phabricator.wikimedia.org/T301170 (10Aklapper) [19:56:56] 10serviceops, 10Phabricator, 10collaboration-services, 10Release-Engineering-Team (Bonus Level 🕹ī¸): Deprecate git-ssh service on phabricator.wikimedia.org - https://phabricator.wikimedia.org/T296022 (10Aklapper) [19:57:08] 10serviceops, 10Phabricator, 10Release-Engineering-Team: Change from ssh to pushing via https for "releng-secrets" git repo - https://phabricator.wikimedia.org/T301170 (10Aklapper)