[03:52:03] FIRING: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [04:02:03] RESOLVED: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [05:01:24] (03open) 10phedenskog: Add a p95 tab to the core daily job time charts [repos/releng/develstats] - 10https://gitlab.wikimedia.org/repos/releng/develstats/-/merge_requests/61 [05:01:31] (03merge) 10phedenskog: Add a p95 tab to the core daily job time charts [repos/releng/develstats] - 10https://gitlab.wikimedia.org/repos/releng/develstats/-/merge_requests/61 [05:14:01] (03PS1) 10Phedenskog: Allow running only the PHPUnit tests that need a database [integration/quibble] - 10https://gerrit.wikimedia.org/r/1337436 (https://phabricator.wikimedia.org/T437134) [05:19:21] (03PS1) 10Phedenskog: jjb: Run only database tests in the sqlite and postgres core jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1337437 (https://phabricator.wikimedia.org/T437134) [05:26:38] (03PS1) 10Phedenskog: jjb: Overlap npm install with the gated browser tests [integration/config] - 10https://gerrit.wikimedia.org/r/1337439 (https://phabricator.wikimedia.org/T436376) [06:31:36] 06Project-Admins, 10Produnto: Phabricator project for bug report, management and moderation of Produnto packages - https://phabricator.wikimedia.org/T437182 (10Bugreporter) 03NEW [07:30:16] (03CR) 10Hashar: [C:03+2] "Done" [performance/WikimediaDebug] - 10https://gerrit.wikimedia.org/r/1335275 (owner: 10Krinkle) [07:30:45] (03Merged) 10jenkins-bot: Fix inclusion non-MediaWiki request noise in the list [performance/WikimediaDebug] - 10https://gerrit.wikimedia.org/r/1335275 (owner: 10Krinkle) [07:36:07] (03PS1) 10Hashar: Tag v3.3.0 [performance/WikimediaDebug] - 10https://gerrit.wikimedia.org/r/1337445 [07:36:29] (03CR) 10Hashar: [C:03+2] Tag v3.3.0 [performance/WikimediaDebug] - 10https://gerrit.wikimedia.org/r/1337445 (owner: 10Hashar) [07:36:59] (03Merged) 10jenkins-bot: Tag v3.3.0 [performance/WikimediaDebug] - 10https://gerrit.wikimedia.org/r/1337445 (owner: 10Hashar) [07:54:07] (03CR) 10Hashar: [C:03+2] zuul: add Akaza24 to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1336998 (owner: 10Apehitkey) [07:55:15] 10GitLab, 06Release-Engineering-Team (Radar), 06Collaboration-Services, 13Patch-For-Review: gitlab behind CDN - https://phabricator.wikimedia.org/T425441#12293630 (10brouberol) It's interesting to see ` an-worker1200.eqiad.wmnet Go-http-client/1.1 -> 401 python-requests/2.32.3 -> 401 ` in the list of a... [07:56:24] (03Merged) 10jenkins-bot: zuul: add Akaza24 to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1336998 (owner: 10Apehitkey) [08:02:46] 10GitLab, 06Release-Engineering-Team (Radar), 06Collaboration-Services, 13Patch-For-Review: gitlab behind CDN - https://phabricator.wikimedia.org/T425441#12293643 (10ABran-WMF) Three corrections to T425441#12288782. Line numbers below are in `modules/profile/templates/cache/haproxy/tls_terminator.cfg.erb`.... [08:06:27] 10GitLab, 06Release-Engineering-Team (Radar), 06Collaboration-Services, 13Patch-For-Review: gitlab behind CDN - https://phabricator.wikimedia.org/T425441#12293649 (10ABran-WMF) >>! In T425441#12293630, @brouberol wrote: > It's interesting to see > ` > an-worker1200.eqiad.wmnet Go-http-client/1.1 -> 401... [08:10:04] FIRING: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [08:11:19] 10GitLab, 06Release-Engineering-Team (Radar), 06Collaboration-Services, 13Patch-For-Review: gitlab behind CDN - https://phabricator.wikimedia.org/T425441#12293663 (10brouberol) What about https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/blob/main/pyproject.toml?ref_type=heads#L59 and http... [08:17:41] 06Release-Engineering-Team (Doing 😎), 07Essential-Work: Failing branch cut job - https://phabricator.wikimedia.org/T437184 (10jnuche) 03NEW [08:18:03] 06Release-Engineering-Team (Doing 😎), 07Essential-Work: Failing branch cut job - https://phabricator.wikimedia.org/T437184#12293689 (10jnuche) [08:21:05] (03open) 10jnuche: settings.yaml: remove archived extension `SimilarEditors` [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/269 (https://phabricator.wikimedia.org/T437184) [08:23:36] (03merge) 10jnuche: settings.yaml: remove archived extension `SimilarEditors` [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/269 (https://phabricator.wikimedia.org/T437184) [08:30:04] 10GitLab, 06Release-Engineering-Team (Radar), 06Collaboration-Services, 13Patch-For-Review: gitlab behind CDN - https://phabricator.wikimedia.org/T425441#12293737 (10ABran-WMF) Good catch, and unlike the dbt row that one is a genuine call across the edge. It passes, because Poetry sets its own User-Agent.... [08:32:33] 10GitLab, 06Collaboration-Services: GitLab Replicas should not page - https://phabricator.wikimedia.org/T437189 (10Jelto) 03NEW [08:33:18] 10GitLab, 06Collaboration-Services: GitLab Replicas should not page - https://phabricator.wikimedia.org/T437189#12293784 (10ABran-WMF) p:05Triage→03High [08:49:13] 06Release-Engineering-Team (Doing 😎), 07Essential-Work: Failing branch cut job - https://phabricator.wikimedia.org/T437184#12293952 (10jnuche) 05Open→03Resolved https://releases-jenkins.wikimedia.org/job/Automatic%20branch%20cut/336/ [08:50:04] RESOLVED: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [09:20:39] 10GitLab, 06Release-Engineering-Team (Radar), 06Collaboration-Services, 13Patch-For-Review: gitlab behind CDN - https://phabricator.wikimedia.org/T425441#12294083 (10brouberol) Great, thank you @ABran-WMF ! [09:47:20] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06SRE: apt-get broken in docker-registry.wikimedia.org/bullseye:20260830 - https://phabricator.wikimedia.org/T437069#12294166 (10MoritzMuehlenhoff) bullseye-security was restored on the Debian archive side, please check if there are still any issues. [09:50:47] 10GitLab, 06Collaboration-Services, 13Patch-For-Review: GitLab Replicas should not page - https://phabricator.wikimedia.org/T437189#12294170 (10ABran-WMF) The primary issue here has been fixed with [[ https://gerrit.wikimedia.org/r/1337543 | the exclusion ]] of Gitlab replicas from the ATS alert that paged.... [10:00:53] 10Phabricator (Upstream), 07Upstream: Add another way of two factor auth to Phabricator other than TOTP (hardware keys) - https://phabricator.wikimedia.org/T187256#12294187 (10Aklapper) [10:01:29] 10Phabricator (Upstream), 07Upstream: Add another way of two factor auth to Phabricator other than TOTP (hardware keys) - https://phabricator.wikimedia.org/T187256#12294194 (10Aklapper) SMS and email are error-prone. If there was another way, it would be hardware keys nowadays. So I edited the ticket title. [10:18:46] 10Phabricator (Upstream), 06Release-Engineering-Team (Doing 😎), 07Upstream: Calendar monthly view makes event names unreadable - https://phabricator.wikimedia.org/T187019#12294269 (10Aklapper) a:03Aklapper [11:05:03] FIRING: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [11:15:03] RESOLVED: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [11:28:43] (03PS1) 10Kevin Bazira: inference-services: Add CI pipeline jobs for tool-server [integration/config] - 10https://gerrit.wikimedia.org/r/1337567 (https://phabricator.wikimedia.org/T436892) [12:04:50] (03CR) 10Hnowlan: [C:03+1] Review access change [software/thumbor-plugins] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1337307 (owner: 10Ladsgroup) [12:05:22] (03CR) 10Ladsgroup: [V:03+2 C:03+2] Review access change [software/thumbor-plugins] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1337307 (owner: 10Ladsgroup) [12:06:24] Hi, I don't have rights to push this change ^ would it be okay to do it since that allows Media platform team to merge patches on thumbor (we own it now)? [12:06:36] (03CR) 10Ladsgroup: [V:03+1 C:03+1] "can't merge it 😞" [software/thumbor-plugins] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1337307 (owner: 10Ladsgroup) [12:10:17] Amir1: +2 and get Ci to submit it? [12:10:40] it doesn't submit it from what I'm seeing [12:10:53] CI doesn't even run on it at all [12:11:14] well it definitely ran at some point https://gerrit.wikimedia.org/r/c/operations/software/thumbor-plugins/+/1333249/4#message-bbf150c6641d8e6a2b7da5509a133e7d577e0088 [12:11:16] and merged [12:11:57] Amir1: which change has an issue? [12:12:11] this is on master, the patch I'm referring to is on meta/config [12:12:16] https://gerrit.wikimedia.org/r/1337307 [12:12:23] CI works on master no problem [12:12:41] I can submit patches on master too, but not this specific patch [12:13:12] this allows members of mediawiki group to +2 patches on the thumbor repo [12:13:25] now I am confused [12:13:36] so I assume for some security reasons we can't merge those patches [12:13:57] you did CR+2 patches on that repo [12:14:03] eg https://gerrit.wikimedia.org/r/c/operations/software/thumbor-plugins/+/1333250/6#message-8fc8455d045b26b169ce41770175aec5947b630f [12:14:14] so you get the permission to Code-Review: +2 there [12:14:23] if you go to https://gerrit.wikimedia.org/r/admin/repos/operations/software/thumbor-plugins,access and try to change the access and who can +2 patches, instead of changing it right away, it creates this patch instead [12:14:38] which I can't merge even though I personally have +2 rights on the repo [12:14:50] note the branch of the patch [12:14:53] it is different permissions [12:15:18] to change the repository configuration (which is held under `refs/meta/config`) one needs the `Owner` permission [12:15:32] https://gerrit.wikimedia.org/r/admin/repos/operations/software/thumbor-plugins,access [12:15:47] that is assigned to the group operations-software-thumbor-plugins https://gerrit.wikimedia.org/r/admin/groups/c63b6fccb478caff84c3c19331358c30d883235e [12:15:56] which has a few members [12:15:57] https://gerrit.wikimedia.org/r/admin/groups/c63b6fccb478caff84c3c19331358c30d883235e,members [12:16:08] so what do you want to achieve ultimately? [12:16:27] have any member of the "mediawiki" group to be able to own/change the config of that repo (and get +2 as well?) [12:16:34] none of those people work at wmf anymore, with excpetion of bill who is on sabbatical [12:16:41] if so the easiest is to have the "mediawiki" group to be included in that operations-software-thumbor-plugins group [12:16:57] (alternatively we delete that group entirely and make thumbor owned by mediawiki) [12:17:24] either way is fine by me. Right now, noone beside me in the team can merge patches in thumbor [12:17:44] oh [12:17:51] you are in the ops ldap group [12:18:24] so yeah essentially operations/software/thumbor-plugins inherits from operations/software which grants +2 to ldap/ops [12:19:58] (03PS1) 10Hashar: Review access change [software/thumbor-plugins] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1337581 [12:22:11] (03PS2) 10Hashar: Change ownership to 'mediawiki' [software/thumbor-plugins] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1337581 [12:22:19] Amir1: https://gerrit.wikimedia.org/r/c/operations/software/thumbor-plugins/+/1337581 with some doc [12:23:05] Code-Review +2 is granted to project owners (there is global rule to grant that) so there is no need to explicitly add it [12:23:14] (03CR) 10Hashar: [V:03+2 C:03+2] Change ownership to 'mediawiki' [software/thumbor-plugins] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1337581 (owner: 10Hashar) [12:23:24] merged [12:23:27] should be good now [12:23:33] (03CR) 10Ladsgroup: [C:03+1] Change ownership to 'mediawiki' [software/thumbor-plugins] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1337581 (owner: 10Hashar) [12:23:54] and the old group operations-software-thumbor-plugins would eventually be removed when we do a cleanup of unused groups [12:24:46] 10GitLab, 06Release-Engineering-Team (Radar), 06cloud-services-team, 06Collaboration-Services, and 4 others: Set an identifiable User-Agent on WMF tools calling gitlab.wikimedia.org - https://phabricator.wikimedia.org/T437216 (10ABran-WMF) 03NEW [12:31:31] (03Abandoned) 10Ladsgroup: Review access change [software/thumbor-plugins] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1337307 (owner: 10Ladsgroup) [12:33:40] 10GitLab, 06Release-Engineering-Team (Radar), 06Collaboration-Services, 06Machine-Learning-Team, and 3 others: Set an identifiable User-Agent on WMF tools calling gitlab.wikimedia.org - https://phabricator.wikimedia.org/T437216#12294601 (10taavi) [12:56:22] 10GitLab, 06Collaboration-Services, 13Patch-For-Review: GitLab Replicas should not page - https://phabricator.wikimedia.org/T437189#12294630 (10ABran-WMF) 05Open→03Resolved a:03ABran-WMF [13:10:20] 06Project-Admins, 10Produnto: Phabricator project for bug report, management and moderation of Produnto packages - https://phabricator.wikimedia.org/T437182#12294657 (10Aklapper) 05Open→03Stalled See https://www.mediawiki.org/wiki/Phabricator/Creating_and_renaming_projects for required data [13:16:04] FIRING: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [13:21:04] RESOLVED: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [13:34:34] (03CR) 10Hashar: [C:03+2] inference-services: Add CI pipeline jobs for tool-server [integration/config] - 10https://gerrit.wikimedia.org/r/1337567 (https://phabricator.wikimedia.org/T436892) (owner: 10Kevin Bazira) [13:35:13] (03CR) 10Hashar: [C:03+2] "I have created the jobs:" [integration/config] - 10https://gerrit.wikimedia.org/r/1337567 (https://phabricator.wikimedia.org/T436892) (owner: 10Kevin Bazira) [13:37:22] (03Merged) 10jenkins-bot: inference-services: Add CI pipeline jobs for tool-server [integration/config] - 10https://gerrit.wikimedia.org/r/1337567 (https://phabricator.wikimedia.org/T436892) (owner: 10Kevin Bazira) [13:38:07] kevinbazira: I have deployed your CI config change for tool_server and did a `recheck` on https://gerrit.wikimedia.org/r/c/machinelearning/liftwing/inference-services/+/1337560 :) [13:38:53] hashar: merci beaucoup : ) [13:39:04] kevinbazira: and something failed https://integration.wikimedia.org/ci/job/inference-services-pipeline-tool-server/1/console :) [13:39:15] looking ... [13:39:18] #17 5.164 E: Release file for http://snapshot.debian.org/archive/debian/20260830T040200Z/dists/bookworm-updates/InRelease is expired (invalid since 1d 11h 31min 7s). Updates for this repository will not be applied. [13:39:18] #17 5.164 E: Release file for http://snapshot.debian.org/archive/debian-security/20260830T040200Z/dists/bookworm-security/InRelease is expired (invalid since 1d 17h 1min 10s). Updates for this repository will not be applied. [13:39:20] fun times [13:39:44] I am not sure why it points to Debian archive [13:40:04] or how long Debian keeps snapshots [13:40:57] 10Phabricator, 06Release-Engineering-Team, 06Collaboration-Services: Move aphlict to k8s - https://phabricator.wikimedia.org/T436657#12294754 (10ABran-WMF) p:05Triage→03Medium [13:41:22] oh docker-registry.wikimedia.org/bookworm:20260830 [13:41:36] so hmm I imagine we now have builds that are more or less reproducible [13:41:49] but that base image has a faulty config that prevent it from installing "expired" packages [13:41:56] s/packages/index/ or whatever [13:42:03] I know nothing about that system though [13:49:56] updated patch to use the latest image: docker-registry.wikimedia.org/bookworm:20260906 [13:50:48] kevinbazira: my guess is that it will fine again in 7 days when the expery is reached [13:51:20] :( [13:51:27] so something is certainly broken in that base image somehow [13:52:06] I see the others have a version pinning as well and apparently are not broken [13:52:14] so something probably go introduced fairly recently [13:52:48] I don't know where our base images are defined though [13:53:11] my guess is SRE would know :) [13:56:09] (03CR) 10Hashar: [C:03+2] "I am happy we have those plan files, that makes reviewing easier 😊" [integration/quibble] - 10https://gerrit.wikimedia.org/r/1337436 (https://phabricator.wikimedia.org/T437134) (owner: 10Phedenskog) [13:57:07] hashar: here is the base images repo: https://gerrit.wikimedia.org/g/operations/docker-images/production-images [13:59:53] kevinbazira: yup, but the base debian images are not maintained there [14:00:23] my guess is they get created from the build server using debootstrap or similar and that would be defined somewhere in Puppet [14:00:30] i don't know about that system though, but SRE would for sure [14:01:26] yep, not familiar with it either. will continue poking around. thanks for your help with the CI patch <3 [14:02:03] :-] [14:03:04] FIRING: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [14:03:29] I think I found it: https://github.com/wikimedia/operations-docker-images-production-images/tree/master/images/wmf-debci/bookworm [14:14:21] (03Merged) 10jenkins-bot: Allow running only the PHPUnit tests that need a database [integration/quibble] - 10https://gerrit.wikimedia.org/r/1337436 (https://phabricator.wikimedia.org/T437134) (owner: 10Phedenskog) [14:18:04] RESOLVED: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [14:29:07] 10Beta-Cluster-Infrastructure, 10CFSSL-PKI, 06Infrastructure-Foundations: profile::pki::client does not create symlinks using hashes of CAs - https://phabricator.wikimedia.org/T436562#12295013 (10elukey) p:05Triage→03Low [14:50:32] Project mediawiki-core-doxygen build #22730: 04FAILURE in 32 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/22730/ [14:53:33] kevinbazira: for the apt get being broken with bookworm:20260830 , the task seems to be https://phabricator.wikimedia.org/T437069 [14:53:50] that task is for bullseye, but I would guess the underlying issue is the same [14:54:15] well hmm no [14:54:18] different one [14:54:20] bah [14:54:28] that task I have linked is for bullseye-security that got removed [14:54:43] and the error for bookworm tool_server image was different [14:55:03] so I guess file a task for SRE + Infrastructure-Foundations [14:55:03] :) [14:55:23] 06Release-Engineering-Team (Radar), 06Infrastructure-Foundations, 06SRE: apt-get broken in docker-registry.wikimedia.org/bullseye:20260830 - https://phabricator.wikimedia.org/T437069#12295132 (10hashar) [14:58:28] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 13Patch-For-Review: Puppet fail on integration-agent-qemu-1004: due to ci-build-images - https://phabricator.wikimedia.org/T436775#12295163 (10hashar) I am pretty sure the issue is the script being invoked on every single run of the puppet... [14:58:38] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 13Patch-For-Review: Puppet fail on integration-agent-qemu-1004: due to ci-build-images - https://phabricator.wikimedia.org/T436775#12295166 (10hashar) p:05Triage→03Low a:03hashar [14:58:59] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Doing 😎), 13Patch-For-Review: Puppet fail on integration-agent-qemu-1004: due to ci-build-images - https://phabricator.wikimedia.org/T436775#12295169 (10hashar) [15:11:16] 10GitLab, 06Release-Engineering-Team (Radar), 06Collaboration-Services, 06Machine-Learning-Team, and 3 others: Set an identifiable User-Agent on WMF tools calling gitlab.wikimedia.org - https://phabricator.wikimedia.org/T437216#12295224 (10Jelto) p:05Triage→03Medium [15:12:07] FIRING: PuppetAgentFailure: Puppet agent failure detected on instance deployment-cp-text09 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [16:05:48] 10WikimediaDebug: Add Test Kitchen functionality to WikimediaDebug - https://phabricator.wikimedia.org/T437240 (10phuedx) 03NEW [16:31:39] Yippee, build fixed! [16:31:39] Project mediawiki-core-doxygen build #22731: 09FIXED in 13 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/22731/ [16:33:08] 10Phabricator, 06SRE: Querying Phabricator DB via phab1005 fails with TLS errors - https://phabricator.wikimedia.org/T437242 (10Urbanecm) 03NEW [16:57:44] 10Phabricator, 06Collaboration-Services: Querying Phabricator DB via phab1005 fails with TLS errors - https://phabricator.wikimedia.org/T437242#12295597 (10Aklapper) Confirming; previously mentioned in T435442#12249892 [16:58:49] ref T406478 [16:58:49] T406478: Scap logs on Grafana dashboards are broken - https://phabricator.wikimedia.org/T406478 [18:00:55] 10Phabricator (Upstream), 07Upstream: Phantom unread Phabricator notification - https://phabricator.wikimedia.org/T399599#12295701 (10Umherirrender) I have now some unread notifications. It seems this is related to some of the phabricator bots, which may not trigger notifications. Often before the unread noti... [19:47:01] (03PS1) 10Umherirrender: zuul: [mediawiki/extensions/Disambiguator] Add MobileFrontend for phan [integration/config] - 10https://gerrit.wikimedia.org/r/1337652 [19:56:01] 10Beta-Cluster-Infrastructure, 06MediaWiki-Engineering, 10Cloud-VPS (Debian Bullseye Deprecation): Beta Cluster: Switch to PHP 8.5, includes writing Puppet changes to support PHP 8.5 on baremetal - https://phabricator.wikimedia.org/T435393#12295799 (10Southparkfan) `deployment-mediawiki15` and `deployment-me... [20:03:38] (03PS3) 10Phedenskog: Run the next npm install during the browser tests [integration/quibble] - 10https://gerrit.wikimedia.org/r/1335750 (https://phabricator.wikimedia.org/T436376) [20:25:22] 10Phabricator, 06Collaboration-Services: Querying Phabricator DB via phab1005 fails with TLS errors - https://phabricator.wikimedia.org/T437242#12295867 (10Arnoldokoth) @Urbanecm We updated the phabricator hosts to a newer Debian version (with a newer mysql client) and from what I've gathered the newer version... [20:28:10] 10Phabricator, 06Collaboration-Services: Querying Phabricator DB via phab1005 fails with TLS errors - https://phabricator.wikimedia.org/T437242#12295869 (10Urbanecm) > You can try adding this to your .my.cnf. That helps, but I can't really add it to .my.cnf (or at least not easily). That file is autocreated b... [20:41:26] 10Phabricator, 06Collaboration-Services: Querying Phabricator DB via phab1005 fails with TLS errors - https://phabricator.wikimedia.org/T437242#12295878 (10Arnoldokoth) Ooh. I had no idea the file is managed by Puppet. I'll push a patch to change it though realistically might be merged tomorrow (reviews and a... [20:47:14] 10Phabricator, 06Collaboration-Services, 13Patch-For-Review: Querying Phabricator DB via phab1005 fails with TLS errors - https://phabricator.wikimedia.org/T437242#12295895 (10Arnoldokoth) a:03Arnoldokoth [21:12:51] 10Phabricator (Upstream), 06Release-Engineering-Team (Doing 😎), 07Upstream: Active Bot usernames are struckout in search suggestions, like disabled accounts - https://phabricator.wikimedia.org/T227963#12295927 (10Aklapper) a:03Aklapper [21:31:12] (03PS9) 10Phedenskog: Run SQLite tests in parallel by having one db per worker [integration/quibble] - 10https://gerrit.wikimedia.org/r/1289330 (https://phabricator.wikimedia.org/T426684) [21:33:21] (03PS4) 10Phedenskog: Run the next npm install during the browser tests [integration/quibble] - 10https://gerrit.wikimedia.org/r/1335750 (https://phabricator.wikimedia.org/T436376) [23:15:41] (03CR) 10Pwangai: "Ready for review." [integration/quibble] - 10https://gerrit.wikimedia.org/r/1289330 (https://phabricator.wikimedia.org/T426684) (owner: 10Phedenskog)