[00:01:24] (03merge) 10dancy: Revert "README:md: Test change, to be reverted" [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/491 [00:03:49] 10Scap, 10MediaWiki-Release-Tools, 13Patch-For-Review: PHP Fatal error: Cannot redeclare wikidiff2_do_diff() - https://phabricator.wikimedia.org/T340862#11059457 (10dancy) @Reedy @legoktm You can now run `scap lint 10Scap, 10MediaWiki-Release-Tools, 13Patch-For-Review: PHP Fatal error: Cannot redeclare wikidiff2_do_diff() - https://phabricator.wikimedia.org/T340862#11059459 (10dancy) xref T301344 [00:15:43] 06Release-Engineering-Team, 10dev-images, 10MediaWiki-Docker, 10MediaWiki-Platform-Team (Radar): Replace php-tideways_xhprof with php-xhprof in MediaWiki-Docker - https://phabricator.wikimedia.org/T401115#11059464 (10Krinkle) [04:06:48] 10GitLab (Integrations), 10Phabricator, 06Release-Engineering-Team: Phorge GitLab widget generates writes to cache table for task views - https://phabricator.wikimedia.org/T401160 (10brennen) 03NEW [04:10:56] (03open) 10brennen: Draft: mr. widget locking: switch to getRuntimeCache() [repos/phabricator/extensions] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/extensions/-/merge_requests/56 (https://phabricator.wikimedia.org/T401160) [04:23:43] (03open) 10brennen: Temporarily disable PhabricatorWebserverSetupCheck [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/93 (https://phabricator.wikimedia.org/T401157) [06:06:27] 10Phabricator, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Phorge setup check caching is misbehaving, leading to many duck-sound=quack requests - https://phabricator.wikimedia.org/T401157#11059621 (10Jelto) [07:01:46] (03PS3) 10Jforrester: Zuul: [mediawiki/vagrant] Make ruby tests non-voting for now [integration/config] - 10https://gerrit.wikimedia.org/r/1175591 (https://phabricator.wikimedia.org/T398469) (owner: 10Gergő Tisza) [07:03:11] (03CR) 10CI reject: [V:04-1] Zuul: [mediawiki/vagrant] Make ruby tests non-voting for now [integration/config] - 10https://gerrit.wikimedia.org/r/1175591 (https://phabricator.wikimedia.org/T398469) (owner: 10Gergő Tisza) [07:03:59] (03PS4) 10Jforrester: Zuul: [mediawiki/vagrant] Make ruby tests non-voting for now [integration/config] - 10https://gerrit.wikimedia.org/r/1175591 (https://phabricator.wikimedia.org/T398469) (owner: 10Gergő Tisza) [07:05:29] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Review and clean up Unresolved Setup Issues in production - https://phabricator.wikimedia.org/T401163 (10Aklapper) 03NEW p:05Triage→03Medium [07:26:15] 10Phabricator, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Phorge setup check caching is misbehaving, leading to many duck-sound=quack requests - https://phabricator.wikimedia.org/T401157#11059702 (10hashar) >>! In T401157#11059391, @Dzahn wrote: > ... > BUT, actually running... [08:12:54] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11059809 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1003 for host gitlab2002.wikimedia.org with OS bookworm [08:14:52] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.45.0-wmf.13 deployment blockers - https://phabricator.wikimedia.org/T396374#11059812 (10hashar) The test wikis have not been promoted, that is supposed to happen automatically via the train presync task. [08:32:39] andre: good morning! :) [08:32:49] hashar: yo yo yo [08:33:06] last night there was a task about strange requests made to Phabricator by ... Phabricator itself [08:33:07] https://phabricator.wikimedia.org/T401157 [08:33:10] i know [08:33:14] ah cool :) [08:33:15] looking into it [08:33:44] time to add some log statements and printf :] [08:34:08] eheh, maybe. first want to understand the code a bit more [08:34:40] (...and close a bunch of other open browser tabs because peace of mind) [08:38:35] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.45.0-wmf.13 deployment blockers - https://phabricator.wikimedia.org/T396374#11059891 (10Aklapper) >>! In T396374#11059812, @hashar wrote: > The test wikis have not been promoted, that is supposed to happen automatically via th... [08:45:26] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/vagrant] Make ruby tests non-voting for now [integration/config] - 10https://gerrit.wikimedia.org/r/1175591 (https://phabricator.wikimedia.org/T398469) (owner: 10Gergő Tisza) [08:46:46] (03Merged) 10jenkins-bot: Zuul: [mediawiki/vagrant] Make ruby tests non-voting for now [integration/config] - 10https://gerrit.wikimedia.org/r/1175591 (https://phabricator.wikimedia.org/T398469) (owner: 10Gergő Tisza) [08:46:58] !log Zuul: [mediawiki/vagrant] Make ruby tests non-voting for now, for T398469 [08:47:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:47:06] T398469: Migrate mediawiki/vagrant CI to Bullseye and remove custom image - https://phabricator.wikimedia.org/T398469 [08:47:25] 10Beta-Cluster-Infrastructure, 06Infrastructure-Foundations, 13Patch-For-Review, 07Puppet: Puppet configures kernel.core_pattern |/usr/lib/systemd/systemd-coredump, but systemd-coredump is not installed - https://phabricator.wikimedia.org/T400247#11059899 (10Lucas_Werkmeister_WMDE) a:05Lucas_Werkmeister_W... [08:57:11] Project beta-scap-sync-world build #218186: 04FAILURE in 1 min 59 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/218186/ [09:02:28] FIRING: PuppetStaleCertificates: Found non-revoked Puppet certificates for 1 deleted instances on deployment-puppetserver-1 - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/PuppetStaleCertificates - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetStaleCertificates [09:02:36] 10Beta-Cluster-Infrastructure: Found non-revoked Puppet certificates for 1 deleted instances on deployment-puppetserver-1 - https://phabricator.wikimedia.org/T401174 (10wmcs-alerts) 03NEW [09:07:12] Project beta-scap-sync-world build #218187: 04STILL FAILING in 1 min 59 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/218187/ [09:17:08] Project beta-scap-sync-world build #218188: 04STILL FAILING in 1 min 55 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/218188/ [09:27:05] Project beta-scap-sync-world build #218189: 04STILL FAILING in 1 min 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/218189/ [09:34:35] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11060070 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1003 for host gitlab2002.wikimedia.org with OS bookworm executed wit... [09:37:05] Yippee, build fixed! [09:37:05] Project beta-scap-sync-world build #218190: 09FIXED in 1 min 54 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/218190/ [09:37:28] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Review and clean up Unresolved Setup Issues in production - https://phabricator.wikimedia.org/T401163#11060073 (10Aklapper) or better https://phabricator.wikimedia.org/config/edit/config.ignore-issues/ [09:37:34] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11060074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1003 for host gitlab2002.wikimedia.org with OS bookworm [09:59:37] 10Continuous-Integration-Infrastructure, 07Jenkins, 06Release-Engineering-Team: Jenkins fails to connect to node on releases2003 - https://phabricator.wikimedia.org/T401180 (10jnuche) 03NEW [10:17:08] 10GitLab (Integrations), 10Phabricator, 06Release-Engineering-Team, 10Wikimedia-Phabricator-Extensions, 13Patch-For-Review: Phorge GitLab widget generates writes to cache table for task views - https://phabricator.wikimedia.org/T401160#11060180 (10Aklapper) [10:23:23] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11060207 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1003 for host gitlab2002.wikimedia.org with OS bookworm completed: -... [10:31:44] 10GitLab (Infrastructure), 06collaboration-services: GitLab partman config is unreliable - https://phabricator.wikimedia.org/T399714#11060241 (10Jelto) p:05Triage→03Medium This happened again and a second reimage of `gitlab2002` was needed. [10:39:28] (03open) 10jnuche: jenkins-rel: determine agent host key dynamically [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/102 (https://phabricator.wikimedia.org/T401180) [10:40:10] (03merge) 10jnuche: jenkins-rel: determine agent host key dynamically [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/102 (https://phabricator.wikimedia.org/T401180) [11:23:14] 10Continuous-Integration-Infrastructure, 07Jenkins, 06Release-Engineering-Team, 13Patch-For-Review: Jenkins fails to connect to node on releases2003 - https://phabricator.wikimedia.org/T401180#11060422 (10jnuche) 05Open→03Resolved a:03jnuche Fixed now, Jenkins can connect to the agent (to its own... [11:34:34] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11060444 (10Jelto) [11:47:28] 10Phabricator, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Phorge setup check caching is misbehaving, leading to many duck-sound=quack requests - https://phabricator.wikimedia.org/T401157#11060461 (10Aklapper) Question: Do we still have data if the number of duck-sound=quack r... [12:55:04] maintenance-disconnect-full-disks build 725461 integration-agent-docker-1063 (/: 25%, /srv: 98%, /var/lib/docker: 33%): OFFLINE due to disk space [13:00:03] maintenance-disconnect-full-disks build 725462 integration-agent-docker-1063 (/: 25%, /srv: 7%, /var/lib/docker: 30%): RECOVERY disk space OK [13:01:56] !log Disable publishing on Phab tasks from mediawiki-libs-IPUtils mirror (rMLIP) to avoid duplicate notifs, T143162 [13:01:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:02:00] T143162: Reduce task notification noise/frequency of changes to associated open patchsets - https://phabricator.wikimedia.org/T143162 [13:34:43] is there a known issue about *phpunit* test builds getting stuck until they time out after an hour? I’ve seen it happen twice today [13:34:55] (IIUC this would be a separate issue from T389536, which affects *selenium* jobs) [13:34:56] T389536: Selenium timeouts can cause the job to remain stuck until the build times out - https://phabricator.wikimedia.org/T389536 [13:35:51] I don’t see anything phpunit-y in T388416… I’ll file a task later if nobody responds [13:35:52] T388416: CI jobs failing with various timeouts (March 2025) - https://phabricator.wikimedia.org/T388416 [13:48:26] (03CR) 10Hashar: [C:03+1] "+1 thanks, I should have done it when I have dropped the obsolete ruby 2.5 job." [integration/config] - 10https://gerrit.wikimedia.org/r/1175591 (https://phabricator.wikimedia.org/T398469) (owner: 10Gergő Tisza) [13:55:58] 10Phabricator, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Phorge setup check caching is misbehaving, leading to many duck-sound=quack requests - https://phabricator.wikimedia.org/T401157#11060873 (10brennen) > Question: Do we still have data if the number of duck-sound=quack... [14:03:22] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11060902 (10Jelto) [14:24:17] I filed T401204 for the PHPUnit timeouts I mentioned above [14:24:18] T401204: PHPUnit tests randomly time out in CI - https://phabricator.wikimedia.org/T401204 [14:35:12] 10Gerrit: Gerrit rate limit exceptions for Wikimania 2025 - https://phabricator.wikimedia.org/T401205 (10Chlod) 03NEW [14:36:18] 10Gerrit: Gerrit rate limit exceptions for Wikimania 2025 - https://phabricator.wikimedia.org/T401205#11061041 (10Chlod) Noting that the Hackathon room is only at the Trademark, so the Novotel IPs may not be necessary, though I wouldn't put it outside the realm of possibility for some late-night lobby hackers to... [15:12:10] (03merge) 10brennen: Temporarily disable PhabricatorWebserverSetupCheck [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/93 (https://phabricator.wikimedia.org/T401157) [15:15:50] 10Phabricator (phabricator-next), 06Release-Engineering-Team, 06collaboration-services: Deploy Phabricator/Phorge 2025-08-05 - https://phabricator.wikimedia.org/T401213 (10brennen) 03NEW [15:16:27] (03open) 10brennen: update submodules for 2025-08-05 deploy [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/76 (https://phabricator.wikimedia.org/T401213) [15:16:32] (03merge) 10brennen: update submodules for 2025-08-05 deploy [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/76 (https://phabricator.wikimedia.org/T401213) [15:26:35] 10Beta-Cluster-Infrastructure: Found non-revoked Puppet certificates for 1 deleted instances on deployment-puppetserver-1 - https://phabricator.wikimedia.org/T401174#11061320 (10bd808) 05Open→03Resolved a:03bd808 `lang=shell-session bd808@deployment-puppetserver-1:~$ sudo /usr/local/sbin/clean-stale-pu... [15:28:39] 10Beta-Cluster-Infrastructure: Found non-revoked Puppet certificates for 1 deleted instances on deployment-puppetserver-1 - https://phabricator.wikimedia.org/T401174#11061329 (10bd808) [15:29:38] (03open) 10swfrench: Draft: make-container-image: switch tideways to xhprof [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/200 (https://phabricator.wikimedia.org/T401152) [15:35:43] 10Phabricator (phabricator-next), 06Release-Engineering-Team, 06collaboration-services: Deploy Phabricator/Phorge 2025-08-05 - https://phabricator.wikimedia.org/T401213#11061388 (10brennen) 05Open→03Resolved [15:37:28] RESOLVED: PuppetStaleCertificates: Found non-revoked Puppet certificates for 1 deleted instances on deployment-puppetserver-1 - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/PuppetStaleCertificates - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetStaleCertificates [15:37:41] 10Beta-Cluster-Infrastructure: Found non-revoked Puppet certificates for 1 deleted instances on deployment-puppetserver-1 - https://phabricator.wikimedia.org/T401218 (10wmcs-alerts) 03NEW [15:49:28] 10Beta-Cluster-Infrastructure: Found non-revoked Puppet certificates for 1 deleted instances on deployment-puppetserver-1 - https://phabricator.wikimedia.org/T401174#11061548 (10bd808) [15:49:29] 10Beta-Cluster-Infrastructure: Found non-revoked Puppet certificates for 1 deleted instances on deployment-puppetserver-1 - https://phabricator.wikimedia.org/T401218#11061550 (10bd808) →14Duplicate dup:03T401174 [15:57:04] !log Cancelled "stuck" https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php81/23837/ job [15:57:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:58:51] bd808: that looks like another case of T401204 to me fwiw [15:58:52] T401204: PHPUnit tests randomly time out in CI - https://phabricator.wikimedia.org/T401204 [16:01:36] Lucas_WMDE: it does look very similar. fun times :/ [16:01:50] yeah :/ [16:02:12] and I don’t have sudo rights on the worker nodes afaict, so I can’t e.g. strace the processes even if I find another one of them [16:07:07] Lucas_WMDE: before I force new hats on you... would you like to be a member of the integration project so you can strace such things? [16:09:38] hi releng folks! would somebody be able to publish this latest update to the dev-images, please? [16:09:41] https://gitlab.wikimedia.org/repos/releng/dev-images/-/commit/a1f75553dd6bef8ec9d10165cf956442d472ef92 [16:11:23] ejegg: I can do that [16:13:24] !log Updating fundraising-civicrm-bookworm-php82-apache2 dev image [16:13:25] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:25:18] (03update) 10swfrench: make-container-image: switch tideways to xhprof [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/200 (https://phabricator.wikimedia.org/T401152) [16:27:21] ejegg: [16:27:21] Successfully published image docker-registry.discovery.wmnet/dev/fundraising-civicrm-bookworm-php82-apache2:1.1.3 [16:27:21] Successfully published image docker-registry.discovery.wmnet/dev/fundraising-bookworm-php82-apache2:1.0.1 [16:29:33] (03approved) 10cgoubert: make-container-image: switch tideways to xhprof [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/200 (https://phabricator.wikimedia.org/T401152) (owner: 10swfrench) [16:31:58] !log Promoted Lucas_WMDE to "member' (admin) in integration project so they can use `sudo`. [16:31:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:32:29] Godspeed Lucas_WMDE! [16:33:23] !log Added self as "member" (admin) in integration project so I don't have to remember to use my cloud root key there anymore. [16:33:24] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:36:48] 10Beta-Cluster-Infrastructure, 06DBA, 07Wikimedia-production-error: DBQueryError "The MariaDB server is running with the --read-only option" fails MainStash in Beta Cluster - https://phabricator.wikimedia.org/T401227 (10Krinkle) 03NEW [16:40:23] (03open) 10dancy: Drop all buster-based images [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/83 (https://phabricator.wikimedia.org/T382379) [16:40:24] (03update) 10dancy: Drop all buster-based images [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/83 (https://phabricator.wikimedia.org/T382379) [16:41:18] (03update) 10dancy: Drop all buster-based images [repos/releng/dev-images] - 10https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/83 (https://phabricator.wikimedia.org/T382379) [16:46:24] (03open) 10dancy: kubernetes: Use separate image repo for single version image build [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/935 (https://phabricator.wikimedia.org/T398875) [16:46:26] (03update) 10dancy: kubernetes: Use separate image repo for single version image build [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/935 (https://phabricator.wikimedia.org/T398875) [16:48:52] (03update) 10dancy: kubernetes: Use separate image repo for single version image build [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/935 (https://phabricator.wikimedia.org/T398875) [16:50:58] (03merge) 10dancy: kubernetes: Use separate image repo for single version image build [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/935 (https://phabricator.wikimedia.org/T398875) [16:51:02] 10Scap, 10MediaWiki-Release-Tools, 13Patch-For-Review: PHP Fatal error: Cannot redeclare wikidiff2_do_diff() - https://phabricator.wikimedia.org/T340862#11061821 (10dancy) [16:52:21] thanks dancy ! [16:53:46] (03open) 10dancy: config.py: Reenable build_mw_next_container_image [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/936 (https://phabricator.wikimedia.org/T398875) [16:53:49] (03update) 10dancy: config.py: Reenable build_mw_next_container_image [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/936 (https://phabricator.wikimedia.org/T398875) [16:54:58] (03merge) 10dancy: config.py: Reenable build_mw_next_container_image [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/936 (https://phabricator.wikimedia.org/T398875) [16:57:04] (03update) 10dancy: Release 4.196.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/937 [16:57:08] (03open) 10dancy: Release 4.196.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/937 [16:58:18] (03merge) 10dancy: Release 4.196.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/937 [17:01:02] 10Beta-Cluster-Infrastructure, 06DBA, 07Wikimedia-production-error: DBQueryError "The MariaDB server is running with the --read-only option" fails MainStash in Beta Cluster - https://phabricator.wikimedia.org/T401227#11061874 (10Krinkle) At glance these db hosts seem up and not marked read-only, according to... [17:05:00] 10WikimediaDebug, 10MediaWiki-Core-Profiler, 06MediaWiki-Platform-Team, 06serviceops, 13Patch-For-Review: Switch wmf-config/Profiler from Tideways to XHProf - https://phabricator.wikimedia.org/T401152#11061882 (10Krinkle) * Apply https://gerrit.wikimedia.org/r/c/1175623 (Puppet) in Beta Cluster. Which un... [17:06:13] (03merge) 10swfrench: make-container-image: switch tideways to xhprof [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/200 (https://phabricator.wikimedia.org/T401152) [18:07:36] Hi releng folks, we had a patch merged to the dev-images repo last night. Could someone with permissions please deploy the Docker image updates or direct me to the right place to request them? Thanks in advance https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/82 [18:19:36] jgleeson: Done! Actually it was done earlier at the request of ejegg. I posted a note in the MR just now. [18:19:48] thanks dancy ! [18:22:15] jgleeson: I merged your fr-dev docker-compose updates too! [18:23:07] oh great! thanks ejegg [18:28:19] 10GitLab (Integrations), 10Phabricator, 06Release-Engineering-Team, 10Wikimedia-Phabricator-Extensions, 13Patch-For-Review: Phorge GitLab widget generates writes to cache table for task views - https://phabricator.wikimedia.org/T401160#11062280 (10brennen) a:03brennen [18:33:20] 10Phabricator, 06Release-Engineering-Team, 06collaboration-services: Phorge setup check caching is misbehaving, leading to many duck-sound=quack requests - https://phabricator.wikimedia.org/T401157#11062294 (10brennen) p:05Triage→03Medium With the above mitigation in place, this seems less urgent in term... [18:42:26] zuul seems to have a 4h backlog on the gate. [18:42:55] nothing obvious stands out in terms of load. there arent' that many things on-going. I guess whatever caused it is gone now. [18:43:05] maybe some flaky tests? [18:43:39] https://grafana.wikimedia.org/d/d4bf3e68-9396-4173-a59c-59e398172ba0/zuul-prometheus?orgId=1&from=now-7d&to=now&timezone=utc&viewPanel=panel-7 [18:43:51] starting creeping up since 1PM UTC [18:48:13] https://integration.wikimedia.org/zuul/ shows its been busy on mediawiki/extensions/Cite since 4h18m but with an ETA of 1 min [18:48:37] literally just finished now [18:49:30] if you reload the grafana link now it also seems done [19:01:46] 10WikimediaDebug, 10MediaWiki-Core-Profiler, 06MediaWiki-Platform-Team, 06serviceops, 13Patch-For-Review: Switch wmf-config/Profiler from Tideways to XHProf - https://phabricator.wikimedia.org/T401152#11062362 (10Krinkle) 05Open→03Resolved p:05Triage→03Medium a:03Krinkle [19:03:02] 10WikimediaDebug, 10MediaWiki-Core-Profiler, 06MediaWiki-Platform-Team, 06serviceops, 13Patch-For-Review: Switch wmf-config/Profiler from Tideways to XHProf - https://phabricator.wikimedia.org/T401152#11062369 (10Krinkle) [19:03:15] 10WikimediaDebug, 10MediaWiki-Core-Profiler, 06MediaWiki-Platform-Team, 06serviceops, 13Patch-For-Review: Switch wmf-config/Profiler from Tideways to XHProf - https://phabricator.wikimedia.org/T401152#11062371 (10Krinkle) [19:03:34] 10WikimediaDebug, 10MediaWiki-Core-Profiler, 06MediaWiki-Platform-Team, 06serviceops, 13Patch-For-Review: Switch wmf-config/Profiler from Tideways to XHProf - https://phabricator.wikimedia.org/T401152#11062373 (10Krinkle) [19:04:05] mutante: ack, before that there was another change also taking 4h [19:04:20] whats on the front of the queue is slowly moving [19:29:39] Re-designed the search bar on both desktop and mobile, but on mobile the search experience has been updated: https://imgur.com/a/xPKbY2Z (easier to search) [19:30:54] you can also create labels now using the UI (no longer need to edit a config refs/meta/config) (not my work). [19:31:45] https://gerrit-review.googlesource.com/admin/repos/gerrit,labels [19:33:56] (03update) 10swfrench: Draft: make-container-image: add mediawiki publish-83 flavour [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/194 (https://phabricator.wikimedia.org/T399884) [19:35:06] (03update) 10swfrench: make-container-image: add mediawiki publish-83 flavour [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/194 (https://phabricator.wikimedia.org/T399884) [19:36:29] paladox: thanks!:) [20:23:43] (03PS1) 10BCornwall: Reapply "Archive operations/debs/trafficserver" [integration/config] - 10https://gerrit.wikimedia.org/r/1175944 [20:24:21] (03PS2) 10BCornwall: Reapply "Archive operations/debs/trafficserver" [integration/config] - 10https://gerrit.wikimedia.org/r/1175944 [20:26:40] (03PS1) 10BCornwall: Archive o/s/prometheus-rdkafka-exporter [integration/config] - 10https://gerrit.wikimedia.org/r/1175946 [20:26:57] (03PS2) 10BCornwall: Archive o/s/prometheus-rdkafka-exporter [integration/config] - 10https://gerrit.wikimedia.org/r/1175946 [20:32:02] 06Release-Engineering-Team, 10dev-images, 10MediaWiki-Docker: Switch PHP 8.3 image of MediaWiki-Docker from Sury to WMF - https://phabricator.wikimedia.org/T401255 (10Krinkle) 03NEW [20:32:14] 06Release-Engineering-Team, 10dev-images, 10MediaWiki-Docker: Switch PHP 8.3 image of MediaWiki-Docker from Sury to WMF - https://phabricator.wikimedia.org/T401255#11062650 (10Krinkle) [20:32:58] 06Release-Engineering-Team, 10dev-images, 10MediaWiki-Docker: Switch PHP 8.3 image of MediaWiki-Docker from Sury to WMF - https://phabricator.wikimedia.org/T401255#11062656 (10Krinkle) [20:32:59] 06Release-Engineering-Team, 10dev-images, 10MediaWiki-Docker, 10MediaWiki-Platform-Team (Radar): Replace php-tideways_xhprof with php-xhprof in MediaWiki-Docker - https://phabricator.wikimedia.org/T401115#11062655 (10Krinkle) [20:33:54] 06Release-Engineering-Team, 10dev-images, 10MediaWiki-Docker, 10MediaWiki-Platform-Team (Radar): Replace php-tideways_xhprof with php-xhprof in MediaWiki-Docker - https://phabricator.wikimedia.org/T401115#11062657 (10Krinkle) [20:34:22] (03CR) 10Ssingh: [C:03+1] Reapply "Archive operations/debs/trafficserver" [integration/config] - 10https://gerrit.wikimedia.org/r/1175944 (owner: 10BCornwall) [21:00:20] 10MediaWiki-Releasing, 10Release-Engineering-Team (Seen), 10MediaWiki-General: Use semantic(-ish) versioning for MediaWiki by dropping the "1." part of the version number - https://phabricator.wikimedia.org/T336128#11062696 (10Randomage) Considering the way in which things such as namespacing changes are app... [21:09:54] (03CR) 10BCornwall: [C:03+1] Reapply "Archive operations/debs/trafficserver" [integration/config] - 10https://gerrit.wikimedia.org/r/1175944 (owner: 10BCornwall) [21:25:12] 10Release-Engineering-Team (Priority Backlog 📥), 07OKR-Work: Publish updated wmf/next container when deploying config backport or security patch - https://phabricator.wikimedia.org/T398875#11062784 (10dancy) 05Open→03Resolved [21:41:57] (03open) 10dancy: pyproject.toml: Set minimum Python version to 3.9 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/938 [21:42:01] (03update) 10dancy: pyproject.toml: Set minimum Python version to 3.9 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/938 [21:42:52] (03open) 10dancy: Bump dependencies based on Dependabot recommendations [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/939 [21:42:56] (03update) 10dancy: Bump dependencies based on Dependabot recommendations [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/939 [21:44:21] (03merge) 10dancy: pyproject.toml: Set minimum Python version to 3.9 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/938 [21:44:51] (03update) 10dancy: Bump dependencies based on Dependabot recommendations [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/939 [21:49:56] (03open) 10dancy: pyproject.toml: Modernize license specification [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/940 [21:50:00] (03update) 10dancy: pyproject.toml: Modernize license specification [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/940 [21:51:33] (03merge) 10dancy: pyproject.toml: Modernize license specification [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/940 [21:51:55] (03update) 10dancy: Bump dependencies based on Dependabot recommendations [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/939 [22:23:07] Project beta-update-databases-eqiad build #86710: 04FAILURE in 3 min 6 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/86710/ [23:24:15] Yippee, build fixed! [23:24:15] Project beta-update-databases-eqiad build #86711: 09FIXED in 4 min 14 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/86711/