[07:19:47] 10Release-Engineering-Team (Radar), 10MediaWiki-Debug-Logger, 10observability, 07Developer Productivity, 13Patch-For-Review: Consider enabling all MW log channels by default for WMF - https://phabricator.wikimedia.org/T228838#9916417 (10hashar) a:05hashar→03None [07:20:08] (03CR) 10Hashar: [C:03+2] jjb: switch jobs to Quibble 1.9.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1047589 (owner: 10Hashar) [07:21:40] !log Switching Jenkins jobs to Quibble 1.9.1 [07:21:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:21:48] (03Merged) 10jenkins-bot: jjb: switch jobs to Quibble 1.9.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1047589 (owner: 10Hashar) [07:23:51] 10Phabricator, 06collaboration-services, 10LDAP-Access-Requests, 06SRE, 13Patch-For-Review: Offboard Lea WMDE (Lea Voget) from the WMF systems - https://phabricator.wikimedia.org/T368139#9916418 (10SLyngshede-WMF) I did find the username in data.yaml, see https://gerrit.wikimedia.org/r/c/operations/puppe... [07:31:43] Hello, the disk on one of the release hosts, releases1003, is almost full. Is there a place where cleanup is possible? I cleaned the apt cache, which resolved the 5% alert, but the disk is still quite full. [07:35:53] (03CR) 10Arthur taylor: [C:03+1] "I'm a little bit nervous about that, but I'm also around all this week and could imagine that we just try it and see how bad it goes." [integration/config] - 10https://gerrit.wikimedia.org/r/1047919 (https://phabricator.wikimedia.org/T361190) (owner: 10Kosta Harlan) [07:52:20] 06Release-Engineering-Team, 06collaboration-services: releases1003 file system over 90% full - https://phabricator.wikimedia.org/T368239 (10LSobanski) 03NEW [07:55:57] (03CR) 10Kosta Harlan: "> I'm a little bit nervous about that" [integration/config] - 10https://gerrit.wikimedia.org/r/1047919 (https://phabricator.wikimedia.org/T361190) (owner: 10Kosta Harlan) [08:01:35] 10Gerrit (Gerrit 3.10): Configure h2 cache pruning - https://phabricator.wikimedia.org/T367504#9916474 (10hashar) 05Open→03Declined `cachePruning.startTime` defaults to 01:00 and `cachePruning.interval` defaults to `1d`, so Gerrit now prune the cache on a daily basis. I don't think the startup clean cau... [08:18:09] 10Gerrit (Gerrit 3.10), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Upgrade to Gerrit 3.10 - https://phabricator.wikimedia.org/T367419#9916510 (10hashar) [08:24:33] 10Gerrit (Gerrit 3.10), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Upgrade to Gerrit 3.10 - https://phabricator.wikimedia.org/T367419#9916517 (10hashar) [08:29:04] 10Continuous-Integration-Infrastructure, 10ci-test-error (WMF-deployed Build Failure): ForeignResourceStructureTest flaky in CI due to "Failed to download resource at https://codeload.github.com" - https://phabricator.wikimedia.org/T362425#9916523 (10Michael) Got this failure again, this time with jQuery: ` 0... [08:31:58] 06Release-Engineering-Team, 06collaboration-services: releases1003 file system over 90% full - https://phabricator.wikimedia.org/T368239#9916528 (10hashar) [08:34:38] (03CR) 10Arthur taylor: [C:03+1] "The patch itself is fine. The risks that I am concerned about are:" [integration/config] - 10https://gerrit.wikimedia.org/r/1047919 (https://phabricator.wikimedia.org/T361190) (owner: 10Kosta Harlan) [08:39:19] !log releases1003: deleting left over temporary files from the MediaWiki branching (`rm -fR /tmp/mw-branching-*`) | T368239 [08:39:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:39:21] T368239: releases1003 file system over 90% full - https://phabricator.wikimedia.org/T368239 [08:50:45] jelto: I have cleared releases1003 [08:51:07] that is entirely due to the Jenkins job which cut the mediawiki deployment on a weekly basis [08:51:12] the job does not remove the previous material [08:51:15] 10Continuous-Integration-Infrastructure, 07Jenkins, 06Release-Engineering-Team, 06collaboration-services: releases1003 file system over 90% full - https://phabricator.wikimedia.org/T368239#9916651 (10hashar) That is the `/` partition being filed, though the application/services should write to a standalone... [08:51:17] and leaks data in /tmp [08:51:54] Filesystem Size Used Avail Use% Mounted on [08:51:54] /dev/vda1 125G 67G 52G 57% / [08:52:03] great thanks a lot [08:52:59] it should probably be repartitionned [08:53:05] cause everything happens in `/` [08:53:18] and surely the jenkins job should garbage collect things ;) [08:56:39] yeah I also thought about that and was surprised /srv is not a dedicated partition [08:59:59] jelto: maybe because it is a ganeti vm [09:00:08] or that got created a long time ago [09:00:10] or whatever :) [09:01:54] /srv/docker has a dedicated disk. We should be able to add another disk and mount it at /srv/ also in Ganeti. [09:34:45] (03open) 10jnuche: branch-cut-test-patches: clean up MW checkouts [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/83 (https://phabricator.wikimedia.org/T368239) [09:37:04] 10Continuous-Integration-Infrastructure, 07Jenkins, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: releases1003 file system over 90% full - https://phabricator.wikimedia.org/T368239#9916945 (10jnuche) Thanks for cleaning up hashar! I've created a patch to make `branch-cut-test... [09:37:42] (03update) 10jnuche: branch-cut-test-patches: clean up MW checkouts [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/83 (https://phabricator.wikimedia.org/T368239) [09:42:06] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 06serviceops, 06SRE, 13Patch-For-Review: Set all appservers to pooled=inactive in scap - https://phabricator.wikimedia.org/T368058#9916968 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by cgoubert@cumin1002 for host mw1407.eqiad.wm... [09:42:41] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 06serviceops, 06SRE, 13Patch-For-Review: Set all appservers to pooled=inactive in scap - https://phabricator.wikimedia.org/T368058#9916971 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by cgoubert@cumin1002 for host mw1420.eqiad.wm... [09:57:30] 10GitLab (Infrastructure), 06Release-Engineering-Team, 06collaboration-services: Upgrade GitLab to major version 17 - https://phabricator.wikimedia.org/T365675#9917016 (10ops-monitoring-bot) Cookbook cookbooks.sre.gitlab.upgrade was started by jelto@cumin1002 on GitLab host gitlab1004.wikimedia.org with reas... [10:04:30] 10GitLab (Infrastructure), 06Release-Engineering-Team, 06collaboration-services: Upgrade GitLab to major version 17 - https://phabricator.wikimedia.org/T365675#9917023 (10ops-monitoring-bot) Cookbook cookbooks.sre.gitlab.upgrade started by jelto@cumin1002 on GitLab host gitlab1004.wikimedia.org with reason:... [10:05:20] 10GitLab (Infrastructure), 06Release-Engineering-Team, 06collaboration-services: Upgrade GitLab to major version 17 - https://phabricator.wikimedia.org/T365675#9917024 (10Jelto) [10:24:23] 10Continuous-Integration-Infrastructure, 07Jenkins, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: releases1003 file system over 90% full - https://phabricator.wikimedia.org/T368239#9917048 (10Jelto) As discussed in IRC, the release VMs should probably have a separate disk moun... [10:32:16] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 06serviceops, 06SRE, 13Patch-For-Review: Set all appservers to pooled=inactive in scap - https://phabricator.wikimedia.org/T368058#9917087 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by cgoubert@cumin1002 for host mw1420.eqiad.wmnet... [10:37:04] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 06serviceops, 06SRE, 13Patch-For-Review: Set all appservers to pooled=inactive in scap - https://phabricator.wikimedia.org/T368058#9917094 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by cgoubert@cumin1002 for host mw1407.eqiad.wmnet... [10:55:24] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 06serviceops, 06SRE, 13Patch-For-Review: Set all appservers to pooled=inactive in scap - https://phabricator.wikimedia.org/T368058#9917115 (10Clement_Goubert) 05Open→03In progress p:05Triage→03Medium [11:17:33] (03open) 10sg912: Adding CIM repo [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/84 [11:18:37] (03update) 10sg912: Adding CIM repo [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/84 [11:19:04] (03update) 10sg912: Adding CIM repo [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/84 [11:22:22] 10Continuous-Integration-Infrastructure, 07Jenkins, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: releases1003 file system over 90% full - https://phabricator.wikimedia.org/T368239#9917196 (10jnuche) I don't think there's an expected increase in disk usage any time soon, mount... [12:01:30] (03PS1) 10Jforrester: Zuul: Fix composer-test-package-php74-or-later to include PHP 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1049135 [12:01:31] (03PS1) 10Jforrester: Zuul: Enable PHP 8.3 for master branch of MW extensions and skins [integration/config] - 10https://gerrit.wikimedia.org/r/1049136 (https://phabricator.wikimedia.org/T353362) [12:03:13] (03CR) 10Jforrester: [C:03+2] Zuul: Fix composer-test-package-php74-or-later to include PHP 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1049135 (owner: 10Jforrester) [12:05:07] (03Merged) 10jenkins-bot: Zuul: Fix composer-test-package-php74-or-later to include PHP 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1049135 (owner: 10Jforrester) [12:05:40] !log Zuul: Fix composer-test-package-php74-or-later to include PHP 8.3 [12:05:40] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:05:54] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 06serviceops, 06SRE, 13Patch-For-Review: Set all appservers to pooled=inactive in scap - https://phabricator.wikimedia.org/T368058#9917301 (10Clement_Goubert) [12:06:01] (03CR) 10Jforrester: [C:03+2] Zuul: Enable PHP 8.3 for master branch of MW extensions and skins [integration/config] - 10https://gerrit.wikimedia.org/r/1049136 (https://phabricator.wikimedia.org/T353362) (owner: 10Jforrester) [12:07:33] (03Merged) 10jenkins-bot: Zuul: Enable PHP 8.3 for master branch of MW extensions and skins [integration/config] - 10https://gerrit.wikimedia.org/r/1049136 (https://phabricator.wikimedia.org/T353362) (owner: 10Jforrester) [12:08:02] !log Zuul: Enable PHP 8.3 for master branch of MW extensions and skins, for T353362 [12:08:04] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:08:04] T353362: Make PHP 8.3 voting on development (master) branch of MW ecosystem (core, vendor, extensions, skins, libraries) - https://phabricator.wikimedia.org/T353362 [12:36:57] 10Continuous-Integration-Config, 07PHP 8.3 support: Make PHP 8.3 voting on development (master) branch of MW ecosystem (core, vendor, extensions, skins, libraries) - https://phabricator.wikimedia.org/T353362#9917383 (10Jdforrester-WMF) [12:39:39] oooh, exciting [12:54:57] 10Continuous-Integration-Config, 07PHP 8.3 support: Make PHP 8.3 voting on development (master) branch of MW ecosystem (core, vendor, extensions, skins, libraries) - https://phabricator.wikimedia.org/T353362#9917416 (10Jdforrester-WMF) OK, situation as of now: * Voting on master for the MW direct ecosystem (MW... [13:07:54] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/DonationInterface] Add experimental PHP 8.2/8.3 jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1049157 [13:07:54] (03PS1) 10Jforrester: Zuul: [mediawiki/services/parsoid] Make PHP 8.3 voting [integration/config] - 10https://gerrit.wikimedia.org/r/1049158 (https://phabricator.wikimedia.org/T353362) [13:08:37] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/DonationInterface] Add experimental PHP 8.2/8.3 jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1049157 (owner: 10Jforrester) [13:08:45] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/services/parsoid] Make PHP 8.3 voting [integration/config] - 10https://gerrit.wikimedia.org/r/1049158 (https://phabricator.wikimedia.org/T353362) (owner: 10Jforrester) [13:09:27] Lucas_WMDE: OK, PHP 8.3 is now voting in master for all[*] extensions and skins, and seems to be passing. Sorry for the slowness! [13:10:11] \o/ \o/ [13:10:14] sounds great, thank you! [13:10:18] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/DonationInterface] Add experimental PHP 8.2/8.3 jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1049157 (owner: 10Jforrester) [13:10:22] (03Merged) 10jenkins-bot: Zuul: [mediawiki/services/parsoid] Make PHP 8.3 voting [integration/config] - 10https://gerrit.wikimedia.org/r/1049158 (https://phabricator.wikimedia.org/T353362) (owner: 10Jforrester) [13:10:50] Lucas_WMDE: Next up, REL1_42, and a huge list of libraries, most of which probably already pass but aren't even voting on PHP 8.2 yet. [13:10:56] (I didn’t notice any slowness FWIW, I had just forgotten that it was intentional that it hadn’t been added months ago together with 8.2) [13:11:15] Yeah, well, I was disappointed in myself for not moving faster. :-) [13:11:55] And now all[*] of prod is running on mw-on-k8s, hopefully we'll jump at least to 8.1 very soon. [13:12:05] And so we can junk the slowest CI jobs. [13:51:02] (03update) 10sg912: Adding CIM repo [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/84 [13:51:23] (03update) 10sg912: Adding CIM repo [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/84 [14:23:36] (03CR) 10Jforrester: "check experimental" [integration/docroot] - 10https://gerrit.wikimedia.org/r/1047613 (owner: 10Libraryupgrader) [14:26:39] 10Phabricator, 06Security-Team, 07SecTeam-Processed, 07Security: Audit members of acl*security for more than 12 months of no activity (May 2024) - https://phabricator.wikimedia.org/T337305#9917802 (10sbassett) >>! In T337305#9916137, @Grunny wrote: > Was I caught up in this cleanup by chance @sbassett?... [14:30:49] (03PS1) 10Jforrester: Zuul: Enable PHP 8.3 as voting on master for all libraries except two [integration/config] - 10https://gerrit.wikimedia.org/r/1049189 (https://phabricator.wikimedia.org/T353362) [14:34:57] (03PS1) 10Jforrester: Zuul: Enable PHP 8.3 as voting on REL1_42 for all extensions and skins [integration/config] - 10https://gerrit.wikimedia.org/r/1049192 (https://phabricator.wikimedia.org/T353362) [14:35:52] (03CR) 10Jforrester: [C:03+2] Zuul: Enable PHP 8.3 as voting on master for all libraries except two [integration/config] - 10https://gerrit.wikimedia.org/r/1049189 (https://phabricator.wikimedia.org/T353362) (owner: 10Jforrester) [14:37:21] (03Merged) 10jenkins-bot: Zuul: Enable PHP 8.3 as voting on master for all libraries except two [integration/config] - 10https://gerrit.wikimedia.org/r/1049189 (https://phabricator.wikimedia.org/T353362) (owner: 10Jforrester) [14:37:53] !log Zuul: Enable PHP 8.3 as voting on master for all libraries except two, for T353362 [14:37:55] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:37:55] T353362: Make PHP 8.3 voting on development (master) branch of MW ecosystem (core, vendor, extensions, skins, libraries) - https://phabricator.wikimedia.org/T353362 [14:39:49] (03CR) 10Jforrester: [C:03+2] Zuul: Enable PHP 8.3 as voting on REL1_42 for all extensions and skins [integration/config] - 10https://gerrit.wikimedia.org/r/1049192 (https://phabricator.wikimedia.org/T353362) (owner: 10Jforrester) [14:40:50] 10Continuous-Integration-Config, 13Patch-For-Review, 07PHP 8.3 support: Make PHP 8.3 voting on development (master) branch of MW ecosystem (core, vendor, extensions, skins, libraries) - https://phabricator.wikimedia.org/T353362#9917894 (10Jdforrester-WMF) [14:41:07] 10Continuous-Integration-Config, 13Patch-For-Review, 07PHP 8.3 support: Make PHP 8.3 voting on development (master) branch of MW ecosystem (core, vendor, extensions, skins, libraries) - https://phabricator.wikimedia.org/T353362#9917898 (10Jdforrester-WMF) 05Open→03Resolved a:03Jdforrester-WMF I thi... [14:41:38] (03Merged) 10jenkins-bot: Zuul: Enable PHP 8.3 as voting on REL1_42 for all extensions and skins [integration/config] - 10https://gerrit.wikimedia.org/r/1049192 (https://phabricator.wikimedia.org/T353362) (owner: 10Jforrester) [14:42:15] !log Zuul: Enable PHP 8.3 as voting on REL1_42 for all extensions and skins, for T353362 [14:42:18] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:44:05] (03PS3) 10Hashar: Gerrit 3.10.x rebuild plugins and update TypeScript API [software/gerrit] (deploy/wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1047175 (https://phabricator.wikimedia.org/T367419) [15:00:17] 10Continuous-Integration-Infrastructure, 07Jenkins, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: releases1003 file system over 90% full - https://phabricator.wikimedia.org/T368239#9917974 (10dancy) >>! In T368239#9916651, @hashar wrote: > There is 3G+ in `/home/dancy`, I have... [15:17:50] 10Continuous-Integration-Infrastructure, 07Jenkins, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: releases1003 file system over 90% full - https://phabricator.wikimedia.org/T368239#9918020 (10LSobanski) Changing the partition layout in the correct way would mean reimaging the... [15:20:20] 10Continuous-Integration-Infrastructure, 07Jenkins, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: releases1003 file system over 90% full - https://phabricator.wikimedia.org/T368239#9918024 (10LSobanski) p:05Triage→03Medium [15:24:20] (03open) 10dancy: Fix indentation of repos/sre/miscweb/security-landing-page entry [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/85 [15:24:23] (03update) 10dancy: Fix indentation of repos/sre/miscweb/security-landing-page entry [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/85 [15:25:00] (03merge) 10dancy: Fix indentation of repos/sre/miscweb/security-landing-page entry [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/85 [15:25:48] 10GitLab (Infrastructure), 06collaboration-services: Import gitlab-runner package for bookworm on apt hosts - https://phabricator.wikimedia.org/T367717#9918052 (10LSobanski) p:05Triage→03Medium [15:28:06] 10GitLab (Infrastructure), 06collaboration-services: GitLab sre.gitlab.upgrade cookbook should lock all other backups during runtime - https://phabricator.wikimedia.org/T367501#9918070 (10LSobanski) p:05Triage→03Medium [15:29:40] 10Phabricator, 06collaboration-services, 10LDAP-Access-Requests, 06SRE: Offboard Lea WMDE (Lea Voget) from the WMF systems - https://phabricator.wikimedia.org/T368139#9918085 (10LSobanski) a:03Dzahn [15:33:48] (03open) 10dancy: scap clean: Add --dry-run mode [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/363 [15:33:50] (03update) 10dancy: scap clean: Add --dry-run mode [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/363 [15:35:53] (03update) 10dancy: scap clean: Add --dry-run mode [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/363 [15:44:53] (03open) 10dancy: Release 4.89.0-1 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/364 [15:46:41] (03merge) 10dancy: Release 4.89.0-1 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/364 [15:58:50] 10Gerrit (Gerrit 3.10), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Upgrade to Gerrit 3.10 - https://phabricator.wikimedia.org/T367419#9918314 (10hashar) [[ https://gerrit-review.googlesource.com/c/gerrit/+/400822 | 400822 - Don't email existing reviewers when adding new ones]... [16:03:36] 06Release-Engineering-Team, 10Scap: If aborting a scap due to test canary error rate, output some errors for reference - https://phabricator.wikimedia.org/T159991#9918324 (10dancy) 05In progress→03Resolved @Reedy wrote: > Why doesn't it give me a sample of the errors? :) As of scap 4.89.0 (deployed to... [16:03:57] jnuche: dancy: thanks for the follow up on release1003 disk space :D [16:05:13] 👍🏾 [16:08:12] 10GitLab (Pipeline Services Migration🐤), 10Data-Engineering (Q4 2024 April 1st - June 30th), 10Event-Platform, 13Patch-For-Review: Migrate Data Engineering Pipelinelib repos to GitLab - https://phabricator.wikimedia.org/T344730#9918332 (10Snwachukwu) [16:09:17] 10Release-Engineering-Team (Seen), 10Scap: Scap canary has a shifting baseline - https://phabricator.wikimedia.org/T183999#9918339 (10dancy) 05In progress→03Resolved As of scap 4.89.0, the threshold canary error rate is set by `canary_threshold` which has a default value of `10` which we expect to be s... [16:09:29] no prob! [16:12:49] 10Phabricator, 06Security-Team, 07SecTeam-Processed: Audit members of acl*security for more than 12 months of no activity (May 2025) - https://phabricator.wikimedia.org/T368224#9918357 (10sbassett) [16:14:10] 06Release-Engineering-Team, 10Scap: Did retrying canary checks do anything? - https://phabricator.wikimedia.org/T367131#9918376 (10dancy) 05In progress→03Resolved @Reedy wrote: > It seemed, that retrying didn't do anything; it seems to be reporting the same avg error rate, even after waiting a couple o... [16:17:47] 10Phabricator, 06Security-Team, 07SecTeam-Processed: Audit members of acl*security for more than 12 months of no activity (May 2025) - https://phabricator.wikimedia.org/T368224#9918402 (10sbassett) Stalled on completion date. If that's not proper, we can set the status to something else. The #security-team... [16:19:18] !log Upgrading scap to 4.89.0 in beta cluster [16:19:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:39:01] 10Gerrit (Gerrit 3.10), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Upgrade to Gerrit 3.10 - https://phabricator.wikimedia.org/T367419#9918536 (10Dzahn) I do think it does actually move a step back - the main problem we have is that reviewers don't react to being added as a re... [16:40:00] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap, 06Data-Platform-SRE, 13Patch-For-Review: "scap deploy"'s config-deploy should check for broken symlinks - https://phabricator.wikimedia.org/T342162#9918540 (10dancy) 05In progress→03Resolved Scap 4.89.0 was deployed today. It has changes rel... [16:48:13] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 07Performance Issue: Submitting actions taking ~10 seconds to load after making changes - https://phabricator.wikimedia.org/T360484#9918576 (10Aklapper) I continue to look into this from time to time but only updates so far are that the preceedi... [16:49:35] !log integration: rearmed keyholder on integration-cumin after the instance got rebooted due to a WMCS maintenance [16:49:35] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:06:05] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 07Performance Issue: Submitting actions taking ~10 seconds to load after making changes - https://phabricator.wikimedia.org/T360484#9918659 (10JWheeler-WMF) @Aklapper this seems to have subsided on my end. [18:10:41] (03PS1) 10Majavah: zuul: [mediawiki/extensions/OpenStackManager] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1049251 (https://phabricator.wikimedia.org/T367220) [18:38:48] 10GitLab (Administration, Settings & Policy), 10gitlab-settings, 10Release-Engineering-Team (Priority Backlog 📥), 13Patch-For-Review: Add a timer to run gitlab-settings/configure-projects - https://phabricator.wikimedia.org/T355097#9919030 (10brennen) 05In progress→03Resolved [18:45:41] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 13Patch-For-Review: Buildkit v0.14.0 released - https://phabricator.wikimedia.org/T367352#9919048 (10dancy) 05Open→03In progress p:05Triage→03Medium [19:37:53] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Yak Shaving 🐃🪒): Have linters/tests results show up as comments in files on gerrit - https://phabricator.wikimedia.org/T209149#9919209 (10Tgr) Gerrit 3.10 introduces a more structured way for suggestions: https://www.gerritcodereview.com/3.10... [20:28:25] 10WikimediaDebug, 10Tool-schedule-deployment: Integrate schedule-deployment with WikimediaDebug - https://phabricator.wikimedia.org/T367213#9919304 (10bd808) >>! In T367213#9906179, @jhsoby wrote: > Can't extensions add CSS to pages? I have some extensions that do that. (It might require more permissions for t... [20:50:18] 10Phabricator, 06Security-Team, 07SecTeam-Processed, 07Security: Audit members of acl*security for more than 12 months of no activity (May 2024) - https://phabricator.wikimedia.org/T337305#9919359 (10Grunny) Great, thanks @sbassett! [21:17:32] 10Phabricator, 06collaboration-services, 10LDAP-Access-Requests, 06SRE: Offboard Lea WMDE (Lea Voget) from the WMF systems - https://phabricator.wikimedia.org/T368139#9919459 (10Dzahn) >>! In T368139#9916418, @SLyngshede-WMF wrote: > I did find the username in data.yaml, see https://gerrit.wikimedia.org/r/... [21:22:56] 10GitLab-Application-Security-Pipeline, 10Security Team AppSec, 06Security-Team, 07Security: Attempt to condense trivy scanning output and avoid false positive exit code - https://phabricator.wikimedia.org/T367440#9919470 (10sbassett) Example of a template keeping track of cumulative bash error codes: http... [21:54:59] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Yak Shaving 🐃🪒): Have linters/tests results show up as comments in files on gerrit - https://phabricator.wikimedia.org/T209149#9919552 (10kostajh) >>! In T209149#9175757, @kostajh wrote: > https://wikitech.wikimedia.org/wiki/Tool:Fix_Suggeste...