[01:40:29] 10Continuous-Integration-Infrastructure, 10Castor, 06Test Platform (San Miguel de Allende 31): Widen the cache proxy npm trial from UploadWizard to six repos - https://phabricator.wikimedia.org/T444254 (10Mhurd) 03NEW [01:46:05] 10Continuous-Integration-Infrastructure, 10Castor, 13Patch-For-Review, 06Test Platform (San Miguel de Allende 31): Widen the cache proxy npm trial from UploadWizard to six repos - https://phabricator.wikimedia.org/T444254#12536944 (10Mhurd) a:03Mhurd [07:57:37] (03PS1) 10Matthias Mullie: Add Thanks dep for ReaderExperiments [integration/config] - 10https://gerrit.wikimedia.org/r/1368163 [08:32:41] (03PS1) 10QChris: Allow โ€œGerrit Managersโ€ to import history [extensions/CollabPatrol] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1368180 [08:32:41] (03CR) 10QChris: [V:03+2 C:03+2] Allow โ€œGerrit Managersโ€ to import history [extensions/CollabPatrol] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1368180 (owner: 10QChris) [08:32:44] (03PS1) 10QChris: Import done. Revoke import grants [extensions/CollabPatrol] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1368181 [08:32:44] (03CR) 10QChris: [V:03+2 C:03+2] Import done. Revoke import grants [extensions/CollabPatrol] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1368181 (owner: 10QChris) [09:15:08] 06Release-Engineering-Team (Doing ๐Ÿ˜Ž), 07Essential-Work, 05Release, 05Train Deployments: 1.48.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T442222#12537810 (10Aklapper) [10:09:28] (03CR) 10Hashar: [C:03+2] "Thx for the massive rebuild of instances! ๐Ÿ˜Ž" [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/1367902 (https://phabricator.wikimedia.org/T440668) (owner: 10Dduvall) [10:10:10] (03Merged) 10jenkins-bot: build: Build images with buildx [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/1367902 (https://phabricator.wikimedia.org/T440668) (owner: 10Dduvall) [10:30:13] (03PS1) 10Hashar: dockerfiles: reparent logstash-filter-verifier to Bookworm [integration/config] - 10https://gerrit.wikimedia.org/r/1368245 (https://phabricator.wikimedia.org/T383337) [10:31:30] (03PS1) 10Hashar: jjb: update logstash-filter-verifier to Bookworm [integration/config] - 10https://gerrit.wikimedia.org/r/1368246 (https://phabricator.wikimedia.org/T383337) [10:31:44] (03CR) 10Hashar: [C:03+2] dockerfiles: reparent logstash-filter-verifier to Bookworm [integration/config] - 10https://gerrit.wikimedia.org/r/1368245 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [10:32:28] (03CR) 10Hashar: "Done" [integration/config] - 10https://gerrit.wikimedia.org/r/1361876 (https://phabricator.wikimedia.org/T443253) (owner: 10Hashar) [10:33:08] (03Merged) 10jenkins-bot: dockerfiles: reparent logstash-filter-verifier to Bookworm [integration/config] - 10https://gerrit.wikimedia.org/r/1368245 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [10:33:11] (03PS1) 10Gkyziridis: inference-services: Trigger jobs on python/requirements.txt changes [integration/config] - 10https://gerrit.wikimedia.org/r/1368247 (https://phabricator.wikimedia.org/T444292) [10:33:17] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Migrate all CI jobs from Bullseye to Bookworm or later and drop Bullseye testing support - https://phabricator.wikimedia.org/T383337#12538285 (10hashar) [10:52:49] (03CR) 10Hashar: "Verified on Ia4308c4a53efc0e6bbb640d52448b5ae4a9a572f" [integration/config] - 10https://gerrit.wikimedia.org/r/1368246 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [10:52:52] (03CR) 10Hashar: [C:03+2] jjb: update logstash-filter-verifier to Bookworm [integration/config] - 10https://gerrit.wikimedia.org/r/1368246 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [10:53:52] (03CR) 10Hashar: [C:03+2] jjb: update logstash-filter-verifier to Bookworm [integration/config] - 10https://gerrit.wikimedia.org/r/1368246 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [10:54:31] (03Merged) 10jenkins-bot: jjb: update logstash-filter-verifier to Bookworm [integration/config] - 10https://gerrit.wikimedia.org/r/1368246 (https://phabricator.wikimedia.org/T383337) (owner: 10Hashar) [10:57:57] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Migrate all CI jobs from Bullseye to Bookworm or later and drop Bullseye testing support - https://phabricator.wikimedia.org/T383337#12538475 (10hashar) The two images lefts are: - `releng/operations-puppet` tracke... [11:36:31] (03merge) 10aklapper: PhabricatorProjectReportsController: Avoid null array key for period [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/132 (https://phabricator.wikimedia.org/T443689) [11:37:02] 10Phabricator (phabricator-next), 06Release-Engineering-Team (Doing ๐Ÿ˜Ž), 07PHP 8.5 support: Null array key error in PhabricatorProjectReportsController::renderDateControls() - https://phabricator.wikimedia.org/T443689#12538640 (10Aklapper) [11:49:08] (03PS1) 10STran: Add Thanks dependency to WikimediaCustomizations [integration/config] - 10https://gerrit.wikimedia.org/r/1368275 (https://phabricator.wikimedia.org/T441426) [12:27:12] 06Release-Engineering-Team (Doing ๐Ÿ˜Ž), 10Catalyst (Luka Ijo Pimeja Jan), 07Essential-Work, 13Patch-For-Review: CAPI/Patchdemo: Do not reset expiration to default on rebuild - https://phabricator.wikimedia.org/T443601#12538872 (10jnuche) 05Openโ†’03Resolved Fix in prod [12:54:18] (03CR) 10AikoChou: [C:03+1] inference-services: Trigger jobs on python/requirements.txt changes [integration/config] - 10https://gerrit.wikimedia.org/r/1368247 (https://phabricator.wikimedia.org/T444292) (owner: 10Gkyziridis) [13:11:02] (03CR) 10Hashar: [C:03+2] inference-services: Trigger jobs on python/requirements.txt changes [integration/config] - 10https://gerrit.wikimedia.org/r/1368247 (https://phabricator.wikimedia.org/T444292) (owner: 10Gkyziridis) [13:12:27] (03Merged) 10jenkins-bot: inference-services: Trigger jobs on python/requirements.txt changes [integration/config] - 10https://gerrit.wikimedia.org/r/1368247 (https://phabricator.wikimedia.org/T444292) (owner: 10Gkyziridis) [13:14:42] (03CR) 10Hashar: [C:03+2] "Deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/1368247 (https://phabricator.wikimedia.org/T444292) (owner: 10Gkyziridis) [13:25:40] (03CR) 10Hashar: "`WikimediaEvents` is a dependency of various extensions. It should not hard require `Thanks` at all, that should be an optional integratio" [integration/config] - 10https://gerrit.wikimedia.org/r/1368275 (https://phabricator.wikimedia.org/T441426) (owner: 10STran) [13:36:16] (03CR) 10Hashar: "Sorry I have pressed reply too early. The issue I referred to is when `TestKitchen`ย had been added as a hard dependency to `WikimediaEvent" [integration/config] - 10https://gerrit.wikimedia.org/r/1368275 (https://phabricator.wikimedia.org/T441426) (owner: 10STran) [13:55:23] (03CR) 10Ottomata: "Agree. Let's fix it in WikimediaEvents." [integration/config] - 10https://gerrit.wikimedia.org/r/1368275 (https://phabricator.wikimedia.org/T441426) (owner: 10STran) [15:29:15] (03CR) 10Krinkle: [C:03+2] Refactor tests framework [fresh] - 10https://gerrit.wikimedia.org/r/1347688 (owner: 10Hashar) [15:32:33] (03CR) 10Krinkle: [C:03+2] test: colorize output of assertions [fresh] - 10https://gerrit.wikimedia.org/r/1347656 (owner: 10Hashar) [15:35:16] (03Merged) 10jenkins-bot: Refactor tests framework [fresh] - 10https://gerrit.wikimedia.org/r/1347688 (owner: 10Hashar) [15:40:41] (03Merged) 10jenkins-bot: test: colorize output of assertions [fresh] - 10https://gerrit.wikimedia.org/r/1347656 (owner: 10Hashar) [16:02:25] 10Beta-Cluster-Infrastructure, 10Shellbox, 10SyntaxHighlight: SyntaxHighlight not working on Beta Cluster wikis (firejail creation failure) - https://phabricator.wikimedia.org/T435866#12540081 (10Krinkle) [16:02:26] 10Beta-Cluster-Infrastructure: syntaxhighlighter throwing errors on beta cluster - https://phabricator.wikimedia.org/T438653#12540084 (10Krinkle) โ†’14Duplicate dup:03T435866 [16:45:24] (03open) 10marostegui: deployments-calendar.yaml: Remove federico3 [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/284 [16:46:39] (03update) 10marostegui: deployments-calendar.yaml: Remove federico3 [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/284 [16:46:56] (03update) 10thcipriani: deployments-calendar.yaml: Remove federico3 [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/284 (owner: 10marostegui) [16:47:24] (03close) 10marostegui: deployments-calendar.yaml: Remove federico3 [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/275 [16:47:25] (03update) 10marostegui: deployments-calendar.yaml: Remove federico3 [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/275 [16:47:54] (03merge) 10thcipriani: deployments-calendar.yaml: Remove federico3 [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/284 (owner: 10marostegui) [17:08:20] (03open) 10project_16_bot_622d7b6d682aa66907ad1bffedf5b9e6: build(deps): bump golang.org/x/sys from 0.49.0 to 0.50.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/804 [17:08:20] (03update) 10project_16_bot_622d7b6d682aa66907ad1bffedf5b9e6: build(deps): bump golang.org/x/sys from 0.49.0 to 0.50.0 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/804 [17:17:25] 06Release-Engineering-Team (Doing ๐Ÿ˜Ž), 10Tool-gitlab-account-approval: Request for Wikimedia GitLab account approval โ€“ varunv04 - https://phabricator.wikimedia.org/T444322#12540727 (10Aklapper) 05Openโ†’03Resolved a:03Aklapper Approved. [18:26:01] 06Release-Engineering-Team (Radar), 07Essential-Work, 10Test Kitchen (Experiment Platform Sprint 32): Determine if we can expand TK experiment deployment windows - https://phabricator.wikimedia.org/T440612#12541164 (10KReid-WMF) I would prefer to keep a guardrail of having experiments start when we reasonabl... [19:00:48] 10GitLab (Account Approval), 06Release-Engineering-Team (Doing ๐Ÿ˜Ž): Request for Wikimedia GitLab account approval โ€“ varunv04 - https://phabricator.wikimedia.org/T444322#12541365 (10Aklapper) [19:18:53] 10Continuous-Integration-Config, 10CheckUser, 10MediaWiki-extensions-WikimediaAntiAbuse, 06Product Safety and Integrity, 07ci-test-error: CheckUser CI on REL1_46 (and older?) fails due to WikimediaAntiAbuse - https://phabricator.wikimedia.org/T444374 (10Reedy) 03NEW [19:19:00] 10Continuous-Integration-Config, 10CheckUser, 10MediaWiki-extensions-WikimediaAntiAbuse, 06Product Safety and Integrity, 07ci-test-error: CheckUser CI on REL1_46 (and older?) fails due to WikimediaAntiAbuse - https://phabricator.wikimedia.org/T444374#12541483 (10Reedy) p:05Triageโ†’03High [19:21:31] 10Continuous-Integration-Config, 10CheckUser, 10MediaWiki-extensions-WikimediaAntiAbuse, 06Product Safety and Integrity, and 4 others: CheckUser CI on REL1_46 (and older?) fails due to WikimediaAntiAbuse - https://phabricator.wikimedia.org/T444374#12541488 (10Reedy) [19:24:11] (03PS1) 10Reedy: zuul: Remove WikimediaAntiAbuse as a dependancy on REL1_43-REL1_46 [integration/config] - 10https://gerrit.wikimedia.org/r/1368371 (https://phabricator.wikimedia.org/T444374) [19:26:30] (03CR) 10Reedy: [C:03+2] zuul: Remove WikimediaAntiAbuse as a dependancy on REL1_43-REL1_46 [integration/config] - 10https://gerrit.wikimedia.org/r/1368371 (https://phabricator.wikimedia.org/T444374) (owner: 10Reedy) [19:27:44] (03Merged) 10jenkins-bot: zuul: Remove WikimediaAntiAbuse as a dependancy on REL1_43-REL1_46 [integration/config] - 10https://gerrit.wikimedia.org/r/1368371 (https://phabricator.wikimedia.org/T444374) (owner: 10Reedy) [19:28:20] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1368371 T444374 [19:28:22] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:28:23] T444374: CheckUser CI on REL1_46 (and older?) fails due to WikimediaAntiAbuse - https://phabricator.wikimedia.org/T444374 [19:30:43] 10Phabricator (Upstream), 07Upstream: Task appears in a search for tasks with me as the 'Other Assignee', after I've already removed myself as that task's 'Other Assignee' - https://phabricator.wikimedia.org/T415462#12541536 (10Aklapper) [20:01:11] (03update) 10dancy: requirements: Bump virtualenv, filelock, urllib3 and setuptools [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1421 [20:01:15] (03update) 10dancy: web: Update npm packages with security advisories [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1422 [20:01:22] (03open) 10dancy: web: Update npm packages with security advisories [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1422 [20:01:22] (03open) 10dancy: requirements: Bump virtualenv, filelock, urllib3 and setuptools [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1421 [20:02:33] (03update) 10dancy: requirements: Bump virtualenv, filelock, urllib3 and setuptools [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1421 [20:05:07] FIRING: [3x] PuppetAgentFailure: Puppet agent failure detected on instance deployment-cp-text09 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [20:05:15] (03merge) 10dancy: requirements: Bump virtualenv, filelock, urllib3 and setuptools [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1421 [20:05:19] (03update) 10dancy: web: Update npm packages with security advisories [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1422 [20:06:18] (03update) 10dancy: web: Update npm packages with security advisories [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1422 [20:11:26] (03merge) 10dancy: web: Update npm packages with security advisories [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1422 [20:14:26] (03CR) 10Ahmon Dancy: [C:03+1] "LGTM!" [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/1367903 (https://phabricator.wikimedia.org/T440668) (owner: 10Dduvall) [20:29:35] (03open) 10dancy: spiderpig: jobrunner: Wait for the database monitors of a job to stop [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1423 [20:29:55] (03update) 10dancy: spiderpig: jobrunner: Wait for the database monitors of a job to stop [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1423 [20:31:50] (03merge) 10dancy: spiderpig: jobrunner: Wait for the database monitors of a job to stop [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1423 [20:38:39] (03CR) 10Dduvall: [C:03+2] build: Support provenance and SBOM attestations [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/1367903 (https://phabricator.wikimedia.org/T440668) (owner: 10Dduvall) [20:39:23] (03Merged) 10jenkins-bot: build: Support provenance and SBOM attestations [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/1367903 (https://phabricator.wikimedia.org/T440668) (owner: 10Dduvall) [21:23:05] FIRING: PuppetAgentStaleLastRun: Last Puppet run was over 24 hours ago on instance deployment-webperf31 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentStaleLastRun [22:04:06] FIRING: PuppetAgentNoResources: No Puppet resources found on instance deployment-webperf31 on project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentNoResources [22:27:13] (03PS1) 10Hashar: dockerfiles: rebuild all base images [integration/config] - 10https://gerrit.wikimedia.org/r/1368395 (https://phabricator.wikimedia.org/T444391) [22:29:14] (03CR) 10Hashar: "And the big questions are:" [integration/config] - 10https://gerrit.wikimedia.org/r/1368395 (https://phabricator.wikimedia.org/T444391) (owner: 10Hashar) [22:32:39] 10Continuous-Integration-Config, 10CheckUser, 10MediaWiki-extensions-WikimediaAntiAbuse, 06Product Safety and Integrity, and 4 others: CheckUser CI on REL1_46 (and older?) fails due to WikimediaAntiAbuse - https://phabricator.wikimedia.org/T444374#12542234 (10Reedy) 05Openโ†’03Resolved a:03Reedy [22:50:36] (03open) 10dancy: spiderpig: Test the job queues without timing [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1424 [22:50:52] (03update) 10dancy: spiderpig: Test the job queues without timing [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1424 [22:50:59] 10GitLab (Administration, Settings & Policy), 06Release-Engineering-Team (Priority Backlog ๐Ÿ“ฅ), 06Collaboration-Services, 06Java-Scala-Standardization: Add CI_RELEASE_TOKEN secret for {name}-maven-release jobs in jenkins - https://phabricator.wikimedia.org/T379203#12542342 (10bd808) Documenting here bec... [22:51:07] FIRING: PuppetStaleCertificates: Found non-revoked Puppet certificates for 1 deleted instances on deployment-puppetserver-1 - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/PuppetStaleCertificates - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetStaleCertificates [22:52:35] (03merge) 10dancy: spiderpig: Test the job queues without timing [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1424 [22:53:20] (03open) 10dancy: spiderpig: Read the error log changes from the event stream [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1425 [22:53:51] (03update) 10dancy: spiderpig: Read the error log changes from the event stream [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1425 [22:56:44] (03merge) 10dancy: spiderpig: Read the error log changes from the event stream [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1425 [22:58:59] (03open) 10dancy: spiderpig: Warn when the logstash poll results are stale [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1426 [22:59:27] (03update) 10dancy: spiderpig: Warn when the logstash poll results are stale [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1426 [23:01:08] (03merge) 10dancy: spiderpig: Warn when the logstash poll results are stale [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1426 [23:07:07] (03CR) 10Snwachukwu: "I have submitted a patch to fix it in me" [integration/config] - 10https://gerrit.wikimedia.org/r/1368275 (https://phabricator.wikimedia.org/T441426) (owner: 10STran) [23:18:06] (03CR) 10LWatson: "Thanks for the update ๐Ÿ˜Š" [integration/config] - 10https://gerrit.wikimedia.org/r/1368275 (https://phabricator.wikimedia.org/T441426) (owner: 10STran) [23:18:46] 06Release-Engineering-Team (Doing ๐Ÿ˜Ž), 06Collaboration-Services, 10Release Pipeline, 13Patch-For-Review: ST6.1.4 Refactor Pipelinelib to use buildx and generate SBOMs - https://phabricator.wikimedia.org/T440668#12542414 (10dduvall) 05In progressโ†’03Resolved [23:59:57] 06Release-Engineering-Team (Doing ๐Ÿ˜Ž), 06WMF-NDA-Requests: Requesting access to WMF-NDA project for dtotten-wmf - https://phabricator.wikimedia.org/T444405#12542521 (10Aklapper) 05Openโ†’03Resolved a:03Aklapper Added after verification.