[00:16:01] 10Release-Engineering-Team (Priority Backlog šŸ“„), 10Scap, 06SRE Observability: Scap prometheus migration: Reduce the cardinality of scap timers/statsd metrics - https://phabricator.wikimedia.org/T377883#10334221 (10colewhite) >>! In T377883#10328206, @dancy wrote: > Is it possible to delete the data series wi... [00:30:42] 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Setup CI build stats reporting tool to produce ongoing reports on capacity and utilization - https://phabricator.wikimedia.org/T376830#10334260 (10bd808) 05In progressā†’03Resolved [x] Move the reporting code from Gerrit to GitLab to match Tool... [04:00:25] (03PS1) 10Samwilson: Zuul: [mediawiki/extensions/CommunityRequests] Add Translate as Phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1092415 (https://phabricator.wikimedia.org/T366194) [08:32:17] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/CommunityRequests] Add Translate as Phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1092415 (https://phabricator.wikimedia.org/T366194) (owner: 10Samwilson) [08:33:49] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/CommunityRequests] Add Translate as Phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1092415 (https://phabricator.wikimedia.org/T366194) (owner: 10Samwilson) [08:35:44] (03CR) 10Hashar: [C:03+2] "Deployed!!" [integration/config] - 10https://gerrit.wikimedia.org/r/1092415 (https://phabricator.wikimedia.org/T366194) (owner: 10Samwilson) [08:42:57] (03CR) 10Hashar: "The repository uses to be named `analytics/statsv` and it is renamed `performance/statsv`. Gerrit now supports renaming repositories since" [integration/config] - 10https://gerrit.wikimedia.org/r/1092319 (owner: 10Krinkle) [08:54:45] 10GitLab (Pipeline Services MigrationšŸ¤), 06collaboration-services, 13Patch-For-Review: move os-reports.wikimedia.org to kubernetes - https://phabricator.wikimedia.org/T350794#10334892 (10MoritzMuehlenhoff) >>! In T350794#10331098, @Arnoldokoth wrote: >> I can expose the generated reports to a read-only rsync... [09:35:03] maintenance-disconnect-full-disks build 650842 integration-agent-docker-1050 (/: 27%, /srv: 97%, /var/lib/docker: 31%): OFFLINE due to disk space [09:40:03] maintenance-disconnect-full-disks build 650843 integration-agent-docker-1050 (/: 27%, /srv: 44%, /var/lib/docker: 29%): RECOVERY disk space OK [10:01:32] (03PS6) 10Arthur taylor: Add daily job for quibble linear phpunit testing [integration/config] - 10https://gerrit.wikimedia.org/r/1087458 (https://phabricator.wikimedia.org/T372618) [10:01:55] (03CR) 10Arthur taylor: Add daily job for quibble linear phpunit testing (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1087458 (https://phabricator.wikimedia.org/T372618) (owner: 10Arthur taylor) [10:04:32] (03CR) 10Arthur taylor: Export list of gated skins and extensions as environment variables (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1088284 (https://phabricator.wikimedia.org/T372618) (owner: 10Arthur taylor) [10:05:09] (03CR) 10Kosta Harlan: [C:03+1] Add daily job for quibble linear phpunit testing [integration/config] - 10https://gerrit.wikimedia.org/r/1087458 (https://phabricator.wikimedia.org/T372618) (owner: 10Arthur taylor) [10:33:22] 10GitLab (Pipeline Services MigrationšŸ¤), 10Citoid, 06Editing-team: Archive Zotero gerrit repo - https://phabricator.wikimedia.org/T380259 (10Mvolz) 03NEW [10:53:25] 10GitLab (Pipeline Services MigrationšŸ¤), 10Citoid, 06Editing-team: Migrate Zotero Server repo from gerrit to GitLab - https://phabricator.wikimedia.org/T374558#10335202 (10Mvolz) FYI I took this opportunity to switch from master to main for the default branch name, so that might require some local updating! [10:53:55] 10GitLab (Pipeline Services MigrationšŸ¤), 10Citoid, 06Editing-team: Migrate Zotero Server repo from gerrit to GitLab - https://phabricator.wikimedia.org/T374558#10335214 (10Mvolz) [11:12:26] 10GitLab (Pipeline Services MigrationšŸ¤), 10Citoid, 06Editing-team: Migrate Zotero Server repo from gerrit to GitLab - https://phabricator.wikimedia.org/T374558#10335238 (10Mvolz) [11:13:34] 10GitLab (Pipeline Services MigrationšŸ¤), 10Citoid, 06Editing-team: Migrate Zotero Server repo from gerrit to GitLab - https://phabricator.wikimedia.org/T374558#10335241 (10Mvolz) [11:14:20] 10GitLab (Pipeline Services MigrationšŸ¤), 10Citoid, 06Editing-team: Archive Zotero gerrit repo - https://phabricator.wikimedia.org/T380259#10335245 (10Mvolz) p:05Triageā†’03Low [11:31:42] 10GitLab (Pipeline Services MigrationšŸ¤), 10Citoid, 06Editing-team, 10Projects-Cleanup: Archive Zotero gerrit repo - https://phabricator.wikimedia.org/T380259#10335278 (10Peachey88) [12:01:12] 10Beta-Cluster-Infrastructure, 10Citoid: Update citoid service on beta - https://phabricator.wikimedia.org/T380165#10335335 (10Mvolz) [12:07:59] 10Phabricator, 10Release-Engineering-Team (Priority Backlog šŸ“„): Decrease number of open Phab tickets with assignee field set for more than two years (aka cookie licking) (Q4/2024 edition) - https://phabricator.wikimedia.org/T365183#10335376 (10Aklapper) >>! In T365183#10308189, @Pppery wrote: > Is it intention... [12:31:59] 10GitLab (Infrastructure), 10Ceph, 06collaboration-services, 06SRE, 10SRE-swift-storage: Migrate gitlab storage to apus - https://phabricator.wikimedia.org/T378922#10335459 (10Jelto) [13:15:03] maintenance-disconnect-full-disks build 650886 integration-agent-docker-1046 (/: 28%, /srv: 95%, /var/lib/docker: 43%): OFFLINE due to disk space [13:20:03] maintenance-disconnect-full-disks build 650887 integration-agent-docker-1046 (/: 27%, /srv: 49%, /var/lib/docker: 42%): RECOVERY disk space OK [13:29:08] (03CR) 10Jforrester: [C:03+2] Docker: [php84] Upgrade to 8.4.0-rc4 (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1084250 (https://phabricator.wikimedia.org/T376274) (owner: 10Jforrester) [13:33:01] 10GitLab (Pipeline Services MigrationšŸ¤), 06collaboration-services, 13Patch-For-Review: move os-reports.wikimedia.org to kubernetes - https://phabricator.wikimedia.org/T350794#10335633 (10akosiaris) > We settled on what a deployment for this could look like. An initContainer that shares an emptyDir volume wit... [13:41:08] 10GitLab (Pipeline Services MigrationšŸ¤), 06collaboration-services, 13Patch-For-Review: move os-reports.wikimedia.org to kubernetes - https://phabricator.wikimedia.org/T350794#10335673 (10Arnoldokoth) >>! In T350794#10334892, @MoritzMuehlenhoff wrote: >>>! In T350794#10331098, @Arnoldokoth wrote: >>> I can ex... [14:03:07] 10GitLab (Pipeline Services MigrationšŸ¤), 06collaboration-services, 13Patch-For-Review: move os-reports.wikimedia.org to kubernetes - https://phabricator.wikimedia.org/T350794#10335759 (10MoritzMuehlenhoff) >>! In T350794#10335673, @Arnoldokoth wrote: >> Why is the initContainer needed if the sidecar will be... [14:19:00] 10GitLab (Pipeline Services MigrationšŸ¤), 10Citoid, 06Editing-team, 10Projects-Cleanup: Archive Zotero gerrit repo - https://phabricator.wikimedia.org/T380259#10335851 (10Jdforrester-WMF) [15:18:57] 10Deployments, 10Release-Engineering-Team (Doing šŸ˜Ž), 07Kubernetes, 13Patch-For-Review: Build and publish multiple MediaWiki production images for a given set of PHP versions - https://phabricator.wikimedia.org/T370934#10336137 (10thcipriani) I see 8.1 images getting published by our deploys: https://docker... [15:29:45] 06Project-Admins: create "ml-model-requests" tag - https://phabricator.wikimedia.org/T379161#10336252 (10XiaoXiao-WMF) Thank you ! [15:37:32] FIRING: [2x] DatasourceNoData: - https://alerts.wikimedia.org/?q=alertname%3DDatasourceNoData [15:47:32] RESOLVED: [2x] DatasourceNoData: - https://alerts.wikimedia.org/?q=alertname%3DDatasourceNoData [15:50:13] 10Deployments, 10Release-Engineering-Team (Doing šŸ˜Ž), 07Kubernetes, 13Patch-For-Review: Build and publish multiple MediaWiki production images for a given set of PHP versions - https://phabricator.wikimedia.org/T370934#10336393 (10Scott_French) No objections from my end - this all seems to be working nicely... [15:54:18] 10GitLab (Pipeline Services MigrationšŸ¤), 10Citoid, 06Editing-team, 10Projects-Cleanup: Archive Zotero gerrit repo - https://phabricator.wikimedia.org/T380259#10336438 (10Mvolz) [15:56:48] 10Release-Engineering-Team (Priority Backlog šŸ“„), 05Release, 05Train Deployments: 1.44.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T375663#10336471 (10Daimona) [15:57:54] 10GitLab (Pipeline Services MigrationšŸ¤), 10Citoid, 06Editing-team, 10Projects-Cleanup: Archive Zotero gerrit repo as it's been replaced with gitlab - https://phabricator.wikimedia.org/T380259#10336479 (10Mvolz) [15:58:07] 10GitLab (Pipeline Services MigrationšŸ¤), 10Citoid, 06Editing-team, 10Projects-Cleanup: Archive Zotero gerrit repo as it's been replaced with gitlab - https://phabricator.wikimedia.org/T380259#10336495 (10Mvolz) [15:58:21] 10GitLab (Pipeline Services MigrationšŸ¤), 10Citoid, 06Editing-team, 10Projects-Cleanup: Archive Zotero gerrit repo as it's been replaced with gitlab - https://phabricator.wikimedia.org/T380259#10336501 (10Mvolz) [16:02:26] 10GitLab (Pipeline Services MigrationšŸ¤), 10Citoid, 06Editing-team, 10Projects-Cleanup: Archive Zotero gerrit repo as it's been replaced with gitlab - https://phabricator.wikimedia.org/T380259#10336520 (10Mvolz) [16:05:01] (03CR) 10Arlolra: [C:03+2] Tweaks for ruwikivoyage & jawikivoyage to suppress noise [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1091788 (owner: 10Subramanya Sastry) [16:05:40] (03Merged) 10jenkins-bot: Tweaks for ruwikivoyage & jawikivoyage to suppress noise [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1091788 (owner: 10Subramanya Sastry) [16:10:25] (03merge) 10dancy: Navigation.vue: Fix computed title for job viewer page [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/580 [16:15:08] (03open) 10dancy: mwscript.py: Bind-mount host SSL cert directories [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/582 [16:15:10] (03update) 10dancy: mwscript.py: Bind-mount host SSL cert directories [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/582 [16:16:06] (03update) 10dancy: mwscript.py: Bind-mount host SSL cert directories [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/582 [16:20:05] 10Release-Engineering-Team (Priority Backlog šŸ“„), 10Scap, 06SRE Observability: Scap prometheus migration: Reduce the cardinality of scap timers/statsd metrics - https://phabricator.wikimedia.org/T377883#10336612 (10dancy) 05Openā†’03Resolved p:05Triageā†’03Low a:03dancy [16:20:21] (03update) 10dancy: mwscript.py: Bind-mount host SSL cert directories [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/582 [16:23:14] (03update) 10dancy: mwscript.py: Bind-mount host SSL cert directories [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/582 [16:47:11] (03PS1) 10Ilias Sarantopoulos: Revert "inference-services: remove llm CI pipeline jobs" [integration/config] - 10https://gerrit.wikimedia.org/r/1092874 [16:47:23] (03PS2) 10Ilias Sarantopoulos: Revert "inference-services: remove llm CI pipeline jobs" [integration/config] - 10https://gerrit.wikimedia.org/r/1092874 [16:51:10] (03CR) 10CI reject: [V:04-1] Revert "inference-services: remove llm CI pipeline jobs" [integration/config] - 10https://gerrit.wikimedia.org/r/1092874 (owner: 10Ilias Sarantopoulos) [17:10:27] (03open) 10dancy: make-container-image/build-images.py: Remove env variable parameter support [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/136 [17:10:29] (03update) 10dancy: make-container-image/build-images.py: Remove env variable parameter support [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/136 [17:12:25] (03open) 10lwatson: web: add a JobCard [repos/releng/scap] (T379412) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/583 (https://phabricator.wikimedia.org/T379412) [17:17:06] (03update) 10lwatson: web: add a JobCard [repos/releng/scap] (T379412) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/583 (https://phabricator.wikimedia.org/T379412) [17:18:37] 10Phabricator (Upstream), 10Release-Engineering-Team (Priority Backlog šŸ“„), 07Upstream: Uploaded files via the drag-and-drop are defaulting to private-access - https://phabricator.wikimedia.org/T310833#10336904 (10Aklapper) a:05brennenā†’03None @brennen: Removing task assignee as this open task has been ass... [17:29:02] 10GitLab (Pipeline Services MigrationšŸ¤), 06collaboration-services: move os-reports.wikimedia.org to kubernetes - https://phabricator.wikimedia.org/T350794#10337004 (10akosiaris) > The initContainer covers us for cases when the volume is empty (like with a new deployment) then the sidecar takes care of subseque... [17:30:42] 10Release-Engineering-Team (Radar), 06tech-decision-forum: Beta Cluster Tech Decision Forum - https://phabricator.wikimedia.org/T308283#10336964 (10Aklapper) a:05TAdeleye_WMFā†’03None @TAdeleye_WMF: Removing task assignee as this open task has been assigned for more than two years - See the email sent to tas... [17:30:44] Project beta-update-databases-eqiad build #80490: 04FAILURE in 10 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/80490/ [17:33:03] 10Phabricator, 07PM: Reset status of tickets that languish in "in progress" - https://phabricator.wikimedia.org/T380300 (10Pppery) 03NEW [17:38:05] (03update) 10lwatson: web: improve main page UI [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/575 (https://phabricator.wikimedia.org/T379412) [17:47:25] (03update) 10lwatson: web: improve main page UI [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/575 (https://phabricator.wikimedia.org/T379412) [17:48:36] brennen: Hey, if you could manage to get hold of Community-Tech folks to look into https://phabricator.wikimedia.org/T380304 that would be awesome as I'm on my way out for a few hours - TIA! [18:01:27] andre: i'll be in and out a bit as well, but i'll do some wrangling. [18:01:34] <3 [18:09:38] (03CR) 10Arlolra: [C:03+2] Change some default paths & settings [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1091789 (owner: 10Subramanya Sastry) [18:10:20] (03Merged) 10jenkins-bot: Change some default paths & settings [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1091789 (owner: 10Subramanya Sastry) [18:21:59] 10Phabricator, 10Release-Engineering-Team (Priority Backlog šŸ“„): Decrease number of open Phab tickets with assignee field set for more than two years (aka cookie licking) (Q4/2024 edition) - https://phabricator.wikimedia.org/T365183#10337669 (10Aklapper) 05Openā†’03Resolved ~165 tasks unassigned from 77 f... [18:22:46] 10Phabricator, 10Release-Engineering-Team (Priority Backlog šŸ“„): Decrease number of open Phab tickets with assignee field set for more than two years (aka cookie licking) (Q2/2025 edition) - https://phabricator.wikimedia.org/T380312 (10Aklapper) 03NEW p:05Triageā†’03Low [18:30:51] Yippee, build fixed! [18:30:51] Project beta-update-databases-eqiad build #80491: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/80491/ [18:54:29] (03update) 10lwatson: web: add a JobCard [repos/releng/scap] (T379412) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/583 (https://phabricator.wikimedia.org/T379412) [19:07:59] 10Deployments, 10Release-Engineering-Team (Doing šŸ˜Ž), 07Kubernetes, 13Patch-For-Review: Build and publish multiple MediaWiki production images for a given set of PHP versions - https://phabricator.wikimedia.org/T370934#10337819 (10dduvall) 05Openā†’03Resolved >>! In T370934#10336393, @Scott_French wro... [19:10:32] (03CR) 10Arlolra: [C:03+2] Add url config for incubator wiki - lets us run tests for that wiki [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1091790 (owner: 10Subramanya Sastry) [19:10:54] (03CR) 10Arlolra: [C:03+2] Add ruwikivoyage CSS workarounds to suppress some noise [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1091800 (owner: 10Subramanya Sastry) [19:11:23] (03Merged) 10jenkins-bot: Add url config for incubator wiki - lets us run tests for that wiki [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1091790 (owner: 10Subramanya Sastry) [19:11:29] (03Merged) 10jenkins-bot: Add ruwikivoyage CSS workarounds to suppress some noise [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1091800 (owner: 10Subramanya Sastry) [19:48:57] 10Release-Engineering-Team (Priority Backlog šŸ“„), 05MediaWiki-backport-deployments: Backport and config deploy: Speculative future backports - https://phabricator.wikimedia.org/T378428#10338010 (10thcipriani) [19:49:09] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for Gian Klug - https://phabricator.wikimedia.org/T379754#10338006 (10thcipriani) 05Openā†’03Resolved a:03thcipriani šŸ‘‹ hi @Gian_Klug I've approved your account. Welcome! [19:52:57] 10Gerrit, 06Release-Engineering-Team, 07Upstream: Link with ampersand is broken in commit messages - https://phabricator.wikimedia.org/T374772#10338028 (10thcipriani) 05Openā†’03Resolved a:03Paladox Looks like @Paladox 's fix landed a while back and we deployed it. https://gerrit.wikimedia.org/r/c/la... [20:11:55] 10GitLab (Pipeline Services MigrationšŸ¤), 06collaboration-services: move os-reports.wikimedia.org to kubernetes - https://phabricator.wikimedia.org/T350794#10338092 (10Arnoldokoth) >>! In T350794#10337004, @akosiaris wrote: >> The initContainer covers us for cases when the volume is empty (like with a new deplo... [20:12:50] 06Release-Engineering-Team, 10Projects-Cleanup, 10Wikimedia-GitHub: Archive unused limn-*-data Gerrit repositories and delete their Github mirrors - https://phabricator.wikimedia.org/T372412#10338097 (10thcipriani) Archived: https://phabricator.wikimedia.org/diffusion/ANLL/manage/ https://github.com/wikimed... [20:17:22] (03update) 10dancy: make-container-image/build-images.py: Remove env variable parameter support [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/136 [20:18:20] (03update) 10lwatson: web: add a JobCard [repos/releng/scap] (T379412) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/583 (https://phabricator.wikimedia.org/T379412) [20:28:00] (03close) 10lwatson: web: add a JobCard [repos/releng/scap] (T379412) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/583 (https://phabricator.wikimedia.org/T379412) [21:03:45] (03update) 10lwatson: web: improve main page UI [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/575 (https://phabricator.wikimedia.org/T379412) [21:09:40] (03CR) 10Ahmon Dancy: [C:03+1] Streamline build process, remove scap, use Puppet for deploys (031 comment) [releng/phatality] - 10https://gerrit.wikimedia.org/r/1088659 (https://phabricator.wikimedia.org/T342476) (owner: 10Cwhite) [21:48:13] (03open) 10dancy: jobrunner: Set suitable job status when awaiting user interaction [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/584 [21:48:16] (03update) 10dancy: jobrunner: Set suitable job status when awaiting user interaction [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/584 [21:53:51] (03merge) 10dancy: jobrunner: Set suitable job status when awaiting user interaction [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/584 [22:02:22] (03PS1) 10Cwhite: update readme with phatality version hiera key [releng/phatality] - 10https://gerrit.wikimedia.org/r/1092927 (https://phabricator.wikimedia.org/T342476) [22:59:09] (03update) 10lwatson: Draft: web: improve main page UI [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/575 (https://phabricator.wikimedia.org/T379412) [23:04:51] (03approved) 10dduvall: make-container-image/build-images.py: Remove env variable parameter support [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/136 (owner: 10dancy)