[00:54:01] !log On deployment-cache-{text,upload}06, ran: touch /srv/trafficserver/tls/etc/ssl_multicert.config && systemctl reload trafficserver-tls.service . Certificate was close to expiry [00:54:02] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:56:32] 10Beta-Cluster-Infrastructure, 10Quality-and-Test-Engineering-Team (QTE), 10SRE, 10Traffic, and 3 others: [epic] The SSL certificate for Beta cluster domains fails to properly renew & deploy - https://phabricator.wikimedia.org/T293585 (10ori) We got alerts about the Beta Cluster cert being close to expiry... [09:23:49] 10Project-Admins: Requests for addition to the #acl*Project-Admins group (in comments) - https://phabricator.wikimedia.org/T706 (10Evelien_WMDE) Hi there, I'm a PM for Wikibase at WMDE, and I'd like to be added to #acl_project-admins. I want to be able to create milestones and sub-projects in #Wikibase.cloud for... [14:27:49] 10Beta-Cluster-Infrastructure, 10Discovery-Search: Verify (and/or fix) Elasticsearch beta cluster problems - https://phabricator.wikimedia.org/T313521 (10bking) Sorry for the delay, we've been pretty busy with production issues. Looking at this now. I suspect we may need more disk space, as hinted at in T308353 . [14:31:06] 10GitLab (Infrastructure), 10Data-Persistence-Backup, 10serviceops, 10serviceops-collab, and 2 others: Backups for GitLab - https://phabricator.wikimedia.org/T274463 (10Jelto) >>! In T274463#8119412, @jcrespo wrote: > **Edit**: There are multiple issues with the current setup- bacula is getting weird files... [14:31:22] 10GitLab (Infrastructure), 10Data-Persistence-Backup, 10serviceops, 10serviceops-collab, and 2 others: Backups for GitLab - https://phabricator.wikimedia.org/T274463 (10Jelto) a:03Jelto [15:16:05] 10Beta-Cluster-Infrastructure, 10Discovery-Search: Verify (and/or fix) Elasticsearch beta cluster problems - https://phabricator.wikimedia.org/T313521 (10bking) Per the comment above, I disabled disk allocation with the following command: `curl -s -H 'Content-Type: Application/json' -XPUT localhost:9200/_clust... [15:57:06] 10Beta-Cluster-Infrastructure, 10Citoid, 10Editing-team, 10SRE: Upgrade deployment-docker-citoid01 host to Buster - https://phabricator.wikimedia.org/T306049 (10Mvolz) [16:44:21] (03PS1) 10Krinkle: jjb: Remove 'compress junit' postbuildscript step [integration/config] - 10https://gerrit.wikimedia.org/r/822663 (https://phabricator.wikimedia.org/T225730) [16:47:28] (03PS3) 10Jeena Huneidi: Scap backport --list: Add mergeable column [tools/scap] - 10https://gerrit.wikimedia.org/r/810078 (https://phabricator.wikimedia.org/T303967) [16:50:57] (03PS1) 10Cwhite: Support OpenSearch-Dashboards 2.2.0 [releng/phatality] - 10https://gerrit.wikimedia.org/r/822664 (https://phabricator.wikimedia.org/T304440) [16:51:21] (03CR) 10CI reject: [V: 04-1] Scap backport --list: Add mergeable column [tools/scap] - 10https://gerrit.wikimedia.org/r/810078 (https://phabricator.wikimedia.org/T303967) (owner: 10Jeena Huneidi) [16:55:19] (03PS2) 10Cwhite: Support OpenSearch-Dashboards 2.2.0 [releng/phatality] - 10https://gerrit.wikimedia.org/r/822664 (https://phabricator.wikimedia.org/T304440) [17:21:31] (03PS4) 10Jeena Huneidi: Scap backport --list: Add mergeable column [tools/scap] - 10https://gerrit.wikimedia.org/r/810078 (https://phabricator.wikimedia.org/T303967) [17:23:24] (03CR) 10Ahmon Dancy: "Testing in progress." [tools/train-dev] - 10https://gerrit.wikimedia.org/r/822463 (https://phabricator.wikimedia.org/T303967) (owner: 10Jeena Huneidi) [17:28:34] (03CR) 10Ahmon Dancy: [C: 03+2] Update prettytable version for scap [tools/train-dev] - 10https://gerrit.wikimedia.org/r/822463 (https://phabricator.wikimedia.org/T303967) (owner: 10Jeena Huneidi) [17:31:48] (03Merged) 10jenkins-bot: Update prettytable version for scap [tools/train-dev] - 10https://gerrit.wikimedia.org/r/822463 (https://phabricator.wikimedia.org/T303967) (owner: 10Jeena Huneidi) [17:40:06] (03PS2) 10Jforrester: jjb: [quibble] Remove 'compress junit' postbuildscript step [integration/config] - 10https://gerrit.wikimedia.org/r/822663 (https://phabricator.wikimedia.org/T225730) (owner: 10Krinkle) [17:41:09] James_F: oops? [17:42:59] !log Restarting Jenkins in an attempt to get CI jobs running again [17:43:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:47:08] !log Restarting zuul [17:47:09] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:48:29] Krinkle ? [17:48:42] (03CR) 10Jeena Huneidi: "recheck" [tools/scap] - 10https://gerrit.wikimedia.org/r/810078 (https://phabricator.wikimedia.org/T303967) (owner: 10Jeena Huneidi) [17:51:15] James_F: output/* files in PS2 [17:51:44] Oh, FFS, this machine is crying from the load. I removed those, but apparently it lost track. [17:52:18] (03PS3) 10Jforrester: jjb: [quibble] Remove 'compress junit' postbuildscript step [integration/config] - 10https://gerrit.wikimedia.org/r/822663 (https://phabricator.wikimedia.org/T225730) (owner: 10Krinkle) [17:53:11] (03CR) 10CI reject: [V: 04-1] Scap backport --list: Add mergeable column [tools/scap] - 10https://gerrit.wikimedia.org/r/810078 (https://phabricator.wikimedia.org/T303967) (owner: 10Jeena Huneidi) [18:04:20] (03CR) 10C. Scott Ananian: [C: 03+2] Strip p-wrappers around "empty" paragraphs [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/820579 (owner: 10Subramanya Sastry) [18:05:08] (03Merged) 10jenkins-bot: Strip p-wrappers around "empty" paragraphs [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/820579 (owner: 10Subramanya Sastry) [18:09:10] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/822663 (https://phabricator.wikimedia.org/T225730) (owner: 10Krinkle) [18:11:12] (03Merged) 10jenkins-bot: jjb: [quibble] Remove 'compress junit' postbuildscript step [integration/config] - 10https://gerrit.wikimedia.org/r/822663 (https://phabricator.wikimedia.org/T225730) (owner: 10Krinkle) [18:16:40] (03PS5) 10Jeena Huneidi: Scap backport --list: Add mergeable column [tools/scap] - 10https://gerrit.wikimedia.org/r/810078 (https://phabricator.wikimedia.org/T303967) [18:16:59] (03PS2) 10Jforrester: jjb: Switch coverage jobs from PHP7.3 to PHP7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/821793 [18:17:04] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/821793 (owner: 10Jforrester) [18:17:08] (03PS2) 10Jforrester: dockerfiles: Drop quibble-buster-php73-coverage, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/821794 [18:17:11] (03CR) 10Jforrester: [C: 03+2] dockerfiles: Drop quibble-buster-php73-coverage, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/821794 (owner: 10Jforrester) [18:18:46] (03PS6) 10Jeena Huneidi: Scap backport --list: Add mergeable column [tools/scap] - 10https://gerrit.wikimedia.org/r/810078 (https://phabricator.wikimedia.org/T303967) [18:19:09] (03Merged) 10jenkins-bot: jjb: Switch coverage jobs from PHP7.3 to PHP7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/821793 (owner: 10Jforrester) [18:19:11] (03Merged) 10jenkins-bot: dockerfiles: Drop quibble-buster-php73-coverage, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/821794 (owner: 10Jforrester) [18:32:48] (03PS3) 10Jforrester: [WIP] dockerfiles: Provide PHP 8.2 CI test images [integration/config] - 10https://gerrit.wikimedia.org/r/818239 (https://phabricator.wikimedia.org/T314093) [18:35:41] (03PS1) 10Jeena Huneidi: Change prettytable version for scap [tools/train-dev] - 10https://gerrit.wikimedia.org/r/822673 (https://phabricator.wikimedia.org/T303967) [18:49:28] (03PS2) 10Jeena Huneidi: Change prettytable version for scap [tools/train-dev] - 10https://gerrit.wikimedia.org/r/822673 (https://phabricator.wikimedia.org/T303967) [18:50:12] (03CR) 10Ahmon Dancy: [C: 03+2] Change prettytable version for scap [tools/train-dev] - 10https://gerrit.wikimedia.org/r/822673 (https://phabricator.wikimedia.org/T303967) (owner: 10Jeena Huneidi) [18:50:56] (03Merged) 10jenkins-bot: Change prettytable version for scap [tools/train-dev] - 10https://gerrit.wikimedia.org/r/822673 (https://phabricator.wikimedia.org/T303967) (owner: 10Jeena Huneidi) [19:08:23] (03CR) 10Ahmon Dancy: Scap backport --list: Add mergeable column (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/810078 (https://phabricator.wikimedia.org/T303967) (owner: 10Jeena Huneidi) [19:47:23] (03PS7) 10Jeena Huneidi: Scap backport --list: Add mergeable column [tools/scap] - 10https://gerrit.wikimedia.org/r/810078 (https://phabricator.wikimedia.org/T303967) [19:47:48] (03CR) 10Jeena Huneidi: Scap backport --list: Add mergeable column (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/810078 (https://phabricator.wikimedia.org/T303967) (owner: 10Jeena Huneidi) [21:25:51] (03PS1) 10Jeena Huneidi: scap backport: Don't allow WIP [tools/scap] - 10https://gerrit.wikimedia.org/r/822687 [23:37:36] (03PS2) 10BryanDavis: Drop Python 2.7 support [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/566406 (owner: 10Legoktm) [23:45:35] (03CR) 10BryanDavis: [C: 03+2] Drop Python 2.7 support [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/566406 (owner: 10Legoktm) [23:46:53] (03Merged) 10jenkins-bot: Drop Python 2.7 support [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/566406 (owner: 10Legoktm) [23:49:16] (03PS1) 10BryanDavis: gitignore: add .python-version [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/822695 [23:49:41] (03CR) 10BryanDavis: [C: 03+2] "trivial change" [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/822695 (owner: 10BryanDavis) [23:50:42] (03Merged) 10jenkins-bot: gitignore: add .python-version [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/822695 (owner: 10BryanDavis) [23:52:15] legoktm: do you want the honors of tagging and releasing commit-message-validator 1.0? (dropping py2 support should be a major version bump) [23:53:13] Oooh. [23:53:18] Yes, can do tonight! [23:53:27] Do we have anything else in the pipeline? [23:54:03] (Hello from the bench next to legoktm.)