[06:42:57] 10Project-Admins: Create project tag for User-EpicPupper - https://phabricator.wikimedia.org/T307832 (10Aklapper) 05Open→03Resolved a:03Aklapper Requested public project #User-EpicPupper has been created: https://phabricator.wikimedia.org/project/view/5932/ (In case you need to edit the project or project... [06:43:01] 10Phabricator (Upstream), 10Upstream: Per-user projects for personal work in progress tracking - https://phabricator.wikimedia.org/T555 (10Aklapper) [07:10:59] (03PS1) 10Hashar: [WMF] update Zuul plugin [software/gerrit] (wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/790261 (https://phabricator.wikimedia.org/T307621) [07:26:33] (03CR) 10Hashar: [C: 03+2] [WMF] update Zuul plugin [software/gerrit] (wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/790261 (https://phabricator.wikimedia.org/T307621) (owner: 10Hashar) [07:26:52] (03PS1) 10Hashar: Update Zuul plugin [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/790262 (https://phabricator.wikimedia.org/T307621) [07:28:24] (03CR) 10jerkins-bot: [V: 04-1] Update Zuul plugin [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/790262 (https://phabricator.wikimedia.org/T307621) (owner: 10Hashar) [07:28:53] (03CR) 10Hashar: "recheck" [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/790262 (https://phabricator.wikimedia.org/T307621) (owner: 10Hashar) [07:34:05] (03Merged) 10jenkins-bot: [WMF] update Zuul plugin [software/gerrit] (wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/790261 (https://phabricator.wikimedia.org/T307621) (owner: 10Hashar) [07:34:57] (03CR) 10Hashar: "recheck due to: rsync: link_stat "/git-fat/2e6a23935370b5571e1a8f92ae9e736f97514142" (in archiva) failed: No such file or directory (2)" [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/790262 (https://phabricator.wikimedia.org/T307621) (owner: 10Hashar) [07:37:56] (03CR) 10Hashar: [C: 03+2] Update Zuul plugin [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/790262 (https://phabricator.wikimedia.org/T307621) (owner: 10Hashar) [07:38:32] (03Merged) 10jenkins-bot: Update Zuul plugin [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/790262 (https://phabricator.wikimedia.org/T307621) (owner: 10Hashar) [07:56:52] 10Phabricator (Upstream), 10Upstream: Per-user projects for personal work in progress tracking - https://phabricator.wikimedia.org/T555 (10Aklapper) 05Open→03Resolved This task nowadays serves no purpose. Resolving. Please do not add subtasks; there is nothing to "track". Thanks. :) [08:02:10] 10Phabricator: sh: 1: svn: not found on phabricator - https://phabricator.wikimedia.org/T307889 (10SunAfterRain) [08:07:31] 10Phabricator: sh: 1: svn: not found on phabricator - https://phabricator.wikimedia.org/T307889 (10hashar) Subversion has been removed from the Phabricator hosts on May 5th by 56c6f3b70afc1547c5af9039b81c65f70f234b4c https://gerrit.wikimedia.org/r/c/operations/puppet/+/785149 [08:11:27] 10Phabricator, 10Release-Engineering-Team (Doing), 10Patch-For-Review: sh: 1: svn: not found on phabricator - https://phabricator.wikimedia.org/T307889 (10hashar) [08:50:08] 10Release-Engineering-Team (Doing), 10Gerrit (Gerrit 3.4): Zuul Depends-On footer processing missing from Gerrit UI after 3.4.4 upgrade - https://phabricator.wikimedia.org/T307621 (10hashar) I have restarted Gerrit on gerrit2001. For gerrit1001 the plugin scanner caught the update after deployment: ` [2022-05... [08:53:51] 10Release-Engineering-Team (Doing), 10Gerrit (Gerrit 3.4): Zuul Depends-On footer processing missing from Gerrit UI after 3.4.4 upgrade - https://phabricator.wikimedia.org/T307621 (10hashar) I have updated the plugin but it does not seem to work with Gerrit 3.4 (I should have tested it locally). Heading to a... [09:03:54] 10Release-Engineering-Team (Doing), 10Gerrit (Gerrit 3.4): Zuul Depends-On footer processing missing from Gerrit UI after 3.4.4 upgrade - https://phabricator.wikimedia.org/T307621 (10hashar) 05Open→03Resolved The JavaScript issue was a caching related issue, force reloading the web UI fixed it. The list of... [10:21:52] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Zuul: Delete .git/logs on zuul-merger git repo to prevent reflog from being created - https://phabricator.wikimedia.org/T307620 (10hashar) On the Zuul merger instances (contint1001.wikimedia.org and contint2001.wikimedia.org) I have delet... [10:24:18] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Zuul: Delete .git/logs on zuul-merger git repo to prevent reflog from being created - https://phabricator.wikimedia.org/T307620 (10hashar) `core.logallrefupdates` is set to `true` in the local repository config, I believe that is done by... [11:14:25] 10Continuous-Integration-Config, 10Quibble, 10Wikidata, 10wdwb-tech, 10ci-test-error (WMF-deployed Build Failure): jenkis CI wikibase-repo-docker failing with new quibble version - https://phabricator.wikimedia.org/T304147 (10kostajh) @hashar are there any other jobs to update? Is every Quibble job now u... [12:13:02] (03CR) 10Hashar: [C: 03+1] Document how to install the Phabricator API token [tools/release] - 10https://gerrit.wikimedia.org/r/788286 (owner: 10Awight) [12:15:27] (03CR) 10Hashar: [C: 03+2] "Jobs updated:" [integration/config] - 10https://gerrit.wikimedia.org/r/787492 (https://phabricator.wikimedia.org/T300340) (owner: 10Hashar) [12:17:41] (03Merged) 10jenkins-bot: jjb: update integration/quibble job to 1.4.5 image [integration/config] - 10https://gerrit.wikimedia.org/r/787492 (https://phabricator.wikimedia.org/T300340) (owner: 10Hashar) [12:17:51] (03CR) 10Hashar: "I have updated the two jobs and triggered builds:" [integration/config] - 10https://gerrit.wikimedia.org/r/787493 (https://phabricator.wikimedia.org/T300340) (owner: 10Hashar) [12:28:44] (03CR) 10Hashar: "https://integration.wikimedia.org/ci/job/quibble-daily-InterwikiSorting-vendor-mysql-php72-docker/259/ failed (UNSTABLE). Apparently cause" [integration/config] - 10https://gerrit.wikimedia.org/r/787493 (https://phabricator.wikimedia.org/T300340) (owner: 10Hashar) [12:30:33] (03PS1) 10Jforrester: Stop branching LocalisationUpdate for Wikimedia production [tools/release] - 10https://gerrit.wikimedia.org/r/790355 (https://phabricator.wikimedia.org/T158360) [12:30:35] (03PS1) 10Jforrester: [DNM] Stop branching LocalisationUpdate for the MediaWiki tarball [tools/release] - 10https://gerrit.wikimedia.org/r/790356 (https://phabricator.wikimedia.org/T300498) [12:38:24] 10Continuous-Integration-Config, 10Toolhub, 10Documentation, 10User-Slst2020: Publish docs on doc.wikimedia.org - https://phabricator.wikimedia.org/T305914 (10Slst2020) >>! In T305914#7910841, @dduvall wrote: > @Slst2020 I'm up for hacking on it together if you'd like to propose a day/time (just not next w... [12:44:57] (03CR) 10Hashar: [C: 03+2] "The webdriver.io failure comes from mediawiki/core tests. Both builds have five tests running. The good one has five XML files generated:" [integration/config] - 10https://gerrit.wikimedia.org/r/787493 (https://phabricator.wikimedia.org/T300340) (owner: 10Hashar) [12:47:40] (03Merged) 10jenkins-bot: jjb: update quibble-daily jobs to Quibble 1.4.5 image [integration/config] - 10https://gerrit.wikimedia.org/r/787493 (https://phabricator.wikimedia.org/T300340) (owner: 10Hashar) [12:49:26] 10GitLab (Infrastructure), 10serviceops: bring new gitlab hardware servers into production - https://phabricator.wikimedia.org/T307142 (10Jelto) [13:14:44] (03CR) 10Hashar: [C: 03+2] "I have filed https://phabricator.wikimedia.org/T307914 for the potential filename time based race condition" [integration/config] - 10https://gerrit.wikimedia.org/r/787493 (https://phabricator.wikimedia.org/T300340) (owner: 10Hashar) [13:15:04] (03PS2) 10Hashar: jjb: update fundraising jobs to Quibble 1.4.5 image [integration/config] - 10https://gerrit.wikimedia.org/r/787494 (https://phabricator.wikimedia.org/T300340) [13:25:44] hashar: Now time for https://gerrit.wikimedia.org/r/c/integration/config/+/777030 ? :-) [13:33:55] (03CR) 10Jforrester: [C: 03+2] make-release: Start branching SimilarEditors extension for production (031 comment) [tools/release] - 10https://gerrit.wikimedia.org/r/789248 (https://phabricator.wikimedia.org/T306909) (owner: 10Tchanders) [13:35:06] (03Merged) 10jenkins-bot: make-release: Start branching SimilarEditors extension for production [tools/release] - 10https://gerrit.wikimedia.org/r/789248 (https://phabricator.wikimedia.org/T306909) (owner: 10Tchanders) [13:37:30] Hi, not sure if this is the right channel but can we add a new entry for mobileapps/wikifeeds deployments? [13:38:03] We used to follow the services deployment window but lately its not very convenient for EU timezones [13:41:45] nemo-yiannis: hi, for https://wikitech.wikimedia.org/wiki/Deployments ? [13:41:52] yes [13:41:59] To add or remove your name from the list, or to schedule a new recurring deploy window, send a patchset to mediawiki/tools/release.git's deployments-calendar.yaml file. Add thcipriani as a reviewer. [13:42:00] :D [13:42:09] sounds good [13:42:13] https://gerrit.wikimedia.org/r/mediawiki/tools/release.git [13:42:15] thanks! [13:42:47] the yaml file has some oddities make-deployment-calendar/deployments-calendar.yaml [13:43:13] (03CR) 10Tchanders: make-release: Start branching SimilarEditors extension for production (031 comment) [tools/release] - 10https://gerrit.wikimedia.org/r/789248 (https://phabricator.wikimedia.org/T306909) (owner: 10Tchanders) [13:43:43] you need to first define a window under "windows" [13:44:30] then add that window under "schedule" which has several days or day ranges (ex: "tue,thus" for Tuesday, Wednesday, Thursday) [13:44:36] tyler reviews those changes [13:45:03] James_F: eventually, I haven't moved all jobs to quibble 1.4.5 + supervisord yet [13:45:20] hashar: Ack. :-( [13:45:34] (03CR) 10Hashar: [C: 03+2] "quibble-donationinterface-REL1_35-php73-docker works :)" [integration/config] - 10https://gerrit.wikimedia.org/r/787494 (https://phabricator.wikimedia.org/T300340) (owner: 10Hashar) [13:45:39] but it is progressing! [13:45:43] yay. [13:48:08] (03Merged) 10jenkins-bot: jjb: update fundraising jobs to Quibble 1.4.5 image [integration/config] - 10https://gerrit.wikimedia.org/r/787494 (https://phabricator.wikimedia.org/T300340) (owner: 10Hashar) [13:54:01] (03PS1) 10Hashar: jjb: update api-testing jobs to Quibble 1.4.5 image [integration/config] - 10https://gerrit.wikimedia.org/r/790370 (https://phabricator.wikimedia.org/T300340) [14:07:17] (03CR) 10Hashar: [C: 03+2] "Passed on https://gerrit.wikimedia.org/r/c/mediawiki/tools/api-testing/+/751431" [integration/config] - 10https://gerrit.wikimedia.org/r/790370 (https://phabricator.wikimedia.org/T300340) (owner: 10Hashar) [14:07:29] 10Project-Admins: Create project tag for User-EpicPupper - https://phabricator.wikimedia.org/T307832 (10EpicPupper) Thanks! [14:09:15] (03Merged) 10jenkins-bot: jjb: update api-testing jobs to Quibble 1.4.5 image [integration/config] - 10https://gerrit.wikimedia.org/r/790370 (https://phabricator.wikimedia.org/T300340) (owner: 10Hashar) [14:12:25] (03PS1) 10Hashar: jjb: update Phan jobs to Quibble 1.4.5 image [integration/config] - 10https://gerrit.wikimedia.org/r/790377 (https://phabricator.wikimedia.org/T300340) [14:19:09] (03PS1) 10Hashar: jjb: update codehealth patches to Quibble image 1.4.5 [integration/config] - 10https://gerrit.wikimedia.org/r/790380 (https://phabricator.wikimedia.org/T300340) [14:20:56] (03PS1) 10Hashar: jjb: update coverage jobs to Quibble image 1.4.5 [integration/config] - 10https://gerrit.wikimedia.org/r/790381 (https://phabricator.wikimedia.org/T300340) [15:24:46] 10Release-Engineering-Team (Doing), 10Gerrit (Gerrit 3.4): Zuul Depends-On footer processing missing from Gerrit UI after 3.4.4 upgrade - https://phabricator.wikimedia.org/T307621 (10dancy) Thanks @hashar! [15:28:10] 10Beta-Cluster-Infrastructure, 10CirrusSearch, 10MediaWiki-Core-JobQueue, 10Beta-Cluster-reproducible, 10Discovery-Search (Current work): Search on betacommons is not indexing anything - https://phabricator.wikimedia.org/T307862 (10MPhamWMF) [15:45:46] 10Beta-Cluster-Infrastructure, 10CirrusSearch, 10MediaWiki-Core-JobQueue, 10Beta-Cluster-reproducible, 10Discovery-Search (Current work): Search on betacommons is not indexing anything - https://phabricator.wikimedia.org/T307862 (10AlexisJazz) [15:54:30] (03PS1) 10Jgiannelos: Add deployment window for mobileapps/wikifeeds [tools/release] - 10https://gerrit.wikimedia.org/r/790396 [15:58:14] 10Beta-Cluster-Infrastructure, 10CirrusSearch, 10MediaWiki-Core-JobQueue, 10Beta-Cluster-reproducible, 10Discovery-Search (Current work): Search on betacommons is not indexing anything - https://phabricator.wikimedia.org/T307862 (10MPhamWMF) [16:06:44] 10GitLab (Integrations), 10GitLab-Test, 10Phabricator, 10Release-Engineering-Team (Next), and 2 others: Experiment with GitLab-Phabricator integration - https://phabricator.wikimedia.org/T265617 (10hashar) >>! In T265617#7880480, @kostajh wrote: >>>! In T265617#7413071, @mmodell wrote: >> So one thing I th... [16:06:51] 10GitLab, 10Release-Engineering-Team (GitLab-a-thon 🦊): Investigate buildkitd instances as image builders for GitLab - https://phabricator.wikimedia.org/T307810 (10dduvall) [16:18:18] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10SecTeam-Processed, and 2 others: Write a phame / tech blog post detailing the appsec pipeline, as it currently exists - https://phabricator.wikimedia.org/T307517 (10sbassett) [16:18:57] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10SecTeam-Processed, and 2 others: Write a phame / tech blog post detailing the appsec pipeline, as it currently exists - https://phabricator.wikimedia.org/T307517 (10sbassett) 05Open→03In progress p:05Triage→03Low [16:19:00] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10Security: Design and Build Application Security Pipeline Components for Gitlab - https://phabricator.wikimedia.org/T289290 (10sbassett) [16:22:45] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10SecTeam-Processed, 10Security: Investigate container scanning options within the context of the Gitlab appsec pipeline - https://phabricator.wikimedia.org/T307523 (10sbassett) [16:24:24] 10GitLab (Integrations), 10GitLab-Test, 10Phabricator, 10Release-Engineering-Team (Next), and 2 others: Experiment with GitLab-Phabricator integration - https://phabricator.wikimedia.org/T265617 (10hashar) The `commentlink` section I describe above is for rendering. The indexing in Gerrit is done by [[ htt... [16:29:43] 10GitLab (Project Migration), 10Release-Engineering-Team (GitLab-a-thon 🦊), 10User-dduvall: Create Blubber repo on GitLab, archive Gerrit repo - https://phabricator.wikimedia.org/T307533 (10hashar) To archive Gerrit repositories we use #projects-cleanup , on the side bar there is the //[[ https://phabricator... [18:09:35] 10GitLab (Project Migration), 10Release-Engineering-Team (GitLab-a-thon 🦊), 10User-brennen, 10User-dduvall: Write a GitLab "Migrating a Project" runbook / manual based on Blubber migration - https://phabricator.wikimedia.org/T307538 (10brennen) [18:18:50] 10GitLab (Integrations), 10GitLab-Test, 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), and 2 others: Experiment with GitLab-Phabricator integration - https://phabricator.wikimedia.org/T265617 (10thcipriani) #releng was poised to look at this this quarter (q4 2022), but the person we intend... [18:29:05] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (GitLab-a-thon 🦊): Investigate alternatives to docker-in-docker for container image creation in GitLab - https://phabricator.wikimedia.org/T307599 (10brennen) [18:30:48] 10GitLab (Project Migration), 10Release-Engineering-Team (GitLab-a-thon 🦊), 10User-dduvall: Migrate Blubber project to GitLab - https://phabricator.wikimedia.org/T301168 (10brennen) [18:40:47] 10Beta-Cluster-Infrastructure, 10CirrusSearch, 10MediaWiki-Core-JobQueue, 10Beta-Cluster-reproducible, 10Discovery-Search (Current work): Search on betacommons is not indexing anything - https://phabricator.wikimedia.org/T307862 (10EBernhardson) CirrusSearch logs in beta cluster are fairly quiet. Since j... [18:51:33] 10Scap, 10SRE, 10SRE-Access-Requests: Add new user identity to Keyholder for scap - https://phabricator.wikimedia.org/T307351 (10thcipriani) Tagging this with #sre-access-requests since (a) it's **sorta** an access request and (b) I'm not sure about the normal process for adding new identities for keyholder... [19:02:45] 10GitLab (Project Migration), 10Release-Engineering-Team (🌱 Spring Cleaning — April 2022), 10User-brennen, 10cloud-services-team (Kanban): GitLab: separate trees for WMCS infrastructure and tenants - https://phabricator.wikimedia.org/T305301 (10brennen) 05In progress→03Resolved [19:12:02] 10GitLab, 10Release-Engineering-Team (GitLab-a-thon 🦊): Investigate buildkitd instances as image builders for GitLab - https://phabricator.wikimedia.org/T307810 (10dduvall) I've set up a personal repo with the manifests, etc. used to set up my evaluation environment. They were applied against a fresh Digital O... [19:26:35] 10GitLab (Project Migration), 10Release-Engineering-Team (GitLab-a-thon 🦊), 10User-brennen, 10User-dduvall: Write a GitLab "Migrating a Project" runbook / manual based on Blubber migration - https://phabricator.wikimedia.org/T307538 (10sbassett) Hey @brennen - This seems like as good a task as any to inq... [19:37:35] 10GitLab (Project Migration), 10Release-Engineering-Team (GitLab-a-thon 🦊), 10User-brennen, 10User-dduvall: Write a GitLab "Migrating a Project" runbook / manual based on Blubber migration - https://phabricator.wikimedia.org/T307538 (10brennen) > This seems like as good a task as any to inquire as to wheth... [19:39:19] 10Beta-Cluster-Infrastructure, 10CirrusSearch, 10MediaWiki-Core-JobQueue, 10Beta-Cluster-reproducible, and 2 others: Search on betacommons is not indexing anything - https://phabricator.wikimedia.org/T307862 (10EBernhardson) a:03EBernhardson [19:43:27] 10GitLab (Project Migration), 10Release-Engineering-Team (GitLab-a-thon 🦊), 10User-brennen, 10User-dduvall: Write a GitLab "Migrating a Project" runbook / manual based on Blubber migration - https://phabricator.wikimedia.org/T307538 (10sbassett) Ok, sounds fair. Thanks for the response. [20:15:21] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10Security, 10user-sbassett: Re-implement semgrep ci includes - https://phabricator.wikimedia.org/T307962 (10sbassett) [20:15:49] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10SecTeam-Processed, and 2 others: Re-implement semgrep ci includes - https://phabricator.wikimedia.org/T307962 (10sbassett) [20:15:57] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10Security: Design and Build Application Security Pipeline Components for Gitlab - https://phabricator.wikimedia.org/T289290 (10sbassett) [20:16:11] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10SecTeam-Processed, and 2 others: Re-implement semgrep ci includes - https://phabricator.wikimedia.org/T307962 (10sbassett) 05Open→03In progress p:05Triage→03Low [20:16:53] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10SecTeam-Processed, and 2 others: Re-implement semgrep ci includes - https://phabricator.wikimedia.org/T307962 (10sbassett) [20:17:58] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10SecTeam-Processed, and 2 others: Re-implement semgrep ci includes - https://phabricator.wikimedia.org/T307962 (10sbassett) [20:22:36] 10Release-Engineering-Team (Radar), 10Infrastructure-Foundations, 10SRE, 10SRE-Access-Requests, 10serviceops: Need a service account on deploy servers for automated train pre-sync operations - https://phabricator.wikimedia.org/T303857 (10thcipriani) >>! In T303857#7901981, @Joe wrote: > To give some cont... [20:25:33] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: Config patches are stuck in the postmerge queue - https://phabricator.wikimedia.org/T307963 (10Tgr) [20:30:25] hmm [20:32:59] !log Manually cancelling deadlock build jobs for beta https://integration.wikimedia.org/ci/view/Beta/ # T307963 [20:33:01] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:33:01] T307963: Config patches are stuck in the postmerge queue - https://phabricator.wikimedia.org/T307963 [20:33:25] thx hashar. [20:33:33] dancy: it is a deadlock of doom between the Jenkins scheduler and the Gearman plugin :-\ [20:33:43] and some job using a lock between themselves [20:33:53] somewhere somehow there is a race condition or a deadlock [20:34:18] I end up convinced that all those beta cluster jobs should be moved to a standalone jenkins [20:34:49] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: Config patches are stuck in the postmerge queue - https://phabricator.wikimedia.org/T307963 (10hashar) 05Open→03Resolved a:03hashar Thank you very much for the task @tgr [20:39:21] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: Config patches are stuck in the postmerge queue - https://phabricator.wikimedia.org/T307963 (10Zabe) This is btw nothing new and has been an issue for some time now, so I guess there might be a deeper problem. [20:41:42] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: Config patches are stuck in the postmerge queue - https://phabricator.wikimedia.org/T307963 (10hashar) yes, that is the infamous {T72597} :-( [20:43:57] looks like the beta jobs are progressing agan [20:43:59] again [20:44:08] I am off > 🛏️ [20:45:06] Sweet dreams [21:38:07] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Radar), 10Discovery, 10Discovery-Search (Current work), 10Patch-For-Review: Deploy new bullseye elastic cluster nodes on deployment-prep - https://phabricator.wikimedia.org/T299797 (10Jdforrester-WMF) [21:39:50] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Radar), 10Discovery, 10Discovery-Search (Current work), 10Patch-For-Review: Deploy new bullseye elastic cluster nodes on deployment-prep - https://phabricator.wikimedia.org/T299797 (10Jdforrester-WMF) FWICT everything seems to be working in Beta... [21:43:05] !log Beta Cluster: Shutting down old deployment-restbase03 instance for T295375 [21:43:07] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:43:07] T295375: Restbase migration to Buster - https://phabricator.wikimedia.org/T295375 [21:44:37] 10Beta-Cluster-Infrastructure, 10Platform Engineering, 10RESTBase: Upgrade deployment-restbase03 host to Buster - https://phabricator.wikimedia.org/T306052 (10Jdforrester-WMF) [22:01:21] 10Beta-Cluster-Infrastructure, 10Platform Engineering, 10RESTBase, 10Patch-For-Review: Upgrade deployment-restbase03 host to Buster - https://phabricator.wikimedia.org/T306052 (10Jdforrester-WMF) Once the above patches are merged, given a dozen manual puppet changes for Beta Cluster I've just made, this sh... [22:19:03] 10Phabricator, 10Release-Engineering-Team (Doing), 10Patch-For-Review: sh: 1: svn: not found on phabricator - https://phabricator.wikimedia.org/T307889 (10Dzahn) I merged the revert and subversion has been installed again on phab1001. Notice: /Stage[main]/Phabricator/Package[subversion]/ensure: created I... [22:20:42] 10Phabricator, 10Release-Engineering-Team (Doing), 10Patch-For-Review: sh: 1: svn: not found on phabricator - https://phabricator.wikimedia.org/T307889 (10Dzahn) 05Open→03Resolved a:03Dzahn [23:08:30] 10Beta-Cluster-Infrastructure, 10CirrusSearch, 10MediaWiki-Core-JobQueue, 10Beta-Cluster-reproducible, and 2 others: Search on betacommons is not indexing anything - https://phabricator.wikimedia.org/T307862 (10AlexisJazz) Some relation with T302699 ? @Majavah @Zabe @dom_walden ?