[05:29:25] Can anyone refresh/update Deployment calendar? [07:08:21] Does anyone here have the magic spells to update the Deployments wiki page? [07:28:28] kart_: awight: good morning. We have a script which is typically run by Tyler [07:29:00] I will give it a try [07:34:07] oh the script fails cause https://phabricator.wikimedia.org/T305216 does not have a owner [07:34:11] it is not assigned [07:37:13] hashar: I can't find the source to these scripts, but if they're public I'm happy to (attempt to) debug. [07:40:17] https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/tools/release/+/refs/heads/master/make-deployment-calendar [07:40:31] ty! [07:41:35] it is largely broken :D [07:42:28] /o\ [07:44:20] the thing is that there is no train next week [07:44:30] and it causes some logic flaws in the script :-\ [07:44:36] :-D [07:44:49] I can do this manually, if it's helpful? [07:45:08] Personally, I just wanted to schedule some backports for today. [07:49:20] I also wanted to few config changes, but seems it will go late today once calendar is fixed. Need to readjust everything around it :) [07:49:59] well you can still do them this morning [07:50:10] without the Deployments page being updated. I am trying to fix the script [08:02:47] (03CR) 10Awight: "This change is ready for review." [tools/release] - 10https://gerrit.wikimedia.org/r/788285 (owner: 10Awight) [08:10:59] hashar: It seems to work for me locally, I don't know why. Mind if I paste the new content into the page? [08:11:19] Or maybe there's a subtle problem with the output, which I'm overlooking... [08:13:16] awight: sure [08:13:29] https://wikitech.wikimedia.org/wiki/User:Awight/sandbox/Deployments [08:13:30] cause it definitely does not work here complaining about a `NoTrain` schedule not existing [08:14:40] but how!? :d [08:15:13] oh `scripts/make-deployments-calendar` does work [08:15:27] I know... I'm more disturbed by something that inexplicably works than one that sometimes fails ;-) [08:15:31] but the `scripts/update-wiki` has a different code path cause it also generate the next week schedule which is broken due to lack of train [08:15:38] ^ exactly, I used the lower-level thing. [08:16:43] (03CR) 10Awight: "This change is ready for review." [tools/release] - 10https://gerrit.wikimedia.org/r/788286 (owner: 10Awight) [08:17:06] awight: +1 can you copy paste what you have generated to [[Deployments]] please ? ;) [08:17:24] done! [08:17:55] I am afraid this whole system has a few issues :≠ [08:18:15] Now that I know where the jewels are kept, I'll also update my availability in the template :-) [08:18:17] at least we have a calendar for this week *success* [08:19:02] kart_: ^ fyi [08:24:51] cool. Thanks hashar :) [08:27:35] (03CR) 10Awight: "This change is ready for review." [tools/release] - 10https://gerrit.wikimedia.org/r/788287 (owner: 10Awight) [08:36:58] 10Release-Engineering-Team, 10Gerrit (Gerrit 3.5): Upgrade to Gerrit 3.5 - https://phabricator.wikimedia.org/T307334 (10hashar) [08:48:32] (03CR) 10Jaime Nuche: [C: 03+2] Fix bad interaction between multiprocessing.Pool and Lock [tools/scap] - 10https://gerrit.wikimedia.org/r/787795 (https://phabricator.wikimedia.org/T307242) (owner: 10Ahmon Dancy) [08:49:26] (03CR) 10Jaime Nuche: [C: 03+2] "Nice fix. I had seen those messages and wondered how the lock could disappear and why that was considered "ok-ish"" [tools/scap] - 10https://gerrit.wikimedia.org/r/787795 (https://phabricator.wikimedia.org/T307242) (owner: 10Ahmon Dancy) [08:52:34] (03Merged) 10jenkins-bot: Fix bad interaction between multiprocessing.Pool and Lock [tools/scap] - 10https://gerrit.wikimedia.org/r/787795 (https://phabricator.wikimedia.org/T307242) (owner: 10Ahmon Dancy) [09:09:05] 10Phabricator, 10Patch-For-Review: Delete unused custom /src/other/CustomLoginHandler.php in Phab extensions - https://phabricator.wikimedia.org/T228518 (10Mainframe98) a:05Mainframe98→03None [09:32:46] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Upstream: Jenkins search results missing /ci/ URL component - https://phabricator.wikimedia.org/T294424 (10hashar) [09:33:26] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen): Store CI builds output outside Jenkins (e.g. static storage) - https://phabricator.wikimedia.org/T53447 (10hashar) [09:33:41] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10SRE: Upgrade ci ssh key to ecdsa - https://phabricator.wikimedia.org/T177826 (10hashar) [09:33:59] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team (Seen), 10Wikimedia-Logstash, and 2 others: Send Jenkins build log and results to ElasticSearch - https://phabricator.wikimedia.org/T78705 (10hashar) 05Open→03Declined Closing this one again since we have no plan to work on it. [09:35:01] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Priority Backlog 📥), 10Elasticsearch: Set up data storage to collect loosely structured data from CI - https://phabricator.wikimedia.org/T211904 (10hashar) 05Open→03Declined Closing this one again since we have no plan to work on it. [09:35:03] 10Release-Engineering-Team (Seen), 10Data³, 10Quality-and-Test-Engineering-Team (QTE): Release Engineering Data Collection and Retention (aka Data³) - https://phabricator.wikimedia.org/T216085 (10hashar) [09:35:05] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen): Store CI builds output outside Jenkins (e.g. static storage) - https://phabricator.wikimedia.org/T53447 (10hashar) [09:35:15] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen): Store CI builds output outside Jenkins (e.g. static storage) - https://phabricator.wikimedia.org/T53447 (10hashar) 05Stalled→03Declined Closing this one again since we have no plan to work on it. [09:35:17] 10Release-Engineering-Team (Seen), 10Data³, 10Quality-and-Test-Engineering-Team (QTE): Release Engineering Data Collection and Retention (aka Data³) - https://phabricator.wikimedia.org/T216085 (10hashar) [09:35:50] 10Continuous-Integration-Infrastructure, 10Jenkins: Increase TTL of failed builds - https://phabricator.wikimedia.org/T228158 (10hashar) 05Open→03Declined Closing this one again since we have no plan to work on it. There is plenty of disk space available on the Jenkins controller. [09:37:24] 10Continuous-Integration-Infrastructure, 10Jenkins: Run two Jenkins master - https://phabricator.wikimedia.org/T254919 (10hashar) 05Stalled→03Declined It could be implemented if we had a way to maintain several Jenkins (via a defined container image and or Jenkins as code). We could then have multiple Jenk... [09:38:56] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team: Upgrade Jenkins to - https://phabricator.wikimedia.org/T307339 (10hashar) [09:41:57] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team: Upgrade Jenkins to - https://phabricator.wikimedia.org/T307339 (10hashar) [09:42:39] (03CR) 10Hashar: [C: 03+1] Sign myself up for the EU afternoon window [tools/release] - 10https://gerrit.wikimedia.org/r/788287 (owner: 10Awight) [09:46:33] (03CR) 10Hashar: [C: 04-1] "The Phabricator API token should be in ~/.arcrc and can be overridden or set via CONDUIT_TOKEN." [tools/release] - 10https://gerrit.wikimedia.org/r/788286 (owner: 10Awight) [09:49:46] (03CR) 10Hashar: [C: 04-1] Define the "init" target described in the README (031 comment) [tools/release] - 10https://gerrit.wikimedia.org/r/788285 (owner: 10Awight) [09:50:32] (03CR) 10Hashar: [C: 03+2] inference-services: articlequality cleanup [integration/config] - 10https://gerrit.wikimedia.org/r/786934 (https://phabricator.wikimedia.org/T301766) (owner: 10AikoChou) [09:50:57] hashar: thanks --^ [09:52:25] (03Merged) 10jenkins-bot: inference-services: articlequality cleanup [integration/config] - 10https://gerrit.wikimedia.org/r/786934 (https://phabricator.wikimedia.org/T301766) (owner: 10AikoChou) [09:55:23] 10Gerrit: Jenkins doesn't have CSS-specific whitepsace rules regarding parentheses - https://phabricator.wikimedia.org/T307313 (10hashar) [10:06:04] !log Reloading Zuul for https://gerrit.wikimedia.org/r/786934 # T301766 [10:06:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:06:06] T301766: Support (or not) the ORES augmented feature output in liftwing - https://phabricator.wikimedia.org/T301766 [10:06:35] elukey: I have reloaded CI [10:06:44] (03CR) 10Hashar: [C: 03+2] "Changed deployed!" [integration/config] - 10https://gerrit.wikimedia.org/r/786934 (https://phabricator.wikimedia.org/T301766) (owner: 10AikoChou) [10:11:43] (03PS2) 10Awight: Remove deprecated make "init" target [tools/release] - 10https://gerrit.wikimedia.org/r/788285 [10:11:59] (03CR) 10Awight: Remove deprecated make "init" target (031 comment) [tools/release] - 10https://gerrit.wikimedia.org/r/788285 (owner: 10Awight) [10:16:59] (03PS2) 10Awight: Document how to install the Phabricator API token [tools/release] - 10https://gerrit.wikimedia.org/r/788286 [10:17:08] (03CR) 10Awight: "Thanks for the hints!" [tools/release] - 10https://gerrit.wikimedia.org/r/788286 (owner: 10Awight) [10:50:39] (03CR) 10Hashar: [C: 03+2] Remove deprecated make "init" target [tools/release] - 10https://gerrit.wikimedia.org/r/788285 (owner: 10Awight) [10:51:17] (03Merged) 10jenkins-bot: Remove deprecated make "init" target [tools/release] - 10https://gerrit.wikimedia.org/r/788285 (owner: 10Awight) [10:51:39] (03CR) 10Hashar: [C: 03+2] Document how to install the Phabricator API token [tools/release] - 10https://gerrit.wikimedia.org/r/788286 (owner: 10Awight) [10:51:43] awight: great :) [10:51:48] (03CR) 10jerkins-bot: [V: 04-1] Document how to install the Phabricator API token [tools/release] - 10https://gerrit.wikimedia.org/r/788286 (owner: 10Awight) [10:52:11] awight: though you broke the dependency chain and have send each of the changes individually [10:52:18] so the second ends up conflicting: https://gerrit.wikimedia.org/r/c/mediawiki/tools/release/+/788286 :D [10:52:39] I am off for lunch, might rebase it myself this afternoon [11:33:03] 10Release-Engineering-Team, 10SRE: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10elukey) p:05Triage→03High [11:35:06] hashar: o/ [11:35:12] are you around by any chance? [11:41:31] 10Release-Engineering-Team, 10SRE: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10elukey) As suggested in the chat, I have created `/var/lock/scap-global-lock` with `Please check https://phabricator.wikimedia.org/T307349` [11:49:39] 10Release-Engineering-Team, 10SRE: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10elukey) A backup is being placed under /srv/restore/srv/deployment on deploy1002 by Jaime. The last backup was taken today at 04:13 UTC. The time of the deleti... [11:50:39] 10Release-Engineering-Team, 10SRE, 10bacula: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10jcrespo) [12:05:42] 10Release-Engineering-Team, 10SRE, 10bacula: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10elukey) The last error that puppet highlights is: ` Error: Execution of '/usr/bin/scap deploy --init' returned 1: Error: /Stage[main]/Profile::Med... [12:07:18] elukey: back from lunch [12:07:44] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team: Upgrade Jenkins to the next LTS, 2.332.2 - https://phabricator.wikimedia.org/T307339 (10Jdforrester-WMF) [12:10:45] 10Release-Engineering-Team, 10SRE, 10bacula: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10jcrespo) Bacula recovery log for the record: {P27348} [12:13:25] (03PS3) 10Hashar: Document how to install the Phabricator API token [tools/release] - 10https://gerrit.wikimedia.org/r/788286 (owner: 10Awight) [12:13:47] (03CR) 10Hashar: "Rebased to fix conflict. Could use a second pair of eyes before a final CR+2 :)" [tools/release] - 10https://gerrit.wikimedia.org/r/788286 (owner: 10Awight) [12:14:04] hashar: hi :) if you see in #security I made a big mistake on deploy1002, if you have a moment for a brainstorm in there I'd be really grateful [12:17:05] 10Deployments, 10Release-Engineering-Team (Doing), 10SRE, 10bacula: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10hashar) p:05High→03Unbreak! This is 100% an unbreak now. [12:25:32] 10Deployments, 10Release-Engineering-Team (Doing), 10SRE, 10bacula: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10jcrespo) Restore finished ok: ` 02-May 12:22 backup1001.eqiad.wmnet-fd JobId 437308: Elapsed time=00:37:57, Transfer rate=2... [12:26:41] 10Deployments, 10Release-Engineering-Team (Doing), 10SRE, 10bacula: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10hashar) [12:36:12] 10Deployments, 10Release-Engineering-Team (Doing), 10SRE, 10bacula: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10hashar) I checked the HEAD of all git repos under `/srv/deployment` with: find /srv/deployment -name .git -print0 -ex... [12:44:48] 10Release-Engineering-Team (Doing), 10Scap: Puppet config for scap-over-scap - https://phabricator.wikimedia.org/T306991 (10jnuche) [12:51:52] 10Deployments, 10Release-Engineering-Team (Doing), 10SRE, 10bacula: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10hashar) After restore: ` $ colordiff -U0 --text deploy2002 deploy1002 --- deploy2002 2022-05-02 14:51:09.946189381 +0200 +... [12:52:39] 10Scap, 10SRE: Add new user identity to Keyholder - https://phabricator.wikimedia.org/T307351 (10jnuche) [12:58:12] 10Deployments, 10Release-Engineering-Team (Doing), 10SRE, 10bacula: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10Volans) >>! In T307349#7895662, @Stashbot wrote: > {nav icon=file, name=Mentioned in SAL (#wikimedia-operations), href=http... [13:18:18] 10Deployments, 10Release-Engineering-Team (Doing), 10SRE, 10bacula: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10hashar) Ran the git rev-parse HEAD again: ` $ diff --text -U0 deploy2002 deploy1002 --- deploy2002 2022-05-02 15:17:27.8861... [13:22:31] 10Deployments, 10Release-Engineering-Team (Doing), 10SRE, 10bacula: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10elukey) p:05Unbreak!→03Medium Everything got restored, and /var/lock/scap-global-lock has been removed, deployments can... [13:38:24] 10Beta-Cluster-Infrastructure, 10Traffic: Betacommons: 504, Connection Timed Out at 2022-05-02 13:35:16 GMT - https://phabricator.wikimedia.org/T307354 (10AlexisJazz) [13:38:44] 10Deployments, 10Release-Engineering-Team (Doing), 10SRE, 10bacula: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10hashar) From IRC the big diff at T307349#7895665 shows that deploy2002 has repositories with a more up to date git commit.... [13:38:50] Project beta-scap-sync-world build #49771: 04FAILURE in 3 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/49771/ [13:42:01] 10Release-Engineering-Team: Add the MWStake logo (and link) to the footer of MediaWiki software - https://phabricator.wikimedia.org/T307243 (10Aklapper) [13:42:13] 10Release-Engineering-Team: Add the MWStake logo (and link) to the footer of MediaWiki software - https://phabricator.wikimedia.org/T307243 (10Aklapper) 05Open→03Stalled [13:46:01] Yippee, build fixed! [13:46:01] Project beta-scap-sync-world build #49772: 09FIXED in 1 min 14 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/49772/ [13:46:46] 10Release-Engineering-Team: Add the MWStake logo (and link) to the footer of MediaWiki software - https://phabricator.wikimedia.org/T307243 (10Bawolff) I'm opposed to this. MW should not be used for what is essentially advertising. [13:51:43] 10Deployments, 10Release-Engineering-Team (Doing), 10Parsoid, 10SRE, 10bacula: Accidental removal of some files under /srv/deployment on deploy1002 - https://phabricator.wikimedia.org/T307349 (10jcrespo) Adding parsoid team here for awareness- please check the repo and all its submodules look as expected... [15:09:17] !log Updating beta cluster scap to 4.7.1-1+0~20220502085300.264~1.gbp367de7? [15:09:18] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:12:10] 10Beta-Cluster-Infrastructure, 10Patch-For-Review: Deployment-prep should host its own statsd/graphite server - https://phabricator.wikimedia.org/T241285 (10Lucas_Werkmeister_WMDE) >>! In T241285#7641040, @Majavah wrote: I haven't heard anyone complaining about the lack of statsd metrics from deployment-prep s... [15:47:37] 10Phabricator: Rename DutchTina to Metaverse on Phabricator - https://phabricator.wikimedia.org/T307267 (10RhinosF1) Note: On-Wiki rename disputed at https://meta.wikimedia.org/wiki/Stewards%27_noticeboard#Please_remove_User%3AMetaverse_from_Global_renamer_group_and_revert_%2391629_rename [15:57:50] 10MediaWiki-Releasing, 10MW-1.38-release: YAML parser not included in REL1_38 - https://phabricator.wikimedia.org/T307307 (10Aklapper) Assuming this is about MediaWiki Core (please add codebase project tags if possible - thanks!). [16:09:54] 10Continuous-Integration-Infrastructure, 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: CiviCRM CI jobs fails when migrating from Stretch to Bullseye - https://phabricator.wikimedia.org/T307178 (10AndyRussG) Thanks so much for all this, @hashar! So, looks like for now we have working CI with the rollba... [16:29:50] 10Continuous-Integration-Infrastructure, 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: CiviCRM CI jobs fails when migrating from Stretch to Bullseye - https://phabricator.wikimedia.org/T307178 (10hashar) That is correct! My ideas is to add a second job to CI which is running with Bullseye and have it... [16:29:58] 10Project-Admins: Archive Kaltura Player Phab project and decline all its open tasks - https://phabricator.wikimedia.org/T306971 (10Jdlrobson) 05Stalled→03Open [16:34:22] hashar (or others) can I shut off integration-castor04 for a few minutes? I want to migrate it to another host but it seems too busy to migrate live :) [16:34:30] * andrewbogott probably needs to learn where the CI folks live these days [16:35:40] 10Continuous-Integration-Infrastructure, 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: CiviCRM CI jobs fails when migrating from Stretch to Bullseye - https://phabricator.wikimedia.org/T307178 (10AndyRussG) @hashar thanks! Just to note, actually a previously passing change is now failing. Seems likely... [16:44:22] andrewbogott: We still live here. :-) [16:44:33] ok :) [16:45:09] I got out of a meeting w/ hashar about 10 minutes ago so he's around somewhere [16:45:28] 10Release-Engineering-Team: Add the MWStake logo (and link) to the footer of MediaWiki software - https://phabricator.wikimedia.org/T307243 (10Legoktm) There are plenty of organizations and individuals who work together to create MediaWiki (if you did a count of orgs' patches/reviews, would MWStake be #1? obviou... [17:09:12] 10Project-Admins: Archive Kaltura Player Phab project and decline all its open tasks - https://phabricator.wikimedia.org/T306971 (10Aklapper) [17:16:25] 10Release-Engineering-Team: Add the MWStake logo (and link) to the footer of MediaWiki software - https://phabricator.wikimedia.org/T307243 (10Aklapper) >>! In T307243#7896629: > There are plenty of organizations and individuals who work together to create MediaWiki **Only slightly related:** There is (incomple... [17:29:16] 10Release-Engineering-Team (Radar), 10Infrastructure-Foundations, 10SRE, 10SRE-Access-Requests, 10serviceops: Need a service account on deploy servers for automated train pre-sync operations - https://phabricator.wikimedia.org/T303857 (10dancy) @Joe Pinging on this ticket. Outstanding issues: * I defi... [18:03:44] 10Release-Engineering-Team (Radar), 10Performance-Team, 10Wikimedia-Rdbms, 10User-brennen: Remove database name from normalized DBQueryError message - https://phabricator.wikimedia.org/T307118 (10Krinkle) [18:08:20] (03PS1) 10Daniel Kinzler: Add nifty pre-commit hook [tools/code-utils] - 10https://gerrit.wikimedia.org/r/788384 [19:34:03] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.39.0-wmf.10 deployment blockers - https://phabricator.wikimedia.org/T305216 (10thcipriani) p:05Triage→03Medium [19:50:33] 10Project-Admins: Requests for addition to the #acl*Project-Admins group (in comments) - https://phabricator.wikimedia.org/T706 (10Aklapper) @xSavitar: Hi! [You should already have permissions to create workboards](https://phabricator.wikimedia.org/tag/trusted-contributors/) for existing project tags. Do you als... [20:04:46] 10Scap, 10SRE: Add new user identity to Keyholder for scap - https://phabricator.wikimedia.org/T307351 (10Peachey88) [20:07:35] (03CR) 10Thcipriani: [C: 03+2] Sign myself up for the EU afternoon window [tools/release] - 10https://gerrit.wikimedia.org/r/788287 (owner: 10Awight) [20:08:14] (03Merged) 10jenkins-bot: Sign myself up for the EU afternoon window [tools/release] - 10https://gerrit.wikimedia.org/r/788287 (owner: 10Awight) [20:25:51] 10Project-Admins: Requests for addition to the #acl*Project-Admins group (in comments) - https://phabricator.wikimedia.org/T706 (10xSavitar) >>! In T706#7897441, @Aklapper wrote: > @xSavitar: Hi! [You should already have permissions to create workboards](https://phabricator.wikimedia.org/tag/trusted-contributors... [20:37:22] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.39.0-wmf.10 deployment blockers - https://phabricator.wikimedia.org/T305216 (10Jdlrobson) [20:47:50] 10Release-Engineering-Team, 10Add-Link, 10Growth-Team, 10Patch-For-Review: mwaddlink CI fails with "No module named pip" - https://phabricator.wikimedia.org/T307209 (10kostajh) >>! In T307209#7891518, @kostajh wrote: >>>! In T307209#7891291, @hashar wrote: >> The lack of pip is T304860 >> >> You would wan... [21:02:10] (03PS3) 10Ahmon Dancy: Add test-scap-backport script to deploy container [tools/train-dev] - 10https://gerrit.wikimedia.org/r/788369 (https://phabricator.wikimedia.org/T287042) [21:36:52] (03PS4) 10Ahmon Dancy: Add test-scap-backport script to deploy container [tools/train-dev] - 10https://gerrit.wikimedia.org/r/788369 (https://phabricator.wikimedia.org/T287042) [22:07:25] (03PS5) 10Ahmon Dancy: Add test-scap-backport script to deploy container [tools/train-dev] - 10https://gerrit.wikimedia.org/r/788369 (https://phabricator.wikimedia.org/T287042) [22:33:14] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10Discovery-Search (Current work): Ensure Phabricator is compatible with Elasticsearch 7.10 - https://phabricator.wikimedia.org/T303445 (10EBernhardson) While it seems unlikely, in case something breaks today i deleted the phabricator indices fr... [23:28:58] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Zuul: Broken dashboard links on Zuul Status page - https://phabricator.wikimedia.org/T307405 (10Reedy)