[08:14:32] (03CR) 10Kosta Harlan: "thank you James for the CR and jjb patch!" [integration/config] - 10https://gerrit.wikimedia.org/r/771470 (https://phabricator.wikimedia.org/T303958) (owner: 10Jforrester) [09:27:20] (03PS3) 10Hashar: dockerfiles: set castor hostname from CASTOR_HOST env [integration/config] - 10https://gerrit.wikimedia.org/r/770921 (https://phabricator.wikimedia.org/T216244) [10:13:02] 10MediaWiki-Releasing, 10MW-1.38-notes, 10MW-1.38-release, 10MW-1.39-notes (1.39.0-wmf.1; 2022-03-21-early): Branch REL1_38 for MediaWiki and all extensions and skins - https://phabricator.wikimedia.org/T302909 (10hashar) We could use an update of https://www.mediawiki.org/wiki/Release_notes and https://ww... [10:51:04] dcausse: hi, jnuche (from releng) has a question for you about open search :] [10:51:18] o/ [10:51:57] "how to get a list of unique values for a field on an OpenSearch dashboard?" [10:52:12] dcausse: hi! I was wondering if it's possible to get a list of unique values for a field from the OpenSearch web UI [10:52:15] but I am lunching ;) [10:52:27] like hashar said ^^^^ [10:52:35] I merely copy pasted :D [10:52:40] I am out for lunch [10:52:44] enjoy! [10:53:34] jnuche: should be if the field is properly indexed as a "keyword", it then becomes a classic aggregation [10:53:50] jnuche: is it on our own logstash setup? [10:55:09] dcausse: no idea, I'm kind of new here :) I'm looking at https://logstash.wikimedia.org/app/dashboards#/view/f7e31de0-9f0d-11eb-863c-3588009e4dd9?_g=(filters%3A!()%2CrefreshInterval%3A(pause%3A!t%2Cvalue%3A0)%2Ctime%3A(from%3Anow-12h%2Cto%3Anow)) [10:55:29] and would like to get all different unique values for `host.name` [10:57:12] jnuche: if it's shown as a top (aggregation) it's possible to extract all of them if the cardinality is not crazy high [10:57:39] from opensearch dashboard I'm not sure how tho [10:59:22] no worries, what would be the way to go about it? can I hit an API endpoint with `curl` calls for example? can you point me to any docs you think could help? [11:01:05] dcausse: didn't ping, silly me :) [11:01:24] jnuche: currently trying to do that in opensearch [11:01:47] dcausse: awesome, thanks! [11:02:26] jnuche: so click the edit button top right [11:02:48] done [11:03:16] then the small gear button top-right of the "Top ECS host" widget and then Edit Vizualization [11:03:39] yep [11:04:03] open "Buckets" and set Size to e.g. 10000 [11:05:40] nice, I can then export the list [11:05:49] Hit "Save and Return" then you might be able to export the top 10000 (which I hope will get you the exhaustive list) [11:05:51] dcausse: thank you very much again [11:05:51] yes [11:06:03] jnuche: yw! [11:06:50] beware that this is the list of hosts having an event in the timeframe selected [11:07:25] dcausse: yep ๐Ÿ‘ [12:32:44] (03PS1) 10Jaime Nuche: Release 4.5.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/771593 [12:57:43] 10Release-Engineering-Team (Radar), 10CommRel-Specialists-Support, 10DBA, 10User-notice: Stop announcing and scheduling primary database switchovers - https://phabricator.wikimedia.org/T303605 (10Trizek-WMF) I assume that the first iteration of this new process will be for {T302283}? [13:08:12] 10Release-Engineering-Team (Radar), 10CommRel-Specialists-Support, 10DBA, 10User-notice: Stop announcing and scheduling primary database switchovers - https://phabricator.wikimedia.org/T303605 (10Marostegui) >>! In T303605#7785370, @Trizek-WMF wrote: > I assume that the first iteration of this new process... [13:09:50] 10MediaWiki-Releasing, 10MW-1.38-notes, 10MW-1.38-release, 10MW-1.39-notes (1.39.0-wmf.1; 2022-03-21-early): Branch REL1_38 for MediaWiki and all extensions and skins - https://phabricator.wikimedia.org/T302909 (10Jdforrester-WMF) >>! In T302909#7785013, @hashar wrote: > We could use an update of https://w... [13:18:48] 10MediaWiki-Releasing, 10MW-1.38-notes, 10MW-1.38-release, 10MW-1.39-notes (1.39.0-wmf.1; 2022-03-21-early): Branch REL1_38 for MediaWiki and all extensions and skins - https://phabricator.wikimedia.org/T302909 (10hashar) Thanks for the [[ https://www.mediawiki.org/wiki/Release_checklist | mw:Release check... [13:21:05] 10Release-Engineering-Team, 10Scap: Add lock to scap prep - https://phabricator.wikimedia.org/T302905 (10jnuche) [13:21:33] 10Release-Engineering-Team, 10Scap: Add lock to scap prep - https://phabricator.wikimedia.org/T302905 (10jnuche) [13:23:27] (03CR) 10Hashar: [C: 03+2] "Well lets do that so :]" [integration/config] - 10https://gerrit.wikimedia.org/r/756042 (owner: 10BryanDavis) [13:25:11] (03CR) 10Hashar: [C: 03+2] "Updating all jobs" [integration/config] - 10https://gerrit.wikimedia.org/r/770460 (owner: 10Hashar) [13:25:14] (03Merged) 10jenkins-bot: zuul: [wikimedia/toolhub] Add commit-message-validator to tests [integration/config] - 10https://gerrit.wikimedia.org/r/756042 (owner: 10BryanDavis) [13:27:02] (03Merged) 10jenkins-bot: jjb: update castor image [integration/config] - 10https://gerrit.wikimedia.org/r/770460 (owner: 10Hashar) [14:03:19] (03PS4) 10Hashar: dockerfiles: set castor hostname from CASTOR_HOST env [integration/config] - 10https://gerrit.wikimedia.org/r/770921 (https://phabricator.wikimedia.org/T216244) [14:03:32] (03CR) 10Hashar: [C: 03+2] dockerfiles: set castor hostname from CASTOR_HOST env [integration/config] - 10https://gerrit.wikimedia.org/r/770921 (https://phabricator.wikimedia.org/T216244) (owner: 10Hashar) [14:06:03] (03Merged) 10jenkins-bot: dockerfiles: set castor hostname from CASTOR_HOST env [integration/config] - 10https://gerrit.wikimedia.org/r/770921 (https://phabricator.wikimedia.org/T216244) (owner: 10Hashar) [14:07:03] !log Building Docker image to support `CASTOR_HOST` | https://gerrit.wikimedia.org/r/770921 | T216244 [14:07:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:07:06] T216244: Don't hardcode castor url in castor docker container - https://phabricator.wikimedia.org/T216244 [14:10:41] (03PS1) 10Hashar: jjb: update castor image to support CASTOR_HOST [integration/config] - 10https://gerrit.wikimedia.org/r/771608 (https://phabricator.wikimedia.org/T216244) [14:10:59] (03CR) 10Hashar: [C: 03+2] jjb: update castor image to support CASTOR_HOST [integration/config] - 10https://gerrit.wikimedia.org/r/771608 (https://phabricator.wikimedia.org/T216244) (owner: 10Hashar) [14:11:47] !log Update all jobs to support `CASTOR_HOST` env variable | https://gerrit.wikimedia.org/r/770921 | T216244 | T252071 [14:11:50] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:11:50] T252071: Move all Wikimedia CI (WMCS integration project) instances from stretch to buster/bullseye - https://phabricator.wikimedia.org/T252071 [14:12:44] (03Merged) 10jenkins-bot: jjb: update castor image to support CASTOR_HOST [integration/config] - 10https://gerrit.wikimedia.org/r/771608 (https://phabricator.wikimedia.org/T216244) (owner: 10Hashar) [14:19:35] 10Release-Engineering-Team (Doing), 10Scap: Add lock to scap prep - https://phabricator.wikimedia.org/T302905 (10jnuche) 05Openโ†’03In progress a:03jnuche [14:26:43] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week), 10Performance-Team (Radar): Talk to Performance team and ServiceOps about caching for train experiment week - https://phabricator.wikimedia.org/T303758 (10cscott) ParserCache (for one) is *not* split by version. This is actually sometimes an issue, since if... [14:27:55] 10Continuous-Integration-Infrastructure, 10Cloud-VPS (Quota-requests), 10castor: Request new flavor for integration Cloud VPS project - https://phabricator.wikimedia.org/T304080 (10hashar) [14:28:36] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Patch-For-Review: Move all Wikimedia CI (WMCS integration project) instances from stretch to buster/bullseye - https://phabricator.wikimedia.org/T252071 (10hashar) [14:29:08] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Patch-For-Review: Move all Wikimedia CI (WMCS integration project) instances from stretch to buster/bullseye - https://phabricator.wikimedia.org/T252071 (10hashar) a:03hashar I have asked a new OpenStack flavor to create the new... [14:29:31] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Cloud-VPS (Quota-requests), 10castor: Request new flavor for integration Cloud VPS project - https://phabricator.wikimedia.org/T304080 (10hashar) [14:32:39] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week), 10Patch-For-Review, 10castor: Don't hardcode castor url in castor docker container - https://phabricator.wikimedia.org/T216244 (10hashar) 05Openโ†’03Resolved The Docker image now supports overriding the Castor i... [14:32:41] 10Continuous-Integration-Infrastructure, 10SRE: jenkins / zuul backing up due to jenkins slaves down - https://phabricator.wikimedia.org/T216039 (10hashar) [14:33:03] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): Determine if we need to communicate anything special about forward and backwards compatibility during train experiment week - https://phabricator.wikimedia.org/T303759 (10cscott) From the perspective of the content transform team: 1. Independent of the /pace/... [14:40:14] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week), 10Scap: Ensure we can leave a single fallback version of MediaWiki in place during train experiment week - https://phabricator.wikimedia.org/T303747 (10cscott) Somewhat related to {T298046} -- if we're going to have very-frequent trains, the content transfor... [14:46:42] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week), 10Scap: Ensure we can leave a single fallback version of MediaWiki in place during train experiment week - https://phabricator.wikimedia.org/T303747 (10dancy) @cscott When you say the trees should be kept around, do you mean the wmf/ branches needs... [15:42:19] (03CR) 10Jforrester: "Aha, I was wondering why my changes weren't syncing." [integration/config] - 10https://gerrit.wikimedia.org/r/767864 (https://phabricator.wikimedia.org/T302905) (owner: 10Ahmon Dancy) [15:43:17] (03CR) 10Ahmon Dancy: Don't trigger beta-mediawiki-config-update-eqiad (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/767864 (https://phabricator.wikimedia.org/T302905) (owner: 10Ahmon Dancy) [15:53:32] (03CR) 10Dduvall: Fix all concurrent access to history log file (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/771465 (https://phabricator.wikimedia.org/T301417) (owner: 10Dduvall) [16:22:27] (03PS1) 10Dduvall: Acquire history log lock in shared mode for read-only access [tools/scap] - 10https://gerrit.wikimedia.org/r/771634 (https://phabricator.wikimedia.org/T301417) [16:23:15] (03PS2) 10Dduvall: Acquire history log lock in shared mode for read-only access [tools/scap] - 10https://gerrit.wikimedia.org/r/771634 (https://phabricator.wikimedia.org/T301417) [16:24:28] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): Ensure train-blockers.toolforge.org will find correct train blocker during train experiment week - https://phabricator.wikimedia.org/T303751 (10brennen) 05In progressโ†’03Resolved [16:24:33] (03PS3) 10Hashar: jjb: Bump Quibble jobs to use memcached [integration/config] - 10https://gerrit.wikimedia.org/r/770468 (https://phabricator.wikimedia.org/T300340) (owner: 10Kosta Harlan) [16:25:01] !log Switching Quibble jobs to use memcached rather than APCu | https://gerrit.wikimedia.org/r/c/integration/config/+/770468 | T300340 [16:25:03] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:25:04] T300340: Use Memcached with Quibble - https://phabricator.wikimedia.org/T300340 [16:25:10] kostajh: forgot to deploy the memcached change [16:26:53] ah no we can't [16:26:59] cause we lack the quibble config :-\ [16:28:05] (03CR) 10Ahmon Dancy: [C: 03+2] "Tested!" [tools/scap] - 10https://gerrit.wikimedia.org/r/771634 (https://phabricator.wikimedia.org/T301417) (owner: 10Dduvall) [16:28:42] (03CR) 10Hashar: [C: 04-1] jjb: Bump Quibble jobs to use memcached (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/770468 (https://phabricator.wikimedia.org/T300340) (owner: 10Kosta Harlan) [16:29:09] (03Merged) 10jenkins-bot: Acquire history log lock in shared mode for read-only access [tools/scap] - 10https://gerrit.wikimedia.org/r/771634 (https://phabricator.wikimedia.org/T301417) (owner: 10Dduvall) [16:34:07] hashar: right, new release needed [16:34:16] working on it ;) [16:36:28] kostajh: https://gerrit.wikimedia.org/r/771636 and https://gerrit.wikimedia.org/r/771637 :] [16:36:39] once they merge I will take care of the git tag and rebuild the images [16:36:50] will do that this evening [16:38:12] 10Phabricator, 10Release-Engineering-Team (Priority Backlog ๐Ÿ“ฅ), 10Discovery-Search (Current work): Ensure Phabricator is compatible with Elasticsearch 7.10 - https://phabricator.wikimedia.org/T303445 (10Aklapper) Past tasks: {T181393}, {T230787}, {T175234}. [16:42:31] (03PS1) 10Hashar: dockerfiles: update to Quibble 1.4.4 [integration/config] - 10https://gerrit.wikimedia.org/r/771639 (https://phabricator.wikimedia.org/T300340) [16:42:37] (03PS2) 10Jaime Nuche: Release 4.5.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/771593 [16:42:44] kostajh: thanks :) [16:43:28] (03CR) 10Hashar: [C: 04-1] "Pending merge of https://gerrit.wikimedia.org/r/771636 and Quibble 1.4.4 tag." [integration/config] - 10https://gerrit.wikimedia.org/r/771639 (https://phabricator.wikimedia.org/T300340) (owner: 10Hashar) [16:43:54] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): What things look at the task for the specific train verison? - https://phabricator.wikimedia.org/T303978 (10dancy) 05Openโ†’03Resolved Thanks @brennen ! I think the `deploy-promote` case is the only one that matters and we expect it to work as desired. [16:44:04] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): What things look at the task for the specific train verison? - https://phabricator.wikimedia.org/T303978 (10dancy) [16:45:36] (03CR) 10Ahmon Dancy: [C: 03+2] Release 4.5.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/771593 (owner: 10Jaime Nuche) [16:45:59] 10Project-Admins: Request for project MediaWiki-extensions-SemanticQueryInterface - https://phabricator.wikimedia.org/T138603 (10Aklapper) @Vedmaka: Could you answer the last question please? Projects shouldn't have two issue trackers, as that brings duplication and fragmentation of issues, plus make planning ha... [16:46:23] (03Merged) 10jenkins-bot: Release 4.5.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/771593 (owner: 10Jaime Nuche) [16:48:25] (03PS1) 10Hashar: jjb: switch jobs to Quibble 1.4.4 [integration/config] - 10https://gerrit.wikimedia.org/r/771640 (https://phabricator.wikimedia.org/T300340) [16:49:13] will do those later tonight [16:49:32] kostajh: I am off will do the 1.4.4 git tag and docker images update to night [16:49:34] tonight [17:08:29] (03PS1) 10Ahmon Dancy: Use USER 33 instead of USER www-data [tools/release] - 10https://gerrit.wikimedia.org/r/771643 [17:08:30] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): Update deployment calendar/roadmap to reflect upcoming train experiment week - https://phabricator.wikimedia.org/T303365 (10thcipriani) 05Openโ†’03Resolved [17:10:05] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): Schedule Google meets for deploy windows during train experiment week - https://phabricator.wikimedia.org/T303366 (10thcipriani) 05Openโ†’03Resolved I've got meetings! Thanks @TAdeleye_WMF <3 [17:10:52] (03CR) 10Giuseppe Lavagetto: [C: 03+1] Use USER 33 instead of USER www-data [tools/release] - 10https://gerrit.wikimedia.org/r/771643 (owner: 10Ahmon Dancy) [17:11:02] (03CR) 10Ahmon Dancy: [C: 03+2] Use USER 33 instead of USER www-data [tools/release] - 10https://gerrit.wikimedia.org/r/771643 (owner: 10Ahmon Dancy) [17:11:15] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): Ensure coordination is set up with SRE for trainsperiment week - https://phabricator.wikimedia.org/T303742 (10thcipriani) I'd like to call this either a dupe of {T303758} or {T303366} [17:11:41] (03Merged) 10jenkins-bot: Use USER 33 instead of USER www-data [tools/release] - 10https://gerrit.wikimedia.org/r/771643 (owner: 10Ahmon Dancy) [17:11:49] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): Ensure coordination is set up with SRE for trainsperiment week - https://phabricator.wikimedia.org/T303742 (10thcipriani) [17:12:05] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week), 10Performance-Team (Radar): Talk to Performance team and ServiceOps about caching for train experiment week - https://phabricator.wikimedia.org/T303758 (10thcipriani) [17:13:45] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week), 10Performance-Team (Radar): Talk to Performance team and ServiceOps about caching for train experiment week - https://phabricator.wikimedia.org/T303758 (10thcipriani) [17:38:40] (03CR) 10Jforrester: Don't trigger beta-mediawiki-config-update-eqiad (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/767864 (https://phabricator.wikimedia.org/T302905) (owner: 10Ahmon Dancy) [17:50:41] maintenance-disconnect-full-disks build 369297 integration-agent-docker-1038 (/: 29%, /srv: 95%, /var/lib/docker: 60%): OFFLINE due to disk space [17:55:53] maintenance-disconnect-full-disks build 369298 integration-agent-docker-1038 (/: 29%, /srv: 44%, /var/lib/docker: 60%): RECOVERY disk space OK [18:07:50] Hi jeena could we backport https://gerrit.wikimedia.org/r/c/mediawiki/skins/Vector/+/771395 now? [18:08:05] I can't make the backport window to do it then and I'd like to get this resolved [18:12:55] hi Jdlrobson [18:13:11] yeah I actually planned to do it right now :) [18:13:32] jeena: sweet! I can test [18:13:51] cool [18:21:00] 10Release-Engineering-Team (Seen), 10Scap: Avoid rsyncing 2GB of L10N each day during train experiment week, if possible - https://phabricator.wikimedia.org/T303749 (10dduvall) I poked at this a bit more this morning and wanted to add some more detail. When `updateLocalisationCache.php` runs, it calls `Locali... [18:25:12] 10Release-Engineering-Team (Seen), 10Scap: Avoid rsyncing 2GB of L10N each day during train experiment week, if possible - https://phabricator.wikimedia.org/T303749 (10dduvall) There's one more dependency to note: - A [[ https://doc.wikimedia.org/mediawiki-core/master/php/classConstantDependency.html | Const... [18:29:22] 10Release-Engineering-Team (Seen), 10Scap: Avoid rsyncing 2GB of L10N each day during train experiment week, if possible - https://phabricator.wikimedia.org/T303749 (10dduvall) Here's a `var_export()` version of the `deps` key from the l10n cache. Note that the classes are not resolved since I acquired it dire... [18:38:58] Jdlrobson: pinged you on -operations but it's ready to test now [19:06:07] !log Tag Quibble 1.4.4 @ 56b2c9ba52c # T300340 [19:06:09] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:06:09] T300340: Use Memcached with Quibble - https://phabricator.wikimedia.org/T300340 [19:07:17] (03CR) 10Hashar: [C: 03+2] dockerfiles: update to Quibble 1.4.4 [integration/config] - 10https://gerrit.wikimedia.org/r/771639 (https://phabricator.wikimedia.org/T300340) (owner: 10Hashar) [19:09:28] (03Merged) 10jenkins-bot: dockerfiles: update to Quibble 1.4.4 [integration/config] - 10https://gerrit.wikimedia.org/r/771639 (https://phabricator.wikimedia.org/T300340) (owner: 10Hashar) [19:11:56] !log Building Docker images for Quibble 1.4.4 [19:11:57] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:15:53] Project mediawiki-core-doxygen-docker build #32683: 04FAILURE in 17 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/32683/ [19:33:01] Yippee, build fixed! [19:33:01] Project mediawiki-core-doxygen-docker build #32684: 09FIXED in 16 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/32684/ [20:22:07] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.26 deployment blockers - https://phabricator.wikimedia.org/T300202 (10Jdlrobson) [20:27:09] I will do the Quibble 1.4.4 switch tomorrow [20:27:17] too late for me to deal with the aftermath this evening ;) [20:35:25] 10Release-Engineering-Team (Seen), 10Scap: Avoid rsyncing 2GB of L10N each day during train experiment week, if possible - https://phabricator.wikimedia.org/T303749 (10dancy) Thanks for the excellent writeup @dduvall ! [21:06:34] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.26 deployment blockers - https://phabricator.wikimedia.org/T300202 (10Ladsgroup) [21:06:45] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.26 deployment blockers - https://phabricator.wikimedia.org/T300202 (10Ladsgroup) [22:23:56] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): Determine if we need to communicate anything special about forward and backwards compatibility during train experiment week - https://phabricator.wikimedia.org/T303759 (10dduvall) Thanks for that explanation, @cscott! >>! In T303759#7785765, @cscott wrote: >... [22:38:56] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): Determine if we need to communicate anything special about forward and backwards compatibility during train experiment week - https://phabricator.wikimedia.org/T303759 (10daniel) Am 17.03.22 um 23:23 schrieb dduvall: > > And @daniel 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): Determine if we need to communicate anything special about forward and backwards compatibility during train experiment week - https://phabricator.wikimedia.org/T303759 (10dduvall) Great. Thank you both for the feedback. >>! In T303759#7787458, @daniel wrote:... [23:58:35] (03PS1) 10Ahmon Dancy: Increase robustness of DeployPromote._get_train_task() [tools/scap] - 10https://gerrit.wikimedia.org/r/771748 (https://phabricator.wikimedia.org/T302488)