[00:04:56] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): Announce Train experiment week to lists and slack - https://phabricator.wikimedia.org/T303361 (10dduvall) [00:09:16] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): Ensure coordination is set up with QTE prior to train experiment week - https://phabricator.wikimedia.org/T303364 (10dduvall) [00:13:29] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): Update deployment calendar/roadmap to reflect upcoming train experiment week - https://phabricator.wikimedia.org/T303365 (10dduvall) [00:17:33] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): Schedule Google meets for deploy windows during train experiment week - https://phabricator.wikimedia.org/T303366 (10dduvall) [00:22:39] 10Release-Engineering-Team (๐Ÿš‚๐Ÿงช Trainsperiment Week): Ensure train-branch-bot can cut 3 branches during Train Experiment Week - https://phabricator.wikimedia.org/T303367 (10dduvall) [00:30:16] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T300201 (10bd808) [05:34:15] PROBLEM - Check systemd state on doc1001 is CRITICAL: CRITICAL - degraded: The following units failed: rsync-doc-doc1002.eqiad.wmnet.service,rsync-doc-doc2001.codfw.wmnet.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [06:20:13] RECOVERY - Check systemd state on doc1001 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [09:33:10] 10Continuous-Integration-Infrastructure, 10Wikidata, 10wdwb-tech, 10Browser-Tests, and 2 others: Centrally look for flakey browser tests - https://phabricator.wikimedia.org/T277205 (10Addshore) Oh neat! Yeah that looks like a lovely approach. I'll run my terrible analysis one more time now and then stop,... [09:35:58] 10Continuous-Integration-Infrastructure, 10Wikidata, 10wdwb-tech, 10Browser-Tests, and 2 others: Centrally look for flakey browser tests - https://phabricator.wikimedia.org/T277205 (10Addshore) I centrally looked at these for quite some time and generally the only thing I found is that most flakiness is ca... [09:46:28] (03PS1) 10Robert Vogel: Refine config for Tuleap* repos [integration/config] - 10https://gerrit.wikimedia.org/r/769399 [10:58:19] I'm planning to deploy WMDE's MediaWiki config patches in a few minutes: https://wikitech.wikimedia.org/wiki/Deployments#Wednesday,_March_9 [10:58:47] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Radar), 10Security-Team, 10serviceops, and 2 others: Setup GitLab Runner in trusted environment - https://phabricator.wikimedia.org/T295481 (10Jelto) [12:19:41] (03CR) 10Hashar: [C: 03+2] helm-linter: Update envoyproxy to 1.18.3 [integration/config] - 10https://gerrit.wikimedia.org/r/769119 (https://phabricator.wikimedia.org/T300324) (owner: 10RLazarus) [12:21:48] (03Merged) 10jenkins-bot: helm-linter: Update envoyproxy to 1.18.3 [integration/config] - 10https://gerrit.wikimedia.org/r/769119 (https://phabricator.wikimedia.org/T300324) (owner: 10RLazarus) [12:22:28] (03PS1) 10Hashar: jjb: update helm-lint for envoyproxy 1.18.3 [integration/config] - 10https://gerrit.wikimedia.org/r/769419 (https://phabricator.wikimedia.org/T300324) [12:27:09] (03CR) 10Hashar: [C: 03+2] "I have build the image and switched the helm-lint CI job to use the new one ;)" [integration/config] - 10https://gerrit.wikimedia.org/r/769419 (https://phabricator.wikimedia.org/T300324) (owner: 10Hashar) [12:29:05] (03Merged) 10jenkins-bot: jjb: update helm-lint for envoyproxy 1.18.3 [integration/config] - 10https://gerrit.wikimedia.org/r/769419 (https://phabricator.wikimedia.org/T300324) (owner: 10Hashar) [13:37:44] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T300201 (10Nikerabbit) Train conductor: please be aware of {T303391}. I'm not marking it as a blocker, but it might be an issue for WMF production if... [14:05:50] (03PS4) 10Jaime Nuche: deploy-promote: integrate into `scap` [tools/scap] - 10https://gerrit.wikimedia.org/r/769023 (https://phabricator.wikimedia.org/T302488) [14:19:39] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Radar), 10Security-Team, 10serviceops, and 2 others: Setup GitLab Runner in trusted environment - https://phabricator.wikimedia.org/T295481 (10Jelto) [14:24:42] (03PS5) 10Jaime Nuche: deploy-promote: integrate into `scap` [tools/scap] - 10https://gerrit.wikimedia.org/r/769023 (https://phabricator.wikimedia.org/T302488) [14:27:23] (03CR) 10Jaime Nuche: deploy-promote: integrate into `scap` (0320 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/769023 (https://phabricator.wikimedia.org/T302488) (owner: 10Jaime Nuche) [14:33:08] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T300201 (10Zabe) [14:34:43] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T300201 (10Zabe) >>! In T300201#7763648, @Nikerabbit wrote: > Train conductor: please be aware of {T303391}. I'm not marking it as a blocker, but it m... [15:17:20] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T300201 (10Zabe) [15:42:46] 10MediaWiki-Releasing, 10MW-1.38-release: Branch REL1_38 for MediaWiki and all extensions and skins - https://phabricator.wikimedia.org/T302909 (10Jdforrester-WMF) a:03Jdforrester-WMF Nominally scheduled as https://wikitech.wikimedia.org/wiki/Deployments#deploycal-item-20220315T0200 and self-assigned; happy... [15:44:53] 10MediaWiki-Releasing, 10MediaWiki-Documentation: Make MediaWiki (major) release task tree - https://phabricator.wikimedia.org/T303238 (10Reedy) [15:45:50] 10MediaWiki-Releasing, 10MediaWiki-Documentation: Make MediaWiki (major) release task tree - https://phabricator.wikimedia.org/T303238 (10Jdforrester-WMF) [15:52:38] 10GitLab (Project Migration), 10Release-Engineering-Team (Doing), 10Product-Analytics: Create new GitLab people & project groups for Product Analytics - https://phabricator.wikimedia.org/T302727 (10mpopov) Thank you very much @brennen! [16:42:04] (03CR) 10RLazarus: [C: 03+1] "A step ahead of me, thank you!" [integration/config] - 10https://gerrit.wikimedia.org/r/769419 (https://phabricator.wikimedia.org/T300324) (owner: 10Hashar) [16:42:42] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T300201 (10Krinkle) [16:42:44] rzl: hopefully that new helm linter is working, I haven't tested it [16:43:02] hashar: no worries, will try it out today [16:43:48] 10Gerrit, 10Release-Engineering-Team (Seen), 10SRE, 10serviceops: Deploy multi-site plugin to gerrit1001 and gerrit2001 - https://phabricator.wikimedia.org/T217174 (10hashar) 05Openโ†’03Declined It is really unlikely we will setup the multsite plugin though if we change our mind later we can always reope... [16:44:57] (03CR) 10Ahmon Dancy: deploy-promote: integrate into `scap` (039 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/769023 (https://phabricator.wikimedia.org/T302488) (owner: 10Jaime Nuche) [16:45:05] 10Gerrit, 10Developer Productivity, 10Upstream, 10User-DannyS712: Feature request: Resolving merge conflicts from web interface - https://phabricator.wikimedia.org/T256731 (10hashar) 05Openโ†’03Declined Declining since we are not going to implement it ourself and there is a feature request filed upstream. [17:08:08] !log Updating Gerrit Comment.soy to get rid of a literal `null` string being inserted in notification emails | https://gerrit.wikimedia.org/r/c/operations/puppet/+/768005 | https://phabricator.wikimedia.org/T288312 [17:08:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:11:23] 10Gerrit: Gerrit notification email contains "null" - https://phabricator.wikimedia.org/T288312 (10hashar) 05Openโ†’03Resolved //This should be solved now// โ„ข [17:14:34] hashar: turned out I introduced a CI error with the envoy upgrade, I'll be sending you another patch shortly if you're still around today :) [17:21:11] (03PS1) 10RLazarus: helm-linter: Set permissions for /var/log/envoy [integration/config] - 10https://gerrit.wikimedia.org/r/769477 (https://phabricator.wikimedia.org/T300324) [17:24:15] (03PS2) 10RLazarus: helm-linter: Set permissions for /var/log/envoy [integration/config] - 10https://gerrit.wikimedia.org/r/769477 (https://phabricator.wikimedia.org/T300324) [17:25:28] (03CR) 10JMeybohm: [C: 03+1] helm-linter: Set permissions for /var/log/envoy [integration/config] - 10https://gerrit.wikimedia.org/r/769477 (https://phabricator.wikimedia.org/T300324) (owner: 10RLazarus) [17:30:37] 10Continuous-Integration-Config: Install unzip in CI images? - https://phabricator.wikimedia.org/T303417 (10Reedy) [17:31:13] 10Continuous-Integration-Config: Install unzip in CI images? - https://phabricator.wikimedia.org/T303417 (10Reedy) [17:42:30] rzl: doing it! [17:42:57] (03CR) 10Hashar: [C: 03+2] helm-linter: Set permissions for /var/log/envoy [integration/config] - 10https://gerrit.wikimedia.org/r/769477 (https://phabricator.wikimedia.org/T300324) (owner: 10RLazarus) [17:44:43] (03Merged) 10jenkins-bot: helm-linter: Set permissions for /var/log/envoy [integration/config] - 10https://gerrit.wikimedia.org/r/769477 (https://phabricator.wikimedia.org/T300324) (owner: 10RLazarus) [17:54:46] hashar: thank you! let me know when to recheck [17:54:58] rzl: I haven't rebuild it sorry [17:55:01] doing it now [17:55:05] then I will switch the jenkins job [17:55:15] much appreciated [17:55:23] we should find a better system for those SRe images :] [17:55:35] and let SRE people rebuild the images used by CI [17:56:07] that'd be great, I hate having to interrupt you all for this :) [17:57:31] (03PS1) 10Hashar: jjb: update helm-lint for /var/log/envoy perms [integration/config] - 10https://gerrit.wikimedia.org/r/769482 (https://phabricator.wikimedia.org/T300324) [17:57:47] Successfully published image docker-registry.discovery.wmnet/releng/helm-linter:0.3.2 [17:58:08] rzl: should be good now [17:58:22] (03CR) 10Hashar: [C: 03+2] "Deployed!" [integration/config] - 10https://gerrit.wikimedia.org/r/769482 (https://phabricator.wikimedia.org/T300324) (owner: 10Hashar) [17:59:29] 10Continuous-Integration-Infrastructure, 10Regression: Install unzip in CI images? - https://phabricator.wikimedia.org/T303417 (10Reedy) It's definitely a regression this time! See {T250496} [18:00:10] (03Merged) 10jenkins-bot: jjb: update helm-lint for /var/log/envoy perms [integration/config] - 10https://gerrit.wikimedia.org/r/769482 (https://phabricator.wikimedia.org/T300324) (owner: 10Hashar) [18:00:49] 10Continuous-Integration-Infrastructure, 10Regression: Install unzip in CI images again - https://phabricator.wikimedia.org/T303417 (10Reedy) [18:02:48] hashar: yep that did it \o/ thanks again [18:07:06] rzl: congratulations! I have added a todo for myself to reach out to the few SRE that relies on CI images so we can find a better workflow [18:37:12] (03CR) 10Ahmon Dancy: Support blubber as a buildkit gateway and Dockerfile frontend (032 comments) [blubber] - 10https://gerrit.wikimedia.org/r/504651 (https://phabricator.wikimedia.org/T301169) (owner: 10Dduvall) [18:37:15] 10Phabricator, 10Release-Engineering-Team, 10Discovery-Search: Phabricator is compatible with Elasticsearch 7.10 - https://phabricator.wikimedia.org/T303445 (10Gehel) [18:38:42] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T300201 (10Zabe) [18:43:02] thcipriani: Hey! Is release engineering the team in charge of phab ? I have just created T303445 [18:43:02] T303445: Phabricator is compatible with Elasticsearch 7.10 - https://phabricator.wikimedia.org/T303445 [18:51:03] (03CR) 10Ahmon Dancy: Provide scap prep auto history browsing and replay (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/763864 (https://phabricator.wikimedia.org/T301417) (owner: 10Dduvall) [19:01:51] 10Release-Engineering-Team, 10serviceops: Add some users to the docker group on deployment servers - https://phabricator.wikimedia.org/T303450 (10dancy) [19:05:16] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T300201 (10brennen) [19:15:42] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T300201 (10Sjoerddebruin) I think the above just broke gadgets on the upgraded wiki's. [19:18:20] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T300201 (10dancy) >>! In T300201#7765164, @Sjoerddebruin wrote: > I think the above just broke gadgets on the upgraded wiki's. Thanks for the report.... [19:21:05] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T300201 (10dancy) [19:24:34] (03PS6) 10Jaime Nuche: deploy-promote: integrate into `scap` [tools/scap] - 10https://gerrit.wikimedia.org/r/769023 (https://phabricator.wikimedia.org/T302488) [19:26:08] (03CR) 10Jaime Nuche: deploy-promote: integrate into `scap` (035 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/769023 (https://phabricator.wikimedia.org/T302488) (owner: 10Jaime Nuche) [19:26:50] (03CR) 10Ahmon Dancy: [C: 03+1] deploy-promote: integrate into `scap` [tools/scap] - 10https://gerrit.wikimedia.org/r/769023 (https://phabricator.wikimedia.org/T302488) (owner: 10Jaime Nuche) [19:32:34] (03CR) 10Ahmon Dancy: [C: 03+1] train: use the new version of `deploy-promote` integrated in `scap` (031 comment) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/769073 (https://phabricator.wikimedia.org/T302488) (owner: 10Jaime Nuche) [19:33:56] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Cloud-VPS, 10Wikidata, and 2 others: Wikibase selenium tests timeout, seemingly due to "memory compaction" events on CI VMs - https://phabricator.wikimedia.org/T281122 (10Addshore) Personally i dont think that more resources would go am... [19:39:19] gehel: we are! thanks for the task, I'll take a look! [19:39:52] thcipriani: hopefully it's just about checking that it's already all good! [19:40:49] 10Phabricator, 10Release-Engineering-Team, 10Discovery-Search: Phabricator is compatible with Elasticsearch 7.10 - https://phabricator.wikimedia.org/T303445 (10RhinosF1) I don't see any docs to say it's explicitly not supported. There seem to be version checks for 2 & 5 in the code. No idea if anything brea... [19:41:38] also, I'm trying to remember if we reverted back to using the built-in sql search or not, anyway, I'll do some digging [19:45:31] i _think_ we did. [19:46:10] same, although I don't know where that config lives, which is what I'm looking for :) [19:47:25] yeah, same [19:47:35] ferret is the thing, right? [19:48:59] digging through phab1001:/srv/deployment/phabricator/deployment/phabricator/conf grepping "ferret" came up empty [19:49:11] although it should, really, be pherret [19:49:41] https://phabricator.wikimedia.org/config/module/searchengine/ [19:49:50] yeah, it has bothered me this whole time that it's not spelled with a ph [19:50:17] huh, cluster.search in local.json is still pointed to elastic [19:53:47] I guess whatever is in the database, though, overrides local. So the page you found may indicate that we've reverted via the ui and left it in local config [19:54:20] https://phabricator.wikimedia.org/D1179 [19:54:28] i think this was the last i thought about this much [20:03:27] 10Release-Engineering-Team (Next), 10Release, 10Train Deployments: VE should preserve .mw-file-description class in Parsoid HTML - https://phabricator.wikimedia.org/T303469 (10ssastry) p:05Triageโ†’03Medium [20:04:20] 10Release-Engineering-Team (Next), 10Editing-team, 10VisualEditor, 10Release, 10Train Deployments: VE should preserve .mw-file-description class in Parsoid HTML - https://phabricator.wikimedia.org/T303469 (10ssastry) a:05dancyโ†’03None [20:04:46] (03CR) 10Ahmon Dancy: [V: 03+1 C: 04-1] "Tested successfully in my train-dev." [tools/scap] - 10https://gerrit.wikimedia.org/r/769023 (https://phabricator.wikimedia.org/T302488) (owner: 10Jaime Nuche) [20:38:19] 10Continuous-Integration-Infrastructure: Install unzip in CI images again - https://phabricator.wikimedia.org/T303417 (10hashar) The message can be found in integration/composer at 1.10.5 from two years ago. We run 2.1.8 now. The commit is https://github.com/composer/composer/commit/379d7691326304b91c250faabbf25... [20:40:47] (03PS1) 10Hashar: dockerfiles: add unzip package to Quibble images [integration/config] - 10https://gerrit.wikimedia.org/r/769517 (https://phabricator.wikimedia.org/T250496) [20:42:37] (03CR) 10Dduvall: [C: 03+1] "The demo hiccup this morning surfaced a major bug whereby all image metadata was missing from the gateway response, and apparently entrypo" [blubber] - 10https://gerrit.wikimedia.org/r/504651 (https://phabricator.wikimedia.org/T301169) (owner: 10Dduvall) [20:43:20] (03PS12) 10Dduvall: Support blubber as a buildkit gateway and Dockerfile frontend [blubber] - 10https://gerrit.wikimedia.org/r/504651 (https://phabricator.wikimedia.org/T301169) [20:47:12] 10Release-Engineering-Team (Next), 10Editing-team, 10VisualEditor, 10Patch-For-Review, and 2 others: VE should preserve .mw-file-description class in Parsoid HTML - https://phabricator.wikimedia.org/T303469 (10brennen) p:05Mediumโ†’03Unbreak! [21:07:34] 10Release-Engineering-Team (Next), 10Editing-team, 10VisualEditor, 10Patch-For-Review, and 2 others: VE should preserve .mw-file-description class in Parsoid HTML - https://phabricator.wikimedia.org/T303469 (10matmarex) The proposed fix is failing unit tests. [21:18:37] (03CR) 10Ahmon Dancy: Support blubber as a buildkit gateway and Dockerfile frontend (031 comment) [blubber] - 10https://gerrit.wikimedia.org/r/504651 (https://phabricator.wikimedia.org/T301169) (owner: 10Dduvall) [22:00:40] (03CR) 10Ahmon Dancy: [V: 03+1 C: 03+1] "The simple stuff I tried worked." [blubber] - 10https://gerrit.wikimedia.org/r/504651 (https://phabricator.wikimedia.org/T301169) (owner: 10Dduvall) [22:31:08] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10Security: Create initial proof of concept application security pipeline repository - https://phabricator.wikimedia.org/T289293 (10sbassett) [22:32:02] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10Security: Create initial proof of concept application security pipeline repository - https://phabricator.wikimedia.org/T289293 (10sbassett) [22:34:06] 10Release-Engineering-Team, 10Security Team AppSec, 10Security-Team, 10SecTeam-Processed, 10Security: Create nancy ci yaml template - https://phabricator.wikimedia.org/T301831 (10sbassett) [[ https://gitlab.wikimedia.org/repos/security/gitlab-ci-security-templates/-/blob/main/golang-nancy/golang-nancy-ci... [22:34:19] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10Security: Create initial proof of concept application security pipeline repository - https://phabricator.wikimedia.org/T289293 (10sbassett) [22:34:48] 10Release-Engineering-Team, 10Security Team AppSec, 10Security-Team, 10SecTeam-Processed, 10Security: Create nancy ci yaml template - https://phabricator.wikimedia.org/T301831 (10sbassett) 05In progressโ†’03Resolved [22:37:49] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10Security: Merge templates that are part of initial release to production branch - https://phabricator.wikimedia.org/T303482 (10sbassett) [22:50:57] there was some interest in the postmortem meeting last month in trying to deploy risky patches separately rather than with the train [22:52:18] so I was thinking about deploying https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralAuth/+/758120 that way [22:52:38] it is approved for self-merge, the table is created, it's ready to go [22:54:36] none of the backport windows are really convenient for me, ideally I would like to do it 23 hours from now, 22:00 UTC [22:55:01] I understand backport windows are for assisted deployment anyway, but I can just do this myself [22:55:31] AFAIK you can still make yourself a deployment window for "bigger" stuff like this [22:56:36] ^ that is my understanding as well [22:56:53] there's a google calendar but I don't appear to have write access to it [22:57:23] just make a new entry on https://wikitech.wikimedia.org/wiki/Deployments [22:57:45] ok [22:58:19] jouncebot will even yell at you when the time comes then :) [22:59:37] TimStarling: alternatively, you could schedule it under a backport deployment window and just let the scheduled deployer know at the beginning of the window (or in the cal notes) that you want to deploy it yourself [23:00:04] but the above suggestion makes sense to me as well [23:00:17] "none of the backport windows are really convenient for me" [23:01:14] ah, i see. i missed that bit :) [23:01:39] 8am-9am this house is full of screaming kids having various dramas in their efforts or non-efforts to get out of the door and off to school [23:01:59] 9am-10am blissfully quiet [23:02:19] hence better to do it at 22:00 [23:02:21] oh yes. i empathize completely [23:02:45] a 10 m/o and 2.5 y/o over here [23:03:14] 10Release-Engineering-Team (Next), 10Editing-team, 10VisualEditor, 10Patch-For-Review, and 2 others: VE should preserve .mw-file-description class in Parsoid HTML - https://phabricator.wikimedia.org/T303469 (10ssastry) After a bunch of testing, we feel it is safe to roll out train as is since Parsoid's htm... [23:03:35] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T300201 (10ssastry) [23:03:39] 10Release-Engineering-Team (Next), 10Editing-team, 10VisualEditor, 10Patch-For-Review, and 2 others: VE should preserve .mw-file-description class in Parsoid HTML - https://phabricator.wikimedia.org/T303469 (10ssastry) [23:04:29] so close together, that must be a handful [23:05:17] 10Release-Engineering-Team (Next), 10Editing-team, 10VisualEditor, 10Patch-For-Review, and 2 others: VE should preserve .mw-file-description class in Parsoid HTML - https://phabricator.wikimedia.org/T303469 (10ssastry) p:05Unbreak!โ†’03Medium We will test this separately and if required, we can roll out... [23:05:49] it really is :/ totally different needs [23:06:57] but i've discovered that i'm much better with toddlers so i wrangle the 2.5 y/o [23:10:48] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T300201 (10Zabe) [23:16:35] (03PS13) 10Dduvall: Support blubber as a buildkit gateway and Dockerfile frontend [blubber] - 10https://gerrit.wikimedia.org/r/504651 (https://phabricator.wikimedia.org/T301169) [23:17:21] (03CR) 10Jforrester: "We should cascade this to all quibble images rather than have it come through the next time they're re-built." [integration/config] - 10https://gerrit.wikimedia.org/r/769517 (https://phabricator.wikimedia.org/T250496) (owner: 10Hashar) [23:26:12] 10Release-Engineering-Team (Next), 10Editing-team, 10VisualEditor, 10Patch-For-Review, and 2 others: VE should preserve .mw-file-description class in Parsoid HTML - https://phabricator.wikimedia.org/T303469 (10Arlolra) A patch to run parserTests with the simulated effects of dropping the class is in https:... [23:58:16] (03PS1) 10Ahmon Dancy: build-mv-image: Don't rsync inactive php- trees [tools/release] - 10https://gerrit.wikimedia.org/r/769577 [23:59:02] (03CR) 10jerkins-bot: [V: 04-1] build-mv-image: Don't rsync inactive php- trees [tools/release] - 10https://gerrit.wikimedia.org/r/769577 (owner: 10Ahmon Dancy)