[03:01:46] Project mwcore-phpunit-coverage-master build #1698: 04STILL FAILING in 1 min 45 sec: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/1698/ [07:09:43] 10Project-Admins, 10Turkish-Sites: trwiki Twinkle project - https://phabricator.wikimedia.org/T289095 (10Aklapper) 05Open→03Resolved a:03Aklapper Requested public parent project #Twinkle-on-Turkish-Wikipedia has been created: https://phabricator.wikimedia.org/project/view/5589/ (I have not made this a s... [07:10:07] 10Project-Admins, 10Turkish-Sites: Create Twinkle-on-Turkish-Wikipedia project - https://phabricator.wikimedia.org/T289095 (10Aklapper) [07:15:05] majavah: legoktm: thank you for the patch to get composer installed via a debian package on toolforge :) [07:28:43] 10Continuous-Integration-Infrastructure, 10Toolforge: Stop using integration/composer and then archive the repo - https://phabricator.wikimedia.org/T249949 (10hashar) [07:29:21] 10Continuous-Integration-Infrastructure: Stop using integration/composer and then archive the repo - https://phabricator.wikimedia.org/T249949 (10hashar) Toolforge is in progress via T287900 (thank you). I filed T293055 for extdist which would require new VM based on Bullseye to replace the Stretch ones. [08:11:31] 10Phabricator: Group Bug reports and Feature Requests closer together - https://phabricator.wikimedia.org/T293059 (10Jidanni) [08:19:12] 10Phabricator: Group Bug reports and Feature Requests closer together - https://phabricator.wikimedia.org/T293059 (10Aklapper) 05Open→03Invalid Please read T293054#7418909 [08:25:22] 10Phabricator, 10Project-Admins: Archive User-Majavah and related Herald rule - https://phabricator.wikimedia.org/T293060 (10Majavah) [08:40:41] 10Gerrit, 10Upstream: Gerrit's New UI reply dialog ignores Ctrl-Z - https://phabricator.wikimedia.org/T192666 (10hashar) [08:59:07] 10Gerrit, 10Upstream: Gerrit's New UI reply dialog ignores Ctrl-Z - https://phabricator.wikimedia.org/T192666 (10hashar) Still happening in Gerrit 3.3.6. It might be due to some Chrome specific code being used that prevented another feature from working properly on Firefox ( https://gerrit-review.googlesource... [09:07:25] (03CR) 10Hashar: [C: 03+2] ":)" [tools/scap] - 10https://gerrit.wikimedia.org/r/728567 (https://phabricator.wikimedia.org/T292638) (owner: 10Ahmon Dancy) [09:08:00] 10Release-Engineering-Team, 10Scap, 10Patch-For-Review: Bad behavior when scap is control-c'd - https://phabricator.wikimedia.org/T292638 (10hashar) a:03dancy [09:08:22] 10Release-Engineering-Team (Doing), 10Scap, 10Patch-For-Review: Bad behavior when scap is control-c'd - https://phabricator.wikimedia.org/T292638 (10hashar) [09:09:26] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10ci-test-error (WMF-deployed Build Failure): TAR_ENTRY_ERROR ENOSPC: no space left on device - https://phabricator.wikimedia.org/T292729 (10hashar) 05Open→03Resolved a:03hashar The agent went out of disk space following the releases... [09:09:45] Project beta-scap-sync-world build #23538: 04FAILURE in 15 min: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/23538/ [09:09:52] (03Merged) 10jenkins-bot: Avoid cascading exception in handle_keyboard_interrupt() [tools/scap] - 10https://gerrit.wikimedia.org/r/728567 (https://phabricator.wikimedia.org/T292638) (owner: 10Ahmon Dancy) [09:16:45] 10Release-Engineering-Team, 10SRE: Reduce latency of new Scap releases - https://phabricator.wikimedia.org/T292646 (10hashar) [09:17:27] 10Release-Engineering-Team (Doing), 10SRE: Reduce latency of new Scap releases - https://phabricator.wikimedia.org/T292646 (10hashar) [09:18:23] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 (10hashar) Thank you for the cleanup @Legoktm ! [09:18:50] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Radar): docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 (10hashar) [09:20:25] 10Project-Admins: Create user board for Majavah - https://phabricator.wikimedia.org/T237402 (10Aklapper) Disabled as requested in T293060 [09:20:43] 10Phabricator, 10Project-Admins: Archive User-Majavah and related Herald rule - https://phabricator.wikimedia.org/T293060 (10Aklapper) 05Open→03Resolved a:03Aklapper Heh, alright! :) Done (thanks for all your work!) [09:20:55] 10Gerrit, 10Release-Engineering-Team (Radar), 10Discovery: Update gerrit submit type for discovery repositories in gerrit - https://phabricator.wikimedia.org/T255509 (10hashar) [09:22:21] 10Gerrit, 10Release-Engineering-Team (Seen), 10User-zeljkofilipin: Gerrit timeout when cloning mediawiki/core - https://phabricator.wikimedia.org/T292858 (10hashar) [09:22:23] 10Release-Engineering-Team (Next), 10Gerrit (Gerrit 3.4): Upgrade to Gerrit 3.4 - https://phabricator.wikimedia.org/T292759 (10hashar) [09:29:49] Yippee, build fixed! [09:29:50] Project beta-scap-sync-world build #23539: 09FIXED in 18 min: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/23539/ [09:44:07] 10Gerrit, 10Release-Engineering-Team (Radar), 10Discovery, 10Discovery-Search: Update gerrit submit type for discovery repositories in gerrit - https://phabricator.wikimedia.org/T255509 (10Gehel) [10:45:46] 10Release-Engineering-Team (Doing), 10SRE: Reduce latency of new Scap releases - https://phabricator.wikimedia.org/T292646 (10jijiki) @Legoktm is working on a cookbook to speed up packaging of scap https://gerrit.wikimedia.org/r/c/operations/cookbooks/+/727605. The rollout process has to stay as it is though (... [11:53:32] 10Beta-Cluster-Infrastructure: upload.wikimedia.beta.wmflabs.org certificate expired - https://phabricator.wikimedia.org/T293070 (10dom_walden) [11:58:57] 10Beta-Cluster-Infrastructure: upload.wikimedia.beta.wmflabs.org certificate expired - https://phabricator.wikimedia.org/T293070 (10RhinosF1) Again? [12:01:10] 10Beta-Cluster-Infrastructure: upload.wikimedia.beta.wmflabs.org certificate expired - https://phabricator.wikimedia.org/T293070 (10RhinosF1) {T271778} [12:03:14] dwalden: I'm sure we fixed that at some point in the last 90 days [12:03:28] Well 90 days ago [12:03:32] And 90 days before [12:03:52] I think it's specific to the upload cache proxy node, it has needed a manual restart before too [12:08:29] majavah: would you be so kind to try? [12:11:37] RhinosF1: no, as an attempt to get some wmf team who cares about deployment-prep working to take over its maintenance [12:13:08] majavah: fair enough [13:22:41] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for scherukuwada - https://phabricator.wikimedia.org/T293087 (10SCherukuwada) [14:04:51] (03PS1) 10C. Scott Ananian: WIP: Translate contains VisualEditor components now [integration/config] - 10https://gerrit.wikimedia.org/r/730207 [14:07:05] (03CR) 10jerkins-bot: [V: 04-1] WIP: Translate contains VisualEditor components now [integration/config] - 10https://gerrit.wikimedia.org/r/730207 (owner: 10C. Scott Ananian) [14:20:47] (03PS1) 10Hashar: Disable PHPUnit Junit report by default [integration/quibble] - 10https://gerrit.wikimedia.org/r/730211 (https://phabricator.wikimedia.org/T256402) [14:22:35] (03CR) 10Hashar: "I kept the logic behind in case we do want a Junit report to be crafted, but this change disables the report by default. That also mean t" [integration/quibble] - 10https://gerrit.wikimedia.org/r/730211 (https://phabricator.wikimedia.org/T256402) (owner: 10Hashar) [14:25:51] (03PS2) 10Isabelle Hurbain-Palatin: WIP: Translate contains VisualEditor components now [integration/config] - 10https://gerrit.wikimedia.org/r/730207 (owner: 10C. Scott Ananian) [14:27:02] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.38.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T281168 (10Zabe) [14:40:56] 10Release-Engineering-Team (Doing), 10Scap: Bad behavior when scap is control-c'd - https://phabricator.wikimedia.org/T292638 (10dancy) 05Open→03Resolved [14:50:06] 10Deployments, 10Release-Engineering-Team: Add the automatic train branch as a 'window' in the deployments calendar so people can be aware of it happening - https://phabricator.wikimedia.org/T293101 (10Jdforrester-WMF) [15:01:11] (03CR) 10Jforrester: [C: 03+2] Disable PHPUnit Junit report by default [integration/quibble] - 10https://gerrit.wikimedia.org/r/730211 (https://phabricator.wikimedia.org/T256402) (owner: 10Hashar) [15:01:34] Project mwcore-phpunit-coverage-master build #1699: 04STILL FAILING in 1 min 33 sec: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/1699/ [15:04:09] 10Deployments, 10Release-Engineering-Team: Add the automatic train branch as a 'window' in the deployments calendar so people can be aware of it happening - https://phabricator.wikimedia.org/T293101 (10Zabe) It is already, see https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/tools/release/+/refs/heads/... [15:04:48] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.38.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T281168 (10dancy) [15:06:18] 10Deployments, 10Release-Engineering-Team: Add the automatic train branch as a 'window' in the deployments calendar so people can be aware of it happening - https://phabricator.wikimedia.org/T293101 (10Jdforrester-WMF) [15:06:23] 10Deployments, 10Release-Engineering-Team: Add the automatic train branch as a 'window' in the deployments calendar so people can be aware of it happening - https://phabricator.wikimedia.org/T293101 (10Jdforrester-WMF) Huh, indeed. [15:06:33] 10Release-Engineering-Team (Doing): Document automatic MW branching time in Deployment Calendar - https://phabricator.wikimedia.org/T284487 (10Jdforrester-WMF) [15:08:25] (03PS1) 10Jforrester: Deployments calendar: Be specific that this is automatic [tools/release] - 10https://gerrit.wikimedia.org/r/730214 [15:17:03] (03Merged) 10jenkins-bot: Disable PHPUnit Junit report by default [integration/quibble] - 10https://gerrit.wikimedia.org/r/730211 (https://phabricator.wikimedia.org/T256402) (owner: 10Hashar) [15:23:56] 10Deployments, 10Release-Engineering-Team: Add the automatic train branch as a 'window' in the deployments calendar so people can be aware of it happening - https://phabricator.wikimedia.org/T293101 (10ihurbain) On the other hand, https://wikitech.wikimedia.org/wiki/Deployments#Monday,_October_11 didn't have a... [15:26:43] 10Deployments, 10Release-Engineering-Team: Add the automatic train branch as a 'window' in the deployments calendar so people can be aware of it happening - https://phabricator.wikimedia.org/T293101 (10Zabe) >>! In T293101#7420233, @ihurbain wrote: > On the other hand, https://wikitech.wikimedia.org/wiki/Deplo... [15:47:11] (03PS1) 10Jforrester: Install 7.21.0 [integration/npm] - 10https://gerrit.wikimedia.org/r/730220 (https://phabricator.wikimedia.org/T293105) [15:58:33] (03CR) 10Jforrester: "I'm sad they dropped the FLOSS bits of the package from the release (changelog, authors list, licence…)." [integration/npm] - 10https://gerrit.wikimedia.org/r/730220 (https://phabricator.wikimedia.org/T293105) (owner: 10Jforrester) [16:07:51] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.38.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T281168 (10dancy) [16:19:22] 10MediaWiki-Releasing, 10Release-Engineering-Team (Doing), 10LibUp, 10MediaWiki-Release-Tools: Work out why initial skin/extension branches into REL1_35/REL1_36 were on the wrong branch - https://phabricator.wikimedia.org/T283883 (10Jdforrester-WMF) 05Open→03Resolved >>! In T283883#7414287, @Legoktm wr... [16:26:24] 10Continuous-Integration-Config, 10Test-Coverage: mwcore-phpunit-coverage-master failing due to composer upgrade - https://phabricator.wikimedia.org/T292910 (10Jdforrester-WMF) >>! In T292910#7414677, @Reedy wrote: > ` > 16:00:07 + exec docker run --volume /srv/git:/srv/git:ro --volume /srv/jenkins/workspace/w... [16:29:31] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.38.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T281168 (10dancy) [17:01:17] (03PS1) 10Ostrzyciel: Remove the THICC extension (archived) [integration/config] - 10https://gerrit.wikimedia.org/r/730250 (https://phabricator.wikimedia.org/T292948) [17:22:36] (03PS2) 10Jforrester: Zuul: [mediawiki/extensions/THICC] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/730250 (https://phabricator.wikimedia.org/T292948) (owner: 10Ostrzyciel) [17:22:41] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/THICC] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/730250 (https://phabricator.wikimedia.org/T292948) (owner: 10Ostrzyciel) [17:24:08] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.38.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T281168 (10dancy) [17:24:24] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/THICC] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/730250 (https://phabricator.wikimedia.org/T292948) (owner: 10Ostrzyciel) [17:28:20] !log Zuul: [mediawiki/extensions/THICC] Mark as archived for T292948 [17:28:22] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:28:22] T292948: Archive the THICC extension - https://phabricator.wikimedia.org/T292948 [17:32:34] 10Phabricator: Allow partially locking a task - https://phabricator.wikimedia.org/T292505 (10Legoktm) I've only had really bad experiences when this feature is used on GitHub, either you/others end up filing a duplicate ticket to make a comment, or you don't leave the comment and everyone else is worse off becau... [18:00:51] James_F: Ty re VE/Parsoid [18:01:03] That's one less blocker for me watch [18:04:08] Ha. Sure. [18:13:31] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10Platform Engineering Roadmap, 10User-Daniel: Convert static mediawiki configuration to form more suitable for containers - https://phabricator.wikimedia.org/T263166 (10Legoktm) >>! In T263166#7389233, @daniel wrote: > I'm currently doing an exploration of h... [18:24:28] 10Release-Engineering-Team (Doing), 10SRE, 10serviceops: Reduce latency of new Scap releases - https://phabricator.wikimedia.org/T292646 (10Legoktm) >>! In T292646#7419245, @jijiki wrote: > The rollout process has to stay as it is though (upgrade on canaries first, and roll out to all hosts after 1-2 days)... [18:39:38] (03CR) 10Krinkle: [V: 03+2 C: 03+2] Install 7.21.0 [integration/npm] - 10https://gerrit.wikimedia.org/r/730220 (https://phabricator.wikimedia.org/T293105) (owner: 10Jforrester) [18:44:17] 10Continuous-Integration-Config, 10Fresnel, 10Performance-Team: All quibble-fresnel jobs crash before starting: "Could not find browser" - https://phabricator.wikimedia.org/T292684 (10dpifke) a:03Krinkle [18:44:25] 10Continuous-Integration-Config, 10Fresnel, 10Performance-Team: All quibble-fresnel jobs crash before starting: "Could not find browser" - https://phabricator.wikimedia.org/T292684 (10Krinkle) I did it locally on Jenkins, I did not commit it. [18:57:54] (03PS1) 10Jforrester: Dockerfiles: [node{12,14}] Upgrade npm from 7.5.2 to 7.21.0 [integration/config] - 10https://gerrit.wikimedia.org/r/730279 (https://phabricator.wikimedia.org/T293105) [19:01:12] good evening [19:07:37] (03CR) 10Krinkle: [C: 03+1] jjb: Bump almost all Quibble jobs to 1.1.0 (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/726880 (https://phabricator.wikimedia.org/T291549) (owner: 10Kosta Harlan) [19:08:24] (03CR) 10Hashar: [C: 03+2] Dockerfiles: [node{12,14}] Upgrade npm from 7.5.2 to 7.21.0 [integration/config] - 10https://gerrit.wikimedia.org/r/730279 (https://phabricator.wikimedia.org/T293105) (owner: 10Jforrester) [19:10:08] 10Continuous-Integration-Config, 10Fresnel, 10Performance-Team (Radar): All quibble-fresnel jobs crash before starting: "Could not find browser" - https://phabricator.wikimedia.org/T292684 (10Krinkle) p:05Medium→03Low a:05Krinkle→03kostajh OK. Assigning over to you tentatively to investigate why cust... [19:10:27] (03Merged) 10jenkins-bot: Dockerfiles: [node{12,14}] Upgrade npm from 7.5.2 to 7.21.0 [integration/config] - 10https://gerrit.wikimedia.org/r/730279 (https://phabricator.wikimedia.org/T293105) (owner: 10Jforrester) [19:11:30] !log Building Docker images for https://gerrit.wikimedia.org/r/730279 [19:11:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:13:28] 10Release-Engineering-Team (Doing), 10SRE, 10serviceops: Reduce latency of new Scap releases - https://phabricator.wikimedia.org/T292646 (10jijiki) @Legoktm we may debdeploy scap everywhere, and then for whatever reason we need to push change Y fast due to issue X. If scap fails everywhere because of a bug w... [19:21:06] 10MediaWiki-Releasing, 10Release-Engineering-Team (Doing), 10LibUp, 10MediaWiki-Release-Tools: Work out why initial skin/extension branches into REL1_35/REL1_36 were on the wrong branch - https://phabricator.wikimedia.org/T283883 (10hashar) Great. Thank you ;) [19:23:04] (03PS5) 10Jeena Huneidi: WIP: mw is deployable to k8s in traindev [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726025 (https://phabricator.wikimedia.org/T287993) [19:30:07] 10Phabricator: Phabricator should not embed videos from Wikimedia Commons when the file is deleted - https://phabricator.wikimedia.org/T293139 (10Dylsss) [19:30:14] 10Deployments, 10Release-Engineering-Team: During holiday: add the automatic train branch as a 'window' in the deployments calendar so people can be aware of it happening - https://phabricator.wikimedia.org/T293101 (10hashar) 05duplicate→03Open [19:31:02] 10Deployments, 10Release-Engineering-Team: During holiday: add the automatic train branch as a 'window' in the deployments calendar so people can be aware of it happening - https://phabricator.wikimedia.org/T293101 (10hashar) Greath catch! I have reopened this task and update the details to give a bit more con... [19:51:24] 10Continuous-Integration-Config, 10Fresnel, 10Performance-Team (Radar): All quibble-fresnel jobs crash before starting: "Could not find browser" - https://phabricator.wikimedia.org/T292684 (10kostajh) >>! In T292684#7421218, @Krinkle wrote: > OK. Assigning over to you tentatively to investigate why custom co... [19:54:33] (03PS1) 10Jforrester: jjb: Update node{12,14} jobs to use images with npm 7.21.0 [integration/config] - 10https://gerrit.wikimedia.org/r/730329 (https://phabricator.wikimedia.org/T293105) [20:03:27] 10Release-Engineering-Team (Seen), 10Documentation, 10GitLab (Auth & Access), 10Upstream, 10User-brennen: Improve / fix request to join project notifications - https://phabricator.wikimedia.org/T293088 (10brennen) Doing some tentative tagging here. [20:10:27] (03CR) 10Hashar: "Successfully published image docker-registry.discovery.wmnet/releng/node12-test-browser-php80-composer:0.2.0-s4" [integration/config] - 10https://gerrit.wikimedia.org/r/730279 (https://phabricator.wikimedia.org/T293105) (owner: 10Jforrester) [20:10:41] James_F: the images are build [20:10:52] Awesome. [20:11:17] I've got a meeting or two left today, but I can deploy the jjb changes (with testing) later if you don't have time. [20:11:56] given it is 10pm yeah I am giving up for today ;) [20:22:37] 10Continuous-Integration-Config, 10Fresnel, 10Performance-Team (Radar): All quibble-fresnel jobs crash before starting: "Could not find browser" - https://phabricator.wikimedia.org/T292684 (10hashar) [20:35:33] (03CR) 10Krinkle: Docker: Drop node10-test-browser, unused (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/720153 (owner: 10Jforrester) [20:37:48] (03CR) 10Krinkle: Docker: Drop node10-test-browser, unused (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/720153 (owner: 10Jforrester) [20:43:38] 10Release-Engineering-Team (Doing), 10GitLab (Project Migration), 10User-brennen: Early adoption signup for WMF GitLab - https://phabricator.wikimedia.org/T282842 (10sbassett) The #security-team would be interested in migrating [[ https://gerrit.wikimedia.org/r/admin/repos/q/filter:wikimedia%252Fsecurity | s... [20:44:08] 10Continuous-Integration-Config, 10Fresnel, 10Performance-Team (Radar): All quibble-fresnel jobs crash before starting: "Could not find browser" - https://phabricator.wikimedia.org/T292684 (10hashar) I have downloaded the build output from good and bad builds: * https://integration.wikimedia.org/ci/job/media... [20:44:18] and that fun thing with fresnel [20:44:25] seems it might be related to the npm 6 > 7 upgrade [20:46:12] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T281168 (10brennen) Scratchpad for any required blocker mails, train summary, etc.: https://etherpad.wikimedia.org/p/train-1.38.0-wmf.4 [20:46:23] 11pm that is all for me tonight ;) [20:51:47] 10Continuous-Integration-Config, 10Fresnel, 10Performance-Team (Radar): All quibble-fresnel jobs crash before starting: "Could not find browser" - https://phabricator.wikimedia.org/T292684 (10Krinkle) `quibble-buster`, which quibble-fresnel inherits from, installs its own copy of npm from Git which hasn't ch... [20:52:54] hashar: aye, yeah, I'm guesing there's an internal conflict there with too new npm with too old node. Those two versions probably don't like each other. [20:53:14] might be [20:53:19] I commented on-task suggesting we get quibble onto node12 as well. which I didn't realie is still on node10! [20:53:52] 10Release-Engineering-Team (Priority Backlog 🔥), 10MW-on-K8s, 10MediaWiki Train Development Environment, 10Release Pipeline, and 2 others: Train-dev: Ability to deploy to k8s - https://phabricator.wikimedia.org/T287993 (10jeena) [20:53:56] it's using the debian nodejs from buster [20:53:58] I think we explciitly did not upgrade quibble-fresnel to node12 since CI was unable to run its testsuite due to a fault in qemu [20:54:02] which is nodejs 10 [20:54:09] and that [20:54:12] fresnel != fresh [20:54:33] oops [20:54:44] node12+ indeed is broken in buster Qemu [20:54:45] cheked bash history, I did try npm install fresnel@1.0.0 [20:55:09] and hence fresh tests break with node10 in Qemu agents [20:55:16] Fresnel should work fine on node12. [20:55:18] I remember I had to mess up with puppeteer at some point to prevent it from download chromium [20:55:23] its own repo afaik already runs on node12 [20:55:26] since in production we want to use the chromium from debian [20:55:52] puppeteet used to not store chromium binary in npm cache [20:55:53] if params['ZUUL_PROJECT'].startswith('mediawiki/services/chromium-render'): [20:55:53] params['PUPPETEER_SKIP_CHROMIUM_DOWNLOAD'] = 'true' [20:55:58] now it does [20:56:03] so we probalby don't need that [20:56:16] it was very specific to that project [20:56:42] puppeteer does not support skipping download unless you ensure you use the same version of chromium as puppeteer expects, it's not likely to work with the stable version we get from debian given npm tooling will move much faster than that [20:56:56] and for fresnel afaik we don't set that env anyway, so should be fine [20:57:01] yup [20:57:12] right chromium-render is for prod, that wants to test against debian [20:57:15] after nuking ~/.npm and running npm 7.5.2 to install fresnel, I don't see any download [20:57:27] which at 80MB or so would definitely be noticeable on my DSL connection [20:57:47] which node version, any env vars relating to puppeteer/node/npm? [20:57:52] but with npm 6, that does download chromium [20:58:07] nodejs 12 [20:58:25] and pretty sure my env is empty [20:58:26] quibble images maybe should descend from the node12 image? [20:58:41] so I suspect the same happens with the releng/quibble-fresnel image. When we build it, it does not download chromium [21:00:12] right, that's consistnet at least. [21:00:38] as for why the behavior change.. I have no clue [21:00:52] maybe puppeteer needs to be bumped due to some change between npm6/7 [21:00:59] like figuring out where the cache is [21:01:19] I imagine if the API changed in npm 7, puppeteer install would fail to lookup in the cache and might just bail out download entirely [21:02:35] then if i install fresnel from npm 6 (which populates the cache) and the install from npm 7, I get chromium ( something like node_modules/fresnel/node_modules/puppeteer/.local/linux-chromium/... [21:04:32] I can reproduce the issue in fresh-node(12) [21:04:59] e.g. mediawiki$ fresh-node -net -env; then cd /tmp; npm install fresnel; cd -; /tmp/node_modules/.bin/fresnel record before [21:05:05] and it won't find chromium on a clean install [21:05:24] so npm 7 and puppeteer aren't wokring together for some reason [21:05:31] \O/ [21:05:51] seems that install puppeteer@4.0.1 [21:05:58] so maybe npm bisect the install? [21:06:07] I was looking at https://github.com/puppeteer/puppeteer/blob/main/CHANGELOG.md [21:06:24] but history is cut before 5.5.0 (I guess they truncate it from time to time) [21:06:35] and I should have tried fresh really [21:07:19] :D [21:08:15] I think before v5 they published the notes only on github releases [21:08:57] you can try npm installing a newer version of puppeteer and see what happens [21:09:06] yeah [21:09:08] but I am giving up for today [21:09:10] thx [21:09:16] point taken about using fresh-node! [21:09:22] (I have no excuse since I have it installed) [21:09:49] at least it got reproduced, which is like a good chunk of solving the puzzle [21:10:55] `npm install puppeteer@4.0.1` [21:10:58] that one installs chromium [21:11:08] so maybe the lock file has somethign to do with it [21:11:30] npm 7 broke so much stuff in the lock file processing.. :/ [21:14:20] I guess we're a bit lucky that so much stuff still works on node 10. So our qunit and selenium tests are now using node10 with npm7, operating on lock files tested with node12. [21:14:28] maybe? [21:15:39] and I can't test updating puppeteer in for fresnel in CI since that runs node12 now [21:15:57] so now we're back to using fresh-node10 to test that part [21:16:08] ok, good thing we have that still :P [21:17:00] the quibble containers having node10 , that means we "test" the lockfiles ;) [21:17:22] and the basic job running "npm install && npm test" uses node12 [21:17:28] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10Platform Engineering Roadmap, 10User-Daniel: Convert static mediawiki configuration to form more suitable for containers - https://phabricator.wikimedia.org/T263166 (10daniel) >>! In T263166#7420935, @Legoktm wrote: >>>! In T263166#7389233, @daniel wrote: >... [21:19:00] (03CR) 10Krinkle: Docker: Drop node10-test-browser, unused (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/720153 (owner: 10Jforrester) [21:19:24] hashar: quibble is for mw repos only, fresnel.git uses node12 images to test its code [21:19:43] whereas post-merge post-publish it'll end up in quibble-fresnel now with node10 which I can't test against [21:19:58] to then run against mw repos to test performance [21:20:26] VE and OOUI are in a similar situation [21:20:54] anyway, I'll use fresh-node10 to run the CI test locally when updating puppeteer in fresnel.git [21:21:23] hopefully there's a working version between puppeteer 4 and 9 that runs fine on Node 10 and npm 7 [21:21:41] and one that also works on node 12 [21:21:45] hpefully yes! [21:22:23] I am following the task at least, but for now I am out! :] happy hackings [21:36:44] 10Continuous-Integration-Config, 10Fresnel, 10Performance-Team (Radar): Update Fresnel to Puppeteer 7 (Chromium 90) - https://phabricator.wikimedia.org/T293150 (10Krinkle)