[00:21:28] (03CR) 10Samwilson: Zuul: Add BetaFeatures to WikiEditor dependencies (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/777438 (https://phabricator.wikimedia.org/T304596) (owner: 10MusikAnimal) [00:24:33] (03PS3) 10MusikAnimal: Zuul: Add BetaFeatures to WikiEditor dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/777438 (https://phabricator.wikimedia.org/T304596) [07:18:24] good morning [07:20:26] morning [07:52:01] there's 119 jobs for Translate extension right now queued up in Zuul [08:43:49] 10GitLab, 10serviceops: Standardize Debian package builds on GitLab CI - https://phabricator.wikimedia.org/T304491 (10MatthewVernon) You have plenty of options there already; presumably another would be to spin up a VM to do a build (then discard it afterwards)? [08:51:00] 10Release-Engineering-Team (Deployment services), 10Scap, 10Patch-For-Review: Experiment rsync "new compression" - https://phabricator.wikimedia.org/T252540 (10hashar) https://gerrit.wikimedia.org/r/c/operations/puppet/+/774824 aligned `scap-master-sync` to use `rsync --new-compress` as well. [09:31:34] 10Release-Engineering-Team, 10Scap, 10User-brennen: scap deploy-promote fails on git push - https://phabricator.wikimedia.org/T304557 (10hashar) @jnuche and I encountered the issue yesterday when he tried `scap deploy-promote` and we tried some debugging this morning again. I think I have found an explanatio... [09:41:26] 10Release-Engineering-Team, 10dev-images, 10mwcli, 10Patch-For-Review: Apache docker image strips Authorization header - https://phabricator.wikimedia.org/T305143 (10Addshore) [09:50:43] 10Project-Admins: Create project tag for Extension WikiSEO - https://phabricator.wikimedia.org/T305534 (10Octfx) [10:04:42] 10Project-Admins: Create project tag for Extension WikiSEO - https://phabricator.wikimedia.org/T305534 (10RhinosF1) I'm pretty sure we should aim to track in a single place. If it's going to be wikimedia deployed, I imagine we must use Phab. [10:09:12] 10Project-Admins: Create project tag for Extension WikiSEO - https://phabricator.wikimedia.org/T305534 (10Octfx) Agreed, Phab should (will be?) be main, but there are numerous people using github that don't know/want to use phabricator. Maybe explicitly stating that phab should be used for reports (while leavi... [10:10:47] 10Project-Admins: Create project tag for Extension WikiSEO - https://phabricator.wikimedia.org/T305534 (10Aklapper) Hi, I strongly believe that there should be one single canonical issue tracker where tickets about a codebase are filed and managed. Having more than one leads to duplicated efforts, fragmented con... [10:47:40] 10Project-Admins: Create project tag for Extension WikiSEO - https://phabricator.wikimedia.org/T305534 (10Octfx) I'm good either way. @RhinosF1 whats your take on this. GitHub or Phabricator? [10:47:57] 10Project-Admins: Create project tag for Extension WikiSEO - https://phabricator.wikimedia.org/T305534 (10RhinosF1) Phab [10:50:27] 10Project-Admins: Create project tag for Extension WikiSEO - https://phabricator.wikimedia.org/T305534 (10Octfx) Allright, I'll close GitHub issues as soon as we are ready to deploy [10:50:54] 10Project-Admins: Create project tag for Extension WikiSEO - https://phabricator.wikimedia.org/T305534 (10Octfx) [11:41:26] (03PS4) 10Jforrester: Zuul: [mediawiki/extensions/WikiEditor] Add BetaFeatures to phan deps [integration/config] - 10https://gerrit.wikimedia.org/r/777438 (https://phabricator.wikimedia.org/T304596) (owner: 10MusikAnimal) [11:41:36] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/WikiEditor] Add BetaFeatures to phan deps [integration/config] - 10https://gerrit.wikimedia.org/r/777438 (https://phabricator.wikimedia.org/T304596) (owner: 10MusikAnimal) [11:44:13] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/WikiEditor] Add BetaFeatures to phan deps [integration/config] - 10https://gerrit.wikimedia.org/r/777438 (https://phabricator.wikimedia.org/T304596) (owner: 10MusikAnimal) [11:44:33] !log Zuul: [mediawiki/extensions/WikiEditor] Add BetaFeatures to phan deps for T304596 [11:44:35] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:44:35] T304596: Add Realtime Preview as Beta Feature - https://phabricator.wikimedia.org/T304596 [11:59:47] 10GitLab (Infrastructure), 10Data-Persistence-Backup, 10serviceops, 10Patch-For-Review, 10User-brennen: Backups for GitLab - https://phabricator.wikimedia.org/T274463 (10Jelto) Backup is present in bacular for the new folder structure: ` jelto@backup1001:~$ sudo bconsole restore Select item: (1-13): 5... [12:13:01] 10GitLab (Infrastructure), 10Data-Persistence-Backup, 10serviceops, 10Patch-For-Review, 10User-brennen: Backups for GitLab - https://phabricator.wikimedia.org/T274463 (10jcrespo) Thanks for the testing, @Jelto: those are the kind of issues I meant that could only be caught when testing restores! Let me k... [14:00:53] (03PS1) 10Jaime Nuche: deploy-promote: create wikiversions update patch using user ssh-agent [tools/scap] - 10https://gerrit.wikimedia.org/r/777812 (https://phabricator.wikimedia.org/T304557) [14:08:57] (03CR) 10Jaime Nuche: "Tested locally in train-dev. The git push command receives the value stored in `cli.user_ssh_auth_sock` before `cli._setup_environ` has a " [tools/scap] - 10https://gerrit.wikimedia.org/r/777812 (https://phabricator.wikimedia.org/T304557) (owner: 10Jaime Nuche) [15:38:47] (03CR) 10Ahmon Dancy: "The change looks good to me. Just one suggestion." [tools/scap] - 10https://gerrit.wikimedia.org/r/777812 (https://phabricator.wikimedia.org/T304557) (owner: 10Jaime Nuche) [15:39:07] 10Release-Engineering-Team, 10Scap, 10Patch-For-Review, 10User-brennen: scap deploy-promote fails on git push - https://phabricator.wikimedia.org/T304557 (10dancy) Nice debugging! [15:44:47] (03PS2) 10Jaime Nuche: deploy-promote: create wikiversions update patch using user ssh-agent [tools/scap] - 10https://gerrit.wikimedia.org/r/777812 (https://phabricator.wikimedia.org/T304557) [15:45:30] (03CR) 10Jaime Nuche: deploy-promote: create wikiversions update patch using user ssh-agent (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/777812 (https://phabricator.wikimedia.org/T304557) (owner: 10Jaime Nuche) [15:45:47] (03CR) 10Ahmon Dancy: [C: 03+2] deploy-promote: create wikiversions update patch using user ssh-agent [tools/scap] - 10https://gerrit.wikimedia.org/r/777812 (https://phabricator.wikimedia.org/T304557) (owner: 10Jaime Nuche) [15:46:45] (03Merged) 10jenkins-bot: deploy-promote: create wikiversions update patch using user ssh-agent [tools/scap] - 10https://gerrit.wikimedia.org/r/777812 (https://phabricator.wikimedia.org/T304557) (owner: 10Jaime Nuche) [16:46:01] 10Release-Engineering-Team (Next), 10Scap: scap proxies are CPU and/or network bound - https://phabricator.wikimedia.org/T305466 (10thcipriani) [16:48:15] 10Release-Engineering-Team (🌱 Spring Cleaning — April 2022), 10dev-images, 10mwcli, 10Patch-For-Review: Apache docker image strips Authorization header - https://phabricator.wikimedia.org/T305143 (10thcipriani) a:03jeena Assigning to @jeena to review: https://gitlab.wikimedia.org/repos/releng/dev-images/... [16:51:40] 10Release-Engineering-Team (Doing), 10Release Pipeline (Blubber): deploy.sh in operations/deployment-charts does not work - https://phabricator.wikimedia.org/T305033 (10thcipriani) a:03thcipriani [16:52:03] 10Release-Engineering-Team (Doing), 10Release Pipeline (Blubber): deploy.sh in operations/deployment-charts does not work - https://phabricator.wikimedia.org/T305033 (10thcipriani) p:05Triage→03Medium [16:53:25] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10serviceops: docker-report-releng failing on multiple image tags because of certificate validation error - https://phabricator.wikimedia.org/T304875 (10thcipriani) [16:55:30] 10Release-Engineering-Team (Next), 10Patch-For-Review, 10Release Pipeline (Blubber), 10User-dduvall: Blubber must not use easy_install to install pip, but python3-pip - https://phabricator.wikimedia.org/T304860 (10thcipriani) 05Open→03Stalled Awaiting @dduvall 's return in May 2022 [16:56:52] 10Release-Engineering-Team (🌱 Spring Cleaning — April 2022), 10dev-images, 10mwcli, 10Patch-For-Review: Apache docker image strips Authorization header - https://phabricator.wikimedia.org/T305143 (10jeena) a:05jeena→03brennen [16:57:33] 10Release-Engineering-Team, 10Scap, 10Infrastructure-Foundations, 10serviceops, 10Patch-For-Review: Use scap to deploy itself to scap targets - https://phabricator.wikimedia.org/T303559 (10jnuche) a:03jnuche [18:25:57] (03CR) 10Ahmon Dancy: backport: compare fetched commits with backports (037 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/773897 (https://phabricator.wikimedia.org/T295495) (owner: 10Jeena Huneidi) [19:19:34] 10Phabricator, 10Security-Team: How to categorise security tasks for beginners? - https://phabricator.wikimedia.org/T305579 (10Reedy) [19:20:14] 10Release-Engineering-Team, 10Scap, 10User-brennen: scap deploy-promote fails on git push - https://phabricator.wikimedia.org/T304557 (10hashar) And that was a quick fix. Well done! [19:21:08] 10Phabricator, 10Security-Team: How to categorise security tasks for beginners? - https://phabricator.wikimedia.org/T305579 (10Reedy) [19:21:34] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10SecTeam-Processed, 10Security: Create best practices / guidelines documentation for Gitlab application security ci templates - https://phabricator.wikimedia.org/T295374 (10sbassett) 05In progress→03Resolved Calling this one done... [19:21:37] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10Security: Create initial proof of concept application security pipeline repository - https://phabricator.wikimedia.org/T289293 (10sbassett) [19:22:57] 10GitLab, 10Release-Engineering-Team (Done by Fri 03 Sep), 10Security Team AppSec, 10Security-Team, and 2 others: Create Security Team group within gitlab.wikimedia.org - https://phabricator.wikimedia.org/T289292 (10sbassett) [19:23:09] 10GitLab (CI & Job Runners), 10Security Team AppSec, 10Security-Team, 10Security: Create initial proof of concept application security pipeline repository - https://phabricator.wikimedia.org/T289293 (10sbassett) 05In progress→03Resolved Resolving for now as sub-tasks for this first phase of work have b... [19:29:09] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10serviceops: docker-report-releng failing on multiple image tags because of certificate validation error - https://phabricator.wikimedia.org/T304875 (10hashar) Those php71 images are obsolete, we have removed them from integration/config i... [19:35:13] 10Phabricator, 10Security-Team, 10SecTeam-Processed: How to categorise security tasks for beginners? - https://phabricator.wikimedia.org/T305579 (10sbassett) @Aklapper - would you have any preference on what we do here? [20:02:53] (03PS1) 10Krinkle: doc: Update WikiLambda links [integration/docroot] - 10https://gerrit.wikimedia.org/r/777865 [20:03:58] !log rebooting phabricator [20:03:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:40:17] (03PS1) 10Eevans: jjb: Define test pipeline for image-suggestions [integration/config] - 10https://gerrit.wikimedia.org/r/777872 (https://phabricator.wikimedia.org/T303408) [20:54:32] Apparently zip's from Gerrit/gitiles come out differently each time. https://github.com/google/gitiles/issues/217 [20:55:01] https://gerrit.wikimedia.org/g/mediawiki/extensions/EventLogging/+/refs/changes/02/754002/22/modules/lib/foreign-resources.yaml#4 [20:55:34] npm and github handle this by setting a deterministic timestamp (either epoch or commit date) [21:44:11] (03CR) 10Ahmon Dancy: "I did more testing today." [tools/scap] - 10https://gerrit.wikimedia.org/r/773897 (https://phabricator.wikimedia.org/T295495) (owner: 10Jeena Huneidi) [21:44:40] 10Continuous-Integration-Infrastructure, 10MinervaNeue, 10Vector, 10Accessibility, and 2 others: Add automated accessibility tests in CI to generate accessibility benchmarks for Skins - https://phabricator.wikimedia.org/T301184 (10Jdlrobson) This is currently being worked on so moving into board for tracking. [21:44:45] 10Continuous-Integration-Infrastructure, 10MinervaNeue, 10Vector, 10Accessibility, and 2 others: Add automated accessibility tests in CI to generate accessibility benchmarks for Skins - https://phabricator.wikimedia.org/T301184 (10Jdlrobson) p:05Triage→03Medium [21:51:27] (03PS1) 10Jdlrobson: Add selenium daily job to Vector [integration/config] - 10https://gerrit.wikimedia.org/r/777884 (https://phabricator.wikimedia.org/T301184) [21:52:48] (03CR) 10Jdlrobson: "Hey all.. we're experimenting with some accessibility testing in Vector which eventually we'd hope to apply to other repos." [integration/config] - 10https://gerrit.wikimedia.org/r/777884 (https://phabricator.wikimedia.org/T301184) (owner: 10Jdlrobson) [21:55:41] 10Phabricator, 10Security-Team, 10SecTeam-Processed: How to categorise security tasks for beginners? - https://phabricator.wikimedia.org/T305579 (10Aklapper) Clearly `#1`. :) `#2` would be more fragmentation via more `good first $somesmallsubset task` tags... [22:04:20] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments: 1.39.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T305212 (10Jdlrobson) Hi there @jnuche! We have an issue with wmf5 (T305171) that we'd rather not be present going into the weekend. If the train runs... [22:11:40] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments: 1.39.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T305212 (10eigyan) @Jdlrobson, thank you for helping me sort a path forward with this patch deployment -> much appreciated! [22:38:31] I'm running into a permission error with npm-test not being allowed to create a temporary file [22:38:34] : Error: EACCES: permission denied, open '/srv/app/.eslintcache' [22:38:35] https://integration.wikimedia.org/ci/job/metrics-platform-pipeline-test-js/90/console [22:38:45] This uses docker-registry.wikimedia.org/nodejs12-devel rather than the CI images [22:39:18] I'm not really familiar with the pipeline stack and what's different there or what this repo might be doing differnt than it shouhld for the 'test' variant to be allowed to write files [22:40:18] looked through https://wikitech.wikimedia.org/wiki/PipelineLib/Tutorial/TestingHelloNode and some adjacent pages [22:44:42] Krinkle: you will need runs.insecurely: true in your blubber config if you want the runtime user to be able to write files into the image. [22:45:44] hidden in https://wikitech.wikimedia.org/wiki/Blubber/User_Guide#Common_Config -- "should the application in the container be run as a user that can't write anything to the filesystem, including caches, or as a user that can? Production variants should have this set to `false`, but other variants may set it to `true` in some circumstances." [22:45:48] Krinkle: if you look at the .pipeline/blubber.yaml in the repo this is from and it has "runs: { insecurely: false }" or insecurely: true that would inf [22:46:08] that would make a difference like that [22:46:31] it's not set at all, so the default of false -- https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/libs/metrics-platform/+/refs/heads/master/.pipeline/blubber/js.yaml [22:46:53] ok, now I need to figure out whether to set insecurely in pipeline/config in pipelines[x] or in blubber/x.yaml somewhere under the variant/test [22:47:15] Krinkle: it goes in the variant in the blubber file [22:48:00] and example -- https://github.com/wikimedia/toolhub/blob/main/.pipeline/blubber.yaml#L95-L99 [22:48:26] ack, trying.. [22:48:55] One thing about procedural APIs, you can relatively quickly find their documentation :) [22:50:10] thx, that worked :) [23:04:33] Whoever added this to scap-deploy, thanks, this is so much clearer than before: [23:04:35] `23:04:10 Copying from deploy1002.eqiad.wmnet:/srv/mediawiki-staging to mwdebug1002.eqiad.wmnet:/srv/mediawiki` [23:04:41] exactly what comes from where to where [23:04:45] I imagine dancy <3 [23:04:48] You're welcome. :-) [23:49:47] 10Release-Engineering-Team (Next), 10Release, 10Train Deployments: 1.39.0-wmf.7 deployment blockers - https://phabricator.wikimedia.org/T305213 (10Krinkle)