[02:32:58] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [03:31:30] 10Release-Engineering-Team (Done by Fri 03 Sep), 10Patch-For-Review, 10Release, 10Train Deployments: 1.37.0-wmf.21 deployment blockers - https://phabricator.wikimedia.org/T281162 (10mmodell) 05Open→03Resolved [03:36:31] (03CR) 1020after4: Automatically mention train blocker task in deploy-promote (031 comment) [tools/release] - 10https://gerrit.wikimedia.org/r/608936 (owner: 1020after4) [08:28:13] (03CR) 10Hashar: "I found out some jq magic to validate the task returned from the toolforge service :)" [tools/release] - 10https://gerrit.wikimedia.org/r/608936 (owner: 1020after4) [08:51:33] (03PS3) 10Hashar: configuring CI pipeline for example-node-api [integration/config] - 10https://gerrit.wikimedia.org/r/716598 (https://phabricator.wikimedia.org/T288130) (owner: 10WQuarshie) [08:51:50] (03CR) 10Hashar: configuring CI pipeline for example-node-api (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/716598 (https://phabricator.wikimedia.org/T288130) (owner: 10WQuarshie) [08:52:21] (03CR) 10Hashar: [C: 03+2] "I have created the jobs:" [integration/config] - 10https://gerrit.wikimedia.org/r/716598 (https://phabricator.wikimedia.org/T288130) (owner: 10WQuarshie) [08:53:34] (03Merged) 10jenkins-bot: configuring CI pipeline for example-node-api [integration/config] - 10https://gerrit.wikimedia.org/r/716598 (https://phabricator.wikimedia.org/T288130) (owner: 10WQuarshie) [09:21:20] Project mediawiki-core-doxygen-docker build #27641: 04FAILURE in 16 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/27641/ [10:05:16] (03CR) 10Hashar: [C: 03+2] "That change was depending on https://gerrit.wikimedia.org/r/c/operations/puppet/+/714624 hence why I haven't updated the job yesterday ;)" [integration/config] - 10https://gerrit.wikimedia.org/r/714633 (https://phabricator.wikimedia.org/T280247) (owner: 10Ebernhardson) [10:06:28] (03Merged) 10jenkins-bot: wdqs-gui: Adjust build to expect configuration in docroot/ [integration/config] - 10https://gerrit.wikimedia.org/r/714633 (https://phabricator.wikimedia.org/T280247) (owner: 10Ebernhardson) [10:13:37] Yippee, build fixed! [10:13:38] Project mediawiki-core-doxygen-docker build #27642: 09FIXED in 9 min 10 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/27642/ [11:14:48] 10Phabricator: Phabricator next hop when trying to upload a file - https://phabricator.wikimedia.org/T290321 (10dcaro) [11:15:57] 10Phabricator: Phabricator next hop when trying to upload a file - https://phabricator.wikimedia.org/T290321 (10dcaro) [11:20:18] Why is https://phabricator.wikimedia.org/p/IN/ duping ancient closed bug reports... [11:37:49] 10Project-Admins: Create project tag for Fix Suggester Bot - https://phabricator.wikimedia.org/T290323 (10kostajh) [12:18:54] (03CR) 10Hashar: [C: 04-1] "The tox-buster image has python 3.5 to 3.9 that comes from:" [integration/config] - 10https://gerrit.wikimedia.org/r/713972 (https://phabricator.wikimedia.org/T289222) (owner: 10Legoktm) [12:36:10] 10Project-Admins, 10Fix-Suggester-Bot: Create project tag for Fix Suggester Bot - https://phabricator.wikimedia.org/T290323 (10Peachey88) 05Open→03Resolved a:03Peachey88 Requested public project #Fix-Suggester-Bot has been created: https://phabricator.wikimedia.org/project/view/5519/ Interested people a... [13:19:06] PROBLEM - SSH on contint2001.mgmt is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [13:21:11] hashar: About? [13:51:55] I find myself wanting redis installed in the container that jenkins uses to run my tox tests. Do I have some ability to configure that container per-repo or is it a once-size-fits-all thing? [13:59:52] andrewbogott: It's not really per repo.. You can create customised images ontop to add extra functionality... [14:01:41] (to make it "per repo") [14:11:14] Reedy: can you tell me more about how to do that? Or are there docs? [14:11:27] (My repo already contains a dockerfile but I'm 90% sure that that's only for local testing) [14:12:29] Yeah, it will be [14:14:27] Which repo is it? [14:14:53] https://github.com/wikimedia/integration-config/tree/master/dockerfiles [14:15:04] We have specific tox- jobs now.. Which may already be being used [14:15:31] this one: https://gerrit.wikimedia.org/r/admin/repos/analytics/quarry/web [14:19:48] RECOVERY - SSH on contint2001.mgmt is OK: SSH OK - OpenSSH_6.6 (protocol 2.0) https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [14:20:03] andrewbogott: you should use the PipelineLib though I am not sure whether it can spawns multiple containers [14:21:00] The pytest lib I'm using only really works if redis is /in/ the same container where the tests are running. [14:21:25] So I don't think an additional container helps, I was hoping to modify/add to the existing one [14:21:33] (keep in mind that I barely understand how this works) [14:23:52] the idea is summarized at https://wikitech.wikimedia.org/wiki/Deployment_pipeline/Components [14:24:14] Blubber being an abstract DSL to generate the image, which would use one of SRE python image and you would get redis added to it [14:24:49] I am looking for a python based example [14:25:38] ok, so if I add a .pipeline to my repo then suddenly jenkins will do a totally different testing pipeline? Or do I need to add config somplace else to tell it to use that? [14:26:21] (this service is not deployed using docker/k8s in production, alas) [14:27:46] oh it is already in ci [14:28:45] so yeah instead of spawning the generic tox / npm test jobs, we would replace those with a job that runs the pipeline [14:29:11] which then use blubber to build the image(s) and run the tests [14:30:25] and i could not find a boilerplate example for a python project, so maybe you should pair with dduvall / jeena [14:31:23] ok -- I think we dcare already has an appointment with one of them to talk about this whole CI setup so I'll just plan to sit in [14:32:15] yeah that sounds like the sane path [14:32:36] and ultimately you would have a docker image published to the registry which might be used to migrate the service to kubernetes [14:32:51] that seems useful :) [14:32:52] thanks hashar ! [14:34:27] (03CR) 10Hashar: "Shouldn't we aim for Node JS 14? It was blocked at some point but we apparently have a node14 image now!" [integration/config] - 10https://gerrit.wikimedia.org/r/715556 (owner: 10Jforrester) [14:39:19] 10Continuous-Integration-Infrastructure, 10Patch-For-Review, 10User-Addshore: Remove DENY rules for https://integration.wikimedia.org/ci/computer/XXXX/builds - https://phabricator.wikimedia.org/T178458 (10hashar) 05Open→03Declined Forgot to decline that last time. We cant allow browsing builds for a sing... [14:40:12] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team (Seen), 10Patch-For-Review: Allow users to view build history in jenkins - https://phabricator.wikimedia.org/T177827 (10hashar) 05Open→03Declined We disallow any `/builds` URL at Apache level: ` modules/contint/templates/apa... [14:42:24] (03Abandoned) 10Hashar: Set maximum line length to 72 characters [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/705438 (owner: 10TK-999) [14:49:18] dduvall and/or jeena, when you're online can we talk about WMCS getting some help with pipline things? tia [14:50:15] (03PS2) 10Hashar: Zuul: [mediawiki/extensions/Math] Add Math to the main gate [integration/config] - 10https://gerrit.wikimedia.org/r/715144 (https://phabricator.wikimedia.org/T232948) (owner: 10Jforrester) [14:50:53] (03CR) 10Hashar: "CI failed since the gated extensions have to be added to zuul/parameter_functions.py . I have amended the patch." [integration/config] - 10https://gerrit.wikimedia.org/r/715144 (https://phabricator.wikimedia.org/T232948) (owner: 10Jforrester) [15:32:07] 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10Code-Health-DevEd-Curriculum: The Train - the workshop - https://phabricator.wikimedia.org/T249152 (10thcipriani) [15:32:23] 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10Code-Health-DevEd-Curriculum: The Train - an overview - https://phabricator.wikimedia.org/T249151 (10thcipriani) [15:33:04] 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10Documentation, 10User-brennen: Update/organize train deployment and related policy documentation - https://phabricator.wikimedia.org/T273802 (10thcipriani) p:05High→03Medium [15:34:06] 10Deployments, 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10Patch-For-Review: deploy mediawiki/tools/releases on deployment server with scap - https://phabricator.wikimedia.org/T274255 (10thcipriani) [15:42:25] (03CR) 10Hashar: "This change is ready for review." [integration/config] - 10https://gerrit.wikimedia.org/r/589336 (owner: 10Hashar) [15:42:31] (03CR) 10Hashar: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/589336 (owner: 10Hashar) [15:56:04] andrewbogott: do you have time in an hour or so? [15:58:22] jeena: yes! [15:58:37] that would be... 10AM pacific? [16:00:14] 10Deployments, 10Release-Engineering-Team (Seen), 10Developer Productivity, 10User-Majavah: Train changelog: Omit empty sections - https://phabricator.wikimedia.org/T249553 (10thcipriani) 05Open→03Resolved [16:00:49] 10Deployments, 10Release-Engineering-Team (Seen), 10Developer Productivity, 10User-Majavah: Train changelog: Omit empty sections - https://phabricator.wikimedia.org/T249553 (10thcipriani) Thanks for the patch @Majavah change log is much improved <3 [16:01:38] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: CI build failure for research landing page - https://phabricator.wikimedia.org/T290100 (10hashar) What Tyler said, change the image to either nodejs10-devel or nodejs12-devel :] [16:02:01] 10Release-Engineering-Team, 10ci-test-error: CI build failure for research landing page - https://phabricator.wikimedia.org/T290100 (10hashar) [16:02:15] (03CR) 10Ahmon Dancy: [C: 03+2] apt: Support user defined APT sources [blubber] - 10https://gerrit.wikimedia.org/r/716519 (https://phabricator.wikimedia.org/T289367) (owner: 10Dduvall) [16:02:44] jeena, btw, since officially speaking there are no meetings today I'm also totally fine waiting until next week [16:03:36] 10Release-Engineering-Team (Yak Shaving 🐃🪒): Automate posting team meeting notes - https://phabricator.wikimedia.org/T290333 (10thcipriani) [16:05:10] (03Merged) 10jenkins-bot: apt: Support user defined APT sources [blubber] - 10https://gerrit.wikimedia.org/r/716519 (https://phabricator.wikimedia.org/T289367) (owner: 10Dduvall) [16:05:24] 10Release-Engineering-Team (Yak Shaving 🐃🪒): Dev/maintainers add common reviewers - https://phabricator.wikimedia.org/T290334 (10thcipriani) [16:08:36] 10Phabricator: HTTP 502 "Next hop connection failed" when trying to upload a file - https://phabricator.wikimedia.org/T290321 (10Aklapper) [16:11:13] 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10Release Pipeline: Implement pipeline config validation - https://phabricator.wikimedia.org/T225335 (10dduvall) [16:11:35] 10Release-Engineering-Team (Yak Shaving 🐃🪒): pipeline-promote: Add patch author as reviewer to promote patch - https://phabricator.wikimedia.org/T281392 (10thcipriani) [16:14:16] 10Release-Engineering-Team, 10ci-test-error: CI build failure for research landing page - https://phabricator.wikimedia.org/T290100 (10Isaac) Hi @hashar -- again, I know very little about how CI works so guidance is appreciated. Is that just changing [[https://github.com/wikimedia/research-landing-page/blob/ma... [16:15:40] 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10mwcli: mwcli: Automate upload of new version to releases server - https://phabricator.wikimedia.org/T290335 (10jeena) [16:15:52] 10Release-Engineering-Team (Yak Shaving 🐃🪒): YAK: Refactor blubber be a native buildkit frontend - https://phabricator.wikimedia.org/T290336 (10thcipriani) [16:17:07] 10Release-Engineering-Team (Yak Shaving 🐃🪒): YAK: Refactor blubber be a native buildkit frontend - https://phabricator.wikimedia.org/T290336 (10thcipriani) [16:17:52] 10Release-Engineering-Team, 10ci-test-error: CI build failure for research landing page - https://phabricator.wikimedia.org/T290100 (10hashar) @Isaac yes! it is that simple :] CI uses that image as a base, copies the `package.json` and `package-lock.json` in the image then run `npm install && npm test`. I wo... [16:20:27] andrewbogott: I don't mind if you don't mind :P [16:20:37] ok! [16:21:31] hm, my test db is mysql 5, do you think the test will still be valid? [16:21:39] oops, wrong channel sorry ^ [16:24:14] 10Release-Engineering-Team (Yak Shaving 🐃🪒): Investigate password store options - https://phabricator.wikimedia.org/T290337 (10thcipriani) [16:25:17] 10Release-Engineering-Team (Yak Shaving 🐃🪒): Investigate password store options - https://phabricator.wikimedia.org/T290337 (10thcipriani) [16:27:15] 10Release-Engineering-Team (Yak Shaving 🐃🪒): Investigate password store options - https://phabricator.wikimedia.org/T290337 (10thcipriani) [16:29:19] 10Release-Engineering-Team (Seen), 10Scap: Automate rebuild l10n cache for Train - https://phabricator.wikimedia.org/T245187 (10hashar) Might require {T210602} [16:31:01] 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10MW-on-K8s, 10MediaWiki Train Development Environment, 10Release Pipeline, 10User-jeena (Sprint 8/18-8/31): Train-dev: Ability to deploy to k8s - https://phabricator.wikimedia.org/T287993 (10thcipriani) [16:33:47] (03CR) 10Jforrester: dockerfiles: Provide node12-test-browser-php{72,80}-composer images (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/715556 (owner: 10Jforrester) [16:34:55] 10Release-Engineering-Team, 10Patch-For-Review, 10ci-test-error: CI build failure for research landing page - https://phabricator.wikimedia.org/T290100 (10Isaac) 05Open→03Resolved a:03Isaac > I would go with node10 which is the version we use pretty much everywhere still. Excellent -- much appreciated... [16:38:54] 10Release-Engineering-Team, 10GitLab: Grant sbassett project owner rights for Gitlab repos they have created - https://phabricator.wikimedia.org/T290276 (10hashar) [16:44:22] 10Release-Engineering-Team (Yak Shaving 🐃🪒): Investigate if GitLab CI can enforce using Blubber as an image builder - https://phabricator.wikimedia.org/T290339 (10thcipriani) [16:45:25] 10Release-Engineering-Team (Yak Shaving 🐃🪒): Investigate if GitLab CI can enforce using Blubber as an image builder - https://phabricator.wikimedia.org/T290339 (10thcipriani) [17:30:25] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [150.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [17:36:01] !log staging blubberoid to deploy https://gerrit.wikimedia.org/r/c/blubber/+/716519 [17:36:03] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:37:37] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [17:42:02] !log deploying blubberoid:2021-09-03-160524-production to eqiad/codfw (https://gerrit.wikimedia.org/r/c/blubber/+/716519) (T289367) [17:42:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:42:06] T289367: blubber should make it convenient to install apt packages from a component - https://phabricator.wikimedia.org/T289367 [17:43:13] 10Release-Engineering-Team (Doing), 10Release Pipeline (Blubber): blubber should make it convenient to install apt packages from a component - https://phabricator.wikimedia.org/T289367 (10dduvall) 05Open→03Resolved Deployed! [17:54:25] The postmerge zuul queue looks stuck. [17:55:12] I've got 2 wikimedia/toolhub jobs sitting there in queued state for 30 minutes now [18:01:28] complaining makes all computers work. about 5 minutes later both started running [18:05:40] I'll have to remember that trick :) [18:41:12] 10Release-Engineering-Team (Doing), 10Release Pipeline (Blubber): blubber should make it convenient to install apt packages from a component - https://phabricator.wikimedia.org/T289367 (10Legoktm) Thank you, works perfectly! [19:18:46] 10Release-Engineering-Team (Doing), 10Release Pipeline: blubber: Execute python module as RUN command of dockerfile - https://phabricator.wikimedia.org/T267685 (10dduvall) p:05Triage→03Medium a:03dduvall [19:19:07] 10Release-Engineering-Team (Doing), 10Release Pipeline (Blubber): blubber: Execute python module as RUN command of dockerfile - https://phabricator.wikimedia.org/T267685 (10dduvall) [19:30:15] (03PS1) 10Krinkle: doc: Fix links to files in doc directories [integration/docroot] - 10https://gerrit.wikimedia.org/r/717574 (https://phabricator.wikimedia.org/T289196) [19:30:21] (03CR) 10Krinkle: [C: 03+2] doc: Fix links to files in doc directories [integration/docroot] - 10https://gerrit.wikimedia.org/r/717574 (https://phabricator.wikimedia.org/T289196) (owner: 10Krinkle) [19:32:01] (03Merged) 10jenkins-bot: doc: Fix links to files in doc directories [integration/docroot] - 10https://gerrit.wikimedia.org/r/717574 (https://phabricator.wikimedia.org/T289196) (owner: 10Krinkle) [19:34:18] 10Continuous-Integration-Infrastructure, 10VisualEditor, 10Editing-team (Tracking), 10Patch-For-Review, 10Performance-Team (Radar): doc.wm.o says "empty directory" for non-empty demos directory - https://phabricator.wikimedia.org/T289196 (10Krinkle) 05Open→03Resolved [19:43:58] 10Release-Engineering-Team (Doing), 10Release Pipeline (Blubber): blubber: Execute python module as RUN command of dockerfile - https://phabricator.wikimedia.org/T267685 (10dduvall) [19:44:00] 10Release-Engineering-Team (Radar), 10Abstract Wikipedia team, 10Release Pipeline (Blubber): More flexible treatment of requirements in `python`; control over `PYTHONPATH` - https://phabricator.wikimedia.org/T282795 (10dduvall) [19:44:23] (03PS1) 10Dduvall: python: Support execution of site package modules in builder [blubber] - 10https://gerrit.wikimedia.org/r/717582 (https://phabricator.wikimedia.org/T267685) [20:07:09] (03CR) 10Ahmon Dancy: [C: 03+1] "LGTM" [blubber] - 10https://gerrit.wikimedia.org/r/717582 (https://phabricator.wikimedia.org/T267685) (owner: 10Dduvall) [20:13:43] (03CR) 10Jforrester: [C: 03+1] "Nice." [blubber] - 10https://gerrit.wikimedia.org/r/717582 (https://phabricator.wikimedia.org/T267685) (owner: 10Dduvall) [20:18:32] 10Release-Engineering-Team (Radar), 10serviceops: nodejs-devel image does not contain npm - https://phabricator.wikimedia.org/T290209 (10thcipriani) [20:18:38] 10Release-Engineering-Team, 10ci-test-error: CI build failure for research landing page - https://phabricator.wikimedia.org/T290100 (10thcipriani) [20:19:47] 10Release-Engineering-Team (Radar), 10serviceops: nodejs-devel image does not contain npm - https://phabricator.wikimedia.org/T290209 (10thcipriani) >>! In T290209#7328337, @Dzahn wrote: > I don't know if this is the reason it was left out of the package but using npm to install software would conflict with L3... [20:27:13] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [150.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [20:28:02] 10Release-Engineering-Team, 10GitLab: Grant sbassett project owner rights for Gitlab repos they have created - https://phabricator.wikimedia.org/T290276 (10thcipriani) 05Open→03Resolved a:03thcipriani I noticed the the `people/security-team` group were not owners of the `security` namespace AND you were... [20:29:15] (03CR) 10Hashar: "I am not sure why we have that "only-standalone" job. When quibble runs for ProofreadPage, it will attempt to run any test marked @standa" [integration/config] - 10https://gerrit.wikimedia.org/r/714328 (https://phabricator.wikimedia.org/T289465) (owner: 10Inductiveload) [20:35:24] (03CR) 10Hashar: [C: 03+1] "+1 lets get rid of stretch indeed even if we later have to further upgrade from node 12 to 14 😐" [integration/config] - 10https://gerrit.wikimedia.org/r/715556 (owner: 10Jforrester) [20:44:19] 10Release-Engineering-Team (Radar), 10serviceops: nodejs-devel image does not contain npm - https://phabricator.wikimedia.org/T290209 (10Legoktm) There have been various related issues like {T286212} (maybe the same issue here) and {T284112}. My recommendation would be to stop using the stretch-based nodejs-de... [20:48:07] 10Release-Engineering-Team (Radar), 10serviceops, 10Regression: nodejs-devel image does not contain npm - https://phabricator.wikimedia.org/T290209 (10Legoktm) ` $ podman run --rm -it --entrypoint=bash docker-registry.wikimedia.org/nodejs-devel:0.1.1 Trying to pull docker-registry.wikimedia.org/nodejs-devel:... [20:50:59] 10Release-Engineering-Team (Radar), 10serviceops, 10Regression: nodejs-devel image does not contain npm - https://phabricator.wikimedia.org/T290209 (10Legoktm) a:03Legoktm Yeah, so same thing as earlier: ` root@be63654e236f:/# apt-cache policy nodejs nodejs: Installed: 6.11.0~dfsg-1+wmf5 Candidate: 6.1... [20:57:27] jenkins seems rather backlogged :/ [21:00:25] there are a lot of MW core jobs in stacked patches, which slows down zuul AIUI [21:03:59] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [21:04:08] 10Release-Engineering-Team (Radar), 10serviceops, 10Patch-For-Review, 10Regression: nodejs-devel image does not contain npm - https://phabricator.wikimedia.org/T290209 (10Legoktm) 05Open→03Resolved ` $ podman run --rm -it --entrypoint=bash docker-registry.wikimedia.org/nodejs-devel:0.1.2 Trying to pull... [21:47:52] (03CR) 10Legoktm: Add tox-bullseye container (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/713972 (https://phabricator.wikimedia.org/T289222) (owner: 10Legoktm) [22:11:21] 10Release-Engineering-Team (Doing), 10Release Pipeline (Blubber): Blubber Docker compiler output does not include copied stages referenced transitively via includes - https://phabricator.wikimedia.org/T289880 (10dduvall) [22:12:34] 10Release-Engineering-Team, 10SRE-Access-Requests: Requesting exec access to pods in 'ci' namespace staging kubernetes - https://phabricator.wikimedia.org/T290360 (10dancy) [22:12:54] 10Release-Engineering-Team, 10SRE-Access-Requests: Requesting exec access to pods in 'ci' namespace staging kubernetes - https://phabricator.wikimedia.org/T290360 (10dancy) [22:12:57] 10Release-Engineering-Team (Doing), 10MW-on-K8s, 10SRE: Automated validation of mediawiki-multiversion images - https://phabricator.wikimedia.org/T288629 (10dancy) [22:13:16] 10Release-Engineering-Team (Doing), 10MW-on-K8s, 10SRE: Automated validation of mediawiki-multiversion images - https://phabricator.wikimedia.org/T288629 (10dancy) 05Stalled→03Open [22:37:50] 10Release-Engineering-Team (Radar), 10Release Pipeline (Blubber): Can't use git master in blubber for dependencies without caching issues - https://phabricator.wikimedia.org/T283981 (10dduvall) 05Open→03Declined IMO the workaround used here is the correct way to do things as Dockerfiles (or any build confi... [22:41:07] 10Release-Engineering-Team (Doing), 10SRE, 10serviceops-radar, 10Release Pipeline (Blubber): build and import blubber package for buster and bullseye (which supports v4) - https://phabricator.wikimedia.org/T283891 (10dduvall) 05Open→03Resolved a:03dduvall Resolving since the docs are now up-to-date. [22:42:41] 10Release-Engineering-Team (Seen): Revise stretch-php72-fpm-apache2 to use current WMF prod packages (PHP 7.2.26+) - https://phabricator.wikimedia.org/T245197 (10dduvall) [22:51:14] 10Release-Engineering-Team (Doing), 10Release Pipeline (Blubber): Blubber: runuser permissions to node_modules/.cache - https://phabricator.wikimedia.org/T255434 (10dduvall) p:05Triage→03Medium a:03dduvall [22:52:29] 10Release-Engineering-Team (Radar), 10Release Pipeline (Blubber): Can't use git master in blubber for dependencies without caching issues - https://phabricator.wikimedia.org/T283981 (10Legoktm) Sorry I missed T283981#7184506. I think declining this is fine, specifically, >>! In T283981#7332401, @dduvall wrote... [23:02:41] !log Creating integration-agent-qemu-1002 (Debian 11 Bullseye, g3.cores8.ram24.disk20.ephemeral40.4xiops), ref T284774 [23:02:43] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:02:44] T284774: Provide one or more Qemu agents in CI that use a newer version than 2.x - https://phabricator.wikimedia.org/T284774 [23:35:46] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10SRE, 10MW-1.37-notes (1.37.0-wmf.15; 2021-07-19), 10Patch-For-Review: Have linters/tests results show up as comments in files on gerrit - https://phabricator.wikimedia.org/T209149 (10Tgr) GerritRobotComments seems to...