[00:26:25] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Cloud-VPS, 10Patch-For-Review, 10cloud-services-team (Kanban): Support Cinder for CI docker workers - https://phabricator.wikimedia.org/T277078 (10Bstorm) @Krinkle and @Jdforrester-WMF I think that should unblock your efforts a... [00:28:18] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Cloud-VPS, 10Patch-For-Review, 10cloud-services-team (Kanban): Support Cinder for CI docker workers - https://phabricator.wikimedia.org/T277078 (10Bstorm) Where it might *not* work is if the init for LVM in general gets applied... [00:28:59] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Cloud-VPS, 10Patch-For-Review, 10cloud-services-team (Kanban): Support Cinder for CI docker workers - https://phabricator.wikimedia.org/T277078 (10Bstorm) If it doesn't, I have another trick that should do it, so let me know. [02:48:31] 10MediaWiki-Releasing, 10MW-1.38-release, 10MobileFrontend (Tracking), 10Readers-Web-Backlog (Tracking): Bundle MobileFrontend extension with MediaWiki - https://phabricator.wikimedia.org/T191734 (10Tgr) By adding it to the make-release script ([[https://gerrit.wikimedia.org/r/c/mediawiki/tools/release/+/5... [05:26:02] anyone mind merging https://gerrit.wikimedia.org/r/c/mediawiki/core/+/720905? updates CREDITS which should be trivial release prep [06:42:01] 10Release-Engineering-Team (Radar), 10GitLab, 10Patch-For-Review, 10User-brennen: WARNING: In GitLab 14.0 we will begin removing all configuration backups older than yourgitlab_rails['backup_keep_time'] setting (currently set to: 259200) - https://phabricator.wikimedia.org/T288324 (10Jelto) 05Open→03Res... [06:53:59] 10Phabricator, 10Developer Productivity, 10MediaWiki-Releasing (Workflow Improvements): Make it easier to track tasks which are stalled pending a MediaWiki release - https://phabricator.wikimedia.org/T290960 (10Aklapper) @DannyS712: Statuses are global across all and any projects; I am very reluctant to add... [07:46:55] 10Release-Engineering-Team (Radar), 10MW-on-K8s, 10SRE, 10serviceops: The restricted/mediawiki-webserver image should include skins and resources - https://phabricator.wikimedia.org/T285232 (10Joe) @dancy I like your idea, even if I generally don't like using rewrite rules much. I'll try to bake a set of... [08:58:38] !log update wb_changes_dispatch set chd_disabled = 1 where chd_site = 'enwiki'; (T290985) [08:58:45] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:58:45] T290985: Enable new dispatching mechanism on beta - https://phabricator.wikimedia.org/T290985 [09:14:06] (03CR) 10Addshore: "I'd be up for giving this a go today!" [integration/config] - 10https://gerrit.wikimedia.org/r/683589 (https://phabricator.wikimedia.org/T278706) (owner: 10Michael Große) [09:16:18] (03CR) 10Michael Große: Add and use job to deploy branch in node project (032 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/683589 (https://phabricator.wikimedia.org/T278706) (owner: 10Michael Große) [09:30:44] akostajh: Is it possible that I've been misunderstanding the new Parsoid architecture all along, and that VE is already browser-testable without our Apache backend work? [09:32:09] AIUI, the issue was that an api.php call would make an outbound request to the Parsoid service, which might call back into the API—but with PHP Parsoid this doesn't happen. [13:55:05] 10Release-Engineering-Team (Radar), 10MW-on-K8s, 10SRE, 10serviceops, 10Patch-For-Review: The restricted/mediawiki-webserver image should include skins and resources - https://phabricator.wikimedia.org/T285232 (10Joe) As it stands, my new configuration would mean that we're going to issue a permanent red... [15:21:46] addshore: I haven't heard mention of backstage.io yet. [15:39:14] Well, I started playing around with it a bit :) https://backstage.toolforge.org/catalog?filters%5Bkind%5D=component&filters%5Buser%5D=all [15:39:40] my gut says it could be a really nice thing for us to adopt in general [15:49:40] * dancy clicks [15:52:31] addshore: someone in product has been talking about an "app catalog"... who was it? Maybe it was Seve Kim? [15:52:43] on IRC? [15:53:12] backstage is pretty neat, you can write plugins and basically auto generate a whole bunch of stuff based on git repos etc [15:53:19] and people from ldap etc [15:53:34] could auto add every tool on toolforge on there to [15:53:44] I can't find him on irc :( [[office:Contact list]] says his nick is skim_ but /whowas has no joy for that [15:54:06] well, im on your slack too, but I can't message people, they need to find me :P [15:54:38] every tool on toolforge is https://toolhub-demo.wmcloud.org/ :) [15:54:46] plus other things not on toolforge [15:55:20] oooh nice, yeah, backstage could just use https://toolhub-demo.wmcloud.org/api-docs to display the same data and stay in sync etc [15:57:12] yeah, one of the goals of toolhub is to act as a repo for other tools to pull/push data [15:57:40] addshore: I pinged Seve on slack about this and gave him your contact there [15:57:45] awesome! [16:23:16] 10Release-Engineering-Team, 10GitLab, 10Security-Team: Add "Samuel (WMF)" account to Security Team group in gitlab.wikimedia.org - https://phabricator.wikimedia.org/T291094 (10sbassett) [16:23:56] 10Continuous-Integration-Config, 10Toolforge, 10cloud-services-team (Kanban): Set up CI for cloud/toolforge/buildpacks repository - https://phabricator.wikimedia.org/T265685 (10Bstorm) I've done this in toolsbeta by using tekton pipelines. The way it works is using an unprivileged pod that uses a new contain... [16:25:01] 10Release-Engineering-Team, 10Scap, 10serviceops: Deploy Scap version 4.0.0 - https://phabricator.wikimedia.org/T291095 (10dancy) [16:25:47] 10Release-Engineering-Team, 10Scap, 10serviceops: Deploy Scap version 4.0.0 - https://phabricator.wikimedia.org/T291095 (10dancy) [16:28:19] 10Phabricator, 10Developer Productivity, 10MediaWiki-Releasing (Workflow Improvements): Make it easier to track tasks which are stalled pending a MediaWiki release - https://phabricator.wikimedia.org/T290960 (10mmodell) Wouldn't a project 'tag' be appropriate for this? [16:30:16] (03PS1) 10Ahmon Dancy: RELEASE.md updates [tools/scap] - 10https://gerrit.wikimedia.org/r/721351 (https://phabricator.wikimedia.org/T277793) [16:32:02] 10Phabricator, 10Patch-For-Review: Evaluate adding "In progress" status to Phabricator. - https://phabricator.wikimedia.org/T288956 (10mmodell) If there are no strong opinions about the icon, I'm going to go with `step-forward` {icon step-forward}, we can change it if there are objections. [16:39:02] 10Phabricator, 10Patch-For-Review: Evaluate adding "In progress" status to Phabricator. - https://phabricator.wikimedia.org/T288956 (10mmodell) 05Open→03progress [16:39:51] Project mediawiki-core-doxygen-docker build #27984: 04FAILURE in 18 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/27984/ [16:39:54] 10Phabricator, 10Patch-For-Review: Evaluate adding "In progress" status to Phabricator. - https://phabricator.wikimedia.org/T288956 (10mmodell) Ok ^ you can see what it looks like above. [16:42:50] 10Phabricator, 10Patch-For-Review: Evaluate adding "In progress" status to Phabricator. - https://phabricator.wikimedia.org/T288956 (10mmodell) 05progress→03Open [16:43:30] 10Phabricator, 10Patch-For-Review: Evaluate adding "In progress" status to Phabricator. - https://phabricator.wikimedia.org/T288956 (10mmodell) 05Open→03progress p:05Triage→03Medium [16:45:38] 10Phabricator, 10Patch-For-Review: Evaluate adding "In progress" status to Phabricator. - https://phabricator.wikimedia.org/T288956 (10mmodell) I don't think it makes sense that in-progress tasks would "need triage." therefore I added the above herald rule: If setting a task to in progress without setting the... [16:48:00] 10Release-Engineering-Team (Radar), 10MW-on-K8s, 10SRE, 10serviceops, 10Patch-For-Review: The restricted/mediawiki-webserver image should include skins and resources - https://phabricator.wikimedia.org/T285232 (10mmodell) >>! In T285232#7355412, @Joe wrote: > at the end of the day it seems to me that we... [16:48:26] 10Phabricator, 10Patch-For-Review: Evaluate adding "In progress" status to Phabricator. - https://phabricator.wikimedia.org/T288956 (10DannyS712) Maybe we should notify wikitech-l about this? I would tag as user-notice but this isn't for tech news [16:49:33] 10Phabricator, 10Patch-For-Review: Evaluate adding "In progress" status to Phabricator. - https://phabricator.wikimedia.org/T288956 (10mmodell) @DannyS712: sure, I'll write a quick notice to the list. [16:49:58] 10Phabricator, 10Patch-For-Review: Evaluate adding "In progress" status to Phabricator. - https://phabricator.wikimedia.org/T288956 (10Majavah) >>! In T288956#7355962, @mmodell wrote: > I don't think it makes sense that in-progress tasks would "need triage." therefore I added the above herald rule: If setting... [16:51:49] 10Phabricator, 10Patch-For-Review: Evaluate adding "In progress" status to Phabricator. - https://phabricator.wikimedia.org/T288956 (10mmodell) hmmm, well needs triage is the highest possible priority besides unbreak now. Medium is the same as "no prioritization" in my mind, but maybe others think of it diffe... [16:53:11] (03PS2) 10Ahmon Dancy: RELEASE.md updates [tools/scap] - 10https://gerrit.wikimedia.org/r/721351 (https://phabricator.wikimedia.org/T277793) [16:54:34] (03CR) 1020after4: "I'm a little confused by this change. Does the readme change correspond to a difference in the process as it seems to delete a bunch of e" [tools/scap] - 10https://gerrit.wikimedia.org/r/721351 (https://phabricator.wikimedia.org/T277793) (owner: 10Ahmon Dancy) [16:56:29] 10Phabricator, 10Patch-For-Review: Evaluate adding "In progress" status to Phabricator. - https://phabricator.wikimedia.org/T288956 (10Majavah) >>! In T288956#7355978, @mmodell wrote: > hmmm, well needs triage is the highest possible priority besides unbreak now. Medium is the same as "no prioritization" in m... [16:59:04] (03PS3) 10Ahmon Dancy: Simplify RELEASE.md steps [tools/scap] - 10https://gerrit.wikimedia.org/r/721351 (https://phabricator.wikimedia.org/T277793) [17:00:38] (03CR) 10Ahmon Dancy: Simplify RELEASE.md steps (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/721351 (https://phabricator.wikimedia.org/T277793) (owner: 10Ahmon Dancy) [17:05:13] 10Phabricator, 10Patch-For-Review: Evaluate adding "In progress" status to Phabricator. - https://phabricator.wikimedia.org/T288956 (10mmodell) I've disabled the herald rule. [17:05:42] 10Phabricator, 10Patch-For-Review: Evaluate adding "In progress" status to Phabricator. - https://phabricator.wikimedia.org/T288956 (10mmodell) >>! In T288956#7355968, @DannyS712 wrote: > Maybe we should notify wikitech-l about this? I would tag as user-notice but this isn't for tech news Email sent. [17:07:14] (03CR) 1020after4: [C: 03+1] "I like simplification!" [tools/scap] - 10https://gerrit.wikimedia.org/r/721351 (https://phabricator.wikimedia.org/T277793) (owner: 10Ahmon Dancy) [17:18:15] Yippee, build fixed! [17:18:16] Project mediawiki-core-doxygen-docker build #27985: 09FIXED in 13 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/27985/ [17:26:43] (03PS1) 10Dduvall: test: validation [blubber] - 10https://gerrit.wikimedia.org/r/721360 [17:27:40] (03CR) 10jerkins-bot: [V: 04-1] test: validation [blubber] - 10https://gerrit.wikimedia.org/r/721360 (owner: 10Dduvall) [17:31:33] I think that mediawiki-core-doxygen-docker times out / is slow because it transfers huge png files [17:31:56] as a result of generating the caller / callees graphs for hooks [17:32:03] :/ [17:32:11] eventually it ends up passing [17:36:20] (03Abandoned) 10Dduvall: test: validation [blubber] - 10https://gerrit.wikimedia.org/r/721360 (owner: 10Dduvall) [17:50:11] 10Continuous-Integration-Infrastructure, 10Patch-For-Review: Add Python 3.9 to Wikimedia CI - https://phabricator.wikimedia.org/T289222 (10Legoktm) p:05Triage→03High Can we keep the discussion in one place please? Either Gerrit or Phab, not both. On Gerrit I wrote: > I not have the capacity to backport Py... [17:50:15] 10Release-Engineering-Team (Doing), 10GitLab, 10User-brennen: Early adoption signup for WMF GitLab - https://phabricator.wikimedia.org/T282842 (10Ostrzyciel) I would be very much interested in participating with the #mediauploader extension. [18:24:08] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.38.0-wmf.1 deployment blockers - https://phabricator.wikimedia.org/T281165 (10thcipriani) [18:58:48] 10Continuous-Integration-Config, 10MW-1.37-notes (1.37.0-wmf.20; 2021-08-23), 10Patch-For-Review: Introduce non-voting jobs with quibble+apache - https://phabricator.wikimedia.org/T276428 (10kostajh) >>! In T276428#7313240, @dancy wrote: > Can someone help diagnose (and ideally fix) this error: > https://int... [19:14:01] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T281164 (10mmodell) [19:18:06] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T281164 (10hashar) [20:44:25] dduvall or jeena: ever since I started using the fancy blubber pipeline for CI I get test failures about half the time that look like this: https://integration.wikimedia.org/ci/job/analytics-quarry-web-pipeline-test/63/console [20:44:54] The first couple of times I figured it was some kind of upstream provider hiccup but now I wonder if one of your workers is broken... is that happening to everyone? [21:03:50] andrewbogott: hmm, this looks to be the relevant part [21:04:03] https://www.irccloud.com/pastebin/kXBy271D [21:04:12] yep, that's the part that breaks [21:04:17] if I 'recheck' it'll most likely just pass [21:04:52] Maybe the debian mirror is down for half the day every day but if that were true I wouldn't think I'm the first to notice [21:05:10] not sure why that would be but perhaps some kind of timeout on the remote end. we'd have to look into logs for the mirrors to be sure [21:05:35] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T281164 (10Urbanecm) [21:05:42] i guess it's possible it's on the wmcs network side as well [21:06:02] But i kinda doubt it's specific to pipelinelib usage [21:07:30] if you file a bug we can look further and/or pass it off [21:07:40] dduvall: It's not really causing me any trouble but I can open a bug if you'd like me to [21:11:50] 10Release-Engineering-Team, 10Release Pipeline (Blubber), 10cloud-services-team (Kanban): Intermittent package download failures during jenkins/pipeline tests - https://phabricator.wikimedia.org/T291135 (10Andrew) [21:11:50] https://phabricator.wikimedia.org/T291135 [21:18:46] andrewbogott: ty! [21:22:48] dduvall: can blubber generate something like 'EXPOSE 5000' ? [21:25:18] it cannot [21:25:44] but can't that be set up at runtime anyway? [21:26:41] I always thought that was a weird dockerfile instruction for that reason. Sorta outside the concern of the image [21:28:15] ok -- right now I'm just blindly trying to produce a file like my sample but I can test and see if it's needed [22:08:32] 10Continuous-Integration-Config, 10Toolforge, 10cloud-services-team (Kanban): Set up CI for cloud/toolforge/buildpacks repository - https://phabricator.wikimedia.org/T265685 (10dduvall) >>! In T265685#7355850, @Bstorm wrote: > I've done this in toolsbeta by using tekton pipelines. The way it works is using a... [22:14:11] !log Cherry-picking https://gerrit.wikimedia.org/r/c/analytics/statsv/+/721044 and https://gerrit.wikimedia.org/r/c/operations/puppet/+/721047 in deployment-prep; should only affect deployment-webperf11. [22:14:15] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:33:19] 10Continuous-Integration-Config, 10Toolforge, 10cloud-services-team (Kanban): Set up CI for cloud/toolforge/buildpacks repository - https://phabricator.wikimedia.org/T265685 (10Bstorm) That version also did require a few docker capacities (also FYI) https://github.com/toolforge/buildservice/blob/main/admin-m... [22:38:38] 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10MW-on-K8s, 10MediaWiki Train Development Environment, 10Release Pipeline, 10User-jeena (Sprint 8/18-8/31): Train-dev: Ability to deploy to k8s - https://phabricator.wikimedia.org/T287993 (10jeena) 05Open→03In progress [22:38:40] 10Release-Engineering-Team (Doing), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url command - https://phabricator.wikimedia.org/T287042 (10jeena) [22:59:17] !log Un-cherry-picked https://gerrit.wikimedia.org/r/c/operations/puppet/+/721047 in deployment-prep for now; more work needed. [22:59:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL