[00:18:50] 10Continuous-Integration-Infrastructure, 10Projects-Cleanup, 10VideoCutTool, 10User-Jayprakash12345: Delete deprecated code repositories and tools - https://phabricator.wikimedia.org/T310726 (10bd808) @Gopavasanth see https://phabricator.wikimedia.org/project/view/3975/ for instructions on requesting tool... [08:23:47] (03PS1) 10Samwilson: Add Foreground skin [integration/config] - 10https://gerrit.wikimedia.org/r/831033 (https://phabricator.wikimedia.org/T317395) [08:45:04] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression since September 6 2022 - https://phabricator.wikimedia.org/T317187 (10kostajh) [08:54:09] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression since September 6 2022 - https://phabricator.wikimedia.org/T317187 (10kostajh) >>! In T317187#8223592, @tstarlin... [08:54:37] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression since September 6 2022 - https://phabricator.wikimedia.org/T317187 (10kostajh) [10:42:34] 10Gerrit, 10Release-Engineering-Team: Make Gerrit deployable with scap on the devtools WMCS project - https://phabricator.wikimedia.org/T317404 (10hashar) [10:45:35] (03PS1) 10Hashar: Add deployment configuration for devtools WMCS project [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/831063 (https://phabricator.wikimedia.org/T317404) [10:46:52] (03CR) 10Hashar: "Looks like that does the right thing on deploy-1004.devtools.eqiad1.wikimedia.cloud:" [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/831063 (https://phabricator.wikimedia.org/T317404) (owner: 10Hashar) [10:49:58] !log devtools: fixed fqdn of instances puppetmaster-1001 and gerrit-prod-1001 by manually editing `/etc/hosts` # T317404 [10:50:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:50:01] T317404: Make Gerrit deployable with scap on the devtools WMCS project - https://phabricator.wikimedia.org/T317404 [10:50:29] 10Gerrit, 10Release-Engineering-Team, 10Patch-For-Review: Make Gerrit deployable with scap on the devtools WMCS project - https://phabricator.wikimedia.org/T317404 (10hashar) [11:20:03] 10Gerrit, 10Release-Engineering-Team, 10Patch-For-Review: Make Gerrit deployable with scap on the devtools WMCS project - https://phabricator.wikimedia.org/T317404 (10hashar) I have generated a local ssh keypair for Gerrit [[ https://wikitech.wikimedia.org/wiki/Keyholder | following the wikitech documentatio... [11:41:41] 10Gerrit, 10Release-Engineering-Team, 10Patch-For-Review: Make Gerrit deployable with scap on the devtools WMCS project - https://phabricator.wikimedia.org/T317404 (10hashar) I had to remove `profile::keyholder::server::agents` from https://horizon.wikimedia.org/project/puppet/ since that was overriding the... [12:14:45] hashar: is it easy for you to have a look at gerrit logs (or is there someone else I could ask)? [12:15:01] kostajh: they should all be in logstash :) [12:15:04] ah [12:15:05] k [12:15:18] https://gerrit.wikimedia.org/r/c/wikimedia/+/831079 made my team's dashboard disappear, and gerrit just says 404 not found [12:15:24] there is a Gerrit link on the home page [12:15:29] yep [12:15:30] https://logstash.wikimedia.org/app/dashboards#/view/AW1f-0k0ZKA7RpirlnKV [12:15:40] NOTICE level is filtered out by default [12:15:48] and there is log for both the primary and the replica gerrit [12:16:27] hmm, I don't see anything relevant in the log [12:16:40] I'll experiment with some things [12:17:30] Cannot parse dashboard wikimedia:refs/meta/dashboards/teams:growth: Bad section entry: section [CONTEXT project="wikimedia" ] [12:17:56] oh [12:17:59] there is a missing space! [12:18:08] [section"Community [12:18:12] ahh [12:18:17] my cursor was just sitting there [12:18:18] there should be a space after `[section` [12:18:18] ;) [12:18:21] well spotted [12:18:23] yeah I hate those [12:18:26] to be fair [12:18:45] Gerrit should definitely validate the config as part of submitting the change [12:18:52] and bails out when it can't be parsed [12:19:24] it works https://gerrit.wikimedia.org/r/p/wikimedia/+/dashboard/teams:growth [12:19:25] thanks! [12:24:30] 10GitLab (Project Migration), 10Quarry, 10Patch-For-Review: Move quarry to gitlab or github - https://phabricator.wikimedia.org/T308978 (10rook) a:03rook [12:27:11] 10GitLab (Infrastructure), 10Data-Persistence-Backup, 10serviceops, 10serviceops-collab, and 2 others: Backups for GitLab - https://phabricator.wikimedia.org/T274463 (10Jelto) Backups on production `gitlab1004` fail with ` Errno::EACCES: Permission denied @ dir_s_mkdir - /srv/gitlab-backup/db ` since Sep... [12:30:47] 10GitLab (Infrastructure), 10Data-Persistence-Backup, 10serviceops, 10serviceops-collab, and 2 others: Backups for GitLab - https://phabricator.wikimedia.org/T274463 (10jbond) @jelto i think this is fall out from https://gerrit.wikimedia.org/r/c/operations/puppet/+/809095 which changed the permission of t... [12:38:36] 10GitLab (Infrastructure), 10Data-Persistence-Backup, 10serviceops, 10serviceops-collab, and 2 others: Backups for GitLab - https://phabricator.wikimedia.org/T274463 (10Jelto) @jbond thanks for the context! I also found the following line in the puppet log: ` Sep 7 11:42:31 gitlab1004 puppet-agent[93484... [12:39:57] 10GitLab (Infrastructure), 10Data-Persistence-Backup, 10serviceops, 10serviceops-collab, and 2 others: Backups for GitLab - https://phabricator.wikimedia.org/T274463 (10jbond) > I guess this permission was automatically setup when installing/bootstrapping GitLab apt package for the first time. yes i imagi... [12:49:12] 10GitLab (Project Migration), 10Quarry, 10Patch-For-Review: Move Quarry from Gerrit to GitHub - https://phabricator.wikimedia.org/T308978 (10Aklapper) [13:03:57] kostajh: that is a nice dashboard ! [13:05:26] 10Gerrit, 10Release-Engineering-Team, 10Patch-For-Review: Make Gerrit deployable with scap on the devtools WMCS project - https://phabricator.wikimedia.org/T317404 (10hashar) 05Open→03Resolved a:03hashar scap / keyholder are set up. I have managed to do a deployment. I can now look at testing the autom... [13:08:24] (03CR) 10Hashar: [C: 03+2] "Tested it and it works!" [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/831063 (https://phabricator.wikimedia.org/T317404) (owner: 10Hashar) [13:09:11] (03Merged) 10jenkins-bot: Add deployment configuration for devtools WMCS project [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/831063 (https://phabricator.wikimedia.org/T317404) (owner: 10Hashar) [13:22:43] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos, 10Patch-For-Review: Failed to convert audio to MP3: /bin/bash: /usr/bin/lame: No such file or directory on Beta Cluster - https://phabricator.wikimedia.org/T317128 (10Clement_Goubert) {F35514869} Following installation of the... [13:23:48] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos, 10Patch-For-Review: Failed to convert audio to MP3: /bin/bash: /usr/bin/lame: No such file or directory on Beta Cluster - https://phabricator.wikimedia.org/T317128 (10Clement_Goubert) It seems we're falling back on this issue h... [13:38:31] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Failed to convert audio to MP3: /bin/bash: /usr/bin/lame: No such file or directory on Beta Cluster - https://phabricator.wikimedia.org/T317128 (10TheresNoTime) 05Open→03Resolved Resolving this as ` samtar@deployment-mediawik... [13:39:08] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10Data-Persistence (Consultation), 10MediaWiki-extensions-Phonos, 10Patch-For-Review: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10TheresNoTime) [13:39:20] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Failed to convert audio to MP3: /bin/bash: /usr/bin/lame: No such file or directory on Beta Cluster - https://phabricator.wikimedia.org/T317128 (10TheresNoTime) [13:40:35] Probing for some support/pointers on T317195 — `Failed to create storage directory Could not create directory "mwstore://phonos-backend/phonos"` [13:40:35] T317195: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 [13:43:37] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10Data-Persistence (Consultation), 10MediaWiki-extensions-Phonos, 10Patch-For-Review: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10TheresNoTime) Apologies for the random subscribe @Legoktm, but you're he... [13:55:02] (03PS1) 10Hashar: scap: automatize plugins handling [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/831093 (https://phabricator.wikimedia.org/T317412) [14:05:35] 10Gerrit, 10Release-Engineering-Team, 10Scap: Automate Gerrit deployment steps - https://phabricator.wikimedia.org/T317412 (10hashar) [14:22:17] (03CR) 10Hashar: [C: 03+2] Add Foreground skin [integration/config] - 10https://gerrit.wikimedia.org/r/831033 (https://phabricator.wikimedia.org/T317395) (owner: 10Samwilson) [14:23:24] 10GitLab (Project Migration), 10Quarry: Move Quarry from Gerrit to GitHub - https://phabricator.wikimedia.org/T308978 (10rook) [14:24:20] (03Merged) 10jenkins-bot: Add Foreground skin [integration/config] - 10https://gerrit.wikimedia.org/r/831033 (https://phabricator.wikimedia.org/T317395) (owner: 10Samwilson) [14:27:48] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10Data-Persistence (Consultation), 10MediaWiki-extensions-Phonos, 10Patch-For-Review: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10TheresNoTime) Confirmed `global-multiwrite` is the correct backend here [14:30:56] (03CR) 10Hashar: [C: 03+2] "Deployed!" [integration/config] - 10https://gerrit.wikimedia.org/r/831033 (https://phabricator.wikimedia.org/T317395) (owner: 10Samwilson) [14:31:48] so I felt like I could use scap to drive Gerrit deployment and it turns out to be a long tail of tasks :-\ [14:31:55] so that will be for next week I guess [14:40:59] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Phonos links to unroutable domain/URL for the MP3 file - https://phabricator.wikimedia.org/T317417 (10TheresNoTime) [14:43:05] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10Data-Persistence (Consultation), 10MediaWiki-extensions-Phonos, 10Patch-For-Review: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10TheresNoTime) 05Open→03Resolved This issue resolved, {T317417} is th... [14:44:31] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team: Upgrade Jenkins to latest LTS 2.361.1 - https://phabricator.wikimedia.org/T317418 (10hashar) [14:46:07] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-Phonos, 10Community-Tech (CommTech-Sprint-32): Phonos links to unroutable domain/URL for the MP3 file - https://phabricator.wikimedia.org/T317417 (10TheresNoTime) [15:24:06] hashar: do you not normally use scap for gerrit? [15:37:44] 10GitLab (Project Migration), 10Quarry: Move Quarry from Gerrit to GitHub - https://phabricator.wikimedia.org/T308978 (10rook) 05Open→03Resolved [15:49:55] thcipriani: we do :D [15:50:27] I thought that `scap` could deal with all the manual step we have to run and listed at https://wikitech.wikimedia.org/wiki/Gerrit/Upgrade#Deploying [15:50:37] and this afternoon I discovered Phabricator does that as well [15:50:48] FYI: I'm going to apply some schema changes to the beta cluster in 15-30 minutes. Will !log when I actually do. [15:50:58] using a privileged user that can run puppet / restart the service etc [15:51:26] while for Gerrit scap deploys using the same user used for running the service (`gerrit2`) [15:51:38] at lesat I have a dev environment on WMCS now! [15:51:48] Daimona: good luck! [15:52:07] Heh :) [16:14:56] ah, yes, I always do a manual restart...for some reason [16:15:32] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-Phonos, 10Community-Tech (CommTech-Sprint-32): Phonos links to unroutable domain/URL for the MP3 file - https://phabricator.wikimedia.org/T317417 (10Legoktm) >>! In T317195#8217912, @MusikAnimal wrote: > I suspect the issue is at [[ https://github.com/wik... [16:18:08] legoktm: I pointed that out on Slack a few hours ago :D [16:18:32] :v no way I could know that [16:18:51] I know [16:19:09] I saw how TMH was using it (cli parameter to ffmpeg) and was like, yeah, that cannae be right :) [16:21:26] 10GitLab (Infrastructure), 10Data-Persistence-Backup, 10serviceops, 10serviceops-collab, and 2 others: Backups for GitLab - https://phabricator.wikimedia.org/T274463 (10Jelto) Backups on production `gitlab1004` are fixed again with https://gerrit.wikimedia.org/r/831083. The puppet run reported ` Notice:... [16:36:13] !log Applying schema change to the wikishared DB on beta for the CampaignEvents extension (1/2) # T316409 [16:36:15] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:36:15] T316409: Re-implement tracking tools schema - https://phabricator.wikimedia.org/T316409 [16:42:11] (03PS2) 10Hashar: scap: automatize plugins handling [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/831093 (https://phabricator.wikimedia.org/T317412) [16:56:59] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-Phonos, 10Community-Tech (CommTech-Sprint-32): Phonos links to unroutable domain/URL for the MP3 file - https://phabricator.wikimedia.org/T317417 (10MusikAnimal) a:03MusikAnimal [17:08:20] !log Applying schema change to the wikishared DB on beta for the CampaignEvents extension (2/2) # T316409 [17:08:23] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:08:24] T316409: Re-implement tracking tools schema - https://phabricator.wikimedia.org/T316409 [17:25:45] !log Applying schema change to the wikishared DB on beta for the CampaignEvents extension (1/2) # T311126 [17:25:47] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:25:47] T311126: Implement strategy to account for DST variations - https://phabricator.wikimedia.org/T311126 [17:46:51] !log Applying schema change to the wikishared DB on beta for the CampaignEvents extension (2/2) # T311126 [17:46:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:46:54] T311126: Implement strategy to account for DST variations - https://phabricator.wikimedia.org/T311126 [18:52:04] 10Gerrit, 10Release-Engineering-Team: Make Gerrit deployable with scap on the devtools WMCS project - https://phabricator.wikimedia.org/T317404 (10Dzahn) It's great to hear this works in devtools. Probably unrelated, but we had some failed puppet alerts on the deployment server there. I'll take a look what t... [19:06:35] (03CR) 10Ahmon Dancy: "Sorry for the delay in reviewing." [tools/scap] - 10https://gerrit.wikimedia.org/r/829249 (https://phabricator.wikimedia.org/T287046) (owner: 10Jeena Huneidi) [19:07:25] ^demon: what is the best way to do an SVN to git migration with a very badly messed up SVN repo that didn't always have a trunk as release was just used and has amazingly broken file permissions and was randomly moved around without preserving history properly [19:07:38] does anyone still have the knowledge how the svn repos were imported into gerrit ?:) [19:08:00] I just logged into rt.wikimedia.org to search for it [19:08:00] <^demon> Ouch. Um. [19:08:10] and then I suggested to download the static-Bugzilla dump [19:08:12] to find it:) [19:08:12] <^demon> There was a tool... [19:09:00] <^demon> I know we used git-filter-branch to convert the mega-repo we got from the blind conversion...but there was another tool I forgot. [19:09:18] ^demon: in my day job I've asked to do an svn2git migration for a really messed up pile of tech debt [19:09:27] <^demon> There's probably better tools by now. Git-filter-branch is definitely slow as hell [19:10:33] <^demon> https://github.com/nirvdrum/svn2git I believe I used this, or some old variation of it [19:12:26] <^demon> RhinosF1: if the history is broken then I would question if it's worth the time to try and reconstruct the history...it's going to be a very labor intensive process. Idk your job situation but is it something you can push back on? [19:12:44] ^demon: oh ye, there's the option of saying it's likely impossible [19:12:53] I need to see how badly it's messed up [19:13:06] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review, 10User-brennen: Deploy buildkitd to trusted GitLab runners - https://phabricator.wikimedia.org/T308271 (10dduvall) Yesterday I circled back to this task to verify that building a basic image via buildkitd and... [19:13:18] But I think someone did something messy restructuring stuff [19:13:37] it's one guy who already said "if not I will just make it readonly". big +1 on pushing back. just point out how you did all this research and how it's NOT trivial because they messed it up in the past before you were even there :) [19:13:55] And the last person to try this gave up because it was getting lost and the git log was very broken apparently [19:14:03] But I'm the new software guy [19:14:16] Whose got fresh eyes to look at it [19:14:44] when they try to find "easy" tasks for new people that are actually not easy at all and the ones they never got to [19:15:29] <^demon> If the history is already a mess then I think short of manually reconstructing history you're up the creek. And as the new guy, are you the best person to manually piece together a history you weren't there for? [19:16:17] <^demon> I don't think there's an easy way out of this without having more organizational context -- that you obviously don't have. I'd take that back to your supervisor. [19:16:30] My boss is nice [19:16:37] He couldn't care less [19:16:40] <^demon> Hah [19:18:13] <^demon> I wish you luck on this endeavor -- it's certainly not one I'm eager to attempt ever again. But yeah, svn2git and git-filter-branch are what I used, to answer your original question. [19:18:18] It was more of a do you have any ideas [19:18:47] So I said I'll find some docs and have a go and if it takes too long we'll scrap it [19:19:18] So I'm going to run svn2git and if it takes either longer than a day or how long until I have headache, whichever is less [19:19:35] I'm saying blame Chad and Daniel and we'll move on [19:22:01] tell him all the details how you researched..we looked through Wikimedia's ancient ticket systems and you found the guy who did that [19:22:08] then at the end .. that it's not worth it :) [19:38:58] Good luck RhinosF1! [20:15:52] (03PS5) 10Jeena Huneidi: Scap backport --revert: improve commit message [tools/scap] - 10https://gerrit.wikimedia.org/r/829249 (https://phabricator.wikimedia.org/T287046) [20:16:06] (03CR) 10Jeena Huneidi: Scap backport --revert: improve commit message (037 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/829249 (https://phabricator.wikimedia.org/T287046) (owner: 10Jeena Huneidi) [20:18:26] (03CR) 10Ahmon Dancy: Scap backport --revert: improve commit message (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/829249 (https://phabricator.wikimedia.org/T287046) (owner: 10Jeena Huneidi) [20:32:05] (03CR) 10Ahmon Dancy: [C: 03+2] "Works great and looks great!" [tools/scap] - 10https://gerrit.wikimedia.org/r/829249 (https://phabricator.wikimedia.org/T287046) (owner: 10Jeena Huneidi) [20:34:31] (03PS1) 10Jeena Huneidi: scap backport: Make internal methods "private" [tools/scap] - 10https://gerrit.wikimedia.org/r/831145 [20:36:43] (03Merged) 10jenkins-bot: Scap backport --revert: improve commit message [tools/scap] - 10https://gerrit.wikimedia.org/r/829249 (https://phabricator.wikimedia.org/T287046) (owner: 10Jeena Huneidi) [20:37:20] (03CR) 10CI reject: [V: 04-1] scap backport: Make internal methods "private" [tools/scap] - 10https://gerrit.wikimedia.org/r/831145 (owner: 10Jeena Huneidi) [20:44:18] (03PS2) 10Jeena Huneidi: scap backport: Make internal methods "private" [tools/scap] - 10https://gerrit.wikimedia.org/r/831145 [20:46:59] 10Release-Engineering-Team (Priority Backlog 📥), 10MW-on-K8s, 10Release Pipeline, 10Patch-For-Review, 10User-brennen: scap backport --revert command - https://phabricator.wikimedia.org/T287046 (10jeena) 05Open→03Resolved [20:47:01] 10Release-Engineering-Team (Priority Backlog 📥), 10MW-on-K8s, 10Release Pipeline, 10Patch-For-Review, 10User-brennen: Design m8s deployment workflows and tooling - https://phabricator.wikimedia.org/T279322 (10jeena) [20:47:52] (03CR) 10CI reject: [V: 04-1] scap backport: Make internal methods "private" [tools/scap] - 10https://gerrit.wikimedia.org/r/831145 (owner: 10Jeena Huneidi) [20:49:32] (03PS3) 10Jeena Huneidi: scap backport: Make internal methods "private" [tools/scap] - 10https://gerrit.wikimedia.org/r/831145 [21:31:39] 10Continuous-Integration-Infrastructure, 10Doxygen, 10Upstream: Doxygen for MediaWiki PHP core: the PHP doc block for Linker::link() does not give a link to the FQCN of MediaWiki\Linker\LinkRenderer, only MediaWiki\Linker - https://phabricator.wikimedia.org/T265853 (10Krinkle) Upstream issue: Is it just me or is anyone else having "Connection loss" errors in their browser when submitting comments on Phabricator? [21:47:31] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MediaWiki-Documentation: MediaWiki core docs unavailable for MW 1.35 and later - https://phabricator.wikimedia.org/T317451 (10Krinkle) [21:47:58] (03PS1) 10Hashar: Support checks running before a stage [tools/scap] - 10https://gerrit.wikimedia.org/r/831152 [21:48:10] I have absolutely no idea what I am doing with ^ [21:51:56] (03CR) 10CI reject: [V: 04-1] Support checks running before a stage [tools/scap] - 10https://gerrit.wikimedia.org/r/831152 (owner: 10Hashar) [21:56:01] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MediaWiki-Documentation: MediaWiki core docs unavailable for MW 1.35 and later - https://phabricator.wikimedia.org/T317451 (10Krinkle) [21:56:52] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MediaWiki-Documentation: MediaWiki core docs unavailable for MW 1.35 and later - https://phabricator.wikimedia.org/T317451 (10Krinkle) Per upstream this is most likely due to an excessively large s... [22:08:16] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MediaWiki-Documentation: MediaWiki core docs unavailable for MW 1.35 and later - https://phabricator.wikimedia.org/T317451 (10Krinkle) Interestingly, the REL branch builds are unaffected. If the presence of extension submodules played a role, I w... [22:13:21] Krinkle: I think you miss the name of the excessively large that is somewhere :P [22:21:36] Platonides: ack, added `file` to the sentence. thx [22:23:02] i was expecting it to be a bit more interesting :D [22:25:46] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MediaWiki-Documentation: MediaWiki core docs unavailable for MW 1.35 and later - https://phabricator.wikimedia.org/T317451 (10Krinkle) The Doxygen version currently deployed in CI (Doxygen 1.9.1) unfortunately does not make mention of the file t... [22:27:32] Platonides: getting there [22:27:49] I'll also patch Doxyfile to exclude vendor and node_modules subdirs [22:27:53] taking foreeeever otherwise [22:35:51] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review, 10User-brennen: Deploy buildkitd to trusted GitLab runners - https://phabricator.wikimedia.org/T308271 (10dduvall) I ran an even more minimal test to see if I could at least build a scratch image containing a... [22:38:29] I got phab1004 "fixed" insofar that the "scap is not functional" part is gone [22:38:48] now it's only failing because of "normal" "has not been deployed yet" [22:39:08] this is when I stopped trying to use systemd::sysuser to create phd [22:39:16] just for debugging / to confirm it's related [22:39:50] maybe deployment works now though because phd and /srv/phab exist [22:42:05] 10GitLab (Project Migration), 10Quarry: Move Quarry from Gerrit to GitHub - https://phabricator.wikimedia.org/T308978 (10rook) [22:43:26] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review, 10User-brennen: Deploy buildkitd to trusted GitLab runners - https://phabricator.wikimedia.org/T308271 (10dduvall) Also cc'ing task {T317341} in case this is related to any changes made on account of security... [23:08:06] (03PS1) 10Krinkle: jjb: Skip submodules in 'mediawiki-core-doxygen-docker-publish' [integration/config] - 10https://gerrit.wikimedia.org/r/831161 (https://phabricator.wikimedia.org/T317451) [23:10:28] Double negatives aren't confusing, unless they're misunderstood. [23:10:39] GIT_NO_SUBMODULES=yes [23:13:04] (03Abandoned) 10Krinkle: jjb: Skip submodules in 'mediawiki-core-doxygen-docker-publish' [integration/config] - 10https://gerrit.wikimedia.org/r/831161 (https://phabricator.wikimedia.org/T317451) (owner: 10Krinkle) [23:19:48] (03CR) 10Samwilson: Add Foreground skin (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/831033 (https://phabricator.wikimedia.org/T317395) (owner: 10Samwilson) [23:25:14] jnuche: dancy: Might be of interest - manually re-triggering a CI build that normally runs in response to a Git release tag being pushed. In case you haven't seen that before - can come in handy at times. https://phabricator.wikimedia.org/T317451 [23:26:49] ah, I documented it last time it came up; https://www.mediawiki.org/w/index.php?title=Continuous_integration/Zuul&diff=3808304&oldid=3747697