[00:40:05] hrm, I wonder if the "provider scap3 is not funcational" refers to the provider's "commands": /usr/bin/scap and /usr/bin/git -- I don't see /usr/bin/scap there [01:04:16] yea, so there is an entire wall of "failed due to dependency" that probably also prevented scap from being installed.. [01:04:35] we'll have to continue debugging more of that tomorrow [01:17:11] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MediaWiki-Stakeholders-Group, 10Epic, 10MW-1.40-release: Expand the set of bundled extensions and skins in MediaWiki 1.40 - https://phabricator.wikimedia.org/T317146 (10Reedy) [01:17:15] 10MediaWiki-Releasing, 10MediaWiki-extensions-LoginNotify, 10MW-1.39-release: Bundle LoginNotify extension with MediaWiki - https://phabricator.wikimedia.org/T191746 (10Reedy) [01:17:23] 10MediaWiki-Releasing, 10Growth-Team, 10Growth-Team-Filtering, 10Thanks, 10MW-1.39-release: Bundle Thanks extension with MediaWiki - https://phabricator.wikimedia.org/T191739 (10Reedy) [01:17:29] 10MediaWiki-Releasing, 10MW-1.39-release, 10MobileFrontend (Tracking): Bundle MobileFrontend extension with MediaWiki - https://phabricator.wikimedia.org/T191734 (10Reedy) [01:17:37] 10MediaWiki-Releasing, 10Growth-Team, 10Growth-Team-Filtering, 10Notifications, and 2 others: Bundle Echo extension with MediaWiki - https://phabricator.wikimedia.org/T191738 (10Reedy) [01:18:19] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MediaWiki-Stakeholders-Group, 10Epic, 10MW-1.40-release: Expand the set of bundled extensions and skins in MediaWiki 1.40 - https://phabricator.wikimedia.org/T317146 (10Reedy) [01:19:04] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MediaWiki-Stakeholders-Group, 10Epic, 10MW-1.40-release: Expand the set of bundled extensions and skins in MediaWiki 1.40 - https://phabricator.wikimedia.org/T317146 (10Reedy) [01:20:06] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MediaWiki-Stakeholders-Group, 10Epic, 10MW-1.40-release: Expand the set of bundled extensions and skins in MediaWiki 1.40 - https://phabricator.wikimedia.org/T317146 (10Reedy) [01:20:20] 10MediaWiki-Releasing, 10Reference Previews, 10MW-1.39-release: Bundle Popups extension with MediaWiki - https://phabricator.wikimedia.org/T231797 (10Reedy) [01:20:26] 10MediaWiki-Releasing, 10MediaWiki-extensions-OATHAuth, 10MW-1.40-release: Bundle WebAuthn extension with MediaWiki - https://phabricator.wikimedia.org/T258007 (10Reedy) [01:20:32] 10MediaWiki-Releasing, 10AntiSpoof, 10MW-1.39-release: Bundle AntiSpoof extension with MediaWiki - https://phabricator.wikimedia.org/T191736 (10Reedy) [01:20:48] 10MediaWiki-Releasing, 10Growth-Team, 10Growth-Team-Filtering, 10Notifications, and 2 others: Bundle Echo extension with MediaWiki - https://phabricator.wikimedia.org/T191738 (10MGChecker) [01:33:04] 10MediaWiki-Releasing, 10Reference Previews, 10MW-1.39-release: Bundle Popups extension with MediaWiki - https://phabricator.wikimedia.org/T231797 (10Reedy) [01:33:15] 10MediaWiki-Releasing, 10Reference Previews, 10MW-1.39-release: Bundle Popups extension with MediaWiki - https://phabricator.wikimedia.org/T231797 (10Reedy) [01:45:50] 10GitLab (Project Migration), 10Release-Engineering-Team (Priority Backlog đŸ“Ĩ), 10User-brennen: Mirror all active Gerrit repositories to GitLab instance - https://phabricator.wikimedia.org/T305984 (10bd808) >>! In T305984#8210132, @bd808 wrote: > Apparently "pull" mirroring (gitlab polling an external repo) i... [05:30:26] 10Phabricator, 10Release-Engineering-Team (Bonus Level 🕹ī¸), 10serviceops, 10serviceops-collab, 10Patch-For-Review: sort out mysql privileges for phab1004/phab2002 - https://phabricator.wikimedia.org/T315713 (10Marostegui) Excellent - thank you Daniel [06:46:48] 10GitLab (Project Migration), 10Quarry: Move quarry to gitlab or github - https://phabricator.wikimedia.org/T308978 (10Aklapper) I believe that random third-party hosting locations like GitHub should be off the table. In my understanding, everything should be on Wikimedia GitLab in the long run, instead of Wi... [06:49:17] 10GitLab (Project Migration), 10Quarry: Move quarry to gitlab or github - https://phabricator.wikimedia.org/T308978 (10Aklapper) (In any case, please disable "Issues" on https://github.com/toolforge/quarry to avoid fragmentation and duplication - thanks.) [07:10:03] good morning [08:41:21] 10Release-Engineering-Team, 10Infrastructure-Foundations, 10SRE-tools: Investigate sharing releng common python code to pywmflib - https://phabricator.wikimedia.org/T316757 (10hashar) Releng might be able to cut a tag a new release which we will then be able to use immediately by bumping the dependency in ou... [08:43:35] (03PS2) 10Jaime Nuche: k8s: push support images to Minikube registry on env startup [tools/train-dev] - 10https://gerrit.wikimedia.org/r/829825 [08:44:12] (03CR) 10Jaime Nuche: k8s: push support images to Minikube registry on env startup (031 comment) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/829825 (owner: 10Jaime Nuche) [08:53:40] 10Gerrit, 10Release-Engineering-Team (Bonus Level 🕹ī¸): gerrit: stop using git-fat - https://phabricator.wikimedia.org/T316878 (10hashar) [08:54:29] (03PS1) 10Mglaser: Fix depencency for BlueSpiceCountThings [integration/config] - 10https://gerrit.wikimedia.org/r/830568 [08:54:51] (03CR) 10Jaime Nuche: [C: 03+2] Only use ANSI colors in scap logo when stdout is a tty [tools/scap] - 10https://gerrit.wikimedia.org/r/830190 (owner: 10Ahmon Dancy) [08:58:43] 10GitLab (Infrastructure), 10serviceops: Reduce usage of public IPv4 addresses on GitLab hosts - https://phabricator.wikimedia.org/T310265 (10ayounsi) [08:59:05] (03Merged) 10jenkins-bot: Only use ANSI colors in scap logo when stdout is a tty [tools/scap] - 10https://gerrit.wikimedia.org/r/830190 (owner: 10Ahmon Dancy) [09:02:35] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Failed to convert audio to MP3: /bin/bash: /usr/bin/lame: No such file or directory on Beta Cluster - https://phabricator.wikimedia.org/T317128 (10TheresNoTime) Tagging #beta-cluster-infrastructure — //should// `lame` be available? [09:14:40] 10Phabricator, 10SRE, 10Traffic, 10Wikimedia-Incident: Phabricator was logging out users repeatedly (2022-08-26) - https://phabricator.wikimedia.org/T316337 (10hashar) I apologize for my unclear comment, I was referring to the notes taking document at https://docs.google.com/document/d/1Ka9MQB8OwdzAzJVfZua... [09:24:42] 10Phabricator, 10SRE, 10Traffic, 10Wikimedia-Incident: Phabricator was logging out users repeatedly (2022-08-26) - https://phabricator.wikimedia.org/T316337 (10jcrespo) > Does it have to be converted to an incident report on Wikitech? It does. > I could do it but could use pairing with someone familiar w... [10:02:02] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10SRE-OnFire, 10User-Ryasmeen, 10Wikimedia-Incident: Beta cluster Error: 502, Next Hop Connection Failed - https://phabricator.wikimedia.org/T315350 (10jbond) 05Open→03Resolved a:03jbond @Zabe thanks for the detailed summary, it looks like... [10:21:06] jnuche: FYI, I'm having a non-critical issue with scap pull throwing rsync errors for "cannot delete non-empty directory: php-1.39.0-wmf.26/cache/l10n" (and upwards dirs). This happens for me on the parse servers I'm putting in production, and occurs whether the host is added to a cluster or not. LMK if you need more info, a bug report in phab, or just a host to check it out. [10:45:07] (03CR) 10Matthias Mullie: "Feel free to abandon this patch. Given the current state of that repo, there's no point doing this." [integration/config] - 10https://gerrit.wikimedia.org/r/803291 (owner: 10Jforrester) [10:47:34] (03PS1) 10Matthias Mullie: Zuul: [mediawiki/extensions/ImageSuggestions] Mark as in production [integration/config] - 10https://gerrit.wikimedia.org/r/830584 (https://phabricator.wikimedia.org/T310367) [10:47:50] (03PS2) 10Matthias Mullie: Zuul: [mediawiki/extensions/SearchVue] Mark as in production [integration/config] - 10https://gerrit.wikimedia.org/r/830584 (https://phabricator.wikimedia.org/T310367) [10:48:22] (03Abandoned) 10Matthias Mullie: Zuul: [mediawiki/extensions/SearchVue] Mark as in production [integration/config] - 10https://gerrit.wikimedia.org/r/830584 (https://phabricator.wikimedia.org/T310367) (owner: 10Matthias Mullie) [10:50:16] (03PS1) 10Matthias Mullie: Start branching SearchVue for Wikimedia production [tools/release] - 10https://gerrit.wikimedia.org/r/830606 (https://phabricator.wikimedia.org/T310367) [10:50:32] (03Abandoned) 10Matthias Mullie: Start branching SearchVue for Wikimedia production [tools/release] - 10https://gerrit.wikimedia.org/r/830606 (https://phabricator.wikimedia.org/T310367) (owner: 10Matthias Mullie) [10:53:37] claime: hi, is parse1001.eqiad.wmnet for example one of those hosts? [10:54:45] jnuche: I can check, I encountered it on parse1017.eqiad.wmnet just now [10:55:25] parse1017.eqiad.wmnet also works, I'll take a look [10:56:03] jnuche: cool, leaving it to you then, thanks [10:58:05] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Failed to convert audio to MP3: /bin/bash: /usr/bin/lame: No such file or directory on Beta Cluster - https://phabricator.wikimedia.org/T317128 (10Joe) The reason lame is not installed in production appservers is that it uses a r... [11:04:20] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Failed to convert audio to MP3: /bin/bash: /usr/bin/lame: No such file or directory on Beta Cluster - https://phabricator.wikimedia.org/T317128 (10Clement_Goubert) a:03Clement_Goubert [11:23:36] (03CR) 10Jaime Nuche: "LGTM and can be merged IMO. I would like to propose an alternative though." [tools/scap] - 10https://gerrit.wikimedia.org/r/830195 (owner: 10Ahmon Dancy) [11:59:22] 10Project-Admins: Create project tag for OurWorldInData - https://phabricator.wikimedia.org/T317066 (10Peachey88) 05Open→03Resolved a:03Peachey88 Hi, requested public project #mediawiki-extensions-ourworldindata has been created: https://phabricator.wikimedia.org/project/manage/6140/ (In case you need to... [12:30:50] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: Investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10kostajh) Tentatively adding as a deployment blocker until we could figure out what is causi... [12:31:33] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10kostajh) [12:32:47] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10kostajh) [12:32:52] jnuche: Think you'll need parse1017 for long? If you do, I'll just put it at the end of the servers I'm putting in production. I don't need it immediately, just need to organize myself :) [12:36:47] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10kostajh) [12:39:31] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10kostajh) [12:39:39] 10GitLab (Project Migration), 10Quarry: Move quarry to gitlab or github - https://phabricator.wikimedia.org/T308978 (10rook) Issues disabled. > everything should be on Wikimedia GitLab in the long run I generally agree. As I mentioned in the email once gitlab has necessary features enabled quarry can move t... [12:41:47] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10kostajh) {T279664} might be related. [12:56:29] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10kostajh) It seems like ElasticSearch integration might... [12:57:02] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10kostajh) p:05Triage→03High [13:02:04] claime: sorry about that, please take parse1017 back [13:02:39] I fixed the issue on that host, I think when we deploy the train to the next group later today that should fix the issue for the rest of the hosts [13:02:40] jnuche: no problem :) [13:02:43] I'll keep an eye on it [13:03:00] Cool, thank you! [13:06:48] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10kostajh) Another possibility is that the interaction wi... [13:16:03] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10dcausse) @kostajh thanks for pinging us, we are in the... [13:25:27] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10kostajh) `Sepcial:Homepage?useCache=0` using no topic f... [13:33:42] (03CR) 10Hashar: [C: 03+2] "GRmbmlblb I should have caught it!" [integration/config] - 10https://gerrit.wikimedia.org/r/830568 (owner: 10Mglaser) [13:36:04] (03Merged) 10jenkins-bot: Fix depencency for BlueSpiceCountThings [integration/config] - 10https://gerrit.wikimedia.org/r/830568 (owner: 10Mglaser) [13:39:06] (03CR) 10Hashar: [C: 03+2] "Deployed!" [integration/config] - 10https://gerrit.wikimedia.org/r/830568 (owner: 10Mglaser) [13:55:16] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10TheresNoTime) [13:59:25] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10TheresNoTime) {T317130} would have caught this, apologies :/ [14:08:30] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10TheresNoTime) https://www.mediawiki.org/wiki/Topic:Shsy2ih2cft90uc8 seems related — resolved for them by: > I'd say your $wgUpload... [14:27:32] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10MusikAnimal) >>! In T317195#8217591, @TheresNoTime wrote: > â€Ļ > I recall a Phonos-specific setting which relates to $wgUploadDirect... [14:28:44] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10TheresNoTime) I note https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/543217 which has: ` 'wgScoreFileBackend' => [... [14:31:51] 10Phabricator, 10Release-Engineering-Team (Doing), 10Dumps-Generation, 10SecTeam-Processed, and 2 others: Public Phabricator dump includes restricted project columns - https://phabricator.wikimedia.org/T309757 (10sbassett) [14:33:14] 10Phabricator, 10Release-Engineering-Team (Doing), 10Dumps-Generation, 10SecTeam-Processed, and 2 others: Public Phabricator dump includes restricted project columns - https://phabricator.wikimedia.org/T309757 (10sbassett) >>! In T309757#8216798, @hashar wrote: > @sbassett we can indeed mark this task reso... [14:37:38] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10TheresNoTime) a:03TheresNoTime [14:51:45] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10TheresNoTime) [14:56:11] (03CR) 10Ahmon Dancy: [C: 03+2] "Verified in train-dev" [tools/train-dev] - 10https://gerrit.wikimedia.org/r/829825 (owner: 10Jaime Nuche) [14:56:44] (03Merged) 10jenkins-bot: k8s: push support images to Minikube registry on env startup [tools/train-dev] - 10https://gerrit.wikimedia.org/r/829825 (owner: 10Jaime Nuche) [14:58:49] 10Release-Engineering-Team, 10Scap: Scap backport should include the deployers username on the Gerrit approval comment - https://phabricator.wikimedia.org/T317163 (10dancy) [15:05:55] !log making hack changes to beta to test T317195 resolution [15:05:57] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:05:57] T317195: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 [15:08:40] 10Phabricator, 10Release-Engineering-Team (Bonus Level 🕹ī¸), 10User-brennen: Deploy Phabricator with scap - https://phabricator.wikimedia.org/T313259 (10dduvall) [15:09:40] 10Phabricator, 10Release-Engineering-Team (Bonus Level 🕹ī¸), 10Patch-For-Review: Move Phabricator configuration into deployment repo - https://phabricator.wikimedia.org/T313950 (10dduvall) 05In progress→03Resolved [15:10:26] (03CR) 10Ahmon Dancy: [C: 03+1] Add RateLimitedProgressReporter (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/830195 (owner: 10Ahmon Dancy) [15:14:47] (03PS1) 10Hashar: Json schema from Gerrit Java event classes [software/gerrit/plugins/events-wikimedia] - 10https://gerrit.wikimedia.org/r/830654 (https://phabricator.wikimedia.org/T304947) [15:21:02] (03PS5) 10Hashar: Implement REST API and Ssh commands [software/gerrit/plugins/events-wikimedia] - 10https://gerrit.wikimedia.org/r/814725 (https://phabricator.wikimedia.org/T304947) [15:22:28] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10TheresNoTime) Hacking in ` $wgPhonosFileBackend = 'global-multiwrite'; $wgPhonosFileBackendDirectory = '//upload.wikimedia.beta.wmf... [15:26:35] (03CR) 10Hashar: "I have extracted code from the original change at https://gerrit.wikimedia.org/r/c/operations/software/gerrit/plugins/events-wikimedia/+/8" [software/gerrit/plugins/events-wikimedia] - 10https://gerrit.wikimedia.org/r/830654 (https://phabricator.wikimedia.org/T304947) (owner: 10Hashar) [15:27:38] (03CR) 10Hashar: "I have extracted some code to a parent change ( https://gerrit.wikimedia.org/r/c/operations/software/gerrit/plugins/events-wikimedia/+/830" [software/gerrit/plugins/events-wikimedia] - 10https://gerrit.wikimedia.org/r/814725 (https://phabricator.wikimedia.org/T304947) (owner: 10Hashar) [15:27:44] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10MusikAnimal) `$wgPhonosFileBackendDirectory` isn't used when `$wgPhonosFileBackend` is set, and I thought Extension:Score was progr... [15:29:59] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10TheresNoTime) I just tried setting `$wgPhonosPath = '//upload.wikimedia.beta.wmflabs.org/phonos';` too, same issue. >>! In T317195... [15:32:27] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos, 10Patch-For-Review: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10MusikAnimal) I suspect the issue is at [[ https://github.com/wikimedia/mediawiki-extensions-Phonos/blob/a9473... [15:47:05] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10Data-Persistence (Consultation), 10MediaWiki-extensions-Phonos, 10Patch-For-Review: Failed to create storage directory on Beta Cluster - https://phabricator.wikimedia.org/T317195 (10TheresNoTime) Or do we need to use [[ https://doc.wikimedia.org/mediawik... [15:51:55] 10Release-Engineering-Team, 10Scap: Scap backport should include the deployers username on the Gerrit approval comment - https://phabricator.wikimedia.org/T317163 (10dancy) Right now the +2 review message is "Approved via scap backport". How about something this instead? ` Approved by dancy@deploy1002 using s... [15:53:50] hi, not sure if this is the right place to ask, but could someone have a look at a task that I've submitted? I just want to know if it's possible or not to run an extension's CI on a release branch: https://phabricator.wikimedia.org/T316479 [15:54:58] Would anyone here know how I might be granted admin bits for the quarry repo on gerrit (https://gerrit.wikimedia.org/r/admin/repos/analytics/quarry/web) ? [15:58:56] Rook: I will add you to the list [15:59:02] Thanks! [15:59:23] Done. [16:01:29] I see access! Thanks! [16:04:08] 10Release-Engineering-Team (Bonus Level 🕹ī¸), 10Scap, 10Research: article-recommender: stop using git-fat - https://phabricator.wikimedia.org/T317212 (10demon) [16:09:41] (03CR) 10Ahmon Dancy: Show CI status while waiting for change to merge (032 comments) [tools/release] - 10https://gerrit.wikimedia.org/r/826675 (https://phabricator.wikimedia.org/T315452) (owner: 10Hashar) [16:12:07] 10Release-Engineering-Team (Bonus Level 🕹ī¸), 10Scap, 10Research, 10Patch-For-Review: article-recommender: clean up git-fat removal - https://phabricator.wikimedia.org/T317212 (10demon) [16:22:56] 10Release-Engineering-Team (Priority Backlog đŸ“Ĩ), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url command - https://phabricator.wikimedia.org/T287042 (10dancy) 05Open→03Resolved [16:23:00] 10Release-Engineering-Team (Priority Backlog đŸ“Ĩ), 10MW-on-K8s, 10Release Pipeline, 10Patch-For-Review, 10User-brennen: Design m8s deployment workflows and tooling - https://phabricator.wikimedia.org/T279322 (10dancy) [16:25:10] 10Release-Engineering-Team (Priority Backlog đŸ“Ĩ), 10MW-on-K8s, 10Release Pipeline, 10Patch-For-Review: Display mergeable status in scap backport --list output - https://phabricator.wikimedia.org/T303967 (10dancy) 05Open→03Resolved [16:25:12] 10Release-Engineering-Team (Priority Backlog đŸ“Ĩ), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url command - https://phabricator.wikimedia.org/T287042 (10dancy) [16:25:42] Hi all, I am not able to login gitlab with my Wikimedia Dev Account, receiving Sign-in using Wikimedia Dev Account auth failed [16:25:42] Sign-in failed because Email has already been taken. and my email address is wfan@wikimedia.org. Thanks :) [16:29:18] wfan: hi, we are currently in our team meeting. May you file a task in https://phabricator.wikimedia.org/ with the project `#gitlab` please? [16:30:20] that should emit a notificaton to the appropriate people and also spam the #wikimedia-gitlab IRC channel [16:30:43] and I will ping releng about it, we are currently in our weekly team meeting [16:31:26] Ok thanks! will do [16:42:50] 10GitLab: Not able to login GitLab with Wikimedia Dev Account - https://phabricator.wikimedia.org/T317218 (10AnnWF) [16:53:18] 10GitLab: Not able to login GitLab with Wikimedia Dev Account - https://phabricator.wikimedia.org/T317218 (10hashar) I can't really look it up cause it is dinner time. Through gitlab search I found https://gitlab.wikimedia.org/wfan219 In LDAP that would be https://ldap.toolforge.org/user/wfan219 And there is... [16:53:36] wfan: looks like there are two accounts sharing the same email: wfan and wfan219 [16:54:02] I imagine one of them is already in Gitlab which then reject the connection for the other account because of the dupe email [16:54:11] there is the same problem with Gerrit [16:55:04] I am off, it is dinner time [16:55:12] hmm, you mean two account name for Wikimedia Dev Account with same email address? Could you help me to delete the wfan219 then, thanks? [16:58:46] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos, 10Patch-For-Review: Failed to convert audio to MP3: /bin/bash: /usr/bin/lame: No such file or directory on Beta Cluster - https://phabricator.wikimedia.org/T317128 (10taavi) There's an instance called [[ https://openstack-brows... [17:36:59] (03CR) 10Jaime Nuche: [C: 03+2] Add RateLimitedProgressReporter (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/830195 (owner: 10Ahmon Dancy) [17:41:17] (03Merged) 10jenkins-bot: Add RateLimitedProgressReporter [tools/scap] - 10https://gerrit.wikimedia.org/r/830195 (owner: 10Ahmon Dancy) [17:46:19] (03PS1) 10Jaime Nuche: K8s: add parsing of deployments configuration file [tools/scap] - 10https://gerrit.wikimedia.org/r/830675 [17:49:49] (03CR) 10Jaime Nuche: "Tested in traindev." [tools/scap] - 10https://gerrit.wikimedia.org/r/830675 (owner: 10Jaime Nuche) [17:49:59] (03CR) 10CI reject: [V: 04-1] K8s: add parsing of deployments configuration file [tools/scap] - 10https://gerrit.wikimedia.org/r/830675 (owner: 10Jaime Nuche) [17:55:12] (03PS2) 10Jaime Nuche: K8s: add parsing of deployments configuration file [tools/scap] - 10https://gerrit.wikimedia.org/r/830675 [17:57:21] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos, 10Patch-For-Review: Failed to convert audio to MP3: /bin/bash: /usr/bin/lame: No such file or directory on Beta Cluster - https://phabricator.wikimedia.org/T317128 (10TheresNoTime) >>! In T317128#8218312, @taavi wrote: > There'... [18:02:18] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-Phonos, 10Patch-For-Review: Failed to convert audio to MP3: /bin/bash: /usr/bin/lame: No such file or directory on Beta Cluster - https://phabricator.wikimedia.org/T317128 (10Legoktm) >>! In T317128#8218312, @taavi wrote: > There's an... [18:05:00] (03CR) 10Ahmon Dancy: [C: 03+1] Add RateLimitedProgressReporter (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/830195 (owner: 10Ahmon Dancy) [18:08:03] (03CR) 10Jaime Nuche: [C: 03+2] Add RateLimitedProgressReporter (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/830195 (owner: 10Ahmon Dancy) [19:02:52] 10Release-Engineering-Team, 10Scap: Scap backport should include the deployers username on the Gerrit approval comment - https://phabricator.wikimedia.org/T317163 (10taavi) >>! In T317163#8218071, @dancy wrote: > Right now the +2 review message is "Approved via scap backport". How about something this instead... [19:26:44] 10GitLab: Not able to login GitLab with Wikimedia Dev Account - https://phabricator.wikimedia.org/T317218 (10Dzahn) Hi @AnnWF, as @hashar already said, there are 2 users in LDAP using the same email address. One is "wfan" and the other is "wfan219". If this is because you want to have separate work and volunte... [19:31:09] 10GitLab: Not able to login GitLab with Wikimedia Dev Account - https://phabricator.wikimedia.org/T317218 (10Dzahn) So, that would be "change email address" at https://wikitech.wikimedia.org/wiki/Special:Preferences after you logged in there. [19:49:14] (03CR) 10Ahmon Dancy: [V: 03+1] "The code looks real nice and tested fine in train-dev." [tools/scap] - 10https://gerrit.wikimedia.org/r/830675 (owner: 10Jaime Nuche) [19:50:22] pcc-worker1003.puppet-diffs.eqiad1.wikimedia.cloud is having disk space issues. "fatal: cannot create directory at 'modules/admin/files/home/otto/bin': No space left on device" [20:06:56] bd808: identified build 37153 as "bad" (huge compared to others). then clicked to delete it in web UI [20:07:15] doesn't mean /srv shows free space just yet.. hmm [20:07:27] (03PS1) 10Ahmon Dancy: scap backport: Include the deployers username on the Gerrit approval comment [tools/scap] - 10https://gerrit.wikimedia.org/r/830692 (https://phabricator.wikimedia.org/T317163) [20:08:11] mutante: thanks for poking at it. I'll pay you back by continuing to work on getting tools off of diffusion :) [20:08:44] :) ok [20:08:58] so clicking in web UI does not mean it is gone from the file system.. at least so far [20:09:02] meh [20:09:18] 36713 and 37153 are to blame [20:09:35] guess I just have to use rm then [20:10:12] also clicked delete for 36713 [20:12:25] bd808: try again. /srv is back to 58% usage.. I used rm -rf [20:12:45] (03CR) 10Ahmon Dancy: "Tested using ~/test-scap-backport in train-dev." [tools/scap] - 10https://gerrit.wikimedia.org/r/830692 (https://phabricator.wikimedia.org/T317163) (owner: 10Ahmon Dancy) [20:13:11] this happens when we build on "*" by leaving the host field empty [20:13:23] more than ~ 2 times within some time frame [20:13:43] mutante: excellent. I got the report i need by just running again right after I reported here. worker round robin can be helpful somedays :) [20:13:44] building on all means about 5GB of space.. and there are only 25 to ues [20:13:59] building on "*" seems like a really bad idea [20:14:21] usually yes, there can be rare cases where you want it though [20:14:36] we should manually delete them after doing so I guess [20:15:04] or it needs to be more aggressive in auto-cleaning up if a job is over X MB in size.. a bit smart [20:15:20] (03PS2) 10Ahmon Dancy: scap backport: Include the deployer's username on the Gerrit approval comment [tools/scap] - 10https://gerrit.wikimedia.org/r/830692 (https://phabricator.wikimedia.org/T317163) [20:15:50] maybe half of them are just forgetting to fill out the form field [20:19:30] (03PS1) 10Ahmon Dancy: Make test-scap-backport bootstrap itself [tools/train-dev] - 10https://gerrit.wikimedia.org/r/830693 [20:50:45] (03CR) 10Ahmon Dancy: [V: 03+1] "Tested by TheresNoTime on deploy1002 too. See https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/830313/" [tools/scap] - 10https://gerrit.wikimedia.org/r/830692 (https://phabricator.wikimedia.org/T317163) (owner: 10Ahmon Dancy) [20:54:15] (03CR) 10Zabe: [C: 03+2] scap backport: Include the deployer's username on the Gerrit approval comment [tools/scap] - 10https://gerrit.wikimedia.org/r/830692 (https://phabricator.wikimedia.org/T317163) (owner: 10Ahmon Dancy) [20:58:41] (03Merged) 10jenkins-bot: scap backport: Include the deployer's username on the Gerrit approval comment [tools/scap] - 10https://gerrit.wikimedia.org/r/830692 (https://phabricator.wikimedia.org/T317163) (owner: 10Ahmon Dancy) [21:03:13] 10Release-Engineering-Team, 10Scap: Make "scap backport" skip syncing steps for labs-only changes - https://phabricator.wikimedia.org/T317242 (10Zabe) [21:28:28] 10GitLab: Not able to login GitLab with Wikimedia Dev Account - https://phabricator.wikimedia.org/T317218 (10AnnWF) Hi Daniel, Thanks for your help, I did update two account to different email and the one with wikimedia email named wfan219 is working properly now, thanks! This ticket can be move to done :) >>... [21:35:20] 10GitLab: Not able to login GitLab with Wikimedia Dev Account - https://phabricator.wikimedia.org/T317218 (10AnnWF) 05Open→03Resolved [21:41:53] 10GitLab: Not able to login GitLab with Wikimedia Dev Account - https://phabricator.wikimedia.org/T317218 (10Dzahn) ok, great, glad it works:) thanks for closing it. [22:39:52] 10GitLab, 10Striker, 10User-bd808: Inactive Diffusion repos for missingpages and wiki2email were migrated to GitLab - https://phabricator.wikimedia.org/T317260 (10Urbanecm) [22:47:09] https://we.phorge.it/phame/post/view/1/going_public/ [23:12:31] 10Phabricator: Herald_rule_for_Product Analytics and Data Engineering tags to Wmfdata-Python tasks Open, Needs TriagePublic - https://phabricator.wikimedia.org/T317266 (10JArguello-WMF) [23:13:25] 10Phabricator: Herald_rule_for_Product Analytics and Data Engineering tags to Wmfdata-Python tasks Open, Needs TriagePublic - https://phabricator.wikimedia.org/T317266 (10JArguello-WMF) [23:13:27] 10Phabricator, 10Data Engineering Planning, 10Product-Analytics, 10wmfdata-python: Herald rule to add Product Analytics and Data Engineering tags to Wmfdata-Python tasks - https://phabricator.wikimedia.org/T304572 (10JArguello-WMF) [23:28:05] bd808: yay [23:55:55] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10tstarling) It can't be due to the wmf.28 deployment bec...