[00:00:25] (03CR) 10Ahmon Dancy: [C: 03+2] train-dev: Add build-image subcommand [tools/train-dev] - 10https://gerrit.wikimedia.org/r/735076 (owner: 10Ahmon Dancy) [00:00:50] (03Merged) 10jenkins-bot: train-dev: Add build-image subcommand [tools/train-dev] - 10https://gerrit.wikimedia.org/r/735076 (owner: 10Ahmon Dancy) [00:12:15] (03CR) 10Ahmon Dancy: Refactor scap.plugins.gerrit for use with backport command (032 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/735064 (https://phabricator.wikimedia.org/T294459) (owner: 10Dduvall) [00:23:54] (03PS1) 10Ahmon Dancy: Use mediawiki/tools/release/make-container-image/webserver to build httpd images [tools/train-dev] - 10https://gerrit.wikimedia.org/r/735077 [00:25:01] (03CR) 10jerkins-bot: [V: 04-1] Use mediawiki/tools/release/make-container-image/webserver to build httpd images [tools/train-dev] - 10https://gerrit.wikimedia.org/r/735077 (owner: 10Ahmon Dancy) [00:26:05] (03PS2) 10Ahmon Dancy: Use mediawiki/tools/release/make-container-image/webserver to build httpd images [tools/train-dev] - 10https://gerrit.wikimedia.org/r/735077 [02:26:37] (03CR) 10Dduvall: Refactor scap.plugins.gerrit for use with backport command (032 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/735064 (https://phabricator.wikimedia.org/T294459) (owner: 10Dduvall) [02:32:21] (03PS4) 10Dduvall: Refactor scap.plugins.gerrit for use with backport command [tools/scap] - 10https://gerrit.wikimedia.org/r/735064 (https://phabricator.wikimedia.org/T294459) [02:32:53] (03CR) 10Dduvall: Refactor scap.plugins.gerrit for use with backport command (032 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/735064 (https://phabricator.wikimedia.org/T294459) (owner: 10Dduvall) [02:33:18] (03CR) 10jerkins-bot: [V: 04-1] Refactor scap.plugins.gerrit for use with backport command [tools/scap] - 10https://gerrit.wikimedia.org/r/735064 (https://phabricator.wikimedia.org/T294459) (owner: 10Dduvall) [02:35:01] (03PS5) 10Dduvall: Refactor scap.plugins.gerrit for use with backport command [tools/scap] - 10https://gerrit.wikimedia.org/r/735064 (https://phabricator.wikimedia.org/T294459) [02:37:57] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for **CBlanton** - https://phabricator.wikimedia.org/T292679 (10thcipriani) 05Open→03Resolved a:03thcipriani ## {icon check-circle-o} Training completed! Hey @CBlanton thanks for attending deployment training I have a... [05:25:11] 10Beta-Cluster-Infrastructure, 10Infrastructure-Foundations, 10Puppet: Puppet failing on deployment-docker-* hosts - https://phabricator.wikimedia.org/T294517 (10Majavah) p:05Triage→03High [05:25:26] 10Beta-Cluster-Infrastructure: Puppet failing on deployment-docker-* hosts - https://phabricator.wikimedia.org/T294517 (10Majavah) [08:16:32] 10Quibble: Quibble should configure php 7.4+ built in web server to use multiple workers - https://phabricator.wikimedia.org/T259456 (10awight) I would expect this to trigger most of the same race conditions as the Apache+quibble configuration. But this was deployed successfully? [10:10:16] (03PS1) 10Zabe: Fix double space [tools/release] - 10https://gerrit.wikimedia.org/r/735313 [10:30:09] (03CR) 10Mvolz: [C: 03+2] Fix double space [tools/release] - 10https://gerrit.wikimedia.org/r/735313 (owner: 10Zabe) [10:31:38] (03Merged) 10jenkins-bot: Fix double space [tools/release] - 10https://gerrit.wikimedia.org/r/735313 (owner: 10Zabe) [10:32:27] (03PS2) 10Majavah: Deployments calendar: Be specific that this is automatic [tools/release] - 10https://gerrit.wikimedia.org/r/730214 (owner: 10Jforrester) [10:32:38] (03CR) 10Majavah: [C: 03+2] Deployments calendar: Be specific that this is automatic [tools/release] - 10https://gerrit.wikimedia.org/r/730214 (owner: 10Jforrester) [10:33:19] (03Merged) 10jenkins-bot: Deployments calendar: Be specific that this is automatic [tools/release] - 10https://gerrit.wikimedia.org/r/730214 (owner: 10Jforrester) [11:20:27] 10Quibble: Quibble should configure php 7.4+ built in web server to use multiple workers - https://phabricator.wikimedia.org/T259456 (10kostajh) >>! In T259456#7464288, @awight wrote: > I would expect this to trigger most of the same race conditions as the Apache+quibble configuration. But this was deployed suc... [11:23:26] (03CR) 10Zfilipin: selenium: Remove wdio-video-reporter npm package (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/734639 (https://phabricator.wikimedia.org/T294341) (owner: 10Zfilipin) [11:23:29] (03Abandoned) 10Zfilipin: selenium: Remove wdio-video-reporter npm package [integration/config] - 10https://gerrit.wikimedia.org/r/734639 (https://phabricator.wikimedia.org/T294341) (owner: 10Zfilipin) [11:41:51] 10Release-Engineering-Team (Done by Thu 04 Nov), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T293947 (10RhinosF1) [11:43:20] new blocker :( [11:43:42] thanks Spookreeeno [11:45:44] zabe: I think revert patch is best [11:46:23] Flaggedrevs is pretty messy [11:46:34] to put it mildly [11:46:41] yeah, trying to fix FlaggedRevs is usually a mess [11:47:17] Lucas_WMDE: should we just upload a revert [11:48:28] I'd give the patch authors/reviewers a bit of time to react (and CC them on the task), getting things sorted without a revert is always preferrable [11:49:42] I pinged Petr + Daniel [11:49:47] And commented on the patch [11:51:23] I agree with majavah, but I also think uploading a revert wouldn’t hurt [11:51:30] especially if you can reproduce the error, and test whether the revert fixes it [11:51:46] then it’s already ready if someone™ decides the revert should be used to unblock the train [11:53:53] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/FlaggedRevs/+/735327 if needed [11:58:12] Lunch [12:19:48] 10Project-Admins: Create project tag for wmfdb - https://phabricator.wikimedia.org/T294357 (10LSobanski) 05Open→03Resolved a:03LSobanski Project created: https://phabricator.wikimedia.org/project/view/5611/ [12:22:44] all my tests so far hasn't really revealed the problem zabe re FlaggedRevs but if the issue persist, we might just revert for now though I don't see a difference in behavior so far. [12:33:44] xSavitar: I think you just found it didn't you? [12:33:56] I did [12:33:58] Revert patch is there though if easier [12:34:00] uploaded a patch now [12:34:10] The fix is 1 line :D [12:34:37] You have patch :) [12:34:54] It's nice when I come back from lunch to see issues solved [12:34:56] Spookreeeno: yes! [12:35:00] :D [12:35:13] Actually, this particular case is an edge case affected by configuration. [12:35:33] I think you can deploy can't you assuming it gets merged [12:35:45] I remember seeing you get access [12:36:01] FlaggedRevs is a stack of edge cases [12:37:28] Yes, I can deploy! [12:37:40] Spookreeeno: a stack of edge cases, yeah :D [12:37:48] :) [12:37:51] xSavitar: cool, so I don't think we need the revert :) [12:38:04] Yes for now, we don't need the revert. [12:38:06] I think Martin is stealing scap atm [12:38:13] I'll abandon later [12:38:37] If Martin is already deploying, maybe he can just do this one and we test instead? I'm up for testing and making sure it works [12:38:53] But I do not have Editors right on dewiktionary [12:39:20] Lucas_WMDE: you maybe? [12:40:16] xSavitar: Martin says you are clear [12:40:23] He's just running scripts now [12:40:47] It probably can be tested on test2wiki [12:41:01] Okay, zabe can you help me test if I deploy? [12:41:49] If someone can give me editor on test2wiki [12:42:06] done [12:43:37] Let me add a test case for the patch as well. [12:43:43] It's safer that way, give me 5mins [12:44:00] majavah: thx [12:54:20] zabe: Once CI confirms verification, I'll deploy [12:58:51] zabe, Spookreeeno, once this is deployed I guess it will be backported? [12:59:16] xSavitar: once merged to master, cherry pick to .6 [12:59:24] Then you deploy the .6 backport [12:59:36] Okay perfect! Just wanted to confirm and be sure. Thanks! [13:00:19] I'm around for moral support [13:00:31] There's a commands tool for backporting [13:01:47] o/ I’m here now but it looks like you don’t need me at the moment :) [13:01:55] (and I’m pretty sure I don’t have special rights on dewiktionary with either account) [13:03:14] (03PS1) 10Hashar: doc: hide the table of content [integration/quibble] - 10https://gerrit.wikimedia.org/r/735379 [13:03:20] (03PS1) 10Hashar: doc: move LICENSE to its own document page [integration/quibble] - 10https://gerrit.wikimedia.org/r/735380 [13:04:11] Lucas_WMDE: just waiting on a happy jerkins [13:04:36] CI is currently failing for that patch. [13:04:49] Ye [13:05:36] (03CR) 10jerkins-bot: [V: 04-1] doc: move LICENSE to its own document page [integration/quibble] - 10https://gerrit.wikimedia.org/r/735380 (owner: 10Hashar) [13:05:49] Yeah, resetting the global should fix it. [13:06:03] yes [13:06:06] Other test are using the stashed global so they're failing [13:06:36] Sorry this is taking a bit more time. Will signal once I'm done. [13:06:43] np [13:08:12] (03PS2) 10Hashar: doc: move LICENSE to its own document page [integration/quibble] - 10https://gerrit.wikimedia.org/r/735380 [13:30:27] Spookreeeno, zabe, FR won't be the end of me. I think CI should pass now and I'll merge :) [13:31:22] xSavitar: :) [13:31:46] * Spookreeeno is currently wondering how code with no references can be called [13:31:58] Is T294559 a blocker? [13:32:01] T294559: Editing page ending with :numbers or going to such page via a namespace alias results in redirect to address with :number used as port - https://phabricator.wikimedia.org/T294559 [13:32:33] (probably not since it isn't that bad) [13:33:07] Spookreeeno: one way or another, sometime needs to be invested to step up FR's game otherwise ... [13:34:04] zabe: i'd block [13:34:30] xSavitar: I don't think that's appropriate to self merge [13:34:35] I can't self merge [13:34:39] Can someone help me? [13:34:49] majavah: Exactly, I was just about to say that before you said it [13:35:01] I'll cherry-pick and merge the backport [13:35:07] But for the main patch, someone has to do it for me [13:35:45] please don't do that either without getting someone else to review/+2 that code on master [13:35:56] majavah: Cool! [13:36:35] 10Release-Engineering-Team (Done by Thu 04 Nov), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T293947 (10Zabe) [13:36:54] I just poked on the patch, hoping it gets merged sooner [13:38:57] majavah: can you +2? [13:39:55] no, I don't have flaggedrevs set up locally and I need to leave a bit so I don't have time to set up or dive deep to how it works for a proper review [13:47:31] Spookreeeno: I just realised the magnitude of the issue, this definetly now looks like something is weirdly going wron [13:48:12] zabe: very [13:49:32] Do mind sharing zabe, is it related to the FR patch? [13:49:47] xSavitar: no [13:49:49] no, I am talking about T294559 [13:49:50] T294559: Editing page ending with :number or going to such page via a namespace alias results in redirect to address with :number used as port - https://phabricator.wikimedia.org/T294559 [13:49:52] another blocker [13:50:00] Ah okay! [13:50:51] cc twentyafterfour hashar [13:51:06] yeah what is happening? [13:51:22] hashar: Do you think T294559 is worth a rollback? [13:51:30] hashar: https://phabricator.wikimedia.org/T294559 is breaking some redirects [13:52:51] someone tried to outsmart the parser [13:52:52] 10Release-Engineering-Team (Doing): Change notification email from jenkins-bot@wikimedia.org to releng internal list - https://phabricator.wikimedia.org/T151642 (10Gehel) discovery-alerts@lists.wikimedia.org is now accepting emails from releng@ [13:56:27] Pchelolo: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/720489 was missed off risky patch? Could it be related? [13:57:26] hashar: commit says risky patch templates got missed :( [13:58:01] it might be unrelated, but might be good to add that to the task [13:58:03] I will rollback [13:58:17] ack [13:58:24] oh wait twentyafterfour is around :) [13:58:42] I am happy to do the rollback twentyafterfour [13:59:21] hashar: up to you [13:59:35] I can do it, I just got here [14:03:16] got a neighborh at home to retrieve stuff [14:03:21] will be available in a few [14:03:48] hashar: I'll do it, no worries [14:07:08] rollback in progress [14:07:30] group1 => 1.38.0-wmf.5 [14:07:50] ty but :( [14:16:21] twentyafterfour: thx ! :) [14:16:43] i have poked platform slack channel [14:19:11] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen): Install php-yaml for Translate - https://phabricator.wikimedia.org/T189244 (10hashar) @Nikerabbit asked about it today. The PHP module is in Debian https://packages.debian.org/search?keywords=php-yaml and it is apparently available in... [14:38:05] 10Release-Engineering-Team (Done by Thu 04 Nov), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T293947 (10Urbanecm_WMF) Note that earlier today, foundation.wikimedia.org was connected to CentralAuth (T205347). That's something we di... [16:13:41] (03CR) 10Ahmon Dancy: [C: 03+1] Refactor scap.plugins.gerrit for use with backport command [tools/scap] - 10https://gerrit.wikimedia.org/r/735064 (https://phabricator.wikimedia.org/T294459) (owner: 10Dduvall) [16:35:47] (03PS1) 10Hashar: Support for ~/.config/quibble.ini [integration/quibble] - 10https://gerrit.wikimedia.org/r/735414 (https://phabricator.wikimedia.org/T238225) [16:38:55] (03CR) 10jerkins-bot: [V: 04-1] Support for ~/.config/quibble.ini [integration/quibble] - 10https://gerrit.wikimedia.org/r/735414 (https://phabricator.wikimedia.org/T238225) (owner: 10Hashar) [16:54:19] (03CR) 10Ahmon Dancy: [C: 03+2] Refactor scap.plugins.gerrit for use with backport command [tools/scap] - 10https://gerrit.wikimedia.org/r/735064 (https://phabricator.wikimedia.org/T294459) (owner: 10Dduvall) [16:55:02] (03Merged) 10jenkins-bot: Refactor scap.plugins.gerrit for use with backport command [tools/scap] - 10https://gerrit.wikimedia.org/r/735064 (https://phabricator.wikimedia.org/T294459) (owner: 10Dduvall) [18:48:27] (03CR) 10Dduvall: [C: 04-1] "In testing I noticed that `last-build` file created by the multiversion build is overwritten by the subsequent webserver build. Is this in" [tools/train-dev] - 10https://gerrit.wikimedia.org/r/735077 (owner: 10Ahmon Dancy) [18:54:57] (03PS2) 10Hashar: Support for ~/.config/quibble.ini [integration/quibble] - 10https://gerrit.wikimedia.org/r/735414 (https://phabricator.wikimedia.org/T238225) [18:58:06] (03CR) 10jerkins-bot: [V: 04-1] Support for ~/.config/quibble.ini [integration/quibble] - 10https://gerrit.wikimedia.org/r/735414 (https://phabricator.wikimedia.org/T238225) (owner: 10Hashar) [19:11:54] 10Release-Engineering-Team (Done by Thu 04 Nov), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T293947 (10Legoktm) Is the train blocked? Typically an email is sent out when that's the case... [19:13:29] I should hope the train's blocked, given that it has rolled back to group0 [19:14:54] "this is what the wikis will be permanently, if you need code changes you need to backport them to 1.38.0-wmf.5" [19:31:17] 10Release-Engineering-Team (Done by Thu 04 Nov), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T293947 (10bd808) >>! In T293947#7466367, @Legoktm wrote: > Is the train blocked? Typically an email is sent out when that's the case...... [19:32:44] 10Release-Engineering-Team (Done by Thu 04 Nov), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T293947 (10mmodell) Indeed it's blocked but getting unblocked soon I hope? [19:55:15] 10Release-Engineering-Team (Done by Thu 04 Nov), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T293947 (10mmodell) [19:57:02] 10Release-Engineering-Team (Done by Thu 04 Nov), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T293947 (10Zabe) [20:53:23] (03CR) 10Hashar: "Ahmon said I could develop that using mediawiki/tools/train-dev so I will definitely give it a try :)" [software/gerrit] (wmf/stable-3.3) - 10https://gerrit.wikimedia.org/r/723992 (owner: 10Hashar) [21:26:29] (03CR) 10Ahmon Dancy: Use mediawiki/tools/release/make-container-image/webserver to build httpd images (032 comments) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/735077 (owner: 10Ahmon Dancy) [21:26:39] (03PS3) 10Ahmon Dancy: Use the mediawiki/tools/release repo to build httpd images [tools/train-dev] - 10https://gerrit.wikimedia.org/r/735077 [21:45:34] (03PS1) 10Dduvall: Configure scap to query the train-dev gerrit service [tools/train-dev] - 10https://gerrit.wikimedia.org/r/735458 [21:47:15] (03CR) 10Ahmon Dancy: [C: 03+2] Configure scap to query the train-dev gerrit service [tools/train-dev] - 10https://gerrit.wikimedia.org/r/735458 (owner: 10Dduvall) [21:47:39] (03Merged) 10jenkins-bot: Configure scap to query the train-dev gerrit service [tools/train-dev] - 10https://gerrit.wikimedia.org/r/735458 (owner: 10Dduvall) [21:51:18] (03CR) 10Dduvall: [C: 03+2] "Looks good! Let's roll." [tools/train-dev] - 10https://gerrit.wikimedia.org/r/735077 (owner: 10Ahmon Dancy) [21:51:46] (03Merged) 10jenkins-bot: Use the mediawiki/tools/release repo to build httpd images [tools/train-dev] - 10https://gerrit.wikimedia.org/r/735077 (owner: 10Ahmon Dancy) [21:53:57] 10Project-Admins: Create project tag for wmfdb - https://phabricator.wikimedia.org/T294357 (10Aklapper) Could someone please edit the Phab project description to link to its codebase URL and/or further documentation? Thanks in advance. [21:56:02] (03PS1) 10Ahmon Dancy: make-container-image: Implement AUTO_STAGE_LANG [tools/release] - 10https://gerrit.wikimedia.org/r/735460 [21:56:49] (03PS2) 10Ahmon Dancy: make-container-image: Implement AUTO_STAGE_LANG [tools/release] - 10https://gerrit.wikimedia.org/r/735460 [21:59:16] (03CR) 10Ahmon Dancy: [C: 03+2] make-container-image: Implement AUTO_STAGE_LANG [tools/release] - 10https://gerrit.wikimedia.org/r/735460 (owner: 10Ahmon Dancy) [22:00:01] (03Merged) 10jenkins-bot: make-container-image: Implement AUTO_STAGE_LANG [tools/release] - 10https://gerrit.wikimedia.org/r/735460 (owner: 10Ahmon Dancy) [22:06:09] !log releases1002 - releases2002, something broke puppet here about 27.5 hours ago. lookup() did not find a value for the name 'profile::docker::storage::physical_volumes' [22:06:10] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:46:20] 10Release-Engineering-Team (Radar), 10Infrastructure-Foundations, 10GitLab (Infrastructure), 10Patch-For-Review, and 3 others: Puppetise gitlab-ansible playbook - https://phabricator.wikimedia.org/T283076 (10Dzahn) >>! In T283076#7454868, @Jelto wrote: > I identified at least two issues which prevent us fr...