[00:50:48] (03PS2) 10Jforrester: Zuul: [mediawiki/extensions/SimpleCalendar] Add basic quibble CI [integration/config] - 10https://gerrit.wikimedia.org/r/710700 (owner: 10Zoranzoki21) [00:50:52] (03PS3) 10Jforrester: Zuul: [mediawiki/extensions/SimpleCalendar] Add basic quibble CI [integration/config] - 10https://gerrit.wikimedia.org/r/710700 (owner: 10Zoranzoki21) [00:50:55] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/SimpleCalendar] Add basic quibble CI [integration/config] - 10https://gerrit.wikimedia.org/r/710700 (owner: 10Zoranzoki21) [00:51:27] (03CR) 10Jforrester: [C: 03+2] Add Aca to the CI allow list [integration/config] - 10https://gerrit.wikimedia.org/r/710628 (owner: 10Zabe) [00:51:58] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/SimpleCalendar] Add basic quibble CI [integration/config] - 10https://gerrit.wikimedia.org/r/710700 (owner: 10Zoranzoki21) [00:52:30] !log Zuul: [mediawiki/extensions/SimpleCalendar] Add basic quibble CI [00:52:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:52:37] (03Merged) 10jenkins-bot: Add Aca to the CI allow list [integration/config] - 10https://gerrit.wikimedia.org/r/710628 (owner: 10Zabe) [00:52:47] !log Zuul: Add Aca to the CI allow list [00:52:49] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [06:49:24] 10Continuous-Integration-Infrastructure, 10MediaWiki-Docker, 10mwcli, 10Patch-For-Review, 10User-Addshore: Set up CI for mwcli - https://phabricator.wikimedia.org/T248779 (10Addshore) 05Openβ†’03Resolved [06:57:11] 10Release-Engineering-Team, 10GitLab: WARNING: In GitLab 14.0 we will begin removing all configuration backups older than yourgitlab_rails['backup_keep_time'] setting (currently set to: 259200) - https://phabricator.wikimedia.org/T288324 (10Jelto) Should be fixed with https://gerrit.wikimedia.org/r/710676. No... [07:01:42] 10Release-Engineering-Team, 10SRE, 10Elections: Create list of developers eligible to vote on the 2021 board vote - https://phabricator.wikimedia.org/T288455 (10Dzahn) Yes, I agree. Back in the days there was only one type of "shell access" and the term is still used in that sense. Then we split into "deploy... [07:08:11] 10Release-Engineering-Team, 10SRE, 10Elections: Create list of developers eligible to vote on the 2021 board vote - https://phabricator.wikimedia.org/T288455 (10Qgil) Thank you @Dzahn! One more piece of data for this puzzle. [07:28:45] (03CR) 10Awight: "@milimetric Just wanted to mention, the code was automatically pulled over to production and I was able to read error logs, thanks for hel" [analytics/reportupdater-queries] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/709646 (https://phabricator.wikimedia.org/T287578) (owner: 10Awight) [08:22:02] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10Release Pipeline, 10serviceops-radar, 10Epic: Deploy MediaWiki to Wikimedia production in containers - https://phabricator.wikimedia.org/T238770 (10JMeybohm) [08:39:41] Project beta-scap-sync-world build #16474: 04FAILURE in 6 min 5 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/16474/ [08:45:37] Project beta-scap-sync-world build #16475: 04STILL FAILING in 4 min 26 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/16475/ [08:50:35] Project beta-scap-sync-world build #16476: 04STILL FAILING in 3 min 27 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/16476/ [08:57:39] Project beta-scap-sync-world build #16477: 04STILL FAILING in 3 min 10 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/16477/ [09:07:47] Yippee, build fixed! [09:07:48] Project beta-scap-sync-world build #16478: 09FIXED in 3 min 15 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/16478/ [09:35:29] anyone available to +2 a really simple patch to disable some BlueSpice tests that are failing? They are preventing a bunch of libup patches that would otherwise merge automatically [09:35:33] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/BlueSpiceFoundation/+/710718 [09:53:40] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-jeena (Sprint 7/28-8/10): 1.37.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T281159 (10Lucas_Werkmeister_WMDE) [10:08:32] could someone add me to the triagers phab acl please? https://phabricator.wikimedia.org/project/view/13/ [10:09:29] in this specific case I want to add a tag to multiple tasks at once and given this is not the first time I want to do that the batch edit ability would be helpful [10:11:22] sure, would you mind creating a task with the request? [10:11:54] oh actually even though I'm a member I can't add others, only members of acl*phabricator can add, so nevermind [11:56:33] majavah: Might be easier to file a task asking... As per DannyS... The people who can add to it seems a little odd [11:59:19] 10Phabricator: Add Majavah to #Triagers - https://phabricator.wikimedia.org/T288613 (10Majavah) [11:59:19] Reedy: T288613 [11:59:20] T288613: Add Majavah to #Triagers - https://phabricator.wikimedia.org/T288613 [12:05:05] 10Phabricator: Add Majavah to #Triagers - https://phabricator.wikimedia.org/T288613 (10Reedy) [13:27:26] Hello! o/ I'm trying to sign-in at gitlab.wikimedia.org but keep getting "Sign-in using Wikimedia Dev Account auth failed: Sign-in failed because Username ending with MIME type format is not allowed.." I've tried logging out and in on Wikitech but nope :( I'm in WMF group [15:02:58] brennen: ^ [15:07:50] Hi! The under-construction gitlab looks great! [15:20:42] thanks razzi <3 [15:23:44] bearloga: what username are you using? [15:24:40] that is, are you using shell username or CN? (though those might be the same...) [15:25:24] https://ldap.toolforge.org/user/bearloga ? [15:25:48] brennen: it didn't give me an option. I clicked sign in button and it took me to an error page [15:25:53] thcipriani: yup! [15:29:42] bearloga: ah, interesting... hrm, trying to reproduce. do you have javascript disabled in any way, by chance? [15:30:42] brennen: js enabled :) by the way the error code is 422 [15:31:25] (using latest firefox, if that helps) [15:31:50] thanks. i'll go spelunking in logs and see if that yields anything. [15:34:51] bearloga: welp: https://gitlab.com/gitlab-org/gitlab/-/issues/335278 [15:34:59] my main reaction to this bug is "you have got to be kidding me" [15:35:09] "send gifs/memes" [15:36:34] lmfao [15:37:29] At least it's been fixed [15:37:36] >will merge into 13-12-stable-ee ready for 13.12.10-ee [15:38:39] brennen: time to upgrade all the things [15:39:02] ah, yep, and there's a release yesterday. we can probably upgrade shortly. [15:45:44] 10Release-Engineering-Team, 10MW-on-K8s, 10SRE: Automated validation of mediawiki-multiversion images - https://phabricator.wikimedia.org/T288629 (10dancy) [15:45:57] 10Release-Engineering-Team, 10MW-on-K8s, 10SRE: Automated validation of mediawiki-multiversion images - https://phabricator.wikimedia.org/T288629 (10dancy) p:05Triageβ†’03Medium [16:04:53] 10Release-Engineering-Team, 10GitLab, 10serviceops, 10User-brennen: GitLab patch release: 13.12.10: Resolves "Username ending with MIME type format is not allowed" errors - https://phabricator.wikimedia.org/T288631 (10brennen) [16:05:13] (03CR) 10Jforrester: [C: 03+2] "No need to sync immediately." [integration/docroot] - 10https://gerrit.wikimedia.org/r/711318 (owner: 10Libraryupgrader) [16:05:30] bearloga: see T288631 [16:05:31] T288631: GitLab patch release: 13.12.10: Resolves "Username ending with MIME type format is not allowed" errors - https://phabricator.wikimedia.org/T288631 [16:06:43] brennen: just catching up on this (was in a meeting) and lol. yes, I am an ogg audio file good job gitlab [16:07:17] thank you!! [16:10:11] thanks for reporting easily the funniest bug i've come across this month. [16:15:31] it's PRETTY GOOD [16:24:02] 10Release-Engineering-Team (Radar), 10Scap, 10Patch-For-Review, 10User-jijiki: Update Scap to perform rolling restart for all MW deploy - https://phabricator.wikimedia.org/T266055 (10dancy) [16:32:20] hahaha: that is an amazing bug. The example of issac.asimov is just *chef's kiss* perfect [16:38:29] 10Release-Engineering-Team (Radar), 10Scap, 10Patch-For-Review, 10User-jijiki: Update Scap to perform rolling restart for all MW deploy - https://phabricator.wikimedia.org/T266055 (10dancy) We ran the test today. @jijiki supplied SRE backup. It ran in two phases: Phase 1: php_fpm_always_restart: false... [16:39:48] 10Release-Engineering-Team (Radar), 10Scap, 10Patch-For-Review, 10User-jijiki: Update Scap to perform rolling restart for all MW deploy - https://phabricator.wikimedia.org/T266055 (10dancy) [17:31:59] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-jeena (Sprint 7/28-8/10): 1.37.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T281159 (10jeena) @Jdlrobson Thanks for the info! [17:36:08] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-jeena (Sprint 7/28-8/10): 1.37.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T281159 (10Jdforrester-WMF) [17:43:55] (03PS6) 10Michael Große: Add and use job to deploy branch in node project [integration/config] - 10https://gerrit.wikimedia.org/r/683589 (https://phabricator.wikimedia.org/T278706) [18:00:39] 10Release-Engineering-Team, 10SRE, 10Elections: Create list of developers eligible to vote on the 2021 board vote - https://phabricator.wikimedia.org/T288455 (10Aklapper) >>! In T288455#7270626, @Tgr wrote: >>>! In T288455#7270569, @thcipriani wrote: >> https://wikimedia.biterg.io/goto/1d62cdd781dbfa9f093dd9... [18:10:03] 10Release-Engineering-Team, 10SRE, 10Elections: Create list of developers eligible to vote on the 2021 board vote - https://phabricator.wikimedia.org/T288455 (10Aklapper) Note that people's names displayed on wikimedia.biterg.io are **not** necessarily their account usernames in some system (Gerrit, Phab, mw... [18:11:45] 10Release-Engineering-Team, 10SRE, 10Elections: Create list of developers eligible to vote on the 2021 board vote - https://phabricator.wikimedia.org/T288455 (10RhinosF1) We still need to map gerrit username to SUL account for it to be useful for votewiki [18:14:55] 10Release-Engineering-Team, 10SRE, 10Elections: Create list of developers eligible to vote on the 2021 board vote - https://phabricator.wikimedia.org/T288455 (10bd808) >>! In T288455#7276112, @Aklapper wrote: >>>! In T288455#7270626, @Tgr wrote: >>>>! In T288455#7270569, @thcipriani wrote: >>> https://wikime... [18:26:09] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments, 10User-jeena: 1.37.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T281160 (10thcipriani) [18:47:59] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Code Health), 10Release-Engineering-Team-TODO, 10Code-Stewardship-Reviews: deployment-prep: Code stewardship request - https://phabricator.wikimedia.org/T215217 (10Majavah) 05Resolvedβ†’03Open a:05Jrbranaaβ†’03None Boldly re-opening this task,... [18:48:54] 10Release-Engineering-Team (Logspam), 10MediaWiki-extensions-LiquidThreads, 10Wikimedia-production-error: PHP Notice: Exception doing lazy updates: Exception: Thread::splitIncrementFromSubject: thread subject has no increment: Comments: - https://phabricator.wikimedia.org/T288646 (10thcipriani) [19:33:11] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Code Health), 10Release-Engineering-Team-TODO, 10Code-Stewardship-Reviews: deployment-prep: Code stewardship request - https://phabricator.wikimedia.org/T215217 (10ArielGlenn) I just linked this task to someone today after explaining that, afaik th... [19:52:41] 10Release-Engineering-Team, 10SRE, 10Elections: Create list of developers eligible to vote on the 2021 board vote - https://phabricator.wikimedia.org/T288455 (10thcipriani) >>! In T288455#7276184, @bd808 wrote: >>>! In T288455#7276112, @Aklapper wrote: >>>>! In T288455#7270626, @Tgr wrote: >>>>>! In T288455#... [19:54:24] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team: Request access to deployment-prep - https://phabricator.wikimedia.org/T286584 (10thcipriani) 05Openβ†’03Resolved a:03thcipriani Thanks for adding #together and sorry for the delay. This should be done now. [19:56:07] 10Release-Engineering-Team (Radar), 10Growth-Team, 10StructuredDiscussions, 10Patch-For-Review, and 2 others: Core tests failing due to Flow HTTP requests and ServiceContainer access - https://phabricator.wikimedia.org/T287001 (10MBinder_WMF) @kostajh to follow up [19:58:16] Hey @jeena we just discovered a critical issue with some critical instrumentation which is part of a core metric. Is it possible to get the backport now rather than wait until 4pm? The patches are ready and merged to master, but there's no ticket yet. [19:58:38] yes, I think it's fine to backport now [19:59:21] 10Phabricator: Add Majavah to #Triagers - https://phabricator.wikimedia.org/T288613 (10Aklapper) 05Openβ†’03Resolved a:03Aklapper Make sense. :) Added! [20:00:56] https://phabricator.wikimedia.org/T288655 [20:02:29] @jeena just finding someone to backport for it [20:02:54] okay, let me know if you need me to do it (or anything else) [20:07:23] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments, 10User-jeena: 1.37.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T281160 (10Jdforrester-WMF) [20:25:51] Why does releng on have have #together [20:26:57] it was our team motto from an offsite so we used it in phab as a team-joke [20:26:57] I guess thcipriani can best explain this [20:27:03] or greg-g :) [20:27:05] from our first offsite, in fact :) [20:27:49] Ah [20:31:09] That must take you back [20:33:00] yeah, it was great [21:04:40] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Code-Stewardship-Reviews: deployment-prep: Code stewardship request - https://phabricator.wikimedia.org/T215217 (10brennen) [21:05:35] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Radar), 10Code-Stewardship-Reviews: deployment-prep: Code stewardship request - https://phabricator.wikimedia.org/T215217 (10brennen) [21:57:08] (03PS1) 10Accraze: inference-services: added draftquality pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/711731 (https://phabricator.wikimedia.org/T287787) [21:59:45] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Radar), 10Code-Stewardship-Reviews: deployment-prep: Code stewardship request - https://phabricator.wikimedia.org/T215217 (10Jrbranaa) @Majavah - yes, this work has stalled due to a shift in my priorities over the last few months. However, it's back... [22:29:39] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-jeena (Sprint 7/28-8/10): 1.37.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T281159 (10Tacsipacsi) [23:15:17] 10Release-Engineering-Team, 10Elections: Create list of developers eligible to vote on the 2021 board vote - https://phabricator.wikimedia.org/T288455 (10Aklapper) [23:15:34] 10Release-Engineering-Team, 10Elections: Create list of developers eligible to vote on the 2021 board vote - https://phabricator.wikimedia.org/T288455 (10Aklapper) **: I dumped 321 SUL accounts that correspond to some of the 399 eligible Gerrit/LDAP accounts into P17004.** I'll repeat what had been sai... [23:17:26] 10Beta-Cluster-Infrastructure, 10Acme-chief: Issues with acme-chief cert rotation on deployment-prep, 2021-01-12 - https://phabricator.wikimedia.org/T271778 (10Samwilson) Looks like *.wikimedia.beta.wmflabs.org expired again, on Tue, 10 Aug 2021 01:01:19 GMT. [23:28:05] 10Beta-Cluster-Infrastructure, 10Acme-chief: Issues with acme-chief cert rotation on deployment-prep, 2021-01-12 - https://phabricator.wikimedia.org/T271778 (10Platonides) It doesn't seem. It is serving a non-expired certificate generated on July: Not Before: 7/11/2021 Not After: 10/9/2021 [23:41:14] 10Release-Engineering-Team (Doing), 10Elections: Create list of developers eligible to vote on the 2021 board vote - https://phabricator.wikimedia.org/T288455 (10brennen) [23:45:17] 10Release-Engineering-Team, 10GitLab (Initialization), 10User-brennen: Set a default project limit for users on GitLab - https://phabricator.wikimedia.org/T288696 (10brennen) [23:45:49] 10Release-Engineering-Team (Doing), 10GitLab (Initialization), 10User-brennen: Set a default project limit for users on GitLab - https://phabricator.wikimedia.org/T288696 (10brennen) p:05Triageβ†’03Medium [23:57:35] 10Release-Engineering-Team (Next), 10GitLab, 10User-brennen: Enable incoming mail handling for GitLab - https://phabricator.wikimedia.org/T284961 (10brennen) [23:58:21] 10Release-Engineering-Team (Doing), 10gitlab-settings, 10GitLab (Initialization), 10User-brennen: Set a default project limit for users on GitLab - https://phabricator.wikimedia.org/T288696 (10brennen)