[00:04:22] (03PS2) 10Jforrester: Zuul: [mediawiki/extensions/SecurityApi] Add basic quibble CI [integration/config] - 10https://gerrit.wikimedia.org/r/744883 (https://phabricator.wikimedia.org/T297243) (owner: 10STran) [00:04:53] (03CR) 10Jforrester: "Is this code going into Wikimedia production? You've put it in the "production" section but not added the "in-wikimedia-production" templa" [integration/config] - 10https://gerrit.wikimedia.org/r/744883 (https://phabricator.wikimedia.org/T297243) (owner: 10STran) [00:15:45] 10Release-Engineering-Team (Next), 10Release, 10Train Deployments: 1.38.0-wmf.13 deployment blockers - https://phabricator.wikimedia.org/T293954 (10matmarex) ##### Risky Patch! πŸš‚πŸ”₯ * **Change**: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiEditor/+/587228 "Tag WikiEditor edits with a hidden ta... [00:22:59] (03PS3) 10BryanDavis: jjb: Define test and publish pipelines for developer-portal [integration/config] - 10https://gerrit.wikimedia.org/r/744871 (https://phabricator.wikimedia.org/T297170) [02:46:48] 10Phabricator, 10Design: Symbol for "Open" on Phabricator tasks suggests "alert" to me, and is inconsistent in its placement when task types are also used - https://phabricator.wikimedia.org/T297249 (10Volker_E) I agree with the problem statement. Note, that I think that's general usability issue, not an acces... [04:39:35] (03CR) 10STran: Zuul: [mediawiki/extensions/SecurityApi] Add basic quibble CI (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/744883 (https://phabricator.wikimedia.org/T297243) (owner: 10STran) [06:04:52] (Queue (Jenkins jobs + Zuul functions) alert) firing: Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org [06:14:52] (Queue (Jenkins jobs + Zuul functions) alert) firing: (2) Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org [06:29:52] (Queue (Jenkins jobs + Zuul functions) alert) resolved: Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org [09:51:54] 10Phabricator, 10Design: Task symbol for "Open" suggests "alert" and is inconsistent in its placement when task types are also used - https://phabricator.wikimedia.org/T297249 (10Aklapper) [14:44:01] (03PS3) 10Jforrester: Zuul: [mediawiki/extensions/SecurityApi] Add basic quibble CI [integration/config] - 10https://gerrit.wikimedia.org/r/744883 (https://phabricator.wikimedia.org/T297243) (owner: 10STran) [14:44:03] (03CR) 10Jforrester: Zuul: [mediawiki/extensions/SecurityApi] Add basic quibble CI (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/744883 (https://phabricator.wikimedia.org/T297243) (owner: 10STran) [15:19:29] 10Project-Admins, 10Campaign-Tools (Campaign-Tools-Sprint-2): Create project tag for the CampaignEvents extension - https://phabricator.wikimedia.org/T297295 (10Daimona) [15:32:33] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/SecurityApi] Add basic quibble CI [integration/config] - 10https://gerrit.wikimedia.org/r/744883 (https://phabricator.wikimedia.org/T297243) (owner: 10STran) [15:35:39] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/SecurityApi] Add basic quibble CI [integration/config] - 10https://gerrit.wikimedia.org/r/744883 (https://phabricator.wikimedia.org/T297243) (owner: 10STran) [15:38:09] !log Zuul: [mediawiki/extensions/SecurityApi] Add basic quibble CI for T297243 [15:38:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:38:16] T297243: Add CI for mediawiki/extensions/SecurityApi - https://phabricator.wikimedia.org/T297243 [15:39:21] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [400.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [15:42:26] ^ apparently there's a $LARGE_AMOUNT of Wikidata patches backlogging everything else [15:42:36] Yes, as ever. :-( [15:46:48] o/ [15:58:51] hey! :P [16:00:17] I see plenty of mediawiki/core patches too, let’s share the blame [16:09:02] 10GitLab (Project Migration), 10Release-Engineering-Team (Doing), 10User-brennen, 10Voice & Tone: Rename mainline Git branch from "master" to "main" on all WMF-hosted repositories during GitLab migration - https://phabricator.wikimedia.org/T281593 (10brennen) [16:09:25] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T293953 (10brennen) [16:09:34] 10Release-Engineering-Team (Doing), 10dev-images, 10mwcli, 10User-brennen: Add php-luasandbox to dev-images used by mwcli - https://phabricator.wikimedia.org/T286678 (10brennen) 05In progressβ†’03Stalled a:05brennenβ†’03None [16:09:52] dancy, brennen: hey! The selenium tests for https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CodeMirror/+/744803 (T296639 fix backport) are failing, but since I can reproduce the test error with an empty patch (https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CodeMirror/+/745265), the selenium tests passed on the master branch and I can [16:09:52] reproduce the warning on mwdebug I'm going to force merge and backport if it's ok with you [16:09:53] T296639: Less_Exception_Parser: File `resources/lib/ooui/wikimedia-ui-base.less` not found. in ve.ui.CodeMirror.init.less - https://phabricator.wikimedia.org/T296639 [16:18:39] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [200.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [16:25:09] majavah: i think that sounds reasonable. [16:25:29] thanks! [16:34:23] and deployed [16:34:24] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T293953 (10Majavah) [16:35:02] (03CR) 10Hashar: [C: 03+2] jjb: Define test and publish pipelines for developer-portal [integration/config] - 10https://gerrit.wikimedia.org/r/744871 (https://phabricator.wikimedia.org/T297170) (owner: 10BryanDavis) [16:37:09] (03CR) 10jerkins-bot: [V: 04-1] jjb: Define test and publish pipelines for developer-portal [integration/config] - 10https://gerrit.wikimedia.org/r/744871 (https://phabricator.wikimedia.org/T297170) (owner: 10BryanDavis) [16:37:49] (03CR) 10Hashar: [C: 03+2] "I forgot to create the jobs:" [integration/config] - 10https://gerrit.wikimedia.org/r/744871 (https://phabricator.wikimedia.org/T297170) (owner: 10BryanDavis) [16:38:33] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [400.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [16:40:49] (03Merged) 10jenkins-bot: jjb: Define test and publish pipelines for developer-portal [integration/config] - 10https://gerrit.wikimedia.org/r/744871 (https://phabricator.wikimedia.org/T297170) (owner: 10BryanDavis) [16:43:01] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Radar), 10Security-Team, 10serviceops, and 2 others: Setup GitLab Runner in trusted environment - https://phabricator.wikimedia.org/T295481 (10Jelto) Trusted runners need some additional configuration parameters (like enable Prometheus metrics). I tr... [16:49:31] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [200.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [17:20:34] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10Scap: Automate scap prep - https://phabricator.wikimedia.org/T245184 (10dancy) I think that https://gerrit.wikimedia.org/r/c/mediawiki/tools/release/+/736072 has sufficiently automated this step and we don't need to do further work on this ticket. [17:28:25] PROBLEM - Check systemd state on doc1001 is CRITICAL: CRITICAL - degraded: The following units failed: rsync-doc-doc2001.codfw.wmnet.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [17:51:39] 10Scap (Scap3-MediaWiki-MVP): Unify scap2 and scap3 - https://phabricator.wikimedia.org/T147477 (10dancy) 05Openβ†’03Declined At this point it is unclear what exactly scap2 and scap3 mean so I'm declining this ticket. [18:15:32] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Done by Thu 04 Nov 🧟), 10Security-Team, 10User-brennen: Enable runners for projects under gitlab.wikimedia.org security group - https://phabricator.wikimedia.org/T294050 (10sbassett) 05Resolvedβ†’03Open Hey @brennen (and maybe @Jelto?) - This seem... [18:19:30] 10Deployments, 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’): Yak:Train stats - https://phabricator.wikimedia.org/T281307 (10thcipriani) 05Openβ†’03Resolved a:03thcipriani It seems we have all that's in the task description and more, now: https://gitlab.wikimedia.org/thcipriani/train-stats#train-stats Calli... [18:19:36] 10Scap (Scap3-MediaWiki-MVP), 10MediaWiki-Configuration: Scap should not rely on extension-list, instead pass --extension-dir to mergeMessageFileList.php - https://phabricator.wikimedia.org/T125678 (10dancy) @hashar Is this ticket resolved? [18:19:43] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10Infrastructure-Foundations, 10Patch-For-Review, 10Puppet, 10User-brennen: logspam-watch: UTF-8 errors for some input - https://phabricator.wikimedia.org/T292246 (10brennen) [18:21:18] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10User-brennen: A tool for quickly answering what groups an extension is deployed to - https://phabricator.wikimedia.org/T296050 (10brennen) a:03brennen [18:22:57] 10GitLab (Project Migration), 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10User-brennen, 10Voice & Tone: Delete temporary main branch in mediawiki/core.git until rename from master to main is ready - https://phabricator.wikimedia.org/T296205 (10thcipriani) [18:23:55] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’): Delete the wmf/1.37.0-wmf.* branches once we've fully deployed to 1.38.0-wmf.1 and finished with them - https://phabricator.wikimedia.org/T291501 (10thcipriani) [18:23:57] RECOVERY - Check systemd state on doc1001 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [18:25:54] 10Phabricator, 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10User-MModell: On Phabricator workboard, show status of associated Gerrit patches - https://phabricator.wikimedia.org/T215148 (10mmodell) [18:26:06] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’): Investigate password store options - https://phabricator.wikimedia.org/T290337 (10hashar) Bubbling up. I have not committed to do the switch after the exploratory phase. The quick presentation I gave to the team has been welcomed and we seem to have consensus that 1pa... [18:28:04] 10Phabricator, 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10Stashbot: Integrate Stashbot better visually with Phabricator - https://phabricator.wikimedia.org/T152729 (10mmodell) [18:28:21] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’): Migrate from pws gpg to 1password - https://phabricator.wikimedia.org/T290337 (10hashar) [18:29:34] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10Security Team AppSec, 10Security-Team, and 2 others: Confirm with releng which docker images make the most sense to use - https://phabricator.wikimedia.org/T296805 (10thcipriani) a:05sbassettβ†’03thcipriani Hi @sbassett [18:30:17] 10GitLab (Project Migration), 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’): Create new GitLab project group: Generated Data Platform - https://phabricator.wikimedia.org/T296381 (10thcipriani) a:03brennen [18:32:22] 10Phabricator, 10Release-Engineering-Team (Next), 10serviceops: Deprecate git-ssh service on phabricator.wikimedia.org - https://phabricator.wikimedia.org/T296022 (10thcipriani) [18:33:15] 10Release-Engineering-Team (Next), 10Security-Team: Determine a better way to track Wikimedia production security patches - https://phabricator.wikimedia.org/T295925 (10thcipriani) [18:34:20] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10Security Team AppSec, 10Security-Team, and 2 others: Confirm with releng which docker images make the most sense to use - https://phabricator.wikimedia.org/T296805 (10thcipriani) [18:34:28] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Done by Wed 24 Nov 🧟), 10Security-Team, 10SecTeam-Processed, 10User-brennen: Limit GitLab shared runners to trusted contributors - https://phabricator.wikimedia.org/T292094 (10brennen) 05Openβ†’03Resolved [18:38:35] 10Diffusion, 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10Developer Productivity: Reduce task notification noise/frequency of changes to associated open patchsets - https://phabricator.wikimedia.org/T143162 (10thcipriani) [18:40:53] 10GitLab (Project Migration), 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10User-brennen, 10Voice & Tone: Delete temporary main branch in mediawiki/core.git until rename from master to main is ready - https://phabricator.wikimedia.org/T296205 (10thcipriani) a:03thcipriani [18:42:34] 10Phabricator, 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10User-MModell: On Phabricator workboard, show status of associated Gerrit patches - https://phabricator.wikimedia.org/T215148 (10thcipriani) [18:43:56] 10Phabricator, 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10Stashbot: Integrate Stashbot better visually with Phabricator - https://phabricator.wikimedia.org/T152729 (10thcipriani) [18:46:19] 10Release-Engineering-Team (Radar), 10Observability-Logging, 10User-brennen: Experiment with automating error log triage - https://phabricator.wikimedia.org/T290267 (10thcipriani) [19:02:34] 10Deployments, 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’): Scope out "deployment-dashboard" project - https://phabricator.wikimedia.org/T137729 (10thcipriani) 05Openβ†’03Resolved a:03thcipriani Declaring https://versions.toolforge.org/ to be a reasonable approximation of the acceptance criteria of this task. [19:03:01] 10Phabricator, 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10User-brennen: Dockerize our Phabricator development environment - https://phabricator.wikimedia.org/T245575 (10mmodell) https://github.com/willson556/phorge-devcontainer [19:04:23] 10Phabricator, 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10User-brennen: Dockerize our Phabricator development environment - https://phabricator.wikimedia.org/T245575 (10mmodell) 05Openβ†’03Declined I don't think it is useful to work on something specific to wikimedia. This is a general enough thing tha... [19:27:06] 10Release-Engineering-Team (Done by Wed 24 Nov 🧟), 10DataΒ³: Demo dataΒ³ - https://phabricator.wikimedia.org/T295497 (10mmodell) 05In progressβ†’03Resolved [19:28:10] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’): Yak: automate the use of https://releng.team - https://phabricator.wikimedia.org/T280093 (10mmodell) https://caddyserver.com/v2 might be an interesting way to implement this. [20:07:25] (03CR) 10Zabe: [C: 03+2] deployments-calendar: train_branch is happening an hour earlier now we're out of DST [tools/release] - 10https://gerrit.wikimedia.org/r/742584 (owner: 10Jforrester) [20:09:12] (03Merged) 10jenkins-bot: deployments-calendar: train_branch is happening an hour earlier now we're out of DST [tools/release] - 10https://gerrit.wikimedia.org/r/742584 (owner: 10Jforrester) [20:59:06] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T293953 (10dancy) [21:19:11] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for David Martin - https://phabricator.wikimedia.org/T297321 (10DMartin-WMF) [21:22:26] 10Scap: bring back scap l10n-update - https://phabricator.wikimedia.org/T210602 (10dancy) 05Openβ†’03Declined I think `scap sync-l10n` covers this need. [21:32:09] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T293953 (10Ladsgroup) [21:46:23] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10Scap: Speed up cache_git_info - https://phabricator.wikimedia.org/T297324 (10dancy) [21:50:50] 10Release-Engineering-Team (Radar), 10Scap, 10SRE: mwdebug1001 and mwdebug1002 are reliably the last two hosts to finish scap-cdb-rebuild - https://phabricator.wikimedia.org/T203625 (10dancy) Is this still a problem? [21:58:52] 10Release-Engineering-Team (Doing), 10Scap, 10MediaWiki-Release-Tools: Output of scap lint.py isn't helpful if it fails the php -l check - https://phabricator.wikimedia.org/T272760 (10dancy) 05In progressβ†’03Resolved a:03dancy This change was included in scap 4.1.0 which was recently deployed. [22:00:57] 10Scap, 10SRE: Decide on /var/lib vs /home as locations of homedir for l10nupdate - https://phabricator.wikimedia.org/T163288 (10dancy) 05Openβ†’03Declined [22:03:01] 10Scap: [scap] Sync fewer files from old deploy branches - https://phabricator.wikimedia.org/T68053 (10dancy) 05Openβ†’03Declined [22:16:25] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’), 10Scap: Stop trying to avoid rsyncing l10n CDB files - https://phabricator.wikimedia.org/T297326 (10dancy) [22:24:05] (03PS1) 10Arlolra: Add an option to disable sandbox to gen.screenshot.js [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/745333 [22:46:57] (03CR) 10Subramanya Sastry: "Did you mean to use this option somewhere? Or only add the option and use it in a later patch?" [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/745333 (owner: 10Arlolra) [23:34:38] (03CR) 10Arlolra: Add an option to disable sandbox to gen.screenshot.js (031 comment) [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/745333 (owner: 10Arlolra) [23:41:50] (03CR) 10Subramanya Sastry: [C: 03+2] "Ah .. I must have done a bad commit where i forgot to add this bit to the repo. I see this same dirtying of this file on parsing-qa-02." [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/745333 (owner: 10Arlolra) [23:43:13] (03Merged) 10jenkins-bot: Add an option to disable sandbox to gen.screenshot.js [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/745333 (owner: 10Arlolra) [23:45:59] (03CR) 10Arlolra: Add an option to disable sandbox to gen.screenshot.js (031 comment) [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/745333 (owner: 10Arlolra) [23:54:06] (03CR) 10Subramanya Sastry: Add an option to disable sandbox to gen.screenshot.js (031 comment) [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/745333 (owner: 10Arlolra)