[00:28:56] 10Continuous-Integration-Config, 10Code-Health-Metrics: Enable codehealth pipeline for more extensions - https://phabricator.wikimedia.org/T224990 (10Pppery) [00:29:33] RECOVERY - Check systemd state on doc1002 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [01:08:46] (03CR) 10Krinkle: jjb: Provide bespoke PHPUnit standalone job for MediaWiki core (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/904100 (https://phabricator.wikimedia.org/T203694) (owner: 10Jforrester) [07:44:26] 10GitLab (Infrastructure), 10SRE, 10ops-codfw, 10serviceops-collab: Install additional SSDs on gitlab2003.wikimedia.org (B5) - https://phabricator.wikimedia.org/T333304 (10Jelto) p:05Triage→03Medium Thanks @Papaul for the quick installation! I can confirm new disks are available on the host: ` gitlab... [08:15:01] 10GitLab (Infrastructure), 10SRE, 10ops-codfw, 10serviceops-collab: Install additional SSDs on gitlab2003.wikimedia.org (B5) - https://phabricator.wikimedia.org/T333304 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin2002 for host gitlab2003.wikimedia.org with OS bullseye [08:48:03] 10GitLab (Infrastructure), 10SRE, 10ops-codfw, 10serviceops-collab: Install additional SSDs on gitlab2003.wikimedia.org (B5) - https://phabricator.wikimedia.org/T333304 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin2002 for host gitlab2003.wikimedia.org with OS bullseye... [09:31:28] 10GitLab, 10Release-Engineering-Team, 10serviceops-collab, 10Patch-For-Review: runner-1030.gitlab-runners.eqiad1.wikimedia.cloud out of space - https://phabricator.wikimedia.org/T333586 (10hashar) >>! In T333586#8743249, @xcollazo wrote: >>>! In T333586#8743236, @hashar wrote: >> (I am pretty sure airflows... [09:48:49] 10GitLab (CI & Job Runners), 10mwbot-rs: mwbot-rs large cache usage on Gitlab CI - https://phabricator.wikimedia.org/T333663 (10hashar) [09:57:01] 10Project-Admins: Creation of a new project "All-and-every-Wikibooks" - https://phabricator.wikimedia.org/T330600 (10Aklapper) @JackPotte: I remain reluctant to create a project tag that sounds like only a single person would actively triage / use / maintain. :-/ [10:14:25] 10GitLab (CI & Job Runners), 10mwbot-rs: mwbot-rs large cache usage on Gitlab CI - https://phabricator.wikimedia.org/T333663 (10hashar) I have inspected a cache for `countcount/mwbot` which has 1734MBytes: A friendly view: | .cargo/registry | 427 MB |--|-- | \- cache | 22 MB | \- index | 225 MB | \- src |... [10:29:16] 10Project-Admins: Creation of a new project "All-and-every-Wikibooks" - https://phabricator.wikimedia.org/T330600 (10hashar) [10:32:52] 10Project-Admins: Creation of a new project "All-and-every-Wikibooks" - https://phabricator.wikimedia.org/T330600 (10hashar) @Aklapper given there are already `#wikisource` #wikisource and `#wiktionary` #wiktionary, I am guessing it makes sense to create one for Wikibooks as well? There seems to be at least two... [10:33:11] (03CR) 10Jaime Nuche: [C: 03+1] Migrate from git fat to git lfs (031 comment) [software/gerrit] (deploy/wmf/stable-3.5) - 10https://gerrit.wikimedia.org/r/904239 (https://phabricator.wikimedia.org/T333465) (owner: 10Hashar) [10:34:52] 10GitLab (CI & Job Runners), 10mwbot-rs: mwbot-rs large cache usage on Gitlab CI - https://phabricator.wikimedia.org/T333663 (10Lucas_Werkmeister_WMDE) >>! In T333663#8744903, @hashar wrote: > ./.cargo/registry/index/github.com-1ecc6299db9ec823/.git is apparently a full clone of a repository with a 214 M packf... [10:35:46] 10GitLab (CI & Job Runners), 10mwbot-rs: mwbot-rs large cache usage on Gitlab CI - https://phabricator.wikimedia.org/T333663 (10hashar) [10:39:33] 10GitLab, 10Release-Engineering-Team, 10serviceops-collab, 10Patch-For-Review: runner-1030.gitlab-runners.eqiad1.wikimedia.cloud out of space - https://phabricator.wikimedia.org/T333586 (10hashar) I have filed {T333663} for `mwbot-rs/mwbot` and `countcount/mwbot`. For data engineering I think {T311111} is... [10:40:14] 10GitLab (CI & Job Runners), 10mwbot-rs: mwbot-rs large cache usage on Gitlab CI - https://phabricator.wikimedia.org/T333663 (10hashar) [10:49:58] 10Project-Admins: Creation of a new project "All-and-every-Wikibooks" - https://phabricator.wikimedia.org/T330600 (10Lionel_Scheepmans) >>! Dans T330600#8744854, @Aklapper a écrit : > @JackPotte: I remain reluctant to create a project tag that sounds like only a single person would actively triage / use / mainta... [11:11:43] 10GitLab (Infrastructure), 10SRE, 10ops-codfw, 10serviceops-collab: Install additional SSDs on gitlab2003.wikimedia.org (B5) - https://phabricator.wikimedia.org/T333304 (10Jelto) 05Open→03Resolved The reimage happend on `gitlab2003` but it seems the partman config is not producing the expected result.... [11:14:59] 10MediaWiki-Releasing, 10NewFunctionality-Worktype: Auto-deploy tarballs to releases.wm.o - https://phabricator.wikimedia.org/T64392 (10Peachey88) [11:18:36] 10MediaWiki-Releasing, 10NewFunctionality-Worktype: Auto-deploy tarballs to releases.wm.o - https://phabricator.wikimedia.org/T64392 (10hashar) 05Open→03Declined Declining to reflect the reality. [11:23:17] 10GitLab (Infrastructure), 10serviceops-collab: Troubleshoot partman config for two additional disks on GitLab hosts - https://phabricator.wikimedia.org/T333674 (10Jelto) [12:03:49] *Brief Gitlab Interruption*: We'll be updating the version of Gitlab in production in the next while, so please expect some brief interruptions in the next 45-60 minutes [12:21:11] (03Abandoned) 10Hashar: jjb: Provide bespoke PHPUnit standalone job for MediaWiki core [integration/config] - 10https://gerrit.wikimedia.org/r/904100 (https://phabricator.wikimedia.org/T203694) (owner: 10Jforrester) [12:21:39] (03Abandoned) 10Hashar: Zuul: [mediawiki/core] Switch standalone job to bespoke one [integration/config] - 10https://gerrit.wikimedia.org/r/904101 (https://phabricator.wikimedia.org/T203694) (owner: 10Jforrester) [13:46:08] (03CR) 10Jaime Nuche: [C: 03+1] Extract and deploy upstream plugins [software/gerrit] (deploy/wmf/stable-3.5) - 10https://gerrit.wikimedia.org/r/904575 (owner: 10Hashar) [14:15:48] 10Phabricator, 10serviceops-collab, 10Patch-For-Review: Phabricator's access log may have some problems in log rotation - https://phabricator.wikimedia.org/T332869 (10Jelto) log rotation on `phab1004` looks good for now. The new log files contain more logs again and the service did not fail. ` phab1004:/var... [14:23:14] (03PS1) 10Robert Vogel: Add new Extenion:ContentDoplets [integration/config] - 10https://gerrit.wikimedia.org/r/904805 [14:26:25] !log (deployment-prep) `dwalden@deployment-mwmaint02:~$ mwscript maintenance/refreshLinks.php --wiki enwiki --category='Pages that use Phonos'` for T327708 [14:26:36] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:26:36] T327708: File usage by Phonos should be registered and tracked - https://phabricator.wikimedia.org/T327708 [14:34:50] !log (deployment-prep) `dwalden@deployment-mwmaint02:~$ mwscript maintenance/refreshLinks.php --wiki en_rtlwiki --category='Pages that use Phonos'` for T327708 [14:34:57] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:34:57] T327708: File usage by Phonos should be registered and tracked - https://phabricator.wikimedia.org/T327708 [14:43:42] (03PS1) 10Robert Vogel: Add new Extenion:ContentStabilization [integration/config] - 10https://gerrit.wikimedia.org/r/904810 [14:44:57] (03CR) 10CI reject: [V: 04-1] Add new Extenion:ContentStabilization [integration/config] - 10https://gerrit.wikimedia.org/r/904810 (owner: 10Robert Vogel) [14:49:27] (03PS2) 10Robert Vogel: Add new Extenion:ContentStabilization [integration/config] - 10https://gerrit.wikimedia.org/r/904810 [14:50:42] (03CR) 10CI reject: [V: 04-1] Add new Extenion:ContentStabilization [integration/config] - 10https://gerrit.wikimedia.org/r/904810 (owner: 10Robert Vogel) [15:11:11] (03PS1) 10Robert Vogel: Add new Extenion:BlueSpiceInstanceStatus [integration/config] - 10https://gerrit.wikimedia.org/r/904818 [15:19:19] 10GitLab (CI & Job Runners), 10mwbot-rs: mwbot-rs large cache usage on Gitlab CI - https://phabricator.wikimedia.org/T333663 (10Legoktm) I actually don't know why files are getting cached in the first place?? I removed (or tried to) all caching back in November: https://gitlab.wikimedia.org/repos/mwbot-rs/rust... [15:28:39] (03CR) 10Ahmon Dancy: "Fancy" [software/gerrit] (deploy/wmf/stable-3.5) - 10https://gerrit.wikimedia.org/r/904575 (owner: 10Hashar) [15:48:11] 10Release-Engineering-Team, 10Security-Team, 10Patch-For-Review: deploy_security.py should check if user.name and user.email git configs are set - https://phabricator.wikimedia.org/T333572 (10Lucas_Werkmeister_WMDE) 05Open→03Resolved a:03Lucas_Werkmeister_WMDE Done. (Note: @dancy requested that it shou... [15:48:21] 10Release-Engineering-Team, 10Security-Team: deploy_security.py should check if user.name and user.email git configs are set - https://phabricator.wikimedia.org/T333572 (10Lucas_Werkmeister_WMDE) [15:52:40] 10GitLab (Project Migration), 10Release-Engineering-Team (GitLab V: Event Horizon 🌄), 10Patch-For-Review, 10User-brennen: Rename mainline branch from "master" to "main" in GitLab:repos/releng/release - https://phabricator.wikimedia.org/T329770 (10dancy) The `master` branch still exists at the moment which... [16:02:14] 10GitLab (CI & Job Runners), 10mwbot-rs: mwbot-rs large cache usage on Gitlab CI - https://phabricator.wikimedia.org/T333663 (10dancy) >>! In T333663#8745979, @Legoktm wrote: > I actually don't know why files are getting cached in the first place?? I removed (or tried to) all caching back in November: https://... [16:05:10] (03CR) 10Hashar: Extract and deploy upstream plugins (032 comments) [software/gerrit] (deploy/wmf/stable-3.5) - 10https://gerrit.wikimedia.org/r/904575 (owner: 10Hashar) [16:05:22] (03PS2) 10Hashar: Extract and deploy upstream plugins [software/gerrit] (deploy/wmf/stable-3.5) - 10https://gerrit.wikimedia.org/r/904575 [16:06:27] (03PS1) 10Bernard Wang: Remove docs from Vector CI [integration/config] - 10https://gerrit.wikimedia.org/r/904825 [16:07:33] (03PS2) 10Bernard Wang: Remove docs from Vector CI [integration/config] - 10https://gerrit.wikimedia.org/r/904825 (https://phabricator.wikimedia.org/T321102) [16:14:45] (03PS2) 10Hashar: Add new ContentDroplets extension [integration/config] - 10https://gerrit.wikimedia.org/r/904805 (owner: 10Robert Vogel) [16:15:22] (03CR) 10Hashar: [C: 03+2] "I have slightly rephrased the commit message." [integration/config] - 10https://gerrit.wikimedia.org/r/904805 (owner: 10Robert Vogel) [16:16:42] (03Merged) 10jenkins-bot: Add new ContentDroplets extension [integration/config] - 10https://gerrit.wikimedia.org/r/904805 (owner: 10Robert Vogel) [16:17:15] (03PS3) 10Hashar: Add new ContentStabilization extension [integration/config] - 10https://gerrit.wikimedia.org/r/904810 (owner: 10Robert Vogel) [16:17:48] (03CR) 10Hashar: [C: 03+2] "I have slightly reworded the commit message and rebased on top of conflicting change https://gerrit.wikimedia.org/r/c/integration/config/+" [integration/config] - 10https://gerrit.wikimedia.org/r/904810 (owner: 10Robert Vogel) [16:18:13] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/904805/ [16:18:15] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:18:57] (03Merged) 10jenkins-bot: Add new ContentStabilization extension [integration/config] - 10https://gerrit.wikimedia.org/r/904810 (owner: 10Robert Vogel) [16:19:25] (03PS2) 10Hashar: Add new BlueSpiceInstanceStatus extension [integration/config] - 10https://gerrit.wikimedia.org/r/904818 (owner: 10Robert Vogel) [16:20:01] (03CR) 10Hashar: [C: 03+2] Add new BlueSpiceInstanceStatus extension [integration/config] - 10https://gerrit.wikimedia.org/r/904818 (owner: 10Robert Vogel) [16:20:09] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/904810 [16:20:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:21:11] (03Merged) 10jenkins-bot: Add new BlueSpiceInstanceStatus extension [integration/config] - 10https://gerrit.wikimedia.org/r/904818 (owner: 10Robert Vogel) [16:24:13] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/904818 [16:24:15] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:25:06] 10Release-Engineering-Team, 10Security-Team, 10SecTeam-Processed: deploy_security.py should check if user.name and user.email git configs are set - https://phabricator.wikimedia.org/T333572 (10sbassett) Thanks for addressing this. [16:32:01] (03CR) 10Hashar: "The `extension-javascript-documentation` template:" [integration/config] - 10https://gerrit.wikimedia.org/r/904825 (https://phabricator.wikimedia.org/T321102) (owner: 10Bernard Wang) [16:33:02] (03CR) 10Bernard Wang: Remove docs from Vector CI (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/904825 (https://phabricator.wikimedia.org/T321102) (owner: 10Bernard Wang) [16:33:17] 10Project-Admins: Creation of a new project "All-and-every-Wikibooks" - https://phabricator.wikimedia.org/T330600 (10Aklapper) @hashar: No, I'd personally prefer not to repeat mistakes only based on the argumentation that they were made before. :) I just cleaned up the #all-and-every-wiktionary workboard a bit,... [16:37:34] (03PS1) 10Majavah: zuul: Move RealMe to in-wikimedia-production [integration/config] - 10https://gerrit.wikimedia.org/r/904828 (https://phabricator.wikimedia.org/T324535) [16:38:39] (03CR) 10CI reject: [V: 04-1] zuul: Move RealMe to in-wikimedia-production [integration/config] - 10https://gerrit.wikimedia.org/r/904828 (https://phabricator.wikimedia.org/T324535) (owner: 10Majavah) [16:39:16] (03PS2) 10Majavah: zuul: Move RealMe to in-wikimedia-production [integration/config] - 10https://gerrit.wikimedia.org/r/904828 (https://phabricator.wikimedia.org/T324535) [16:55:46] (03CR) 10Nray: [C: 03+1] "I don't have +2 rights to this repo, but this looks right to me" [integration/config] - 10https://gerrit.wikimedia.org/r/904825 (https://phabricator.wikimedia.org/T321102) (owner: 10Bernard Wang) [16:57:01] 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10Data-Engineering-Planning, 10Performance Issue: Improve speed of Gitlab CI - https://phabricator.wikimedia.org/T311111 (10xcollazo) @lbowmaker for your consideration. We could win significant developer productivity by tackling this one. [17:10:52] 10GitLab, 10Release-Engineering-Team, 10serviceops-collab: runner-1030.gitlab-runners.eqiad1.wikimedia.cloud out of space - https://phabricator.wikimedia.org/T333586 (10xcollazo) >>! In T333586#8743160, @Dzahn wrote: >>>! In T333586#8743072, @xcollazo wrote: >> Does it make sense to consider running the cro... [17:34:56] 10GitLab, 10Release-Engineering-Team, 10serviceops-collab: runner-1030.gitlab-runners.eqiad1.wikimedia.cloud out of space - https://phabricator.wikimedia.org/T333586 (10Dzahn) >>! In T333586#8746429, @xcollazo wrote: > Now that I think about it more, perhaps running `clear-docker-cache.timer` hourly is too a... [17:52:39] (03CR) 10Jdlrobson: [C: 03+1] "Hey james: could I trouble you for a review here, please? :)" [integration/config] - 10https://gerrit.wikimedia.org/r/904825 (https://phabricator.wikimedia.org/T321102) (owner: 10Bernard Wang) [17:57:32] 10GitLab (CI & Job Runners), 10mwbot-rs: mwbot-rs large cache usage on Gitlab CI - https://phabricator.wikimedia.org/T333663 (10Legoktm) a:03Legoktm Ohhhh got it, thanks. So then I can add `rm -rf target && rm -rf .cargo` to the end of the job as cleanup. Will get to that in the next few days. [18:17:05] 10GitLab, 10Release-Engineering-Team, 10serviceops-collab: runner-1030.gitlab-runners.eqiad1.wikimedia.cloud out of space - https://phabricator.wikimedia.org/T333586 (10Dzahn) >>! In T333586#8746491, @Dzahn wrote: >>> I wouldn't mind merging a patch or uploading one that goes back to "every 8 hours". But t... [19:05:10] 10GitLab, 10Release-Engineering-Team, 10serviceops-collab: runner-1030.gitlab-runners.eqiad1.wikimedia.cloud out of space - https://phabricator.wikimedia.org/T333586 (10xcollazo) >>! In T333586#8746569, @Dzahn wrote: >>>! In T333586#8746491, @Dzahn wrote: >>>> I wouldn't mind merging a patch or uploading one... [19:24:55] I've seen andre doing it but I don't remember how: I have a list of task numbers (eg: T125108, T519135,...), can I url-hack a phab query with them to then enable easy bulk editing of them? [19:24:56] T125108: sync-masters slow on mira - https://phabricator.wikimedia.org/T125108 [19:25:26] (made up Ts there, I don't care about mira, blast from the past) [19:40:40] greg-g: this could be easier to find! Check out and ^F for "hidden" [19:41:48] Would be something like [20:14:23] bd808: that's it! thank you! [20:15:48] yw. That might be worth an entry under https://www.mediawiki.org/wiki/Phabricator/Help#Searching_for_items too. [20:31:33] 10GitLab (Project Migration), 10Release-Engineering-Team (GitLab V: Event Horizon 🌄), 10Patch-For-Review, 10User-brennen: Rename mainline branch from "master" to "main" in GitLab:repos/releng/release - https://phabricator.wikimedia.org/T329770 (10jeena) >>! In T329770#8746098, @dancy wrote: > The `master`...