[02:04:26] 10GitLab (Project Migration), 10Release-Engineering-Team: Create new GitLab project group: Admin - https://phabricator.wikimedia.org/T312057 (10Familyfirst407) [06:40:49] 10Phabricator, 10I18n: 2 simple typos in English for Phabricator (Differential) - https://phabricator.wikimedia.org/T312051 (10Aklapper) [06:41:35] 10Phabricator, 10I18n: Typo in Phabricator (Arcanist) in English - https://phabricator.wikimedia.org/T312052 (10Aklapper) [06:42:16] 10Phabricator, 10I18n: Typo in Phabricator (Audit) in English - https://phabricator.wikimedia.org/T312053 (10Aklapper) [08:18:40] hashar: hi, are you around? [08:49:31] !log Diffusion rORES repository. Changed URI settings: enabled SSH push for mirroring; disabled HTTP | T311390 [08:49:34] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:49:34] T311390: Github's wikimedia/ores not mirroring to Gerrit's scoring/ores/ores - https://phabricator.wikimedia.org/T311390 [08:52:12] 10Phabricator, 10Release-Engineering-Team, 10Machine-Learning-Team (Active Tasks), 10User-MarcoAurelio: Github's wikimedia/ores not mirroring to Gerrit's scoring/ores/ores - https://phabricator.wikimedia.org/T311390 (10MarcoAurelio) @elukey With the Phab SSH replication service fixed in T311624 I went ahea... [10:15:11] 10GitLab (Infrastructure), 10Data-Persistence-Backup, 10serviceops, 10serviceops-collab, and 2 others: Backups for GitLab - https://phabricator.wikimedia.org/T274463 (10Jelto) >>! In T274463#8045578, @Dzahn wrote: > unfortunately just noticed an Icinga alert for gitlab1003 (nothing mails us about this, th... [10:49:00] 10Release-Engineering-Team, 10serviceops, 10Patch-For-Review: PendingDeprecationWarning on update_version.py - https://phabricator.wikimedia.org/T310133 (10TheresNoTime) 05Open→03Stalled Stalling on figuring out resolution to [[ https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/803886/1#mes... [10:56:38] TheresNoTime: it's a debian package so you'd first need to know if any debian repo has a more up to date version [10:59:01] https://packages.debian.org/buster/python3-ruamel.yaml says yes [11:00:13] RhinosF1: if you wouldn't mind making a comment at the task, I'd really appreciate it ^^ (I'm in a "task update" mood at the moment! :P) [11:03:11] 10Release-Engineering-Team, 10serviceops, 10Patch-For-Review: PendingDeprecationWarning on update_version.py - https://phabricator.wikimedia.org/T310133 (10RhinosF1) >>! In T310133#8051077, @TheresNoTime wrote: > Stalling on figuring out resolution to [[ https://gerrit.wikimedia.org/r/c/operations/deployment... [11:03:13] TheresNoTime: done with my very low amount of wisdom beyond it's possible [11:03:25] TheresNoTime: the tests for that run in a docker container and install their dependencies form pypi via tox, why does the package versions on the runner host even matter? [11:03:32] s/form/from/ [11:04:10] tbh, I wondered why it wasn't failing anyway if it was using an incompatible version [11:04:20] taavi: 🤷 not looked at the merits of the comment past the point of "this stalls my task" :) [11:04:33] wait, I think I'm wrong [11:05:00] i didn't go deeper than does a new package exist [11:05:40] in addition to the tests, it's also used by pipelinelib which may very well run outside docker [11:05:46] which versiou do you actually need? [11:05:58] jeena: which does it need ^ [11:07:42] okay now looking at this closer, I don't think it does need updating? I certainly didn't update it locally when testing my change [11:09:20] https://gerrit.wikimedia.org/r/plugins/gitiles/operations/deployment-charts/+/refs/heads/master/update_version/requirements.txt nor is it locked? [11:10:05] (oh wait no we're talking two different packages here) [11:32:37] (03PS1) 10Hashar: zuul: skip selenium for Wikibase repo/rest-api [integration/config] - 10https://gerrit.wikimedia.org/r/811258 (https://phabricator.wikimedia.org/T307090) [11:36:19] (03CR) 10Hashar: [C: 03+2] zuul: skip selenium for Wikibase repo/rest-api [integration/config] - 10https://gerrit.wikimedia.org/r/811258 (https://phabricator.wikimedia.org/T307090) (owner: 10Hashar) [11:38:54] (03Merged) 10jenkins-bot: zuul: skip selenium for Wikibase repo/rest-api [integration/config] - 10https://gerrit.wikimedia.org/r/811258 (https://phabricator.wikimedia.org/T307090) (owner: 10Hashar) [11:39:35] !log Reloaded Zuul for `skip selenium for Wikibase repo/rest-api` https://gerrit.wikimedia.org/r/c/integration/config/+/811258 [11:39:36] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:44:50] GitLab needs a short maintenance break of around 10 minutes at 12:30 UTC (in 45 minutes) [12:19:20] 10Beta-Cluster-Infrastructure, 10ContentTranslation: Content security policy error when using ContentTranslation on the beta cluster - https://phabricator.wikimedia.org/T312092 (10awight) [12:19:22] (03PS1) 10Jaime Nuche: utc morning backport/training: add Jaime to scheduled window [tools/release] - 10https://gerrit.wikimedia.org/r/811293 [12:22:43] 10Beta-Cluster-Infrastructure, 10ContentTranslation: ContentTranslation can't retrieve wiki source on the beta cluster - https://phabricator.wikimedia.org/T312093 (10awight) [12:25:11] (03PS1) 10Hashar: zuul: skip the other selenium job for Wikibase repo/rest-api [integration/config] - 10https://gerrit.wikimedia.org/r/811297 (https://phabricator.wikimedia.org/T307090) [12:27:26] (03CR) 10Hashar: "Same as what we did earlier but for the generic job quibble-vendor-mysql-php72-selenium-docker" [integration/config] - 10https://gerrit.wikimedia.org/r/811297 (https://phabricator.wikimedia.org/T307090) (owner: 10Hashar) [12:54:19] jelto: gitlab was working a few minutes ago, but now getting 502 errors. Is the maintenance still ongoing? [12:55:16] GitLab should be available again. Took a little bit longer than expected, sorry for that! [12:55:21] kostajh: ^ [13:00:12] jelto: working again, yes. thanks! [13:06:52] 10Phabricator (Upstream), 10Release-Engineering-Team, 10Upstream, 10User-brennen: Uploaded files via the drag-and-drop are defaulting to private-access - https://phabricator.wikimedia.org/T310833 (10brennen) p:05High→03Medium @matmarex thanks for the documentation changes. I can probably investigate s... [13:37:40] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10SRE, 10serviceops, and 2 others: replace doc1001.eqiad.wmnet with a buster VM and create the codfw equivalent - https://phabricator.wikimedia.org/T247653 (10Volker_E) Same, thanks @Dzahn for making the OOUIPHP demos work again! [14:11:15] 10Release-Engineering-Team, 10Scap, 10Patch-For-Review: scap-o-scap: Bootstrapping a new host fails - https://phabricator.wikimedia.org/T310740 (10jnuche) It turns out the opening paragraph of the doc I pasted in my previous comment refers to lexical scope only. For the relationship to actually exist, you ne... [14:17:44] !log restarted mathoid service on deployment-docker-mathoid01 [14:17:46] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:51:43] 10Project-Admins, 10Stewards-and-global-tools: Decide on a project name for replacing T43492 "Steward, global sysop and SWMT tracking bug" - https://phabricator.wikimedia.org/T112891 (10FriedrickMILBarbarossa) [14:54:34] 10GitLab (CI & Job Runners), 10serviceops, 10serviceops-collab: DNS/networking not working on Trusted Runners - https://phabricator.wikimedia.org/T311241 (10Jelto) >>! In T311241#8042109, @Dzahn wrote: > currently the issue here is not DNS anymore. > > but it is now: 'This job is stuck because you don't ha... [15:20:36] 10GitLab (CI & Job Runners), 10serviceops, 10serviceops-collab: DNS/networking not working on Trusted Runners - https://phabricator.wikimedia.org/T311241 (10Jelto) I run the script in [repos/releng/gitlab-trusted-runner/](https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/) manually: ` $ add-pr... [15:57:35] (03PS1) 10Ahmon Dancy: Set SCAP_MW_LANG=en in environment in deploy container [tools/train-dev] - 10https://gerrit.wikimedia.org/r/811330 [16:11:18] (03CR) 10Thcipriani: [C: 03+2] "❤️ thanks jnuche" [tools/release] - 10https://gerrit.wikimedia.org/r/811293 (owner: 10Jaime Nuche) [16:12:03] (03Merged) 10jenkins-bot: utc morning backport/training: add Jaime to scheduled window [tools/release] - 10https://gerrit.wikimedia.org/r/811293 (owner: 10Jaime Nuche) [17:53:12] 10Phabricator, 10I18n, 10Patch-For-Review: Typo in Phabricator (Audit) in English - https://phabricator.wikimedia.org/T312053 (10Aklapper) [17:53:22] 10Phabricator, 10I18n, 10Patch-For-Review: Typo in Phabricator (Arcanist) in English - https://phabricator.wikimedia.org/T312052 (10Aklapper) [17:53:33] 10Phabricator, 10I18n, 10Patch-For-Review: 2 simple typos in English for Phabricator (Differential) - https://phabricator.wikimedia.org/T312051 (10Aklapper) [18:05:02] 10Release-Engineering-Team (Priority Backlog 📥), 10Release, 10Train Deployments: 1.39.0-wmf.21 deployment blockers - https://phabricator.wikimedia.org/T308074 (10Jdlrobson) [18:15:54] 10GitLab (Infrastructure), 10serviceops, 10Patch-For-Review: bring new gitlab hardware servers into production - https://phabricator.wikimedia.org/T307142 (10Dzahn) gitlab2001 has been removed from the acme_chief yaml, that allowed it to request certs but it's still up and has the puppet role applied. This... [18:24:09] (03CR) 10Ahmon Dancy: main._apache_sync_command: add missing parameter default value (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/785121 (owner: 10Jaime Nuche) [18:32:00] 10GitLab (Infrastructure), 10serviceops, 10Patch-For-Review: bring new gitlab hardware servers into production - https://phabricator.wikimedia.org/T307142 (10Dzahn) >>! In T307142#7967245, @Jelto wrote: > https://gitlab-replica.wikimedia.org/ will be migrated from `gitlab2001` (the old ganeti VM) to `gitlab1... [18:39:47] 10GitLab (Infrastructure), 10serviceops, 10Patch-For-Review: bring new gitlab hardware servers into production - https://phabricator.wikimedia.org/T307142 (10ops-monitoring-bot) cookbooks.sre.hosts.decommission executed by dzahn@cumin2002 for hosts: `gitlab2001.codfw.wmnet` - gitlab2001.codfw.wmnet (**FAIL**... [18:52:22] 10GitLab (Infrastructure), 10serviceops, 10Patch-For-Review: bring new gitlab hardware servers into production - https://phabricator.wikimedia.org/T307142 (10ops-monitoring-bot) cookbooks.sre.hosts.decommission executed by dzahn@cumin2002 for hosts: `gitlab2001.wikimedia.org` - gitlab2001.wikimedia.org (**PA... [18:56:02] 10GitLab (Infrastructure), 10serviceops, 10Patch-For-Review: bring new gitlab hardware servers into production - https://phabricator.wikimedia.org/T307142 (10Dzahn) [18:56:49] 10GitLab (Infrastructure), 10serviceops, 10Patch-For-Review: bring new gitlab hardware servers into production - https://phabricator.wikimedia.org/T307142 (10Dzahn) [18:57:10] 10Release-Engineering-Team, 10serviceops, 10Patch-For-Review: PendingDeprecationWarning on update_version.py - https://phabricator.wikimedia.org/T310133 (10jeena) I saw a question on IRC about which version is needed. According to the docs, https://pypi.org/project/ruamel.yaml/ version 0.16.0 or greater woul... [19:03:46] (03CR) 10Jeena Huneidi: [C: 03+2] "LGTM" [tools/train-dev] - 10https://gerrit.wikimedia.org/r/810083 (owner: 10Ahmon Dancy) [19:04:12] (03Merged) 10jenkins-bot: Disable build_mw_container_image to save testing time [tools/train-dev] - 10https://gerrit.wikimedia.org/r/810083 (owner: 10Ahmon Dancy) [19:13:24] (03CR) 10Jeena Huneidi: [C: 03+2] "LGTM" [tools/train-dev] - 10https://gerrit.wikimedia.org/r/810082 (owner: 10Ahmon Dancy) [19:13:52] (03Merged) 10jenkins-bot: train-dev ssh: Use login shell [tools/train-dev] - 10https://gerrit.wikimedia.org/r/810082 (owner: 10Ahmon Dancy) [19:34:14] (03CR) 10Jeena Huneidi: [C: 03+2] "LGTM" [tools/train-dev] - 10https://gerrit.wikimedia.org/r/810084 (owner: 10Ahmon Dancy) [19:34:54] (03Merged) 10jenkins-bot: train-dev: Another round of removal of relative paths [tools/train-dev] - 10https://gerrit.wikimedia.org/r/810084 (owner: 10Ahmon Dancy) [19:45:09] (03PS1) 10Kosta Harlan: VueTest: Add to extension-codehealth pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/811368 [19:52:01] (03CR) 10Jeena Huneidi: [C: 03+2] "tested, LGTM" [tools/train-dev] - 10https://gerrit.wikimedia.org/r/811330 (owner: 10Ahmon Dancy) [19:52:27] (03Merged) 10jenkins-bot: Set SCAP_MW_LANG=en in environment in deploy container [tools/train-dev] - 10https://gerrit.wikimedia.org/r/811330 (owner: 10Ahmon Dancy) [20:18:43] 10MediaWiki-Releasing, 10MediaWiki-Vendor, 10MW-1.37-release: Wrong version of guzzle shipped with 1.37.3 (breaks update.php) - https://phabricator.wikimedia.org/T312059 (10Aklapper) [20:25:07] 10GitLab (Infrastructure), 10Data-Persistence-Backup, 10serviceops, 10serviceops-collab, and 2 others: Backups for GitLab - https://phabricator.wikimedia.org/T274463 (10Dzahn) >>! In T274463#8051032, @Jelto wrote: > I reverted the change which introduced the Blackbox exporter in https://gerrit.wikimedia.or... [20:37:18] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MediaWiki-Stakeholders-Group, 10Epic, 10MW-1.39-release: Expand the set of bundled extensions and skins in MediaWiki 1.39 - https://phabricator.wikimedia.org/T305072 (10Bawolff) There was recently an irc discussion about adding checkuser to bundled extensio... [20:43:59] (03CR) 10Chad: [C: 03+2] scap prep: Ensure umask is 002 before running [tools/scap] - 10https://gerrit.wikimedia.org/r/809708 (owner: 10Ahmon Dancy) [20:44:53] (03CR) 10Chad: [C: 03+2] scap prep chmod 777 cache dir bugfix [tools/scap] - 10https://gerrit.wikimedia.org/r/810396 (owner: 10Ahmon Dancy) [20:45:27] (03CR) 10Chad: [C: 03+2] Move serializing_lock_file to a setgid directory [tools/scap] - 10https://gerrit.wikimedia.org/r/810397 (https://phabricator.wikimedia.org/T310395) (owner: 10Ahmon Dancy) [20:48:47] (03PS1) 10Ahmon Dancy: Clean up php fpm restart [tools/scap] - 10https://gerrit.wikimedia.org/r/811373 [20:49:54] (03Merged) 10jenkins-bot: scap prep: Ensure umask is 002 before running [tools/scap] - 10https://gerrit.wikimedia.org/r/809708 (owner: 10Ahmon Dancy) [20:49:56] (03Merged) 10jenkins-bot: scap prep chmod 777 cache dir bugfix [tools/scap] - 10https://gerrit.wikimedia.org/r/810396 (owner: 10Ahmon Dancy) [20:50:59] (03Merged) 10jenkins-bot: Move serializing_lock_file to a setgid directory [tools/scap] - 10https://gerrit.wikimedia.org/r/810397 (https://phabricator.wikimedia.org/T310395) (owner: 10Ahmon Dancy) [20:52:48] (03PS2) 10Ahmon Dancy: Clean up php fpm restart [tools/scap] - 10https://gerrit.wikimedia.org/r/811373 [20:58:16] (03PS3) 10Ahmon Dancy: Clean up php fpm restart [tools/scap] - 10https://gerrit.wikimedia.org/r/811373 (https://phabricator.wikimedia.org/T266055) [21:03:40] 10Release-Engineering-Team, 10serviceops, 10Patch-For-Review: PendingDeprecationWarning on update_version.py - https://phabricator.wikimedia.org/T310133 (10Dzahn) @jeena Debian buster has 0.15.34 and Debian bullseye has 0.16.12. The puppet line "ensure_packages('python3-ruamel.yaml')" just tells it to make... [21:19:38] 10MediaWiki-Releasing, 10MediaWiki-Vendor, 10MW-1.37-release: Wrong version of guzzle shipped with 1.37.3 (breaks update.php) - https://phabricator.wikimedia.org/T312059 (10Legoktm) Ugh, we should probably add a check for this in make-release. I'll work on that later... @Reedy could we get a maintenance rel... [21:21:37] 10MediaWiki-Releasing, 10MediaWiki-Vendor, 10MW-1.37-release: Wrong version of guzzle shipped with 1.37.3 (breaks update.php) - https://phabricator.wikimedia.org/T312059 (10Reedy) >>! In T312059#8053776, @Legoktm wrote: > Ugh, we should probably add a check for this in make-release. I'll work on that later..... [21:34:51] 10Release-Engineering-Team (Priority Backlog 📥), 10Release, 10Train Deployments: 1.39.0-wmf.21 deployment blockers - https://phabricator.wikimedia.org/T308074 (10matmarex) [23:46:12] 10Release-Engineering-Team, 10serviceops, 10Patch-For-Review: PendingDeprecationWarning on update_version.py - https://phabricator.wikimedia.org/T310133 (10jeena) @Dzahn I was only commenting to provide information that in order to merge the patch in question, a different version would be needed, and finding... [23:54:22] 10Release-Engineering-Team, 10serviceops, 10Patch-For-Review: PendingDeprecationWarning on update_version.py - https://phabricator.wikimedia.org/T310133 (10Dzahn) @jeena And I just wanted to answer the question I saw "I'm not sure how to update it." literally. That is what is needed to update it. Also doesn'... [23:58:45] 10Release-Engineering-Team, 10serviceops, 10Patch-For-Review: PendingDeprecationWarning on update_version.py - https://phabricator.wikimedia.org/T310133 (10jeena) 👍 thanks @Dzahn, sorry for the misunderstanding