[01:25:39] 10Release-Engineering-Team (GitLab V: Event Horizon 🌄), 10Release, 10Train Deployments: 1.40.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T325588 (10Tgr) If you encounter frequent PageViewInfo-related timeouts on Wednesday, see T327920#8647335 (not related to the train in any way but coul... [05:39:14] 10Phabricator: No document engine can render the contents of this file - https://phabricator.wikimedia.org/T330618 (10I) [05:40:19] 10Phabricator: No document engine can render the contents of this file - https://phabricator.wikimedia.org/T330618 (10I) [05:45:43] 10Phabricator: No document engine can render the contents of this file - https://phabricator.wikimedia.org/T330618 (10I) [08:05:44] GitLab will be down at 10:00 UTC for around 2 hours. The active host will be switched to the other datacenter. See https://phabricator.wikimedia.org/T329931 for more information. [08:24:11] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) [09:02:30] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team, 10Patch-For-Review: Upgrade Jenkins to latest LTS 2.375.3 - https://phabricator.wikimedia.org/T330045 (10hashar) Checking https://debmonitor.wikimedia.org/packages/jenkins , the two releases Jenkins got upgraded to 2.375.3. I g... [09:19:55] 10Phabricator: No document engine can render the contents of this file - https://phabricator.wikimedia.org/T330618 (10Aklapper) p:05Triage→03Low We don't plan to install document engines for every potential file format. This is a H264 file. Cisco's open source H.264 encoder could be used server-side. [09:20:28] 10Phabricator: MP4/H264 rendering support in Phabricator ("No document engine can render the contents of this file") - https://phabricator.wikimedia.org/T330618 (10Aklapper) [09:27:01] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team, 10Patch-For-Review: Upgrade Jenkins to latest LTS 2.375.3 - https://phabricator.wikimedia.org/T330045 (10hashar) 05Open→03Resolved a:03hashar Upgraded with @jnuche {F36874036 size=full} [09:35:40] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) [09:54:10] 10Phabricator, 10PM, 10Patch-For-Review: Merge the Phabricator Priority values "Low" and "Lowest" - https://phabricator.wikimedia.org/T228759 (10Aklapper) > It doesn't speak to the value of distinguishing tasks around product alignment, I'm not sure if I get "product alignment" correctly then - if some task... [09:55:20] 10Phabricator, 10PM, 10Patch-For-Review: Merge the Phabricator Priority values "Low" and "Lowest" - https://phabricator.wikimedia.org/T228759 (10hashar) [09:59:50] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) a:03eoghan [10:03:54] 10Phabricator, 10PM, 10Patch-For-Review: Merge the Phabricator Priority values "Low" and "Lowest" - https://phabricator.wikimedia.org/T228759 (10hashar) I took the liberty to slightly enhance the task description making use of `{nav Low}` and `{nav Lowest}` to make them stand out a bit in the text. My under... [10:06:36] GitLab maintenance starting now [10:07:30] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) [10:18:18] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) [10:19:53] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) [10:28:59] 10GitLab, 10serviceops-collab: Clarify the process of logging into gitlab-replica using 2FA - https://phabricator.wikimedia.org/T330639 (10LSobanski) [10:30:41] 10GitLab, 10serviceops-collab: Clarify the process of logging into gitlab-replica using 2FA - https://phabricator.wikimedia.org/T330639 (10LSobanski) p:05Triage→03Low [11:03:23] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) [11:25:39] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) [11:36:55] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) [11:51:24] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) [11:58:10] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) [12:00:40] Maintenance finished, GitLab is back again. If you face any issues feel free to comment in T329931. [12:00:40] T329931: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 [12:02:26] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) [12:03:46] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) Maintenance finished, GitLab is back again. If you face any issues feel free to post it here. I'm leaving the task... [12:38:27] (03PS1) 10Zoranzoki21: Zuul: Archive the SecureSessions extension [integration/config] - 10https://gerrit.wikimedia.org/r/892435 (https://phabricator.wikimedia.org/T327583) [12:40:53] (03CR) 10Hashar: "The dependency should be the other way around. We want CI to be configured first, then recheck the Metrolook pending change ;)" [integration/config] - 10https://gerrit.wikimedia.org/r/891968 (owner: 10Jack Phoenix) [12:45:40] (03PS3) 10Hashar: zuul: [mediawiki/skins/Metrolook] Add SocialProfile as a phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/891968 (owner: 10Jack Phoenix) [12:46:27] (03CR) 10Hashar: [C: 03+2] "I have read it wrong, the header was Needed-By, not Depends-On. The CI failure was because the commit message validator does not recogniz" [integration/config] - 10https://gerrit.wikimedia.org/r/891968 (owner: 10Jack Phoenix) [12:47:38] (03Merged) 10jenkins-bot: zuul: [mediawiki/skins/Metrolook] Add SocialProfile as a phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/891968 (owner: 10Jack Phoenix) [12:50:43] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/891968 [12:50:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:59:16] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) A "Backup freshness" alert with summary:"No backups: 1 (gitlab2002), Fresh: 116 jobs"" triggered. This should resol... [12:59:58] (03CR) 10Hashar: [C: 03+2] "Thank you!" [integration/config] - 10https://gerrit.wikimedia.org/r/892435 (https://phabricator.wikimedia.org/T327583) (owner: 10Zoranzoki21) [13:01:10] (03Merged) 10jenkins-bot: Zuul: Archive the SecureSessions extension [integration/config] - 10https://gerrit.wikimedia.org/r/892435 (https://phabricator.wikimedia.org/T327583) (owner: 10Zoranzoki21) [13:02:43] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/892435 | T327583 [13:02:45] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:02:45] T327583: Archive the SecureSessions extension - https://phabricator.wikimedia.org/T327583 [13:31:55] (03PS1) 10Zoranzoki21: Zuul: Archive the CatTalk extension [integration/config] - 10https://gerrit.wikimedia.org/r/892440 (https://phabricator.wikimedia.org/T305636) [14:30:48] (03CR) 10Hashar: [C: 03+2] Zuul: Archive the CatTalk extension [integration/config] - 10https://gerrit.wikimedia.org/r/892440 (https://phabricator.wikimedia.org/T305636) (owner: 10Zoranzoki21) [14:31:53] (03Merged) 10jenkins-bot: Zuul: Archive the CatTalk extension [integration/config] - 10https://gerrit.wikimedia.org/r/892440 (https://phabricator.wikimedia.org/T305636) (owner: 10Zoranzoki21) [14:41:01] (03PS1) 10Jforrester: build: Pin PHPUnit to 9.5.28 like in other repos [integration/docroot] - 10https://gerrit.wikimedia.org/r/892452 [14:47:03] (03PS1) 10Zoranzoki21: Zuul: Archive the Carp extension [integration/config] - 10https://gerrit.wikimedia.org/r/892458 (https://phabricator.wikimedia.org/T299283) [14:48:59] 10Release-Engineering-Team, 10Gerrit (Gerrit 3.5): Upgrade Gerrit from 3.5.4 to 3.5.5 - https://phabricator.wikimedia.org/T330663 (10hashar) [14:59:18] 10Release-Engineering-Team, 10Gerrit (Gerrit 3.5): Upgrade Gerrit from 3.5.4 to 3.5.5 - https://phabricator.wikimedia.org/T330663 (10hashar) [15:01:13] 10Release-Engineering-Team, 10Gerrit (Gerrit 3.5): Upgrade Gerrit from 3.5.4 to 3.5.5 - https://phabricator.wikimedia.org/T330663 (10hashar) There are also some updates to sshd (new version is 2.9.2) which might address some issues due to newer keys or old algo being disabled on some distro. [15:05:55] (03CR) 10Hashar: [C: 03+2] Zuul: Archive the Carp extension [integration/config] - 10https://gerrit.wikimedia.org/r/892458 (https://phabricator.wikimedia.org/T299283) (owner: 10Zoranzoki21) [15:06:57] (03Merged) 10jenkins-bot: Zuul: Archive the Carp extension [integration/config] - 10https://gerrit.wikimedia.org/r/892458 (https://phabricator.wikimedia.org/T299283) (owner: 10Zoranzoki21) [15:08:24] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/892458 | T299283 [15:08:26] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:08:27] T299283: Archive the Carp extension - https://phabricator.wikimedia.org/T299283 [15:14:36] (03CR) 10Jforrester: [C: 03+1] "Have fun!" [integration/config] - 10https://gerrit.wikimedia.org/r/891857 (owner: 10Hashar) [15:58:05] (03CR) 10Hashar: [C: 03+2] "Thanks and lets try!" [integration/config] - 10https://gerrit.wikimedia.org/r/891857 (owner: 10Hashar) [15:59:12] (03Merged) 10jenkins-bot: tox: unpin python-jenkins [integration/config] - 10https://gerrit.wikimedia.org/r/891857 (owner: 10Hashar) [15:59:32] 10GitLab (Infrastructure), 10serviceops-collab: Investigate gitlab maintenance/read-only mode - https://phabricator.wikimedia.org/T330277 (10eoghan) 05Open→03Resolved [16:00:04] 10GitLab (Infrastructure), 10serviceops-collab: Investigate gitlab maintenance/read-only mode - https://phabricator.wikimedia.org/T330277 (10eoghan) Nothing further to do here for the time being, we have options that we can use and notes on where to look in the future if we want to improve the user experience... [16:01:18] (03CR) 10Hashar: [C: 03+2] build: Pin PHPUnit to 9.5.28 like in other repos [integration/docroot] - 10https://gerrit.wikimedia.org/r/892452 (owner: 10Jforrester) [16:01:58] (03Merged) 10jenkins-bot: build: Pin PHPUnit to 9.5.28 like in other repos [integration/docroot] - 10https://gerrit.wikimedia.org/r/892452 (owner: 10Jforrester) [16:28:15] 10Beta-Cluster-Infrastructure, 10Discovery-Search: Puppet not running on deployment-awx-* - https://phabricator.wikimedia.org/T329792 (10MPhamWMF) p:05Triage→03Medium [16:58:32] 10Gerrit, 10serviceops-collab, 10Patch-For-Review: Issues with Gerrit test instance - https://phabricator.wikimedia.org/T329444 (10Jelto) a:03Dzahn @Dzahn can you follow up on the task status? [16:58:58] Is it just me, or does adding a comment in gerrit cause browser cpu usage to spike to 100%/everything become laggy for everyone? [16:59:10] 10GitLab (Infrastructure), 10serviceops-collab: gitlab_default_can_create_group setting deprecation - https://phabricator.wikimedia.org/T330282 (10Jelto) p:05Triage→03Medium a:03eoghan [17:00:41] bawolff: I don't have the issue. If it always trigger maybe a profiling / the browser performance tab has some info? [17:22:39] 10Beta-Cluster-Infrastructure, 10Discovery-Search: Puppet not running on deployment-awx-* - https://phabricator.wikimedia.org/T329792 (10bking) I deleted these instances. Closing... [17:22:56] 10Beta-Cluster-Infrastructure, 10Discovery-Search: Puppet not running on deployment-awx-* - https://phabricator.wikimedia.org/T329792 (10bking) 05Open→03Resolved [17:35:17] 10Release-Engineering-Team (Seen), 10Scap: Running "scap pull" on deploy1001 halts for 2 minutes, then reports a php-fpm error - https://phabricator.wikimedia.org/T246959 (10dancy) [18:04:30] (03PS1) 10Pwangai: Zuul: [mediawiki/extensions/Linter] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/892523 (https://phabricator.wikimedia.org/T321837) [18:31:07] (03PS1) 10Pwangai: Zuul: [mediawiki/extensions/WikiEditor] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/892528 (https://phabricator.wikimedia.org/T321837) [18:31:44] (03PS1) 10Zoranzoki21: Zuul: Archive two extensions (MolHandler and StlHandler) [integration/config] - 10https://gerrit.wikimedia.org/r/892530 (https://phabricator.wikimedia.org/T299158) [18:56:39] 10Continuous-Integration-Config, 10Accessibility, 10MinervaNeue (Tracking), 10User-zeljkofilipin: Save artifacts in Jenkins for selenium daily Minerva - https://phabricator.wikimedia.org/T328684 (10bwang) 05Open→03Resolved [18:56:52] 10Continuous-Integration-Config, 10Accessibility, 10MinervaNeue (Tracking), 10User-zeljkofilipin: Save artifacts in Jenkins for selenium daily Minerva - https://phabricator.wikimedia.org/T328684 (10bwang) oh yep looks like its working after all, thank you! [19:35:31] (03PS1) 10Jforrester: Zuul: [mediawiki/libs/Dodo] Make CI voting on PHP 8.1 [integration/config] - 10https://gerrit.wikimedia.org/r/892546 (https://phabricator.wikimedia.org/T311462) [19:35:33] (03PS1) 10Jforrester: Zuul: [mediawiki/libs/Dodo] Switch to library template [integration/config] - 10https://gerrit.wikimedia.org/r/892547 [19:36:35] (03CR) 10CI reject: [V: 04-1] Zuul: [mediawiki/libs/Dodo] Make CI voting on PHP 8.1 [integration/config] - 10https://gerrit.wikimedia.org/r/892546 (https://phabricator.wikimedia.org/T311462) (owner: 10Jforrester) [19:38:24] (03PS1) 10Jforrester: Zuul: [mediawiki/tools/phan/PerfCheckPlugin] Collapse template [integration/config] - 10https://gerrit.wikimedia.org/r/892548 [19:39:56] (03PS2) 10Jforrester: Zuul: [mediawiki/libs/Dodo] Make CI voting on PHP 8.1 [integration/config] - 10https://gerrit.wikimedia.org/r/892546 (https://phabricator.wikimedia.org/T311462) [19:44:36] 10Release-Engineering-Team (Seen), 10Wikidata, 10wdwb-tech, 10MW-1.35-notes (1.35.0-wmf.16; 2020-01-21), 10Technical-Debt: Stop using jsonlint and instead use eslint-plugin-json for the linting - https://phabricator.wikimedia.org/T220036 (10Kizule) No patches to review so far. [20:24:01] 10Phabricator: Custom task form for session template for Wikimedia Hackathon 2023 - https://phabricator.wikimedia.org/T330699 (10srishakatux) [20:51:24] 10Phabricator: Custom task form for session template for Wikimedia Hackathon 2023 - https://phabricator.wikimedia.org/T330699 (10Aklapper) @srishakatux: Hi, if a form is wanted, then please follow https://www.mediawiki.org/wiki/Phabricator/Help/Forms#Creating_custom_forms and provide the information requested in... [21:10:46] 10Continuous-Integration-Config, 10Accessibility, 10MinervaNeue (Tracking), 10User-zeljkofilipin: Save artifacts in Jenkins for selenium daily Minerva - https://phabricator.wikimedia.org/T328684 (10hashar) You are welcome @bwang ;) [21:12:12] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (GitLab V: Event Horizon 🌄), 10serviceops-collab, 10serviceops-radar: Set up mirror of the docker hub registry for gitlab-runners - https://phabricator.wikimedia.org/T329679 (10dancy) p:05Triage→03Medium a:03dancy [21:17:09] 10Gerrit, 10GitLab (Project Migration), 10Release-Engineering-Team (GitLab V: Event Horizon 🌄), 10User-brennen: Script closing/archiving of migrated repositories on Gerrit - https://phabricator.wikimedia.org/T330345 (10hashar) This task is similar to what is done when archiving Gerrit project as part of th... [21:20:20] 10Phabricator: Custom task form for session template for Wikimedia Hackathon 2023 - https://phabricator.wikimedia.org/T330699 (10srishakatux) >>! In T330699#8650574, @Aklapper wrote: > @srishakatux: Hi, if a form is wanted, then please follow https://www.mediawiki.org/wiki/Phabricator/Help/Forms#Creating_custom_... [21:23:15] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (GitLab V: Event Horizon 🌄), 10serviceops-collab, 10serviceops-radar: Set up mirror of the docker hub registry for gitlab-runners - https://phabricator.wikimedia.org/T329679 (10dancy) [21:46:50] 10Gerrit, 10GitLab (Project Migration), 10Release-Engineering-Team (GitLab V: Event Horizon 🌄), 10User-Kizule, 10User-brennen: Script closing/archiving of migrated repositories on Gerrit - https://phabricator.wikimedia.org/T330345 (10Kizule) [22:16:16] 10GitLab (Auth & Access), 10serviceops-collab: Clarify the process of logging into gitlab-replica using 2FA - https://phabricator.wikimedia.org/T330639 (10brennen)