[00:13:37] 10Phabricator: Custom task form for session template for Wikimedia Hackathon 2023 - https://phabricator.wikimedia.org/T330699 (10Aklapper) 05Open→03Declined > the url with parameters that you generated is all what I needed. In that case I'm declining this task but if i misunderstood please do reopen! Thanks :) [04:20:42] 10phabricator maintenance bot: Maintenance bot creates duplicates for new wiki tasks - https://phabricator.wikimedia.org/T286740 (10JJMC89) [07:41:16] PROBLEM - Check systemd state on doc1002 is CRITICAL: CRITICAL - degraded: The following units failed: rsync-doc-doc2001.codfw.wmnet.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [08:27:31] 10GitLab (Infrastructure), 10serviceops-collab, 10Patch-For-Review: ProbeDown - https://phabricator.wikimedia.org/T330717 (10Jelto) [08:33:22] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) As mentioned in T330717, the new production host `gitlab2002` still had the restore enabled and executed a restore... [08:38:14] RECOVERY - Check systemd state on doc1002 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [09:01:42] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) I started a incident report in [2023-02-28_GitLab_data_loss](https://wikitech.wikimedia.org/wiki/Incidents/2023-02-... [09:14:29] (03CR) 10Hashar: [C: 03+2] Zuul: Archive two extensions (MolHandler and StlHandler) [integration/config] - 10https://gerrit.wikimedia.org/r/892530 (https://phabricator.wikimedia.org/T299158) (owner: 10Zoranzoki21) [09:15:57] (03Merged) 10jenkins-bot: Zuul: Archive two extensions (MolHandler and StlHandler) [integration/config] - 10https://gerrit.wikimedia.org/r/892530 (https://phabricator.wikimedia.org/T299158) (owner: 10Zoranzoki21) [09:16:37] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/892530 | T299158 | T299161 [09:16:40] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:16:40] T299158: Archive the MolHandler extension - https://phabricator.wikimedia.org/T299158 [09:16:41] T299161: Archive the StlHandler extension - https://phabricator.wikimedia.org/T299161 [09:24:41] 10Continuous-Integration-Infrastructure, 10Quibble, 10Developer Productivity: Provide early feedback when a patch has job failures - https://phabricator.wikimedia.org/T323750 (10hashar) a:03kostajh [09:25:58] 10Continuous-Integration-Config, 10Quibble: Update quibble jobs to run maintenance scripts via maintenance/run.php - https://phabricator.wikimedia.org/T326333 (10hashar) 05Open→03Resolved Quibble 1.5.0 uses `maintenance/run.php` when it exists (else fallback to directly invoke the requested PHP script). It... [10:06:47] (03PS1) 10Zoranzoki21: Zuul: Archive the BlockAndNuke extension [integration/config] - 10https://gerrit.wikimedia.org/r/892910 (https://phabricator.wikimedia.org/T299262) [10:16:32] (03CR) 10Hashar: [C: 03+2] Zuul: Archive the BlockAndNuke extension [integration/config] - 10https://gerrit.wikimedia.org/r/892910 (https://phabricator.wikimedia.org/T299262) (owner: 10Zoranzoki21) [10:17:42] (03Merged) 10jenkins-bot: Zuul: Archive the BlockAndNuke extension [integration/config] - 10https://gerrit.wikimedia.org/r/892910 (https://phabricator.wikimedia.org/T299262) (owner: 10Zoranzoki21) [10:19:38] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/892910/ | T299262 [10:19:40] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:19:41] T299262: Archive the BlockAndNuke extension - https://phabricator.wikimedia.org/T299262 [10:34:43] 10Gerrit, 10Upstream: Cloning commit message hooks from Gerrit via SSH/SCP isn't working on MacOS - https://phabricator.wikimedia.org/T330740 (10Kizule) [10:35:23] 10Gerrit, 10Upstream: Cloning commit message hooks from Gerrit via SSH/SCP isn't working on MacOS - https://phabricator.wikimedia.org/T330740 (10Kizule) [10:35:43] 10Gerrit, 10Upstream, 10User-Kizule: Cloning commit message hooks from Gerrit via SSH/SCP isn't working on MacOS - https://phabricator.wikimedia.org/T330740 (10Kizule) a:03Kizule [10:36:00] 10Gerrit, 10Upstream: Cloning commit message hooks from Gerrit via SSH/SCP isn't working on MacOS - https://phabricator.wikimedia.org/T330740 (10Kizule) a:05Kizule→03None [10:38:14] 10GitLab (Infrastructure), 10serviceops-collab, 10Patch-For-Review: ProbeDown - https://phabricator.wikimedia.org/T330717 (10Jelto) 05Open→03Resolved After merging the above change the restore timer is gone form `gitlab2002`. So we should not see a ProbeDown alert again due to restores on the production... [10:40:38] 10phabricator maintenance bot: Maintenance bot creates duplicates for new wiki tasks - https://phabricator.wikimedia.org/T286740 (10Ladsgroup) A simple solution would be to check wiki exists (by hitting e.g. az.wikimedia.org and making sure it's not 404) before creating them, if the wiki exists, it shouldn't cre... [10:40:56] 10Gerrit, 10Upstream: Cloning commit message hooks from Gerrit via SSH/SCP isn't working on MacOS - https://phabricator.wikimedia.org/T330740 (10Kizule) [10:41:19] (03PS1) 10Zoranzoki21: Zuul: Archive the News extension [integration/config] - 10https://gerrit.wikimedia.org/r/892919 (https://phabricator.wikimedia.org/T299263) [10:44:10] 10Gerrit, 10Upstream: Cloning commit message hooks from Gerrit via SSH/SCP isn't working on MacOS - https://phabricator.wikimedia.org/T330740 (10hashar) This is due to OpenSSH 9 making `scp` to use `sftp` which is not supported by Gerrit. The workaround is to fetch the commit hook over https. [10:44:26] 10Continuous-Integration-Infrastructure, 10Quibble, 10Developer Productivity: Create customized Gerrit label for "All-Jobs-Passed" - https://phabricator.wikimedia.org/T330741 (10kostajh) [10:44:29] 10Gerrit, 10Upstream: Cloning commit message hooks from Gerrit via SSH/SCP isn't working on MacOS - https://phabricator.wikimedia.org/T330740 (10hashar) [10:55:28] 10Continuous-Integration-Infrastructure, 10Quibble, 10Developer Productivity: Provide early feedback when a patch has job failures - https://phabricator.wikimedia.org/T323750 (10kostajh) [10:56:02] 10Continuous-Integration-Infrastructure, 10Quibble, 10Developer Productivity, 10Patch-For-Review: Create customized Gerrit label for "All-Jobs-Passed" - https://phabricator.wikimedia.org/T330741 (10kostajh) 05Open→03Declined On second thought, it would be confusing to have this label always display, ev... [11:19:23] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10Jelto) The rsync jobs between production host and replica are only created but not removed, when the list of replicas chan... [11:57:33] 10GitLab: gitlab backup timer failing - https://phabricator.wikimedia.org/T330744 (10jbond) 05Open→03In progress p:05Triage→03Medium [12:01:36] 10GitLab, 10serviceops-collab: gitlab backup timer failing - https://phabricator.wikimedia.org/T330744 (10Jelto) thanks for keeping an eye on that! I also posted this issue to the gitlab switchover task: T329931#8652138 I would be happy to review a patch! Thanks a lot. [12:09:30] (03CR) 10Hashar: [C: 03+2] Zuul: Archive the News extension [integration/config] - 10https://gerrit.wikimedia.org/r/892919 (https://phabricator.wikimedia.org/T299263) (owner: 10Zoranzoki21) [12:10:39] (03Merged) 10jenkins-bot: Zuul: Archive the News extension [integration/config] - 10https://gerrit.wikimedia.org/r/892919 (https://phabricator.wikimedia.org/T299263) (owner: 10Zoranzoki21) [12:11:53] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/892919 | T299263 [12:11:55] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:11:55] T299263: Archive the News extension - https://phabricator.wikimedia.org/T299263 [12:12:09] 10GitLab, 10serviceops-collab: gitlab backup timer failing - https://phabricator.wikimedia.org/T330744 (10jbond) [12:13:29] 10GitLab, 10serviceops-collab, 10Patch-For-Review: gitlab backup timer failing - https://phabricator.wikimedia.org/T330744 (10jbond) [12:13:34] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10jbond) [12:14:25] 10GitLab, 10serviceops-collab, 10Patch-For-Review: gitlab backup timer failing - https://phabricator.wikimedia.org/T330744 (10jbond) >>! In T330744#8652227, @Jelto wrote: > thanks for keeping an eye on that! > Oncall so just cleaning up active alerts :) > I also posted this issue to the gitlab switchover t... [12:50:00] (03PS1) 10Zoranzoki21: Zuul: Archive the Ids extension [integration/config] - 10https://gerrit.wikimedia.org/r/892952 (https://phabricator.wikimedia.org/T296348) [13:02:38] (03CR) 10Hashar: [C: 03+2] Zuul: Archive the Ids extension [integration/config] - 10https://gerrit.wikimedia.org/r/892952 (https://phabricator.wikimedia.org/T296348) (owner: 10Zoranzoki21) [13:03:43] (03Merged) 10jenkins-bot: Zuul: Archive the Ids extension [integration/config] - 10https://gerrit.wikimedia.org/r/892952 (https://phabricator.wikimedia.org/T296348) (owner: 10Zoranzoki21) [13:06:17] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/892952 | T296348 [13:06:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:06:20] T296348: Archive the Ids extension - https://phabricator.wikimedia.org/T296348 [13:13:52] 10GitLab, 10serviceops-collab: gitlab backup timer failing - https://phabricator.wikimedia.org/T330744 (10Jelto) Time/jobs for `gitlab2002` were removed on `gitlab1004`: ` Notice: /Stage[main]/Gitlab::Rsync/Systemd::Timer::Job[rsync-data-backup-gitlab2002.wikimedia.org]/Systemd::Timer[rsync-data-backup-gitlab... [13:27:49] (03CR) 10Hashar: [C: 03+1] Zuul: [mediawiki/tools/phan/PerfCheckPlugin] Collapse template [integration/config] - 10https://gerrit.wikimedia.org/r/892548 (owner: 10Jforrester) [13:30:17] (03CR) 10Hashar: [C: 03+2] Zuul: [mediawiki/extensions/Linter] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/892523 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [13:30:22] (03CR) 10Hashar: [C: 03+2] Zuul: [mediawiki/extensions/WikiEditor] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/892528 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [13:31:13] 10Continuous-Integration-Infrastructure, 10Quibble, 10Developer Productivity: Quibble: Include the failed command in data passed to external endpoint - https://phabricator.wikimedia.org/T330750 (10kostajh) [13:31:30] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/Linter] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/892523 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [13:31:34] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/WikiEditor] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/892528 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [13:43:36] (03CR) 10Hashar: [C: 03+2] "Deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/892523 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [13:43:41] (03CR) 10Hashar: [C: 03+2] "Deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/892528 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [13:51:23] (03CR) 10Hashar: [C: 03+2] "https://gerrit.wikimedia.org/r/c/mediawiki/libs/Dodo/+/892545 passed the experimental pipeline / php 8.1" [integration/config] - 10https://gerrit.wikimedia.org/r/892546 (https://phabricator.wikimedia.org/T311462) (owner: 10Jforrester) [13:54:42] (03CR) 10Hashar: [C: 03+2] Zuul: [mediawiki/libs/Dodo] Make CI voting on PHP 8.1 [integration/config] - 10https://gerrit.wikimedia.org/r/892546 (https://phabricator.wikimedia.org/T311462) (owner: 10Jforrester) [13:54:45] 10Gerrit, 10Wikidata, 10Wikidata Query Builder, 10wdwb-tech, 10wmde-wikidata-tech: Remove master branch from wikidata/query-builder/deploy.git - https://phabricator.wikimedia.org/T330752 (10Lucas_Werkmeister_WMDE) [13:55:52] (03Merged) 10jenkins-bot: Zuul: [mediawiki/libs/Dodo] Make CI voting on PHP 8.1 [integration/config] - 10https://gerrit.wikimedia.org/r/892546 (https://phabricator.wikimedia.org/T311462) (owner: 10Jforrester) [13:56:29] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/892546 | T311462 [13:56:31] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:56:31] T311462: Re-enable PHP 8.1 CI on Dodo - https://phabricator.wikimedia.org/T311462 [13:56:43] (03PS2) 10Hashar: Zuul: [mediawiki/libs/Dodo] Switch to library template [integration/config] - 10https://gerrit.wikimedia.org/r/892547 (owner: 10Jforrester) [13:57:02] (03PS2) 10Hashar: Zuul: [mediawiki/tools/phan/PerfCheckPlugin] Collapse template [integration/config] - 10https://gerrit.wikimedia.org/r/892548 (owner: 10Jforrester) [13:58:30] (03CR) 10Hashar: "I think we can keep the `composer-test-package-php74-to-php80` template even if it has a single use since it is clearly expressed which ph" [integration/config] - 10https://gerrit.wikimedia.org/r/892548 (owner: 10Jforrester) [13:59:07] (03CR) 10Hashar: [C: 03+2] "No change in Zuul config diff :]" [integration/config] - 10https://gerrit.wikimedia.org/r/892547 (owner: 10Jforrester) [14:00:15] (03Merged) 10jenkins-bot: Zuul: [mediawiki/libs/Dodo] Switch to library template [integration/config] - 10https://gerrit.wikimedia.org/r/892547 (owner: 10Jforrester) [14:07:20] (03CR) 10Hashar: [C: 03+2] jjb: Switch quibble jobs to images based on the node14 image [integration/config] - 10https://gerrit.wikimedia.org/r/891742 (owner: 10Jforrester) [14:08:10] !log Updating Quibble jobs for https://gerrit.wikimedia.org/r/c/integration/config/+/891742 (node 14 shipped from the node14 images instead of being manually installed at build time) [14:08:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:08:25] so well [14:08:30] (03Merged) 10jenkins-bot: jjb: Switch quibble jobs to images based on the node14 image [integration/config] - 10https://gerrit.wikimedia.org/r/891742 (owner: 10Jforrester) [14:08:35] it is 3pm local time, and I have send my full day solely doing reviews :-\ [14:10:58] 10Continuous-Integration-Config, 10Patch-For-Review: Re-enable PHP 8.1 CI on Dodo - https://phabricator.wikimedia.org/T311462 (10hashar) [14:11:25] 10Continuous-Integration-Config: Re-enable PHP 8.1 CI on Dodo - https://phabricator.wikimedia.org/T311462 (10hashar) 05Stalled→03Resolved a:03Jdforrester-WMF Phan has been made to pass under php 8.1 ( T307618 ). The CI change has been deployed. [14:13:07] 10Continuous-Integration-Config, 10Wikimedia-Site-requests, 10User-Urbanecm: CI should ensure that wmf-config/logos.php matches logos/config.yaml - https://phabricator.wikimedia.org/T329231 (10hashar) >>! In T329231#8640239, @Urbanecm wrote: > Did a final test with [r891505](https://gerrit.wikimedia.org/r/89... [14:15:42] 10Continuous-Integration-Config, 10MediaWiki-Core-Tests, 10Browser-Tests, 10User-zeljkofilipin: Jenkins job should fail when all (selenium) tests are skipped - https://phabricator.wikimedia.org/T324480 (10hashar) Is that for the selenium-daily jobs which run against the beta cluster? For #quibble , it wil... [14:21:13] hashar: Thank you for all that deploy work [14:21:55] :] [14:31:41] 10Scap: Global lock file not being honored by Scap - https://phabricator.wikimedia.org/T330756 (10jnuche) [14:32:21] 10Gerrit, 10Wikidata, 10Wikidata Query Builder, 10wdwb-tech, 10wmde-wikidata-tech: Remove master branch from wikidata/query-builder/deploy.git - https://phabricator.wikimedia.org/T330752 (10hashar) To clarify, it is for the Gerrit `wikidata/query-builder/deploy` repository which has: https://gerrit.wiki... [14:35:53] (03PS1) 10Zoranzoki21: Zuul: Archive the Tabber extension [integration/config] - 10https://gerrit.wikimedia.org/r/892993 (https://phabricator.wikimedia.org/T271418) [14:37:39] (03CR) 10Jforrester: Zuul: [mediawiki/tools/phan/PerfCheckPlugin] Collapse template (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/892548 (owner: 10Jforrester) [14:40:46] (03CR) 10Hashar: [C: 03+2] Zuul: Archive the Tabber extension [integration/config] - 10https://gerrit.wikimedia.org/r/892993 (https://phabricator.wikimedia.org/T271418) (owner: 10Zoranzoki21) [14:41:58] (03Merged) 10jenkins-bot: Zuul: Archive the Tabber extension [integration/config] - 10https://gerrit.wikimedia.org/r/892993 (https://phabricator.wikimedia.org/T271418) (owner: 10Zoranzoki21) [14:43:07] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/892993 | T271418 [14:43:09] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:43:09] T271418: Archive the Tabber extension - https://phabricator.wikimedia.org/T271418 [15:01:00] James_F: Reedy: do we have an IRC bouncer service for volunteers? [15:01:15] I think I somehow recalled that maybe one of you two might have mentioned it [15:36:27] (03CR) 10Hashar: [C: 03+2] "Indeed :-]" [integration/config] - 10https://gerrit.wikimedia.org/r/892548 (owner: 10Jforrester) [15:37:40] (03Merged) 10jenkins-bot: Zuul: [mediawiki/tools/phan/PerfCheckPlugin] Collapse template [integration/config] - 10https://gerrit.wikimedia.org/r/892548 (owner: 10Jforrester) [15:47:52] 10Gerrit, 10Wikidata, 10Wikidata Query Builder, 10wdwb-tech, 10wmde-wikidata-tech: Remove master branch from wikidata/query-builder/deploy.git - https://phabricator.wikimedia.org/T330752 (10Lucas_Werkmeister_WMDE) Yeah, I seem to have the required access at https://gerrit.wikimedia.org/r/admin/repos/wiki... [15:49:58] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/892548/ [15:49:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:50:14] I am off, dinner time! [16:13:48] 10Scap: Global lock file not being honored by Scap - https://phabricator.wikimedia.org/T330756 (10jnuche) [16:26:25] 10phabricator maintenance bot: Maintenance bot creates duplicates for new wiki tasks - https://phabricator.wikimedia.org/T286740 (10JJMC89) [16:26:45] 10Continuous-Integration-Infrastructure, 10Quibble, 10Developer Productivity, 10Patch-For-Review: Quibble: Include the failed command in data passed to external endpoint - https://phabricator.wikimedia.org/T330750 (10kostajh) a:03kostajh [16:28:19] 10Beta-Cluster-Infrastructure, 10Wiki-Setup: Add a wiki that doesn't use the Gregorian calendar to beta cluster - https://phabricator.wikimedia.org/T298970 (10TheresNoTime) 05Open→03Declined Boldly closing per the above [16:50:41] maintenance-disconnect-full-disks build 469506 integration-agent-docker-1032 (/: 29%, /srv: 95%, /var/lib/docker: 46%): OFFLINE due to disk space [16:55:34] maintenance-disconnect-full-disks build 469507 integration-agent-docker-1032 (/: 29%, /srv: 37%, /var/lib/docker: 44%): RECOVERY disk space OK [17:11:09] 10GitLab (Infrastructure), 10serviceops-collab, 10Datacenter-Switchover, 10Patch-For-Review: Switchover gitlab (gitlab1004 -> gitlab2002) - https://phabricator.wikimedia.org/T329931 (10jbond) [17:11:15] 10GitLab, 10serviceops-collab: gitlab backup timer failing - https://phabricator.wikimedia.org/T330744 (10jbond) 05In progress→03Resolved thanks, i have ran `sudo systemctl reset-failed` to clean up the systemd status. closing [17:32:44] (03PS1) 10Lucas Werkmeister (WMDE): Allow more BuildKit frontend image names [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/893028 (https://phabricator.wikimedia.org/T329553) [17:35:18] 10Beta-Cluster-Infrastructure: en.wikipedia.beta.wmflabs.org — MWException: Error: invalid magic word 'pendingchangelevel' - https://phabricator.wikimedia.org/T330779 (10TheresNoTime) [17:35:37] (03CR) 10Lucas Werkmeister (WMDE): "Similar caveat as in https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/40 applies: I think this is the first Groovy code " [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/893028 (https://phabricator.wikimedia.org/T329553) (owner: 10Lucas Werkmeister (WMDE)) [17:37:32] 10Beta-Cluster-Infrastructure: en.wikipedia.beta.wmflabs.org — MWException: Error: invalid magic word 'pendingchangelevel' - https://phabricator.wikimedia.org/T330779 (10TheresNoTime) https://integration.wikimedia.org/ci/view/Beta/job/beta-scap-sync-world/92376/console is the first beta-scap-sync-world with `Che... [17:41:09] 10Beta-Cluster-Infrastructure: en.wikipedia.beta.wmflabs.org — MWException: Error: invalid magic word 'pendingchangelevel' - https://phabricator.wikimedia.org/T330779 (10TheresNoTime) -operations suggests it //was// https://gerrit.wikimedia.org/r/889259 for T326800 that is a potential cause ` 17:38:22 <+logmsgb... [17:53:02] 10Beta-Cluster-Infrastructure: en.wikipedia.beta.wmflabs.org — MWException: Error: invalid magic word 'pendingchangelevel' - https://phabricator.wikimedia.org/T330779 (10TheresNoTime) 05Open→03Resolved Resolved by [[ https://gerrit.wikimedia.org/r/893030 | revert ]] [19:17:29] does anyone know if it's possible for gitlab web UI to display longer commit msgs? re: https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/commit/fba7afb52113c77663f0764c2067007ec427589c [19:18:45] (03PS1) 10Subramanya Sastry: Remove bogus rules from early days of visual diff testing [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/893043 [19:20:25] (03CR) 10Subramanya Sastry: [C: 03+2] Remove bogus rules from early days of visual diff testing [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/893043 (owner: 10Subramanya Sastry) [19:21:04] (03Merged) 10jenkins-bot: Remove bogus rules from early days of visual diff testing [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/893043 (owner: 10Subramanya Sastry) [19:26:06] 10Release-Engineering-Team (Radar), 10MediaWiki-Core-Tests, 10Quality-and-Test-Engineering-Team (QTE) (Test engineering): Ensure we're testing appropriately and not over-testing across Wikimedia-deployed code - https://phabricator.wikimedia.org/T221434 (10TAdeleye_WMF) [19:26:12] 10Continuous-Integration-Config, 10Browser-Tests, 10Quality-and-Test-Engineering-Team (QTE) (Test engineering): Drop Selenium tests from gate-and-submit-wmf - https://phabricator.wikimedia.org/T307180 (10TAdeleye_WMF) [19:26:53] 10Continuous-Integration-Infrastructure, 10Code-Review-Workgroup, 10Quibble, 10Quality-and-Test-Engineering-Team (QTE) (Test engineering): Lightweight preview environment for gerrit changes - https://phabricator.wikimedia.org/T241140 (10TAdeleye_WMF) [19:27:11] 10Release-Engineering-Team (Radar), 10Epic, 10Platform Team Initiatives (API Integration Tests), 10Quality-and-Test-Engineering-Team (QTE) (Test engineering), 10Testing-Roadblocks: Create and run a suite of end-to-end tests for the Wikimedia environment - https://phabricator.wikimedia.org/T248683 (10TAdel... [19:27:21] 10Continuous-Integration-Config, 10MediaWiki-Core-Tests, 10Browser-Tests, 10Patch-For-Review, and 2 others: Make MediaWiki Wdio tests less slow (Sept 2019) - https://phabricator.wikimedia.org/T234002 (10TAdeleye_WMF) [19:36:16] inflatador: ugh. I was hoping that truncation was just a CSS rule, but it seems to be in the HTML source they emit. :/ [19:38:46] This might be https://gitlab.com/gitlab-org/gitlab/-/issues/357459 upstream. [19:45:21] bd808 dang. Thanks for investigating though! cc: ebernhardson [19:49:14] well that's an infuriating behavior. [19:50:03] Who could _possibly_ want to see more than 1KB of commit message?! [19:50:06] i initially misread the upstream ticket and thought "who's writing 1k commit *subjects*?", but truncating the whole message is not great. [19:58:30] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (GitLab V: Event Horizon 🌄), 10Patch-For-Review: Isito interferes with HTTP traffic from buildkitd build containers - https://phabricator.wikimedia.org/T330433 (10dduvall) p:05Triage→03Medium a:03dduvall [20:01:03] 10GitLab (Misc), 10Upstream: GitLab truncates commit messages over 1k of text - https://phabricator.wikimedia.org/T330790 (10brennen) [20:01:56] i was getting kind of irritable about people filing local tasks for upstream gitlab UI stuff we can't do anything about, but after some discussion i'm like well, we may as well keep a list. [20:27:03] 10Phabricator: Help with Phabricator MFA - https://phabricator.wikimedia.org/T329964 (10SimoneThisDot) Thank you for the answer Andre.. Do you know "Who" can help me disable his account? he tried himself but he does not have the privileges to do so: {F36885062} [20:31:12] 10GitLab (Administration, Settings & Policy), 10Release-Engineering-Team (Priority Backlog 📥), 10Privacy Engineering, 10Product-Analytics, and 2 others: Request for Private repos to be enabled - https://phabricator.wikimedia.org/T305082 (10JFishback_WMF) [20:31:16] 10Phabricator: Help with Phabricator MFA - https://phabricator.wikimedia.org/T329964 (10RhinosF1) A combination of ITS (mainly for WMF staff) and SRE process will result in it. A member of SRE will be alerted when the date SRE were given to expire the access (or when they are informed otherwise) and they’ll run... [20:39:00] 10Phabricator: Help with Phabricator MFA - https://phabricator.wikimedia.org/T329964 (10RhinosF1) I’ve had a look and I don’t see any privileged access nor am I sure the user was created by ITS and there’s no -ctr email. I assume the best solution is to block the user on wikitech and lock SUL on leaving day. I... [20:39:26] Why is ^ not a -ctr email [20:40:28] Because they weren't directly employed by the WMF [20:43:04] Reedy: is there any process for accounts no one knows about than bribe a wikitech admin and steward? [20:43:26] Because not employed by WMF means it’s not surely ITS [20:43:44] And it doesn’t have nda so SRE won’t [20:44:13] -ctr emails are kind of dehumanizing, especially in an org where contractor probably just means foreigner [20:46:41] it's possible to be in the US and have a -ctr [20:46:56] Besides that aside, i don't see why we're refusing to allow them to transfer. I kind of think we should be happy to have ex-employed people to transition to volunteer capacity if they feel like it [20:47:37] -ctr means "part time" mostly [20:47:49] 10Phabricator (Upstream), 10Upstream: Allow disabling Audit application - https://phabricator.wikimedia.org/T330794 (10Aklapper) p:05Triage→03Low [20:48:00] 10Phabricator (Upstream), 10Upstream: Allow disabling Audit application - https://phabricator.wikimedia.org/T330794 (10Aklapper) [20:48:04] 10Phabricator: Make sure anti-vandalism features are up to snuff - https://phabricator.wikimedia.org/T84 (10Aklapper) [20:48:32] or maybe better stated "does not have a job tied to a requisition number and needs to be approved to contract renewal periodically" [20:49:02] 10Phabricator: Help with Phabricator MFA - https://phabricator.wikimedia.org/T329964 (10Bawolff) Personally I feel like if someone wants to contribute post-employment, we should make that easy (With of course no expectation that they do unless they want to). In practice this rarely comes up, as most people who... [20:49:28] I mean, regardless, i still think its a bit dehumanizing [20:50:34] bawolff: I mean I kinda agree with you on both counts [20:50:39] what part of getting paid by the WMF is humanizing? ;) [20:50:40] -ctr being stupid [20:50:52] And why can’t they just change their email [20:51:04] There is 0 special about that account [20:51:11] they wanted to do that, and needed 2fa disabled so they could [20:51:27] and then it spun into some weirder thing [20:51:31] Apart from the username but accounts can be renamed so drop the dot [20:51:48] bd808: yep but Andre said no [20:52:09] and he's the one to answer why questions [20:52:50] Which i also get why andre said no. Its weird for someone to use their account which has their employer's name in it, in a personal capacity afterwards [20:53:04] Rename it [20:53:16] Can you even rename phab accounts [20:53:26] Ye [20:53:27] yes [20:53:31] that's not why he said no. He said no because there is no clear way to verify the request [20:53:35] Been done many times [20:53:41] the 2fa removal request [20:53:43] * bawolff did not know that [20:53:54] bd808: surely someone at the wmf has met the guy [20:54:05] My reading of the task is the Adam Basso is willing to vouch that said person is really that person. Did i misread? [20:54:16] You did not misread [20:54:41] Or well, i sort of did [20:54:48] Because Adam did not comment on the task [20:54:56] the unknown person asserted that Adam would [20:55:20] Another this dot employee said they could [20:55:23] So yeah, this probably would have been resolved trivially if everyone on task was a bit more clear and straight forward [20:56:44] Also, it would be really nice if all these WMF subcontractors filled out their userpage on wiki with info [20:56:50] especially if they are going to edit wikis [20:56:54] even the test ones [21:03:42] 10GitLab (Auth & Access), 10Release-Engineering-Team (GitLab V: Event Horizon 🌄), 10Infrastructure-Foundations, 10SRE, and 3 others: migrate gitlab away from the CAS protocol - https://phabricator.wikimedia.org/T320390 (10demon) From the looks of it, we can add OIDC as a second [omniauth provider](https://... [21:47:23] 10Phabricator: Uninstall Differential - https://phabricator.wikimedia.org/T330797 (10Aklapper) 05Open→03Stalled [21:47:40] 10Phabricator: Uninstall Differential - https://phabricator.wikimedia.org/T330797 (10Aklapper) [21:47:46] 10GitLab (Project Migration), 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10serviceops-collab, and 3 others: Migrate active repositories in Phabricator Differential to GitLab - https://phabricator.wikimedia.org/T191182 (10Aklapper) [21:53:10] 10Phabricator: Help with Phabricator MFA - https://phabricator.wikimedia.org/T329964 (10Aklapper) WMF ITS does not disable any Phabricator accounts - SRE or Phab admins disable Phabricator accounts. WMF ITS would handle [on-wiki SUL accounts](https://meta.wikimedia.org/wiki/Special:CentralAuth?target=AdesojiThis... [22:22:15] doing a quick scap test deploy to aphlict2001 for debugging purposes. [22:58:24] 10Gerrit, 10GitLab (Project Migration), 10Release-Engineering-Team (GitLab V: Event Horizon 🌄), 10User-Kizule, 10User-brennen: Script closing/archiving of migrated repositories on Gerrit - https://phabricator.wikimedia.org/T330345 (10brennen) > In my imagination the perfect migration and least error pron... [23:13:22] (03PS1) 10JHathaway: helm-linter: add semver-cli [integration/config] - 10https://gerrit.wikimedia.org/r/893074 (https://phabricator.wikimedia.org/T320554) [23:37:45] (03PS1) 10Subramanya Sastry: Fix svwiki & eswiki rules + use .mw-ref not sup.reference [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/893081 [23:56:47] 10Phabricator: Uninstall Differential - https://phabricator.wikimedia.org/T330797 (10valerio.bozzolan) I wonder if Evan Priestley ever even imagined such a situation, which is somewhat post-apocalyptic, premising that Differential was the first component of Phabricator, as far as I can have seen from its history... [23:59:43] (03CR) 10Subramanya Sastry: [C: 03+2] Fix svwiki & eswiki rules + use .mw-ref not sup.reference [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/893081 (owner: 10Subramanya Sastry)