[00:39:10] 10Continuous-Integration-Config, 10PHP 8.0 support: Make PHP 8.0 voting on MW master - https://phabricator.wikimedia.org/T300463 (10Reedy) 05Open→03Stalled [01:13:50] 10Continuous-Integration-Config, 10MW-1.35-release, 10MW-1.36-release, 10MW-1.37-release, 10PHP 8.0 support: Make PHP 8.0 voting on MW 1.35, 1.36, 1.37 - https://phabricator.wikimedia.org/T274965 (10Reedy) [01:13:52] 10Continuous-Integration-Config, 10MW-1.35-release: Add PHP 8.0 non voting jobs to 1.35 - https://phabricator.wikimedia.org/T269714 (10Reedy) [01:36:48] (03PS1) 10Ahmon Dancy: Release 4.3.1-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/761084 [01:36:50] (03CR) 10Ahmon Dancy: [C: 03+2] Release 4.3.1-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/761084 (owner: 10Ahmon Dancy) [01:37:17] (03CR) 10jerkins-bot: [V: 04-1] Release 4.3.1-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/761084 (owner: 10Ahmon Dancy) [01:38:10] (03CR) 10Ahmon Dancy: Release 4.3.1-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/761084 (owner: 10Ahmon Dancy) [01:38:13] (03CR) 10Ahmon Dancy: [C: 03+2] Release 4.3.1-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/761084 (owner: 10Ahmon Dancy) [01:38:53] (03Merged) 10jenkins-bot: Release 4.3.1-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/761084 (owner: 10Ahmon Dancy) [01:41:55] 10Release-Engineering-Team, 10Scap, 10serviceops: Deploy Scap version 4.3.1 - https://phabricator.wikimedia.org/T301307 (10dancy) [05:19:52] (03CR) 10Krinkle: Make sync-world, etc work without /srv/mediawiki on deploy server (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/757070 (https://phabricator.wikimedia.org/T253822) (owner: 10Ahmon Dancy) [09:13:53] 10Release-Engineering-Team, 10Scap, 10serviceops: Deploy Scap version 4.3.1 - https://phabricator.wikimedia.org/T301307 (10Jelto) 05Open→03In progress p:05Triage→03High a:03Jelto [10:07:53] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments: 1.38.0-wmf.21 deployment blockers - https://phabricator.wikimedia.org/T300197 (10Lucas_Werkmeister_WMDE) T301324 is a new frontend error in wmf.21, affecting mobile versions of Test Wikidata and (once the train reaches... [10:28:34] 10Release-Engineering-Team, 10Scap, 10serviceops: Deploy Scap version 4.3.1 - https://phabricator.wikimedia.org/T301307 (10Jelto) scap was updated to `4.3.1` on canary machines (`A:mw-canary or A:parsoid-canary or A:mw-jobrunner-canary or A:restbase-canary`) and test were ok. I'll proceed with all other mac... [11:41:15] 10GitLab (Infrastructure), 10SRE, 10serviceops: gitlab: enable IPv6 for https - https://phabricator.wikimedia.org/T300816 (10Jelto) 05In progress→03Resolved `gitlab.wikimedia.org` and `gitlab-replica.wikimedia.org` can be reached over IPv6 now! ` $ curl -s -I -6 https://gitlab.wikimedia.org/explore | gr... [13:35:57] 10Scap, 10Documentation: scap help needs updating - https://phabricator.wikimedia.org/T301343 (10Reedy) [13:40:21] 10Scap: Can't sync vendor - https://phabricator.wikimedia.org/T301344 (10Reedy) [13:55:03] 10Release-Engineering-Team, 10Scap, 10serviceops: Deploy Scap version 4.3.1 - https://phabricator.wikimedia.org/T301307 (10Jelto) 05In progress→03Resolved scap `4.3.1` is deployed on all machines. I'm closing this task. [14:01:49] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.38.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T300198 (10Nikerabbit) [15:09:05] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.38.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T300198 (10Reedy) [15:22:45] !log deleted shutoff deployment-mx02 [15:22:46] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:27:44] is there a way users can get "banned" from gerrit? [15:28:00] got someone struggling to log in, whoose account was also accidently blocked on wikitech some time ago [15:31:14] addshore: ja [15:31:21] Ollie_S: is the user! [15:31:24] when they get blocked on wikitech, their gerrit account is disabled [15:31:29] can we has enable? [15:31:31] there's a hook for it in CS.php [15:32:09] I can reenable a gerrit account [15:32:32] Ollie_S: if you PM dancy with your email, i guess we can get this sorted! [15:32:48] unblocking the wikitech account should re-enable it on gerrit [15:33:08] I think the user account was unblocked on wikitech before the re-enabling part of code was written or deployed [15:33:12] or it had a bug in it or something [15:33:20] block them again! [15:33:28] both definitely possible! [15:33:31] I did that already xD [15:33:40] (but some time ago) [15:35:12] ohia James_F [15:35:16] .... *jakob_WMDE [15:35:21] gj [15:35:25] * addshore slaps Reedy [15:35:36] :D [15:35:45] Oh no. [15:35:54] too many brits [15:36:04] Oh, gerrit bans? Sorry, no, I'm not one of the Blesséd Ones. [15:36:18] (Blessèd Ones, my typo.) [15:36:24] reenabling gerrit should work since https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/751941 (and follow-ups), which got deployed on 6th of january, they got reblocked on the 7th of january so it seems that that did not work? [15:37:01] Ollie_S: do you get a nice informative error message? [15:37:54] addshore: The finest! `Authentication failed.` [15:43:23] All good. dancy has unblocked me and I'm now able to log in again. Thanks for the help all :D [15:43:28] WOO! [15:43:35] * Reedy blocks Ollie_S for good measure [15:43:38] * addshore slaps Reedy again for good measure [15:43:41] xD [16:17:11] (03PS1) 10Joal: Fix analytics gobblin-wmf maven release job [integration/config] - 10https://gerrit.wikimedia.org/r/761386 [16:19:03] (03CR) 10jerkins-bot: [V: 04-1] Fix analytics gobblin-wmf maven release job [integration/config] - 10https://gerrit.wikimedia.org/r/761386 (owner: 10Joal) [16:21:54] (03CR) 10Ahmon Dancy: Make sync-world, etc work without /srv/mediawiki on deploy server (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/757070 (https://phabricator.wikimedia.org/T253822) (owner: 10Ahmon Dancy) [16:28:59] 10Release-Engineering-Team, 10Scap, 10serviceops: Deploy Scap version 4.3.1 - https://phabricator.wikimedia.org/T301307 (10dancy) Thanks @Jelto ! [16:36:21] (03PS2) 10Joal: Fix analytics gobblin-wmf maven release job [integration/config] - 10https://gerrit.wikimedia.org/r/761386 (https://phabricator.wikimedia.org/T297938) [17:00:32] 10GitLab (Infrastructure), 10SRE, 10serviceops: gitlab: enable IPv6 for https - https://phabricator.wikimedia.org/T300816 (10Dzahn) confirmed working: ` nmap -6 gitlab.wikimedia.org -p 443 .. PORT STATE SERVICE 443/tcp open https ` and also A+ now on https://www.ssllabs.com/ssltest/analyze.html?d=git... [17:35:46] 10Release-Engineering-Team (Done by Feb 23🔥), 10MediaWiki Train Development Environment: Train-dev: Update to helm 3 - https://phabricator.wikimedia.org/T301266 (10thcipriani) [17:40:49] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap, 10Patch-For-Review: Improve efficiency of scap l10n operations - https://phabricator.wikimedia.org/T295304 (10thcipriani) [17:43:03] 10Release-Engineering-Team (Doing), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url command - https://phabricator.wikimedia.org/T287042 (10thcipriani) [17:43:22] 10Release-Engineering-Team (Done by Feb 23🔥), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url: copy files for legacy deployment - https://phabricator.wikimedia.org/T294457 (10thcipriani) 05In progress→03Invalid Now using the deployment server as the source of truth for the con... [17:45:45] 10Release-Engineering-Team (Done by Feb 23🔥), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url: legacy deployment - https://phabricator.wikimedia.org/T294466 (10thcipriani) [17:51:26] 10Beta-Cluster-Infrastructure, 10Beta-Cluster-reproducible: Beta cluster down: Error: 502, Next Hop Connection Failed - https://phabricator.wikimedia.org/T300525 (10thcipriani) [17:54:51] 10Release-Engineering-Team (Seen), 10Scap: scap should automatically update mediawiki.org roadmap pages - https://phabricator.wikimedia.org/T300384 (10thcipriani) The [[ https://versions.toolforge.org/ | status board ]] is updated by reading the wikiversions file on https://noc.wikimedia.org/conf/highlight.php... [17:55:45] 10Release-Engineering-Team (Done by Feb 23🔥), 10MW-on-K8s, 10SRE, 10serviceops: Make scap deploy to kubernetes together with the legacy systems - https://phabricator.wikimedia.org/T299648 (10thcipriani) [17:56:31] 10Release-Engineering-Team (Done by Feb 23🔥), 10MW-on-K8s, 10SRE, 10serviceops: Make scap deploy to kubernetes together with the legacy systems - https://phabricator.wikimedia.org/T299648 (10thcipriani) [17:56:59] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Next), 10Scap: Replace most of beta::autoupdater with scap prep auto - https://phabricator.wikimedia.org/T299163 (10thcipriani) [17:57:51] 10Release-Engineering-Team, 10Scap: scap should not let you deploy to a non-existent environment - https://phabricator.wikimedia.org/T298274 (10thcipriani) 05Open→03Resolved a:03Majavah Should be resolved, thanks for the patch! [17:58:15] 10Deployments, 10Release-Engineering-Team (Doing): Deployment calendar: MediaWiki branch cut reported as between 3-4 AM UTC but actually it seems to be between 2-3 AM - https://phabricator.wikimedia.org/T297724 (10thcipriani) [17:59:53] 10Release-Engineering-Team (Done by Feb 23🔥), 10MW-on-K8s, 10serviceops: Build MediaWiki images for kubernetes on the deployment servers - https://phabricator.wikimedia.org/T297673 (10thcipriani) [18:18:57] 10Phabricator, 10SRE, 10vm-requests: VM Request template (form 84) title doesn't make sense - https://phabricator.wikimedia.org/T301387 (10RhinosF1) p:05Low→03Triage [18:23:59] (03CR) 10Ahmon Dancy: Make sync-world, etc work without /srv/mediawiki on deploy server (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/757070 (https://phabricator.wikimedia.org/T253822) (owner: 10Ahmon Dancy) [18:59:27] 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10Scap, 10Patch-For-Review, 10Performance-Team (Radar), 10Performance-Team-publish: Stop trying to avoid rsyncing l10n CDB files - https://phabricator.wikimedia.org/T297326 (10dancy) 05Open→03Declined Collected data is in the description. T99740 is reall... [19:00:33] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap, 10Patch-For-Review: Improve efficiency of scap l10n operations - https://phabricator.wikimedia.org/T295304 (10dancy) 05Open→03Resolved Some improvements were made but I couldn't do everything. Closing as resolved. [19:13:39] (03PS1) 10Ahmon Dancy: Add test case for sync-world's use of --exclude-wikiversions.php [tools/scap] - 10https://gerrit.wikimedia.org/r/761445 [19:20:24] (03CR) 10Ahmon Dancy: [C: 03+2] Add test case for sync-world's use of --exclude-wikiversions.php [tools/scap] - 10https://gerrit.wikimedia.org/r/761445 (owner: 10Ahmon Dancy) [19:21:22] (03Merged) 10jenkins-bot: Add test case for sync-world's use of --exclude-wikiversions.php [tools/scap] - 10https://gerrit.wikimedia.org/r/761445 (owner: 10Ahmon Dancy) [19:21:58] (03PS1) 10Ahmon Dancy: Don't update wikiversions.php if contents haven't changed [tools/scap] - 10https://gerrit.wikimedia.org/r/761446 [19:22:10] (03CR) 10Ahmon Dancy: [C: 03+2] Make www container handle www.mediawiki.org too [tools/train-dev] - 10https://gerrit.wikimedia.org/r/761065 (owner: 10Ahmon Dancy) [19:22:12] (03CR) 10Ahmon Dancy: [C: 03+2] train-dev: misc improvements [tools/train-dev] - 10https://gerrit.wikimedia.org/r/761066 (owner: 10Ahmon Dancy) [19:24:45] (03Merged) 10jenkins-bot: Make www container handle www.mediawiki.org too [tools/train-dev] - 10https://gerrit.wikimedia.org/r/761065 (owner: 10Ahmon Dancy) [19:24:47] (03Merged) 10jenkins-bot: train-dev: misc improvements [tools/train-dev] - 10https://gerrit.wikimedia.org/r/761066 (owner: 10Ahmon Dancy) [19:32:45] (03PS1) 10Ahmon Dancy: scap prep auto fingerprint [tools/scap] - 10https://gerrit.wikimedia.org/r/761451 [19:33:49] (03CR) 10Thcipriani: [V: 03+2 C: 03+2] "Seems to build anyway ¯\_(ツ)_/¯" [releng/phatality] - 10https://gerrit.wikimedia.org/r/759293 (https://phabricator.wikimedia.org/T299168) (owner: 10Cwhite) [19:46:49] (03CR) 10Krinkle: [C: 03+1] Make sync-world, etc work without /srv/mediawiki on deploy server (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/757070 (https://phabricator.wikimedia.org/T253822) (owner: 10Ahmon Dancy) [19:48:11] (03CR) 10Ahmon Dancy: [C: 03+2] Make sync-world, etc work without /srv/mediawiki on deploy server (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/757070 (https://phabricator.wikimedia.org/T253822) (owner: 10Ahmon Dancy) [19:48:53] (03Merged) 10jenkins-bot: Make sync-world, etc work without /srv/mediawiki on deploy server [tools/scap] - 10https://gerrit.wikimedia.org/r/757070 (https://phabricator.wikimedia.org/T253822) (owner: 10Ahmon Dancy) [19:52:25] 10Deployments, 10Release-Engineering-Team (Radar), 10serviceops, 10Patch-For-Review, and 2 others: Remove provisioning for 'mwscript', 'foreachwikiindblist' etc from deployment host - https://phabricator.wikimedia.org/T253822 (10dancy) [20:10:50] 10Continuous-Integration-Infrastructure, 10LibUp, 10ci-test-error: Git repos with direct pushes bypassing gerrit/CI could be a maintenance burden for libup patches - https://phabricator.wikimedia.org/T301201 (10Umherirrender) Does that mean all extensions with `push` and/or `pushMerge` in the access rules sh... [20:12:24] (03PS1) 10Kosta Harlan: jjb: Temporarily disable ci-fullrun-extensions for Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/761454 [20:23:17] (03CR) 10Daniel Kinzler: "We want to disable this, I'll just trust that this is the correct way to do it 😊" [integration/config] - 10https://gerrit.wikimedia.org/r/761454 (owner: 10Kosta Harlan) [20:23:22] (03CR) 10Daniel Kinzler: [C: 03+1] jjb: Temporarily disable ci-fullrun-extensions for Quibble [integration/config] - 10https://gerrit.wikimedia.org/r/761454 (owner: 10Kosta Harlan) [20:40:47] 10Beta-Cluster-Infrastructure, 10SRE: Enable search team SRE access to deployment-prep VMs - https://phabricator.wikimedia.org/T301408 (10bking) [20:43:05] 10Beta-Cluster-Infrastructure, 10SRE: Enable search team SRE access to deployment-prep VMs - https://phabricator.wikimedia.org/T301408 (10bking) a:03bking [20:59:12] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Radar), 10Discovery, 10Discovery-Search (Current work): Deploy new elastic cluster nodes on deployment-prep - https://phabricator.wikimedia.org/T299797 (10bd808) [20:59:13] 10Beta-Cluster-Infrastructure, 10SRE, 10Patch-For-Review: Enable search team SRE access to deployment-prep VMs - https://phabricator.wikimedia.org/T301408 (10bd808) [21:20:03] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Radar), 10Discovery, 10Discovery-Search (Current work): Deploy new bullseye elastic cluster nodes on deployment-prep - https://phabricator.wikimedia.org/T299797 (10bking) [21:46:07] 10Release-Engineering-Team (Done by Feb 23🔥), 10Scap: Add rollback mechanism to `scap prep auto` - https://phabricator.wikimedia.org/T301417 (10dancy) [21:46:50] 10Release-Engineering-Team (Done by Feb 23🔥), 10Scap: Add rollback mechanism to `scap prep auto` - https://phabricator.wikimedia.org/T301417 (10dancy) [21:52:24] 10Release-Engineering-Team (Done by Feb 23🔥), 10Scap: Add rollback mechanism to `scap prep auto` - https://phabricator.wikimedia.org/T301417 (10dancy) [22:04:08] (03CR) 10Ahmon Dancy: [C: 03+1] "I cherry-picked and ran ./jjb-update integration-quibble-tox-docker. Let me know if that did the trick." [integration/config] - 10https://gerrit.wikimedia.org/r/761454 (owner: 10Kosta Harlan) [22:07:50] 10Release-Engineering-Team: Should UI regressions (e.g. no fatals) with the non-default skins ever block the train? - https://phabricator.wikimedia.org/T300169 (10Jdlrobson) 05Open→03Resolved a:03Jdlrobson > Developers SHOULD develop for multiple skins and they should use 3G download speeds to do so, using... [23:19:47] (03PS1) 10Ahmon Dancy: Don't use /srv/mediawiki in deploy container [tools/train-dev] - 10https://gerrit.wikimedia.org/r/761493 [23:20:18] (03CR) 10Ahmon Dancy: [C: 03+2] Don't use /srv/mediawiki in deploy container [tools/train-dev] - 10https://gerrit.wikimedia.org/r/761493 (owner: 10Ahmon Dancy) [23:25:32] (03Merged) 10jenkins-bot: Don't use /srv/mediawiki in deploy container [tools/train-dev] - 10https://gerrit.wikimedia.org/r/761493 (owner: 10Ahmon Dancy)