[04:07:03] 10Release-Engineering-Team (Radar), 10Scap, 10MediaWiki-Internationalization, 10Performance-Team: Use static php array files for l10n cache at WMF (instead of CDB) - https://phabricator.wikimedia.org/T99740 (10Krinkle) 05Stalledβ†’03Open a:05Krinkleβ†’03None No longer stalled as T266055 is now resolved... [06:11:18] 10Release-Engineering-Team (Radar), 10Scap, 10Patch-For-Review, 10User-jijiki: Update Scap to perform rolling restart for all MW deploy - https://phabricator.wikimedia.org/T266055 (10Joe) [06:11:35] 10Release-Engineering-Team (Radar), 10Scap, 10Patch-For-Review, 10User-jijiki: Update Scap to perform rolling restart for all MW deploy - https://phabricator.wikimedia.org/T266055 (10Joe) 05Openβ†’03Resolved [08:05:17] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for mfossati - https://phabricator.wikimedia.org/T313812 (10ArielGlenn) We missed you at the training this morning, @mfossati , I guess something came up? If you'd like to reschedule, that would be great :-) [08:18:38] 10GitLab (Infrastructure), 10Release-Engineering-Team, 10serviceops, 10serviceops-collab, 10User-brennen: GitLab major release: 15.x - https://phabricator.wikimedia.org/T309062 (10Jelto) GitLab Test instance was upgraded by unattended update to `15.0.4`. Update logs in `/var/log/unattended-upgrades/unatt... [09:00:48] So, strange question (and correct me if I am wrong) but it appears of the 131 wikis in group0 only 7 are not "closed". Are we aware of this? [09:05:39] dwalden: yes? group0 by definition is test wikis, mw.o and closed wikis [09:06:23] honestly 7 is more than what I expected [09:07:23] OK, fair enough. [09:08:22] It was a surprise to me, but OK. [09:10:52] I have now found https://wikitech.wikimedia.org/wiki/Deployments/Train#Groups, which I wasn't aware of before. [09:16:53] GitLab needs a short maintenance break at 10 UTC for abot 5 minutes. CI (gitlab-runner) will need some more minutes [09:34:37] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for mfossati - https://phabricator.wikimedia.org/T313812 (10mfossati) Sorry for that @ArielGlenn , I'm feeling sick today. Let me reschedule. [09:40:50] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for mfossati - https://phabricator.wikimedia.org/T313812 (10mfossati) [10:25:03] 10GitLab (Infrastructure), 10Release-Engineering-Team, 10serviceops, 10serviceops-collab, 10User-brennen: GitLab major release: 15.x - https://phabricator.wikimedia.org/T309062 (10Jelto) 05Openβ†’03Resolved All GitLab instances upgraded by unattended update to `15.0.4` and Runner to `15.0.1` Test infr... [13:31:51] 10Phabricator, 10translatewiki.net, 10I18n: Adding sicilian language (scn) - https://phabricator.wikimedia.org/T299694 (10Nikerabbit) Mukunda was handling this process before. I hope it's documented somewhere so that someone else can take it up. [13:40:15] (03PS3) 10Jforrester: Zuul: [mediawiki/extensions/OfflineImportLexicon] Mark asarchived [integration/config] - 10https://gerrit.wikimedia.org/r/817897 (https://phabricator.wikimedia.org/T313860) (owner: 10Mstyles) [13:45:03] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/OfflineImportLexicon] Mark asarchived [integration/config] - 10https://gerrit.wikimedia.org/r/817897 (https://phabricator.wikimedia.org/T313860) (owner: 10Mstyles) [13:48:02] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/OfflineImportLexicon] Mark asarchived [integration/config] - 10https://gerrit.wikimedia.org/r/817897 (https://phabricator.wikimedia.org/T313860) (owner: 10Mstyles) [14:31:05] 10Continuous-Integration-Config, 10MediaWiki-Internationalization, 10affects-translatewiki.net: Make messages with raw HTML translatable on translatewiki.net - https://phabricator.wikimedia.org/T312787 (10Nikerabbit) It is also to protect the users of translatewiki.net, in case we use those translations dire... [14:31:44] 10GitLab (Project Migration), 10Release-Engineering-Team: Create new GitLab project group: Community Resources Team - https://phabricator.wikimedia.org/T313593 (10Osnard) [15:02:35] 10Release-Engineering-Team (Radar), 10Security-Team, 10Security: Security Issue Access Request for (Your Phabricator Username) - https://phabricator.wikimedia.org/T314055 (10thcipriani) [15:02:51] 10Release-Engineering-Team (Radar), 10Security-Team, 10Security: Security Issue Access Request for @jnuche - https://phabricator.wikimedia.org/T314055 (10thcipriani) [15:45:20] 10Gerrit, 10Release-Engineering-Team (The Decommission Mission πŸ’€): Add Gerrit2002 as a replica of Gerrit1001 - https://phabricator.wikimedia.org/T313972 (10Volans) For context on a similar discussion for gitlab, please see T310265. In particular I have some questions/comments: * Could `gerrit-replica-new.wiki... [16:48:14] 10Gerrit, 10Release-Engineering-Team (The Decommission Mission πŸ’€), 10Patch-For-Review: Bring up Gerrit2002 - https://phabricator.wikimedia.org/T313250 (10Dzahn) https://puppet-compiler.wmflabs.org/pcc-worker1002/36495/gerrit2002.wikimedia.org/index.html from https://gerrit.wikimedia.org/r/c/operations/puppe... [16:48:39] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review, 10Release, 10Train Deployments: 1.39.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T308075 (10Ladsgroup) There are some templatelinks errors now on testwiki (`ApiQueryBacklinks::runFirstQuery`) but those has nothing to... [16:49:29] When I've been trying to use "check experimental" to run mediawiki-quibble-composer-mysql-php81-docker tests on REL1_38, i keep getting "Fatal error: Class "PHPUnit\Framework\TestFailure" not found in /workspace/src/includes/CommentFormatter/StringCommentIterator.php on line 0" [16:49:48] I'm pretty sure that the error has nothing to do with any mediawiki code [16:50:06] Any ideas on what the cause might be, or what i can do to debug it? [16:50:07] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review, 10Release, 10Train Deployments: 1.39.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T308075 (10brennen) > There are some templatelinks errors now on testwiki (ApiQueryBacklinks::runFirstQuery) but those has nothing to do... [16:50:13] example: https://integration.wikimedia.org/ci/job/mediawiki-quibble-composer-mysql-php81-docker/22/console [16:51:18] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review, 10Release, 10Train Deployments: 1.39.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T308075 (10Zabe) Sorry, a bit late. ##### Risky Patch! πŸš‚πŸ”₯ * **Change**: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CheckU... [16:55:40] Alert: wikibugs talks about Risky Patches, wah [17:21:14] (03PS1) 10Zabe: zuul: add CheckUser as phan dependency for Flow [integration/config] - 10https://gerrit.wikimedia.org/r/818189 [17:35:20] 10Phabricator, 10Release-Engineering-Team (The Decommission Mission πŸ’€), 10serviceops, 10serviceops-collab, 10Patch-For-Review: Setup rsync for phab data on disk - https://phabricator.wikimedia.org/T313360 (10Dzahn) ^ sorry, I mixed up tickets for phabricator and gerrit because we are working on both at t... [17:38:44] Could someone merge https://gerrit.wikimedia.org/r/c/integration/config/+/818189/ in order to make the fix for T314058 pass phan [17:38:45] T314058: TypeError: Argument 1 passed to Flow\Hooks::onSpecialCheckUserGetLinksFromRow() must be SpecialPage, CheckUserGetEditsPager given - https://phabricator.wikimedia.org/T314058 [17:38:45] ? [17:40:35] zabe: on it [17:41:19] hrm, i suppose this is going to need deployed [17:41:25] * brennen hasn't touched zuul in quite some time [17:44:06] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review, 10Release, 10Train Deployments: 1.39.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T308075 (10Zabe) [17:44:44] 10Gerrit, 10Release-Engineering-Team (The Decommission Mission πŸ’€), 10Patch-For-Review: Bring up Gerrit2002 - https://phabricator.wikimedia.org/T313250 (10Dzahn) on gerrit2002 we now have, created by the migration class: - a group "gerrit2" - a user "gerrit2" - a directory /srv/gerrit - package rsync install... [17:45:02] progress on gerrit2002: https://phabricator.wikimedia.org/T313250#8112877 [17:45:24] (03CR) 10Brennen Bearnes: [C: 03+2] zuul: add CheckUser as phan dependency for Flow [integration/config] - 10https://gerrit.wikimedia.org/r/818189 (owner: 10Zabe) [17:47:27] (03Merged) 10jenkins-bot: zuul: add CheckUser as phan dependency for Flow [integration/config] - 10https://gerrit.wikimedia.org/r/818189 (owner: 10Zabe) [17:49:39] bawolff: the TestFailure class is referred to by MWs phpunit printer [17:49:52] bawolff: is it specific to your patch even if you don't yet know why? [17:50:24] Might by that newer phpunit version is chosen [17:50:32] be* [17:50:44] The patch is just no changes, so its running off current REL1_38 branch. Its possible there's something in REL1_38 breaking it, but I'm not sure what it would be [17:52:28] Its also only happening on php 8.1 tests, but not php 8.0 or 7.2. And just on REL1_38 not master [17:52:35] 10Gerrit, 10Release-Engineering-Team (The Decommission Mission πŸ’€), 10SRE, 10serviceops, and 2 others: replacement for gerrit2001 - https://phabricator.wikimedia.org/T243027 (10Dzahn) on gerrit2002 we now have, created by the migration class: - a group "gerrit2" - a user "gerrit2" - a directory /srv/gerrit... [17:53:57] 10Gerrit, 10Release-Engineering-Team (The Decommission Mission πŸ’€): Add Gerrit2002 as a replica of Gerrit1001 - https://phabricator.wikimedia.org/T313972 (10Dzahn) [17:54:04] 10Gerrit, 10Release-Engineering-Team (The Decommission Mission πŸ’€), 10SRE, 10serviceops, and 2 others: replacement for gerrit2001 - https://phabricator.wikimedia.org/T243027 (10Dzahn) [17:54:57] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/818189/ [17:54:58] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:55:59] Like the only difference between REL1_38 and master in MediaWikiPHPUnitResultPrinter.php is a code comment [17:56:33] Hmm, maybe i should try and get quibble working locally. That'd probably make this easier [18:00:54] And both REL1_38 and master are using the same phpunit requirements in composer.jsob [18:01:12] brennen, thanks! [18:02:48] zabe: sure thing [18:04:41] Ahh, nevermind, I managed to reproduce it locally, which should give me something to go on [18:05:28] 10Gerrit, 10Release-Engineering-Team (The Decommission Mission πŸ’€), 10Patch-For-Review: Add Gerrit2002 as a replica of Gerrit1001 - https://phabricator.wikimedia.org/T313972 (10Dzahn) >>! In T313972#8112442, @Volans wrote: > The current one is set as VIP in Netbox, @Volans When I looked at netbox I noticed t... [18:05:37] hmm, does a recheck comment need to be _just_ "recheck"? [18:10:42] brennen: yes [18:22:08] huh, well seems like my issue is caused by a7b946631394599 [18:23:43] or maybe that just gets rid of the particular failure i was testing with, but not the general issue [18:28:28] !log gerrit: rsyncing /home from prod gerrit1001 to /srv/home-gerrit1001.wikimedia.org on gerrit2002 new replica T243027 T313250 [18:28:29] T243027: replacement for gerrit2001 - https://phabricator.wikimedia.org/T243027 [18:28:29] T313250: Bring up Gerrit2002 - https://phabricator.wikimedia.org/T313250 [19:07:12] # !log gerrit: starting rsync of /srv/gerrit (>240GB) from prod gerrit1001 to /srv/gerrit on gerrit2002 new replica .. ..slowly ..with --bwlimit=1000 [19:07:36] (in a screen on gerrit1001 it's pushing to the new host) [19:08:03] nice [19:08:12] the bwlimit is because last time I skipped that.. cough... [19:08:23] i broke things by taking all the bandwidth [19:08:35] so it's gonna be slow but I like it ..and just check on it later [19:10:37] reason this works and also see: https://phabricator.wikimedia.org/T243027#8112882 [19:11:06] that was that "check firewall / rsync etc" part - done [19:36:19] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review, 10Release, 10Train Deployments: 1.39.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T308075 (10brennen) [19:57:19] shell groups gerrit-root or gerrit-admin can ssh to gerrti2002.wikimedia.org [20:46:06] 10Gerrit, 10Release-Engineering-Team (The Decommission Mission πŸ’€), 10Patch-For-Review: Add gerrit2002 as a replica of gerrit1001 - https://phabricator.wikimedia.org/T313972 (10Krinkle) [20:46:22] 10Gerrit, 10Release-Engineering-Team (The Decommission Mission πŸ’€), 10Patch-For-Review: Bring up gerrit2002 - https://phabricator.wikimedia.org/T313250 (10Krinkle) [20:58:20] 10Phabricator, 10Release-Engineering-Team (The Decommission Mission πŸ’€): Scap3-ify Phabricator - https://phabricator.wikimedia.org/T313953 (10brennen) [21:01:26] 10Phabricator, 10Release-Engineering-Team (The Decommission Mission πŸ’€): Scap3-ify Phabricator - https://phabricator.wikimedia.org/T313953 (10brennen) [21:23:54] 10MediaWiki-Releasing, 10MW-1.39-release: Announce 1.39 forthcoming "pencils down" and likely timeline to wikitech-l etc. - https://phabricator.wikimedia.org/T313918 (10Jdforrester-WMF) [21:25:07] 10Phabricator, 10Release-Engineering-Team (The Decommission Mission πŸ’€): Scap3-ify Phabricator - https://phabricator.wikimedia.org/T313953 (10brennen) Further experimented with `checks.yaml` and got `PermissionError`s because there's a root-owned symlink in each checkout, put in place by puppet. We'll need to... [21:35:07] 10Continuous-Integration-Infrastructure, 10PHP 8.2 support: Create PHP 8.2 CI images and jobs for early testing - https://phabricator.wikimedia.org/T314093 (10Jdforrester-WMF) [21:43:04] thcipriani: Heads-up that it's about time to branch REL1_39 and switch to 1.40.0-wmf alphas; I've created https://phabricator.wikimedia.org/T313918 but dates aren't final yet. I'll make sure I don't break anything for RelEng of course. [22:01:12] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review, 10Release, 10Train Deployments: 1.39.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T308075 (10brennen) 05Openβ†’03Resolved Stable at all wikis. [22:08:27] James_F: ty for the reviews :) [22:08:59] bawolff: Thank you for being awesome and writing them! I'm trying to work out whether it's worth building a php82 beta image. [22:09:55] James_F: Crazy talk. PHP 8.2 isn't even EOL yet ;) [22:10:25] IKR. [22:15:24] Looking at php 8.2, the big ones for us look to be deprecating dynamic properties, and maybe some of the changes to the Callable type [22:17:20] Huh, and also "${...}" being deprecated. That's a rather random thing to deprecate [22:23:11] huh. did someone spend time with a python 2 to 3 conversion and enjoy the experience enough to decide string manipulation should get harder? [22:23:24] I've been backporting these patches after they get merged at least if they apply clearly, but I'm not sure if there's much point to do so for 1.37 and 1.35, as that seems a bit of a hopeless cause [22:24:41] also TIL that "${foo->bar}" and "$foo->bar" are the same in php, but "{$foo->bar}" is not allowed [22:24:47] https://wiki.php.net/rfc/deprecate_dollar_brace_string_interpolation is the discussion about it [22:25:03] bawolff: Well, we've been so terrible at getting 8.0 support out that 1.35 may go deprecated before we ship it… [22:29:18] ('Deprecated' aka 1.39 LTS has shipped; EOL won't be until November 2023, and even I am not that pessimistic.) [22:55:27] (03PS1) 10Jforrester: [WIP] dockerfiles: Provide PHP 8.2 CI test images [integration/config] - 10https://gerrit.wikimedia.org/r/818239 (https://phabricator.wikimedia.org/T314093) [22:55:38] 10Continuous-Integration-Infrastructure, 10PHP 8.2 support, 10Patch-For-Review: Create PHP 8.2 CI images and jobs for early testing - https://phabricator.wikimedia.org/T314093 (10Jdforrester-WMF) 05Openβ†’03In progress [23:04:58] <3 James_F as always