[05:40:46] 10Release-Engineering-Team (Priority Backlog 📥), 10Release, 10Train Deployments: 1.42.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T350080 (10Tgr) ##### Risky Patch! 🚂🔥 * **Change**: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralAuth/+/960643 * **Summary**: Key format mig... [07:10:04] (03CR) 10WMDE-Fisch: [C: 03+1] Zuul: [mediawiki/extensions/FileImporter] Add VisualEditor for phan [integration/config] - 10https://gerrit.wikimedia.org/r/971586 (owner: 10Umherirrender) [07:42:22] 10GitLab (Upstream pit of despair 🕳️), 10Release-Engineering-Team (Radar), 10Upstream, 10User-brennen: Investigate whether issues, operations, wikis, etc. can be disabled globally on GitLab - https://phabricator.wikimedia.org/T264231 (10Nikki) I was going to move my repos for Wikimedia-related things from... [07:47:59] 10GitLab (Upstream pit of despair 🕳️), 10Release-Engineering-Team (Radar), 10Upstream, 10User-brennen: Investigate whether issues, operations, wikis, etc. can be disabled globally on GitLab - https://phabricator.wikimedia.org/T264231 (10RhinosF1) @Nikki: no one should have been using gitlab issues, project... [09:15:51] 10Continuous-Integration-Config, 10SonarQube Bot, 10Quality-and-Test-Engineering-Team (SonarCloud Admin): Upgrade SonarQube / codehealth jobs to use Java 17 - https://phabricator.wikimedia.org/T349983 (10hashar) [09:17:10] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Provide Java 17 CI images - https://phabricator.wikimedia.org/T350569 (10hashar) [09:29:24] (03PS1) 10Hashar: dockerfiles: Java 17 images based on Bullseye [integration/config] - 10https://gerrit.wikimedia.org/r/971905 (https://phabricator.wikimedia.org/T350569) [09:32:10] (03CR) 10Hashar: [C: 03+2] dockerfiles: Java 17 images based on Bullseye [integration/config] - 10https://gerrit.wikimedia.org/r/971905 (https://phabricator.wikimedia.org/T350569) (owner: 10Hashar) [09:33:23] (03Merged) 10jenkins-bot: dockerfiles: Java 17 images based on Bullseye [integration/config] - 10https://gerrit.wikimedia.org/r/971905 (https://phabricator.wikimedia.org/T350569) (owner: 10Hashar) [09:37:49] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Patch-For-Review: Provide Java 17 CI images - https://phabricator.wikimedia.org/T350569 (10hashar) 05Open→03Resolved a:03hashar Successfully published image docker-registry.wikimedia.org/releng/maven-java17:1.0.0 Successfully publis... [09:40:18] (03PS1) 10Hashar: dockerfiles: sonar-scanner: upgrade from Java 11 to 17 [integration/config] - 10https://gerrit.wikimedia.org/r/971906 (https://phabricator.wikimedia.org/T349983) [09:42:54] (03CR) 10Hashar: [C: 03+2] dockerfiles: sonar-scanner: upgrade from Java 11 to 17 [integration/config] - 10https://gerrit.wikimedia.org/r/971906 (https://phabricator.wikimedia.org/T349983) (owner: 10Hashar) [09:44:32] (03Merged) 10jenkins-bot: dockerfiles: sonar-scanner: upgrade from Java 11 to 17 [integration/config] - 10https://gerrit.wikimedia.org/r/971906 (https://phabricator.wikimedia.org/T349983) (owner: 10Hashar) [09:49:21] 10Continuous-Integration-Config, 10SonarQube Bot, 10Patch-For-Review, 10Quality-and-Test-Engineering-Team (SonarCloud Admin): Upgrade SonarQube / codehealth jobs to use Java 17 - https://phabricator.wikimedia.org/T349983 (10hashar) I have build the image based on Java 17: `docker-registry.wikimedia.org/son... [09:52:39] 10GitLab (Upstream pit of despair 🕳️), 10Release-Engineering-Team (Radar), 10Upstream, 10User-brennen: Investigate whether issues, operations, wikis, etc. can be disabled globally on GitLab - https://phabricator.wikimedia.org/T264231 (10Peachey88) To reframe how the last comment was written, @Nikki, Could... [10:32:07] 10Phabricator, 10Release-Engineering-Team (Social Piranhas 🐟): Remove "Active Differential users" stats from monthly community_metrics.sh email - https://phabricator.wikimedia.org/T324131 (10Aklapper) 05Open→03Resolved [10:35:22] 10Continuous-Integration-Config, 10SonarQube Bot, 10Quality-and-Test-Engineering-Team (SonarCloud Admin): Upgrade SonarQube / codehealth jobs to use Java 17 - https://phabricator.wikimedia.org/T349983 (10pwangai) Thank you @hashar I will monitor the codehealth jobs to ensure they are running smoothly. [10:56:03] 10GitLab (Project Migration), 10collaboration-services: Migrate SRE repositories to GitLab - operations/software - https://phabricator.wikimedia.org/T341504 (10LSobanski) [11:00:19] 10GitLab (Project Migration), 10SRE, 10Traffic: Migrate Traffic repositories from Gerrit to Gitlab - https://phabricator.wikimedia.org/T347623 (10LSobanski) Would operations/software/knead-wikidough and operations/software/liberica fit here as well? [11:00:42] 10GitLab (Project Migration), 10collaboration-services: Migrate SRE repositories to GitLab - operations/software - https://phabricator.wikimedia.org/T341504 (10LSobanski) [11:02:03] (03PS1) 10Hashar: jjb: use Java 17 for MediaWiki core codehealth jobs [integration/config] - 10https://gerrit.wikimedia.org/r/971919 (https://phabricator.wikimedia.org/T349983) [11:02:05] (03PS1) 10Hashar: jjb: use Java 17 for extensions codehealth jobs [integration/config] - 10https://gerrit.wikimedia.org/r/971920 (https://phabricator.wikimedia.org/T349983) [11:04:24] (03CR) 10Hashar: [C: 03+2] jjb: use Java 17 for MediaWiki core codehealth jobs [integration/config] - 10https://gerrit.wikimedia.org/r/971919 (https://phabricator.wikimedia.org/T349983) (owner: 10Hashar) [11:04:39] 10GitLab (Project Migration), 10collaboration-services: Migrate SRE repositories to GitLab - operations/software - https://phabricator.wikimedia.org/T341504 (10LSobanski) [11:05:08] (03CR) 10Hashar: "The Java 11 -> Java 17 is first being tested for mediawiki/core with https://gerrit.wikimedia.org/r/c/integration/config/+/971919/ and if " [integration/config] - 10https://gerrit.wikimedia.org/r/971920 (https://phabricator.wikimedia.org/T349983) (owner: 10Hashar) [11:06:18] (03Merged) 10jenkins-bot: jjb: use Java 17 for MediaWiki core codehealth jobs [integration/config] - 10https://gerrit.wikimedia.org/r/971919 (https://phabricator.wikimedia.org/T349983) (owner: 10Hashar) [11:06:44] 10Continuous-Integration-Config, 10SonarQube Bot, 10Patch-For-Review, 10Quality-and-Test-Engineering-Team (SonarCloud Admin): Upgrade SonarQube / codehealth jobs to use Java 17 - https://phabricator.wikimedia.org/T349983 (10hashar) I have updated the two jobs that runs for mediawiki/core * https://integrat... [11:10:39] 10GitLab (Project Migration), 10collaboration-services: Migrate SRE repositories to GitLab - https://phabricator.wikimedia.org/T341468 (10LSobanski) [11:14:59] 10Release-Engineering-Team (Priority Backlog 📥), 10Projects-Cleanup, 10collaboration-services: Migrate SRE repositories to GitLab - Archiving unused Gerrit repositories - https://phabricator.wikimedia.org/T343707 (10LSobanski) All good recommendations, so far I confirmed: - operations/docker-images/debian -... [11:15:55] 10GitLab (Project Migration), 10collaboration-services: Migrate SRE repositories to GitLab - operations/software - https://phabricator.wikimedia.org/T341504 (10LSobanski) [11:25:15] 10GitLab (Project Migration), 10collaboration-services: Migrate SRE repositories to GitLab - operations/debs - https://phabricator.wikimedia.org/T341991 (10LSobanski) [11:57:09] 10Phabricator: 2FA Reset Request - https://phabricator.wikimedia.org/T350576 (10Deniz_WMDE) [12:02:17] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog 📥), 10collaboration-services, 10Patch-For-Review: Migrate to using new GitLab CI runner authentication scheme - https://phabricator.wikimedia.org/T344951 (10Jelto) In https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/merge_... [14:29:11] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Remove Java 8 - https://phabricator.wikimedia.org/T350587 (10hashar) [14:32:34] 10Continuous-Integration-Config, 10Pywikibot, 10Language-Team (Language-2023-October-December), 10Patch-For-Review, 10affects-translatewiki.net: Translation updates to pywikibot are blocked: pywikibot-i18n-tox-docker has started failing - https://phabricator.wikimedia.org/T349599 (10Xqt) [14:36:23] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Remove Java 8 - https://phabricator.wikimedia.org/T350587 (10hashar) [14:51:31] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Remove Java 8 - https://phabricator.wikimedia.org/T350587 (10Gehel) General note: I think we should use the same Java version in CI and in production. While backward compatibility tends to be really good with Java, I think it is best to test... [14:55:13] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Remove Java 8 - https://phabricator.wikimedia.org/T350587 (10Gehel) [14:55:40] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Remove Java 8 - https://phabricator.wikimedia.org/T350587 (10Gehel) Should we take this opportunity to migrate those repositories and jobs to Gitlab? [15:10:10] 10Continuous-Integration-Config, 10Pywikibot, 10Language-Team (Language-2023-October-December), 10Patch-For-Review, 10affects-translatewiki.net: Translation updates to pywikibot are blocked: pywikibot-i18n-tox-docker has started failing - https://phabricator.wikimedia.org/T349599 (10hashar) >>! In T34959... [15:16:18] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog 📥), 10collaboration-services, 10Patch-For-Review: Migrate to using new GitLab CI runner authentication scheme - https://phabricator.wikimedia.org/T344951 (10Jelto) A rough checklist for the actual migration of a runner: [] merge pup... [15:16:45] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog 📥), 10collaboration-services, 10Patch-For-Review: Migrate to using new GitLab CI runner authentication scheme - https://phabricator.wikimedia.org/T344951 (10Jelto) [15:16:53] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, and 2 others: Move 25% of mediawiki external requests to mw on k8s - https://phabricator.wikimedia.org/T348122 (10Joe) >>! In T348122#9240973, @matmarex wrote: > The Kubernetes work so far has caused problems with cross-wiki Echo notificatio... [15:20:34] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/UploadWizard] Add CodeEditor for phan [integration/config] - 10https://gerrit.wikimedia.org/r/971585 (owner: 10Umherirrender) [15:21:52] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/UploadWizard] Add CodeEditor for phan [integration/config] - 10https://gerrit.wikimedia.org/r/971585 (owner: 10Umherirrender) [15:24:00] !log Zuul: [mediawiki/extensions/UploadWizard] Add CodeEditor for phan [15:24:02] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:24:36] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/FileImporter] Add VisualEditor for phan [integration/config] - 10https://gerrit.wikimedia.org/r/971586 (owner: 10Umherirrender) [15:25:23] 10Continuous-Integration-Config, 10SonarQube Bot, 10Patch-For-Review, 10Quality-and-Test-Engineering-Team (SonarCloud Admin): Upgrade SonarQube / codehealth jobs to use Java 17 - https://phabricator.wikimedia.org/T349983 (10hashar) Looking at the reports for mediawiki/core on https://sonarcloud.io/project/... [15:26:04] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Remove Java 8 images from integration/config - https://phabricator.wikimedia.org/T350587 (10hashar) [15:26:24] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/FileImporter] Add VisualEditor for phan [integration/config] - 10https://gerrit.wikimedia.org/r/971586 (owner: 10Umherirrender) [15:26:46] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure: Migrate all CI jobs from buster to bullseye or later and drop buster testing support - https://phabricator.wikimedia.org/T335765 (10hashar) [15:26:48] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Remove Java 8 images from integration/config - https://phabricator.wikimedia.org/T350587 (10hashar) [15:27:51] 10Phabricator: Phabricator forgot about (some?) Gerrit commits - https://phabricator.wikimedia.org/T278893 (10Aklapper) I guess my question is how to first end up finding the initial URI. :) For the record, there are four such entries and only one of those is a commit that "exists": ` mysql:phstats@m3-slave.eq... [15:28:51] (03Abandoned) 10Hashar: [WIP] Docker: Provide bullseye-based Quibble images [integration/config] - 10https://gerrit.wikimedia.org/r/914316 (https://phabricator.wikimedia.org/T335766) (owner: 10Jforrester) [15:30:01] 10Phabricator (Upstream), 10Upstream: tooltip mentions non-existent "Wishlist priority" in Open Tasks by Project and Priority report - https://phabricator.wikimedia.org/T91428 (10Aklapper) I'd say upstream shouldn't hardcode default priority field value names [15:30:53] hashar: I totally don't agree with this approach. It's the anti-pattern we're trying to avoid. [15:31:17] James_F: yeah but that always happened this way [15:31:20] CI should *always* ship on a new platform *before* production, otherwise it's significantly worthless. [15:31:23] prod get upgraded under the hood [15:31:25] Yes, I'm trying to fix that. [15:31:27] eventually stuff explodes [15:31:39] * James_F sighs. [15:31:42] and we suddenly discover CI got forgotten or starts breaking [15:31:46] Indeed. [15:31:54] I have long gave up on that [15:32:29] so eventually something will end up corrupted / with differences in prod [15:32:32] resulting in an UBN [15:32:43] You giving up doesn't mean everyone else has to. [15:32:47] we catch up on CI front to get whatever new state of thing is as an UBN [15:33:02] developers fix up the code/tests to adjust to the new state [15:33:14] (meanwhile CI is broken until the fix is done) [15:33:28] fixed code land in prod and we resume [15:33:58] I am more or less exaggerating though, I have only learned about that libicu last week for unrelated reasons [15:34:09] I've been following that for a few months. [15:34:33] I guess updating the CI images is rather straightforward [15:34:50] but I have no clue what the new libicu version can end up breaking [15:34:59] Mostly collation of categories. [15:35:14] I'm not worried about the libicu change, we've done several of them over the past few years. [15:35:29] maybe a new image can be crafted with yet another extra job to test it [15:35:38] and we run both in parallel until the migration is complete [15:35:47] (to ensure code works in both case if that is at all possible) [15:35:51] Yes, that was my plan. [15:35:55] But you just abandoned it. [15:36:02] AH [15:36:05] bummer :-\ [15:36:53] ah yeah cause that change is for Bullseye and I don't think we have any plan to upgrade prod to bullseye [15:37:12] my understanding is we do the libicu upgrade then go straight to php 8.1 [15:37:14] My understanding is that that's planned for some time next year. [15:37:22] (03Restored) 10Hashar: [WIP] Docker: Provide bullseye-based Quibble images [integration/config] - 10https://gerrit.wikimedia.org/r/914316 (https://phabricator.wikimedia.org/T335766) (owner: 10Jforrester) [15:37:28] anyway I restored it :) [15:37:35] <3 [15:38:03] we'd need the CI part to be identified in whatever roadmap exists :] [15:38:28] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/TemplateStyles] Add CodeEditor for phan [integration/config] - 10https://gerrit.wikimedia.org/r/971565 (owner: 10Umherirrender) [15:39:43] Yes! Hope to help make this a proper process in future. [15:39:53] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/TemplateStyles] Add CodeEditor for phan [integration/config] - 10https://gerrit.wikimedia.org/r/971565 (owner: 10Umherirrender) [15:40:02] as for "things that might break around libicu", N.ikerabbit just pinged us about https://phabricator.wikimedia.org/T294727 with a link to ICU 67 upgrade (I *think* this is relevant to the current conversation; if not, plz ignore and apologies for the noise) [15:40:05] "Theoretically" it'll be lots easier to migrate in a mw-on-k8s world. We'll see. [15:40:18] (03CR) 10Hashar: [C: 03+2] "Those are all on me, I have introduced those repositories under the `archived` section. Thank you for the fixes!" [integration/config] - 10https://gerrit.wikimedia.org/r/971593 (owner: 10Pppery) [15:40:59] Yeah, T294727 looks like an issue, definitely. [15:41:00] T294727: Bengali digits are shown on Meetei Wikipedia and translatewiki.net - https://phabricator.wikimedia.org/T294727 [15:41:27] !log Zuul: [mediawiki/extensions/TemplateStyles] Add CodeEditor for phan [15:41:29] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:41:56] (03Merged) 10jenkins-bot: Move active extensions out of archived section [integration/config] - 10https://gerrit.wikimedia.org/r/971593 (owner: 10Pppery) [15:42:23] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/971593 Move active extensions out of archived section [15:42:25] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:47:13] 10Continuous-Integration-Config, 10Pywikibot, 10Language-Team (Language-2023-October-December), 10affects-translatewiki.net: Translation updates to pywikibot are blocked: pywikibot-i18n-tox-docker has started failing - https://phabricator.wikimedia.org/T349599 (10Xqt) 05Open→03Resolved [15:52:36] 10Continuous-Integration-Config, 10Pywikibot, 10Language-Team (Language-2023-October-December), 10affects-translatewiki.net: Translation updates to pywikibot are blocked: pywikibot-i18n-tox-docker has started failing - https://phabricator.wikimedia.org/T349599 (10Nikerabbit) Thank you both! [15:56:10] 10Continuous-Integration-Config, 10Release-Engineering-Team: tox v4 replacement for TOX_TESTENV_PASSENV - https://phabricator.wikimedia.org/T348871 (10hashar) Setting `TOX_OVERRIDE=testenv.pass_env=XDG_CACHE_HOME,testenv.set_env=PY_COLORS=1` apparently replaces `pass_env` and `set_env` stanzas which might be d... [15:58:07] 10Continuous-Integration-Config, 10Pywikibot, 10Language-Team (Language-2023-October-December), 10affects-translatewiki.net: Translation updates to pywikibot are blocked: pywikibot-i18n-tox-docker has started failing - https://phabricator.wikimedia.org/T349599 (10hashar) It is possible that CI injecting `T... [16:02:06] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Remove Java 8 images from integration/config - https://phabricator.wikimedia.org/T350587 (10hashar) [16:09:22] 10Continuous-Integration-Config, 10Release-Engineering-Team: tox v4 replacement for TOX_TESTENV_PASSENV - https://phabricator.wikimedia.org/T348871 (10Xqt) >>! In T348871#9308999, @hashar wrote: > Setting `TOX_OVERRIDE=testenv.pass_env=XDG_CACHE_HOME,testenv.set_env=PY_COLORS=1` apparently replaces `pass_env`... [16:32:29] 10Continuous-Integration-Config, 10Infrastructure-Foundations, 10Puppet CI, 10SRE, 10SRE-tools: Add shell scripts CI validations - https://phabricator.wikimedia.org/T148494 (10Volans) [16:43:54] 10Phabricator: Phabricator forgot about (some?) Gerrit commits - https://phabricator.wikimedia.org/T278893 (10Dzahn) If it's such a small number and since I don't have an answer to the question, I would be ok just closing this. Maybe "declined" ? [16:44:08] how do you find out these days which time slot the train is happening in? [16:44:34] tgr: You mean, which of the two? [16:44:42] train-blockers.toolforge.org and knowing which region the person is in? [16:45:30] 10Release-Engineering-Team, 10Security-Team, 10SecTeam-Processed: Notify MediaWiki security tasks as soon as an uploaded patch fails to apply - https://phabricator.wikimedia.org/T350065 (10sbassett) [16:46:20] jnuche is utc+2 so I'd assume the eu day slot [16:46:48] 10Release-Engineering-Team, 10Security-Team, 10SecTeam-Processed: Ensure no security patches is an error condition for MediaWiki deploys - https://phabricator.wikimedia.org/T350070 (10sbassett) [16:47:40] tgr, RhinosF1: that's right, train deploys will happen this week in UTC times [17:37:28] 10Phabricator: Phabricator forgot about (some?) Gerrit commits - https://phabricator.wikimedia.org/T278893 (10hashar) The commit rOPUP6dcbacd92e01b303bbe58ce1d3fad5bfbb9558d5 is Patchset 2 of https://gerrit.wikimedia.org/r/c/operations/puppet/+/237299/2 which is available in Gerrit under the ref `refs/changes/9... [17:42:45] I assume there is a sync script to push new tags from gerrit/github to packagist .... i wonder if that has broken somehow for parsoid ... it has been almost 15 mins since I pushed a new parsoid tag to "origin" and I see that github has it (v0.19.0-a5) but packagist is still showing the older -a4 one ( https://packagist.org/packages/wikimedia/parsoid ). [17:43:14] I need that new tag in packagist to release a new parsoid version to the vendor/ repo. [17:44:34] subbu: typically the update is triggered by a git hook on github that pings the packagist API. Let me take a look at the github config... [17:45:02] ah, okay ... thanks! [17:46:26] I triggered a check via the packagist page to pick up the new tag. Still need to look at the github hook to see if it is broken [17:49:31] thanks .. i see it now in packagist. [17:50:33] (DatasourceError) firing: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [17:51:51] subbu: I can see in the github console that the hook was triggered 2023-11-06T10:18:24-0700 (17:18 UTC). Packagist acked with a 202 response. I'm not sure why nothing actually updated on the Packagist side until I poked the "update" button that maintainers can see at https://packagist.org/packages/wikimedia/parsoid. [17:52:27] I did just update the hook config on the github side to match the latest from https://packagist.org/about#how-to-update-packages [17:52:27] huh .. presumably some transient issue on the packagist side? [17:52:51] could be yeah, or maybe some work queue that would have gotten to it eventually? [17:53:27] possible ... I'll alert folks on the team about this for next week just in case. Scott has rights on the packagist repo and should also be able to hit the update button if necessary. [17:54:03] subbu: I think you should be able to see the github side of things at https://github.com/wikimedia/mediawiki-services-parsoid/settings/hooks/179265473?tab=deliveries [17:55:33] (DatasourceError) resolved: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [17:55:36] noted .. thanks for the pointers to that page. [18:06:53] 10Phabricator: Phabricator incorrectly states "Commit No Longer Exists" for some non-committed Gerrit patchsets (due to pulling all refs/changes/* into Diffusion before 2016-06-21) - https://phabricator.wikimedia.org/T278893 (10Aklapper) [18:07:01] 10Phabricator: Phabricator incorrectly states "Commit No Longer Exists" for some non-committed Gerrit patchsets (due to pulling all refs/changes/* into Diffusion before 2016-06-21) - https://phabricator.wikimedia.org/T278893 (10Aklapper) 05Open→03Declined Uh, thanks Hashar for exploring, explaining and docum... [18:24:37] 10Release-Engineering-Team (Priority Backlog 📥), 10Release, 10Train Deployments: 1.42.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T350080 (10Dreamy_Jazz) [18:36:35] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, and 2 others: Move 25% of mediawiki external requests to mw on k8s - https://phabricator.wikimedia.org/T348122 (10matmarex) It seems it was the same cause, as both issues look fixed to me. Thanks! [19:20:49] 10Phabricator (Upstream), 10Wikimedia-Planet, 10Upstream: missing html links for Phabricator Blog feeds - https://phabricator.wikimedia.org/T191683 (10Dzahn) ` # warn on missing links if not data.feed.has_key('planet_message'): if not data.feed.has_key('links'): data.feed['links']... [19:23:31] 10Phabricator (Upstream), 10Wikimedia-Planet, 10Upstream: missing html links for Phabricator Blog feeds - https://phabricator.wikimedia.org/T191683 (10Dzahn) This is code from the old "planet-venus" code base (https://www.intertwingly.net/code/venus/) that we do not use anymore. Also this does not appear in... [19:23:41] 10Phabricator (Upstream), 10Wikimedia-Planet, 10Upstream: missing html links for Phabricator Blog feeds - https://phabricator.wikimedia.org/T191683 (10Dzahn) 05Stalled→03Declined [19:26:57] 10Continuous-Integration-Config, 10PHP 8.3 support: Build PHP 8.3 CI images - https://phabricator.wikimedia.org/T339350 (10Jdforrester-WMF) a:03Jdforrester-WMF [19:39:51] (03PS4) 10Jforrester: dockerfiles: [php-ast] Build a PHP 8.3 AST [integration/config] - 10https://gerrit.wikimedia.org/r/954336 [19:39:53] (03PS3) 10Jforrester: [DNM] dockerfiles: Provide php83 test image [integration/config] - 10https://gerrit.wikimedia.org/r/954337 (https://phabricator.wikimedia.org/T339350) [19:42:07] (03CR) 10Jforrester: [C: 03+2] dockerfiles: [php-ast] Build a PHP 8.3 AST [integration/config] - 10https://gerrit.wikimedia.org/r/954336 (owner: 10Jforrester) [19:43:48] (03Merged) 10jenkins-bot: dockerfiles: [php-ast] Build a PHP 8.3 AST [integration/config] - 10https://gerrit.wikimedia.org/r/954336 (owner: 10Jforrester) [19:59:05] 10Continuous-Integration-Config, 10SonarQube Bot, 10Patch-For-Review, 10Quality-and-Test-Engineering-Team (SonarCloud Admin): Upgrade SonarQube / codehealth jobs to use Java 17 - https://phabricator.wikimedia.org/T349983 (10pwangai) It seems to be working fine with our sonar scanner version. We can go ahea... [20:24:08] (03CR) 10Jforrester: "This currently fails due to lacking four packages upstream:" [integration/config] - 10https://gerrit.wikimedia.org/r/954337 (https://phabricator.wikimedia.org/T339350) (owner: 10Jforrester) [20:59:51] 10Release-Engineering-Team (Priority Backlog 📥), 10Web-Team-Backlog (Web Team FY2023-24 Q2 Sprint 3), 10Wikimedia-production-error: Mobile section toggles have beeb - https://phabricator.wikimedia.org/T350627 (10Jdlrobson) p:05Triage→03Medium [21:00:07] 10Release-Engineering-Team (Priority Backlog 📥), 10Web-Team-Backlog (Web Team FY2023-24 Q2 Sprint 3), 10Wikimedia-production-error: No longer possible to collapse sections on mobile - https://phabricator.wikimedia.org/T350627 (10Jdlrobson) p:05Medium→03Unbreak! [21:05:07] 10Release-Engineering-Team (Priority Backlog 📥), 10Release, 10Train Deployments: 1.42.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T350080 (10Jdlrobson) T350627 is a deploy blocker. It seems to have been caused by https://gerrit.wikimedia.org/r/c/mediawiki/core/+/939783 [21:06:52] 10Release-Engineering-Team (Priority Backlog 📥), 10Content-Transform-Team, 10Web-Team-Backlog (Web Team FY2023-24 Q2 Sprint 3), 10Wikimedia-production-error: No longer possible to collapse sections on mobile - https://phabricator.wikimedia.org/T350627 (10Jdlrobson) Seems to have been caused by https://gerr... [21:11:49] 10Release-Engineering-Team (Priority Backlog 📥), 10Content-Transform-Team, 10MediaWiki-Platform-Team, 10Web-Team-Backlog (Web Team FY2023-24 Q2 Sprint 3), 10Wikimedia-production-error: No longer possible to collapse sections on mobile - https://phabricator.wikimedia.org/T350627 (10Jdlrobson) [21:29:28] Note for relengers: (task hopefully coming) cjming discovered that if you scap backport a patch that gets a -2 while Jenkins is running +2 checks, scap will hang forever [21:29:43] I guess scap should eventually go, hey, this has a -2, aborting [21:37:12] 10Scap: Scap backporting a patch that gets a -2 hangs - https://phabricator.wikimedia.org/T350628 (10cjming) [21:37:22] filed ^^ [21:37:48] not sure what tags it should have [21:48:16] 10Release-Engineering-Team (Priority Backlog 📥), 10Content-Transform-Team, 10MediaWiki-Platform-Team, 10Web-Team-Backlog (Web Team FY2023-24 Q2 Sprint 3), 10Wikimedia-production-error: No longer possible to collapse sections on mobile - https://phabricator.wikimedia.org/T350627 (10DLynch) Looks like the... [21:49:32] (03PS2) 10Jforrester: dockerfiles: update commit-message-validator to v2.0.0 [integration/config] - 10https://gerrit.wikimedia.org/r/971546 (https://phabricator.wikimedia.org/T339147) (owner: 10BryanDavis) [21:50:13] (03CR) 10Jforrester: [C: 03+2] dockerfiles: update commit-message-validator to v2.0.0 [integration/config] - 10https://gerrit.wikimedia.org/r/971546 (https://phabricator.wikimedia.org/T339147) (owner: 10BryanDavis) [21:52:28] 10Scap: Scap backporting a patch that gets a -2 hangs - https://phabricator.wikimedia.org/T350628 (10RhinosF1) Context is this evening's window and https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/971529/ [21:53:52] (03Merged) 10jenkins-bot: dockerfiles: update commit-message-validator to v2.0.0 [integration/config] - 10https://gerrit.wikimedia.org/r/971546 (https://phabricator.wikimedia.org/T339147) (owner: 10BryanDavis) [21:54:57] (03PS1) 10Jforrester: jjb: [commit-message-validator] Update to image 2.0.0 [integration/config] - 10https://gerrit.wikimedia.org/r/972053 (https://phabricator.wikimedia.org/T339147) [21:55:58] (03CR) 10Jforrester: [C: 03+2] "Deployed, test run at https://integration.wikimedia.org/ci/job/commit-message-validator/19478/console worked fine." [integration/config] - 10https://gerrit.wikimedia.org/r/972053 (https://phabricator.wikimedia.org/T339147) (owner: 10Jforrester) [21:57:29] (03Merged) 10jenkins-bot: jjb: [commit-message-validator] Update to image 2.0.0 [integration/config] - 10https://gerrit.wikimedia.org/r/972053 (https://phabricator.wikimedia.org/T339147) (owner: 10Jforrester) [21:58:12] (03PS2) 10Jforrester: dockerfiles: Upgrade to Doxygen 1.9.8 [integration/config] - 10https://gerrit.wikimedia.org/r/971554 (owner: 10Krinkle) [21:58:48] (03CR) 10Jforrester: [C: 03+2] dockerfiles: Upgrade to Doxygen 1.9.8 [integration/config] - 10https://gerrit.wikimedia.org/r/971554 (owner: 10Krinkle) [22:00:36] 10Release-Engineering-Team (Priority Backlog 📥), 10Content-Transform-Team, 10MediaWiki-Platform-Team, 10Web-Team-Backlog (Web Team FY2023-24 Q2 Sprint 3), 10Wikimedia-production-error: No longer possible to collapse sections on mobile - https://phabricator.wikimedia.org/T350627 (10ssastry) I thought MFE... [22:04:14] (03Merged) 10jenkins-bot: dockerfiles: Upgrade to Doxygen 1.9.8 [integration/config] - 10https://gerrit.wikimedia.org/r/971554 (owner: 10Krinkle) [22:04:58] 10Release-Engineering-Team (Priority Backlog 📥), 10Release, 10Train Deployments: 1.42.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T350080 (10Jdforrester-WMF) [22:08:08] 10Release-Engineering-Team (Priority Backlog 📥), 10Content-Transform-Team, 10MediaWiki-Platform-Team, 10Web-Team-Backlog (Web Team FY2023-24 Q2 Sprint 3), 10Wikimedia-production-error: No longer possible to collapse sections on mobile - https://phabricator.wikimedia.org/T350627 (10ssastry) Looks like thi... [22:09:07] 10Release-Engineering-Team (Priority Backlog 📥), 10Content-Transform-Team, 10MediaWiki-Platform-Team, 10Web-Team-Backlog (Web Team FY2023-24 Q2 Sprint 3), 10Wikimedia-production-error: No longer possible to collapse sections on mobile - https://phabricator.wikimedia.org/T350627 (10DLynch) Sorry, I guess... [22:10:06] (03PS1) 10Jforrester: jjb: Update Doxygen-based jobs to 1.9.8 [integration/config] - 10https://gerrit.wikimedia.org/r/972056 [22:24:46] 10Release-Engineering-Team (Priority Backlog 📥), 10Content-Transform-Team, 10MediaWiki-Platform-Team, 10Web-Team-Backlog (Web Team FY2023-24 Q2 Sprint 3), 10Wikimedia-production-error: No longer possible to collapse sections on mobile - https://phabricator.wikimedia.org/T350627 (10DLynch) Quick explanati...