[00:12:19] !log Trying the simplest thing that might work by adding a CNAME record for parsoid.svc.deployment-prep.eqiad1.wikimedia.cloud. (T389252) [00:12:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:12:21] T389252: deployment-restbase05.deployment-prep.eqiad1.wikimedia.cloud configured to talk to parsoid.svc.deployment-prep.eqiad1.wikimedia.cloud - https://phabricator.wikimedia.org/T389252 [00:32:55] 10Beta-Cluster-Infrastructure: deployment-restbase05.deployment-prep.eqiad1.wikimedia.cloud configured to talk to parsoid.svc.deployment-prep.eqiad1.wikimedia.cloud - https://phabricator.wikimedia.org/T389252#10649885 (10bd808) Adding the CNAME seems to have made things happier in the near term. I also just mana... [02:44:13] 10Beta-Cluster-Infrastructure, 10gadget-Cat-a-lot: Temporary interface admin right to beta commons for updating the Cat-a-lot gadget - https://phabricator.wikimedia.org/T387080#10650203 (10Zache) @Multichill Could you do the updating the Beta Commons gadget files? [07:18:53] (03update) 10oblivian: Allow multiple kubernetes clusters to be used [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/681 (https://phabricator.wikimedia.org/T388761) [07:47:37] I'm getting timeouts from the gerrit webpage... is anything going on? [07:47:49] It's not down for me, just flaky and slow. [08:33:50] 10Scap: Integrate deploy security patch script to scap - https://phabricator.wikimedia.org/T389325 (10hashar) 03NEW [08:43:30] 10Scap: Integrate deploy security patch script to scap - https://phabricator.wikimedia.org/T389325#10650629 (10hashar) [08:45:41] 10Scap: Integrate deploy security patch script to scap - https://phabricator.wikimedia.org/T389325#10650639 (10taavi) dupe of {T361709}? [08:54:34] 10Gerrit, 06Release-Engineering-Team: Enable browser notifications system in Gerrit - https://phabricator.wikimedia.org/T389327 (10hashar) 03NEW [09:31:12] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 05Release, 05Train Deployments: 1.44.0-wmf.21 deployment blockers - https://phabricator.wikimedia.org/T386216#10650821 (10jnuche) [11:52:30] 10Continuous-Integration-Config, 10Testing Support, 10Test-Platform (The Next One): Remove wdio-video-reporter from all repositories - https://phabricator.wikimedia.org/T294341#10651461 (10zeljkofilipin) 05Openβ†’03In progress p:05Triageβ†’03Low [12:54:11] (03Restored) 10Zfilipin: selenium: Remove wdio-video-reporter npm package [integration/config] - 10https://gerrit.wikimedia.org/r/734639 (https://phabricator.wikimedia.org/T294341) (owner: 10Zfilipin) [12:54:36] (03Abandoned) 10Zfilipin: selenium: Remove wdio-video-reporter npm package [integration/config] - 10https://gerrit.wikimedia.org/r/734639 (https://phabricator.wikimedia.org/T294341) (owner: 10Zfilipin) [13:21:59] (03PS1) 10Hashar: Fix logging exception when using --resolve-requires [integration/quibble] - 10https://gerrit.wikimedia.org/r/1129251 [13:25:36] (03CR) 10Hashar: "If you are up for some Python fun? :) `--resolve-requires` processes the requires field in `extension.json` and recursively clone the re" [integration/quibble] - 10https://gerrit.wikimedia.org/r/1129251 (owner: 10Hashar) [13:26:46] jnuche: hi! btullis and I have merged this stack this morning https://gerrit.wikimedia.org/r/c/operations/dumps/+/1128897/ I was wondering what we needed to do to pull the changes in a new restricted/mediawiki-multiversion-debug image release. Thanks! [13:27:17] (and given the upcoming dc switchover for mediawiki, is it even a good time?) [13:30:41] brouberol: hi there, it sounds like you want to see your changes in the debug image, but you don't want to backport the change? [13:31:01] that's right [13:31:03] and yeah, the dc switchover is upon us soon [13:31:21] (which takes full priority of course, I'm happy to wait for a better time) [13:32:22] hmm, I don't know if that's a use case we support tbh. You will see your changes included in the image once the branch is cut for next week's train and a new image is built off that [13:32:53] or if we do a scap with a full rebuild, right? [13:33:58] claime: that would make sense, give me a few minutes to see what I can dig up [13:37:22] btw, until recently the automated train presync was being instructed explicitly to always do a full rebuild. That recent change in behavior may be messing with your expectations [13:46:01] ok, so the full image build has no bearing on what images are built from what I can see: https://gitlab.wikimedia.org/repos/releng/release/-/blob/main/make-container-image/build_image_incr.py?ref_type=heads#L194 [13:47:01] my guess is nothing off `master `gets built, since the purpose of generating the images is to deploy to prod (but that's just an assumption) [13:47:57] maybe it's possible to invoke a few scripts manually and pass `master` as a branch to generate the image, but I don't even know if that's desirable [13:48:31] I'm gonna fall back to dancy again, he'll know for sure. Sry about that [13:48:42] brouberol, claime: ^ [14:13:49] no worries jnuche, and thanks [14:36:52] can't we scap backport? [14:37:27] I wonder whether operations/dumps should be part of the same branching model as core/extension/skins etc [14:40:24] jnuche: Looks like we'd need to invalidate the build cache for the first section of https://gitlab.wikimedia.org/repos/releng/release/-/blob/main/make-container-image/mediawiki-debug/Dockerfile?ref_type=heads#L3 [14:42:52] For what it's worth, I don't think that that combining mediawiki-debug with the dumps code is intended to be a long-term solution. It was a pragmatic decision made here: https://phabricator.wikimedia.org/T381473#10389702 [14:44:29] jnuche, brouberol, btullis: I just deleted a couple of images from the deploy server which look to be the debug base image. If I targeted the right images, the next backport should perform a fresh clone of operations/dumps when it builds the debug image. [14:44:48] For the record I located the images using `docker image ls -f label=org.wikimedia.mediawiki-dumps` [14:45:02] dancy: Ack, thanks. [14:45:22] backport.. or just a scap sync-world will do. [14:45:42] thanks dancy! [14:46:21] Those are not commands that I've ever been called upon to run, yet. [14:48:38] thanks dancy [14:49:34] btullis see https://wikitech.wikimedia.org/wiki/MediaWiki_On_Kubernetes#Full_image_rebuild_and_deployment [14:50:01] Nice, thanks. [14:51:29] 10Continuous-Integration-Config, 06Security-Team, 07Security: Add some form of static analysis for package-lock.json - https://phabricator.wikimedia.org/T242058#10652378 (10Aklapper) 05In progressβ†’03Open Resetting task status from "In Progress" to "Open" as this task has not seen updates for two years. [15:04:33] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 10ci-test-error (WMF-deployed Build Failure): CI jobs failing with various timeouts (March 2025) - https://phabricator.wikimedia.org/T388416#10652519 (10Daimona) I spot-checked the [[https://grafana.wmclou... [15:35:02] maintenance-disconnect-full-disks build 685470 integration-agent-docker-1044 (/: 26%, /srv: 99%, /var/lib/docker: 70%): OFFLINE due to disk space [15:35:46] 06Project-Admins, 06Security-Team, 10Vulnerability Management, 07SecTeam-Processed, 07Security: Add numerous Security Vuln-* Tags in Phabricator - https://phabricator.wikimedia.org/T387508#10652690 (10mmartorana) I have updated all the vulnerability descriptions. While reviewing them, I noticed a fe... [15:38:56] 06Project-Admins, 06Security-Team, 10Vulnerability Management, 07SecTeam-Processed, 07Security: Add numerous Security Vuln-* Tags in Phabricator - https://phabricator.wikimedia.org/T387508#10652723 (10sbassett) 05Resolvedβ†’03Open >>! In T387508#10652690, @mmartorana wrote: > I have updated all the vul... [15:40:02] maintenance-disconnect-full-disks build 685471 integration-agent-docker-1044 (/: 26%, /srv: 81%, /var/lib/docker: 70%): RECOVERY disk space OK [15:41:49] 06Project-Admins, 06Security-Team, 10Vulnerability Management, 07SecTeam-Processed, 07Security: Add numerous Security Vuln-* Tags in Phabricator - https://phabricator.wikimedia.org/T387508#10652753 (10mmartorana) >> - Similarly, #vuln-authn_session and #vuln-missingauthz seem somewhat redundant... I wo... [15:47:19] dancy: jnuche: Just checking; would you happy for me to kick off a `scap sync-world --k8s-only --k8s-confirm-diff -D full_image_build:true`on a deployment server now? [15:47:51] btullis: no prob from my side [15:47:54] I recommend skipping `-D full_image_build:true` for the first attempt. [15:47:58] That's a very expensive flag. [15:48:29] (assuming the goal is still get get a mediawiki-debug image w/ the latest from operations/dumps) [15:49:26] dancy: Yes, that's right. So we should try using the generated image from `scap sync-world --k8s-only --k8s-confirm-diff` and see if the updated dumps code is included? Is that right? [15:49:39] btullis: hnowlan is running backports right now, but you can probably do it right after [15:49:46] Yes please. That will be much faster [15:50:08] claime: OK, thanks. [15:50:49] (currently at 62%) [15:51:30] Ack. Maybe we will try your image first, as well. [15:51:30] 06Project-Admins, 06Security-Team, 10Vulnerability Management, 07SecTeam-Processed, 07Security: Add numerous Security Vuln-* Tags in Phabricator - https://phabricator.wikimedia.org/T387508#10652845 (10sbassett) >>! In T387508#10652753, @mmartorana wrote: > I'd merge them. So a merge would be a "rename o... [15:54:42] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 05Release, 05Train Deployments: 1.44.0-wmf.21 deployment blockers - https://phabricator.wikimedia.org/T386216#10652876 (10jnuche) [16:13:24] 06Project-Admins, 06Security-Team, 10Vulnerability Management, 07SecTeam-Processed, 07Security: Add numerous Security Vuln-* Tags in Phabricator - https://phabricator.wikimedia.org/T387508#10652998 (10mmartorana) >>! In T387508#10652845, @sbassett wrote: >>>! In T387508#10652753, @mmartorana wrote: >> I'... [16:20:03] maintenance-disconnect-full-disks build 685479 integration-agent-docker-1042 (/: 26%, /srv: 96%, /var/lib/docker: 34%): OFFLINE due to disk space [16:25:02] maintenance-disconnect-full-disks build 685480 integration-agent-docker-1042 (/: 26%, /srv: 50%, /var/lib/docker: 31%): RECOVERY disk space OK [16:31:23] 06Project-Admins, 06Security-Team, 10Vulnerability Management, 07SecTeam-Processed, 07Security: Add numerous Security Vuln-* Tags in Phabricator - https://phabricator.wikimedia.org/T387508#10653081 (10A_smart_kitten) >>! In T387508#10652998, @mmartorana wrote: >>>! In T387508#10652845, @sbassett wrote: >... [16:38:11] 06Project-Admins, 06Security-Team, 10Vulnerability Management, 07SecTeam-Processed, 07Security: Add numerous Security Vuln-* Tags in Phabricator - https://phabricator.wikimedia.org/T387508#10653150 (10A_smart_kitten) >>! In T387508#10652723, @sbassett wrote: >>>! In T387508#10652690, @mmartorana wrote: >... [16:40:02] maintenance-disconnect-full-disks build 685483 integration-agent-docker-1041 (/: 26%, /srv: 100%, /var/lib/docker: 36%): OFFLINE due to disk space [16:45:03] maintenance-disconnect-full-disks build 685484 integration-agent-docker-1041 (/: 26%, /srv: 70%, /var/lib/docker: 34%): RECOVERY disk space OK [16:48:51] releng/andre: I've been pointed in your direction... I inadvertently left https://phabricator.wikimedia.org/P74253 publicly-visible for 16 minutes; it contains some officewiki titles. Reedy suggested you could see if anyone else accessed it in that time for me? [16:49:45] zip: I could try but #collaboration-services might be more skilled, please file a ticket [16:50:30] okay, thank you [16:52:20] NDA phabricator ticket tagged with them do the trick? [16:53:50] see: https://phabricator.wikimedia.org/T389392 [16:55:08] zip: looks good, just one thing, could you be as specific as possible about the timespan (and timezone) [16:55:24] It's on the paste, but sure [16:56:57] we'll file this one under "list of fuckups I won't do twice" [16:57:05] ... not to jinx it [16:58:13] thanks! I wouldn't worry. The ticket itself is the same thing that was said here and here is public as well. [16:58:37] and I hope there is nothing so secret that the mere mention of a page title is a big deal [16:58:43] hopefully not [16:59:47] though I'm a bit concerned if someone spots Talk:List of secret evil projects [17:00:02] the move itself might cause a bigger discussion than the "leak" [17:00:15] I assume those wont be moves with redirects [17:00:43] naw, they're with redirects. we're archiving Flow pages so people can switch back to regular discussion tools [17:01:20] ah! but wouldn't that mean the regular talk pages always redirect to the flow page? [17:01:52] I will check the access stuff.. give me a few minutes [17:02:17] yes, the regular talk pages will have a link to the flow page [17:02:41] we do want people to be able to create the flow pages, we just don't want them editing them or creating new ones [17:02:45] er, s/create/find [17:03:13] https://phabricator.wikimedia.org/T380911, if you're interested [17:03:21] I am not sure it can be both at the same time. redirect the whole page and also use it and have links there. But maybe I am just confused. [17:03:24] be right back [17:03:37] thanks for the link, ack [17:03:40] I think we move the Flow page, create a new page, and put a link at the top to the archive page [17:04:29] *nod* [17:04:43] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 05Release, 05Train Deployments: 1.44.0-wmf.21 deployment blockers - https://phabricator.wikimedia.org/T386216#10653332 (10jnuche) [17:05:39] zip: was the paste included in any tickets or just "standalone" ? [17:05:51] asking because it matters what I have to search for in logs [17:08:05] Indeed, I included it in a ticket [17:08:51] https://phabricator.wikimedia.org/T380911 from 16:18GMT [17:08:57] alright, thanks [17:14:46] I wonder if it would be a good idea to decrease the jenkins timeout for builds from 60 minutes to 45 or even 30 minutes [17:15:03] the selenium builds sometimes seem to get stuck (example: https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php74-selenium/78319/console), and then in the worst case all of gate-and-submit is just blocked for close to an hour :( [17:15:38] (that build got stuck after 12 minutes, so 48 minutes was spent just waiting for the timeout…) [17:15:53] but I don’t know what our longest legitimate builds are [17:16:07] maybe some of them need 50 minutes, no idea [17:21:51] 06Project-Admins: Delete Tags that were created wrong - https://phabricator.wikimedia.org/T385488#10653411 (10Aklapper) [17:25:16] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 05Release, 05Train Deployments: 1.44.0-wmf.21 deployment blockers - https://phabricator.wikimedia.org/T386216#10653436 (10SToyofuku-WMF) [17:38:34] zip: I answered on the ticket. please reload. maybe you can exclude some as yourself or known [17:42:27] yup [17:42:31] two of them are. any idea who olum is? [17:43:15] 06Project-Admins, 06Security-Team, 10Vulnerability Management, 07SecTeam-Processed, 07Security: Add numerous Security Vuln-* Tags in Phabricator - https://phabricator.wikimedia.org/T387508#10653543 (10mmartorana) >>! In T387508#10653150, @A_smart_kitten wrote: >>>! In T387508#10652723, @sbassett wrote: >... [17:44:19] the ticket is fine to be public AIUI [17:44:27] but the paste... was an error [17:46:34] zip: Olum is a family name and it's a family website. Can't find staff with that last name though. [17:46:41] zip: I would guess that olum is Pppery [17:47:06] https://phabricator.wikimedia.org/p/Pppery/ [17:47:29] I think we're probably fine then [17:47:33] zip: I almost said the same about the ticket being public.. but since I responded now with the IPs it's not true anymore :p [17:47:48] oh the ticket ABOUT the ticket has PII, sure, that's definitely best kept secret [17:47:57] possibly best edited out now also [17:47:58] oh, the other one. gotcha! [17:48:42] now we've resolved this could you redact the IPs? [17:53:37] ?from where? [17:54:06] from the ticket? not really. it will just have to stay private forever [17:55:18] well, okay [17:55:20] that's normal though for some NDAd tickets [17:56:06] well, I _can_ click edit.. [17:57:56] zip: nevermind, I DID replaced them with "redacted" in the comment [17:58:13] 'ppreciated :) [17:58:29] please hit resolve if you think that's it [17:59:03] Done! Thanks for checking [17:59:12] yep, you're welcome [17:59:22] it's been A Day [17:59:48] thanks as well, get a hot drink now [18:01:28] 10Gerrit, 10Release-Engineering-Team (Seen): Gerrit timeout when cloning mediawiki/core - https://phabricator.wikimedia.org/T292858#10653606 (10Aklapper) 05Openβ†’03Resolved No reply; optimistically resolving. Please reopen if I'm wrong. [18:02:30] zip: one last thing though. you still can't make the ticket public. just leave it as is. reason: there is still an edit history [18:02:39] this was like an edit on a wiki [18:03:52] 10Beta-Cluster-Infrastructure, 10gadget-Cat-a-lot: Temporary interface admin right to beta commons for updating the Cat-a-lot gadget - https://phabricator.wikimedia.org/T387080#10653615 (10Zache) [18:03:52] 10Release-Engineering-Team (Doing 😎), 10wikimedia.biterg.io: Enable domain matching in wikimedia.biterg.io configuration - https://phabricator.wikimedia.org/T327785#10653618 (10Aklapper) 05Openβ†’03Resolved At some point months ago I switched on "Affiliate to organizations automatically" at https://wikim... [18:05:45] 10Beta-Cluster-Infrastructure, 10gadget-Cat-a-lot: Temporary interface admin right to beta commons for updating the Cat-a-lot gadget - https://phabricator.wikimedia.org/T387080#10653630 (10Zache) @Multichill and others. If you have time to check the beta commins you could give me a admin rights too for T389317... [18:06:50] Project beta-scap-sync-world build #198543: 04FAILURE in 1 min 38 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/198543/ [18:08:59] sure. I don't plan to make the ticket public [18:11:02] alright, ty [18:11:42] 10Beta-Cluster-Infrastructure, 10gadget-Cat-a-lot: Temporary interface admin right to beta commons for updating the Cat-a-lot gadget - https://phabricator.wikimedia.org/T387080#10653694 (10TheresNoTime) Saw this in passing β€” have given you admin/interface admin on beta.commons [18:11:53] 10Beta-Cluster-Infrastructure, 06Abstract Wikipedia team, 10Wikifunctions: "Exec error in changeprop" for wikifunctions.beta.wmflabs.org - https://phabricator.wikimedia.org/T389274#10653699 (10Jdforrester-WMF) We should probably just decom Beta Wikifunctions, it doesn't and won't ever work. [18:12:02] 10Beta-Cluster-Infrastructure, 10Wikifunctions, 10Abstract Wikipedia team (25Q3 (Jan–Mar)): "Exec error in changeprop" for wikifunctions.beta.wmflabs.org - https://phabricator.wikimedia.org/T389274#10653703 (10Jdforrester-WMF) [18:16:56] Yippee, build fixed! [18:16:56] Project beta-scap-sync-world build #198544: 09FIXED in 1 min 46 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/198544/ [18:29:33] 10Beta-Cluster-Infrastructure, 10gadget-Cat-a-lot: Temporary interface admin right to beta commons for updating the Cat-a-lot gadget - https://phabricator.wikimedia.org/T387080#10653918 (10Zache) Thank you! [19:21:16] hi releng, anyone in-timezone with context on a.kosiaris's recent spiderpig work? we're discussing in #wikimedia-sre [19:25:46] o/ [19:34:19] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 05Release, 05Train Deployments: 1.44.0-wmf.21 deployment blockers - https://phabricator.wikimedia.org/T386216#10654198 (10jnuche) [19:42:09] (03open) 10dancy: local-dev: Create deployer02 user [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/692 [19:42:09] (03update) 10dancy: local-dev: Create deployer02 user [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/692 [19:42:47] (03update) 10dancy: local-dev: Create deployer02 user [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/692 [19:42:51] (03update) 10dancy: local-dev: Create deployer02 user [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/692 [19:53:57] (03merge) 10dancy: local-dev: Create deployer02 user [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/692 [20:25:50] (03PS1) 10Reedy: zuul: Always test and gate operations/mediawiki-config on PHP 8.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1129364 [20:27:36] (03CR) 10CI reject: [V:04-1] zuul: Always test and gate operations/mediawiki-config on PHP 8.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1129364 (owner: 10Reedy) [20:28:58] (03PS2) 10Reedy: zuul: Always test and gate operations/mediawiki-config on PHP 8.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1129364 [20:29:42] (03CR) 10Reedy: [C:03+2] "This is overdue ;)" [integration/config] - 10https://gerrit.wikimedia.org/r/1129364 (owner: 10Reedy) [20:31:25] (03Merged) 10jenkins-bot: zuul: Always test and gate operations/mediawiki-config on PHP 8.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1129364 (owner: 10Reedy) [20:32:58] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1129364 [20:32:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:34:14] (03CR) 10Jforrester: "We weren't running these in parallel because they're performance-critical, but sure." [integration/config] - 10https://gerrit.wikimedia.org/r/1129364 (owner: 10Reedy) [20:51:28] 10Beta-Cluster-Infrastructure, 07Wikimedia-production-error: PHP Warning: include(/srv/mediawiki/php-master/vendor/guzzlehttp/guzzle/src/Exception/ConnectException.php): Failed to open stream: Too many open files - https://phabricator.wikimedia.org/T389422 (10Reedy) 03NEW [21:23:56] (03PS4) 10Hashar: Update list of phpunit config files to copy to log directory [integration/quibble] - 10https://gerrit.wikimedia.org/r/1113983 (https://phabricator.wikimedia.org/T378797) (owner: 10Arthur taylor) [21:28:33] (03CR) 10Hashar: [C:03+2] "Note that CI requires a release of Quibble and CI images to be updated" [integration/quibble] - 10https://gerrit.wikimedia.org/r/1113983 (https://phabricator.wikimedia.org/T378797) (owner: 10Arthur taylor) [21:40:11] (03PS3) 10Jforrester: [DNM] Zuul: Drop all PHP 7.4 testing for MediaWiki things [integration/config] - 10https://gerrit.wikimedia.org/r/1127083 (https://phabricator.wikimedia.org/T328921) [21:40:11] (03PS2) 10Jforrester: Zuul: [integration/quibble] Stop testing in PHP 7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/1127093 [21:40:11] (03PS2) 10Jforrester: Zuul: [mediawiki/services/parsoid] Stop testing in PHP 7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/1127094 [21:40:11] (03PS2) 10Jforrester: Zuul: [operations/mediawiki-config] Stop testing in PHP 7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/1127095 [21:40:12] (03PS2) 10Jforrester: jjb: Provide api-testing-{db}-php81 [integration/config] - 10https://gerrit.wikimedia.org/r/1127115 [21:40:14] (03PS3) 10Jforrester: Zuul: [mediawiki/tools/api-testing] Stop testing in PHP 7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/1127096 [21:40:18] (03PS2) 10Jforrester: jjb: Provide php81-compile-coverage-publish [integration/config] - 10https://gerrit.wikimedia.org/r/1127116 [21:40:22] (03PS3) 10Jforrester: Zuul: Stop testing PHP extensions with PHP 7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/1127097 [21:40:26] (03PS3) 10Jforrester: Zuul: Stop testing most libraries and tools in PHP 7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/1127098 [21:40:30] (03PS3) 10Jforrester: Zuul: [labs/tools/heritage] Raise PHP testing from 7.4 to 8.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1127099 [21:40:34] (03PS3) 10Jforrester: Zuul: [mediawiki/tools/phan/SecurityCheckPlugin] Stop testing in PHP 7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/1127100 [21:40:38] (03PS3) 10Jforrester: Zuul: [mediawiki/tools/phan/PerfCheckPlugin] Use a template for CI [integration/config] - 10https://gerrit.wikimedia.org/r/1127101 [21:40:42] (03PS2) 10Jforrester: Docker: Provide quibble-bullseye-php81-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/1127117 [21:40:46] (03PS2) 10Jforrester: jjb: Migrate jobs silently using PHP 7.4 to 8.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1127118 [21:40:50] (03PS2) 10Jforrester: jjb: Drop PHP 7.4 jobs, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/1127119 [21:43:47] I'm not sure how important it is, i don't know enough about wikisource and the proofread page extension. But the wmf.21 extension today caused the page and index namespaces used by ProofreadPage to be removed from ContentNamespaces due to a change in loading order. Ticket: https://phabricator.wikimedia.org/T389430 [21:44:04] s/wmf.21 extension/wmf.21 deployment/ [21:45:24] (03Merged) 10jenkins-bot: Update list of phpunit config files to copy to log directory [integration/quibble] - 10https://gerrit.wikimedia.org/r/1113983 (https://phabricator.wikimedia.org/T378797) (owner: 10Arthur taylor) [21:45:32] ebernhardson: you can mark it a blocker of this week train blocker [21:45:36] https://train-blockers.toolforge.org/ [21:46:24] hashar: thanks, i'll let someone else figure out how important it is :) [21:47:04] and I guess poke #mediawiki-engineering-teams on Slack [21:47:24] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 05Release, 05Train Deployments: 1.44.0-wmf.21 deployment blockers - https://phabricator.wikimedia.org/T386216#10654715 (10EBernhardson) [21:47:31] ebernhardson: the thing is Proofreadpage is solely maintained by a sole volunteer as far as I remember [21:47:46] it's listed as tpt, haven't seen them too recently [21:47:58] yeah [21:48:15] jeena: some something is broken on Wikisource :) [21:48:31] I need to roll back train? [21:48:56] I have no idea but Erik found an issue that is worth investigating [21:49:01] I don't know whether it is serious enough [21:49:11] looking [21:49:49] i'm not sure if it needs to or not...on the search side it means much of the proofread page namespaces (a significant amount of the content) isn't searchable like it should be (but still findable under the "right" circumstances) [21:50:26] we have an automated system that fixes it up...it's moved ~300k pages across various wikisources to the right place, but that will take a few weeks to do them all (it intentionally goes slowly across all pages on all wikis and makes sure they are correct) [21:50:49] moves not in the mediawiki sense, but in the backend search indexes sense. Since they are no longer content they get indexed to a different place [21:52:01] well if Page and Index namespaces are supposed to be content namespaces, they should be flagged as such? [21:52:10] so that your search backend does the right thing [21:52:33] hashar: they are, via a hook in ProofreadPageInit. But loading order means NamespaceInfo takes a snapshot of $wgContentNamespaces before ProofreadPageInit runs [21:52:55] bummer :/ [21:52:57] what changed is the loading order, NamespaceInfo must have previously run after the hook, now it runs before [21:53:27] i tried finding what changed the loading order...but thats not an easy question :( [21:53:36] are the new search indexes going to be a problem after the issue is fixed? [21:53:38] short of git bisecting it I guess [21:53:53] https://www.mediawiki.org/wiki/MediaWiki_1.44/wmf.21 [21:53:59] jeena: well, yes it's a problem both directions. Right now its moving everything out of the content search indexes. Then once we fix it everything has to get moved back [21:54:13] so maybe we should roll back to stop more indexes from happening? [21:54:18] I'm not sure [21:54:34] whether the small time difference will matter [21:55:05] for the search systems, it's all automated and will fix itself either way. What it effects is the searchability of those things, most probably cant be found unless you select 'all namespaces' in search. They wont be found in normal search [21:55:20] it might also effect other parts of wikisource, but i don't know enough about that [21:56:11] that also alters the sites statistics :) [21:56:28] and rolling back would automatically fix the searchability? [21:56:32] https://en.wikisource.org/wiki/Special:Statistics [21:56:44] jeena: for most pages, except the 300k that already got moved [21:57:08] okay...I guess I'll roll back then [21:59:11] \o/ [21:59:24] I can't follow up, it is late and well I am in vacation tomorrow :] [21:59:43] thanks jeena for rolling back out of caution and ebernhardson for having flagged it! [21:59:54] Thanks for the ping! [22:00:00] * hashar bows [22:00:12] i'm crafting a mesage for #mediawiki-engineering-teams on slack, hopefully someone has ideas about what the fix should be [22:03:48] thanks ebernhardson [22:07:17] thank you as well [22:07:24] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 13Patch-For-Review, 05Release, 05Train Deployments: 1.44.0-wmf.21 deployment blockers - https://phabricator.wikimedia.org/T386216#10654778 (10jeena) ^ I commented on the task, but I rolled back because of T389430 [22:10:56] (03CR) 10Hashar: [C:03+1] Zuul: [integration/quibble] Stop testing in PHP 7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/1127093 (owner: 10Jforrester) [23:25:21] 10WikimediaDebug, 10Mobile-Content-Service, 10Parsoid (Tracking): Support X-Wikimedia-Debug header for services - https://phabricator.wikimedia.org/T182827#10655022 (10Krinkle) [23:25:26] 10WikimediaDebug, 10Mobile-Content-Service: Support X-Wikimedia-Debug header for services - https://phabricator.wikimedia.org/T182827#10655023 (10Krinkle) [23:28:04] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for davidcoronel - https://phabricator.wikimedia.org/T389444 (10DidiCoronel) 03NEW [23:32:23] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for davidcoronel - https://phabricator.wikimedia.org/T389444#10655042 (10DidiCoronel) [23:45:02] maintenance-disconnect-full-disks build 685568 integration-agent-docker-1041 (/: 27%, /srv: 95%, /var/lib/docker: 36%): OFFLINE due to disk space [23:46:06] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for davidcoronel - https://phabricator.wikimedia.org/T389444#10655073 (10bd808) Your https://ldap.toolforge.org/user/davidcoronel Developer account doesn't have Toolforge membership yet (https://wikitech.wikimedia.org... [23:50:03] maintenance-disconnect-full-disks build 685569 integration-agent-docker-1041 (/: 27%, /srv: 31%, /var/lib/docker: 33%): RECOVERY disk space OK