[01:15:19] 10MediaWiki-Releasing: Bundle Skin:Chameleon with MediaWiki core - https://phabricator.wikimedia.org/T391928#10746072 (10Bugreporter) 05Declinedβ†’03Stalled Reopen as stalled since there is at least some sort of consensus to do so as described in task description, but it can not be done yet until skin is moved... [01:25:39] 10MediaWiki-Releasing, 06Growth-Team, 10WikiLove: Bundle Extension:WikiLove with MediaWiki core - https://phabricator.wikimedia.org/T391952#10746101 (10Bugreporter) > WikiLove is a WMF specific extension May be no longer once we resolved {T392041}. Also we may want {T62984} first - having a table not being... [01:27:36] 10MediaWiki-Releasing, 10CirrusSearch, 06Discovery-Search: Bundle Extension:CirrusSearch with MediaWiki core - https://phabricator.wikimedia.org/T391922#10746106 (10Bugreporter) CirrusSearch can not do anything without ElasticSearch. What's worse, this extension is bound to a specific (and EOLed) ElasticSear... [01:39:32] 10MediaWiki-Releasing, 10MediaWiki-Installer, 06MediaWiki-Stakeholders-Group, 07Epic, 05MW-1.44-release: Expand the set of bundled extensions and skins in MediaWiki 1.44 - https://phabricator.wikimedia.org/T333405#10746139 (10Bugreporter) [02:06:30] 10MediaWiki-Releasing, 10MediaWiki-Installer, 06MediaWiki-Stakeholders-Group, 07Epic, 05MW-1.44-release: Expand the set of bundled extensions and skins in MediaWiki 1.44 - https://phabricator.wikimedia.org/T333405#10746150 (10Bugreporter2) [02:11:12] 10MediaWiki-Releasing, 06Growth-Team, 10WikiLove: Bundle Extension:WikiLove with MediaWiki core - https://phabricator.wikimedia.org/T391952#10746158 (10Pppery) > with just three supports over time That's mostly due to the obscurity of https://www.mediawiki.org/wiki/Suggestions_for_extensions_to_be_integrated [02:15:49] 10MediaWiki-Releasing, 10MassMessage: Bundle Extension:MassMessage with MediaWiki core - https://phabricator.wikimedia.org/T391927#10746160 (10Pppery) [07:06:41] 10GitLab (Infrastructure), 10Ceph, 06collaboration-services, 10Data-Persistence-Backup, and 2 others: Migrate gitlab storage to apus (also: backups from S3?) - https://phabricator.wikimedia.org/T378922#10746407 (10Jelto) >>! In T378922#10743705, @MatthewVernon wrote: > Looking at the Ceph metrics, it seems... [07:06:57] 10GitLab (Infrastructure), 10Ceph, 06collaboration-services, 10Data-Persistence-Backup, and 2 others: Migrate gitlab storage to apus (also: backups from S3?) - https://phabricator.wikimedia.org/T378922#10746410 (10Jelto) [09:20:59] !log integration: restarted integration-puppetserver-01 [09:20:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:22:08] 10GitLab (Infrastructure), 10Ceph, 06collaboration-services, 10Data-Persistence-Backup, and 2 others: Migrate gitlab storage to apus (also: backups from S3?) - https://phabricator.wikimedia.org/T378922#10746665 (10Jelto) [09:29:38] 10GitLab (Infrastructure), 10Ceph, 06collaboration-services, 10Data-Persistence-Backup, and 2 others: Migrate gitlab storage to apus (also: backups from S3?) - https://phabricator.wikimedia.org/T378922#10746694 (10Jelto) I've triggered a backup on the GitLab replica, which has been switched to object stora... [10:25:25] 10GitLab (Infrastructure), 10Ceph, 06collaboration-services, 10Data-Persistence-Backup, and 2 others: Migrate gitlab storage to apus (also: backups from S3?) - https://phabricator.wikimedia.org/T378922#10746807 (10MatthewVernon) That fits with what I see from bucket stats: gitlab-packages has 3,938 objects... [10:37:13] 06Project-Admins: Create project tag for MediaWiki-extension-DiscordRCFeed - https://phabricator.wikimedia.org/T392060 (10Lens0021) 03NEW [10:43:49] 06Project-Admins: Create project tag for MediaWiki-extension-DiscordRCFeed - https://phabricator.wikimedia.org/T392060#10746894 (10Peachey88) 05Openβ†’03Resolved a:03Peachey88 Project created :) [10:49:44] (03update) 10cgoubert: mediawiki-cli: Copy and link mwcron scripts [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/161 (https://phabricator.wikimedia.org/T391665) [10:52:02] (03merge) 10cgoubert: mediawiki-cli: Copy and link mwcron scripts [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/161 (https://phabricator.wikimedia.org/T391665) [11:07:22] 10Continuous-Integration-Config, 13Patch-For-Review: Several recent slow (>15 minute) helm-lint job runs - https://phabricator.wikimedia.org/T387781#10747004 (10kamila) 05Openβ†’03Stalled ServiceOps is currently working on several things that require modifying the CI code (k8s 1.31 compatibility, mw-script a... [11:08:24] 10Continuous-Integration-Config, 06serviceops, 13Patch-For-Review: Several recent slow (>15 minute) helm-lint job runs - https://phabricator.wikimedia.org/T387781#10747006 (10kamila) [11:29:32] 10Continuous-Integration-Config, 10ci-test-error (WMF-deployed Build Failure), 05MW-1.42-release, 05MW-1.43-release: quibble-composer-mysql-php8[23] fail since 31/3 - https://phabricator.wikimedia.org/T390754#10747071 (10abi_) [11:30:24] 10Continuous-Integration-Config, 10ci-test-error (WMF-deployed Build Failure), 05MW-1.42-release, 05MW-1.43-release: quibble-composer-mysql-php8[23] fail since 31/3 - https://phabricator.wikimedia.org/T390754#10747078 (10abi_) This is blocking merging of TwnMainPage extension pages. See: 1133093: TwnMainPa... [11:43:25] 10Continuous-Integration-Config, 10ci-test-error (WMF-deployed Build Failure), 05MW-1.42-release, 05MW-1.43-release: quibble-composer-mysql-php8[23] fail since 31/3 - https://phabricator.wikimedia.org/T390754#10747104 (10Reedy) Has someone regressed it… [11:48:32] 10GitLab (Pipeline Services Migration🐀), 06collaboration-services, 10Wikidata, 10Wikidata Query UI, and 2 others: move query.wikidata.org to kubernetes - https://phabricator.wikimedia.org/T350793#10747126 (10Jelto) I temporarily disabled all query sites on the legacy miscweb vms to test no service is using... [12:05:47] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Priority Backlog πŸ“₯), 06collaboration-services, 07Datacenter-Switchover: Create a runbook for switching CI master - https://phabricator.wikimedia.org/T256396#10747196 (10hashar) a:05hasharβ†’03None [12:06:22] 10Continuous-Integration-Config, 10ci-test-error (WMF-deployed Build Failure), 05MW-1.42-release, 05MW-1.43-release: quibble-composer-mysql-php8[23] fail since 31/3 - https://phabricator.wikimedia.org/T390754#10747200 (10Reedy) I note that this is slightly different, as it's master and an MW extension... [12:15:41] 10Continuous-Integration-Config, 10ci-test-error (WMF-deployed Build Failure), 05MW-1.42-release, 05MW-1.43-release: quibble-composer-mysql-php8[23] fail since 31/3 - https://phabricator.wikimedia.org/T390754#10747233 (10Reedy) I think the problem is still the original cause... But TwnMainPage uses `quibbl... [12:21:18] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 07Zuul: Zuul jenkins-bot user holding open SSH sessions - https://phabricator.wikimedia.org/T309376#10747248 (10hashar) 05Openβ†’03Resolved I am quite convinced it is a race condition in the code as found on T309376#8770561. Th... [12:40:29] 10MediaWiki-Releasing, 10MediaWiki-Installer, 06MediaWiki-Stakeholders-Group, 07Epic, 05MW-1.44-release: Expand the set of bundled extensions and skins in MediaWiki 1.44 - https://phabricator.wikimedia.org/T333405#10747331 (10Jdforrester-WMF) [12:41:15] (03update) 10jforrester: settings.yaml: Promote TemplateStyles to being bundled [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/160 (https://phabricator.wikimedia.org/T327006) (owner: 10reedy) [12:42:17] (03merge) 10jforrester: settings.yaml: Promote TemplateStyles to being bundled [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/160 (https://phabricator.wikimedia.org/T327006) (owner: 10reedy) [12:42:53] (03update) 10jforrester: settings.yaml: Promote CheckUser to being bundled [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/159 (https://phabricator.wikimedia.org/T317147) (owner: 10reedy) [12:43:58] (03merge) 10jforrester: settings.yaml: Promote CheckUser to being bundled [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/159 (https://phabricator.wikimedia.org/T317147) (owner: 10reedy) [12:44:27] 10MediaWiki-Releasing, 10MediaWiki-Installer, 05MW-1.45-release: Expand the set of bundled extensions and skins in MediaWiki 1.45 - https://phabricator.wikimedia.org/T392080 (10Jdforrester-WMF) 03NEW [12:45:21] 10MediaWiki-Releasing, 10MediaWiki-Installer, 06MediaWiki-Stakeholders-Group, 07Epic, 05MW-1.44-release: Expand the set of bundled extensions and skins in MediaWiki 1.44 - https://phabricator.wikimedia.org/T333405#10747363 (10Jdforrester-WMF) [12:46:07] 10MediaWiki-Releasing, 10MediaWiki-Installer, 06MediaWiki-Stakeholders-Group, 05MW-1.44-release: Expand the set of bundled extensions and skins in MediaWiki 1.44 - https://phabricator.wikimedia.org/T333405#10747367 (10Jdforrester-WMF) (This is obviously not an Epic.) I don't think any further ones are goi... [12:46:45] 10MediaWiki-Releasing, 10MobileFrontend (Tracking): Bundle MobileFrontend extension with MediaWiki - https://phabricator.wikimedia.org/T191734#10747370 (10Jdforrester-WMF) [12:46:46] 10MediaWiki-Releasing, 10MediaWiki-Installer, 05MW-1.45-release: Expand the set of bundled extensions and skins in MediaWiki 1.45 - https://phabricator.wikimedia.org/T392080#10747371 (10Jdforrester-WMF) [12:46:46] 10MediaWiki-Releasing, 10MediaWiki-Installer, 06MediaWiki-Stakeholders-Group, 05MW-1.44-release: Expand the set of bundled extensions and skins in MediaWiki 1.44 - https://phabricator.wikimedia.org/T333405#10747372 (10Jdforrester-WMF) [12:46:54] 10MediaWiki-Releasing, 10MediaWiki-Installer, 05MW-1.45-release: Expand the set of bundled extensions and skins in MediaWiki 1.45 - https://phabricator.wikimedia.org/T392080#10747374 (10Jdforrester-WMF) [12:46:54] 10MediaWiki-Releasing, 10MediaWiki-extensions-OATHAuth: Bundle WebAuthn extension with MediaWiki - https://phabricator.wikimedia.org/T258007#10747373 (10Jdforrester-WMF) [12:46:59] 10MediaWiki-Releasing, 10MediaWiki-Installer, 06MediaWiki-Stakeholders-Group, 05MW-1.44-release: Expand the set of bundled extensions and skins in MediaWiki 1.44 - https://phabricator.wikimedia.org/T333405#10747375 (10Jdforrester-WMF) [12:47:12] 10MediaWiki-Releasing, 10MediaWiki-Installer, 05MW-1.45-release: Expand the set of bundled extensions and skins in MediaWiki 1.45 - https://phabricator.wikimedia.org/T392080#10747377 (10Jdforrester-WMF) [12:47:13] 10MediaWiki-Releasing: Bundle Popups extension with MediaWiki - https://phabricator.wikimedia.org/T231797#10747376 (10Jdforrester-WMF) [12:47:18] 10MediaWiki-Releasing, 10MediaWiki-Installer, 06MediaWiki-Stakeholders-Group, 05MW-1.44-release: Expand the set of bundled extensions and skins in MediaWiki 1.44 - https://phabricator.wikimedia.org/T333405#10747378 (10Jdforrester-WMF) [12:47:24] 10MediaWiki-Releasing, 10MediaWiki-Installer, 05MW-1.45-release: Expand the set of bundled extensions and skins in MediaWiki 1.45 - https://phabricator.wikimedia.org/T392080#10747380 (10Jdforrester-WMF) [12:47:34] 10MediaWiki-Releasing, 10MediaWiki-Installer, 06MediaWiki-Stakeholders-Group, 05MW-1.44-release: Expand the set of bundled extensions and skins in MediaWiki 1.44 - https://phabricator.wikimedia.org/T333405#10747381 (10Jdforrester-WMF) [12:47:40] 10MediaWiki-Releasing, 10MediaWiki-Installer, 05MW-1.45-release: Expand the set of bundled extensions and skins in MediaWiki 1.45 - https://phabricator.wikimedia.org/T392080#10747384 (10Jdforrester-WMF) [12:47:46] 10MediaWiki-Releasing, 10AntiSpoof, 06Trust and Safety Product Team: Bundle AntiSpoof extension with MediaWiki - https://phabricator.wikimedia.org/T191736#10747383 (10Jdforrester-WMF) [12:47:52] 10MediaWiki-Releasing, 10MediaWiki-Installer, 06MediaWiki-Stakeholders-Group, 05MW-1.44-release: Expand the set of bundled extensions and skins in MediaWiki 1.44 - https://phabricator.wikimedia.org/T333405#10747385 (10Jdforrester-WMF) [12:48:02] 10MediaWiki-Releasing, 10MediaWiki-Installer, 05MW-1.45-release: Expand the set of bundled extensions and skins in MediaWiki 1.45 - https://phabricator.wikimedia.org/T392080#10747398 (10Jdforrester-WMF) [12:48:08] 10MediaWiki-Releasing, 10TimedMediaHandler: Bundle Extension:TimedMediaHandler with MediaWiki core - https://phabricator.wikimedia.org/T391756#10747397 (10Jdforrester-WMF) [12:48:14] 10MediaWiki-Releasing, 10MediaWiki-Installer, 06MediaWiki-Stakeholders-Group, 05MW-1.44-release: Expand the set of bundled extensions and skins in MediaWiki 1.44 - https://phabricator.wikimedia.org/T333405#10747399 (10Jdforrester-WMF) [12:48:39] 10MediaWiki-Releasing, 10MediaWiki-Installer, 05MW-1.45-release: Expand the set of bundled extensions and skins in MediaWiki 1.45 - https://phabricator.wikimedia.org/T392080#10747406 (10Jdforrester-WMF) [12:48:44] 10MediaWiki-Releasing, 10MediaWiki-Installer, 06MediaWiki-Stakeholders-Group, 05MW-1.44-release: Expand the set of bundled extensions and skins in MediaWiki 1.44 - https://phabricator.wikimedia.org/T333405#10747407 (10Jdforrester-WMF) [12:51:48] 10MediaWiki-Releasing, 10CheckUser, 06Trust and Safety Product Team, 05MW-1.44-release, 13Patch-For-Review: Bundle CheckUser extension with MediaWiki - https://phabricator.wikimedia.org/T317147#10747413 (10Jdforrester-WMF) OK, this will Just Workβ„’ when the bundle is made using the script. Remaining work... [12:52:09] (03update) 10jforrester: branch.py: ensure we're tracking skins/* changes [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/139 (owner: 10thcipriani) [12:53:37] 10MediaWiki-Releasing, 10TemplateStyles, 05MW-1.44-release: Bundle Extension:TemplateStyles with MediaWiki core - https://phabricator.wikimedia.org/T327006#10747416 (10Jdforrester-WMF) OK, this will Just Workβ„’ when the bundle is made using the script. Remaining work: * Add an entry about this to the release... [12:55:15] 10MediaWiki-Releasing, 10TemplateStyles, 05MW-1.44-release: Bundle Extension:TemplateStyles with MediaWiki core - https://phabricator.wikimedia.org/T327006#10747422 (10Jdforrester-WMF) [13:15:51] 10Continuous-Integration-Config, 06serviceops, 13Patch-For-Review: Several recent slow (>15 minute) helm-lint job runs - https://phabricator.wikimedia.org/T387781#10747502 (10Joe) >>! In T387781#10737662, @hashar wrote: > I think that should be changed to compare against the parent instead (`HEAD^`). As exp... [13:38:12] 10MediaWiki-Releasing, 10MediaWiki-Installer, 05MW-1.45-release: Expand the set of bundled extensions and skins in MediaWiki 1.45 - https://phabricator.wikimedia.org/T392080#10747589 (10Aklapper) [13:38:52] 10MediaWiki-Releasing, 10MediaWiki-Installer, 05MW-1.45-release: Expand the set of bundled extensions and skins in MediaWiki 1.45 - https://phabricator.wikimedia.org/T392080#10747592 (10Aklapper) [13:39:37] 10MediaWiki-Releasing, 10MediaWiki-Installer, 05MW-1.45-release: Expand the set of bundled extensions and skins in MediaWiki 1.45 - https://phabricator.wikimedia.org/T392080#10747593 (10Bugreporter) [13:39:38] 10MediaWiki-Releasing, 06Growth-Team, 10WikiLove: Bundle Extension:WikiLove with MediaWiki core - https://phabricator.wikimedia.org/T391952#10747594 (10Bugreporter) [13:39:39] 10MediaWiki-Releasing, 10MassMessage: Bundle Extension:MassMessage with MediaWiki core - https://phabricator.wikimedia.org/T391927#10747595 (10Bugreporter) [13:39:42] 10MediaWiki-Releasing, 10MediaWiki-extensions-CodeMirror: Bundle Extension:CodeMirror with MediaWiki core - https://phabricator.wikimedia.org/T391926#10747596 (10Bugreporter) [13:39:48] 10MediaWiki-Releasing, 10CirrusSearch, 06Discovery-Search: Bundle Extension:CirrusSearch with MediaWiki core - https://phabricator.wikimedia.org/T391922#10747597 (10Bugreporter) [13:40:19] 10MediaWiki-Releasing, 10CirrusSearch, 06Discovery-Search: Bundle Extension:CirrusSearch with MediaWiki core - https://phabricator.wikimedia.org/T391922#10747599 (10Reedy) p:05Mediumβ†’03Low [13:40:28] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [YOUR DEVELOPER ACCOUNT USERNAME HERE] - https://phabricator.wikimedia.org/T392088 (10Spartan.arbinger) 03NEW [13:41:43] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for spartan.arbinger - https://phabricator.wikimedia.org/T392088#10747628 (10Reedy) [13:42:49] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Disable "To:" and "CC:" Recipient Hint footers in mail notifications (metamta.recipients.show-hints) - https://phabricator.wikimedia.org/T391924#10747631 (10Aklapper) Ah, thanks! That may have made sense years ago, nowadays mail notifications incl... [13:43:59] 10MediaWiki-Releasing, 10MediaWiki-Installer, 05MW-1.45-release: Expand the set of bundled extensions and skins in MediaWiki 1.45 - https://phabricator.wikimedia.org/T392080#10747635 (10Bugreporter) [13:44:01] 10MediaWiki-Releasing: Bundle Skin:Chameleon with MediaWiki core - https://phabricator.wikimedia.org/T391928#10747634 (10Bugreporter) [13:45:34] 10Phabricator: Dark mode (experimental) has contrast issues - https://phabricator.wikimedia.org/T391929#10747661 (10Aklapper) Ponder linked on their frontpage might be an option then. [13:51:06] 10MediaWiki-Releasing, 10CirrusSearch, 06Discovery-Search: Bundle Extension:CirrusSearch with MediaWiki core - https://phabricator.wikimedia.org/T391922#10747705 (10Jdforrester-WMF) I can pretty confidently say that this is never going to happen, or at least not any time soon, as it's contrary to the entire... [13:56:13] 10Continuous-Integration-Config, 10ci-test-error (WMF-deployed Build Failure), 05MW-1.42-release, 05MW-1.43-release: quibble-composer-mysql-php8[23] fail since 31/3 - https://phabricator.wikimedia.org/T390754#10747761 (10hashar) > I suspect TwnMainPage may be an edge case in this case with this cross deplo... [13:58:51] (03PS1) 10Hashar: zuul: [mediawiki/extensions/TwnMainPage] use vendor [integration/config] - 10https://gerrit.wikimedia.org/r/1137009 (https://phabricator.wikimedia.org/T390754) [13:59:54] 10Release-Engineering-Team (Doing 😎), 07Essential-Work, 05Release, 05Train Deployments: 1.44.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T386220#10747791 (10Reedy) [14:00:25] 10Release-Engineering-Team (Doing 😎), 07Essential-Work, 05Release, 05Train Deployments: 1.44.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T386220#10747797 (10Reedy) Not sure if {T392086} is actually a train blocker, but *seems* to be new on .25, but could be just something weird on the... [14:00:41] 10MediaWiki-Releasing: Bundle Skin:Chameleon with MediaWiki core - https://phabricator.wikimedia.org/T391928#10747811 (10Aklapper) I am very very hesitant to add more skins by default after Timeless became quickly unmaintained after tarball inclusion (which is of course unrelated to Chameleon which might be a gr... [14:01:45] (03CR) 10Hashar: [C:03+2] zuul: [mediawiki/extensions/TwnMainPage] use vendor [integration/config] - 10https://gerrit.wikimedia.org/r/1137009 (https://phabricator.wikimedia.org/T390754) (owner: 10Hashar) [14:03:26] (03Merged) 10jenkins-bot: zuul: [mediawiki/extensions/TwnMainPage] use vendor [integration/config] - 10https://gerrit.wikimedia.org/r/1137009 (https://phabricator.wikimedia.org/T390754) (owner: 10Hashar) [14:12:14] 10Continuous-Integration-Config, 10ci-test-error (WMF-deployed Build Failure), 05MW-1.42-release, 05MW-1.43-release, 13Patch-For-Review: quibble-composer-mysql-php8[23] fail since 31/3 - https://phabricator.wikimedia.org/T390754#10747897 (10hashar) I have triggered the php 8.1, 8.2 and 8.3 job by [[ http... [14:19:18] 10Continuous-Integration-Config, 10ci-test-error (WMF-deployed Build Failure), 05MW-1.42-release, 05MW-1.43-release, 13Patch-For-Review: quibble-composer-mysql-php8[23] fail since 31/3 - https://phabricator.wikimedia.org/T390754#10747955 (10hashar) 05Openβ†’03Resolved > PHP build succeeded. > > *... [15:16:43] 10Phabricator, 06Infrastructure-Foundations: Removing an offboarded user from privileged Phab ACL projects should not require admins - https://phabricator.wikimedia.org/T392111 (10Aklapper) 03NEW [15:23:19] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [Ironblade101] - https://phabricator.wikimedia.org/T391203#10748383 (10Ironblade101) [15:27:55] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/WikiLambda] Make Catalyst non-voting for now [integration/config] - 10https://gerrit.wikimedia.org/r/1137025 [15:29:31] (03CR) 10CI reject: [V:04-1] Zuul: [mediawiki/extensions/WikiLambda] Make Catalyst non-voting for now [integration/config] - 10https://gerrit.wikimedia.org/r/1137025 (owner: 10Jforrester) [15:29:36] (03PS1) 10Jforrester: jjb: [wikilambda-catalyst-end-to-end] E-mail Vaughn on failures [integration/config] - 10https://gerrit.wikimedia.org/r/1137026 [15:30:25] (03PS2) 10Jforrester: Zuul: [mediawiki/extensions/WikiLambda] Make Catalyst non-voting for now [integration/config] - 10https://gerrit.wikimedia.org/r/1137025 [15:30:25] (03PS2) 10Jforrester: jjb: [wikilambda-catalyst-end-to-end] E-mail Vaughn on failures [integration/config] - 10https://gerrit.wikimedia.org/r/1137026 [16:00:02] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/WikiLambda] Make Catalyst non-voting for now [integration/config] - 10https://gerrit.wikimedia.org/r/1137025 (owner: 10Jforrester) [16:01:48] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/WikiLambda] Make Catalyst non-voting for now [integration/config] - 10https://gerrit.wikimedia.org/r/1137025 (owner: 10Jforrester) [16:01:59] !log Zuul: [mediawiki/extensions/WikiLambda] Make Catalyst non-voting for now [16:02:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:03:51] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Priority Backlog πŸ“₯): Experiment with Quibble job migration - https://phabricator.wikimedia.org/T392117 (10thcipriani) 03NEW [16:24:03] 10Beta-Cluster-Infrastructure: HIgh load on deployment-mediawiki14 and slow responses - https://phabricator.wikimedia.org/T392003#10748772 (10bd808) I told folks about all this blocking: https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/thread/2RFFHXSI6INQDJ2AQ7U3IQ2HTHT5J4VL/ [16:57:52] 06Release-Engineering-Team, 06serviceops: scap train-presync failed to push image: blob upload unknown - https://phabricator.wikimedia.org/T391935#10748926 (10dduvall) β†’14Duplicate dup:03T390251 [16:58:14] 06Release-Engineering-Team, 06serviceops: scap train-presync failed to push image: blob upload unknown - https://phabricator.wikimedia.org/T391935#10748931 (10dduvall) Closed as a duplicate that, while ongoing, is not strictly a train blocker. If folks run into this issue during a deployment, they should... [17:06:04] 10Phabricator, 06Infrastructure-Foundations: Removing an offboarded user from privileged Phab ACL projects should not require admins - https://phabricator.wikimedia.org/T392111#10748972 (10Pppery) > T39100 That task number is missing a digit. [17:08:14] 10Phabricator: Dark mode (experimental) has contrast issues - https://phabricator.wikimedia.org/T391929#10748977 (10Pppery) > I can't write on that task because I don't have "Trusted Contributors" I've added you to that group. [17:23:36] 10Phabricator, 06Infrastructure-Foundations: Removing an offboarded user from privileged Phab ACL projects should not require admins - https://phabricator.wikimedia.org/T392111#10749047 (10Aklapper) [17:23:48] 10Phabricator, 06Infrastructure-Foundations: Removing an offboarded user from privileged Phab ACL projects should not require admins - https://phabricator.wikimedia.org/T392111#10749053 (10Aklapper) Thanks! Fixed [17:33:26] 10Phabricator, 06Infrastructure-Foundations: Removing an offboarded user from privileged Phab ACL projects should not require admins - https://phabricator.wikimedia.org/T392111#10749094 (10Pppery) The list of projects there: [] #acl_security - This is a parent project that has no members distinct from its s... [17:37:47] 10Phabricator, 06Infrastructure-Foundations: Removing an offboarded user from privileged Phab ACL projects should not require admins - https://phabricator.wikimedia.org/T392111#10749103 (10Dzahn) Please see T392100#10749086 for a comment about this. [17:44:14] (03open) 10swfrench: make-container-image: sleep only after full mediawiki image build [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/165 (https://phabricator.wikimedia.org/T390251) [17:48:28] (03approved) 10dancy: make-container-image: sleep only after full mediawiki image build [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/165 (https://phabricator.wikimedia.org/T390251) (owner: 10swfrench) [18:23:32] 10Release-Engineering-Team (Doing 😎), 07Essential-Work, 05Release, 05Train Deployments: 1.44.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T386220#10749394 (10dduvall) [19:09:41] 10Release-Engineering-Team (Doing 😎), 07Essential-Work, 05Release, 05Train Deployments: 1.44.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T386220#10749571 (10Aklapper) T392142 might potentially turn out to be a blocker [19:27:58] (03merge) 10swfrench: make-container-image: sleep only after full mediawiki image build [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/165 (https://phabricator.wikimedia.org/T390251) [19:38:22] (03CR) 10Hashar: [C:03+2] jjb: [wikilambda-catalyst-end-to-end] E-mail Vaughn on failures (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1137026 (owner: 10Jforrester) [19:39:30] (03CR) 10Hashar: [C:03+2] "I have updated the job:" [integration/config] - 10https://gerrit.wikimedia.org/r/1137026 (owner: 10Jforrester) [19:39:41] (03Merged) 10jenkins-bot: jjb: [wikilambda-catalyst-end-to-end] E-mail Vaughn on failures [integration/config] - 10https://gerrit.wikimedia.org/r/1137026 (owner: 10Jforrester) [20:03:10] 10GitLab (Project Migration), 10BDC-Implementation: Move Analytics from Gerrit to Gitlab - https://phabricator.wikimedia.org/T391404#10749735 (10IAckerman-WMF) Hi @thcipriani! What info would you all need for this move? We can gather it so that when your team is ready for this task, we have everything document... [20:05:43] (03CR) 10Jforrester: "Thanks!" [integration/config] - 10https://gerrit.wikimedia.org/r/1137026 (owner: 10Jforrester) [20:12:39] 10GitLab (Project Migration), 10BDC-Implementation: Move Analytics from Gerrit to Gitlab - https://phabricator.wikimedia.org/T391404#10749779 (10Dzahn) This would probably start with a list of all (active) repos that you would like to move. [20:22:52] beta cluster is overwhelmed again, it seems [20:23:17] bd808: ^^ if you're bored [20:25:11] i didn't want to ping you because you helped with it yesterday, but no good deed goes unpunished, i guess ;) [20:26:13] heh. it's a fair ping [20:26:18] He'll tell me off in private, it's fine [20:26:30] stupid idea: put beta behind https://anubis.techaro.lol/ [20:26:46] taavi: I've been thinking about it [20:26:47] taavi: Bit late for april fools :D [20:46:14] looks like the big flood is coming from a telco in Vietnam [20:46:34] 14.160.0.0/11 [20:46:44] sudo ip route add country vietnam to blackhole [20:47:28] that is going to be a never ending chase :( [20:56:22] we have profile::firewall::nftables_throttling puppet class which is about throttling based on number of connections, not based on allow/deny lists [20:59:42] !log Blocked 193.43.72.0/24 and 14.160.0.0/11 because beta was very, very sad [20:59:42] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:00:08] modules/profile/templates/firewall/throttling.nft.erb contains mostly raw nftables code. the relevant part from there could be copied and executed without puppet too. but it still needs a list of PROD and CLOUD networks (to never throttle those) [21:01:49] I actually think taavi's half-joke idea isn't the worst: putting a proof of work proxy in front of beta. The one he mentioned is being used to protect a number of things these days including gitlab at Gnome [21:02:16] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [Ironblade101] - https://phabricator.wikimedia.org/T391203#10750108 (10Aklapper) 05Openβ†’03Resolved a:03Aklapper Done. [21:04:00] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for Rahulkundu1209 - https://phabricator.wikimedia.org/T382355#10750115 (10Aklapper) 05Openβ†’03Invalid There is no such user `Rahulkundu1209` in GNOME GitLab... [21:05:08] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for Austin Varshney - https://phabricator.wikimedia.org/T383205#10750118 (10Aklapper) 05Openβ†’03Invalid There is no such user `austinvarshney` in GNOME GitLab... [21:08:18] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [Sirchanmp] - https://phabricator.wikimedia.org/T391434#10750134 (10Aklapper) Hi, which projects hosted in GitLab do you plan to work on? [22:10:37] (03PS4) 10JHathaway: puppet: use bind mounts for local runs [integration/config] - 10https://gerrit.wikimedia.org/r/1135116 [22:16:17] (03CR) 10JHathaway: "Thanks for the careful review @hashar@free.fr, I've replied to your suggestions, as well as improving the commit comments." [integration/config] - 10https://gerrit.wikimedia.org/r/1135116 (owner: 10JHathaway) [22:33:29] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Doing 😎): Stand up Zuul 11 experiment environment in zuul3 cloud VPS project - https://phabricator.wikimedia.org/T391374#10750390 (10thcipriani) @dduvall mentioned a couple updates here in the #together team meeting: - that thi... [22:56:00] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Doing 😎): Stand up Zuul 11 experiment environment in zuul3 cloud VPS project - https://phabricator.wikimedia.org/T391374#10750425 (10thcipriani) I wonder if this is because the tenant config calls the [[https://zuul-ci.org/docs... [23:06:45] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Doing 😎): Stand up Zuul 11 experiment environment in zuul3 cloud VPS project - https://phabricator.wikimedia.org/T391374#10750463 (10Paladox) >>! In T391374#10750390, @thcipriani wrote: > @dduvall mentioned a couple updates her... [23:36:00] 10Phabricator: Phabricator workboard import: Array for %Ls conversion is empty. Query: projectPHID IN (%Ls) - https://phabricator.wikimedia.org/T392168 (10thcipriani) 03NEW [23:38:25] 10Phabricator: Phabricator workboard import: Array for %Ls conversion is empty. Query: projectPHID IN (%Ls) - https://phabricator.wikimedia.org/T392168#10750519 (10thcipriani) In case it's useful while digging through logs, quotes above are from 2025-04-11 14:55 UTC.