[06:10:07] <wikibugs>	 10Phabricator, 10Release-Engineering-Team (Bonus Level 🕹️), 10DBA, 10serviceops, and 2 others: sort out mysql privileges for phab1004/phab2002 - https://phabricator.wikimedia.org/T315713 (10Marostegui) @Aklapper this is now fixed: ` aklapper@phab1001:~$ mysql Reading table information for completion of tab...
[08:56:58] <wikibugs>	 (03CR) 10Jaime Nuche: [C: 03+1] python: add no-deps flag for pip installation (031 comment) [blubber] - 10https://gerrit.wikimedia.org/r/833778 (https://phabricator.wikimedia.org/T316755) (owner: 10Hashar)
[09:10:45] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] "Thanks for taking care of pywikibot CI :)  I have created the new job and will reload Zuul has soon as this is merged." [integration/config] - 10https://gerrit.wikimedia.org/r/834717 (https://phabricator.wikimedia.org/T301908) (owner: 10Xqt)
[09:12:45] <wikibugs>	 (03Merged) 10jenkins-bot: Zuul: [pywikibot/core] Use Python 3.6 instead of 3.5 for fasttest [integration/config] - 10https://gerrit.wikimedia.org/r/834717 (https://phabricator.wikimedia.org/T301908) (owner: 10Xqt)
[09:35:02] <hashar>	 !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/834717/
[09:35:04] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[09:45:16] <wikibugs>	 10GitLab (Infrastructure), 10serviceops, 10serviceops-collab, 10Patch-For-Review: Migrate gitlab-test instance to puppet - https://phabricator.wikimedia.org/T297411 (10Jelto) `gitlab-prod-1001` had failing puppet runs (root@ mail):  ` Sep 24 08:12:47 gitlab-prod-1001 systemd[1]: Starting OpenBSD Secure She...
[09:45:29] <wikibugs>	 10GitLab (Infrastructure), 10serviceops, 10serviceops-collab, 10Patch-For-Review: Migrate gitlab-test instance to puppet - https://phabricator.wikimedia.org/T297411 (10Jelto) 05Open→03Resolved
[09:53:05] <wikibugs>	 10GitLab (Infrastructure), 10serviceops, 10serviceops-collab: Migrate gitlab-test instance to bullseye - https://phabricator.wikimedia.org/T318521 (10Jelto)
[10:13:00] <wikibugs>	 (03Restored) 10Matthias Mullie: Zuul: [mediawiki/extensions/SearchVue] Mark as in production [integration/config] - 10https://gerrit.wikimedia.org/r/830584 (https://phabricator.wikimedia.org/T310367) (owner: 10Matthias Mullie)
[10:13:25] <wikibugs>	 (03Abandoned) 10Matthias Mullie: Zuul: [mediawiki/extensions/SearchVue] Mark as in production [integration/config] - 10https://gerrit.wikimedia.org/r/830584 (https://phabricator.wikimedia.org/T310367) (owner: 10Matthias Mullie)
[10:13:31] <wikibugs>	 (03Restored) 10Matthias Mullie: Start branching SearchVue for Wikimedia production [tools/release] - 10https://gerrit.wikimedia.org/r/830606 (https://phabricator.wikimedia.org/T310367) (owner: 10Matthias Mullie)
[10:13:49] <wikibugs>	 (03PS2) 10Matthias Mullie: Start branching SearchVue for Wikimedia production [tools/release] - 10https://gerrit.wikimedia.org/r/830606 (https://phabricator.wikimedia.org/T310367)
[10:18:18] <wikibugs>	 (03CR) 10Matthias Mullie: "Per T315250#8236613, security is fine with starting to branch." [tools/release] - 10https://gerrit.wikimedia.org/r/830606 (https://phabricator.wikimedia.org/T310367) (owner: 10Matthias Mullie)
[10:19:58] <wikibugs>	 10Phabricator: Set visibility policy of the Phabricator's Countdown application to "public" - https://phabricator.wikimedia.org/T258106 (10Aklapper) a:05mmodell→03None Removing task assignee due to inactivity as this open task has been assigned for more than two years. See the email sent to the task assignee...
[10:20:54] <wikibugs>	 10Release-Engineering-Team (Seen), 10TimedMediaHandler, 10local-charts, 10User-brennen: Add Timed media handler to local-charts - https://phabricator.wikimedia.org/T253773 (10Aklapper) a:05jeena→03None Removing task assignee due to inactivity as this open task has been assigned for more than two years....
[10:30:25] <wikibugs>	 (03CR) 10Hashar: [C: 04-1] "I have build the buildkit image locally using:" [blubber] - 10https://gerrit.wikimedia.org/r/833778 (https://phabricator.wikimedia.org/T316755) (owner: 10Hashar)
[10:35:31] <hashar>	 pff
[10:35:35] <wikibugs>	 10Release-Engineering-Team, 10Wikidata, 10wdwb-tech, 10ci-test-error: Wikibase CI failure: quibble-buster-php74-bundle does not exist - https://phabricator.wikimedia.org/T318525 (10Jakob_WMDE)
[10:35:39] <hashar>	 docker buildkit 1   -  0  hashar
[10:36:46] <tarrow>	 was just about to post in here about https://phabricator.wikimedia.org/T318525; missing docker image is a known problem?
[10:39:30] <wikibugs>	 (03CR) 10Hashar: [C: 04-1] "Another thing I have noticed is my local blubber/buildkit image has sha256 starting with 350e9c0753cd  but when running buildkit against s" [blubber] - 10https://gerrit.wikimedia.org/r/833778 (https://phabricator.wikimedia.org/T316755) (owner: 10Hashar)
[10:42:07] <wikibugs>	 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-Phonos, 10Community-Tech (CommTech-Sprint-33), 10MW-1.40-notes (1.40.0-wmf.2; 2022-09-19), 10Patch-For-Review: Phonos links to unroutable domain/URL for the MP3 file - https://phabricator.wikimedia.org/T317417 (10MatthewVernon) Reading the discussion...
[10:42:31] <wikibugs>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Zuul: CI does not trigger for https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Echo/+/832380 - https://phabricator.wikimedia.org/T318528 (10hashar)
[10:43:26] <wikibugs>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Zuul: CI does not trigger for https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Echo/+/832380 - https://phabricator.wikimedia.org/T318528 (10hashar) From the Zuul logs: ` 2022-09-22 15:49:59,737 DEBU...
[10:43:41] <wikibugs>	 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Zuul: CI does not trigger for https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Echo/+/832380 - https://phabricator.wikimedia.org/T318528 (10hashar) 05Open→03Resolved a:03hashar The series is:...
[10:49:14] <wikibugs>	 (03PS1) 10Hashar: Revert "jjb: Migrate mediawiki-selenium-integration-docker from php72 to php74" [integration/config] - 10https://gerrit.wikimedia.org/r/834685 (https://phabricator.wikimedia.org/T318525)
[10:49:16] <wikibugs>	 (03PS1) 10Hashar: Revert "jjb: Migrate legacy-quibble-rubyselenium-docker from php72 to php74" [integration/config] - 10https://gerrit.wikimedia.org/r/835126 (https://phabricator.wikimedia.org/T318525)
[10:52:19] <hashar>	 !log Rolling quibble/ruby jobs from php 7.4 to 7.2: `mediawiki-selenium-integration-docker` `legacy-quibble-rubyselenium-docker` # T318525
[10:52:21] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[10:52:22] <stashbot>	 T318525: Wikibase CI failure: quibble-buster-php74-bundle does not exist - https://phabricator.wikimedia.org/T318525
[10:52:30] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] Revert "jjb: Migrate mediawiki-selenium-integration-docker from php72 to php74" [integration/config] - 10https://gerrit.wikimedia.org/r/834685 (https://phabricator.wikimedia.org/T318525) (owner: 10Hashar)
[10:52:35] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] Revert "jjb: Migrate legacy-quibble-rubyselenium-docker from php72 to php74" [integration/config] - 10https://gerrit.wikimedia.org/r/835126 (https://phabricator.wikimedia.org/T318525) (owner: 10Hashar)
[10:52:49] <wikibugs>	 (03CR) 10Tarrow: [C: 03+1] Revert "jjb: Migrate legacy-quibble-rubyselenium-docker from php72 to php74" [integration/config] - 10https://gerrit.wikimedia.org/r/835126 (https://phabricator.wikimedia.org/T318525) (owner: 10Hashar)
[10:53:17] <hashar>	 tarrow: sorry for the mess! :D
[10:53:32] <hashar>	 tarrow: there is nothing preventing us from having a Jenkins job using a non existent image
[10:53:39] <hashar>	 and i guess some mass text replacement was made
[10:53:53] <hashar>	 I will look at creating a php7.4 based images for the ruby selenium tests
[10:54:12] <wikibugs>	 10Release-Engineering-Team, 10Wikidata, 10wdwb-tech, 10Patch-For-Review, 10ci-test-error: Wikibase CI failure: quibble-buster-php74-bundle does not exist - https://phabricator.wikimedia.org/T318525 (10hashar) @Jdforrester-WMF has switched a lot of jobs from php 7.2 to 7.4 unfortunately we never haven't c...
[10:54:25] <wikibugs>	 (03Merged) 10jenkins-bot: Revert "jjb: Migrate mediawiki-selenium-integration-docker from php72 to php74" [integration/config] - 10https://gerrit.wikimedia.org/r/834685 (https://phabricator.wikimedia.org/T318525) (owner: 10Hashar)
[10:54:26] <hashar>	 or maybe it has already been proposed but not created
[10:54:38] <hashar>	 anyway lunch time!
[10:54:58] <wikibugs>	 (03Merged) 10jenkins-bot: Revert "jjb: Migrate legacy-quibble-rubyselenium-docker from php72 to php74" [integration/config] - 10https://gerrit.wikimedia.org/r/835126 (https://phabricator.wikimedia.org/T318525) (owner: 10Hashar)
[10:55:30] <tarrow>	 no problemo! Thanks for the speedy patch :)
[11:32:41] <James_F>	 hashar: Argh. Sorry tarrow.
[11:35:07] <tarrow>	 hehe, no worries :)
[11:35:20] <wikibugs>	 (03PS1) 10Jforrester: dockerfiles: Provide quibble-buster-php74-bundle [integration/config] - 10https://gerrit.wikimedia.org/r/835152
[11:36:15] <wikibugs>	 (03CR) 10Jforrester: [C: 03+2] dockerfiles: Provide quibble-buster-php74-bundle [integration/config] - 10https://gerrit.wikimedia.org/r/835152 (owner: 10Jforrester)
[11:38:11] <wikibugs>	 (03Merged) 10jenkins-bot: dockerfiles: Provide quibble-buster-php74-bundle [integration/config] - 10https://gerrit.wikimedia.org/r/835152 (owner: 10Jforrester)
[11:40:12] <James_F>	 !log Docker
[11:40:13] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[11:40:22] <James_F>	 Bah.
[11:40:30] <James_F>	 !log Docker: Building and publishing quibble-buster-php74-bundle
[11:40:31] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[11:46:08] <wikibugs>	 (03PS1) 10Jforrester: jjb: Migrate mediawiki-selenium-integration-docker from php72 to php74 [Re-apply] [integration/config] - 10https://gerrit.wikimedia.org/r/835128
[11:46:31] <wikibugs>	 (03PS1) 10Jforrester: jjb: Migrate legacy-quibble-rubyselenium-docker from php72 to php74 [Re-apply] [integration/config] - 10https://gerrit.wikimedia.org/r/835129
[11:48:07] <wikibugs>	 (03CR) 10CI reject: [V: 04-1] jjb: Migrate mediawiki-selenium-integration-docker from php72 to php74 [Re-apply] [integration/config] - 10https://gerrit.wikimedia.org/r/835128 (owner: 10Jforrester)
[11:59:36] <wikibugs>	 10Release-Engineering-Team, 10Scap, 10MW-on-K8s: Scap Mediawiki K8s deployments - https://phabricator.wikimedia.org/T318536 (10jnuche)
[12:02:42] <wikibugs>	 10Release-Engineering-Team (Bonus Level 🕹️), 10MW-on-K8s, 10serviceops, 10Patch-For-Review: Make scap deploy to kubernetes together with the legacy systems - https://phabricator.wikimedia.org/T299648 (10jnuche) 05Open→03Resolved Scap is already generating Helmfile configuration files as described here...
[12:02:51] <wikibugs>	 10Release-Engineering-Team, 10Scap, 10MW-on-K8s: Scap Mediawiki K8s deployments - https://phabricator.wikimedia.org/T318536 (10jnuche)
[12:02:54] <wikibugs>	 10Release-Engineering-Team (Bonus Level 🕹️), 10MW-on-K8s, 10serviceops, 10Patch-For-Review: Make scap deploy to kubernetes together with the legacy systems - https://phabricator.wikimedia.org/T299648 (10jnuche)
[12:03:20] <wikibugs>	 10Release-Engineering-Team, 10Scap: Perform mediawiki k8s deployments in parallel - https://phabricator.wikimedia.org/T318181 (10jnuche)
[12:03:24] <wikibugs>	 10Release-Engineering-Team (Bonus Level 🕹️), 10MW-on-K8s, 10serviceops, 10Patch-For-Review: Make scap deploy to kubernetes together with the legacy systems - https://phabricator.wikimedia.org/T299648 (10jnuche)
[12:03:49] <wikibugs>	 10Release-Engineering-Team, 10Scap: Honor flavor fields in MW K8s deployment config files - https://phabricator.wikimedia.org/T318538 (10jnuche)
[12:04:06] <wikibugs>	 10Release-Engineering-Team, 10Scap: Honor flavor fields in MW K8s deployment config files - https://phabricator.wikimedia.org/T318538 (10jnuche)
[12:04:08] <wikibugs>	 10Release-Engineering-Team, 10Scap, 10MW-on-K8s: Scap Mediawiki K8s deployments - https://phabricator.wikimedia.org/T318536 (10jnuche)
[12:04:20] <wikibugs>	 10Release-Engineering-Team, 10Scap: Verify images before MW K8s deployments - https://phabricator.wikimedia.org/T318223 (10jnuche)
[12:04:22] <wikibugs>	 10Release-Engineering-Team, 10Scap, 10MW-on-K8s: Scap Mediawiki K8s deployments - https://phabricator.wikimedia.org/T318536 (10jnuche)
[12:04:40] <wikibugs>	 10Release-Engineering-Team, 10Scap: Perform mediawiki k8s deployments in parallel - https://phabricator.wikimedia.org/T318181 (10jnuche)
[12:04:42] <wikibugs>	 10Release-Engineering-Team, 10Scap, 10MW-on-K8s: Scap Mediawiki K8s deployments - https://phabricator.wikimedia.org/T318536 (10jnuche)
[12:54:43] <wikibugs>	 (03PS2) 10Hashar: python: add no-deps flag for pip installation [blubber] - 10https://gerrit.wikimedia.org/r/833778 (https://phabricator.wikimedia.org/T316755)
[12:54:45] <wikibugs>	 (03PS1) 10Hashar: Add make blubber-buildkit-docker [blubber] - 10https://gerrit.wikimedia.org/r/835173
[12:59:09] <wikibugs>	 10Release-Engineering-Team, 10Scap: Migrate code for generating MW K8s images to Scap repository - https://phabricator.wikimedia.org/T318544 (10jnuche)
[12:59:27] <wikibugs>	 10Release-Engineering-Team, 10Scap: Migrate code for generating MW K8s images to Scap repository - https://phabricator.wikimedia.org/T318544 (10jnuche)
[12:59:29] <wikibugs>	 10Release-Engineering-Team, 10Scap, 10MW-on-K8s: Scap Mediawiki K8s deployments - https://phabricator.wikimedia.org/T318536 (10jnuche)
[13:16:38] <wikibugs>	 (03PS1) 10Zabe: zuul: Add Pikne to the allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/835182
[13:16:55] <wikibugs>	 (03PS2) 10Zabe: zuul: Add Pikne to the allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/835182
[13:20:11] <wikibugs>	 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-Phonos, 10Community-Tech (CommTech-Sprint-33), 10MW-1.40-notes (1.40.0-wmf.2; 2022-09-19), 10Patch-For-Review: Phonos links to unroutable domain/URL for the MP3 file - https://phabricator.wikimedia.org/T317417 (10TheresNoTime) >>! In T317417#8259615,...
[13:20:22] <James_F>	 hashar: For some reason I can't deploy https://gerrit.wikimedia.org/r/c/integration/config/+/835129 - it ends with "jenkins-jobs: commands succeeded" but it doesn't list the job being changed and https://integration.wikimedia.org/ci/job/legacy-quibble-rubyselenium-docker/jobConfigHistory/ doesn't update. Can you?
[13:26:37] <James_F>	 I manually switched it in the config and it seems fine.
[13:26:38] <James_F>	 Odd.
[13:29:31] <wikibugs>	 (03PS2) 10Jforrester: jjb: Migrate legacy-quibble-rubyselenium-docker from php72 to php74 [Re-apply] [integration/config] - 10https://gerrit.wikimedia.org/r/835129
[13:29:45] <wikibugs>	 (03CR) 10Jforrester: [C: 03+2] "Deployed (and tested)." [integration/config] - 10https://gerrit.wikimedia.org/r/835129 (owner: 10Jforrester)
[13:32:31] <wikibugs>	 (03Merged) 10jenkins-bot: jjb: Migrate legacy-quibble-rubyselenium-docker from php72 to php74 [Re-apply] [integration/config] - 10https://gerrit.wikimedia.org/r/835129 (owner: 10Jforrester)
[13:32:37] <wikibugs>	 (03PS2) 10Jforrester: jjb: Migrate mediawiki-selenium-integration-docker from php72 to php74 [Re-apply] [integration/config] - 10https://gerrit.wikimedia.org/r/835128
[13:32:45] <wikibugs>	 (03PS3) 10Jforrester: jjb: Migrate mediawiki-selenium-integration-docker from php72 to php74 [Re-try] [integration/config] - 10https://gerrit.wikimedia.org/r/835128
[13:36:27] <wikibugs>	 10Phabricator: Deleting a file server-side results in exception: Unknown edge constant "26"! at PhabricatorEdgeType.php:214 - https://phabricator.wikimedia.org/T318547 (10Aklapper)
[13:38:22] <wikibugs>	 (03CR) 10Jforrester: [C: 03+2] "Deployed. No new breakages." [integration/config] - 10https://gerrit.wikimedia.org/r/835128 (owner: 10Jforrester)
[13:40:16] <wikibugs>	 (03Merged) 10jenkins-bot: jjb: Migrate mediawiki-selenium-integration-docker from php72 to php74 [Re-try] [integration/config] - 10https://gerrit.wikimedia.org/r/835128 (owner: 10Jforrester)
[13:53:26] <wikibugs>	 (03PS7) 10Jforrester: [DNM] Zuul: [mediawiki/vendor] Drop PHP72 jobs, use only PHP74 ones [integration/config] - 10https://gerrit.wikimedia.org/r/833024
[13:53:27] <wikibugs>	 (03PS8) 10Jforrester: [DNM] Zuul: [mediawiki/core] Drop PHP 7.2 and PHP 7.3 testing for master and wmf [integration/config] - 10https://gerrit.wikimedia.org/r/827496 (https://phabricator.wikimedia.org/T261872)
[13:57:58] <wikibugs>	 (03PS1) 10Hashar: Define requirements.txt exhaustively [tools/scap] - 10https://gerrit.wikimedia.org/r/835193 (https://phabricator.wikimedia.org/T316755)
[13:59:58] <wikibugs>	 (03CR) 10CI reject: [V: 04-1] Define requirements.txt exhaustively [tools/scap] - 10https://gerrit.wikimedia.org/r/835193 (https://phabricator.wikimedia.org/T316755) (owner: 10Hashar)
[14:05:37] <wikibugs>	 (03PS1) 10Hashar: Normalize requirements files [tools/scap] - 10https://gerrit.wikimedia.org/r/835196
[14:06:50] <wikibugs>	 (03PS2) 10Hashar: Define requirements.txt exhaustively [tools/scap] - 10https://gerrit.wikimedia.org/r/835193 (https://phabricator.wikimedia.org/T316755)
[14:08:07] <wikibugs>	 (03CR) 10Hashar: "I think I found the issue (PythonConfig objects were not merged!)." [blubber] - 10https://gerrit.wikimedia.org/r/833778 (https://phabricator.wikimedia.org/T316755) (owner: 10Hashar)
[14:08:50] <wikibugs>	 (03CR) 10CI reject: [V: 04-1] Define requirements.txt exhaustively [tools/scap] - 10https://gerrit.wikimedia.org/r/835193 (https://phabricator.wikimedia.org/T316755) (owner: 10Hashar)
[14:10:17] <wikibugs>	 (03CR) 10Hashar: Define requirements.txt exhaustively (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/835193 (https://phabricator.wikimedia.org/T316755) (owner: 10Hashar)
[14:54:39] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+1] python: add no-deps flag for pip installation [blubber] - 10https://gerrit.wikimedia.org/r/833778 (https://phabricator.wikimedia.org/T316755) (owner: 10Hashar)
[14:56:03] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+2] Normalize requirements files [tools/scap] - 10https://gerrit.wikimedia.org/r/835196 (owner: 10Hashar)
[14:58:49] <hashar>	 I have fixed `go` locally :-]
[15:01:16] <wikibugs>	 (03Merged) 10jenkins-bot: Normalize requirements files [tools/scap] - 10https://gerrit.wikimedia.org/r/835196 (owner: 10Hashar)
[15:13:21] <wikibugs>	 10Release-Engineering-Team, 10Scap: Perform mediawiki k8s deployments in parallel - https://phabricator.wikimedia.org/T318181 (10dancy) 05Open→03Resolved
[15:13:23] <wikibugs>	 10Release-Engineering-Team, 10Scap, 10MW-on-K8s: Scap Mediawiki K8s deployments - https://phabricator.wikimedia.org/T318536 (10dancy)
[15:53:07] <wikibugs>	 (03PS1) 10Ahmon Dancy: Release 4.23.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/835224
[15:53:09] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+2] Release 4.23.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/835224 (owner: 10Ahmon Dancy)
[15:55:14] <wikibugs>	 (03Abandoned) 10Ahmon Dancy: Release 4.23.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/835224 (owner: 10Ahmon Dancy)
[15:59:22] <wikibugs>	 (03PS1) 10Ahmon Dancy: Release 4.23.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/835227
[16:17:06] <wikibugs>	 (03Abandoned) 10Ahmon Dancy: Release 4.23.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/835227 (owner: 10Ahmon Dancy)
[16:18:59] <wikibugs>	 (03PS1) 10Ahmon Dancy: Release 4.23.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/835229
[16:19:01] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+2] Release 4.23.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/835229 (owner: 10Ahmon Dancy)
[16:21:44] <wikibugs>	 (03Merged) 10jenkins-bot: Release 4.23.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/835229 (owner: 10Ahmon Dancy)
[17:08:54] <wikibugs>	 (03PS1) 10Ahmon Dancy: Add release-scripts/perform-release [tools/scap] - 10https://gerrit.wikimedia.org/r/835238
[17:14:13] <wikibugs>	 (03CR) 10CI reject: [V: 04-1] Add release-scripts/perform-release [tools/scap] - 10https://gerrit.wikimedia.org/r/835238 (owner: 10Ahmon Dancy)
[17:24:12] <wikibugs>	 (03PS2) 10Ahmon Dancy: Add release-scripts/perform-release [tools/scap] - 10https://gerrit.wikimedia.org/r/835238
[17:25:50] <wikibugs>	 10GitLab (Project Migration), 10Release-Engineering-Team: Create new GitLab project group: mwbot-rs - https://phabricator.wikimedia.org/T318068 (10brennen) 05Open→03Resolved a:03brennen Created group and added yourself and @ssastry as owners - couldn't find others as active users. Sorry for delay; group...
[18:55:28] <wikibugs>	 (03PS1) 10Dduvall: Provide a method of access control for stage actions [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835250
[18:55:32] <wikibugs>	 (03PS1) 10Dduvall: Restrict image names to derivations of the project name [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835251
[19:00:39] <wikibugs>	 (03PS2) 10Dduvall: Provide a method of access control for stage actions [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835250
[19:00:41] <wikibugs>	 (03PS2) 10Dduvall: Restrict image names to derivations of the project name [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835251
[19:01:37] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+2] Provide a method of access control for stage actions [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835250 (owner: 10Dduvall)
[19:02:48] <wikibugs>	 (03Merged) 10jenkins-bot: Provide a method of access control for stage actions [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835250 (owner: 10Dduvall)
[19:37:37] <wikibugs>	 (03PS1) 10Dduvall: Add @NonCPS annotation to ValidationException.getMessage [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835259
[19:38:22] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+2] Add @NonCPS annotation to ValidationException.getMessage [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835259 (owner: 10Dduvall)
[19:39:03] <wikibugs>	 (03CR) 10CI reject: [V: 04-1] Add @NonCPS annotation to ValidationException.getMessage [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835259 (owner: 10Dduvall)
[19:46:49] <wikibugs>	 (03PS2) 10Dduvall: Add @NonCPS annotation to ValidationException.getMessage [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835259
[19:48:38] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+2] Add @NonCPS annotation to ValidationException.getMessage [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835259 (owner: 10Dduvall)
[19:49:41] <wikibugs>	 (03Merged) 10jenkins-bot: Add @NonCPS annotation to ValidationException.getMessage [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835259 (owner: 10Dduvall)
[19:51:49] <wikibugs>	 (03PS1) 10Dduvall: jjb: Allow only certain Zuul pipelines to perform certain PipelineLib actions [integration/config] - 10https://gerrit.wikimedia.org/r/835262
[19:52:05] <wikibugs>	 (03PS2) 10Dduvall: jjb: Allow only certain Zuul pipelines to perform certain PipelineLib actions [integration/config] - 10https://gerrit.wikimedia.org/r/835262
[19:54:07] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+2] jjb: Allow only certain Zuul pipelines to perform certain PipelineLib actions [integration/config] - 10https://gerrit.wikimedia.org/r/835262 (owner: 10Dduvall)
[19:57:07] <wikibugs>	 (03Merged) 10jenkins-bot: jjb: Allow only certain Zuul pipelines to perform certain PipelineLib actions [integration/config] - 10https://gerrit.wikimedia.org/r/835262 (owner: 10Dduvall)
[20:00:19] <dduvall>	 !log regenerating 314 jobs for deployment of https://gerrit.wikimedia.org/r/835262
[20:00:20] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[20:24:54] <James_F>	 Ah, is this why I'm getting UnapprovedUsageException on test jobs?
[20:25:33] <James_F>	 https://integration.wikimedia.org/ci/job/function-evaluator-pipeline-test/jobConfigHistory/showDiffFiles?timestamp1=2022-05-04_13-37-57&timestamp2=2022-09-26_19-58-39 seemingly so.
[20:26:25] <wikibugs>	 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-Phonos, 10Community-Tech (CommTech-Sprint-34), 10MW-1.40-notes (1.40.0-wmf.2; 2022-09-19), 10Patch-For-Review: Phonos links to unroutable domain/URL for the MP3 file - https://phabricator.wikimedia.org/T317417 (10JMcLeod_WMF)
[20:27:42] <James_F>	 dduvall: Is it possible for the script to say what action isn't being allowed?
[20:33:46] <dancy>	 James_F: I re-approved the relevant scripts using the Jenkins UI
[20:34:53] <dancy>	 (so it should be happy now)
[20:37:04] <wikibugs>	 (03PS1) 10Ahmon Dancy: Don't call realpath in jjb-{delete,test,update} [integration/config] - 10https://gerrit.wikimedia.org/r/835266
[20:46:00] <wikibugs>	 (03CR) 10Dduvall: [C: 03+2] Don't call realpath in jjb-{delete,test,update} [integration/config] - 10https://gerrit.wikimedia.org/r/835266 (owner: 10Ahmon Dancy)
[20:47:53] <wikibugs>	 (03Merged) 10jenkins-bot: Don't call realpath in jjb-{delete,test,update} [integration/config] - 10https://gerrit.wikimedia.org/r/835266 (owner: 10Ahmon Dancy)
[20:49:03] <dduvall>	 James_F: `UnapprovedUsageException` is a bit different than what we just deployed, but it may have started as a result of job regeneration. i'm not sure
[20:51:58] <wikibugs>	 (03PS1) 10Ahmon Dancy: Double maintenance-disconnect-full-disks timeout to 360 seconds [integration/config] - 10https://gerrit.wikimedia.org/r/835270
[20:52:33] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+2] "I made this change manually in the UI a while back.  This commit makes it permanent." [integration/config] - 10https://gerrit.wikimedia.org/r/835270 (owner: 10Ahmon Dancy)
[20:52:54] <dancy>	 dduvall: That is the impression I get.
[20:53:23] <James_F>	 dancy: Thanks, checking now.
[20:54:20] <wikibugs>	 (03Merged) 10jenkins-bot: Double maintenance-disconnect-full-disks timeout to 360 seconds [integration/config] - 10https://gerrit.wikimedia.org/r/835270 (owner: 10Ahmon Dancy)
[20:54:31] <James_F>	 dduvall: Aha, yes, now it does say what's not allowed.
[20:54:38] <James_F>	  - stage `production` contains an action (`publish`) that is not allowed by this pipeline
[20:55:02] <dduvall>	 James_F: i see. ok, taking a look
[20:55:04] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Blocking 🧱), 10Wikimedia-Fundraising-CiviCRM: CI has somehow completely stopped for a pr at wikimedia/fundraising/crm master - https://phabricator.wikimedia.org/T317928 (10thcipriani) 05Open→03Resolved a:03dancy >>! In T317928#8242575...
[20:55:10] <James_F>	 How do we define the production stage to not exist?
[20:55:11] <dduvall>	 James_F: which job?
[20:55:17] <James_F>	 https://integration.wikimedia.org/ci/job/function-evaluator-pipeline-test/137/console
[20:55:22] <dduvall>	 ty
[20:55:45] <James_F>	 -> https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/services/function-evaluator/+/refs/heads/master/.pipeline/blubber.yaml
[20:56:01] <dduvall>	 seem i didn't catch your project when doing codesearch :)
[20:56:22] <James_F>	 Ha. :-)
[20:56:45] <James_F>	 Should we rename a stage or something?
[20:56:46] <dduvall>	 we're no longer allowing image publishing during pipelines other than gate-and-submit/postmerge/publish/post
[20:56:52] <James_F>	 (Our config is a real mess.)
[20:57:04] <dduvall>	 let me see what your pipeline is doing and how it might be refactored
[20:57:12] <James_F>	 Yeah, the only publish is for production: https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/services/function-evaluator/+/refs/heads/master/.pipeline/config.yaml
[21:00:07] <wikibugs>	 (03CR) 10Ahmon Dancy: Restrict image names to derivations of the project name (031 comment) [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835251 (owner: 10Dduvall)
[21:04:04] <dduvall>	 oh phooey. i know what's going on
[21:04:17] <James_F>	 dduvall: Did we screw up?
[21:04:20] <James_F>	 Sorry!
[21:04:25] <dduvall>	 no you're good :)
[21:04:32] <dduvall>	 we (i) messed up
[21:04:36] <James_F>	 Cool. (At least in this regard.)
[21:05:01] <dancy>	 Change is bad
[21:05:16] <dduvall>	 dancy: looks like PipelineBuilder is validating all defined pipelines, not just the ones that will run in the current context
[21:05:27] * dduvall hacks on a fix
[21:05:36] <James_F>	 Ah, yeah, that'd do it.
[21:05:56] <wikibugs>	 (03PS1) 10Ahmon Dancy: systemtest: Allow Jenkins port to be overridden [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835272
[21:06:05] <wikibugs>	 (03PS1) 10Ahmon Dancy: Makefile: fix mktemp warning [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835273
[21:07:58] <James_F>	 (Also I saw the task that Blubberoid is going away, so I'm going to have to re-work our team's local build scripts.)
[21:09:44] * bd808 needs to poke at that too for 3 projects
[21:10:24] <James_F>	 https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/WikiLambda/+/refs/heads/master/README.md#locally_built-services-for-development is our current set of instructions.
[21:10:48] <James_F>	 We probably should have a single source of truth for that stuff, it's not AW-specific (or Toolforge-specific I imagine?).
[21:14:23] <wikibugs>	 (03PS1) 10Dduvall: Only validate pipelines that are about to be executed [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835274
[21:14:48] <dduvall>	 ^ dancy if you have a moment
[21:15:13] <dancy>	 looking
[21:15:59] <dduvall>	 currently running system tests locally as well
[21:16:26] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+1] Only validate pipelines that are about to be executed [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835274 (owner: 10Dduvall)
[21:16:53] <dduvall>	 James_F, bd808: let me know if you need any direction with that (migration to blubber buildkit frontend)
[21:17:06] <bd808>	 James_F: I need to find my notes on this all again, but it should be possible to make a docker-compose file that points to your blubber.yaml and gives the variant name to build.
[21:17:12] <James_F>	 I was hoping someone else would move first so I could just copy-paste. :-)
[21:17:23] <James_F>	 bd808: Ooh, that sounds helpful.
[21:17:35] <bd808>	 I made that work at some point with some hints from dduvall 
[21:17:37] <dduvall>	 you should be able to treat the blubber.yaml file just as you would a Dockerfile once you add the magic `# syntax=` line
[21:17:52] <James_F>	 Right.
[21:17:54] <dduvall>	 the only thing that's different is that the target needs to be explicit
[21:17:58] <wikibugs>	 (03PS1) 10Ahmon Dancy: TESTING: Testing a commit that will fail CI [tools/scap] - 10https://gerrit.wikimedia.org/r/835275
[21:18:58] <dduvall>	 dancy: system tests pass
[21:19:07] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+2] Only validate pipelines that are about to be executed [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835274 (owner: 10Dduvall)
[21:20:09] <wikibugs>	 (03Merged) 10jenkins-bot: Only validate pipelines that are about to be executed [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835274 (owner: 10Dduvall)
[21:21:19] <dduvall>	 James_F: https://integration.wikimedia.org/ci/job/function-evaluator-pipeline-test/138/console has at least passed the validation stage
[21:21:31] <James_F>	 Woo.
[21:21:46] <dduvall>	 dancy: thanks for the review (again) :)
[21:21:48] <James_F>	 (eventually we'll want the publish step to create multiple images, but eh.)
[21:21:54] <James_F>	 Thank you dduvall and dancy!
[21:21:54] <dduvall>	 James_F: thanks for the patience :)
[21:21:57] <James_F>	 Any time.
[21:22:00] <wikibugs>	 (03CR) 10CI reject: [V: 04-1] TESTING: Testing a commit that will fail CI [tools/scap] - 10https://gerrit.wikimedia.org/r/835275 (owner: 10Ahmon Dancy)
[21:22:16] <James_F>	 Speaking of nerd-sniping, is T307797 something that only SRE ServiceOps can deal with or can this be a RelEng fix?
[21:22:17] <dduvall>	 James_F: yeah, i looked at your `.pipeline/config.yaml` and it should work as written
[21:22:18] <stashbot>	 T307797: Clean-up / delete old versions of service pipeline created docker images from the public docker registry? - https://phabricator.wikimedia.org/T307797
[21:22:24] <James_F>	 dduvall: Awesome.
[21:22:47] <dduvall>	 the publish action is allowed when running during gate-and-submit, postmerge, publish, and post
[21:22:58] <dduvall>	 as is `deploy`, `publish`, `trigger`
[21:23:02] <bd808>	 dduvall: should I expect to use a tag like "2022-09-21-184314-publish-buildkit-gateway" or do y'all plan to have more semver tags at some point?
[21:23:33] <dduvall>	 we're building an image upon tag pushes, so that's what i expect us to use going forward
[21:23:44] <dduvall>	 getting all git-opy on you
[21:23:49] <dduvall>	 er git-ops-y
[21:23:50] <James_F>	 bd808: Just use :latest. ;-)
[21:23:58] <dancy>	 haha
[21:24:00] <dduvall>	 :yolo
[21:24:05] <dancy>	 What could go wrong
[21:24:06] <James_F>	 What could possibly go wrong?
[21:24:20] <James_F>	 TBF, we actually need :latest for our images for the Beta Cluster hack.
[21:24:42] <dduvall>	 yeah, we have a cron to tag random past shas as `:yolo`
[21:28:14] <wikibugs>	 (03CR) 10Dduvall: Restrict image names to derivations of the project name (031 comment) [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835251 (owner: 10Dduvall)
[21:28:40] <James_F>	 https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/production/modules/role/manifests/beta/docker_services.pp for the curious/shocked.
[21:31:49] <Daimona>	 !log Applying schema change to the wikishared DB on beta for the CampaignEvents extension (1/2) # T318120
[21:31:52] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[21:31:52] <stashbot>	 T318120: Update database schema to support private registration - https://phabricator.wikimedia.org/T318120
[21:32:11] <bd808>	 James_F: I'll see your beta cluster and raise you a prod use of service::docker -- https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/production/modules/profile/manifests/wmcs/striker/docker.pp
[21:32:43] <bd808>	 I am using explict tags there though and not just :latest
[21:32:58] <dduvall>	 i like this game :)
[21:34:01] <James_F>	 This is “never have I ever” for prod cruft?
[21:35:53] <Reedy>	 Never have I ever committed a hack that I could almost guarantee would never be fixed
[21:36:02] <wikibugs>	 (03CR) 10Ahmon Dancy: Restrict image names to derivations of the project name (031 comment) [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835251 (owner: 10Dduvall)
[21:36:06] <bd808>	 or a version of "You Wanna See Something Really Scary?" from the Twilight Zone movie. ;)
[21:41:19] <James_F>	 Let's Play A Game: Wikimedia Hackathon edition.
[21:46:21] <Daimona>	 !log Applying schema changes to the wikishared DB on beta for the CampaignEvents extension # T318379 T318120
[21:46:24] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[21:46:25] <stashbot>	 T318379: Change storage of organizer roles - https://phabricator.wikimedia.org/T318379
[21:46:25] <stashbot>	 T318120: Update database schema to support private registration - https://phabricator.wikimedia.org/T318120
[21:46:40] <wikibugs>	 (03Abandoned) 10Ahmon Dancy: TESTING: Testing a commit that will fail CI [tools/scap] - 10https://gerrit.wikimedia.org/r/835275 (owner: 10Ahmon Dancy)
[21:47:22] <wikibugs>	 (03PS3) 10Dduvall: Restrict image names to derivations of the project name [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835251
[21:50:26] <wikibugs>	 (03PS4) 10Dduvall: Restrict image names to derivations of the project name [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835251
[21:50:33] <wikibugs>	 (03CR) 10Dduvall: Restrict image names to derivations of the project name (031 comment) [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835251 (owner: 10Dduvall)
[21:53:33] <wikibugs>	 (03CR) 10Dduvall: [C: 03+2] Makefile: fix mktemp warning [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835273 (owner: 10Ahmon Dancy)
[21:54:13] <wikibugs>	 (03CR) 10Dduvall: [C: 03+2] systemtest: Allow Jenkins port to be overridden [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835272 (owner: 10Ahmon Dancy)
[21:55:00] <wikibugs>	 (03Merged) 10jenkins-bot: Makefile: fix mktemp warning [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835273 (owner: 10Ahmon Dancy)
[22:00:43] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+2] Restrict image names to derivations of the project name [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835251 (owner: 10Dduvall)
[22:01:21] <wikibugs>	 (03PS2) 10Ahmon Dancy: systemtest: Allow Jenkins port to be overridden [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835272
[22:01:29] <wikibugs>	 (03CR) 10Ahmon Dancy: [C: 03+2] systemtest: Allow Jenkins port to be overridden [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835272 (owner: 10Ahmon Dancy)
[22:02:59] <wikibugs>	 (03Merged) 10jenkins-bot: Restrict image names to derivations of the project name [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835251 (owner: 10Dduvall)
[22:03:09] <wikibugs>	 10Release-Engineering-Team (Priority Backlog 📥), 10Epic, 10Release Pipeline (Blubber): Deprecate Blubber's CLI and microservice (blubberoid) interfaces - https://phabricator.wikimedia.org/T318289 (10bd808)
[22:04:43] <wikibugs>	 (03Merged) 10jenkins-bot: systemtest: Allow Jenkins port to be overridden [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/835272 (owner: 10Ahmon Dancy)
[22:30:00] <bd808>	 James_F: this seems to be the basic magic with docker-compose and blubber-buildkit -- https://gerrit.wikimedia.org/r/c/labs/striker/+/835287
[22:49:49] <wikibugs>	 10Release-Engineering-Team (Priority Backlog 📥), 10Epic, 10Release Pipeline (Blubber): Deprecate Blubber's CLI and microservice (blubberoid) interfaces - https://phabricator.wikimedia.org/T318289 (10bd808)
[23:01:05] <wikibugs>	 10Release-Engineering-Team (Priority Backlog 📥), 10Epic, 10Release Pipeline (Blubber): Deprecate Blubber's CLI and microservice (blubberoid) interfaces - https://phabricator.wikimedia.org/T318289 (10bd808)
[23:16:49] <wikibugs>	 10Release-Engineering-Team, 10Discovery-Search, 10Elasticsearch, 10Developer Productivity: Developer productivity: Shared ElasticSearch instance for local development environments, Quibble CI, and Patch Demo - https://phabricator.wikimedia.org/T312198 (10bd808) There is some precedent for this in https://w...
[23:32:41] <wikibugs>	 10Release-Engineering-Team (Priority Backlog 📥), 10Epic, 10Release Pipeline (Blubber): Deprecate Blubber's CLI and microservice (blubberoid) interfaces - https://phabricator.wikimedia.org/T318289 (10bd808)
[23:40:19] <wikibugs>	 10Release-Engineering-Team (Priority Backlog 📥), 10Epic, 10Release Pipeline (Blubber): Deprecate Blubber's CLI and microservice (blubberoid) interfaces - https://phabricator.wikimedia.org/T318289 (10bd808) >>! In T318289#8253975, @dduvall wrote: > @bd808, bringing you into the discussion since I know you are...