[00:02:45] 06Project-Admins: Create project tag for Wikinews volunteer developed software - https://phabricator.wikimedia.org/T402150#11096576 (10Gryllida) I already provided links and a description. "Project tag for volunteers to coordinate development of wiki software, including mediawiki, its extensions, and third party... [00:03:23] 06Project-Admins: Create project tag for Wikinews volunteer developed software - https://phabricator.wikimedia.org/T402150#11096577 (10Gryllida) "The only thing blocking this at the moment is the project description." 😢 [00:14:26] 06Project-Admins: Create project tag for Wikinews volunteer developed software - https://phabricator.wikimedia.org/T402150#11096597 (10thcipriani) 05Open→03Resolved a:03thcipriani >>! In T402150#11096576, @Gryllida wrote: > I already provided links and a description. "Project tag for volunteers to coor... [01:16:13] (03open) 10swfrench: Draft: make-container-image: introduce flavour metadata [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/207 (https://phabricator.wikimedia.org/T401721) [01:25:45] 10Scap, 06serviceops, 13Patch-For-Review: Provide MediaWiki app image PHP version in helm values - https://phabricator.wikimedia.org/T401721#11096756 (10Scott_French) Alright, after a bit of thought, I believe the simplest option is use the build report. As noted in [[ https://gitlab.wikimedia.org/repos/rele... [01:32:17] 10GitLab (Project Migration), 06Community-Tech, 06translatewiki.net, 10WS Export, 13Patch-For-Review: Migrate ws-export repo from GitHub to GitLab - https://phabricator.wikimedia.org/T395398#11096758 (10Samwilson) [01:32:23] 10GitLab (Project Migration), 06Community-Tech, 06translatewiki.net, 10WS Export, 13Patch-For-Review: Migrate ws-export repo from GitHub to GitLab - https://phabricator.wikimedia.org/T395398#11096759 (10Samwilson) Translatewiki.net is now updating the new repo, e.g. https://gitlab.wikimedia.org/toolforge... [02:59:11] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for awesomeaasim - https://phabricator.wikimedia.org/T402100#11096807 (10bd808) The shell account name from your Developer account is the gitlab username. https://ldap.toolforge.org/user/awesomeaasim [05:58:18] (03PS1) 10Jeena Huneidi: Update wikilambda daily selenium job [integration/config] - 10https://gerrit.wikimedia.org/r/1179783 (https://phabricator.wikimedia.org/T388056) [05:59:54] (03CR) 10Jeena Huneidi: "I added the parameters but in the jenkins UI I see them twice and the second ones are empty. I don't know how to override the existing par" [integration/config] - 10https://gerrit.wikimedia.org/r/1179783 (https://phabricator.wikimedia.org/T388056) (owner: 10Jeena Huneidi) [06:14:22] Project beta-code-update-eqiad build #561786: 04FAILURE in 1 min 21 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/561786/ [06:25:15] Yippee, build fixed! [06:25:16] Project beta-code-update-eqiad build #561787: 09FIXED in 2 min 14 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/561787/ [07:13:04] (03CR) 10Hashar: "The empty parameters are added by:" [integration/config] - 10https://gerrit.wikimedia.org/r/1179783 (https://phabricator.wikimedia.org/T388056) (owner: 10Jeena Huneidi) [08:09:39] 10Scap: SecurityPatchBot: Notify task when patch fails to apply during train deployment - https://phabricator.wikimedia.org/T402262 (10jnuche) 03NEW [10:06:44] 10GitLab (Project Migration), 06Community-Tech, 06translatewiki.net, 10WS Export, 13Patch-For-Review: Migrate ws-export repo from GitHub to GitLab - https://phabricator.wikimedia.org/T395398#11097429 (10Samwilson) 05Open→03In progress [10:32:02] 10Continuous-Integration-Config, 10MediaWiki-Core-Tests, 10MW-1.45-notes (1.45.0-wmf.16; 2025-08-26), 13Patch-For-Review, 07Technical-Debt: Drop deprecated phpunit.php and suite.xml - https://phabricator.wikimedia.org/T395470#11097580 (10Daimona) [11:02:51] (03update) 10sfaci: Added Metrics Platform Experimentation Lab deployment window to deployments calendar [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/186 (https://phabricator.wikimedia.org/T396742) [11:05:05] (03update) 10sfaci: Added Metrics Platform Experimentation Lab deployment window to deployments calendar [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/186 (https://phabricator.wikimedia.org/T396742) [11:17:53] 10GitLab (Project Migration), 06Community-Tech, 06translatewiki.net, 10WS Export, 13Patch-For-Review: Migrate ws-export repo from GitHub to GitLab - https://phabricator.wikimedia.org/T395398#11097865 (10Samwilson) [11:28:09] 10Scap: SecurityPatchBot: Notify task when patch fails to apply during train deployment - https://phabricator.wikimedia.org/T402262#11097908 (10jnuche) [11:30:04] maintenance-disconnect-full-disks build 729476 integration-agent-docker-1054 (/: 40%, /srv: 97%, /var/lib/docker: 31%): OFFLINE due to disk space [11:35:04] maintenance-disconnect-full-disks build 729477 integration-agent-docker-1054 (/: 40%, /srv: 27%, /var/lib/docker: 30%): RECOVERY disk space OK [11:49:46] dancy: thanks for the review! [11:52:03] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Allow admins to rename personal Herald rules - https://phabricator.wikimedia.org/T386703#11097996 (10Aklapper) 05Declined→03Open p:05Triage→03Low a:03Aklapper I'm going to reopen because once again came back to this thought today as it i... [12:01:08] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.45.0-wmf.15 deployment blockers - https://phabricator.wikimedia.org/T396376#11098036 (10Urbanecm_WMF) [12:12:04] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Allow admins to rename personal Herald rules - https://phabricator.wikimedia.org/T386703#11098052 (10Aklapper) Playing locally with a "Personal Rule" created by a non-admin account, ` diff --git a/src/applications/herald/storage/HeraldRule.php b/s... [12:17:09] Yippee, build fixed! [12:17:09] Project beta-scap-sync-world build #220169: 09FIXED in 1 min 58 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/220169/ [12:20:12] 06Project-Admins: Create project tag for Wikinews volunteer developed software - https://phabricator.wikimedia.org/T402150#11098065 (10Gryllida) Thank you @thcipriani 😊 Please could you help with access or point me to some documentation, {F65786491} [12:23:59] 06Project-Admins: Create project tag for Wikinews volunteer developed software - https://phabricator.wikimedia.org/T402150#11098066 (10Aklapper) Hi, see https://www.mediawiki.org/wiki/Phabricator/Project_management#Boards : "Editing a project and its columns requires membership in [Trusted-Contributors](http... [12:27:21] 06Project-Admins: Create project tag for Wikinews volunteer developed software - https://phabricator.wikimedia.org/T402150#11098068 (10Aklapper) I realize I should probably just post the usual text after project creation here. :) Let me do that: Requested public project has been created: https://phabricator... [12:31:13] (03open) 10aklapper: Allow admins to rename / edit personal Herald rules [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/94 (https://phabricator.wikimedia.org/T386703) [13:12:51] 10Phabricator, 10Release-Engineering-Team (Doing 😎): Herald rule to add Connection team tag to maintained projects - https://phabricator.wikimedia.org/T402240#11098244 (10Aklapper) 05Open→03Resolved a:03Aklapper Hi, done in H466. I also boldly added #Campaign-Registration: * When any of these condit... [13:13:06] 10Phabricator, 10Release-Engineering-Team (Doing 😎), 06Connection-Team: Herald rule to add Connection team tag to maintained projects - https://phabricator.wikimedia.org/T402240#11098248 (10Aklapper) [13:36:05] !log lucaswerkmeister-wmde@deployment-puppetserver-1:/srv/git/operations/puppet$ pgit revert 20ea82337d # T401592 [13:36:07] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:36:07] T401592: Property suggestor not appearing on Beta - https://phabricator.wikimedia.org/T401592 [13:38:52] (03open) 10jnuche: patches: enhance patch failure notifications [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/986 (https://phabricator.wikimedia.org/T402262) [13:39:39] hm, `sudo puppetserver-deploy-code` doesn’t work [13:39:45] (on deployment-puppetserver-1) [13:40:02] “WARN: git repository /srv/git/operations/puppet does not exist or is unreachable at this moment!” due to a “dubious ownership” error [13:41:18] (03update) 10jnuche: patches: enhance patch failure notifications [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/986 (https://phabricator.wikimedia.org/T402262) [13:43:21] Lucas_WMDE: I know nothing about that script sorry :/ [13:43:26] (`sudo -u gitpuppet puppetserver-deploy-code` also doesn’t work fwiw) [13:46:49] huh, according to https://gerrit.wikimedia.org/r/c/operations/puppet/+/1060919 “we're running puppetserver-deploy-code as gitpuppet” [13:46:58] why does that give me an error then [13:46:59] “Error: /var/cache/g10k exists, but is not writable! Exiting!” [13:47:21] did I break it by running it as root first (per the docs at https://wikitech.wikimedia.org/wiki/Help:Project_puppetserver#Deploying_changes ?) [13:47:53] no, it looks like that directory has existed since 2024 [13:48:37] 10Phabricator, 10Release-Engineering-Team (Doing 😎), 06Connection-Team: Herald rule to add Connection team tag to maintained projects - https://phabricator.wikimedia.org/T402240#11098388 (10Daimona) Looks correct to me, thank you! [13:49:54] anyone around who knows how to make (or rather revert) puppet changes in beta correctly? [13:56:03] Lucas_WMDE: I think that wrapper was added by t.aavi. But you should be able to use raw git commands in /srv/git/operations/puppet [13:56:23] that’s what I did (using the pgit wrapper so it’s the right user) [13:56:39] I’m mainly not sure how to get it “deployed” after that [13:56:49] yeah let me try [13:57:11] I think I originally wrote `pgit`, but definitely not `puppetserver-deploy-code` [13:57:31] if I SSH into another deployment-prep instance I can see my revert as the “last puppet commit” in the banner, so that part looks okay [13:57:35] anyway, that script is a total mess, but I think the deploy script is supposed to be run as root [13:57:44] I have reset --hard HEAD^^ [13:57:59] which dropped: [13:57:59] 33d05192cd (HEAD -> production) Revert "mediawiki: Remove unused wikidata.org vhost and fix redirect in beta" [13:57:59] 20ea82337d (tag: snapshot-202508191327) mediawiki: Remove unused wikidata.org vhost and fix redirect in beta [13:58:45] + g10k_envdir=/home/gitpuppet/.puppet/code/environments_staging [13:58:45] + g10k -quiet -branch production -config /etc/puppet/g10k.conf [13:58:45] checkDirAndCreate(): Error: /var/cache/g10k exists, but is not writable! Exiting! [13:58:48] * hashar whistles [13:58:58] who knows what `g10k` is [13:59:14] ok, so that’s as far as I got, except you’ve erased the Git record of what I did… [13:59:25] which record? [13:59:37] the revert commit, with a message explaining why I reverted it [13:59:57] well I reset back to that state [14:00:37] rerr [14:00:45] Lucas_WMDE: I can reset to the state as I pasted above [14:01:42] but if your patch broke something you can simply remove it from the stack of patches [14:01:45] instead of doing a revert [14:01:52] the patch I reverted wasn’t mine [14:02:18] ah yeah that was by Timo [14:02:24] I had told Krinkle a few hours ago on https://gerrit.wikimedia.org/r/c/operations/puppet/+/1179719 that it seemed to break mobile beta wikidata, and got no reply, so I thought I’d try reverting it [14:02:40] and do it as a git revert instead of a reset so that there would be an explanation in the git log, besides the comments on the Gerrit change [14:02:42] well I guess we can drop both patches and mentions the patch got removed from beta [14:04:15] well, it looks like mobile beta wikidata is working again now [14:04:34] I suspect we just needed to wait half an hour for puppet to apply changes [14:04:38] !log deployment-prep: sudo systemctl start puppet-git-sync-upstream.service [14:04:39] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:04:41] but maybe the reset vs revert made the difference, idk [14:04:49] or that… [14:04:51] who knows [14:04:59] then we can run puppet on the instance that needs it [14:06:03] hmm dpeloyment-mediawiki14 had puppet running: [14:06:05] The last Puppet run was at Tue Aug 19 13:58:03 UTC 2025 (7 minutes ago). [14:06:05] Last Puppet commit: (33d05192cd) Lucas Werkmeister - Revert "mediawiki: Remove unused wikidata.org vhost and fix redirect in beta" [14:06:35] same for deployment-mediawiki13 [14:07:32] that’s what I mentioned above already, I think? [14:07:34] “if I SSH into another deployment-prep instance I can see my revert as the “last puppet commit” in the banner” [14:08:01] presumably that’ll go away soon now, to reflect the git reset [14:10:09] * Lucas_WMDE is done touching puppet for now [14:25:57] 10Phabricator (phabricator-next), 06Release-Engineering-Team, 06collaboration-services: Deploy Phabricator/Phorge 2025-08-19 - https://phabricator.wikimedia.org/T402309 (10brennen) 03NEW [14:26:53] (03open) 10brennen: update submodules for 2025-08-19 deploy [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/77 (https://phabricator.wikimedia.org/T402309) [14:27:32] brennen: hi, I have fixed up the puppet patch to bump the php APC limit from 32MB to 256MB https://gerrit.wikimedia.org/r/c/operations/puppet/+/1175916 [14:27:59] maybe you want it applied as before/after/as part of the upgrade [14:29:41] Lucas_WMDE: I think periodic rebase made (or would have made) the revert dissappear anyhow :) [14:29:50] Re-applied now with an attempted fix [14:30:10] thanks! [14:30:22] (03open) 10swfrench: Draft: kubernetes: adopt new report structure and set php.version [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/987 (https://phabricator.wikimedia.org/T401721) [14:31:01] what I had in mind was that you’d clean up the git history when you got back fwiw, I just didn’t know when that would be ^^ [14:31:32] (03merge) 10brennen: update submodules for 2025-08-19 deploy [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/77 (https://phabricator.wikimedia.org/T402309) [14:32:36] 10Scap, 07Essential-Work, 13Patch-For-Review: SecurityPatchBot: Notify task when patch fails to apply during train deployment - https://phabricator.wikimedia.org/T402262#11098651 (10jnuche) [14:34:21] Lucas_WMDE: np yeah, sounds good :) [14:34:28] hm, now I get “We can’t connect to the server at m.wikidata.beta.wmcloud.org.” [14:35:35] maybe I caught the server in the middle of a restart or something [14:36:37] and now it seems to be timing out instead [14:37:28] FIRING: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [14:37:35] 10Beta-Cluster-Infrastructure: Project deployment-prep instance deployment-cache-text08 is down - https://phabricator.wikimedia.org/T402311 (10wmcs-alerts) 03NEW [14:37:37] Yeah, it's still running puppet [14:37:40] ok [14:38:08] Notice: /Stage[main]/Profile::Cache::Varnish::Frontend/Varnish::Instance[text-frontend]/Exec[load-new-vcl-file-frontend]/returns: subprocess.CalledProcessError: Command '['/usr/bin/varnishadm', '-n', 'frontend', 'vcl.label', 'wikimedia_misc', 'vcl-a1f9de39-9991-434d-9249-bbb27c9d2c52']' returned non-zero exit status 2. [14:38:18] Not sure what it's doing , it's seemingly just spinning its wheels [14:38:32] Notice: /Stage[main]/Profile::Cache::Varnish::Frontend/Varnish::Instance[text-frontend]/Exec[load-new-vcl-file-frontend]/returns: Rejected 400 [14:38:32] Notice: /Stage[main]/Profile::Cache::Varnish::Frontend/Varnish::Instance[text-frontend]/Exec[load-new-vcl-file-frontend]/returns: CLI communication error (hdr) [14:38:32] Notice: /Stage[main]/Profile::Cache::Varnish::Frontend/Varnish::Instance[text-frontend]/Exec[load-new-vcl-file-frontend]/returns: Executing: "/usr/bin/varnishadm -n frontend vcl.load vcl-a1f9de39-9991-434d-9249-bbb27c9d2c52 /etc/varnish/wikimedia_misc-frontend.vcl" [14:38:38] Once it exists I'll run it again [14:39:18] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.45.0-wmf.15 deployment blockers - https://phabricator.wikimedia.org/T396376#11098703 (10jnuche) [14:39:39] 10Beta-Cluster-Infrastructure: Project deployment-prep instance deployment-cache-text08 is down - https://phabricator.wikimedia.org/T402311#11098711 (10Lucas_Werkmeister_WMDE) Possibly related to ongoing work at T401592 / [Gerrit](https://gerrit.wikimedia.org/r/c/operations/puppet/+/1179719). (Or the timing coul... [14:40:01] It's not even responding to ctrl-C [14:41:56] https://grafana.wmcloud.org/d/0g9N-7pVz/cloud-vps-project-board?var-project=deployment-prep&orgId=1&from=now-1h&to=now&timezone=browser&var-instance=deployment-cache-text08 [14:42:28] RESOLVED: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [14:42:37] 10Beta-Cluster-Infrastructure: Project deployment-prep instance deployment-cache-text08 is down - https://phabricator.wikimedia.org/T402311#11098727 (10wmcs-alerts) [14:43:34] the whole VM is overloaded? (or was?) [14:43:47] no [14:43:50] I was expecting it to be [14:43:52] but it wasn't [14:44:04] re-running puppet now to see if it sorts stuff out or not [14:44:17] well... [14:44:21] I thought the gap in Grafana meant it was too busy to even respond to Prometheus [14:44:25] I suppose if it was unresponsive for a while, it wouldn't... ^ [14:44:26] yeah [14:44:30] (until it managed to report load values in the double digits) [14:44:32] ok ^^ [14:45:29] maybe whatever state our VCL code is in as of $when means Varnish can't do live reloads anymore [14:45:36] given the size of the VM [14:45:55] it's probably going to tail spin again during this run [14:46:40] * Lucas_WMDE knows nothing about VCL [14:47:27] but Grafana looks like it’s very busy again yeah [14:49:54] well, it finished running on its own eventually [14:50:13] seems to work fine [14:50:37] Lucas_WMDE: let me know if it (beta wikidata) works correclty for you as well! [14:50:44] trying it now [14:51:57] seems to work fine \o/ [14:52:10] (except that firefox refuses to reset the zoom level to 100% o_O but that’s certainly unrelated ^^) [14:52:28] both on mobile and when using either m. or www. on desktop [14:52:29] \o/ [14:55:26] thanks a lot Krinkle! [14:55:42] (Grafana also looks alright again) [14:57:06] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.45.0-wmf.15 deployment blockers - https://phabricator.wikimedia.org/T396376#11098822 (10jnuche) [14:58:23] 10Beta-Cluster-Infrastructure: Project deployment-prep instance deployment-cache-text08 is down - https://phabricator.wikimedia.org/T402311#11098827 (10Lucas_Werkmeister_WMDE) 05Open→03Resolved Gonna boldly close this – the instance was temporarily overloaded during some VCL work (see [Grafana](https://g... [15:07:12] 10Phabricator (2025-08-19), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Deploy Phabricator/Phorge 2025-08-19 - https://phabricator.wikimedia.org/T402309#11098874 (10Aklapper) Included: * {T399604} / https://gerrit.wikimedia.org/r/c/phabricator/translations/+/1175157 * Some sma... [15:15:38] 10Release-Engineering-Team (Doing 😎): Improve garbage collection of unused MediaWiki images on deployment host - https://phabricator.wikimedia.org/T387927#11098914 (10dancy) Noting that as currently written, `scap clean-images` can only be executed by people in the `docker` group. We should improve that to make... [15:25:40] (03approved) 10dancy: patches: enhance patch failure notifications [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/986 (https://phabricator.wikimedia.org/T402262) (owner: 10jnuche) [16:02:07] 06Release-Engineering-Team, 06Content-Transform-Team, 10Quibble, 10Catalyst (PatchDemo), and 3 others: Not adding trailing / to autoload patches breaks Parsoid CI - https://phabricator.wikimedia.org/T400299#11099148 (10hashar) @cscott CI should be all set now :) Your patch //[[ https://gerrit.wikimedia.or... [16:21:14] 10Phabricator, 10Release-Engineering-Team (Doing 😎), 06Connection-Team: Herald rule to add Connection team tag to maintained projects - https://phabricator.wikimedia.org/T402240#11099242 (10Daimona) 05Resolved→03Open Apologies, I just noticed a serious issue with the current rule: whenever a task is tagg... [16:23:54] 10Phabricator, 10Release-Engineering-Team (Doing 😎), 06Connection-Team: Herald rule to add Connection team tag to maintained projects - https://phabricator.wikimedia.org/T402240#11099255 (10Aklapper) 05Open→03Resolved Done. [16:25:59] (03update) 10swfrench: make-container-image: introduce flavour metadata [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/207 (https://phabricator.wikimedia.org/T401721) [16:26:04] (03update) 10swfrench: make-container-image: introduce flavour metadata [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/207 (https://phabricator.wikimedia.org/T401721) [16:26:52] 10GitLab (Upstream pit of despair 🕳️), 10Release-Engineering-Team (Seen): Gitlab "Review changes" button not working - https://phabricator.wikimedia.org/T400091#11099287 (10dancy) Upstream has closed https://gitlab.com/gitlab-org/gitlab/-/issues/553219 as "cannot reproduce". I have confirmed that the "Review... [16:27:23] 10GitLab (Upstream pit of despair 🕳️), 10Release-Engineering-Team (Seen): Gitlab "Review changes" button not working - https://phabricator.wikimedia.org/T400091#11099290 (10dancy) @Jelto Adding you to make you aware of this bug that probably won't be fixed until we upgrade our Gitlab instance. [16:27:38] (03approved) 10dancy: make-container-image: introduce flavour metadata [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/207 (https://phabricator.wikimedia.org/T401721) (owner: 10swfrench) [17:13:10] 10Phabricator, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Phorge setup check caching is misbehaving, leading to many duck-sound=quack requests - https://phabricator.wikimedia.org/T401157#11099479 (10Dzahn) After deploying the config change (and restarting php7.4-fpm) we can n... [17:13:32] (03merge) 10swfrench: make-container-image: introduce flavour metadata [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/207 (https://phabricator.wikimedia.org/T401721) [17:20:19] 10Scap, 06serviceops, 13Patch-For-Review: Provide MediaWiki app image PHP version in helm values - https://phabricator.wikimedia.org/T401721#11099540 (10Scott_French) Alright, [[ https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/207 | release!207 ]] is live, and the updated build report look... [17:21:15] (03update) 10swfrench: kubernetes: adopt new report structure and set php.version [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/987 (https://phabricator.wikimedia.org/T401721) [17:30:57] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services, 07Essential-Work: Puppetize systemd unit for zuul's nodepool - https://phabricator.wikimedia.org/T401614#11099578 (10Dzahn) Hello, so regarding the mount points, I ensured thes... [17:42:57] (03update) 10sfaci: Added Metrics Platform Experimentation Lab deployment window to deployments calendar [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/186 (https://phabricator.wikimedia.org/T396742) [17:58:02] 10Beta-Cluster-Infrastructure, 13Patch-For-Review: Beta cluster IP block page should not point to noc@wikimedia.org - https://phabricator.wikimedia.org/T393404#11099686 (10bd808) 05Open→03Resolved [18:01:33] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services, 07Essential-Work, 13Patch-For-Review: Puppetize systemd unit for zuul's nodepool - https://phabricator.wikimedia.org/T401614#11099704 (10bd808) a:05bd808→03Dzahn Unlickin... [18:09:51] (03update) 10dancy: kubernetes: adopt new report structure and set php.version [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/987 (https://phabricator.wikimedia.org/T401721) (owner: 10swfrench) [18:09:58] (03merge) 10dancy: kubernetes: adopt new report structure and set php.version [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/987 (https://phabricator.wikimedia.org/T401721) (owner: 10swfrench) [18:10:45] (03open) 10dancy: Release 4.206.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/988 [18:11:43] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services, 07Essential-Work, 13Patch-For-Review: Puppetize systemd unit for zuul's nodepool - https://phabricator.wikimedia.org/T401614#11099734 (10bd808) >>! In T401614#11099578, @Dzah... [18:13:10] (03merge) 10dancy: Release 4.206.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/988 [18:15:37] 06Release-Engineering-Team, 06collaboration-services: ProbeDown - https://phabricator.wikimedia.org/T402331 (10phaultfinder) 03NEW [18:15:38] FIRING: ProbeDown: Service gerrit1003:443 has failed probes (http_gerrit_tls_ip6) - https://wikitech.wikimedia.org/wiki/Runbook#gerrit1003:443 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://alerts.wikimedia.org/?q=alertname%3DProbeDown [18:20:31] RESOLVED: [2x] ProbeDown: Service gerrit1003:443 has failed probes (http_gerrit_tls_ip4) - https://wikitech.wikimedia.org/wiki/Runbook#gerrit1003:443 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://alerts.wikimedia.org/?q=alertname%3DProbeDown [18:23:55] hi releng folks, we're seeing an odd failure on gate-and-submit for the DonationInterface extension [18:24:07] I think this is the relevant line: [18:24:10] https://integration.wikimedia.org/ci/job/mwgate-composer-php82/1539/console [18:24:21] 14:20:02 + git init --initial-branch=master [18:24:22] 14:20:02 /src/.git: Permission denied [18:24:33] any suggestions? [18:24:53] I hit rebuild to see if that happens a second time. [18:25:07] https://integration.wikimedia.org/ci/job/mwgate-composer-php82/1540/console [18:25:46] looks like it's fine now? permission denied on /srv/.git that goes away sounds like unlucky timing / race ? [18:26:59] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services, 07Essential-Work, 13Patch-For-Review: Puppetize systemd unit for zuul's nodepool - https://phabricator.wikimedia.org/T401614#11099815 (10bd808) >>! In T401614#11099578, @Dzah... [18:27:42] thanks mutante [18:29:35] ejegg: did you get a V+1 on the change now from jenkins bot? if not you can comment "recheck" to trigger that [18:31:11] thanks, got it merged now! [18:34:02] ok, cool [18:46:11] 10Scap, 06serviceops: Provide MediaWiki app image PHP version in helm values - https://phabricator.wikimedia.org/T401721#11099918 (10Scott_French) This is now live and exports `php.version` as expected, e.g. ` swfrench@deploy1003:~$ cat /etc/helmfile-defaults/mediawiki/release/mw-web-main.yaml docker: regi... [21:37:55] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for SaintWolf - https://phabricator.wikimedia.org/T399836#11100432 (10Aklapper) 05Open→03Stalled @SaintWolf That sounds like a great project! Did you manage to get the code that you need? If not, could you provide... [21:39:42] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for Meetmendapara - https://phabricator.wikimedia.org/T398769#11100441 (10Aklapper) 05Open→03Declined Unfortunately closing this Phabricator task as no further information has been provided. @Meetmendapara: A... [21:55:03] 06Release-Engineering-Team: Add a staging kubernetes cluster to train-dev - https://phabricator.wikimedia.org/T402350 (10dancy) 03NEW [21:56:47] 10Release-Engineering-Team (Priority Backlog 📥), 07Epic, 07OKR-Work: [WE6.1.1] Move image build to deployment server and update for backports - https://phabricator.wikimedia.org/T398868#11100503 (10bd808) [21:56:48] 06Release-Engineering-Team: Add a staging kubernetes cluster to train-dev - https://phabricator.wikimedia.org/T402350#11100504 (10bd808) [22:05:33] 10Release-Engineering-Team (Priority Backlog 📥): Add a staging kubernetes cluster to train-dev - https://phabricator.wikimedia.org/T402350#11100541 (10bd808) [22:38:28] FIRING: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [22:38:37] 10Beta-Cluster-Infrastructure: Project deployment-prep instance deployment-cache-text08 is down - https://phabricator.wikimedia.org/T402355 (10wmcs-alerts) 03NEW [22:43:28] RESOLVED: InstanceDown: Project deployment-prep instance deployment-cache-text08 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [22:43:39] 10Beta-Cluster-Infrastructure: Project deployment-prep instance deployment-cache-text08 is down - https://phabricator.wikimedia.org/T402355#11100632 (10wmcs-alerts) [23:12:27] 10Beta-Cluster-Infrastructure: Project deployment-prep instance deployment-cache-text08 is down - https://phabricator.wikimedia.org/T402355#11100674 (10bd808) 05Open→03Invalid Flapping check? {T402311} [23:53:41] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.45.0-wmf.16 deployment blockers - https://phabricator.wikimedia.org/T396377#11100705 (10SecurityPatchBot)