[01:05:18] 10Release-Engineering-Team (Next), 10Release, 10Train Deployments: 1.38.0-wmf.9 deployment blockers - https://phabricator.wikimedia.org/T293950 (10Jdlrobson) [03:11:24] PROBLEM - SSH on contint1001.mgmt is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [03:13:15] https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar says that Thursday should be a no deploy day, but it isn't currently marked as such [07:15:42] RECOVERY - SSH on contint1001.mgmt is OK: SSH OK - OpenSSH_6.6 (protocol 2.0) https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [07:19:19] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10dev-images, 10mwcli, 10User-brennen: Add php-luasandbox to dev-images used by mwcli - https://phabricator.wikimedia.org/T286678 (10brennen) [07:21:14] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10GitLab (Auth & Access), 10Upstream, 10User-brennen: Gitlab 2fa password validation seems bugged - https://phabricator.wikimedia.org/T292431 (10brennen) [07:23:00] 10Release-Engineering-Team (Priority Backlog 📥), 10Security-Team, 10GitLab (CI & Job Runners), 10SecTeam-Processed, 10User-brennen: Limit GitLab shared runners to trusted contributors - https://phabricator.wikimedia.org/T292094 (10brennen) [07:23:28] 10Release-Engineering-Team (Priority Backlog 📥), 10dev-images, 10mwcli, 10User-brennen: Add php-luasandbox to dev-images used by mwcli - https://phabricator.wikimedia.org/T286678 (10brennen) [09:14:15] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Zuul, 10ci-test-error (WMF-deployed Build Failure): fatal: remote error: access denied or repository not exported: /mediawiki/extensions/ReadingLists - https://phabricator.wikimedia.org/T187897 (10kostajh) Just saw this for Growt... [09:59:38] 10Phabricator, 10SRE: migrate RT maint-announce into phabricator - https://phabricator.wikimedia.org/T118176 (10Aklapper) [10:00:10] 10Release-Engineering-Team, 10GitLab: GitLab should not display ads for paid versions - https://phabricator.wikimedia.org/T295453 (10Majavah) [10:00:12] 10Phabricator (Upstream), 10Developer Productivity, 10Upstream: Disable parent task metadata by default for new sub tasks - https://phabricator.wikimedia.org/T239378 (10Aklapper) [10:08:54] (03PS3) 10Hashar: jjb: add the template name to the job description [integration/config] - 10https://gerrit.wikimedia.org/r/737723 [10:13:22] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Doing), 10Datacenter-Switchover: Create a runbook for switching CI master - https://phabricator.wikimedia.org/T256396 (10hashar) a:05hashar→03None [10:13:40] 10Continuous-Integration-Config, 10Release-Engineering-Team (Doing), 10Patch-For-Review: Add a CI Job for running bundlesize tests in Vector - https://phabricator.wikimedia.org/T255149 (10hashar) a:05hashar→03None [10:13:46] 10Continuous-Integration-Config, 10Release-Engineering-Team (Doing), 10phan: Phan job should use composer instead of vendor for release branches ( undeclared class \Wikimedia\Equivset\Equivset ) - https://phabricator.wikimedia.org/T231966 (10hashar) a:05hashar→03None [10:13:56] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Doing), 10ci-test-error: Various CI jobs failing after "mkdir: cannot create directory ‘log’: Permission denied" - https://phabricator.wikimedia.org/T282893 (10hashar) a:05hashar→03None [10:29:40] 10Phabricator: Autogeneration of Subtasks - https://phabricator.wikimedia.org/T244782 (10Aklapper) (see also slightly related {T239378} about the problems with creating subtasks nowadays) [11:28:45] 10Continuous-Integration-Config, 10Release-Engineering-Team (Next), 10MediaWiki-Core-Tests, 10Code-Health, and 6 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10kostajh) >>! In T225730#7482641, @Ladsgroup wrote: > If we move selenium to daily jo... [15:04:27] (03CR) 10Jforrester: [C: 03+1] "Neat." [integration/config] - 10https://gerrit.wikimedia.org/r/737677 (owner: 10Hashar) [15:05:49] 10Beta-Cluster-Infrastructure, 10Abstract Wikipedia team, 10function-evaluator, 10function-orchestrator: Work out how we're going to have "production-like" versions of the wikifunctions evaluator and orchestrator services in Beta Cluster - https://phabricator.wikimedia.org/T289315 (10Jdforrester-WMF) >>! I... [15:19:24] 10Beta-Cluster-Infrastructure, 10Abstract Wikipedia team, 10Cloud-VPS (Project-requests): Request creation of integration-k8s VPS project - https://phabricator.wikimedia.org/T295477 (10Jdforrester-WMF) [15:19:47] 10Beta-Cluster-Infrastructure, 10Abstract Wikipedia team, 10function-evaluator, 10function-orchestrator: Work out how we're going to have "production-like" versions of the wikifunctions evaluator and orchestrator services in Beta Cluster - https://phabricator.wikimedia.org/T289315 (10Jdforrester-WMF) [15:19:49] 10Beta-Cluster-Infrastructure, 10Abstract Wikipedia team, 10function-evaluator, 10function-orchestrator: Work out how we're going to have "production-like" versions of the wikifunctions evaluator and orchestrator services in Beta Cluster - https://phabricator.wikimedia.org/T289315 (10Jdforrester-WMF) Filed... [15:19:51] 10Beta-Cluster-Infrastructure, 10Abstract Wikipedia team, 10Cloud-VPS (Project-requests): Request creation of integration-k8s VPS project - https://phabricator.wikimedia.org/T295477 (10Jdforrester-WMF) [15:23:34] PROBLEM - SSH on contint1001.mgmt is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [15:26:25] 10Beta-Cluster-Infrastructure, 10Abstract Wikipedia team, 10Cloud-VPS (Project-requests): Request creation of integration-k8s VPS project - https://phabricator.wikimedia.org/T295477 (10bd808) In my understanding "beta cluster" refers to [[https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep|depl... [15:29:15] 10Beta-Cluster-Infrastructure, 10Abstract Wikipedia team, 10Cloud-VPS (Project-requests): Request creation of integration-k8s VPS project - https://phabricator.wikimedia.org/T295477 (10Jdforrester-WMF) >>! In T295477#7496065, @bd808 wrote: > In my understanding "beta cluster" refers to [[https://wikitech.wik... [15:29:55] 10Beta-Cluster-Infrastructure, 10Abstract Wikipedia team, 10Cloud-VPS (Project-requests): Request creation of deployment-prep-k8s VPS project - https://phabricator.wikimedia.org/T295477 (10Jdforrester-WMF) [15:31:35] 10Beta-Cluster-Infrastructure, 10Abstract Wikipedia team, 10Cloud-VPS (Project-requests): Request creation of deployment-prep-k8s VPS project - https://phabricator.wikimedia.org/T295477 (10bd808) >>! In T295477#7496090, @Jdforrester-WMF wrote: >>>! In T295477#7496065, @bd808 wrote: >> I'm not sure I understa... [15:35:15] 10Release-Engineering-Team, 10GitLab, 10Security-Team, 10serviceops: Setup GitLab Runner in trusted environment - https://phabricator.wikimedia.org/T295481 (10Jelto) [15:35:37] !log delete shut down deployment-logstash* and deployment-mwmaint01 instances [15:35:39] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:59:38] majavah: Looks like we're just going to create a beta k8s cluster inside the deployment-prep project. Fun! [15:59:47] (03CR) 10Hashar: "+ Timo since you wrote the original code ;)" [integration/config] - 10https://gerrit.wikimedia.org/r/737677 (owner: 10Hashar) [16:02:09] (03CR) 10Krinkle: [C: 03+1] zuul: polish set_doc_variables() [integration/config] - 10https://gerrit.wikimedia.org/r/737677 (owner: 10Hashar) [16:02:26] speed light reviewing ;) [16:02:30] thanks Krinkle ! [16:03:03] (03CR) 10Hashar: [C: 03+2] zuul: polish set_doc_variables() [integration/config] - 10https://gerrit.wikimedia.org/r/737677 (owner: 10Hashar) [16:04:38] James_F: ooh! how are you planning on implementing it? [16:04:59] also, who exactly is "we" and how much are they committed into maintaining it in the future? [16:05:03] (03Merged) 10jenkins-bot: zuul: polish set_doc_variables() [integration/config] - 10https://gerrit.wikimedia.org/r/737677 (owner: 10Hashar) [16:07:46] !log reloading Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/737677 [16:07:48] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:09:42] majavah: my stance in the paste was that releng should not have to maintain a k8s cluster [16:10:01] then I was not directly involved in the discussion to host containerized service on deployment-prep [16:10:24] I guess that settled on just using a docker daemon and pull the new image , which is probably good enough [16:10:54] majavah: "We" is me and Geno from the AW team. We need it for beta Wikifunctions, but this isn't our 'day job' sadly. :-( [16:11:26] majavah: As to 'how', we don't know yet. [16:13:08] * Reedy hands James_F some bubble gum and duct tape [16:21:49] <_joe_> !log cherry-picking puppet patches for mediawiki on php 7.4 to deployment-prep [16:21:50] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:28:54] (03PS1) 10Ahmon Dancy: deploy-promote get_train_task() Handle null task_id [tools/release] - 10https://gerrit.wikimedia.org/r/737966 [16:29:48] <_joe_> James_F: who committed to that? [16:29:49] <_joe_> :P [16:30:16] <_joe_> James_F: btw, I would suggest you (AW) reach out to serviceops about such stuff [16:30:28] <_joe_> including discussing a bit how wikifunctions will be implemented [16:31:23] <_joe_> given I guess you'd like to run it in production as well [16:32:05] <_joe_> I'd add, it's better if we try to understand early what your needs are, as our team is currently a bit thin on resources [16:32:29] <_joe_> so the sooner we understand what would be needed infra-wise for AW together, the least surprises will appear [16:32:46] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url: Validate that the specified changes are suitable - https://phabricator.wikimedia.org/T294453 (10thcipriani) [16:33:19] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url command - https://phabricator.wikimedia.org/T287042 (10thcipriani) [16:33:53] <_joe_> so yeah, before you install a k8s cluster in deployment-prep, let's talk :) [16:34:21] _joe_: are we migrating to php 7.4? [16:34:34] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10GitLab (CI & Job Runners), 10User-brennen: runner-1002 is out of space - https://phabricator.wikimedia.org/T291221 (10thcipriani) [16:34:51] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥): docker-gc: A tool for partially pruning docker resources - https://phabricator.wikimedia.org/T294034 (10thcipriani) [16:34:57] <_joe_> hashar: yes, for now I'm just doing the puppet prep work [16:35:44] I am guessing CI uses the package from sury [16:36:32] ah yeah :-\ [16:36:39] <_joe_> hashar: yeah that needs to change [16:36:49] <_joe_> legoktm did package 7.4 [16:37:20] James_F already switched CI to use our packages [16:37:47] it is definitely still using sury [16:38:06] https://gerrit.wikimedia.org/r/c/integration/config/+/732112/ ? [16:38:52] quibble-buster-php74 has `FROM {{ "sury-php" | image_tag }} AS sury-php` [16:39:36] ah, guess he missed that one [16:40:15] 10Continuous-Integration-Config: Also run PHP 7.4 jobs on wmf branch patches - https://phabricator.wikimedia.org/T293924 (10Legoktm) [16:40:24] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Patch-For-Review: Re-build all CI images of PHP 7.4 from sury's package to Wikimedia's one, to assure us that it will work - https://phabricator.wikimedia.org/T293851 (10Legoktm) 05Resolved→03Open @hashar pointed out that quibb... [16:42:28] hmm, with 7.4 packaged we should migrate the dev-images stuff to that. [16:43:09] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Patch-For-Review: Re-build all CI images of PHP 7.4 from sury's package to Wikimedia's one, to assure us that it will work - https://phabricator.wikimedia.org/T293851 (10hashar) We would need the Quibble images to be migrated as we... [16:43:40] it is nearly impossible to keep tracki of everything nowadays [16:44:04] we really could use a php74-migration project in Phab and one solely in charge of tracking all the steps :D [16:44:15] but for quibble that is probably going to be not too hard [16:44:24] though we need xdebug / pcov, but I commented on the task [16:44:37] I already packaged xdebug [16:45:10] if pcov is needed, please file a separate task [16:45:22] 10MediaWiki-Releasing, 10MW-1.37-notes, 10MW-1.37-release: Write release announcement for 1.37.0-rc.2 - https://phabricator.wikimedia.org/T295178 (10Reedy) 05In progress→03Resolved [16:45:28] 10MediaWiki-Releasing, 10MW-1.37-notes, 10MW-1.37-release, 10Patch-For-Review: Release 1.37.0-rc.2 - https://phabricator.wikimedia.org/T295176 (10Reedy) [16:46:43] 10MediaWiki-Releasing, 10MW-1.37-notes, 10MW-1.37-release, 10Patch-For-Review: Release 1.37.0-rc.2 - https://phabricator.wikimedia.org/T295176 (10Reedy) 05In progress→03Resolved [16:46:49] 10MediaWiki-Releasing, 10MW-1.37-notes, 10MW-1.37-release: Release MW 1.37.0 - https://phabricator.wikimedia.org/T289585 (10Reedy) [16:54:01] _joe_: The Board. :-) Happy to talk, and I'm sure there's lots to learn from how prod works. :-) [16:54:55] <_joe_> James_F: I doubt the board committed to install a k8s cluster in deployment prep [16:55:02] <_joe_> I wasn't questioning AW ;) [16:56:41] _joe_: Yeah, it's a real mess to stand up a Beta service. [16:57:17] _joe_: Would a meeting next week work? [16:57:24] <_joe_> James_F: sure [16:58:00] <_joe_> please add alex and janis too from serviceops [16:58:34] Will do. [16:58:49] <_joe_> James_F: do you have any doc outlining your current plans? [16:58:56] <_joe_> re: implementation [16:59:29] (03PS3) 10Daimona Eaytoy: dockerfiles: Install pcov in base PHP images [integration/config] - 10https://gerrit.wikimedia.org/r/694621 (https://phabricator.wikimedia.org/T280170) [16:59:39] Nothing specific about k8s, no. [17:00:59] <_joe_> do you have a prototype service? [17:02:12] I assume pcov is needed, unless we want to go back to xdebug for test coverage. I'm writing a task. [17:03:10] _joe_: Right now we have Lucas's http://notwikilambda.toolforge.org/ but we want to replace that with a 'real' system using k8s (not directly running node services). [17:03:38] Actually, I guess I could just reopen https://phabricator.wikimedia.org/T243847 ? [17:04:04] <_joe_> James_F: ok, I want to understand how that works under the hood [17:04:11] <_joe_> e.g. how are functions executed? [17:04:31] _joe_: In meeting now; will reply later with my full attention. [17:05:16] <_joe_> sure no problem, we can talk tomorrow too, I'm going afk anyways [17:06:14] +1 [17:07:05] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Patch-For-Review, 10Test-Coverage: Switch mediawiki code coverage from xdebug to pcov - https://phabricator.wikimedia.org/T234020 (10Daimona) [17:07:07] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10SRE, and 2 others: Add pcov PHP extension to wikimedia apt so it can be used in Wikimedia CI - https://phabricator.wikimedia.org/T243847 (10Daimona) 05Invalid→03Open This is actually... [17:08:17] !log Cherry-picking https://gerrit.wikimedia.org/r/c/operations/puppet/+/737970 in deployment-prep. Should only affect deployment-webperf11. [17:08:18] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:13:03] 10Release-Engineering-Team, 10Scap: Use the restricted/mediawiki-multiversion container image as source of /srv/mediawiki for scap sync - https://phabricator.wikimedia.org/T295493 (10dancy) [17:15:04] 10Release-Engineering-Team, 10Scap: Use the restricted/mediawiki-multiversion container image as source of /srv/mediawiki for scap sync - https://phabricator.wikimedia.org/T295493 (10dancy) [17:18:12] 10Release-Engineering-Team (Priority Backlog 📥), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url: copy files for legacy deployment - https://phabricator.wikimedia.org/T294457 (10dancy) [17:18:14] 10Release-Engineering-Team, 10Scap: Use the restricted/mediawiki-multiversion container image as source of /srv/mediawiki for scap sync - https://phabricator.wikimedia.org/T295493 (10dancy) [17:19:26] 10Release-Engineering-Team (Priority Backlog 📥), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url: copy files for legacy deployment - https://phabricator.wikimedia.org/T294457 (10dancy) [17:20:32] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Patch-For-Review: Re-build all CI images of PHP 7.4 from sury's package to Wikimedia's one, to assure us that it will work - https://phabricator.wikimedia.org/T293851 (10hashar) [17:20:37] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10SRE, and 2 others: Add pcov PHP extension to wikimedia apt so it can be used in Wikimedia CI - https://phabricator.wikimedia.org/T243847 (10hashar) [17:20:42] Daimona: good finding about pcov task ;) [17:21:25] Yeah, I don't know why I closed it. I mean, yes, we didn't need that for the 73 job. But that doesn't mean we won't need it for the others... [17:21:52] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Patch-For-Review: Re-build all CI images of PHP 7.4 from sury's package to Wikimedia's one, to assure us that it will work - https://phabricator.wikimedia.org/T293851 (10hashar) We would need pcov which is used to generate coverage... [17:21:58] cause pcov got made available for 7.2 I guess [17:22:22] which was sufficient for the coverage need [17:22:27] anyway task is around ;) [17:22:27] 10Release-Engineering-Team, 10Scap: Scap backport change_url: Implement “wait for my commits to be merged” - https://phabricator.wikimedia.org/T295494 (10dancy) [17:22:49] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url command - https://phabricator.wikimedia.org/T287042 (10dancy) [17:22:51] 10Release-Engineering-Team, 10Scap: Scap backport change_url: Implement “wait for my commits to be merged” - https://phabricator.wikimedia.org/T295494 (10dancy) [17:23:05] legoktm: Daimona got us the task to build pcov for our php 7.4 : https://phabricator.wikimedia.org/T243847 ;) [17:24:11] 10Release-Engineering-Team, 10Scap: Scap backport change_url: Implement “what are the commit hashes from the two repos that correspond to my merged commit(s)”. - https://phabricator.wikimedia.org/T295495 (10dancy) [17:24:11] We couldn't install on 7.2 because that used wm apt, not sury. So the workaround was to switch all coverage jobs to 7.3 :P [17:24:28] 10Release-Engineering-Team, 10Scap: Scap backport change_url: Implement “what are the commit hashes from the two repos that correspond to my merged commit(s)”. - https://phabricator.wikimedia.org/T295495 (10dancy) [17:24:30] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url command - https://phabricator.wikimedia.org/T287042 (10dancy) [17:28:40] 10Release-Engineering-Team, 10Scap: Scap backport change_url: Implement “wait for my commits to be merged” - https://phabricator.wikimedia.org/T295494 (10dancy) [17:30:24] 10Release-Engineering-Team, 10Scap: Scap backport change_url: Implement “what are the commit hashes from the two repos that correspond to my merged commit(s)”. - https://phabricator.wikimedia.org/T295495 (10dancy) [17:37:59] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap: Scap backport change_url: Implement “wait for my commits to be merged” - https://phabricator.wikimedia.org/T295494 (10dancy) [17:38:23] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap: Scap backport change_url: Implement “what are the commit hashes from the two repos that correspond to my merged commit(s)”. - https://phabricator.wikimedia.org/T295495 (10dancy) [17:39:03] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap, 10Patch-For-Review: Improve efficiency of scap l10n operations - https://phabricator.wikimedia.org/T295304 (10dancy) [17:41:37] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10GitLab (CI & Job Runners), 10User-brennen: runner-1002 is out of space - https://phabricator.wikimedia.org/T291221 (10dancy) a:05dancy→03None [17:45:15] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap: Scap backport change_url: Implement “what are the commit hashes from the two repos that correspond to my merged commit(s)”. - https://phabricator.wikimedia.org/T295495 (10dancy) [17:46:29] _joe_: looks like your cherry-picks broke puppet on deployment-jobrunner04 https://phabricator.wikimedia.org/P17718 [17:46:47] <_joe_> majavah: sigh let me take a look [17:47:10] <_joe_> oh my, this is my past sins biting me [17:47:25] <_joe_> majavah: I'll fix it tomorrow morning, if that's not blocking your work [17:47:56] it's not, it just popped up in our monitoring system as a puppet failure [17:47:58] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10Release, 10Train Deployments: 1.38.0-wmf.9 deployment blockers - https://phabricator.wikimedia.org/T293950 (10jeena) [17:50:59] 10Release-Engineering-Team (Priority Backlog 📥), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url: copy files for legacy deployment - https://phabricator.wikimedia.org/T294457 (10dancy) [17:58:54] 10Continuous-Integration-Config, 10Release-Engineering-Team (Next), 10MediaWiki-Core-Tests, 10Code-Health, and 6 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10Ladsgroup) That sounds good to me as long as have policies to make sure selenium tes... [17:59:16] 10Release-Engineering-Team, 10GitLab, 10Security-Team, 10serviceops: Setup GitLab Runner in trusted environment - https://phabricator.wikimedia.org/T295481 (10Dzahn) [18:03:31] 10Release-Engineering-Team, 10GitLab, 10Security-Team, 10serviceops: Setup GitLab Runner in trusted environment - https://phabricator.wikimedia.org/T295481 (10Dzahn) > I would like to reuse the existing puppet code for the Shared Runners in WMCS. Yes, this is great, all for this. > we could start with V... [18:11:32] 10Release-Engineering-Team, 10GitLab, 10Security-Team, 10serviceops: Setup GitLab Runner in trusted environment - https://phabricator.wikimedia.org/T295481 (10Joe) The only things I could see as potentially different between these runners and those running in wmcs are: - Need to use the http proxy to reac... [18:22:19] 10Release-Engineering-Team (Radar), 10GitLab, 10Security-Team, 10serviceops: Setup GitLab Runner in trusted environment - https://phabricator.wikimedia.org/T295481 (10thcipriani) [18:23:41] 10Release-Engineering-Team (Priority Backlog 📥), 10GitLab: GitLab should not display ads for paid versions - https://phabricator.wikimedia.org/T295453 (10thcipriani) [18:24:38] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Config, 10Release-Engineering-Team (Seen): Make icinga-wm complain if beta CI jobs haven't run in some period of time - https://phabricator.wikimedia.org/T295241 (10thcipriani) [18:25:40] 10MediaWiki-Releasing: Provide machine readable information about MediaWiki versions - https://phabricator.wikimedia.org/T294746 (10thcipriani) [18:26:52] 10Release-Engineering-Team (Priority Backlog 📥), 10GitLab (Auth & Access), 10User-brennen, 10cloud-services-team (Kanban): Create top level 'cloud' group on Gitlab - https://phabricator.wikimedia.org/T293741 (10brennen) [18:28:03] 10Release-Engineering-Team, 10serviceops: contint hardware refresh? - https://phabricator.wikimedia.org/T294276 (10thcipriani) [18:28:33] 10Release-Engineering-Team, 10serviceops: contint hardware refresh - https://phabricator.wikimedia.org/T294276 (10thcipriani) [18:29:52] 10Release-Engineering-Team (Priority Backlog 📥): Demo data³ - https://phabricator.wikimedia.org/T295497 (10mmodell) [18:29:54] 10Release-Engineering-Team (Seen), 10serviceops: contint hardware refresh - https://phabricator.wikimedia.org/T294276 (10thcipriani) Runs out of warranty this calendar year or fiscal year? I'm inclined to go ahead and replace this hardware. [18:30:22] 10Release-Engineering-Team (Priority Backlog 📥), 10Data³: Demo data³ - https://phabricator.wikimedia.org/T295497 (10mmodell) [18:31:18] 10Release-Engineering-Team (Radar), 10serviceops: Puppet failure on deploy-1002.devtools.eqiad1.wikimedia.cloud due to missing profile::kubernetes::deployment_server::user_defaults - https://phabricator.wikimedia.org/T294174 (10thcipriani) [18:31:53] 10Release-Engineering-Team (Seen), 10serviceops: contint hardware refresh - https://phabricator.wikimedia.org/T294276 (10Dzahn) The purchase date was 2016-03-24 (and one day earlier for contint1001) and I expect it to last 5 years. [18:34:40] 10Release-Engineering-Team, 10Security Team AppSec, 10Security-Team, 10GitLab (CI & Job Runners), and 3 others: Migrate existing proof-of-concept node ci templates to slim node wm node docker images - https://phabricator.wikimedia.org/T294306 (10thcipriani) 05Invalid→03Open a:05sbassett→03thcipriani... [18:34:45] 10Release-Engineering-Team (Doing), 10Security Team AppSec, 10Security-Team, 10GitLab (CI & Job Runners), and 3 others: Migrate existing proof-of-concept node ci templates to slim node wm node docker images - https://phabricator.wikimedia.org/T294306 (10thcipriani) [18:34:49] 10Release-Engineering-Team (Priority Backlog 📥), 10Data³: Demo data³ - https://phabricator.wikimedia.org/T295497 (10mmodell) [18:36:37] 10Release-Engineering-Team (Doing), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url command - https://phabricator.wikimedia.org/T287042 (10thcipriani) [18:36:47] with all that spam you all can follow the releng team triaging phabricator tasks! [18:38:45] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10GitLab: GitLab should not display ads for paid versions - https://phabricator.wikimedia.org/T295453 (10brennen) [18:39:13] 10Release-Engineering-Team (Priority Backlog 📥), 10GitLab (Auth & Access), 10User-brennen, 10cloud-services-team (Kanban): Create top level 'cloud' group on Gitlab - https://phabricator.wikimedia.org/T293741 (10brennen) 05Stalled→03Open [18:39:24] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10GitLab (Auth & Access), 10User-brennen, 10cloud-services-team (Kanban): Create top level 'cloud' group on Gitlab - https://phabricator.wikimedia.org/T293741 (10brennen) [18:40:12] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10Scap: Scap backport change_url: Implement “what are the commit hashes from the two repos that correspond to my merged commit(s)”. - https://phabricator.wikimedia.org/T295495 (10thcipriani) [18:40:46] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10Scap: Scap backport change_url: Implement “wait for my commits to be merged” - https://phabricator.wikimedia.org/T295494 (10thcipriani) [18:41:08] 10Release-Engineering-Team (Priority Backlog 📥), 10Data³: Demo data³ - https://phabricator.wikimedia.org/T295497 (10mmodell) [18:41:28] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10GitLab (Administration, Settings & Policy), 10User-brennen: GitLab should not display ads for paid versions - https://phabricator.wikimedia.org/T295453 (10brennen) a:03brennen We theoretically have the option that disables this kind of thing turned on, b... [18:41:46] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10Scap, 10Patch-For-Review: Improve efficiency of scap l10n operations - https://phabricator.wikimedia.org/T295304 (10thcipriani) [18:41:58] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10GitLab (Auth & Access), 10User-brennen, 10cloud-services-team (Kanban): Create top level 'cloud' group on Gitlab - https://phabricator.wikimedia.org/T293741 (10brennen) a:03brennen [18:42:02] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10Data³: Demo data³ - https://phabricator.wikimedia.org/T295497 (10mmodell) [18:42:05] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10Data³: Demo data³ - https://phabricator.wikimedia.org/T295497 (10mmodell) [18:42:28] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10Data³: Demo data³ - https://phabricator.wikimedia.org/T295497 (10mmodell) [18:43:10] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10Security-Team, 10GitLab (CI & Job Runners), 10SecTeam-Processed, 10User-brennen: Limit GitLab shared runners to trusted contributors - https://phabricator.wikimedia.org/T292094 (10thcipriani) [18:43:45] 10Phabricator, 10Release-Engineering-Team (Doing), 10Data³: Phab feature request: Cycle time for a task entering a column to resolution, with support for wildcards - https://phabricator.wikimedia.org/T148805 (10mmodell) [18:45:06] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10Security-Team, 10GitLab (CI & Job Runners), 10Patch-For-Review, and 2 others: Limit GitLab shared runners to images from Wikimedia Docker registry - https://phabricator.wikimedia.org/T291978 (10thcipriani) [18:45:20] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10SRE, and 2 others: Add pcov PHP extension to wikimedia apt so it can be used in Wikimedia CI - https://phabricator.wikimedia.org/T243847 (10Legoktm) a:03Legoktm Sure. [18:45:59] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url: copy files for legacy deployment - https://phabricator.wikimedia.org/T294457 (10thcipriani) [18:46:13] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10GitLab (Auth & Access), 10Upstream, 10User-brennen: Gitlab 2fa password validation seems bugged - https://phabricator.wikimedia.org/T292431 (10brennen) 05Open→03In progress a:03brennen [18:47:48] 10Release-Engineering-Team (Priority Backlog 📥), 10GitLab (Auth & Access): Create a top level wmde group on Gitlab - https://phabricator.wikimedia.org/T291388 (10thcipriani) a:03thcipriani [18:49:06] 10Phabricator, 10Release-Engineering-Team (Seen), 10Wikimedia-Phabricator-Extensions: Phabricator Project Reports have inaccurate counts for the age histogram - https://phabricator.wikimedia.org/T294998 (10thcipriani) [18:50:17] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10Data³: Demo data³ - https://phabricator.wikimedia.org/T295497 (10mmodell) 05Open→03In progress [18:50:35] 10Release-Engineering-Team (Doing), 10dev-images, 10mwcli, 10User-brennen: Add php-luasandbox to dev-images used by mwcli - https://phabricator.wikimedia.org/T286678 (10thcipriani) [18:51:26] 10Continuous-Integration-Config, 10Release-Engineering-Team (Next), 10MediaWiki-Core-Tests, 10Code-Health, and 6 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10Legoktm) I understand the general request of making CI faster but I think the tradeo... [18:52:27] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url: legacy deployment - https://phabricator.wikimedia.org/T294466 (10thcipriani) [18:57:16] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10Security-Team, 10GitLab (CI & Job Runners), 10Patch-For-Review, and 2 others: Limit GitLab shared runners to images from Wikimedia Docker registry - https://phabricator.wikimedia.org/T291978 (10Legoktm) >>! In T291978#7490374, @thcipriani wrote: > These... [19:23:03] dduvall: o/ [19:23:06] https://gerrit.wikimedia.org/r/c/operations/puppet/+/737764 [19:23:08] look okay to you? [19:23:22] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Patch-For-Review: Re-build all CI images of PHP 7.4 from sury's package to Wikimedia's one, to assure us that it will work - https://phabricator.wikimedia.org/T293851 (10Legoktm) [19:23:24] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10SRE, and 2 others: Add pcov PHP extension to wikimedia apt so it can be used in Wikimedia CI - https://phabricator.wikimedia.org/T243847 (10Legoktm) 05Open→03Resolved [19:23:32] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Patch-For-Review, 10Test-Coverage: Switch mediawiki code coverage from xdebug to pcov - https://phabricator.wikimedia.org/T234020 (10Legoktm) [19:23:54] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Patch-For-Review: Re-build all CI images of PHP 7.4 from sury's package to Wikimedia's one, to assure us that it will work - https://phabricator.wikimedia.org/T293851 (10Legoktm) Both php7.4-xdebug and php7.4-pcov packages should b... [19:25:00] 10Phabricator (Upstream), 10Developer Productivity, 10Upstream: Disable parent task metadata by default for new sub tasks - https://phabricator.wikimedia.org/T239378 (10Quiddity) +1 to the proposal that seems to have unanimous support so far, for not including the Subscribers from the parent task. A few non-... [19:25:32] or thcipriani does that look okay to merge? [19:25:42] * thcipriani takes a look [19:26:54] RECOVERY - SSH on contint1001.mgmt is OK: SSH OK - OpenSSH_6.6 (protocol 2.0) https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [19:29:17] ottomata: after squinting at irb, lgtm :) [19:29:24] thank you thcipriani i'll follow up with patches if somethign doesn't work :) [19:29:29] <3 [19:30:07] and <3 for the patch [20:19:40] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url: copy files for legacy deployment - https://phabricator.wikimedia.org/T294457 (10dancy) 05Open→03In progress [20:19:42] 10Release-Engineering-Team (Doing), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url command - https://phabricator.wikimedia.org/T287042 (10dancy) [20:26:26] 10Release-Engineering-Team (Radar), 10serviceops, 10Patch-For-Review: Puppet failure on deploy-1002.devtools.eqiad1.wikimedia.cloud due to missing profile::kubernetes::deployment_server::user_defaults - https://phabricator.wikimedia.org/T294174 (10Dzahn) >>! In T294174#7454163, @hashar wrote: > #beta-cluster... [20:28:46] 10Release-Engineering-Team (Radar), 10serviceops, 10Patch-For-Review: Puppet failure on deploy-1002.devtools.eqiad1.wikimedia.cloud due to missing profile::kubernetes::deployment_server::user_defaults - https://phabricator.wikimedia.org/T294174 (10Dzahn) puppet run finished on deploy1002 again for the first... [20:51:32] James_F: Krinkle: the Quibble jobs got switched from Node 10 to 14. So I guess we can finally remove the releng/*node10* images ( https://gerrit.wikimedia.org/r/c/integration/config/+/720156/ and parents ) [20:51:48] if I understand it migrating Quibble images was the last blocker [20:52:03] and given they got moved to node14 , I guess we can get fresh to drop node 10 support [20:53:55] (03CR) 10Hashar: [C: 03+2] "The Quibble images have been ported by James ( T294931 ) apparently with success. So I think we can now drop all the node 10 based images" [integration/config] - 10https://gerrit.wikimedia.org/r/720153 (owner: 10Jforrester) [20:54:40] 10Release-Engineering-Team (Radar), 10serviceops, 10Patch-For-Review: Puppet failure on deploy-1002.devtools.eqiad1.wikimedia.cloud due to missing profile::kubernetes::deployment_server::user_defaults - https://phabricator.wikimedia.org/T294174 (10Dzahn) 05Open→03Resolved a:03Dzahn Horizon Hiera empty,... [20:54:42] brennen: I got a change to turn on gitlab csp if you ever get some bandwith with sre this week ( https://gerrit.wikimedia.org/r/c/operations/puppet/+/737968 ) [20:54:50] or i will try to remember about it next week ;) [20:55:32] 10Release-Engineering-Team (Done by Wed 24 Nov 🔥), 10Scap, 10Patch-For-Review: Improve efficiency of scap l10n operations - https://phabricator.wikimedia.org/T295304 (10dancy) 05Open→03In progress [21:02:20] hashar: the problem is fresh-node is testing its code via node10 because node12 and node14 are both broken with Qemu [21:02:27] I am waiting for a new Qemu agent. [21:02:37] (03CR) 10Jeena Huneidi: [C: 03+2] "LGTM" [tools/train-dev] - 10https://gerrit.wikimedia.org/r/737780 (owner: 10Ahmon Dancy) [21:03:06] but as long as the image is not deleted from the docker registry, the source can be removed, although keeping it in the repo would be harmless I suppose until we're done. [21:03:07] (03CR) 10jerkins-bot: [V: 04-1] Install docker in deploy container [tools/train-dev] - 10https://gerrit.wikimedia.org/r/737780 (owner: 10Ahmon Dancy) [21:03:19] it's not like it needs updating [21:04:13] Krinkle: OH my bad, I thought the Qemu issue had been solved by moving the underlying instance to Bullseye [21:07:54] oh https://phabricator.wikimedia.org/T284774 [21:08:49] integration-agent-qemu-1003 is the one [21:09:23] it's still an empty instance [21:09:25] not yet provisioned [21:10:00] isn't it all about applying the proper puppet role / profiles ? [21:10:59] hashar: it also needs an actual VM to be provisioned within the VM. [21:11:05] a Qemu base image [21:11:12] with newer Debian and docker-ce [21:11:28] oh I used a system named diskimage to provide those back in the Nodepool days [21:11:29] this was done by lars last time, and isn't puppetised [21:11:37] eek [21:11:44] well guess we can copy the existing one over [21:11:54] (03PS2) 10Ahmon Dancy: Install docker in deploy container [tools/train-dev] - 10https://gerrit.wikimedia.org/r/737780 [21:11:56] we can, but that wouldn't solve the problem, it needs newer debian [21:12:01] and port whatever script was used to create the image to whatever is the SRE standard tool to craft an image [21:12:08] although I didn't know the .iso is compatible between qemu 2 and 4, maybe it is [21:12:16] (03CR) 10jerkins-bot: [V: 04-1] Install docker in deploy container [tools/train-dev] - 10https://gerrit.wikimedia.org/r/737780 (owner: 10Ahmon Dancy) [21:12:24] so yeah that is a long tail [21:12:27] it wasn't created by script, it was created by lars and uploaded over ssh [21:16:35] (03PS3) 10Ahmon Dancy: Install docker in deploy container [tools/train-dev] - 10https://gerrit.wikimedia.org/r/737780 [21:16:35] yeah, I think it's fine to do by hand, it just needs to be documented and is presumably only 2 or 3 steps to do once a year or so when we re-create the image. [21:17:04] we documented the last steps after the image is up: https://www.mediawiki.org/wiki/Continuous_integration/Qemu [21:17:28] but not the steps about where to get the debian image, and install it, and to configure/size the qemu format etc. [21:18:41] maybe the keypair can be puppetized [21:18:49] that is on the host rather than guest [21:19:43] (03CR) 10Ahmon Dancy: [C: 04-1] "splitting" [tools/train-dev] - 10https://gerrit.wikimedia.org/r/737780 (owner: 10Ahmon Dancy) [21:19:58] 10Continuous-Integration-Infrastructure, 10Performance-Team: Provide one or more Qemu agents in CI that use a newer version than 2.x - https://phabricator.wikimedia.org/T284774 (10hashar) integration-agent-qemu-1003 is ready but: * we do not have the base Qemu image available. It was created by @LarsWirzenius... [21:21:20] knowing Lars he probably used his image building system https://vmdb2.liw.fi/ ;) [21:22:19] (03PS1) 10Ahmon Dancy: Adjustments for new shellcheck complaints [tools/train-dev] - 10https://gerrit.wikimedia.org/r/738011 [21:24:02] (03CR) 10Ahmon Dancy: [C: 03+2] Adjustments for new shellcheck complaints [tools/train-dev] - 10https://gerrit.wikimedia.org/r/738011 (owner: 10Ahmon Dancy) [21:24:27] (03Merged) 10jenkins-bot: Adjustments for new shellcheck complaints [tools/train-dev] - 10https://gerrit.wikimedia.org/r/738011 (owner: 10Ahmon Dancy) [21:24:43] (03PS4) 10Ahmon Dancy: Install docker in deploy container [tools/train-dev] - 10https://gerrit.wikimedia.org/r/737780 [21:25:20] (03CR) 10Ahmon Dancy: [C: 03+2] Install docker in deploy container [tools/train-dev] - 10https://gerrit.wikimedia.org/r/737780 (owner: 10Ahmon Dancy) [21:25:44] (03Merged) 10jenkins-bot: Install docker in deploy container [tools/train-dev] - 10https://gerrit.wikimedia.org/r/737780 (owner: 10Ahmon Dancy) [21:40:16] hashar: ack, thanks [21:46:06] (03PS1) 10Ahmon Dancy: Update FailedCommand documentation [tools/scap] - 10https://gerrit.wikimedia.org/r/738014 [21:46:41] (03CR) 10Ahmon Dancy: [C: 03+2] Update FailedCommand documentation [tools/scap] - 10https://gerrit.wikimedia.org/r/738014 (owner: 10Ahmon Dancy) [21:46:57] (03PS1) 10Ahmon Dancy: Make _runcmd and gitcmd return strings, not bytes [tools/scap] - 10https://gerrit.wikimedia.org/r/738015 [21:47:21] (03Merged) 10jenkins-bot: Update FailedCommand documentation [tools/scap] - 10https://gerrit.wikimedia.org/r/738014 (owner: 10Ahmon Dancy) [21:56:45] https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar lists tomorrow as a no deploy day, but it isn't marked as such on https://wikitech.wikimedia.org/wiki/Deployments#Thursday,_November_11 [22:16:52] 10Release-Engineering-Team (Radar), 10Front-end-Standards-Group, 10Product-Infrastructure-Team-Backlog, 10Platform Engineering (Icebox): Should npm packages maintained by Wikimedia be scoped or unscoped? - https://phabricator.wikimedia.org/T239742 (10Krinkle) 05Open→03Resolved a:03Krinkle https://www... [22:57:19] (03PS2) 10Ahmon Dancy: Make _runcmd and gitcmd return strings, not bytes [tools/scap] - 10https://gerrit.wikimedia.org/r/738015 [22:57:21] (03PS1) 10Ahmon Dancy: runcmd/FailedCommand error reporting improvements [tools/scap] - 10https://gerrit.wikimedia.org/r/738023 [22:57:23] (03PS1) 10Ahmon Dancy: lint.py: Supply useful output on lint fail [tools/scap] - 10https://gerrit.wikimedia.org/r/738024 (https://phabricator.wikimedia.org/T272760) [22:58:19] (03CR) 10jerkins-bot: [V: 04-1] lint.py: Supply useful output on lint fail [tools/scap] - 10https://gerrit.wikimedia.org/r/738024 (https://phabricator.wikimedia.org/T272760) (owner: 10Ahmon Dancy) [23:00:36] (03CR) 10Ahmon Dancy: [C: 03+2] Make _runcmd and gitcmd return strings, not bytes [tools/scap] - 10https://gerrit.wikimedia.org/r/738015 (owner: 10Ahmon Dancy) [23:01:44] (03Merged) 10jenkins-bot: Make _runcmd and gitcmd return strings, not bytes [tools/scap] - 10https://gerrit.wikimedia.org/r/738015 (owner: 10Ahmon Dancy) [23:02:19] kostajh: if we run fresnel on skins/extension (opt-in) we could set a threshold there as well. We currently have it run on core only, and only alert in relative increases [23:02:50] we could set let .fresnel.yml e.g. specify that on scenario "Page view" (or something else) set a budget for x KB for CSS, for example. [23:03:14] which core could set to a base line, but Vector can increase and/or set additional scenarios to measure. [23:03:25] it's less flebixle, but could be a good thing / more meaningful. [23:03:39] We can also do both of course :) [23:06:43] (03PS2) 10Ahmon Dancy: lint.py: Supply useful output on lint fail [tools/scap] - 10https://gerrit.wikimedia.org/r/738024 (https://phabricator.wikimedia.org/T272760) [23:06:58] (03CR) 10Ahmon Dancy: [C: 03+2] runcmd/FailedCommand error reporting improvements [tools/scap] - 10https://gerrit.wikimedia.org/r/738023 (owner: 10Ahmon Dancy) [23:07:38] (03Merged) 10jenkins-bot: runcmd/FailedCommand error reporting improvements [tools/scap] - 10https://gerrit.wikimedia.org/r/738023 (owner: 10Ahmon Dancy) [23:10:28] 10Release-Engineering-Team (Seen), 10Scap, 10MediaWiki-Release-Tools, 10Patch-For-Review: Output of scap lint.py isn't helpful if it fails the php -l check - https://phabricator.wikimedia.org/T272760 (10dancy) I ran into this today while experimenting, so I fixed it. [23:12:46] 10Release-Engineering-Team (Seen), 10Scap, 10MediaWiki-Release-Tools, 10Patch-For-Review: Output of scap lint.py isn't helpful if it fails the php -l check - https://phabricator.wikimedia.org/T272760 (10dancy) 05Open→03In progress [23:52:51] 10Release-Engineering-Team (Doing), 10Scap, 10MediaWiki-Release-Tools, 10Patch-For-Review: Output of scap lint.py isn't helpful if it fails the php -l check - https://phabricator.wikimedia.org/T272760 (10dancy) [23:53:25] I'm scratching my head over a strange Phan warning, in case anyone has ideas :) https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaMaintenance/+/737799 [23:53:27] > blameStartupRegistry.php:161 PhanPossiblyUndeclaredVariable Variable $component is possibly undeclared [23:53:47] line 159 (no warning): ( $startupBreakdown[$component] ?? [] ); [23:53:55] line 160 (no warning): ( $startupBreakdown[$component]['modules'] ?? 0 ) + 1; [23:54:07] line 161 (oh?): ( $startupBreakdown[$component]['startupBytes'] ?? 0 ) + $startupBytes; [23:54:27] I added line 159 thinking it was about the key $component, and not $component itself. [23:54:49] but now I'm not sure. Is it somehow thinking $component can magically become undefined between those two lines?