[02:27:30] RECOVERY - SSH on contint2001.mgmt is OK: SSH OK - OpenSSH_6.6 (protocol 2.0) https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [06:40:53] (03PS3) 10Thiemo Kreuz (WMDE): UnsortedUseStatementsSniff: add early returns [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698045 (owner: 10DannyS712) [06:41:59] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+1] "Feel free to +2 this yourself now, if you like." (031 comment) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698045 (owner: 10DannyS712) [06:51:30] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+1] "I think this makes sense." [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698289 (https://phabricator.wikimedia.org/T273575) (owner: 10DannyS712) [07:00:49] (03CR) 10DannyS712: [C: 03+2] "> Patch Set 3: Code-Review+1" (031 comment) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698045 (owner: 10DannyS712) [07:00:53] (03PS4) 10DannyS712: UnsortedUseStatementsSniff: add support for scoped namespaces [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698286 (https://phabricator.wikimedia.org/T278348) [07:02:37] (03Merged) 10jenkins-bot: UnsortedUseStatementsSniff: add early returns [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698045 (owner: 10DannyS712) [08:37:44] 10Continuous-Integration-Config, 10Browser-Tests: Don't run quibble-vendor-mysql-php72-selenium-docker in gate-and-submit - https://phabricator.wikimedia.org/T284434 (10kostajh) [08:46:55] 10Continuous-Integration-Config, 10Release-Engineering-Team (Doing), 10MW-1.37-notes (1.37.0-wmf.9; 2021-06-07), 10Patch-For-Review: Upgrade all CI jobs for WMF-deployed projects from Node 10 to Node 12 - https://phabricator.wikimedia.org/T284345 (10hashar) [08:52:12] (03PS5) 10DannyS712: UnsortedUseStatementsSniff: add support for bracketed namespaces [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698286 (https://phabricator.wikimedia.org/T278348) [08:52:27] (03CR) 10DannyS712: "PS5: replace "scoped namespace" with "bracketed namespace" to match php documentation" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698286 (https://phabricator.wikimedia.org/T278348) (owner: 10DannyS712) [08:54:32] (03PS3) 10DannyS712: UnusedUseStatementSniff: add support for bracketed namespaces [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698287 (https://phabricator.wikimedia.org/T278348) [08:54:42] (03CR) 10DannyS712: "PS3: replace "scoped namespace" with "bracketed namespace" to match php documentation" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698287 (https://phabricator.wikimedia.org/T278348) (owner: 10DannyS712) [08:55:34] 10MediaWiki-Codesniffer, 10Patch-For-Review, 10User-DannyS712: UnsortedUseStatementsSniff doesn't complain about use statements in namespace {} - https://phabricator.wikimedia.org/T278348 (10DannyS712) >>! In T278348#6942036, @thiemowmde wrote: > Can we please stop people writing code like this? I'm dead ser... [09:04:29] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.9 deployment blockers - https://phabricator.wikimedia.org/T281150 (10Ladsgroup) {meme, src=whereismypopcorn} [09:27:13] (03PS1) 10Hashar: zuul: fix typo in jobs-framework-api project name [integration/config] - 10https://gerrit.wikimedia.org/r/698466 [09:27:32] (03CR) 10Hashar: [C: 03+2] zuul: fix typo in jobs-framework-api project name [integration/config] - 10https://gerrit.wikimedia.org/r/698466 (owner: 10Hashar) [09:28:39] (03Merged) 10jenkins-bot: zuul: fix typo in jobs-framework-api project name [integration/config] - 10https://gerrit.wikimedia.org/r/698466 (owner: 10Hashar) [09:33:11] (03CR) 10Arturo Borrero Gonzalez: "thanks!" [integration/config] - 10https://gerrit.wikimedia.org/r/698466 (owner: 10Hashar) [09:45:18] 10Continuous-Integration-Config, 10Release-Engineering-Team (Doing), 10MW-1.37-notes (1.37.0-wmf.9; 2021-06-07), 10Patch-For-Review: Upgrade all CI jobs for WMF-deployed projects from Node 10 to Node 12 - https://phabricator.wikimedia.org/T284345 (10WMDE-Fisch) While playing with this locally I realized th... [09:46:26] (03CR) 10DannyS712: "This change is ready for review." [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698359 (https://phabricator.wikimedia.org/T278348) (owner: 10DannyS712) [10:09:21] (03CR) 10Hashar: [C: 04-1] "Kartotherian has been added to the Pipeline/Blubber a while back. The Blubber config has a test variant that runs 'npm test'. So I guess " [integration/config] - 10https://gerrit.wikimedia.org/r/698258 (https://phabricator.wikimedia.org/T284345) (owner: 10Jforrester) [10:28:01] hello! I have a question about failed post-merge build steps in the pipeline, how re-trigger then? patch for context https://gerrit.wikimedia.org/r/c/operations/software/tegola/+/698470 [10:52:36] 10Continuous-Integration-Config, 10Patch-For-Review: Add CI for mediawiki/extensions/WikibaseStatementUpdater - https://phabricator.wikimedia.org/T270369 (10Nikerabbit) 05Open→03Resolved a:03Nikerabbit Just Phan missing? Seems good enough for me. [11:35:48] mbsantos: The easiest way would probably just be create a new patchset on the commit by editing the commit message or such [11:48:58] mbsantos: hi, you can't retrigger a postmerge job yourself but I can do it [11:49:07] mbsantos: else the job will be run next time a change get merged [11:49:52] I don't know whether there is a way to easily rebuild an image with the Pipeline [11:49:54] hashar: thanks, I thought that there was something similar to `recheck` comment to re-trigger failed post-merge jobs [11:50:44] ah there is https://phabricator.wikimedia.org/T214431 "Pipeline: provide a way to rebuild all blubber images" [11:50:49] anyway, I will re build it [11:51:25] thanks! [11:51:53] !log zuul enqueue --trigger gerrit --pipeline postmerge --project operations/software/tegola --change 698470,1 # request by mbsantos for https://gerrit.wikimedia.org/r/c/operations/software/tegola/+/698470 [11:51:55] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:52:12] mbsantos: it shows up at https://integration.wikimedia.org/zuul/#q=tegola [11:52:22] and there is no `recheck` magic keyword for postmerge pipeline [11:52:38] ack! [11:53:00] I loved the gerrit shortcut command [11:53:26] the idea is postmerge is triggered solely after someone has been able to +2 [11:53:37] so the permission to run a postmerge job depends on one ability to +2 [11:53:42] for a specific repo [11:54:02] and we can't apply any access list on the `recheck` keyword beside the hardcoded allow list of users allowed to do it [11:54:20] anyway, looks like the build has worked this time! [11:58:48] (03CR) 10Hashar: [C: 03+2] "INFO:jenkins_jobs.builder:Creating jenkins job trigger-wikilambda-function-evaluator-pipeline-test" [integration/config] - 10https://gerrit.wikimedia.org/r/691243 (https://phabricator.wikimedia.org/T282828) (owner: 10Cory Massaro) [12:00:13] (03Merged) 10jenkins-bot: Add .pipeline/config.yaml-style CI to mediawiki/services/function-evaluator. [integration/config] - 10https://gerrit.wikimedia.org/r/691243 (https://phabricator.wikimedia.org/T282828) (owner: 10Cory Massaro) [12:10:44] (03CR) 10Hashar: "I have send a dummy change against the repository: https://gerrit.wikimedia.org/r/c/mediawiki/services/function-evaluator/+/698483 and it " [integration/config] - 10https://gerrit.wikimedia.org/r/691243 (https://phabricator.wikimedia.org/T282828) (owner: 10Cory Massaro) [12:27:52] (03CR) 10Thiemo Kreuz (WMDE): Add new sniffs for a single non-bracketed namespace per file (034 comments) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698359 (https://phabricator.wikimedia.org/T278348) (owner: 10DannyS712) [12:34:20] PROBLEM - SSH on contint2001.mgmt is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [12:52:10] (03PS3) 10DannyS712: Add new sniffs for a single non-bracketed namespace per file [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698359 (https://phabricator.wikimedia.org/T278348) [12:53:09] (03CR) 10DannyS712: Add new sniffs for a single non-bracketed namespace per file (034 comments) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698359 (https://phabricator.wikimedia.org/T278348) (owner: 10DannyS712) [13:30:48] 10MediaWiki-Codesniffer: Forbid usage of array_push with a single element - https://phabricator.wikimedia.org/T242218 (10DannyS712) I propose to decline this - as @thiemowmde points out, there are reasons to use `array_push` with a single element (eg to match array_pop) and while I agree that it takes slightly m... [13:34:43] 10Continuous-Integration-Config, 10Code-Health, 10Discovery-Search (Current work), 10Patch-For-Review: Ensure that SonarQube is commenting on gerrit code reviews of the Search Platform team - https://phabricator.wikimedia.org/T264873 (10Gehel) Sonar analysis is enabled for all Search Platform projects. The... [13:35:04] RECOVERY - SSH on contint2001.mgmt is OK: SSH OK - OpenSSH_6.6 (protocol 2.0) https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [13:50:44] 10Release-Engineering-Team (Doing), 10Gerrit (Gerrit 3.3): Upgrade Gerrit to 3.3 - https://phabricator.wikimedia.org/T262241 (10hashar) [13:50:50] 10Release-Engineering-Team (Doing), 10Gerrit (Gerrit 3.3): Switch Gerrit from Java 8 to Java 11 - https://phabricator.wikimedia.org/T268225 (10hashar) 05Open→03Resolved All done. Thank you @MoritzMuehlenhoff [13:55:04] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Patch-For-Review, 10Technical-Debt: Move integration/docroot.git deployment to scap - https://phabricator.wikimedia.org/T256005 (10hashar) [13:55:06] (03CR) 10DannyS712: "This change is ready for review." [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698526 (https://phabricator.wikimedia.org/T270830) (owner: 10DannyS712) [13:55:07] 10Scap: scap deploy --init on deployment server fails on first puppet run - https://phabricator.wikimedia.org/T257317 (10hashar) [13:57:41] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), 10Patch-For-Review, 10Technical-Debt: Move integration/docroot.git deployment to scap - https://phabricator.wikimedia.org/T256005 (10hashar) @Izno... [13:58:51] 10Continuous-Integration-Infrastructure, 10SRE, 10SRE-Access-Requests: Requesting access to contint-admins for Ladsgroup - https://phabricator.wikimedia.org/T283925 (10hashar) A bit late but `+1` since Amir has a tendency of being super helpful on everything he touches and I trust him to ask questions when h... [14:00:23] 10MediaWiki-Codesniffer: MediaWiki.WhiteSpace.SpaceBeforeSingleLineComment.NewLineComment is too strict - https://phabricator.wikimedia.org/T168476 (10DannyS712) So, to put in terms that can be implemented in the sniff: don't complain about not having a newline before a single line comment, if after the comment... [14:00:50] 10Scap: scap deploy --init on deployment server fails on first puppet run - https://phabricator.wikimedia.org/T257317 (10hashar) [14:00:56] 10Release-Engineering-Team (Radar), 10SRE, 10serviceops, 10Patch-For-Review: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10hashar) [14:01:30] 10Scap: scap deploy --init on deployment server fails on first puppet run - https://phabricator.wikimedia.org/T257317 (10hashar) [14:01:34] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10serviceops, 10Patch-For-Review: Replace production deployment servers and update them to Buster - https://phabricator.wikimedia.org/T265963 (10hashar) [14:02:55] 10Scap: scap deploy --init on deployment server fails on first puppet run - https://phabricator.wikimedia.org/T257317 (10hashar) I have reverted the task dependencies changes made by @Izno. This task is not a blocker of updating MediaWiki app servers to Buster, it is merely an actionable that needs to be solved... [14:23:25] Can I get a comment from a Gerrit person on https://phabricator.wikimedia.org/T282074? Quick summary is: the wikitech databases currently have a grant for access from the gerrit host. Everyone is pretty sure that this is obsolete and/or a mistake but no one is confident enough to close it without input from releng. [14:23:43] Project mediawiki-core-doxygen-docker build #25486: 04FAILURE in 19 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/25486/ [14:31:00] (03CR) 10Jforrester: "> Patch Set 1: Code-Review-1" [integration/config] - 10https://gerrit.wikimedia.org/r/698258 (https://phabricator.wikimedia.org/T284345) (owner: 10Jforrester) [14:35:30] 10Continuous-Integration-Config, 10Browser-Tests: Don't run quibble-vendor-mysql-php72-selenium-docker in gate-and-submit - https://phabricator.wikimedia.org/T284434 (10Jdforrester-WMF) The first job is "with this patch, do this extension's selenium test (and its dependencies) still pass?" The second is "with... [14:39:20] 10Beta-Cluster-Infrastructure, 10Beta-Cluster-reproducible, 10User-Majavah, 10User-Ryasmeen, 10User-zeljkofilipin: Cannot login to beta cluster: "There seems to be a problem with your login session..." - https://phabricator.wikimedia.org/T263617 (10dom_walden) 05Resolved→03Open We are seeing this pro... [14:42:22] 10LibUp, 10MediaWiki-Releasing, 10Release-Engineering-Team (Seen), 10MediaWiki-Release-Tools: Work out why initial skin/extension branches into REL1_35/REL1_36 were on the wrong branch - https://phabricator.wikimedia.org/T283883 (10hashar) For Interwiki on the Gerrit server: ` git -C /srv/gerrit/git/mediaw... [14:42:57] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10MediaWiki Train Development Environment: Add CI to mediawiki/tools/train-dev - https://phabricator.wikimedia.org/T259586 (10hashar) [14:42:59] 10Release-Engineering-Team (Seen), 10GitLab (Initialization), 10Software-Licensing, 10User-brennen: Add LICENSE/COPYING to operations/gitlab-ansible - https://phabricator.wikimedia.org/T282496 (10hashar) [14:44:01] 10Beta-Cluster-Infrastructure, 10Wikimedia-Logstash, 10observability: logstash-beta.wmflabs.org does not receive any mediawiki events - https://phabricator.wikimedia.org/T233134 (10hashar) [14:45:19] 10Deployments, 10Prod-Kubernetes, 10serviceops: Evaluate cost/benefits of switching to using a programming language to define kubernetes resources - https://phabricator.wikimedia.org/T254739 (10hashar) [14:46:32] 10Gerrit, 10Phabricator, 10Release-Engineering-Team (Seen), 10Wikimedia-GitHub: Automate or create a check of mirrors of WMF Gerrit git repos into Phabricator Diffussion and Github - https://phabricator.wikimedia.org/T251132 (10hashar) [14:48:52] 10Release-Engineering-Team (Seen), 10Platform Engineering, 10Release Pipeline, 10SRE, and 5 others: Kask functional testing with Cassandra via the Deployment Pipeline - https://phabricator.wikimedia.org/T224041 (10hashar) [14:50:33] 10LibUp, 10MediaWiki-Releasing, 10Release-Engineering-Team (Seen), 10MediaWiki-Release-Tools: Work out why initial skin/extension branches into REL1_35/REL1_36 were on the wrong branch - https://phabricator.wikimedia.org/T283883 (10Jdforrester-WMF) >>! In T283883#7138728, @hashar wrote: > So REL1_34 has be... [15:00:34] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Platform Team Initiatives (API Integration Tests): Expose mediawiki/tools/api-testing doc on doc.wikimedia.org - https://phabricator.wikimedia.org/T236915 (10hashar) I have completely missed the previous reply b @apaskulin . Looks like we just ne... [15:00:40] (03CR) 10Thiemo Kreuz (WMDE): Add new sniffs for a single non-bracketed namespace per file (034 comments) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698359 (https://phabricator.wikimedia.org/T278348) (owner: 10DannyS712) [15:01:50] (03CR) 10Thiemo Kreuz (WMDE): [C: 04-1] FunctionCommentSniff: also skip @{inheritDoc} (031 comment) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698526 (https://phabricator.wikimedia.org/T270830) (owner: 10DannyS712) [15:06:42] (03CR) 10Thiemo Kreuz (WMDE): [C: 04-1] "I disagree with this being a PHPCS sniff. Searching for unused variables is something IDEs or Phan need to do. This is a job for tools tha" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/693531 (https://phabricator.wikimedia.org/T283446) (owner: 10DannyS712) [15:14:50] 10MediaWiki-Codesniffer: MediaWiki.WhiteSpace.SpaceBeforeSingleLineComment.NewLineComment is too strict - https://phabricator.wikimedia.org/T168476 (10thiemowmde) Note this will become more and more irrelevant when we can start using [PHP 8's named arguments](https://www.php.net/manual/en/functions.arguments.php... [15:20:59] Project mediawiki-core-doxygen-docker build #25487: 04STILL FAILING in 16 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/25487/ [15:27:20] (03PS1) 10Hashar: api-testing: add npm run doc and publishing [integration/config] - 10https://gerrit.wikimedia.org/r/698563 (https://phabricator.wikimedia.org/T236915) [15:28:07] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Patch-For-Review, 10Platform Team Initiatives (API Integration Tests): Expose mediawiki/tools/api-testing doc on doc.wikimedia.org - https://phabricator.wikimedia.org/T236915 (10hashar) a:03hashar [15:28:29] (03CR) 10jerkins-bot: [V: 04-1] api-testing: add npm run doc and publishing [integration/config] - 10https://gerrit.wikimedia.org/r/698563 (https://phabricator.wikimedia.org/T236915) (owner: 10Hashar) [15:29:40] (03PS2) 10Hashar: api-testing: add npm run doc and publishing [integration/config] - 10https://gerrit.wikimedia.org/r/698563 (https://phabricator.wikimedia.org/T236915) [15:30:20] (03CR) 10Hashar: "On hold until jsdoc is ready to be added to the repository: https://gerrit.wikimedia.org/r/c/mediawiki/tools/api-testing/+/698562" [integration/config] - 10https://gerrit.wikimedia.org/r/698563 (https://phabricator.wikimedia.org/T236915) (owner: 10Hashar) [15:30:45] (03CR) 10jerkins-bot: [V: 04-1] api-testing: add npm run doc and publishing [integration/config] - 10https://gerrit.wikimedia.org/r/698563 (https://phabricator.wikimedia.org/T236915) (owner: 10Hashar) [15:32:40] (03PS3) 10Hashar: api-testing: add npm run doc and publishing [integration/config] - 10https://gerrit.wikimedia.org/r/698563 (https://phabricator.wikimedia.org/T236915) [15:39:41] (03CR) 10Hashar: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/698563 (https://phabricator.wikimedia.org/T236915) (owner: 10Hashar) [15:45:03] (03PS1) 10Hashar: Add mediawiki/tools/api-testing JSDoc [integration/docroot] - 10https://gerrit.wikimedia.org/r/698564 (https://phabricator.wikimedia.org/T236915) [15:46:31] 10Release-Engineering-Team (Radar), 10Release Pipeline, 10SRE, 10serviceops, and 2 others: Remove obsoleted docker images - https://phabricator.wikimedia.org/T242604 (10JMeybohm) [15:46:39] 10Continuous-Integration-Config, 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Platform Team Initiatives (API Integration Tests): Expose mediawiki/tools/api-testing doc on doc.wikimedia.org - https://phabricator.wikimedia.org/T236915 (10hashar) p:05Triage→03Medium [15:46:43] triaged and fixed a task! [15:46:52] 10Continuous-Integration-Config, 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Platform Team Initiatives (API Integration Tests): Expose mediawiki/tools/api-testing doc on doc.wikimedia.org - https://phabricator.wikimedia.org/T236915 (10hashar) JSDoc is introduced by: https://gerrit.wikimedia.o... [15:52:14] 10MediaWiki-Codesniffer: Forbid usage of array_push with a single element - https://phabricator.wikimedia.org/T242218 (10Daimona) >>! In T242218#7138400, @DannyS712 wrote: > I propose to decline this - as @thiemowmde points out, there are reasons to use `array_push` with a single element (eg to match array_pop)... [15:52:23] 10MediaWiki-Codesniffer: Forbid usage of array_push with a single element - https://phabricator.wikimedia.org/T242218 (10Daimona) 05Open→03Declined [15:57:23] (03CR) 10Daimona Eaytoy: "> Patch Set 2: Code-Review-1" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/693531 (https://phabricator.wikimedia.org/T283446) (owner: 10DannyS712) [16:20:15] Yippee, build fixed! [16:20:15] Project mediawiki-core-doxygen-docker build #25488: 09FIXED in 15 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/25488/ [16:20:58] (03CR) 10Gergő Tisza: FunctionCommentSniff: also skip @{inheritDoc} (031 comment) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698526 (https://phabricator.wikimedia.org/T270830) (owner: 10DannyS712) [16:26:10] (03CR) 10Cory Massaro: "> Patch Set 10:" [integration/config] - 10https://gerrit.wikimedia.org/r/691243 (https://phabricator.wikimedia.org/T282828) (owner: 10Cory Massaro) [16:28:43] 10Release-Engineering-Team (Seen), 10Packaging, 10SRE: Debian-glue doesn't check for the validity of the distribution in the changelog. - https://phabricator.wikimedia.org/T252619 (10hashar) The job uses dpkg-parsechangelog and sets `distribution` which is then used to set `DIST`. That is then passed to pbui... [16:36:33] 10Continuous-Integration-Config, 10Release-Engineering-Team (Seen), 10MediaWiki-extensions-MediaModeration, 10Platform Team Initiatives (Hash Checking): Enable integration testing in CI - https://phabricator.wikimedia.org/T246914 (10hashar) Sorry that task has not been noticed. My rule of thumb is that te... [16:37:07] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Readers-Web-Backlog, 10Vector: [CI] Replace npm-test job with npm-test + MediaWiki Core - https://phabricator.wikimedia.org/T252772 (10hashar) [16:38:34] 10Continuous-Integration-Config, 10Release-Engineering-Team (Seen), 10Readers-Web-Backlog, 10Vector: [CI] Replace npm-test job with npm-test + MediaWiki Core - https://phabricator.wikimedia.org/T252772 (10hashar) That is similar to T243263 which is about retrieving those resources using `curl https://raw.g... [16:39:12] 10Release-Engineering-Team (Seen), 10Page-Previews, 10Readers-Web-Backlog (Tracking), 10patch-welcome: Popups should not use github.com for documentation generation - https://phabricator.wikimedia.org/T243263 (10hashar) T252772 is similar and propose to use relative link / MW_INSTALL_PATH in order to fetch... [16:39:50] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Librarization, 10Quibble: Install extension require-dev dependencies in wmf-quibble-vendor-mysql-hhvm-docker - https://phabricator.wikimedia.org/T220723 (10hashar) [16:40:54] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Analytics-Radar, 10ChangeProp, and 3 others: Run EventBus tests in MediaWiki core CI - https://phabricator.wikimedia.org/T257583 (10hashar) [16:41:21] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10SRE, 10observability, 10Patch-For-Review: Export zuul metrics to Prometheus - https://phabricator.wikimedia.org/T233089 (10hashar) [16:41:59] 10Release-Engineering-Team (Seen), 10SRE, 10SRE-tools: Support running puppet Beaker on CI - https://phabricator.wikimedia.org/T253635 (10hashar) [16:45:07] 10Continuous-Integration-Config, 10Release-Engineering-Team (Seen), 10MediaWiki-extensions-MediaModeration, 10Platform Team Initiatives (Hash Checking): Enable integration testing in CI - https://phabricator.wikimedia.org/T246914 (10Pchelolo) 05Open→03Declined Ok, thank you @hashar This is mostly what... [16:49:38] 10Release-Engineering-Team (Doing), 10Instrument-ClientError, 10Documentation, 10User-brennen: Deploy process should be updated so JavaScript errors block the train - https://phabricator.wikimedia.org/T274000 (10Jdlrobson) @thcipriani I am going to assume resolving this means that release engineering are u... [17:25:07] <_joe_> hi releng friends, I have a blubber question! I want to copy files over from one image to another, but only selected directories. Up to now, I'd do so by using requirements: - but now I'm removing the "command:" stanza from that image, and this means the requirements section is not ran anymore. [17:25:27] <_joe_> so, what instruction should I use instead? [17:28:03] <_joe_> ok, simply swapping "builder.requirements" with "copies" did the trick :) [17:31:30] Happy to help. :-) [17:34:40] <_joe_> dancy: ahah yeah sorry for using you all as a virtual rubberduck [17:35:23] <_joe_> I find it incredibly useful to actually write down my doubts, because in the effort not to look too clueless, I often figure out what I didn't know how to do in the first place [17:35:36] Nod. Same [17:38:00] (03PS1) 10Martaannaj: Add EventLogging to phan dependencies of PropertySuggester [integration/config] - 10https://gerrit.wikimedia.org/r/698584 [17:52:57] 10Release-Engineering-Team, 10Documentation: Document branching time in Deployment Calendar - https://phabricator.wikimedia.org/T284487 (10Reedy) [17:54:26] 10Release-Engineering-Team, 10Documentation: Document automatic MW branching time in Deployment Calendar - https://phabricator.wikimedia.org/T284487 (10Reedy) [18:23:50] 10Continuous-Integration-Config, 10Release-Engineering-Team (Doing), 10MW-1.37-notes (1.37.0-wmf.9; 2021-06-07), 10Patch-For-Review: Upgrade all CI jobs for WMF-deployed projects from Node 10 to Node 12 - https://phabricator.wikimedia.org/T284345 (10zeljkofilipin) Selenium jobs are not updated to node v12?... [18:24:18] andrewbogott: hi, about db grant related to Gerrit, it is probably better to tag #gerrit / #releng on the task ;] I will reply [18:24:46] thanks! [18:25:41] 10Gerrit, 10Release-Engineering-Team, 10wikitech.wikimedia.org, 10cloud-services-team (Kanban): Audit labswiki grants - https://phabricator.wikimedia.org/T282074 (10Andrew) [18:31:47] 10Gerrit, 10Release-Engineering-Team (Radar), 10wikitech.wikimedia.org, 10cloud-services-team (Kanban): Audit labswiki grants - https://phabricator.wikimedia.org/T282074 (10hashar) | wikiadmin | 208.80.154.136 | | wikiuser | 208.80.154.136 | Gerrit no more use a database since we upgrade it back in June... [18:34:30] 10Gerrit, 10Release-Engineering-Team (Radar), 10wikitech.wikimedia.org, 10cloud-services-team (Kanban): Audit labswiki grants - https://phabricator.wikimedia.org/T282074 (10hashar) And still from dns.git: | wikiadmin | 208.80.153.14 | NXDOMAIN | wikiuser | 208.80.153.14 | NXDOMAIN That was `labtestweb... [18:34:38] andrewbogott: tldr , the extra grants are silver and labtestweb2001 . I found the decom boxes via operations/dns.git which has a bunch of history. More recent things would be in Netbox [18:34:50] so essentially the grants can be removed :] [18:40:47] 10Continuous-Integration-Config, 10Release-Engineering-Team (Doing), 10MW-1.37-notes (1.37.0-wmf.9; 2021-06-07), 10Patch-For-Review: Upgrade all CI jobs for WMF-deployed projects from Node 10 to Node 12 - https://phabricator.wikimedia.org/T284345 (10hashar) >>! In T284345#7139795, @zeljkofilipin wrote: > S... [18:44:06] James_F: if around, about Kartotherian I am confident we can drop our custom node10 job given it is now using the Pipeline ( https://gerrit.wikimedia.org/r/c/integration/config/+/698258 ) [18:44:32] James_F: I guess it is now up to the dev to switch their app to whatever base image [18:44:54] unknown is who would be in charge of tracking the migration progress for all the services ;D [18:54:24] thanks hashar [18:55:29] 10Continuous-Integration-Config, 10Release-Engineering-Team (Doing), 10MW-1.37-notes (1.37.0-wmf.9; 2021-06-07), 10Patch-For-Review: Upgrade all CI jobs for WMF-deployed projects from Node 10 to Node 12 - https://phabricator.wikimedia.org/T284345 (10Jdforrester-WMF) >>! In T284345#7139795, @zeljkofilipin w... [19:00:39] hashar: Yeah, I was waiting for one of them to agree it was good to dorp. [19:00:52] hashar: Migrating the services would be SRE's problem. :-) [19:01:16] I think they have analysis tool/script to track obsolete images indeed [19:01:26] problem solved! [19:01:46] so yeah, in short I am in favor of just dropping our CI image from karthoterian [19:02:07] we probably forgot to drop it as part of migrating to the Pipeline [19:14:49] (03PS3) 10DannyS712: FunctionCommentSniff: also skip {@inheritDoc} [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698526 (https://phabricator.wikimedia.org/T270830) [19:15:24] (03CR) 10DannyS712: FunctionCommentSniff: also skip {@inheritDoc} (031 comment) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698526 (https://phabricator.wikimedia.org/T270830) (owner: 10DannyS712) [19:37:40] hashar: No, I remember having to keep it around for some reason, but it was ages ago. [19:38:05] hashar: See also maps/kartotherian/deploy, maps/tilerator/deploy, and maps/kartotherian/package. :-( [19:39:23] 10Continuous-Integration-Infrastructure, 10Gerrit, 10Release-Engineering-Team (Seen), 10Zuul: Display Zuul status of jobs for a change on Gerrit UI - https://phabricator.wikimedia.org/T214068 (10hashar) @Paladox I feel sorry this has taken sometime. Now that we have Gerrit 3.2 and I have entirely overhaule... [19:40:11] James_F: the /deploy repo were for deploy node_modules but if they have moved to k8S I guess they are obsolete [19:40:19] such a legacy stack :-\\ [19:40:46] anyway, the Pipeline does seem to run npm test so I think that covers the same thing we are doing in our node10 job [19:41:23] Yeah, hopefully? [19:41:26] Will ping them. [19:41:46] (03PS1) 10Jforrester: Zuul: [mediawiki/services/kartotherian] Drop non-pipeline job [integration/config] - 10https://gerrit.wikimedia.org/r/698610 [19:41:48] (03PS1) 10Jforrester: jjb: Drop kartotherian-node10-browser-docker, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/698611 [19:41:50] (03PS1) 10Jforrester: dockerfiles: Drop node10-kartotherian, no longer used [integration/config] - 10https://gerrit.wikimedia.org/r/698612 [19:51:27] (03PS3) 10DannyS712: Add new AnonymousFunctionUseSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/693531 (https://phabricator.wikimedia.org/T283446) [19:52:58] (03CR) 10DannyS712: "> Patch Set 2: Code-Review-1" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/693531 (https://phabricator.wikimedia.org/T283446) (owner: 10DannyS712) [19:56:57] hashar: I vaguely think they're testing in the pipeline but still deployed manually, or something dreadful. Certainly the deploy repos have commits in the past few months, though I think that might be just for Beta Cluster. A mess, anyway. :-( [19:57:26] (03Abandoned) 10Jforrester: [WIP] dockerfiles: Provide a node12-based Kartotherian image [integration/config] - 10https://gerrit.wikimedia.org/r/698258 (https://phabricator.wikimedia.org/T284345) (owner: 10Jforrester) [19:59:45] 10Continuous-Integration-Infrastructure: CI failures in "git-status" task related to submodules - https://phabricator.wikimedia.org/T284492 (10matmarex) [20:01:28] (03CR) 10Umherirrender: [C: 03+1] DocumentationTypeTrait: complain about `type` as a type [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698290 (https://phabricator.wikimedia.org/T273806) (owner: 10DannyS712) [20:15:14] 10Continuous-Integration-Infrastructure: CI failures in "git-status" task related to submodules - https://phabricator.wikimedia.org/T284492 (10Jdforrester-WMF) Given that 1f76dea7e63ca6b7bd5f1ef6a372c1878429f973 is the current value pointer for VE-MW HEAD, but 3891890c29a72941b0eae1bfd7b3b829bc58b343 is the curr... [20:16:52] 10Continuous-Integration-Infrastructure: CI failures in "git-status" task related to submodules - https://phabricator.wikimedia.org/T284492 (10Jdforrester-WMF) The node10 images have git 2.20.1; the node12 ones have git 2.30.2. Possibly something changed about how `git submodule` checkouts work? [20:17:47] liw: andrewbogott: I seem to be hitting a problem with Qemu VM in WMCS. Something random and low-level in Node.js is consistently failing in containers with Debian 11 Bullseye + Node 12, but passing in containers with Debian 9 Stretch + Node 10. I'm told the magic wand to fix this is to "enable KVM" https://github.com/nodejs/node/issues/34709 [20:17:59] but I'm not sure how or where to go with that, or whether that's possible/allowed [20:18:36] given we're inside another VM :) [20:20:04] I suppose one option I have is to try to re-create this agent with a newer debian version for the outer VM [20:20:10] Krinkle: I don't think we support VMs in VMs. There was a trial for that recently, let me see if I can find the history. [20:20:14] James_F: per your email, cirrus doesn't pass mwgate-node12-docker, assuming it's because of the node 10 deps used in the browser testing suite: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/698608 [20:20:37] ebernhardson: Ah, fun. :-( In a meeting right now. Will have a look after the hour. [20:20:42] andrewbogott: CI has been running VMS for over a year now. [20:21:04] it's the only way to test software that invokes docker [20:21:23] 10Release-Engineering-Team, 10Release Pipeline, 10function-evaluator, 10Abstract Wikipedia team (Phase ε): Service pipeline publish job creates and then immediately deletes its tags - https://phabricator.wikimedia.org/T284495 (10Jdforrester-WMF) [20:21:26] well, maybe not the only, but the one feasible option we ended up going with [20:22:09] Project mediawiki-core-doxygen-docker build #25492: 04FAILURE in 17 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/25492/ [20:23:17] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.9 deployment blockers - https://phabricator.wikimedia.org/T281150 (10matmarex) [20:24:13] Krinkle: here is the only context i know: T267433 T276208 [20:24:14] T267433: Enable support for nested VMs - https://phabricator.wikimedia.org/T267433 [20:24:14] T276208: cloud: libvirt doesn't support live migration when using nested KVM - https://phabricator.wikimedia.org/T276208 [20:24:33] My understanding is that non-native nesting (which might be 'qemu') works but is very slow [20:24:41] and kvm-on-kvm is not supported [20:24:50] But it's starting to seem like you know more about this than I do [20:24:54] I'm also curious about https://www.mediawiki.org/wiki/Topic:W7zkraklqyav2z6d :) [20:24:58] gah [20:25:15] (wrong window) [20:25:30] Krinkle: and of course Bullseye isn't in final release yet so that could confound things [20:26:36] andrewbogott: ah I see, yeah, I remember liw saying it would be faster if we use kvm, which we indeed didn't/couldn't do at the time. I didn't realize dancy had since requested to enable it, and I understand now that that was reverted. I hadn't connected the dots between "kvm for perf (1 year ago)" and the "enable kvm" I read now. [20:26:39] Makes sense , thx [20:28:55] Krinkle: it's not 100% out of the question to enable it again if it turns out to be necessary. We'd probably need to set up some kind of special handling for things that use nested kvm. [20:29:11] And it would be a non-trivial amount of work [20:29:58] andrewbogott: yeah, I wasn't aware of the migration caveat that makes sense. [20:30:28] bullseye is only used in the container, not either of the VMs [20:30:44] I don't know why that image uses bullseye already, that does seem a bit early indeed [20:31:01] I suspect it's to use the debian node12 package without needing SRE to backport it [20:31:13] Krinkle: Yes. [20:31:29] but this might end up incurring more cost that it saves if it means everything else it uses is out of step / untested by us [20:31:54] backporting is "usually" easy, but I don't know if that's the case with node [20:32:06] or whether it woudl be okay to use a third-party PPA given that it won't match prod either way [20:32:29] then we could still use the main buster base image [20:32:33] I wanted to avoid a third-party PPA as it's such a nightmare keeping things working. [20:33:08] yeah, for sure. [20:33:44] For the very rare qemu use case we could have a distinct node12-on-buster-nightmare image that's only used for that use case? [20:34:07] so our options are 1) try node12 as buster with ppa node and hope bullseye is the issue, 2) try updating the qemu image from stretch to buster and hope newer qemu will somehow fix it, or 3) experimentaly ask kvm to be enabled just for this one VM [20:34:40] s/qemu image/qemu agent/ [20:35:10] And possibly none of those three will fix it. [20:36:18] 10Gerrit, 10Release-Engineering-Team (Radar), 10wikitech.wikimedia.org, 10cloud-services-team (Kanban): Audit labswiki grants - https://phabricator.wikimedia.org/T282074 (10Andrew) a:05Andrew→03Marostegui I think that's all of them explained! Back to you @Marostegui [20:36:29] 4) replace Jenkins with GitLab ;) [20:36:59] 5) run the same dead node+npm versions in CI as we run in prod [20:37:45] James_F: I forgot about the beta cluster thingie, thought I think there is some support to run containerized service there (somehow) :-\ [20:37:53] hashar: Nope. :-( [20:38:37] bd808: Losing updates to all our CI tools for a year or two until SRE moves us to node12/bullseye wouldn't be great (and by that time node12 would be EOL). [20:38:44] and git 2.20 .. 2.30 upgrade bah too many things [20:38:50] hashar: Yeah. :-( [20:40:37] we can always rollback mwgate-node12-dockerto node10 [20:40:56] or it is a mess with the git-status grunt task [20:41:12] hashar: As I said, I'm in meetings. Let's discuss in half an hour. [20:41:25] James_F: I will be asleep probably :/ [20:41:33] I'll work something out. [20:50:55] 10Continuous-Integration-Infrastructure: CI failures in "git-status" task related to submodules - https://phabricator.wikimedia.org/T284492 (10hashar) [20:57:30] James_F: found it :] [20:57:34] 10Continuous-Integration-Infrastructure: CI failures in "git-status" task related to submodules - https://phabricator.wikimedia.org/T284492 (10hashar) Looking at the build for https://gerrit.wikimedia.org/r/c/mediawiki/extensions/VisualEditor/+/698547 which is https://integration.wikimedia.org/ci/job/mwgate-node... [20:57:52] node12-test/run.sh has a git submodule update --init --remote [20:58:03] which update the submodules from the upstream repository instead of what is registered [20:58:31] I guess it got copied from the -portals image which is an adhoc job for mediawiki portals [20:59:02] the entry point for node12-test should not process submodules [20:59:11] that is managed by the ci-src-setup image :] [20:59:57] Oh duh, yes. [21:00:10] But I wanted to scrap the portals-specific image. [21:03:00] (03PS1) 10Hashar: dockerfiles: node12-test do not process submodules [integration/config] - 10https://gerrit.wikimedia.org/r/698624 (https://phabricator.wikimedia.org/T284492) [21:03:06] James_F: ^ will let you manage that one [21:03:14] Ack. [21:03:26] the portals that was a quick sprint hack with Jan [21:03:31] but surely it can be revisited [21:03:40] Right. [21:05:48] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Cloud-VPS (Quota-requests): Request increased quota for 'integration' Cloud VPS project - https://phabricator.wikimedia.org/T284507 (10Krinkle) [21:06:06] anyway bug sliced [21:06:28] i guess that can be rebuild, jobs updated and done [21:06:39] Thanks! Will have to see if it breaks the portals job. [21:06:57] for portals you will need a dedicated job [21:07:56] or maybe the --remote was never needed [21:13:15] 10Continuous-Integration-Infrastructure, 10Patch-For-Review: CI failures in "git-status" task related to submodules - https://phabricator.wikimedia.org/T284492 (10hashar) And for the context, the `--remote` used to be in the original job we crafted for portals ( https://gerrit.wikimedia.org/r/c/integration/con... [21:13:23] traced the history on the task in case someone searches for it later on [21:18:46] Yippee, build fixed! [21:18:46] Project mediawiki-core-doxygen-docker build #25493: 09FIXED in 14 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/25493/ [21:32:39] (03PS2) 10Jforrester: dockerfiles: node12-test do not process submodules [integration/config] - 10https://gerrit.wikimedia.org/r/698624 (https://phabricator.wikimedia.org/T284492) (owner: 10Hashar) [21:34:32] (03PS3) 10Jforrester: dockerfiles: [node12-test] Do not process submodules [integration/config] - 10https://gerrit.wikimedia.org/r/698624 (https://phabricator.wikimedia.org/T284492) (owner: 10Hashar) [21:34:38] (03CR) 10Jforrester: [C: 03+2] dockerfiles: [node12-test] Do not process submodules [integration/config] - 10https://gerrit.wikimedia.org/r/698624 (https://phabricator.wikimedia.org/T284492) (owner: 10Hashar) [21:37:07] (03Merged) 10jenkins-bot: dockerfiles: [node12-test] Do not process submodules [integration/config] - 10https://gerrit.wikimedia.org/r/698624 (https://phabricator.wikimedia.org/T284492) (owner: 10Hashar) [21:40:25] !log Docker: Pushing node12-test ano node12-test-browser 0.0.2 for T284492 [21:40:28] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:40:28] T284492: CI failures in "git-status" task related to submodules - https://phabricator.wikimedia.org/T284492 [21:41:35] PROBLEM - SSH on contint2001.mgmt is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [21:43:02] 10Phabricator, 10Release-Engineering-Team (Seen), 10User-Matthewrbowker: Phacility (Maintainer of Phabricator) is winding down. Upstream support ending. - https://phabricator.wikimedia.org/T283980 (10DonTrung) This happened in a really short time after the whole IRC debacle, why can't there be a Phabricator... [21:45:35] 10Phabricator, 10Release-Engineering-Team (Seen), 10User-Matthewrbowker: Phacility (Maintainer of Phabricator) is winding down. Upstream support ending. - https://phabricator.wikimedia.org/T283980 (10Reedy) >>! In T283980#7140531, @DonTrung wrote: > This happened in a really short time after the whole IRC de... [21:55:14] 10Phabricator, 10Release-Engineering-Team (Seen), 10User-Matthewrbowker: Phacility (Maintainer of Phabricator) is winding down. Upstream support ending. - https://phabricator.wikimedia.org/T283980 (10bd808) >>! In T283980#7140531, @DonTrung wrote: > This happened in a really short time after the whole IRC de... [22:02:35] !log urbanecm@deployment-sessionstore04:~$ sudo touch /etc/cassandra/service-enabled #T263617 [22:02:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:02:37] T263617: Cannot login to beta cluster: "There seems to be a problem with your login session..." - https://phabricator.wikimedia.org/T263617 [22:02:53] !log urbanecm@deployment-sessionstore04:~$ sudo service cassandra start # T263617 [22:02:55] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:04:20] 10Beta-Cluster-Infrastructure, 10Beta-Cluster-reproducible, 10User-Majavah, 10User-Ryasmeen, 10User-zeljkofilipin: Cannot login to beta cluster: "There seems to be a problem with your login session..." - https://phabricator.wikimedia.org/T263617 (10Urbanecm) [22:04:56] James_F: checking back in to see if you had ideas on the node 12 job and cirrus [22:07:01] (03PS1) 10Jforrester: jjb: Switch node12 images to ones without a git submodule over-ride [integration/config] - 10https://gerrit.wikimedia.org/r/698627 (https://phabricator.wikimedia.org/T284492) [22:07:16] ebernhardson: Just pushing new node12 images for a different thing, then looking at Cirrus. Sorry! [22:07:17] 10Beta-Cluster-Infrastructure, 10Beta-Cluster-reproducible, 10User-Majavah, 10User-Ryasmeen, 10User-zeljkofilipin: Cannot login to beta cluster: "There seems to be a problem with your login session..." - https://phabricator.wikimedia.org/T263617 (10Urbanecm) @dom_walden I temporarily "fixed" this by manu... [22:13:38] (03CR) 10Jforrester: [C: 03+2] "Deployed. wikimedia-portals-build seems to still work?" [integration/config] - 10https://gerrit.wikimedia.org/r/698627 (https://phabricator.wikimedia.org/T284492) (owner: 10Jforrester) [22:14:50] (03Merged) 10jenkins-bot: jjb: Switch node12 images to ones without a git submodule over-ride [integration/config] - 10https://gerrit.wikimedia.org/r/698627 (https://phabricator.wikimedia.org/T284492) (owner: 10Jforrester) [22:19:43] ebernhardson: It's most perplexing. I've managed to replicate locally, but also fix it locally, except git claims it's a no-op change. [22:22:19] hmm [22:22:36] James_F: whats the change that git is ignoring? [22:22:37] Aha, npm 6.x vs. 7.x perhaps? [22:22:52] ebernhardson: I re-built package-lock.json from scratch. One mo. [22:22:56] ahh [22:25:01] Yeah, looks like fibers 3.1.1 isn't compatible? [22:25:35] seems likely, in general we are on 3 yr old wdio-* packages which are likely not compatible with node 12 [22:25:44] How do they even work? [22:25:49] ✔ Do you want to upgrade: wdio-mediawiki 0.4.0 → 1.1.1? … no [22:25:51] Ouch. [22:26:05] They work by having worked one day, and then not changing things :P [22:26:24] Uh-huh. [22:26:35] Not changing things generally leads to things breaking IME. [22:27:14] depends on what else holds constant, in thise case all this node js is typically run from a single host still provisioned with generally the same things as 3 years ago [22:27:32] Ah, you're not using any of the general tools? [22:28:07] no, this is still the same basic thing as when they made the first push to shift all the integration tests from ruby to nodejs [22:29:46] Ah you really still using cucumber? Wow. [22:29:50] That's totally unsupported in CI. [22:30:03] Ci never supported multi-wikis, so it didn't really matter :S [22:30:34] As in we do things that will actively break it. [22:31:13] Why? [22:31:17] Switching from "wdio-cucumber-framework" to "@wdio/cucumber-framework" still doesn't get us to a new enough version of fibers. [22:32:03] Because cucumber does a bunch of screwy stuff with threads (hence use of fibers) that aren't how the rest of CI node processes work, so every other tool is written carefully not to do that. [22:32:32] The move to Apache-based quibble will be multi-threaded and I imagine would break with that framework? Nothing tested, and I don't work on this stuff any more. [22:32:48] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.9 deployment blockers - https://phabricator.wikimedia.org/T281150 (10thcipriani) >>! In T281150#7137851, @Ladsgroup wrote: > {meme, src=whereismypopcorn} Is there a risky patch or something we should be worried about in this train? [22:33:59] interesting, i imagine some day we'll move things but i wouldn't expect any time soon [23:05:09] ebernhardson: OK, https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/698631 passes on node12 locally and in CI. No idea about your jerry-rigged cucumber runner, sorry. I note it's also been attempted before in https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/610396 :-( [23:10:11] it's not so jerry-rigged right now, just a standard mwv. Getting node 12 into mwv will probably cause that though :P [23:10:29] So, it can't continue running node10? It's a bit difficult to come in monday morning with no ability to merge things [23:11:22] "Standard" MWV, a technology we dropped support for years ago? :-) [23:11:50] Yeah, I'm really sorry about this. Node10 went EOL over a month ago now and so many of our upstreams are dropping it. [23:11:52] merging the change doesn't really change anything, some set of tests still don't run, but it shifts it to a set of tests we care more about [23:12:02] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.9 deployment blockers - https://phabricator.wikimedia.org/T281150 (10DannyS712) >>! In T281150#7140642, @thcipriani wrote: > Is there a risky patch or something we should be worried about in this train? The planned removal of the... [23:12:04] Yeah. :-( [23:12:24] James_F: Hey i've asked about twenty times for something, anything to support multi-wiki. The sad thing is no one cares enough, so we can't move [23:12:33] I would love to move, but our use case doesn't seem to be important [23:12:55] ebernhardson: Yeah, I know. This was the number one ask in the APP process from Search Platform for Engineering Productivity, I guess? [23:13:11] It's hugely important to me. I've got a bunch of things that operate cross-wiki that we can't test in CI. [23:13:27] Not least, all the CentralAuth, Wikidata, or Commons stuff. [23:15:34] Not sure what we put in on the engineering productivity, seems plausible. I certainly mention it on any surveys :) [23:15:56] Surveys butter no parsnips. :-) This is a management thing. We've been talking about this for years. [23:16:40] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.9 deployment blockers - https://phabricator.wikimedia.org/T281150 (10Ladsgroup) >>! In T281150#7140642, @thcipriani wrote: > Is there a risky patch or something we should be worried about in this train? No risky patch but it'll b... [23:16:52] I'm wary to drop any of these multi-wiki tests. I'm just about sure they will break due to some unrelated refactoring without tests....i guess have to see what it takes to get node 12 in mwv [23:21:52] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.9 deployment blockers - https://phabricator.wikimedia.org/T281150 (10thcipriani) >>! In T281150#7140695, @Ladsgroup wrote: >>>! In T281150#7140642, @thcipriani wrote: >> Is there a risky patch or something we should be worried abo... [23:23:52] 10Continuous-Integration-Infrastructure: CI failures in "git-status" task related to submodules - https://phabricator.wikimedia.org/T284492 (10Jdforrester-WMF) 05Open→03Resolved a:03Jdforrester-WMF This should be back to working everywhere. If not, please shout. [23:39:58] ebernhardson: maybe T256822 would help, but we would still need a backport of node12 to Buster. [23:39:59] T256822: [EPIC] Upgrade MediaWiki-Vagrant to Debian Buster - https://phabricator.wikimedia.org/T256822 [23:40:37] bd808: hmm, i'll check it out. the repo we use seems to have a node12 build for buster [23:41:16] I started on making the Buster branch work and then got sidetracked by lots of other things [23:49:35] (03CR) 10DannyS712: Add new sniffs for a single non-bracketed namespace per file (033 comments) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/698359 (https://phabricator.wikimedia.org/T278348) (owner: 10DannyS712)