[01:06:13] !log unstick beta ci jobs [01:06:15] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [01:31:48] (03PS2) 10Jforrester: jjb: Bump Quibble jobs to 1.1.0 [integration/config] - 10https://gerrit.wikimedia.org/r/726880 (https://phabricator.wikimedia.org/T291549) (owner: 10Kosta Harlan) [01:33:04] (03CR) 10Jforrester: [C: 03+1] "PS2: Re-based and added the affected jobs list. LGTM, good luck!" [integration/config] - 10https://gerrit.wikimedia.org/r/726880 (https://phabricator.wikimedia.org/T291549) (owner: 10Kosta Harlan) [02:09:58] ejegg: I'm working on some FR-vendor patches to make your patches work. [02:38:23] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T281167 (10Jdforrester-WMF) Note that I've upgraded the CI world to composer 2.x, which works fine, but which brought along a new ve... [03:01:07] thanks James_F! [04:27:50] 10Continuous-Integration-Config, 10Fresnel, 10Performance-Team: All quibble-fresnel jobs crash before starting: "Could not find browser" - https://phabricator.wikimedia.org/T292684 (10Krinkle) [04:29:05] 10Continuous-Integration-Config, 10Fresnel, 10Performance-Team: All quibble-fresnel jobs crash before starting: "Could not find browser" - https://phabricator.wikimedia.org/T292684 (10Krinkle) The only recent change I could find to the job is "Quibble 1.1.0" * https://gerrit.wikimedia.org/r/c/integration/co... [04:31:42] 10Continuous-Integration-Config, 10Fresnel, 10Performance-Team: All quibble-fresnel jobs crash before starting: "Could not find browser" - https://phabricator.wikimedia.org/T292684 (10Krinkle) Nevermind, the JJB patch there is not yet applied to the live Jenkins job. [04:41:04] 10Continuous-Integration-Config, 10Fresnel, 10Performance-Team: All quibble-fresnel jobs crash before starting: "Could not find browser" - https://phabricator.wikimedia.org/T292684 (10Krinkle) * Last successful build: [45642](https://integration.wikimedia.org/ci/job/mediawiki-fresnel-patch-docker/45642/) at... [04:41:54] !log Reconfigure `mediawiki-fresnel-patch-docker` job to revert quibble change back to quibble-fresnel:1.0.1 [04:41:56] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [04:46:25] !log Temporarily change `mediawiki-fresnel-patch-docker` job change back to quibble-fresnel:1.0.1 ref T292684 [04:46:27] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [04:46:27] T292684: All quibble-fresnel jobs crash before starting: "Could not find browser" - https://phabricator.wikimedia.org/T292684 [04:49:12] 10Continuous-Integration-Config, 10Fresnel, 10Performance-Team: All quibble-fresnel jobs crash before starting: "Could not find browser" - https://phabricator.wikimedia.org/T292684 (10Krinkle) p:05Triageβ†’03Medium That fixed the issue for now. Lowering priority for now. This can investigated now without t... [06:49:46] 10Project-Admins: Create project tag for me User-Juan90264 - https://phabricator.wikimedia.org/T292687 (10Juan90264) [06:51:52] 10Project-Admins: Create project tag for me User-Juan90264 - https://phabricator.wikimedia.org/T292687 (10Juan90264) [07:22:22] 10Continuous-Integration-Config, 10Fresnel, 10Performance-Team: All quibble-fresnel jobs crash before starting: "Could not find browser" - https://phabricator.wikimedia.org/T292684 (10kostajh) @Krinkle did you change this in integration/config? If so, I'm not seeing it. I'd like to roll out this patch https:... [07:24:12] (03PS3) 10Kosta Harlan: jjb: Bump almost all Quibble jobs to 1.1.0 [integration/config] - 10https://gerrit.wikimedia.org/r/726880 (https://phabricator.wikimedia.org/T291549) [07:24:34] (03CR) 10Kosta Harlan: jjb: Bump almost all Quibble jobs to 1.1.0 (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/726880 (https://phabricator.wikimedia.org/T291549) (owner: 10Kosta Harlan) [08:04:55] hashar: bonjour [08:22:34] (03PS2) 10Gehel: Initial configuration of the new analytics/gobblin-wmf project. [integration/config] - 10https://gerrit.wikimedia.org/r/726907 [08:22:49] (03CR) 10Gehel: Initial configuration of the new analytics/gobblin-wmf project. (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/726907 (owner: 10Gehel) [09:12:59] 10Release-Engineering-Team (Done by Wed 06 Oct), 10Code-Health, 10Developer Productivity, 10GitLab (Integrations), 10User-brennen: Investigate whether we can/should integrate Git/Reviewers with GitLab - https://phabricator.wikimedia.org/T289712 (10kostajh) If you're mentioned in a MR, it shows up in your... [10:03:32] 10Continuous-Integration-Config, 10Wikidata, 10Wikidata-Campsite, 10wdwb-tech, 10User-awight: Migrate Wikibase selenium tests to Quibble+Apache, enable concurrency - https://phabricator.wikimedia.org/T291476 (10awight) [10:04:29] 10Continuous-Integration-Config, 10Wikidata, 10Wikidata-Campsite, 10wdwb-tech, 10User-awight: Migrate Wikibase selenium tests to Quibble+Apache, enable concurrency - https://phabricator.wikimedia.org/T291476 (10awight) Just discovered that @Addshore has already migrated Wikibase CI to use the apache job,... [10:14:25] 10Continuous-Integration-Config, 10Wikidata, 10Wikidata-Campsite, 10wdwb-tech, 10User-awight: Migrate Wikibase selenium tests to Quibble+Apache, enable concurrency - https://phabricator.wikimedia.org/T291476 (10awight) [10:20:52] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, and 2 others: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10jijiki) [10:27:21] 10Continuous-Integration-Config, 10Wikidata, 10Wikidata-Campsite, 10wdwb-tech, 10User-awight: Migrate Wikibase selenium tests to Quibble+Apache, enable concurrency - https://phabricator.wikimedia.org/T291476 (10awight) >>! In T291476#7407856, @awight wrote: > Just discovered that @Addshore has already mi... [10:43:02] 10Continuous-Integration-Config, 10Wikidata, 10Wikidata-Campsite, 10wdwb-tech, 10User-awight: Migrate Wikibase selenium tests to Quibble+Apache, enable concurrency - https://phabricator.wikimedia.org/T291476 (10awight) [11:05:35] 10Phabricator: Decide guidelines for creation and how to distribute phabricator badges - https://phabricator.wikimedia.org/T196390 (10zeljkofilipin) [11:05:53] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, and 2 others: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10jijiki) [11:11:04] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, and 2 others: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10jijiki) [11:13:00] 10Continuous-Integration-Config, 10Wikidata, 10Wikidata-Campsite, 10wdwb-tech, 10User-awight: Migrate Wikibase selenium tests to Quibble+Apache, enable concurrency - https://phabricator.wikimedia.org/T291476 (10awight) More glitches which happen locally and I can see in a few CI failures, ` [Chrome 90.0.... [11:13:11] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Radar), 10Browser-Tests, 10User-zeljkofilipin: Revisit how we run Selenium tests in our repos - https://phabricator.wikimedia.org/T282840 (10zeljkofilipin) [11:25:50] kostajh: I am lunching, will do the deploy after. In roughly an hour [11:46:48] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T281167 (10Zabe) [12:21:15] hashar: ping me when you have a moment to merge https://gerrit.wikimedia.org/r/c/integration/config/+/726907 (I'll be there until 4pm) [12:23:04] I'll be in https://meet.google.com/cmx-srik-fht [12:33:12] hashar: sounds good, thx [12:39:00] :) [12:43:23] gehel: looks straight forward [12:43:59] (03CR) 10Hashar: [C: 03+2] Initial configuration of the new analytics/gobblin-wmf project. [integration/config] - 10https://gerrit.wikimedia.org/r/726907 (owner: 10Gehel) [12:44:19] eventually we will want to look at migrating to java 11 [12:46:47] once we migrate hadoop to jav 11! [12:46:48] (03Merged) 10jenkins-bot: Initial configuration of the new analytics/gobblin-wmf project. [integration/config] - 10https://gerrit.wikimedia.org/r/726907 (owner: 10Gehel) [12:47:44] oops the project name is wrong in zuul/layout.yaml . Will amend [12:48:34] what was it supposed to be? [12:48:41] (03PS1) 10Hashar: zuul: fix gobblin-wmf Gerrit project name [integration/config] - 10https://gerrit.wikimedia.org/r/727330 [12:48:47] I've created a stupid commit just to test: https://gerrit.wikimedia.org/r/c/analytics/gobblin-wmf/+/727329 [12:48:51] analytics-gobblin-wmf > analytics/gobblin-wmf [12:48:54] dash > slash [12:49:05] Oh, ofc [12:49:33] I have pushed the change [12:49:40] and rechecked [12:49:45] kostajh: I will do the quibble jobs updated [12:50:21] thanks, i'm around for another ~45 minutes or so [12:50:21] 10Release-Engineering-Team (Doing), 10mwcli, 10GitLab (CI & Job Runners), 10User-brennen: Increase maximum artifacts size for gitlab CI - https://phabricator.wikimedia.org/T292372 (10Addshore) >>! In T292372#7400629, @brennen wrote: > Will test this tomorrow: https://gitlab.wikimedia.org/releng/gitlab-sett... [12:51:26] hashar: thanks! I'll keep a look on the build, but if it has started, it's probably good! [12:55:54] probably yes ;) [12:56:03] the publish step might fail later on cause it runs site:site target [12:58:00] (03PS4) 10Hashar: jjb: Bump almost all Quibble jobs to 1.1.0 [integration/config] - 10https://gerrit.wikimedia.org/r/726880 (https://phabricator.wikimedia.org/T291549) (owner: 10Kosta Harlan) [12:58:29] kostajh: I have amended james/your change to also update releng/quibble-buster-php73-coverage [12:58:34] the 1.1.0 version got mis numbered [12:59:27] ah, oops [12:59:32] hashar: but does 1.1.0-s0 exist? [12:59:40] https://gerrit.wikimedia.org/r/c/integration/config/+/726880/2..3/jjb/mediawiki.yaml [12:59:42] yup [12:59:53] so someone bumped that coverage specific image to 1.1.0 [13:00:05] which was an arbitrary number and was done before 1.1.0 got cut [13:00:18] and later the image got a 1.1.1 version [13:00:29] so I rolled back the version numbering back to 1.1.0 with -s0 since a 1.1.0 already exited [13:00:32] existed [13:01:19] deploying those [13:01:48] (03CR) 10jerkins-bot: [V: 04-1] jjb: Bump almost all Quibble jobs to 1.1.0 [integration/config] - 10https://gerrit.wikimedia.org/r/726880 (https://phabricator.wikimedia.org/T291549) (owner: 10Kosta Harlan) [13:01:54] grblb [13:02:04] ah [13:02:29] 10Release-Engineering-Team, 10ci-test-error (WMF-deployed Build Failure): TAR_ENTRY_ERROR ENOSPC: no space left on device - https://phabricator.wikimedia.org/T292729 (10kostajh) [13:02:56] (03PS5) 10Hashar: jjb: Bump almost all Quibble jobs to 1.1.0 [integration/config] - 10https://gerrit.wikimedia.org/r/726880 (https://phabricator.wikimedia.org/T291549) (owner: 10Kosta Harlan) [13:04:19] (03CR) 10Hashar: [C: 03+2] "I have amended the patch to also update jobs that had releng/quibble-buster-php73-coverage:1.1.0. The image got erroneously numbered and d" [integration/config] - 10https://gerrit.wikimedia.org/r/726880 (https://phabricator.wikimedia.org/T291549) (owner: 10Kosta Harlan) [13:04:47] !log Updating Jenkins jobs for Quibble 1.1.0 # T291549 [13:04:50] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:04:50] T291549: Evaluate phpbench for benchmarking and guarding against performance regressions - https://phabricator.wikimedia.org/T291549 [13:07:13] (03Merged) 10jenkins-bot: jjb: Bump almost all Quibble jobs to 1.1.0 [integration/config] - 10https://gerrit.wikimedia.org/r/726880 (https://phabricator.wikimedia.org/T291549) (owner: 10Kosta Harlan) [13:10:08] !log Upgraded releases Jenkins from 2.289.2 to 2.303.2 [13:10:09] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:10:19] kostajh: deployed! [13:10:29] hashar: cheers! [13:10:37] I am going to upgrade Jenkins [13:14:34] !log Upgraded CI Jenkins on contint2001 [13:14:38] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:15:34] good luck :) [13:28:14] gehel: tadam https://doc.wikimedia.org/analytics-gobblin-wmf/gobblin-wmf/ [13:28:48] there is some repetition in that url ;D [13:29:07] obblin-wmf/ [13:29:15] !log restarting CI Jenkins for git plugin update [13:29:17] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:33:22] hashar: cool! [13:33:29] not really sure how we construct those URLs... [13:37:17] hashar: I get a Not Found on https://integration.wikimedia.org/job/wikimedia-event-utilities-maven-release-docker [13:37:25] gehel: it copies src/target/staging to /analytics-gobblin-wmf/ (the jjb project name) [13:37:53] I got this URL from jenkins search, so I suspect it exists, but I don't have access [13:38:00] missing /ci/ https://integration.wikimedia.org/ci/job/wikimedia-event-utilities-maven-release-docker [13:38:01] Or I don't understand how to find a job [13:38:19] I guess their search result got broken after the Jenkins upgrade :( [13:38:25] cool! thanks! [13:38:41] about the maven site URL, I don't think I care about it enough to clean it up :/ [13:39:04] yeah it is probably [13:39:05] fine [13:39:13] we can just add it to doc.wikimedia.org homepage as is [14:02:51] 10Project-Admins: Create project tag for me User-Juan90264 - https://phabricator.wikimedia.org/T292687 (10Juan90264) @Aklapper: Can you accomplish my task? [14:20:26] (03CR) 10Hashar: [C: 03+2] Gerrit v3.3.6 and rebuild plugins [software/gerrit] (deploy/wmf/stable-3.3) - 10https://gerrit.wikimedia.org/r/716317 (https://phabricator.wikimedia.org/T290236) (owner: 10Hashar) [14:20:44] (03Merged) 10jenkins-bot: Gerrit v3.3.6 and rebuild plugins [software/gerrit] (deploy/wmf/stable-3.3) - 10https://gerrit.wikimedia.org/r/716317 (https://phabricator.wikimedia.org/T290236) (owner: 10Hashar) [14:24:54] I forgot about that one [14:44:34] (03CR) 10Ahmon Dancy: Fix build-mw-image-loop.py startup race (031 comment) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/724168 (owner: 10Ahmon Dancy) [14:44:46] 10Release-Engineering-Team, 10SRE: Add Ahmon and Brennen to Icinga contact list - https://phabricator.wikimedia.org/T292753 (10hashar) [14:46:32] brennen: dancy: I am going to upgrade the Gerrit replica since I completely forgot about upgrading gerrit :-\ [14:46:35] it is a bit overdue [14:46:53] πŸ‘πŸΎ [14:52:23] it is good we have a clear runbook [14:52:34] Nice work on that. [14:52:50] I went to craft a patch that port those commands to scap [14:52:56] but well .. I have been too lazy to complete it [15:03:13] (03CR) 10Ahmon Dancy: Replace simple git server with Gerrit (034 comments) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [15:06:12] I am going to do the primary [15:06:26] 10Gerrit, 10Release-Engineering-Team (Doing), 10Patch-For-Review: Upgrade Gerrit from 3.3.5 to 3.3.6 - https://phabricator.wikimedia.org/T290236 (10hashar) 05Openβ†’03In progress ` name=ssh -p 29418 gerrit-replica.wikimedia.org gerrit plugin ls Name Version Api-Version Sta... [15:06:28] 10Gerrit, 10Upstream: Can't dismiss reviewer autosuggestion dropdown in new Gerrit - https://phabricator.wikimedia.org/T288193 (10hashar) [15:06:30] 10Gerrit, 10Release-Engineering-Team (Radar), 10Regression, 10Upstream: Gerrit names misalignment in 3.3.5 - https://phabricator.wikimedia.org/T288035 (10hashar) [15:06:32] 10Gerrit, 10Upstream: Editing via the web interface, "Drag and drop a file here" does not accept files in "Add a new file or open an existing file" - https://phabricator.wikimedia.org/T274699 (10hashar) [15:09:35] heh, I was about to complain about Gerrit being un-SSHable and then remembered the recent comments here ^^ [15:11:15] Project mwcore-phpunit-coverage-master build #1689: 04FAILURE in 11 min: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/1689/ [15:11:15] 10Gerrit, 10Release-Engineering-Team (Doing), 10Patch-For-Review: Upgrade Gerrit from 3.3.5 to 3.3.6 - https://phabricator.wikimedia.org/T290236 (10hashar) gerrit.wikimedia.org updated: ` Name Version Api-Version Status File ------------------------------------------------... [15:13:24] dancy: that went easily with little downtime! \o/ [15:13:30] woot! [15:18:10] 10Gerrit, 10Upstream: Can't dismiss reviewer autosuggestion dropdown in new Gerrit - https://phabricator.wikimedia.org/T288193 (10hashar) [15:18:12] 10Gerrit, 10Release-Engineering-Team (Doing): Upgrade Gerrit from 3.3.5 to 3.3.6 - https://phabricator.wikimedia.org/T290236 (10hashar) 05In progressβ†’03Resolved a:03hashar Finally I have deployed the update. With the[[https://wikitech.wikimedia.org/wiki/Gerrit/Upgrade | upgrade runbook]] the downtime has... [15:18:14] 10Gerrit, 10Release-Engineering-Team (Radar), 10Regression, 10Upstream: Gerrit names misalignment in 3.3.5 - https://phabricator.wikimedia.org/T288035 (10hashar) [15:18:16] 10Gerrit, 10Upstream: Editing via the web interface, "Drag and drop a file here" does not accept files in "Add a new file or open an existing file" - https://phabricator.wikimedia.org/T274699 (10hashar) [15:19:45] 10Gerrit, 10Release-Engineering-Team (Radar), 10Regression, 10Upstream: Gerrit names misalignment in 3.3.5 - https://phabricator.wikimedia.org/T288035 (10hashar) 05Openβ†’03Resolved a:03hashar This has been fixed upstream and released in Gerrit 3.3.6 which I have deployed some minutes ago. I have confi... [15:19:46] 10Gerrit, 10Upstream: Can't dismiss reviewer autosuggestion dropdown in new Gerrit - https://phabricator.wikimedia.org/T288193 (10hashar) 05Openβ†’03Resolved This has been fixed upstream and released in Gerrit 3.3.6 which I have deployed some minutes ago. [15:21:17] 10Release-Engineering-Team (Seen), 10MediaWiki-Core-Tests, 10Quality-and-Test-Engineering-Team (QTE), 10User-zeljkofilipin: Run Selenium tests using Firefox - https://phabricator.wikimedia.org/T161697 (10zeljkofilipin) [15:23:07] 10Gerrit, 10Upstream: Editing via the web interface, "Drag and drop a file here" does not accept files in "Add a new file or open an existing file" - https://phabricator.wikimedia.org/T274699 (10hashar) 05Openβ†’03Resolved I have deployed Gerrit 3.3.6 and I have been able to drag'n drop a file using Firefox 78. [15:28:28] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Notifications, 10Browser-Tests, and 2 others: New usermessage browser test is blocking merges in Minerva skin and Echo extension - https://phabricator.wikimedia.org/T185928 (10zeljkofilipin) [15:28:42] 10Phabricator, 10Release-Engineering-Team (Seen), 10observability, 10Browser-Tests, 10User-MModell: Develop tests for phabricator search to detect regressions / search quality issues - https://phabricator.wikimedia.org/T182160 (10zeljkofilipin) [15:29:44] 10Beta-Cluster-Infrastructure, 10Browser-Tests, 10User-zeljkofilipin, 10WorkType-NewFunctionality: Make selenium users use botflags at beta-cluster by setting the bot flag (passing "bot=1" parameter) when saving edits - https://phabricator.wikimedia.org/T116027 (10zeljkofilipin) p:05Lowβ†’03Triage [15:30:19] 10Release-Engineering-Team (Seen), 10Scap, 10Browser-Tests: Running smoke tests during deployment - https://phabricator.wikimedia.org/T187733 (10zeljkofilipin) [15:31:02] 10Release-Engineering-Team (Seen), 10MediaWiki-extensions-Other, 10Browser-Tests, 10ci-test-error: EditSubpages extension fails mediawiki/core webdriver.io tests - https://phabricator.wikimedia.org/T196436 (10zeljkofilipin) [15:32:30] 10Release-Engineering-Team (Seen), 10MediaWiki-Vagrant, 10Browser-Tests: Running selenium inside Vagrant with xvfb or X11 does not work - https://phabricator.wikimedia.org/T196646 (10zeljkofilipin) [15:37:01] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10User-zeljkofilipin: Q1 Selenium framework improvements - https://phabricator.wikimedia.org/T198389 (10zeljkofilipin) [15:38:01] 10Release-Engineering-Team (Radar), 10Browser-Tests, 10Patch-For-Review: wdio-cucumber-framework fails on NodeJS due to fibers@2.x under nodejs10 - https://phabricator.wikimedia.org/T229033 (10zeljkofilipin) [15:43:04] 10Release-Engineering-Team (Seen), 10MediaWiki-Core-Tests, 10Browser-Tests, 10Developer Productivity, and 2 others: TEC12:O1:O1.4 Goal – Support running Selenium browser tests in the docker local development environment - https://phabricator.wikimedia.org/T222234 (10zeljkofilipin) 05Openβ†’03Resolved a:... [15:43:08] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10Developer Productivity: FY201819 TEC12:O1 Outcome – Local development is unified with testing and production - https://phabricator.wikimedia.org/T222237 (10zeljkofilipin) [15:43:12] 10Release-Engineering-Team, 10Release-Engineering-Team-TODO, 10Developer Productivity, 10Epic: FY201819 TEC12 Program – Developer Productivity - https://phabricator.wikimedia.org/T212449 (10zeljkofilipin) [15:43:17] 10Release-Engineering-Team (Seen), 10MediaWiki-Core-Tests, 10Browser-Tests, 10Developer Productivity, and 2 others: TEC12:O1:O1.4 Goal – Support running Selenium browser tests in the docker local development environment - https://phabricator.wikimedia.org/T222234 (10zeljkofilipin) a:05zeljkofilipinβ†’03No... [15:43:50] 10Continuous-Integration-Config, 10Release-Engineering-Team (Seen), 10Browser-Tests, 10ci-test-error: Selenium tests failing for PluggableAuth plugins - https://phabricator.wikimedia.org/T236030 (10zeljkofilipin) [15:48:24] (03CR) 10Ahmon Dancy: Replace simple git server with Gerrit (031 comment) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [15:54:00] (03CR) 10Dduvall: Replace simple git server with Gerrit (033 comments) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [15:55:34] (03PS6) 10Dduvall: Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 [15:59:41] (03CR) 10Ahmon Dancy: "Looks good. I will try it out on Linux and Mac Docker today." [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [16:06:52] (03CR) 10Dduvall: [C: 03+2] Fix build-mw-image-loop.py startup race (031 comment) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/724168 (owner: 10Ahmon Dancy) [16:07:29] (03Merged) 10jenkins-bot: Fix build-mw-image-loop.py startup race [tools/train-dev] - 10https://gerrit.wikimedia.org/r/724168 (owner: 10Ahmon Dancy) [16:07:59] is there a known issue with gerrit (cc hashar ) [16:08:14] jbond: I have upgraded it roughly an hour ago [16:08:17] what is happening? [16:08:26] its just timing out for me [16:09:29] looks like a local issue for me [16:09:49] 10Release-Engineering-Team, 10Gerrit (Gerrit 3.4): Upgrade to Gerrit 3.4 - https://phabricator.wikimedia.org/T292759 (10hashar) [16:10:13] jbond: maybe something has regressed :( [16:10:26] v [16:11:08] if fetching over https: GIT_TRACE_CURL=1 git fetch [16:11:33] check you get protocol version 2 enabled: `git config --get protocol.version` [16:12:59] no its routing [16:14:20] Yeah, I'm getting ERR_CONNECTION_TIMED_OUT on the site as well. [16:14:26] other reports about a broader problem in -operations [16:14:43] yeah, looks not gerrit specific [16:15:15] i was just like "hrm, is gitlab down or..." and tabbed over here to check. [16:15:57] network issue apparently [16:16:11] nonetheless, thx for the notification jbond! [16:23:33] yes sorry hashar was chatting in -sre, but should be fixed now [16:28:12] jbond: no worries :) [16:28:25] it could well have been an issue with gerrit itself [16:28:37] I am happy to learn the network issue got quickly resolved [16:28:40] (03CR) 10Ahmon Dancy: Replace simple git server with Gerrit (032 comments) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [16:30:35] yes i think i blamed gerrit as i saw the upgrade :) [16:30:36] (03CR) 10Ahmon Dancy: Replace simple git server with Gerrit (031 comment) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [16:46:27] (03PS7) 10Dduvall: Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 [16:46:46] (03CR) 10jerkins-bot: [V: 04-1] Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [16:47:05] (03CR) 10Dduvall: Replace simple git server with Gerrit (032 comments) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [16:47:43] (03PS1) 10Mainframe98: Archive the ParsoidBatchAPI extension [integration/config] - 10https://gerrit.wikimedia.org/r/727409 (https://phabricator.wikimedia.org/T252744) [16:49:05] (03CR) 10Ahmon Dancy: Replace simple git server with Gerrit (031 comment) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [16:49:11] (03PS8) 10Dduvall: Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 [16:49:36] (03CR) 10jerkins-bot: [V: 04-1] Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [16:50:54] (03PS9) 10Dduvall: Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 [16:51:02] (03CR) 10Dduvall: Replace simple git server with Gerrit (031 comment) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [16:52:21] (03CR) 10jerkins-bot: [V: 04-1] Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [16:53:48] (03PS10) 10Dduvall: Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 [17:02:51] (03PS2) 10Jforrester: Zuul: [mediawiki/extensions/ParsoidBatchAPI] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/727409 (https://phabricator.wikimedia.org/T252744) (owner: 10Mainframe98) [17:05:26] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/ParsoidBatchAPI] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/727409 (https://phabricator.wikimedia.org/T252744) (owner: 10Mainframe98) [17:05:42] (03PS11) 10Dduvall: Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 [17:06:59] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/ParsoidBatchAPI] Mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/727409 (https://phabricator.wikimedia.org/T252744) (owner: 10Mainframe98) [17:07:50] !log Zuul: [mediawiki/extensions/ParsoidBatchAPI] Mark as archived T252744 [17:07:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:07:55] T252744: Archive the ParsoidBatchAPI extension - https://phabricator.wikimedia.org/T252744 [17:29:04] 10Project-Admins: Create project tag for me User-Juan90264 - https://phabricator.wikimedia.org/T292687 (10Anoop) [17:29:08] 10Phabricator (Upstream), 10Upstream: Per-user projects for personal work in progress tracking - https://phabricator.wikimedia.org/T555 (10Anoop) [17:29:30] (03PS1) 10Dduvall: Use the remote origin url for Gerrit/git heuristic [tools/release] - 10https://gerrit.wikimedia.org/r/727433 [17:30:23] (03PS12) 10Dduvall: Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 [17:32:40] (03PS13) 10Dduvall: Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 [17:38:22] 10Beta-Cluster-Infrastructure, 10Browser-Tests, 10User-zeljkofilipin, 10WorkType-NewFunctionality: Make selenium users use botflags at beta-cluster by setting the bot flag (passing "bot=1" parameter) when saving edits - https://phabricator.wikimedia.org/T116027 (10zeljkofilipin) [17:57:46] 10Continuous-Integration-Config, 10Browser-Tests, 10Patch-For-Review, 10Ruby, 10User-zeljkofilipin: Drop Ruby Selenium CI jobs; we don't support them any more - https://phabricator.wikimedia.org/T220035 (10zeljkofilipin) [18:19:40] 10Release-Engineering-Team (Radar), 10Infrastructure-Foundations, 10GitLab (Infrastructure), 10Patch-For-Review, and 3 others: Puppetise gitlab-ansible playbook - https://phabricator.wikimedia.org/T283076 (10Dzahn) @Jelto As agreed we reimaged gitlab2001 together in a call. We confirmed a few things: -... [18:20:41] (03PS1) 10Kosta Harlan: phpbench: Run composer install first [integration/quibble] - 10https://gerrit.wikimedia.org/r/727443 (https://phabricator.wikimedia.org/T291549) [18:21:06] 10Release-Engineering-Team (Radar), 10Infrastructure-Foundations, 10GitLab (Infrastructure), 10Patch-For-Review, and 3 others: Puppetise gitlab-ansible playbook - https://phabricator.wikimedia.org/T283076 (10Dzahn) Also we spoke about the !log command and Arnold used that though we still need to solve priv... [18:23:06] (03PS1) 10Kosta Harlan: [DNM] Test run [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 [18:26:41] 10Release-Engineering-Team (Priority Backlog πŸ”₯), 10Release Pipeline, 10GitLab (CI & Job Runners): Proof of concept for Buildpacks as WMF image build tool in GitLab - https://phabricator.wikimedia.org/T291017 (10dduvall) [18:26:44] (03CR) 10jerkins-bot: [V: 04-1] [DNM] Test run [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 (owner: 10Kosta Harlan) [18:28:05] (03PS2) 10Kosta Harlan: [DNM] Test run [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 [18:34:08] (03CR) 10jerkins-bot: [V: 04-1] [DNM] Test run [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 (owner: 10Kosta Harlan) [18:35:33] (03PS3) 10Kosta Harlan: [DNM] Test run [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 [18:39:46] (03CR) 10jerkins-bot: [V: 04-1] [DNM] Test run [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 (owner: 10Kosta Harlan) [18:41:20] (03PS4) 10Kosta Harlan: [DNM] Test run [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 [18:44:53] (03PS1) 10Dduvall: Display timing for previous section when using TRAIN_DEV_TIMESTAMP [tools/train-dev] - 10https://gerrit.wikimedia.org/r/727461 [18:46:50] (03CR) 10jerkins-bot: [V: 04-1] [DNM] Test run [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 (owner: 10Kosta Harlan) [18:56:19] (03PS5) 10Kosta Harlan: [DNM] Test run [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 [19:06:22] (03PS6) 10Kosta Harlan: [DNM] Test run [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 [19:13:55] (03PS7) 10Kosta Harlan: [DNM] Test run [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 [19:21:04] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T281167 (10brennen) [19:29:41] (03CR) 10Kosta Harlan: "PS4 correctly clones the patch (I487357e97ce6436ed11614a182b3e0fe562ab2af) that has "phpbench" in composer.json for GrowthExperiments. Sub" [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 (owner: 10Kosta Harlan) [19:36:49] (03PS8) 10Kosta Harlan: [DNM] Test run [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 [19:37:44] kostajh: night hacking ;) [19:38:38] something like that, anyway [19:39:04] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T281167 (10Majavah) [19:39:47] (03CR) 10Hashar: [DNM] Test run (032 comments) [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 (owner: 10Kosta Harlan) [19:39:56] kostajh: got you some tricks^ [19:40:14] I missed the lack of composer install step, to be honest I haven't tested it [19:41:18] for mediawiki/core we do install the composer dev dependencies before phpbench, so there is no need to "composer install" [19:41:18] I had assumed we were already doing that via the `composer-test` step, but it was a bad assumption [19:41:45] and there is a crazy use case when dependencies are shipped by vendor.git , in this case we dont want to run composer install [19:41:49] and solely rely on vendor.git [19:42:04] for extensions as well? [19:42:15] for extensions nop afaik [19:42:24] and I don't think the composer merge plugin merges dev dependencies [19:42:58] right, I didn't think so [19:43:02] so for extensions, we might want to add the phpbench step to the one running composer test && npm test [19:43:45] hmm, I don't think so, because in some cases you will want MediaWiki installed, and I think we do composer test before that [19:44:01] yeah that kind of target what phpbench requires [19:44:16] so, any ideas what I am doing wrong in that the relevant patch from GrowthExperiments is not checked out by my hack of ci-fullrun.sh? https://gerrit.wikimedia.org/r/c/integration/quibble/+/727444 [19:45:42] I have send a comment about it [19:45:55] ZUUL_REF points to refs/zuul/master/XXXXX which is a ref internal to zuul [19:45:59] and not available in gerrit [19:46:13] so it would fallback to whatever ZUUL_BRANCH is set to [19:46:30] refs/changes/53/726653/8 is the one you want [19:47:07] clearly I should write down how to run quibble against a given change [19:47:27] ah, thx [19:47:27] (03PS9) 10Kosta Harlan: [DNM] Test run [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 [19:47:59] or even better add an option so one can `--change 726533[,8]` [19:48:12] it is written down in https://doc.wikimedia.org/quibble/#create-a-env-file-to-specify-the-variables-needed-to-replicate-a-ci-run but that references the type of ZUUL_REF which was incorrect according to what you wrote [19:48:23] let quibble to query gerrit and retrieve the change and internall set all the ZUUL_* vars [19:48:25] yeah, that would be very nice [19:51:12] * kostajh watches https://integration.wikimedia.org/ci/job/integration-quibble-fullrun/388/console [19:51:42] an alternative is to run phpbench in a dedicated job [19:52:14] in the `patch-performance` pipeline along fresnel [19:52:47] still didn't work [19:53:21] yes, possible but adds more complexity, and fresnel AFAIK was never adapted for working with extensions? [19:53:39] T220162 [19:53:40] T220162: Make it possible to run Fresnel on MediaWiki extension repos - https://phabricator.wikimedia.org/T220162 [19:54:01] 00:00:32.990 INFO:zuul.Cloner.mediawiki/extensions/GrowthExperiments:Prepared mediawiki/extensions/GrowthExperiments repo with branch master at commit 60cf9428a69e0a44b81c964972d82ddac4029cba [19:54:45] (03CR) 10Kosta Harlan: "I can't get the child patch to run, but I have verified that this works locally." [integration/quibble] - 10https://gerrit.wikimedia.org/r/727443 (https://phabricator.wikimedia.org/T291549) (owner: 10Kosta Harlan) [19:54:59] "zuul_project": "mediawiki/extensions/GrowthExperiments", "zuul_ref": "refs/changes/53/726653/8", "zuul_url": "git://contint1001.wikimedia.org"}, [19:55:20] so it tries to fetch refs/changes/63/726653/8 from git://contint1001 (the zuul merger git repo) [19:55:23] cant find the ref there [19:55:33] and fallback to ZUUL_BRANCH (=master) [19:55:46] so gotta set ZUUL_URL=https://gerrit.wikimedia.org/r [19:56:20] clearly an argument in favor of adding an option to just have to give the change # [19:56:36] (03PS10) 10Kosta Harlan: [DNM] Test run [integration/quibble] - 10https://gerrit.wikimedia.org/r/727444 [19:56:37] so in summary you need: ZUUL_PROJECT ZUUL_BRANCH ZUUL_REF and ZUUL_URL [19:56:38] ok [19:56:43] i still don't understand why it worked in PS4 [19:57:07] * hashar look at the sky and points at cosmic rays [19:58:01] i see 'em [19:58:20] on patchset 4 I guess your run happened to use the same git://contintXXXX [19:58:27] we have two [20:01:12] we can talk about it tomorrow whenever you want [20:01:33] I just have an informal meeting from 9 to 10, but free otherwise [20:01:40] yay, it works https://integration.wikimedia.org/ci/job/integration-quibble-fullrun/389/console [20:01:47] great [20:01:52] sorry for the mess :-\ [20:02:04] (03CR) 10Kosta Harlan: phpbench: Run composer install first (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/727443 (https://phabricator.wikimedia.org/T291549) (owner: 10Kosta Harlan) [20:02:15] thanks for your help! [20:02:17] i'm off [20:02:28] as for phpbench execution, I guess the idea is what to do when triggered from core vs extension [20:02:38] and when dependencies are provided by vendor vs composer merge plugin [20:02:40] ditto [20:02:48] kostajh: thank you for introducing yet another tool! :] [20:06:18] fyi - Ive reproduced bug T292777 on beta cluster by creating a gadget with a colon in its name. [20:06:18] T292777: MWException: Invalid name 'wpgadget-Special:WhatLinksHere_action_links' passed to HTMLFormField::__construct - https://phabricator.wikimedia.org/T292777 [20:06:37] This means Special:Preferences is now fatalling on beta until my fix is merged by CI. This is known/intentional. [20:09:53] 10Release-Engineering-Team (Priority Backlog πŸ”₯), 10GitLab (Auth & Access), 10User-brennen: Gitlab 2fa password validation seems bugged - https://phabricator.wikimedia.org/T292431 (10brennen) p:05Triageβ†’03High This has come up repeatedly of late, definitely should get something figured out. [20:12:00] 10Release-Engineering-Team (Radar), 10Infrastructure-Foundations, 10GitLab (Infrastructure), 10Patch-For-Review, and 3 others: Puppetise gitlab-ansible playbook - https://phabricator.wikimedia.org/T283076 (10brennen) Nice work, all! > if you have a user on gitlab-replica.wikimedia.org could you check some... [20:22:57] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T281167 (10Krinkle) [20:29:22] 10Release-Engineering-Team (Radar), 10Infrastructure-Foundations, 10GitLab (Infrastructure), 10Patch-For-Review, and 3 others: Puppetise gitlab-ansible playbook - https://phabricator.wikimedia.org/T283076 (10Dzahn) @brennen The VM has been reinstalled and puppet reinstalled gitlab but the actual gitlab dat... [20:32:02] Is it just me or is it possible that the usernames in the new gerrit version are slightly shifted? [20:33:43] Wouldn't be the first time there was a UI regression in a gerrit upgrade [20:41:23] 10Release-Engineering-Team, 10SRE: Add Ahmon and Brennen to Icinga contact list - https://phabricator.wikimedia.org/T292753 (10Dzahn) Hi, confirmed they don't have contacts yet. Will add them with standard email but without phone numbers for now. Note this means _no notifications_ from Icinga though about aler... [20:42:11] 10Release-Engineering-Team, 10SRE: Add Ahmon and Brennen to Icinga contact list - https://phabricator.wikimedia.org/T292753 (10Dzahn) I am not sure how helpful it is to have a contact group that does not actually get email when there are alerts but that is a slightly unrelated issue. [20:49:56] 10Release-Engineering-Team, 10SRE: Add Ahmon and Brennen to Icinga contact list - https://phabricator.wikimedia.org/T292753 (10Dzahn) 05Openβ†’03Resolved a:03Dzahn done. contacts added in private repo. the names are "brennen" and "dancy". Please not you will be able to login the web UI as both Brennen/br... [20:56:44] 10Release-Engineering-Team, 10SRE, 10Patch-For-Review: Add Ahmon and Brennen to Icinga contact list - https://phabricator.wikimedia.org/T292753 (10Dzahn) @hashar That being said, this ticket should not have been needed because we recently already did a more global solution: T289746 specifically see the opt... [20:57:11] 10Gerrit, 10Release-Engineering-Team (Doing), 10GitLab, 10Icinga, and 4 others: RelEng access to downtime alerts in Icinga for gitlab, gerrit, possibly other services? - https://phabricator.wikimedia.org/T289746 (10Dzahn) We also just did T292753 but that was kind of duplicate work then. [21:02:07] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T281167 (10Zabe) [21:08:15] 10MediaWiki-Releasing, 10Security, 10User-RhinosF1, 10user-sbassett: Write and send supplementary release announcement for extensions and skins with security patches (1.31.16/1.35.4/1.36.2) - https://phabricator.wikimedia.org/T285414 (10sbassett) 05Openβ†’03Resolved a:05sbassettβ†’03Mstyles And **sent*... [21:08:20] 10MediaWiki-Releasing, 10Security, 10User-RhinosF1, 10user-sbassett: Write and send supplementary release announcement for extensions and skins with security patches (1.31.16/1.35.4/1.36.2) - https://phabricator.wikimedia.org/T285414 (10sbassett) [21:12:48] 10Continuous-Integration-Config, 10Release-Engineering-Team (Seen), 10Vector: [CI] Replace npm-test job with npm-test + MediaWiki Core - https://phabricator.wikimedia.org/T252772 (10LGoto) [21:20:10] 10MediaWiki-Releasing, 10Documentation: Formally EOL REL1_31 - https://phabricator.wikimedia.org/T279858 (10Reedy) [21:20:24] 10MediaWiki-Releasing, 10Security: Release MediaWiki 1.31.16/1.35.4/1.36.2 - https://phabricator.wikimedia.org/T285404 (10Reedy) 05In progressβ†’03Resolved a:03Reedy [21:20:30] 10MediaWiki-Releasing, 10Security: Release MediaWiki 1.31.16/1.35.4/1.36.2 - https://phabricator.wikimedia.org/T285404 (10Reedy) [21:46:46] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for **CBlanton** - https://phabricator.wikimedia.org/T292679 (10thcipriani) Hi @CBlanton! These happen on Thurs at 23:00UTC. I've added you the training that happens in 2 hours, but don't feel obligated to attend that one on s... [21:47:16] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for **CBlanton** - https://phabricator.wikimedia.org/T292679 (10thcipriani) [21:47:40] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for **CBlanton** - https://phabricator.wikimedia.org/T292679 (10thcipriani) p:05Triageβ†’03Medium [21:54:43] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for **CBlanton** - https://phabricator.wikimedia.org/T292679 (10CBlanton) Thank you! Le jeu. 7 oct. 2021 Γ  14:47, thcipriani a Γ©crit : > thcipriani triaged this task as "Medium" priority.... [21:54:48] (03PS1) 10Jforrester: Dockerfiles: [node12] Re-build to upgrade node12 from 12.21.0 to 12.22.5 [integration/config] - 10https://gerrit.wikimedia.org/r/727617 [22:03:38] (03PS2) 10Jforrester: Dockerfiles: [node12] Re-build to upgrade node12 from 12.21.0 to 12.22.5 [integration/config] - 10https://gerrit.wikimedia.org/r/727617 [22:03:40] (03PS1) 10Jforrester: jjb: Provide node14 jobs (copied from the node12 ones) [integration/config] - 10https://gerrit.wikimedia.org/r/727622 [22:03:42] (03PS1) 10Jforrester: Zuul: [design/codex] Provide initial CI, node14-browser suite [integration/config] - 10https://gerrit.wikimedia.org/r/727623 [22:04:36] (03PS2) 10Jforrester: jjb: Provide node14 jobs (copied from the node12 ones) [integration/config] - 10https://gerrit.wikimedia.org/r/727622 [22:04:38] (03PS2) 10Jforrester: Zuul: [design/codex] Provide initial CI, node14-browser suite [integration/config] - 10https://gerrit.wikimedia.org/r/727623 [22:04:40] (03PS3) 10Jforrester: Dockerfiles: [node12] Re-build to upgrade node12 from 12.21.0 to 12.22.5 [integration/config] - 10https://gerrit.wikimedia.org/r/727617 [22:08:13] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/727622 (owner: 10Jforrester) [22:08:19] (03CR) 10Jforrester: [C: 03+2] Zuul: [design/codex] Provide initial CI, node14-browser suite [integration/config] - 10https://gerrit.wikimedia.org/r/727623 (owner: 10Jforrester) [22:11:19] (03Merged) 10jenkins-bot: jjb: Provide node14 jobs (copied from the node12 ones) [integration/config] - 10https://gerrit.wikimedia.org/r/727622 (owner: 10Jforrester) [22:11:21] (03Merged) 10jenkins-bot: Zuul: [design/codex] Provide initial CI, node14-browser suite [integration/config] - 10https://gerrit.wikimedia.org/r/727623 (owner: 10Jforrester) [22:12:38] !log Zuul: [design/codex] Provide initial CI, node14-browser suite [22:12:40] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:27:34] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [400.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [22:31:10] (03CR) 10Ahmon Dancy: Replace simple git server with Gerrit (032 comments) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [22:32:17] (03PS2) 10Dduvall: Display timing for previous section when using TRAIN_DEV_TIMESTAMP [tools/train-dev] - 10https://gerrit.wikimedia.org/r/727461 [22:35:06] (03PS14) 10Dduvall: Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 [22:36:37] (03CR) 10Dduvall: Replace simple git server with Gerrit (032 comments) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [22:40:16] (03CR) 10Ahmon Dancy: [C: 03+2] "Worked for me." [tools/train-dev] - 10https://gerrit.wikimedia.org/r/727461 (owner: 10Dduvall) [22:51:36] (03Merged) 10jenkins-bot: Display timing for previous section when using TRAIN_DEV_TIMESTAMP [tools/train-dev] - 10https://gerrit.wikimedia.org/r/727461 (owner: 10Dduvall) [23:06:28] (03CR) 10Ahmon Dancy: "Please update exp/files/deploy/scripts/with-real-gerrit w/ the new url." [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [23:12:55] (03PS15) 10Dduvall: Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 [23:13:56] (03CR) 10Dduvall: Replace simple git server with Gerrit (031 comment) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [23:20:47] (03PS16) 10Dduvall: Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 [23:25:50] (03CR) 10Dduvall: "Profiling from `TRAIN_DEV_TIMESTAMPS=yes ./train-dev build`" [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [23:49:47] fyi https://gerrit.wikimedia.org/r/c/operations/cookbooks/+/727605 is towards improving the scap release process on the SRE side