[00:56:12] 10Continuous-Integration-Infrastructure, 10SRE, 10serviceops-collab, 10Patch-For-Review: contint2002 service implementation tracking - https://phabricator.wikimedia.org/T324659 (10Dzahn) > contint.wikimedia.org minor correction, it's https://integration.wikimedia.org >Daniel proposed to reiimage it... [01:10:00] (03PS1) 10Dzahn: update IP in gerrit_ssh_host_key to new gerrit IP [integration/config] - 10https://gerrit.wikimedia.org/r/928183 [01:10:16] (03CR) 10Dzahn: "https://gerrit.wikimedia.org/r/c/integration/config/+/928183" [integration/config] - 10https://gerrit.wikimedia.org/r/928102 (https://phabricator.wikimedia.org/T278203) (owner: 10Hashar) [02:58:33] 10Release-Engineering-Team (They Live 🕶️🧟), 10Patch-For-Review, 10Release, 10Train Deployments: 1.41.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T337526 (10TheresNoTime) [05:06:20] 10Phabricator, 10Release-Engineering-Team, 10User-brennen: Temporarily replace the Phabricator logo for Pride Month - https://phabricator.wikimedia.org/T337964 (10brennen) > I'm wondering if this is an example of unintentional logo misuse? The WMF brand guidelines discourage modifying the Foundation logo's b... [05:09:01] 10GitLab (Account Approval), 10Release-Engineering-Team: Requesting GitLab account activation for dvrandecic - https://phabricator.wikimedia.org/T338389 (10brennen) 05Open→03Resolved [05:09:17] 10GitLab (Account Approval), 10Release-Engineering-Team: Requesting GitLab account activation for dvrandecic - https://phabricator.wikimedia.org/T338389 (10brennen) [07:36:36] (03CR) 10Hashar: "(this int" [integration/config] - 10https://gerrit.wikimedia.org/r/928183 (owner: 10Dzahn) [07:41:03] 10Continuous-Integration-Config, 10Data Pipelines, 10Data-Engineering, 10Event-Platform Value Stream (Sprint 14 B), and 2 others: Wikimedia-event-utilities jenkins build failure - https://phabricator.wikimedia.org/T338343 (10hashar) The root cause is those java8 CI images are still based on Debian Stretch... [07:55:50] 10Beta-Cluster-Infrastructure, 10VisualEditor, 10VisualEditor-VisualDiffs: Visual diff of page creation fails with “Invalid response from server” - https://phabricator.wikimedia.org/T338388 (10Tacsipacsi) Sorry, I was so certain that it’s because of the new page creation that I didn’t test further. Beta Incu... [08:11:30] !log Manually building php8.1 dev images to update php 8.1 to the latest provided by sury.org. Change is https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/33 for T338208 [08:11:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:11:33] T338208: xdebug causes segmentation fault in PHP when hitting a breakpoint in a phpunit test - https://phabricator.wikimedia.org/T338208 [08:57:41] 10Project-Admins: Creation of a new project "All-and-every-Wikibooks" - https://phabricator.wikimedia.org/T330600 (10Aklapper) @JackPotte: Hmm, any take on the last comments? [09:02:31] 10Phabricator, 10User-DannyS712: Form 4 (create project) should set sprint to 0 by default - https://phabricator.wikimedia.org/T247687 (10Aklapper) 05Open→03Resolved Neither recently created project tags like https://phabricator.wikimedia.org/project/manage/6597/ nor the examples here from three years ago... [09:37:43] 10Phabricator (Upstream), 10Documentation, 10Upstream: Document "Contact Numbers" (and disable them if not yet available for use on Wikimedia Phabricator) - https://phabricator.wikimedia.org/T240109 (10Aklapper) Having looked into this, I think an upstream ticket to disable exposing SMS support functionality... [09:38:09] 10Phabricator (Upstream), 10Upstream: Disable "Contact Numbers" settings panel (and SMS support functionality that we do not use) - https://phabricator.wikimedia.org/T240109 (10Aklapper) p:05Lowest→03Low [10:29:10] 10Project-Admins: Create project tag for Tech Docs Team - https://phabricator.wikimedia.org/T338387 (10Aklapper) 05Open→03Resolved a:03Aklapper Requested public project #tech-docs-team has been created: https://phabricator.wikimedia.org/project/view/6599/ (In case you need to edit the project or project w... [11:08:29] I have a question about the jenkins instance that runs on releases hosts. How come we don't use the main CI instance and just have it upload to the releases hosts instead? [12:08:30] 10Phabricator: Upgrade license of Phabricator TO CC BY-SA 4.0 - https://phabricator.wikimedia.org/T338440 (10Bugreporter) [12:37:33] 10Release-Engineering-Team, 10Data-Engineering, 10Event-Platform Value Stream (Sprint 14 B), 10Patch-For-Review: eventutillities-python should publish python doc to doc.wikimedia.org - https://phabricator.wikimedia.org/T337475 (10CodeReviewBot) gmodena merged https://gitlab.wikimedia.org/repos/data-enginee... [12:57:21] 10Beta-Cluster-Infrastructure, 10VisualEditor, 10VisualEditor-VisualDiffs: Visual diff of page creation fails with “Invalid response from server” - https://phabricator.wikimedia.org/T338388 (10Tacsipacsi) Although having oldid 45 after more than a year shows how much it’s used for testing… [13:10:10] 10Phabricator: Add a "Thanks" token - https://phabricator.wikimedia.org/T338133 (10kostajh) >>! In T338133#8910190, @Aklapper wrote: >> U+1F64F Person with folded hands - can indicate sorrow or regret, can also indicate pleading, praying, bowing, or thanking > > So I'm not convinced about this one. Anyone has a... [13:15:41] 10Phabricator (Upstream), 10Upstream: Add option to display task history in reverse-chronological order - https://phabricator.wikimedia.org/T100534 (10Aklapper) 05Open→03Declined [14:16:29] 10GitLab (Project Migration), 10Release-Engineering-Team (They Live 🕶️🧟), 10User-brennen: Define a permissions model for the /repos/mediawiki/ namespace on GitLab - https://phabricator.wikimedia.org/T336807 (10hashar) I have noticed `extensions-all` and `skins-all`, I suspect those are place holders to poten... [14:18:20] (03PS1) 10Majavah: zuul: Add some dependencies for CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/928559 [14:20:18] (03CR) 10Majavah: [C: 03+2] zuul: Add some dependencies for CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/928559 (owner: 10Majavah) [14:21:35] (03Merged) 10jenkins-bot: zuul: Add some dependencies for CentralAuth [integration/config] - 10https://gerrit.wikimedia.org/r/928559 (owner: 10Majavah) [14:21:58] 10GitLab (Infrastructure), 10Release-Engineering-Team, 10serviceops-collab: Upgrade GitLab to major version 16 - https://phabricator.wikimedia.org/T338460 (10Jelto) [14:22:08] !log reloading zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/928559/ [14:22:09] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:36:07] 10Deployments, 10Release-Engineering-Team, 10Patch-For-Review, 10Sustainability (Incident Followup): Scap train-presync failed to prepare 1.41.0-wmf.12 - https://phabricator.wikimedia.org/T338205 (10hashar) a:03hashar [14:36:33] 10Phabricator, 10Code-Review-Workgroup, 10Wikimedia-Phabricator-Extensions, 10Developer Productivity, 10Patch-For-Review: Indicate WIP status of patches on Phabricator patch list - https://phabricator.wikimedia.org/T325239 (10CodeReviewBot) aklapper opened https://gitlab.wikimedia.org/repos/phabricator/e... [14:36:44] 10Phabricator, 10Code-Review-Workgroup, 10Wikimedia-Phabricator-Extensions, 10Developer Productivity, 10Patch-For-Review: Indicate WIP status of patches on Phabricator patch list - https://phabricator.wikimedia.org/T325239 (10CodeReviewBot) [14:41:53] 10Phabricator, 10Release-Engineering-Team (They Live 🕶️🧟), 10Code-Review-Workgroup, 10Wikimedia-Phabricator-Extensions, and 2 others: Indicate WIP status of patches on Phabricator patch list - https://phabricator.wikimedia.org/T325239 (10Aklapper) p:05Triage→03Low a:03Aklapper [14:57:01] 10Phabricator, 10Security-Team, 10WMF-Legal, 10SecTeam-Processed, 10Security: WMF's Phabricator is duplicated on a protentionally malicious/phishing site - https://phabricator.wikimedia.org/T327589 (10Aklapper) True; I guess not [14:57:47] 10Phabricator, 10Security-Team, 10WMF-Legal, 10SecTeam-Processed, 10Security: WMF's Phabricator is duplicated on a protentionally malicious/phishing site - https://phabricator.wikimedia.org/T327589 (10Aklapper) [15:01:09] 10Gerrit, 10Release-Engineering-Team, 10serviceops-collab: Rename gerrit2 unix user to gerrit and assign a fixed uid - https://phabricator.wikimedia.org/T338470 (10hashar) [15:02:28] 10Phabricator: Upgrade license of Phabricator TO CC BY-SA 4.0 - https://phabricator.wikimedia.org/T338440 (10Aklapper) @Bugreporter: Phabricator license is Apache License 2.0. Do you mean the page footer instead? Please elaborate and provide steps to reproduce when creating tasks, in order to avoid misunderstan... [15:02:40] 10Phabricator: Upgrade license of Phabricator To CC BY-SA 4.0 - https://phabricator.wikimedia.org/T338440 (10Aklapper) [15:10:31] 10Gerrit, 10Release-Engineering-Team, 10serviceops-collab: Rename gerrit2 unix user to gerrit and assign a fixed uid - https://phabricator.wikimedia.org/T338470 (10hashar) a:03hashar [15:13:39] 10Deployments, 10Scap, 10SRE: setup automatic deletion of old l10nupdate - https://phabricator.wikimedia.org/T130317 (10taavi) [15:14:00] 10Scap: Completely port l10nupdate to scap - https://phabricator.wikimedia.org/T133913 (10taavi) 05Open→03Declined Per https://gerrit.wikimedia.org/r/c/operations/puppet/+/896318. [15:14:24] 10Deployments, 10Infrastructure-Foundations, 10SRE, 10serviceops-radar: Make l10nupdate user a system user - https://phabricator.wikimedia.org/T120585 (10taavi) 05Open→03Declined Per https://gerrit.wikimedia.org/r/c/operations/puppet/+/896318. [15:14:52] 10Deployments: [l10n] LocalisationUpdate can incorrectly report/log success - https://phabricator.wikimedia.org/T49301 (10taavi) 05Open→03Declined Per https://gerrit.wikimedia.org/r/c/operations/puppet/+/896318. [15:14:54] 10Deployments, 10observability: Setup alerting for l10nupdate - https://phabricator.wikimedia.org/T107515 (10taavi) [15:14:58] 10Deployments, 10observability: Setup alerting for l10nupdate - https://phabricator.wikimedia.org/T107515 (10taavi) 05Open→03Declined Per https://gerrit.wikimedia.org/r/c/operations/puppet/+/896318. [15:15:00] 10Deployments, 10User-greg: Remove (?) l10nupdate from !log'ing - https://phabricator.wikimedia.org/T107480 (10taavi) [15:36:30] 10Phabricator: "In progress" tasks should appear above other tasks in workboards sorted by status - https://phabricator.wikimedia.org/T308307 (10Aklapper) I'd generally agree with this proposal for workboards. The order of statūs is defined by the order in JSON at https://phabricator.wikimedia.org/config/edit/ma... [15:38:02] (03PS1) 10Ilias Sarantopoulos: fix: rename bloom to llm [integration/config] - 10https://gerrit.wikimedia.org/r/928592 [15:38:58] (03PS2) 10Ilias Sarantopoulos: fix: rename bloom pipelines to llm [integration/config] - 10https://gerrit.wikimedia.org/r/928592 [15:39:34] (03CR) 10Hashar: [C: 03+2] "INFO:jenkins_jobs.builder:Number of jobs generated: 4" [integration/config] - 10https://gerrit.wikimedia.org/r/928592 (owner: 10Ilias Sarantopoulos) [15:42:23] (03Merged) 10jenkins-bot: fix: rename bloom pipelines to llm [integration/config] - 10https://gerrit.wikimedia.org/r/928592 (owner: 10Ilias Sarantopoulos) [15:43:31] (03CR) 10Hashar: [C: 03+2] "I have reloaded CI :]" [integration/config] - 10https://gerrit.wikimedia.org/r/928592 (owner: 10Ilias Sarantopoulos) [16:09:04] 10Phabricator, 10Security-Team, 10WMF-Legal, 10SecTeam-Processed, 10Security: WMF's Phabricator is duplicated on a protentionally malicious/phishing site - https://phabricator.wikimedia.org/T327589 (10KHurd-WMF) Nothing major on our end. Posting shouldn't lead to a problem. [16:10:23] 10GitLab (Integrations), 10Phabricator, 10Release-Engineering-Team: GitLab repos are not correctly mirrored to Diffusion - https://phabricator.wikimedia.org/T324151 (10Jdforrester-WMF) Do we want to sync code into Diffusion? It's a massive point of confusion for no added value. I'd rather this task was "Ensu... [16:34:35] 10GitLab (Integrations), 10Phabricator, 10Release-Engineering-Team: GitLab repos are not correctly mirrored to Diffusion - https://phabricator.wikimedia.org/T324151 (10bd808) > It's a massive point of confusion for no added value. I find it useful for the ability to embed nice links like {03cff13258dbb76740e... [16:40:00] 10Phabricator: Some deadline stamps change color, while others do not, despite having the same deadline - https://phabricator.wikimedia.org/T311168 (10Aklapper) 05Open→03Declined Custom code in rPHEX's `src/customfields/DeadlineEditEngineSubtype.php` to colorize is not based on calendar days in some timezone... [16:43:10] 10Project-Admins: Create project tag for Tech Docs Team - https://phabricator.wikimedia.org/T338387 (10apaskulin) Thanks, @Aklapper! [16:47:45] 10Phabricator, 10Release-Engineering-Team (They Live 🕶️🧟), 10Diffusion-Repository-Administrators: Git hashes for SyntaxHighlight_GeSHi not recognised in Phabricator (due to Pygments fork repo import) - https://phabricator.wikimedia.org/T119588 (10Aklapper) 05Open→03Resolved a:03Aklapper Resolved by del... [17:51:02] 10GitLab (Integrations), 10Phabricator, 10Release-Engineering-Team: GitLab repos are not correctly mirrored to Diffusion - https://phabricator.wikimedia.org/T324151 (10Dzahn) >>! In T324151#8914724, @Jdforrester-WMF wrote: > Do we want to sync code into Diffusion? Definitely not. Diffusion is already suppo... [17:53:07] 10GitLab (Integrations), 10Phabricator, 10Release-Engineering-Team: GitLab repos are not correctly mirrored to Diffusion - https://phabricator.wikimedia.org/T324151 (10Dzahn) also see T191182 and duplicate of this T262219 (which I would say should be declined along with everything else related to Diffusion) [18:05:41] Project beta-scap-sync-world build #106732: 04FAILURE in 24 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/106732/ [18:15:36] Project beta-scap-sync-world build #106733: 04STILL FAILING in 22 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/106733/ [18:20:03] Project beta-update-databases-eqiad build #67890: 04FAILURE in 2.7 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/67890/ [18:20:55] 10GitLab (Integrations), 10Phabricator, 10Release-Engineering-Team: GitLab repos are not correctly mirrored to Diffusion - https://phabricator.wikimedia.org/T324151 (10Aklapper) >>! In T324151#8915197, @Dzahn wrote: > Diffusion is already supposed to be decom'ed. That's Differential (code review), not Diff... [18:21:01] 10GitLab (Integrations), 10Phabricator, 10Release-Engineering-Team: GitLab repos are not correctly mirrored to Diffusion - https://phabricator.wikimedia.org/T324151 (10bd808) >>! In T324151#8915197, @Dzahn wrote: >>>! In T324151#8914724, @Jdforrester-WMF wrote: >> Do we want to sync code into Diffusion? >... [18:25:31] Project beta-scap-sync-world build #106734: 04STILL FAILING in 22 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/106734/ [18:32:11] Project beta-scap-sync-world build #106735: 04STILL FAILING in 22 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/106735/ [18:35:38] Project beta-scap-sync-world build #106736: 04STILL FAILING in 22 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/106736/ [18:39:07] HMM [18:42:01] Project beta-scap-sync-world build #106737: 04STILL FAILING in 22 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/106737/ [18:42:21] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: beta-scap-sync-world failure - https://phabricator.wikimedia.org/T338495 (10TheresNoTime) [18:42:52] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: beta-scap-sync-world failure — SSL peer certificate or SSH remote key was not OK - https://phabricator.wikimedia.org/T338495 (10TheresNoTime) [18:45:33] Project beta-scap-sync-world build #106738: 04STILL FAILING in 21 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/106738/ [18:49:55] TheresNoTime: isbetabroken! [18:53:01] https://www.isbetabroken.com :D [18:53:50] TheresNoTime: powered by you :) [18:55:41] Project beta-scap-sync-world build #106739: 04STILL FAILING in 23 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/106739/ [18:55:51] its another T296125 [18:55:52] T296125: Fatal error: Uncaught ConfigException: Failed to load configuration from etcd - https://phabricator.wikimedia.org/T296125 [18:56:26] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: beta-scap-sync-world failure — SSL peer certificate or SSH remote key was not OK - https://phabricator.wikimedia.org/T338495 (10TheresNoTime) Prev: {T296125} [19:03:28] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: beta-scap-sync-world failure — SSL peer certificate or SSH remote key was not OK - https://phabricator.wikimedia.org/T338495 (10TheresNoTime) ` samtar@deployment-mediawiki12:~$ sudo puppet agent -tv Info: Using configured environment 'prod... [19:05:36] Project beta-scap-sync-world build #106740: 04STILL FAILING in 22 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/106740/ [19:08:27] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: beta-scap-sync-world failure — SSL peer certificate or SSH remote key was not OK - https://phabricator.wikimedia.org/T338495 (10TheresNoTime) All I can find that looks sorta-related is {T286375} [19:09:05] (I have no idea, outside of my scope of knowledge — cc taavi as it looks like you fixed something related to T338495 a while back..) [19:09:05] T338495: beta-scap-sync-world failure — SSL peer certificate or SSH remote key was not OK - https://phabricator.wikimedia.org/T338495 [19:15:36] Project beta-scap-sync-world build #106741: 04STILL FAILING in 23 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/106741/ [19:20:03] Project beta-update-databases-eqiad build #67891: 04STILL FAILING in 2.7 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/67891/ [19:25:32] Project beta-scap-sync-world build #106742: 04STILL FAILING in 22 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/106742/ [19:35:36] Project beta-scap-sync-world build #106743: 04STILL FAILING in 22 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/106743/ [19:45:42] Project beta-scap-sync-world build #106744: 04STILL FAILING in 22 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/106744/ [19:49:01] ^ some ssl certificate expired for etcd [19:49:17] seems on beta the host is _etcd._tcp.svc.deployment-prep.eqiad1.wikimedia.cloud [19:52:13] deployment-etcd02.deployment-prep.eqiad1.wikimedia.cloud. [19:52:33] but beside that I am not investigating it is 10pm here [19:55:39] Project beta-scap-sync-world build #106745: 04STILL FAILING in 23 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/106745/ [19:58:31] "failed POST to https://pki-intermediate.pki.eqiad1.wikimedia.cloud:443/api/v1/cfssl/authsign: Post \"https://pki-intermediate.pki.eqiad1.wikimedia.cloud:443/api/v1/cfssl/authsign\": remote error: tls: expired certificate"} [19:58:40] so looks like it depend son another server [19:58:42] out of beta cluster [19:59:48] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: beta-scap-sync-world failure — SSL peer certificate or SSH remote key was not OK - https://phabricator.wikimedia.org/T338495 (10dancy) I have re-disabled https://integration.wikimedia.org/ci/job/beta-scap-sync-world/ [20:00:14] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: beta-scap-sync-world failure — SSL peer certificate or SSH remote key was not OK - https://phabricator.wikimedia.org/T338495 (10thcipriani) >>! In T338495#8915394, @TheresNoTime wrote: > All I can find that looks sorta-related is {T286375}... [20:04:58] TheresNoTime, thcipriani, dancy: not sure if helpful, but https://wikitech.wikimedia.org/wiki/PKI/Clients is the page I could find on cfssl things [20:10:59] Thanks! [20:11:02] also found https://wikitech.wikimedia.org/wiki/PKI/Cloud [20:20:04] Project beta-update-databases-eqiad build #67892: 04STILL FAILING in 3.7 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/67892/ [20:54:25] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: beta-scap-sync-world failure — SSL peer certificate or SSH remote key was not OK - https://phabricator.wikimedia.org/T338495 (10dancy) I wonder what the time is on host `pki-intermediate.pki.eqiad1.wikimedia.cloud`. [21:09:24] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: beta-scap-sync-world failure — SSL peer certificate or SSH remote key was not OK - https://phabricator.wikimedia.org/T338495 (10dancy) Adding another datapoint: ` root@deployment-etcd02:~# hostname -f deployment-etcd02.deployment-prep.eqia... [21:20:05] Project beta-update-databases-eqiad build #67893: 04STILL FAILING in 4 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/67893/ [21:39:20] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: beta-scap-sync-world failure — SSL peer certificate or SSH remote key was not OK - https://phabricator.wikimedia.org/T338495 (10bd808) >>! In T338495#8915911, @dancy wrote: > The connection fails with a code indicating that the client cert... [21:52:40] wikifunctions.beta is now not loading, same error as T338495 [21:52:41] T338495: beta-scap-sync-world failure — SSL peer certificate or SSH remote key was not OK - https://phabricator.wikimedia.org/T338495 [21:53:24] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure: beta-scap-sync-world failure — SSL peer certificate or SSH remote key was not OK - https://phabricator.wikimedia.org/T338495 (10TheresNoTime) https://wikifunctions.beta.wmflabs.org is now failing with the same error [22:20:05] Project beta-update-databases-eqiad build #67894: 04STILL FAILING in 3.9 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/67894/ [22:28:22] (03PS2) 10Dzahn: remove IPs from gerrit_ssh_host_key file [integration/config] - 10https://gerrit.wikimedia.org/r/928183 (https://phabricator.wikimedia.org/T336427) [22:28:49] (03CR) 10Dzahn: remove IPs from gerrit_ssh_host_key file (032 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/928183 (https://phabricator.wikimedia.org/T336427) (owner: 10Dzahn) [22:50:45] 10Release-Engineering-Team (They Live 🕶️🧟), 10serviceops-collab: upgrade gerrit servers to bullseye - https://phabricator.wikimedia.org/T334521 (10Dzahn) 05In progress→03Resolved Declaring this resolved because: - both production servers with the gerrit role are on bullseye - the former gerrit prod server... [23:15:29] 10Phabricator, 10serviceops-collab, 10Developer-Advocacy (Apr-Jun 2023), 10Patch-For-Review: Automate SQL queries for annual end-of-year Phabricator statistics/metrics on wikitech-l@ - https://phabricator.wikimedia.org/T337388 (10Dzahn) [23:20:04] Project beta-update-databases-eqiad build #67895: 04STILL FAILING in 3.7 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/67895/