[03:33:47] 10GitLab (Integrations), 10Release-Engineering-Team (Priority Backlog 📥), 10Wikibugs: Connect WikiBugs IRC bot to Wikimedia GitLab - https://phabricator.wikimedia.org/T288381#9712181 (10bd808) Getting [[https://docs.gitlab.com/ee/user/project/integrations/webhook_events.html#comment-events|comment]], [[https... [06:56:52] 10Release-Engineering-Team (Now this 🫠), 06cloud-services-team, 06collaboration-services, 10VPS-project-devtools, and 2 others: Update devtools project puppetmaster - https://phabricator.wikimedia.org/T360470#9712252 (10Jelto) Puppet runs on some machines which use the new puppetmaster in devtools fail, he... [07:47:18] 10Gerrit (Gerrit 3.9), 07Regression, 07Upstream: Gerrit: All WIP changes are marked as "Merge Conflict" in the dashboard, even when they are not - https://phabricator.wikimedia.org/T328843#9712330 (10hashar) The fix is in v3.8.5, v3.9.4 and will be in v3.10.0 [08:18:43] (03PS1) 10Hashar: Merge tag 'v3.8.5' into wmf/stable-3.8 [software/gerrit] (wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1019674 (https://phabricator.wikimedia.org/T354886) [08:19:29] (03CR) 10CI reject: [V:04-1] Merge tag 'v3.8.5' into wmf/stable-3.8 [software/gerrit] (wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1019674 (https://phabricator.wikimedia.org/T354886) (owner: 10Hashar) [08:58:15] (03CR) 10Hashar: "recheck" [software/gerrit] (wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1019674 (https://phabricator.wikimedia.org/T354886) (owner: 10Hashar) [09:12:27] (03CR) 10Hashar: [C:03+2] Merge tag 'v3.8.5' into wmf/stable-3.8 [software/gerrit] (wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1019674 (https://phabricator.wikimedia.org/T354886) (owner: 10Hashar) [09:16:17] (03CR) 10Lucas Werkmeister (WMDE): [WIP] jjb: Specify PHP flavour in Wikibase special quibble jobs, add 81/83 (033 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/1019298 (https://phabricator.wikimedia.org/T362412) (owner: 10Jforrester) [09:18:56] (03Merged) 10jenkins-bot: Merge tag 'v3.8.5' into wmf/stable-3.8 [software/gerrit] (wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1019674 (https://phabricator.wikimedia.org/T354886) (owner: 10Hashar) [09:21:23] (03PS1) 10Hashar: Gerrit 3.8.5 and rebuild plugins [software/gerrit] (deploy/wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1019685 (https://phabricator.wikimedia.org/T354886) [09:22:27] 06Release-Engineering-Team, 06Data-Engineering, 06Discovery-Search, 06Java-Scala-Standardization, and 2 others: 14Adapt gitlab pipelines for the new wmf-jvm-parent-pom - 14https://phabricator.wikimedia.org/T358841#9712584 (10Gehel) 05Open→03Declined [09:23:58] (03CR) 10Hashar: [C:03+2] Gerrit 3.8.5 and rebuild plugins [software/gerrit] (deploy/wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1019685 (https://phabricator.wikimedia.org/T354886) (owner: 10Hashar) [09:25:12] (03Merged) 10jenkins-bot: Gerrit 3.8.5 and rebuild plugins [software/gerrit] (deploy/wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1019685 (https://phabricator.wikimedia.org/T354886) (owner: 10Hashar) [09:33:29] 10Phabricator, 06Infrastructure-Foundations, 10Mail: 14Interacting with Phabricator via mail is broken - 14https://phabricator.wikimedia.org/T92724#9712606 (10Mvolz) 14I'm having this issue again. Looks like a different error though: ` A message that you sent could not be delivered to one or more of... [09:55:30] PROBLEM - gerrit process on gerrit2002 is CRITICAL: PROCS CRITICAL: 0 processes with regex args ^/usr/lib/jvm/java-11-openjdk-amd64/bin/java .*-jar /var/lib/gerrit2/review_site/bin/gerrit.war daemon -d /var/lib/gerrit2/review_site https://wikitech.wikimedia.org/wiki/Gerrit [09:56:30] RECOVERY - gerrit process on gerrit2002 is OK: PROCS OK: 1 process with regex args ^/usr/lib/jvm/java-11-openjdk-amd64/bin/java .*-jar /var/lib/gerrit2/review_site/bin/gerrit.war daemon -d /var/lib/gerrit2/review_site https://wikitech.wikimedia.org/wiki/Gerrit [10:04:54] 10Phabricator, 06Infrastructure-Foundations, 10Mail: 14Interacting with Phabricator via mail is broken - 14https://phabricator.wikimedia.org/T92724#9712695 (10Aklapper) 14@Mvolz: This ticket got closed nine years ago. You are probably looking for T356077 instead. [10:06:41] (03PS1) 10Hashar: Merge branch 'deploy/wmf/stable-3.7' into deploy/wmf/stable-3.8 [software/gerrit] (deploy/wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1019709 [10:08:46] (03CR) 10Hashar: [C:03+2] Merge branch 'deploy/wmf/stable-3.7' into deploy/wmf/stable-3.8 [software/gerrit] (deploy/wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1019709 (owner: 10Hashar) [10:09:24] (03Merged) 10jenkins-bot: Merge branch 'deploy/wmf/stable-3.7' into deploy/wmf/stable-3.8 [software/gerrit] (deploy/wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1019709 (owner: 10Hashar) [10:09:38] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518 (10Clement_Goubert) 03NEW [10:10:01] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9712711 (10Clement_Goubert) 05Open→03In progress p:05Triage→03Unbreak! [10:23:34] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9712726 (10MoritzMuehlenhoff) [10:26:02] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9712728 (10Clement_Goubert) [10:33:01] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9712750 (10jijiki) [10:33:04] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9712748 (10hashar) > integration/config/operations-puppet (shellcheck) The image is based on Buster since the Puppet master in production use /... [10:33:59] 10Gerrit (Gerrit 3.8), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: 14Upgrade to Gerrit 3.8 - 14https://phabricator.wikimedia.org/T354886#9712760 (10hashar) 05Open→03Resolved [10:40:59] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9712771 (10hnowlan) >>! In T362518#9712748, @hashar wrote: >> mediawiki/vagrant (php7.4-wikidiff2) > > #mediawiki-vagrant is based on Buster, t... [10:45:15] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9712773 (10MoritzMuehlenhoff) >>! In T362518#9712748, @hashar wrote: >> integration/config/operations-puppet (shellcheck) > > The image is base... [10:48:45] 10Phabricator, 06Moderator-Tools-Team: Herald rule for Moderator Tools maintained extension - https://phabricator.wikimedia.org/T362057#9712779 (10Aklapper) Hi, what does that mean regarding the Kanban subproject at https://phabricator.wikimedia.org/project/view/5880/ ? You very likely only want the #Moderato... [10:52:16] 10Gerrit (Gerrit 3.8), 07Regression, 07Upstream: 14Gerrit: All WIP changes are marked as "Merge Conflict" in the dashboard, even when they are not - 14https://phabricator.wikimedia.org/T328843#9712784 (10Winston_Sung) 05Stalled→03Resolved 14Thanks. Confirmed already fixed in Gerrit v3.8.5: https:/... [10:53:04] 10Phabricator, 06Moderator-Tools-Team: 14Herald rule for Moderator Tools maintained extension - 14https://phabricator.wikimedia.org/T362057#9712788 (10Aklapper) 05Open→03Resolved a:03Aklapper 14I created H437: * When **all** of these conditions are met: ** Project tags include **any** of #Library-Ca... [10:55:49] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9712795 (10MoritzMuehlenhoff) [10:57:16] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9712798 (10JMeybohm) [11:17:31] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9712847 (10Clement_Goubert) [11:20:18] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9712849 (10Clement_Goubert) [12:22:09] 10Diffusion, 10Gerrit, 10Phabricator: Diffusion fails to push into Gerrit - https://phabricator.wikimedia.org/T362524 (10Aklapper) 03NEW [12:51:30] 06Release-Engineering-Team, 06collaboration-services, 06Java-Scala-Standardization, 10Data-Platform-SRE (2024.04.15 - 2024.05.05), 07Security: Investigate replacing Archiva with Gitlab repositories - https://phabricator.wikimedia.org/T358612#9713130 (10Gehel) [12:55:08] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 06collaboration-services: 14Define access to external resources for GitLab CI Runners - 14https://phabricator.wikimedia.org/T320730#9713150 (10Jelto) 05Open→03Resolved 14I think we are mostly settled about which runners have which kind of acce... [13:21:47] 06Release-Engineering-Team, 10Scap, 06Data-Platform-SRE, 10Wikidata, and 2 others: scap deploy should not repool a wdqs node that is depooled - https://phabricator.wikimedia.org/T361246#9713312 (10Gehel) [13:22:28] 06Release-Engineering-Team, 10Scap, 06Data-Platform-SRE, 10Wikidata, 10Wikidata-Query-Service: scap deploy should not repool a wdqs node that is depooled - https://phabricator.wikimedia.org/T361246#9713314 (10Gehel) [13:24:17] (03CR) 10Jforrester: [WIP] jjb: Specify PHP flavour in Wikibase special quibble jobs, add 81/83 (033 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/1019298 (https://phabricator.wikimedia.org/T362412) (owner: 10Jforrester) [13:31:00] hashar: If you're around, some jjb advice on ^^^ 1019298 would be helpful. [13:44:18] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Decrease number of open Phab tickets with assignee field set for more than two years (aka cookie licking) (Q2/2024 edition) - https://phabricator.wikimedia.org/T351691#9713390 (10Aklapper) [13:50:59] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9713418 (10JMeybohm) [13:51:16] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9713416 (10JMeybohm) Production images rebuild is done: ` Apr 15 13:46:55 build2001 update-production-images[4127624]: Successfully published i... [13:51:47] 10Release-Engineering-Team (Seen), 06Content-Transform-Team-WIP, 10MW-on-K8s, 06serviceops, and 4 others: A lot of `[info] Wikitext for this page has duplicate ids:` in logstash for mw-parsoid. Possibly related to PageBundle - https://phabricator.wikimedia.org/T358588#9713422 (10MSantos) [13:56:50] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Decrease number of open Phab tickets with assignee field set for more than two years (aka cookie licking) (Q2/2024 edition) - https://phabricator.wikimedia.org/T351691#9713442 (10Aklapper) Sent emails about 275 tasks to 140 task assignees. [14:14:01] 10Release-Engineering-Team (Now this 🫠), 06cloud-services-team, 06collaboration-services, 10VPS-project-devtools, and 2 others: Update devtools project puppetmaster - https://phabricator.wikimedia.org/T360470#9713527 (10Dzahn) That's new. It worked before and "no route to host" sounds like the machine is d... [14:19:43] (03CR) 10Lucas Werkmeister (WMDE): [WIP] jjb: Specify PHP flavour in Wikibase special quibble jobs, add 81/83 (033 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/1019298 (https://phabricator.wikimedia.org/T362412) (owner: 10Jforrester) [14:29:36] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9713615 (10JMeybohm) [14:32:48] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9713630 (10JMeybohm) [14:35:02] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9713636 (10akosiaris) p:05Unbreak!→03High The immediate issue blocking the train has been resolved and new images have been pushed. Hence, l... [14:53:10] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.43.0-wmf.1 deployment blockers - https://phabricator.wikimedia.org/T361395#9713737 (10thcipriani) p:05Triage→03Medium a:03dancy [14:55:45] 06Release-Engineering-Team, 10Scap, 13Patch-For-Review: scap should support --pause-after-testserver-sync and testserver checks in --k8s-only mode - https://phabricator.wikimedia.org/T362437#9713747 (10CodeReviewBot) dancy merged https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/274 Run testse... [14:56:02] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.43.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T361396#9713765 (10thcipriani) p:05Triage→03Medium a:03brennen [15:05:29] 06Release-Engineering-Team, 10Scap, 13Patch-For-Review: scap should support --pause-after-testserver-sync and testserver checks in --k8s-only mode - https://phabricator.wikimedia.org/T362437#9713820 (10dancy) Scott your changes are included in scap 4.76.0 which I just deployed. [15:15:04] Creating patches via Gerrit doesn't seem to work right now - looking at the browser console, the problem appears to be that this file is missing: https://gerrit.wikimedia.org/r/plugins/codemirror-editor/static/codemirror-element.html [15:15:20] Via the Gerrit web interface, that is. [15:17:38] hashar, thcipriani, dancy: ^ [15:19:44] Yaron: I managed to create https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1019796 so it can't be broken for all [15:20:06] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 06serviceops, 06SRE, and 3 others: A lot of `[info] Wikitext for this page has duplicate ids:` in logstash for mw-parsoid. Possibly related to PageBundle - https://phabricator.wikimedia.org/T358588#9713923 (10MSantos) [15:20:42] Yaron: Can you supply the steps needed to demonstrate the problem? [15:24:08] RhinosF1: hm, okay. [15:24:21] dancy: I go to a URL like this, and click "Create Change": https://gerrit.wikimedia.org/r/admin/repos/mediawiki/extensions/PageForms,commands [15:25:48] Then I fill out the mini-form, click "Mark as Active", click "Edit", select a file - and then the file doesn't appear. [15:26:03] Thanks. Testing [15:26:14] This was all working fine at least until Friday. Thanks! [15:26:36] (It seems to happen for every extension, by the way.) [15:28:28] So I've made it to the click "Edit" step. Now when you say "select a file", what are you doing? [15:28:31] 10Phabricator, 06Moderator-Tools-Team: 14Herald rule for Moderator Tools maintained extension - 14https://phabricator.wikimedia.org/T362057#9714028 (10Samwalton9-WMF) 14Thanks @Aklapper! [15:28:48] Do you mean using "Add/open/upload" ? [15:29:11] Oh, sorry - I click on "Add/Open/Upload", then select any file that shows up in autocompletion. [15:31:56] Yaron: https://issues.gerritcodereview.com/issues/318802106 ? [15:32:49] 10GitLab (Administration, Settings & Policy), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: gitlab: enforce 2fa for admins - https://phabricator.wikimedia.org/T361277#9714043 (10LSobanski) a:03eoghan [15:32:55] 10GitLab (Administration, Settings & Policy), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: gitlab: enforce 2fa for admins - https://phabricator.wikimedia.org/T361277#9714044 (10LSobanski) p:05Triage→03High [15:33:46] 10GitLab, 06Release-Engineering-Team, 06collaboration-services: GitLab archival by Software Heritage - https://phabricator.wikimedia.org/T362258#9714046 (10LSobanski) No concerns from #collaboration-services [15:34:30] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services, 13Patch-For-Review, 07Python3-Porting: Many (all?) of the phabricator/tools scripts are in Python 2 - https://phabricator.wikimedia.org/T355574#9714048 (10LSobanski) p:05Medium→03Lowest [15:34:54] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services, 13Patch-For-Review, 07Python3-Porting: Many (all?) of the phabricator/tools scripts are in Python 2 - https://phabricator.wikimedia.org/T355574#9714050 (10LSobanski) p:05Lowest→03Low [15:39:26] dancy: sorry for the delay. Oh, interesting. For me, this is happening in Chrome - and it is apparently that same HTML file that the guy was complaining about for Firefox. [15:39:48] Nod. Seemed about the same. I tested w/ Chrome too and saw what you described. [15:40:02] Yaron: Would you mind filing a phabricator ticket? [15:40:05] Okay, that's a relief, that you saw the same problem. [15:40:08] Okay, sure. [15:40:14] Thanks! [15:43:36] 06Release-Engineering-Team, 10Scap, 06serviceops: scap should support --pause-after-testserver-sync and testserver checks in --k8s-only mode - https://phabricator.wikimedia.org/T362437#9714084 (10Scott_French) [15:44:21] 10Gerrit: Inline editing of files no longer works in Gerrit - https://phabricator.wikimedia.org/T362545 (10Yaron_Koren) 03NEW [15:44:35] dancy: I created it; see above. [15:44:43] Thanks! [15:45:19] 10Gerrit: Inline editing of files no longer works in Gerrit - https://phabricator.wikimedia.org/T362545#9714108 (10dancy) [15:47:12] 10Gerrit: Inline editing of files no longer works in Gerrit - https://phabricator.wikimedia.org/T362545#9714118 (10dancy) @hashar Bringing this to your attention [15:47:15] 06Release-Engineering-Team, 10Scap, 06serviceops: 14scap should support --pause-after-testserver-sync and testserver checks in --k8s-only mode - 14https://phabricator.wikimedia.org/T362437#9714122 (10Scott_French) 05In progress→03Resolved 14Many thanks for your help, @dancy! I'd like to add support... [15:52:40] 10Gerrit: Inline editing of files no longer works in Gerrit - https://phabricator.wikimedia.org/T362545#9714139 (10dancy) @Paladox [15:53:45] 10Gerrit: Inline editing of files no longer works in Gerrit - https://phabricator.wikimedia.org/T362545#9714144 (10Paladox) I think you need to clear your browser cache. It works for me and Polymer isn't use any more for the codemirror gerrit plugin. [15:56:14] 10Gerrit: Inline editing of files no longer works in Gerrit - https://phabricator.wikimedia.org/T362545#9714146 (10dancy) >>! In T362545#9714144, @Paladox wrote: > I think you need to clear your browser cache. It works for me and Polymer isn't use any more for the codemirror gerrit plugin. @Yaron_Koren I just t... [15:57:37] 10Gerrit: 14Inline editing of files no longer works in Gerrit - 14https://phabricator.wikimedia.org/T362545#9714164 (10Yaron_Koren) 05Open→03Invalid 14@Paladox - oh! How strange. I had tried clearing the browser cache several times before, but now when I tried it again, it did seem to solve the problem.... [16:00:07] 10Gerrit: Unchanged file with unresolved comment doesn't show up in file list - https://phabricator.wikimedia.org/T362269#9714170 (10Paladox) Does this work for you now? [16:03:15] 10Release-Engineering-Team (Now this 🫠), 06cloud-services-team, 06collaboration-services, 10VPS-project-devtools, and 2 others: Update devtools project puppetmaster - https://phabricator.wikimedia.org/T360470#9714191 (10Andrew) The latest puppetserver code is prone to gobbling RAM; I'd check for oom messag... [16:08:57] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.43.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T361396#9714241 (10brennen) [16:11:54] (03CR) 10Hashar: "I don't know whether `wikibase-client` and `wikibase-repo` need to be run with different flavor of PHP. If I remember well they always ran" [integration/config] - 10https://gerrit.wikimedia.org/r/1019298 (https://phabricator.wikimedia.org/T362412) (owner: 10Jforrester) [16:12:08] o/ [16:15:20] 10Gerrit: 14Inline editing of files no longer works in Gerrit - 14https://phabricator.wikimedia.org/T362545#9714295 (10hashar) 14I have upgraded Gerrit this morning and if I remember well there are some issues with some caches not being invalidated somewhere in the stack. Thus eventually the browsers runs o... [16:17:24] !log Changed operations/software/gerrit HEAD to `deploy/wmf/stable-3.8` # T354886 [16:17:28] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:17:28] T354886: Upgrade to Gerrit 3.8 - https://phabricator.wikimedia.org/T354886 [16:29:56] Project beta-update-databases-eqiad build #75282: 04FAILURE in 9 min 55 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/75282/ [16:36:19] 10Release-Engineering-Team (Now this 🫠), 06cloud-services-team, 06collaboration-services, 10VPS-project-devtools, and 2 others: Update devtools project puppetmaster - https://phabricator.wikimedia.org/T360470#9714432 (10brennen) > Also, just so i'm clear... your new puppet7 server is still named 'puppetmas... [16:48:02] 10Release-Engineering-Team (Now this 🫠), 06cloud-services-team, 06collaboration-services, 10VPS-project-devtools, and 2 others: Update devtools project puppetmaster - https://phabricator.wikimedia.org/T360470#9714532 (10Dzahn) Currently can't ssh to puppetmaster-1003 from external either - also with "no ro... [16:49:09] 10Release-Engineering-Team (Now this 🫠), 06cloud-services-team, 06collaboration-services, 10VPS-project-devtools, and 2 others: Update devtools project puppetmaster - https://phabricator.wikimedia.org/T360470#9714541 (10Dzahn) ` [?2004hroot@puppetmaster-1003:~# [ 899.382711] /dev/sdb: Can't open blockdev ` [16:49:47] 06Release-Engineering-Team, 06Infrastructure-Foundations, 06serviceops, 13Patch-For-Review: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9714546 (10JMeybohm) [16:52:12] 10Release-Engineering-Team (Now this 🫠), 06cloud-services-team, 06collaboration-services, 10VPS-project-devtools, and 2 others: Update devtools project puppetmaster - https://phabricator.wikimedia.org/T360470#9714557 (10Dzahn) "soft reboot" resurrected it, can ssh to it again. running puppet on the puppet... [16:55:16] 10Release-Engineering-Team (Now this 🫠), 06cloud-services-team, 06collaboration-services, 10VPS-project-devtools, and 2 others: Update devtools project puppetmaster - https://phabricator.wikimedia.org/T360470#9714583 (10Dzahn) Unfortunately this hasn't fixed the issue for agents yet, for example gitlab-run... [17:10:07] 10Release-Engineering-Team (Now this 🫠), 06cloud-services-team, 06collaboration-services, 10VPS-project-devtools, and 2 others: Update devtools project puppetmaster - https://phabricator.wikimedia.org/T360470#9714686 (10Dzahn) ` Exception in thread "main" java.lang.IllegalStateException: Unable to borrow J... [17:11:04] 10Release-Engineering-Team (Now this 🫠), 06cloud-services-team, 06collaboration-services, 10VPS-project-devtools, and 2 others: Update devtools project puppetmaster - https://phabricator.wikimedia.org/T360470#9714687 (10Dzahn) well, "parent directory /srv/puppet/server does not exist" sounds at least fixab... [17:13:12] (03PS1) 10Dreamrimmer: zuul: Add DreamRimmer to the CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1019850 [17:14:07] 10Release-Engineering-Team (Now this 🫠), 06cloud-services-team, 06collaboration-services, 10VPS-project-devtools, and 2 others: Update devtools project puppetmaster - https://phabricator.wikimedia.org/T360470#9714700 (10Dzahn) ` # @param ssldir_on_srv used on cloud-vps; it allows storing certs on a detacha... [17:27:46] (03CR) 10Sohom Datta: [C:03+1] "Trusted user on enwiki, Global renamer. User has expressed interesting in contributing to mediawiki-config related changes and PageTriage" [integration/config] - 10https://gerrit.wikimedia.org/r/1019850 (owner: 10Dreamrimmer) [17:28:19] (03CR) 10Sohom Datta: [C:03+1] zuul: Add DreamRimmer to the CI allowlist (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1019850 (owner: 10Dreamrimmer) [17:30:28] Yippee, build fixed! [17:30:28] Project beta-update-databases-eqiad build #75283: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/75283/ [17:31:48] (03PS2) 10Dreamrimmer: zuul: Add DreamRimmer to the CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1019850 [17:40:24] (03CR) 10Sohom Datta: [C:03+1] zuul: Add DreamRimmer to the CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1019850 (owner: 10Dreamrimmer) [17:40:37] (03CR) 10Sohom Datta: [C:03+1] zuul: Add DreamRimmer to the CI allowlist (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1019850 (owner: 10Dreamrimmer) [17:54:24] (03CR) 10Jforrester: "Not running the job in non-master would 'fix' the bug, yes, but I don't think that's acceptable for WMDE?" [integration/config] - 10https://gerrit.wikimedia.org/r/1019298 (https://phabricator.wikimedia.org/T362412) (owner: 10Jforrester) [18:07:29] 10GitLab, 10Release-Engineering-Team (Radar), 06collaboration-services: GitLab archival by Software Heritage - https://phabricator.wikimedia.org/T362258#9714971 (10thcipriani) Reading the [[https://docs.softwareheritage.org/user/faq/#add-forge-now|faq]] answered all the questions I had. No concerns here either. [18:10:27] 10GitLab, 10Release-Engineering-Team (Radar), 06collaboration-services: GitLab archival by Software Heritage - https://phabricator.wikimedia.org/T362258#9714975 (10JJMC89) @LSobanski and @thcipriani Would you like me to respond to them or would you like to? [18:16:01] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 07Zuul: 14Jenkins 'test' pipeline should cancel old jobs after new patch sets - 14https://phabricator.wikimedia.org/T229708#9714979 (10Dreamy_Jazz) 14@hashar Example: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CheckUs... [18:17:46] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 07Zuul: Jenkins 'test' pipeline should cancel old jobs after new patch sets - https://phabricator.wikimedia.org/T229708#9714981 (10Dreamy_Jazz) 05Declined→03Open Marking this as open again because I have an example of this. [18:32:53] 10Phabricator, 10Wikimedia-Phabricator-Extensions: Make OAuth session persistent in Phabricator - https://phabricator.wikimedia.org/T140448#9715024 (10Pppery) >>! In T140448#9711855, @Aklapper wrote: > Current https://www.mediawiki.org/w/index.php?title=OAuth/For_Developers&oldid=6456529 says: >> Applications... [18:49:33] 10Phabricator, 10Wikimedia-Phabricator-Extensions, 07Epic, 13Patch-For-Review, 07PHP 8.0 support: Make WMF custom Phab extensions and code work with PHP8 - https://phabricator.wikimedia.org/T342244#9715080 (10CodeReviewBot) brennen merged https://gitlab.wikimedia.org/repos/phabricator/extensions/-/merge_... [18:51:36] 10Release-Engineering-Team (Priority Backlog 📥), 10Wikimedia-Phabricator-Extensions, 13Patch-For-Review: Improve error handling in "Escalate security issue" code - https://phabricator.wikimedia.org/T337654#9715089 (10CodeReviewBot) brennen merged https://gitlab.wikimedia.org/repos/phabricator/extensions/-/me... [19:02:22] urbanecm: I see a stack of 8 patches from you in CI; please remember that every extra patch in the chain doubles the number of fork targets for CI and so is very expensive beyond 3–4 patches. [19:02:26] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 13Patch-For-Review: 14Remove script to convert Phabricator parent project into subproject - 14https://phabricator.wikimedia.org/T342275#9715124 (10CodeReviewBot) 14brennen closed https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/... [19:02:55] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 13Patch-For-Review: 14Remove script to convert Phabricator parent project into subproject - 14https://phabricator.wikimedia.org/T342275#9715121 (10brennen) 05Open→03Resolved a:03Frostly 14Applied delete from the merge request at: htt... [19:03:26] James_F: thanks for the info. just curious, would you mind expanding on what does "doubles the number of fork targets for CI" mean? [19:05:02] urbanecm: Every dependency causes zuul-cloner to create the "patch+head" and "patch+dependency+head" pair; if there are two deep, that's ["patch+head", "patch+dep1+head", "patch+dep2+head", "patch+dep1+dep2+head" ] etc. [19:06:50] James_F: interesting, good to know. sorry for the huge chain then – my team's been quite short on staff last week, so i spent it experimenting on something and the chain's the result. will get it reviewed&merged soon though [19:08:30] No worries. :-) [19:09:05] Not a crisis (until the chain gets to > 10 patches, at which point it can DoS CI which is why it's only allowed for C+2ers). [19:10:41] 10Release-Engineering-Team (Priority Backlog 📥), 10Wikimedia-Phabricator-Extensions, 13Patch-For-Review: Train blockers task throws exceptions on strange version numbers - https://phabricator.wikimedia.org/T304540#9715134 (10CodeReviewBot) brennen merged https://gitlab.wikimedia.org/repos/phabricator/extensi... [19:11:54] 10Phabricator (phabricator-next), 10Release-Engineering-Team (Priority Backlog 📥), 13Patch-For-Review: 14Remove script to convert Phabricator parent project into subproject - 14https://phabricator.wikimedia.org/T342275#9715140 (10brennen) [19:13:17] 10Phabricator (phabricator-next), 10Release-Engineering-Team (Priority Backlog 📥), 10Wikimedia-Phabricator-Extensions, 13Patch-For-Review: 14Train blockers task throws exceptions on strange version numbers - 14https://phabricator.wikimedia.org/T304540#9715135 (10brennen) 05Open→03Resolved a:03bren... [19:13:50] 10Phabricator (phabricator-next), 10Release-Engineering-Team (Priority Backlog 📥), 13Patch-For-Review: Remove script to convert Phabricator parent project into subproject - https://phabricator.wikimedia.org/T342275#9715143 (10brennen) 05Resolved→03Open May be lingering documentation cleanup here after sc... [19:19:02] 14Phabricator-Sprint-Extension, 10Release-Engineering-Team (Priority Backlog 📥), 06Diffusion-Repository-Administrators, 10Projects-Cleanup, and 2 others: Archive broken and unmaintained Phabricator Sprint extension - https://phabricator.wikimedia.org/T275325#9715198 (10CodeReviewBot) brennen merged https:/... [19:20:16] 14Phabricator-Sprint-Extension, 10Release-Engineering-Team (Priority Backlog 📥), 06Diffusion-Repository-Administrators, 10Projects-Cleanup, and 2 others: 14Archive broken and unmaintained Phabricator Sprint extension - 14https://phabricator.wikimedia.org/T275325#9715199 (10brennen) 05Open→03Resolved [19:28:50] 10Phabricator, 10Wikimedia-Phabricator-Extensions, 13Patch-For-Review: Remove custom src/customfields/DifferentialApplyPatchWithOnlyGitField.php - https://phabricator.wikimedia.org/T342041#9715286 (10CodeReviewBot) brennen merged https://gitlab.wikimedia.org/repos/phabricator/extensions/-/merge_requests/27... [19:32:36] 10Phabricator (phabricator-next), 10Wikimedia-Phabricator-Extensions: 14Remove custom src/customfields/DifferentialApplyPatchWithOnlyGitField.php - 14https://phabricator.wikimedia.org/T342041#9715305 (10brennen) 05Open→03Resolved 14Should deploy tomorrow, 2024-04-16. [19:44:07] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 07Zuul: Jenkins 'test' pipeline should cancel old jobs after new patch sets - https://phabricator.wikimedia.org/T229708#9715352 (10hashar) @Dreamy_Jazz awesome! Everything ends up easier when there is a very recent case since that me... [19:57:01] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 13Patch-For-Review: Make Global Search link to application specific Advanced Search - https://phabricator.wikimedia.org/T360518#9715392 (10CodeReviewBot) brennen merged https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/... [20:00:43] 10Phabricator (phabricator-next), 10Release-Engineering-Team (Priority Backlog 📥), 13Patch-For-Review: 14Make Global Search link to application specific Advanced Search - 14https://phabricator.wikimedia.org/T360518#9715395 (10brennen) 05Open→03Resolved 14Seems like a good idea to me. Tweaked the MR... [20:02:40] 10Release-Engineering-Team (Now this 🫠), 06cloud-services-team, 06collaboration-services, 10VPS-project-devtools, and 2 others: Update devtools project puppetmaster - https://phabricator.wikimedia.org/T360470#9715403 (10Dzahn) After the above created /srv/puppet the service could start again and puppet run... [20:03:25] 10Diffusion, 10Phabricator, 13Patch-For-Review: Enable git repo creation in Diffusion again - https://phabricator.wikimedia.org/T355644#9715406 (10CodeReviewBot) brennen merged https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/37 Reinstate custom rPHABd8dd5f4b71ea8af64831dde779c598... [20:07:51] 10Phabricator, 10Wikimedia-Phabricator-Extensions: Make OAuth session persistent in Phabricator - https://phabricator.wikimedia.org/T140448#9715427 (10bd808) >>! In T140448#9711855, @Aklapper wrote: > So this is "just" about replacing the URI in `getAuthorizeTokenURI()` at https://gitlab.wikimedia.org/repos/ph... [20:27:33] 10Diffusion, 10Phabricator, 13Patch-For-Review: Enable git repo creation in Diffusion again - https://phabricator.wikimedia.org/T355644#9715469 (10brennen) Will deploy https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/37 tomorrow 2024-04-16. [20:29:10] 10Phabricator, 13Patch-For-Review: Revert rPHDEP024c5b0d1d05cbac148128eb0f77df7ac578220b after upstream Phorge 2023.49 deployment - https://phabricator.wikimedia.org/T354277#9715482 (10CodeReviewBot) brennen merged https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/36 Revert rPHDEP024c... [20:31:19] 10Phabricator (phabricator-next): 14Revert rPHDEP024c5b0d1d05cbac148128eb0f77df7ac578220b after upstream Phorge 2023.49 deployment - 14https://phabricator.wikimedia.org/T354277#9715485 (10brennen) 05Open→03Resolved 14Will deploy tomorrow, 2024-04-16. [21:37:19] brennen: Thanks for the code reviews <3 [21:37:40] andre: yw. :) sorry so many of these have languished for so long. [21:37:54] nah, happy to see shit merged [21:44:45] 10GitLab (Integrations), 10Phabricator, 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10Wikimedia-Phabricator-Extensions, 13Patch-For-Review: Phabricator gitlab panel is hard to read on narrow screens - https://phabricator.wikimedia.org/T362047#9715644 (10CodeReviewBot) brennen merged https://gitlab.wikim... [21:47:15] 10Phabricator: Changing a non-access request ticket into an access request ticket does not create restricted blocker - https://phabricator.wikimedia.org/T103194#9715649 (10CodeReviewBot) brennen updated https://gitlab.wikimedia.org/repos/phabricator/extensions/-/merge_requests/30 Remove unused createPrivate... [21:47:17] 10Phabricator, 06SRE, 13Patch-For-Review: have any task put into ops-access-requests automatically generate an ops-access-review task - https://phabricator.wikimedia.org/T87467#9715648 (10CodeReviewBot) brennen updated https://gitlab.wikimedia.org/repos/phabricator/extensions/-/merge_requests/30 Remove... [21:47:24] 10GitLab (Integrations), 10Phabricator, 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10Wikimedia-Phabricator-Extensions, 13Patch-For-Review: Phabricator gitlab panel is hard to read on narrow screens - https://phabricator.wikimedia.org/T362047#9715645 (10brennen) 05Open→03In progress We'll deploy @Ak... [21:48:18] 10Phabricator: Changing a non-access request ticket into an access request ticket does not create restricted blocker - https://phabricator.wikimedia.org/T103194#9715654 (10CodeReviewBot) brennen merged https://gitlab.wikimedia.org/repos/phabricator/extensions/-/merge_requests/30 Remove unused createPrivateS... [21:48:23] 10Phabricator, 06SRE, 13Patch-For-Review: have any task put into ops-access-requests automatically generate an ops-access-review task - https://phabricator.wikimedia.org/T87467#9715653 (10CodeReviewBot) brennen merged https://gitlab.wikimedia.org/repos/phabricator/extensions/-/merge_requests/30 Remove u... [22:30:34] 10Release-Engineering-Team (Now this 🫠), 10Scap, 10MediaWiki-libs-Mime, 10MediaWiki-Platform-Team (Radar), and 2 others: Get rid of the /srv/mediawiki/php symbolic link - https://phabricator.wikimedia.org/T359643#9715719 (10CodeReviewBot) dancy opened https://gitlab.wikimedia.org/repos/releng/scap/-/merge_... [23:41:59] 10GitLab (Account Approval): Correct and then approve https://gitlab.wikimedia.org/sethabathaba - https://phabricator.wikimedia.org/T353857#9715880 (10bd808) I was doing some other cleanup of pending GitLab accounts and I think I may have rejected the https://gitlab.wikimedia.org/sethabathaba account. Attempting... [23:44:33] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for safan41 - https://phabricator.wikimedia.org/T361018#9715882 (10bd808) I was doing some other cleanup of pending GitLab accounts and I think I rejected the https://gitlab.wikimedia.org/safan41 account. Logging in t...