[01:13:37] 10Continuous-Integration-Config, 10MediaWiki-Vendor, 07Composer, 05MW-1.39-release, and 4 others: Upgrade composer from 2.6.4 to 2.7.x - https://phabricator.wikimedia.org/T360973 (10Reedy) 03NEW [01:27:06] bd808: for the calendar, I had a look at how OpenDev is doing it. They have a system to keep track of weekly meetings slots and generate a webpage / .ical file [01:27:19] https://meetings.opendev.org/ [01:28:23] and an example entry https://opendev.org/opendev/irc-meetings/src/branch/master/meetings/qa-team-meeting.yaml [01:30:27] and the code is at https://pypi.org/project/yaml2ical/ [01:32:16] it is probably fine to define the weekly slot, but not really for the self serve "I want to get X deployed during the window, please help" [02:01:20] 10GitLab, 10Release-Engineering-Team (Radar), 10ChangeProp, 06collaboration-services, and 9 others: Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596#9660226 (10tstarling) PhpRedis is getting behind KeyDB with [[https://github.com/phpredis/ph... [02:55:21] 10Phabricator, 10Observability-Logging: Phabricator error log PHP entries truncated - https://phabricator.wikimedia.org/T360975 (10Aklapper) 03NEW [02:57:51] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): 14Clean up Phabricator production error logs - 14https://phabricator.wikimedia.org/T337500#9660273 (10Aklapper) 05Openβ†’03Resolved 14Let's declare this resolved. There's always more to do but that should be covered in dedicated tickets, an... [02:57:55] 10Phabricator: 14MediaWiki OAuth panel could be more clear in Phabricator - 14https://phabricator.wikimedia.org/T91584#9660275 (10Aklapper) 05Openβ†’03Resolved 14I think this is sufficiently clear nowadays - everything mentioned in the original task description seems to have been fixed in the meantime. [08:40:38] 10Deployments, 06Release-Engineering-Team, 06serviceops: httpbb appserver test breaks deployment of the week due to a timeout parsing page - https://phabricator.wikimedia.org/T360867#9660563 (10hashar) [09:20:05] (03CR) 10Isabelle Hurbain-Palatin: [C:03+2] Remove adaptor to suppress catlinks - Parsoid generates them now [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1013720 (https://phabricator.wikimedia.org/T351931) (owner: 10Subramanya Sastry) [09:20:54] (03Merged) 10jenkins-bot: Remove adaptor to suppress catlinks - Parsoid generates them now [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1013720 (https://phabricator.wikimedia.org/T351931) (owner: 10Subramanya Sastry) [09:39:11] 10Release-Engineering-Team (Now this 🫠), 06Security-Team, 13Patch-For-Review, 07SecTeam-Processed: Notify MediaWiki security tasks as soon as an uploaded patch fails to apply - https://phabricator.wikimedia.org/T350065#9660656 (10CodeReviewBot) jnuche opened https://gitlab.wikimedia.org/repos/releng/scap/-... [09:45:01] 10GitLab (CI & Job Runners), 06collaboration-services, 13Patch-For-Review: Create a special-purpose Trusted Runner with Dockerfile frontend - https://phabricator.wikimedia.org/T357612#9660680 (10Jelto) [09:47:27] (03CR) 10Hashar: [C:03+2] jjb: switch jobs to Quibble 1.7.0 [integration/config] - 10https://gerrit.wikimedia.org/r/1014043 (owner: 10Hashar) [09:47:38] !log Upgrading Quibble to 1.7.0 | https://gerrit.wikimedia.org/r/c/integration/config/+/1014043 [09:47:39] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:48:48] (03Merged) 10jenkins-bot: jjb: switch jobs to Quibble 1.7.0 [integration/config] - 10https://gerrit.wikimedia.org/r/1014043 (owner: 10Hashar) [09:49:07] 10Release-Engineering-Team (Now this 🫠), 06Security-Team, 13Patch-For-Review, 07SecTeam-Processed: Notify MediaWiki security tasks as soon as an uploaded patch fails to apply - https://phabricator.wikimedia.org/T350065#9660681 (10CodeReviewBot) jnuche merged https://gitlab.wikimedia.org/repos/releng/scap/-... [09:50:48] 10Continuous-Integration-Infrastructure, 06Growth-Team, 10Quibble, 10Thanks, and 2 others: 14CI test failure due to Git error - 14https://phabricator.wikimedia.org/T356247#9660682 (10hashar) 05Stalledβ†’03Resolved 14>>! In T356247#9509151, @hashar wrote: > Stalled pending a release of #quibble to be... [09:51:10] 06Release-Engineering-Team, 10Quibble, 13Patch-For-Review: 14Quibble --command is no more interactive causing "bash" to show no prompt - 14https://phabricator.wikimedia.org/T360443#9660688 (10hashar) 05Openβ†’03Resolved a:03hashar [10:41:07] 10Phabricator, 06collaboration-services, 06WMF-NDA-Requests: Add WMDE staff who have signed the NDA with the WMF to the WMF-NDA phabricator policy group - https://phabricator.wikimedia.org/T358578#9660824 (10AndrewTavis_WMDE) Thanks for looking into this, @Dzahn! A bit confused as the task for my NDA was com... [10:48:39] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for EEnabulele - https://phabricator.wikimedia.org/T360992 (10E.Enabulele) 03NEW [10:53:10] (03CR) 10Ilias Sarantopoulos: "Thanks a lot Antoine!" [integration/config] - 10https://gerrit.wikimedia.org/r/1013161 (https://phabricator.wikimedia.org/T357986) (owner: 10Ilias Sarantopoulos) [10:54:15] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for EEnabulele - https://phabricator.wikimedia.org/T360992#9660899 (10Aklapper) Hi @E.Enabulele, in case that you are WMF staff or a contractor, could you please [connect](https://phabricator.wikimedia.org/settings/pa... [10:57:52] 10Gerrit (Gerrit 3.8), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Upgrade to Gerrit 3.8 - https://phabricator.wikimedia.org/T354886#9660906 (10hashar) My message sent to upstream: I have an issue with the Gerrit 3.8.4 release, it looks like it got tagged on Feb 23 and is mis... [10:58:25] (03CR) 10Hashar: Merge tag 'v3.8.4' into wmf/stable-3.8 (031 comment) [software/gerrit] (wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1013953 (https://phabricator.wikimedia.org/T354886) (owner: 10Hashar) [10:59:17] 10GitLab (CI & Job Runners), 06collaboration-services, 13Patch-For-Review: Create a special-purpose Trusted Runner with Dockerfile frontend - https://phabricator.wikimedia.org/T357612#9660907 (10Jelto) [10:59:18] 10GitLab (CI & Job Runners), 06collaboration-services, 13Patch-For-Review: Create a special-purpose Trusted Runner with Dockerfile frontend - https://phabricator.wikimedia.org/T357612#9660911 (10Jelto) [11:04:14] (03CR) 10Hashar: [C:03+2] Merge tag 'v3.8.4' into wmf/stable-3.8 [software/gerrit] (wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1013953 (https://phabricator.wikimedia.org/T354886) (owner: 10Hashar) [11:11:35] (03Merged) 10jenkins-bot: Merge tag 'v3.8.4' into wmf/stable-3.8 [software/gerrit] (wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1013953 (https://phabricator.wikimedia.org/T354886) (owner: 10Hashar) [11:13:55] 10GitLab, 10Release-Engineering-Team (Radar), 10ChangeProp, 06collaboration-services, and 9 others: Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596#9660965 (10larissagaulia) [11:21:48] 10Continuous-Integration-Config, 10MW-1.42-notes (1.42.0-wmf.19; 2024-02-20), 07PHP 8.3 support: Make PHP 8.3 voting on development (master) branch of MW ecosystem (core, vendor, extensions, skins, libraries) - https://phabricator.wikimedia.org/T353362#9661036 (10Jdforrester-WMF) [11:21:49] 10Continuous-Integration-Config, 07PHP 8.2 support: Make PHP 8.2 voting on development (master) branch of MW ecosystem (core, vendor, extensions, skins, libraries) - https://phabricator.wikimedia.org/T352085#9661037 (10Jdforrester-WMF) [11:33:04] (03PS1) 10Hashar: Gerrit 3.8.4 and rebuild plugins [software/gerrit] (deploy/wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1014488 (https://phabricator.wikimedia.org/T354886) [11:33:22] (03CR) 10CI reject: [V:04-1] Gerrit 3.8.4 and rebuild plugins [software/gerrit] (deploy/wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1014488 (https://phabricator.wikimedia.org/T354886) (owner: 10Hashar) [11:56:46] (03CR) 10Hashar: "recheck after pushing LFS objects" [software/gerrit] (deploy/wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1014488 (https://phabricator.wikimedia.org/T354886) (owner: 10Hashar) [12:00:55] (03PS2) 10Hashar: Gerrit 3.8.4 and rebuild plugins [software/gerrit] (deploy/wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1014488 (https://phabricator.wikimedia.org/T354886) [13:20:37] 10GitLab: Clicking outside the name of a repository on a group project list ignores request to open in a new tab - https://phabricator.wikimedia.org/T361004 (10taavi) 03NEW [13:21:07] (03CR) 10Hashar: [C:03+2] Gerrit 3.8.4 and rebuild plugins [software/gerrit] (deploy/wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1014488 (https://phabricator.wikimedia.org/T354886) (owner: 10Hashar) [13:21:13] (03Merged) 10jenkins-bot: Gerrit 3.8.4 and rebuild plugins [software/gerrit] (deploy/wmf/stable-3.8) - 10https://gerrit.wikimedia.org/r/1014488 (https://phabricator.wikimedia.org/T354886) (owner: 10Hashar) [13:24:22] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for resquito - https://phabricator.wikimedia.org/T361008 (10REsquito-WMF) 03NEW [13:24:26] (03CR) 10Hashar: [C:03+2] tox.ini: require virtualenv<20.22.0 for Python 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1014040 (owner: 10Hashar) [13:24:34] (03PS2) 10Hashar: tox.ini: require virtualenv<20.22.0 for Python 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1014040 (https://phabricator.wikimedia.org/T345695) [13:24:45] (03CR) 10Hashar: [C:03+2] tox.ini: require virtualenv<20.22.0 for Python 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1014040 (https://phabricator.wikimedia.org/T345695) (owner: 10Hashar) [13:25:29] (03Merged) 10jenkins-bot: tox.ini: require virtualenv<20.22.0 for Python 2.7 [integration/config] - 10https://gerrit.wikimedia.org/r/1014040 (https://phabricator.wikimedia.org/T345695) (owner: 10Hashar) [13:25:39] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for WMF-Ldlulisa - https://phabricator.wikimedia.org/T361011 (10LDlulisa-WMF) 03NEW [13:25:45] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for WMF-Ldlulisa - https://phabricator.wikimedia.org/T361011#9661450 (10LDlulisa-WMF) [13:26:29] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for `Ruairi O'Donnell` - https://phabricator.wikimedia.org/T361012 (10ROdonnell-WMF) 03NEW [13:28:13] 10Phabricator, 10Observability-Logging: Phabricator error log PHP entries truncated - https://phabricator.wikimedia.org/T360975#9661489 (10colewhite) I located some instances [[ https://logstash.wikimedia.org/goto/fbc8ebb5efe29b5879b337382f45106a | here ]]. The count of these truncated logs is very small - 13... [13:34:36] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team: Rebuild integration-cumin to get rid of Debian Buster - https://phabricator.wikimedia.org/T360784#9661506 (10hashar) @joanna_borun I am mostly likely going to need assistance from people knowing Cumin. Notably I don't know which Debian distri... [13:59:41] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for safan41 - https://phabricator.wikimedia.org/T361018 (10Safan41) 03NEW [14:42:32] 06Release-Engineering-Team, 10Scap, 06Data-Platform-SRE: "scap deploy"'s config-deploy should check for broken symlinks - https://phabricator.wikimedia.org/T342162#9661817 (10Sandeeps) a:03Sandeeps [14:57:54] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for renilthomas - https://phabricator.wikimedia.org/T361022 (10RThomas-WMF) 03NEW [14:58:11] 10GitLab (Project Migration), 06collaboration-services, 13Patch-For-Review, 10SRE Observability (FY2023/2024-Q4): Migrate SRE repositories to GitLab - operations/alerts - https://phabricator.wikimedia.org/T349626#9661875 (10fgiunchedi) [15:10:36] 10Release-Engineering-Team (Now this 🫠), 06Security-Team, 07SecTeam-Processed: 14Notify MediaWiki security tasks as soon as an uploaded patch fails to apply - 14https://phabricator.wikimedia.org/T350065#9661915 (10jnuche) 05In progressβ†’03Resolved 14This should be complete now, including notifying fa... [15:26:19] 10Release-Engineering-Team (Now this 🫠), 06Security-Team, 07SecTeam-Processed: 14Notify MediaWiki security tasks as soon as an uploaded patch fails to apply - 14https://phabricator.wikimedia.org/T350065#9661984 (10sbassett) 14Thanks! [16:28:54] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for renilthomas - https://phabricator.wikimedia.org/T361022#9662284 (10bd808) 05Openβ†’03In progress a:03bd808 [16:34:31] 10GitLab (Account Approval), 06Release-Engineering-Team: 14Requesting GitLab account activation for renilthomas - 14https://phabricator.wikimedia.org/T361022#9662336 (10bd808) 05In progressβ†’03Resolved 14* Added @RThomas-WMF to #trusted-contributors * Waited for #tool-gitlab-account-approval to do the... [16:38:18] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for safan41 - https://phabricator.wikimedia.org/T361018#9662362 (10bd808) @Yug can you vouch for @Safan41? [16:42:55] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for `Ruairi O'Donnell` - https://phabricator.wikimedia.org/T361012#9662390 (10bd808) 05Openβ†’03In progress a:03bd808 [16:43:17] is ther a way to disable the 'New signins require approval' popup on every push? [16:44:36] 10GitLab (Account Approval), 06Release-Engineering-Team: 14Requesting GitLab account activation for `Ruairi O'Donnell` - 14https://phabricator.wikimedia.org/T361012#9662400 (10bd808) 05In progressβ†’03Resolved 14* Added @ROdonnell-WMF to #trusted-contributors * Manually approved https://gitlab.wikimedia... [16:46:53] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for WMF-Ldlulisa - https://phabricator.wikimedia.org/T361011#9662424 (10bd808) 05Openβ†’03In progress a:03bd808 [16:48:20] 10GitLab (Account Approval), 06Release-Engineering-Team: 14Requesting GitLab account activation for WMF-Ldlulisa - 14https://phabricator.wikimedia.org/T361011#9662442 (10bd808) 05In progressβ†’03Resolved 14* Added @LDlulisa-WMF to #trusted-contributors * Manually approved https://gitlab.wikimedia.org/w... [16:49:14] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for resquito - https://phabricator.wikimedia.org/T361008#9662448 (10bd808) 05Openβ†’03In progress a:03bd808 [16:50:45] 10GitLab (Account Approval), 06Release-Engineering-Team: 14Requesting GitLab account activation for WMF-Ldlulisa - 14https://phabricator.wikimedia.org/T361011#9662458 (10LDlulisa-WMF) 14Thank you @@bd808.  [16:50:58] 10GitLab (Account Approval), 06Release-Engineering-Team: 14Requesting GitLab account activation for resquito - 14https://phabricator.wikimedia.org/T361008#9662460 (10bd808) 05In progressβ†’03Resolved 14* Added @REsquito-WMF to #trusted-contributors * Manually approved https://gitlab.wikimedia.org/resqui... [17:05:58] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for EEnabulele - https://phabricator.wikimedia.org/T360992#9662512 (10bd808) 05Openβ†’03In progress a:03bd808 [17:11:25] 10GitLab (Account Approval), 06Release-Engineering-Team: 14Requesting GitLab account activation for EEnabulele - 14https://phabricator.wikimedia.org/T360992#9662541 (10bd808) 05In progressβ†’03Resolved 14* Manually checked `uid=eenabulele,ou=people,dc=wikimedia,dc=org` Developer account and office.wikim... [17:41:21] 06Release-Engineering-Team, 10Scap, 06Data-Platform-SRE: "scap deploy"'s config-deploy should check for broken symlinks - https://phabricator.wikimedia.org/T342162#9662628 (10Sandeeps) 05Openβ†’03In progress [18:33:50] 10Phabricator (Upstream), 07Regression, 07Upstream: PhabricatorDataNotAttachedException when rendering project hovercard with username mentioned in project description - https://phabricator.wikimedia.org/T360530#9662816 (10Aklapper) Fixed in upstream: https://we.phorge.it/rP216d308507edd54a68cb392df9f4701126... [20:00:43] 10Phabricator, 06collaboration-services, 06WMF-NDA-Requests: Add WMDE staff who have signed the NDA with the WMF to the WMF-NDA phabricator policy group - https://phabricator.wikimedia.org/T358578#9663292 (10Dzahn) Thank you @AndrewTavis_WMDE I think there was a mismatch because of your last names. I have al... [20:02:39] 10Release-Engineering-Team (Yak Shaving πŸƒπŸͺ’): 14Yak: Investigate HTTPBB during scap (to replace simple smoke-test) - 14https://phabricator.wikimedia.org/T280094#9663306 (10thcipriani) 05Openβ†’03Resolved a:03dancy [20:30:20] tip of the day: `GitLab Mono` font is not monospace :/ [20:30:48] it's probably black and white [20:33:06] Reedy: https://phabricator.wikimedia.org/F43461159 :) [20:33:17] see, I was right; P [20:35:41] yeah [20:35:49] well [20:35:49] no [20:35:53] it is not even black and white [20:37:33] it is #737278 [20:37:37] anyway [20:43:42] white-space: pre-line; [20:46:12] which collapses whitespaces [20:46:16] {solved} [21:34:24] 10GitLab (Infrastructure), 10Release-Engineering-Team (Radar), 06collaboration-services: Add GitLab upgrades and maintenance to deployment calendar - https://phabricator.wikimedia.org/T336470#9663733 (10CodeReviewBot) dzahn updated https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/66 add Da... [21:35:54] 10GitLab (Infrastructure), 10Release-Engineering-Team (Radar), 06collaboration-services: Add GitLab upgrades and maintenance to deployment calendar - https://phabricator.wikimedia.org/T336470#9663740 (10Dzahn) I learned this is the place to add a regular window: https://gitlab.wikimedia.org/repos/releng/rel... [21:38:12] 10GitLab (Infrastructure), 10Release-Engineering-Team (Radar), 06collaboration-services: Add GitLab upgrades and maintenance to deployment calendar - https://phabricator.wikimedia.org/T336470#9663754 (10Dzahn) >>! In T336470#9642258, @Jelto wrote: > I'm happy to get other opinions, especially from Americas S... [22:30:14] hi, was there a phab upgrade recently? i noticed that some bots seem to be having problems. stashbot didn't post comments during a backport earlier, and also whatever is supposed to remove patch-for-review tags doesn't do it any more. see https://phabricator.wikimedia.org/T359751 for an example [22:30:43] (i didn't file a bug yet, but i will if this isn't a known issue) [22:31:14] MatmaRex: last week iirc [22:31:59] MatmaRex: https://phabricator.wikimedia.org/T358610 [22:35:48] alright, i'll phile it [22:44:48] also… is it just me, or is phab slow? [22:45:48] it's struggling to show me comment previews for example, with requests taking multiple seconds or failing [22:46:49] 10Phabricator, 10Stashbot: Stashbot seems down - https://phabricator.wikimedia.org/T361073 (10matmarex) 03NEW [22:47:10] 10Phabricator, 10Phabricator maintenance bot: #patch-for-review tags are not being removed from tasks - https://phabricator.wikimedia.org/T361074 (10matmarex) 03NEW [22:48:29] glad to see these notifications, because i can't load either of these tasks i just filed :) [22:48:57] seems ok to me [22:52:04] maybe it's some network thing closer to my side then. the rest of the internet works fine though [22:55:07] !log unclear if stashbot is working (T361073) [22:55:10] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:55:10] T361073: Stashbot seems down - https://phabricator.wikimedia.org/T361073 [22:55:28] https://phabricator.wikimedia.org/T361073#9664035 [22:55:34] so it's working at least _some_ of the time... [22:59:31] MatmaRex: any chance the backport it didn't log for was the one that hitched along with another? [23:00:56] like code got merged but wasn't mentioned in the log message from scap backport? [23:01:00] oh, hmm. maybe, maybe it just wasn't logged correctly [23:03:08] stashbot doesn't talk back to logmsgbot in -operations either to try and keep bot noise down a bit [23:03:08] See https://wikitech.wikimedia.org/wiki/Tool:Stashbot for help. [23:04:24] 10Phabricator, 10Stashbot: Stashbot seems down - https://phabricator.wikimedia.org/T361073#9664040 (10brennen) I suspect we have an explanation here: `lang=irc 16:30 hi, was there a phab upgrade recently? i noticed that some bots seem to be having problems. stashbot didn't post comments during a ba... [23:11:04] 10Phabricator, 10Stashbot: 14Stashbot seems down - 14https://phabricator.wikimedia.org/T361073#9664057 (10bd808) 05Openβ†’03Invalid 14I see no signs of Stashbot being sad recently. I think this is the more likely explantion: `lang=irc [20:27] !log catrope@deploy1002 catrope: Backport for [[... [23:15:11] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 06Infrastructure-Foundations, 10observability, and 2 others: beta-scap-sync-world fails: logstash_checker.py: KeyError: 'aggregations' - https://phabricator.wikimedia.org/T360595#9664059 (10Ladsgroup) I issued a new certificate from the puppetmaste... [23:15:55] (03CR) 10Ladsgroup: "This probably should be reverted I think" [integration/config] - 10https://gerrit.wikimedia.org/r/1013598 (https://phabricator.wikimedia.org/T360595) (owner: 10Hashar)