[00:46:17] Project beta-scap-sync-world build #147652: 04FAILURE in 1 min 2 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147652/ [00:48:59] 00:46:17 00:46:17 Check 'Logstash canary error rate' failed: Traceback (most recent call last): [00:48:59] 00:46:17 File "/usr/local/bin/logstash_checker.py", line 421, in [00:48:59] 00:46:17 main() [00:48:59] 00:46:17 File "/usr/local/bin/logstash_checker.py", line 412, in main [00:49:00] 00:46:17 sys.exit(checker.run()) [00:49:02] 00:46:17 File "/usr/local/bin/logstash_checker.py", line 309, in run [00:49:04] 00:46:17 entries = r['aggregations']['2']['buckets'] [00:49:06] 00:46:17 KeyError: 'aggregations' [00:56:13] Project beta-scap-sync-world build #147653: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147653/ [01:06:17] Project beta-scap-sync-world build #147654: 04STILL FAILING in 1 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147654/ [01:16:21] Project beta-scap-sync-world build #147655: 04STILL FAILING in 1 min 6 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147655/ [01:26:22] Project beta-scap-sync-world build #147656: 04STILL FAILING in 1 min 6 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147656/ [01:36:19] Project beta-scap-sync-world build #147657: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147657/ [01:46:20] Project beta-scap-sync-world build #147658: 04STILL FAILING in 1 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147658/ [01:46:31] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10CAS-SSO, 06Infrastructure-Foundations: Correct IDP login page Privacy Policy - https://phabricator.wikimedia.org/T350129#9648092 (10Pppery) a:03Pppery [01:56:16] Project beta-scap-sync-world build #147659: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147659/ [02:06:16] Project beta-scap-sync-world build #147660: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147660/ [02:16:19] Project beta-scap-sync-world build #147661: 04STILL FAILING in 1 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147661/ [02:26:17] Project beta-scap-sync-world build #147662: 04STILL FAILING in 1 min 5 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147662/ [02:36:19] Project beta-scap-sync-world build #147663: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147663/ [02:46:23] Project beta-scap-sync-world build #147664: 04STILL FAILING in 1 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147664/ [02:56:11] Project beta-scap-sync-world build #147665: 04STILL FAILING in 1 min 2 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147665/ [03:06:18] Project beta-scap-sync-world build #147666: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147666/ [03:16:17] Project beta-scap-sync-world build #147667: 04STILL FAILING in 1 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147667/ [03:26:18] Project beta-scap-sync-world build #147668: 04STILL FAILING in 1 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147668/ [03:36:22] Project beta-scap-sync-world build #147669: 04STILL FAILING in 1 min 5 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147669/ [03:46:16] Project beta-scap-sync-world build #147670: 04STILL FAILING in 1 min 1 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147670/ [03:56:16] Project beta-scap-sync-world build #147671: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147671/ [04:06:19] Project beta-scap-sync-world build #147672: 04STILL FAILING in 1 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147672/ [04:16:17] Project beta-scap-sync-world build #147673: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147673/ [04:26:16] Project beta-scap-sync-world build #147674: 04STILL FAILING in 1 min 5 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147674/ [04:36:15] Project beta-scap-sync-world build #147675: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147675/ [04:46:18] Project beta-scap-sync-world build #147676: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147676/ [04:56:15] Project beta-scap-sync-world build #147677: 04STILL FAILING in 1 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147677/ [05:06:18] Project beta-scap-sync-world build #147678: 04STILL FAILING in 1 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147678/ [05:16:17] Project beta-scap-sync-world build #147679: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147679/ [05:26:15] Project beta-scap-sync-world build #147680: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147680/ [05:36:17] Project beta-scap-sync-world build #147681: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147681/ [05:46:22] Project beta-scap-sync-world build #147682: 04STILL FAILING in 1 min 5 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147682/ [05:56:15] Project beta-scap-sync-world build #147683: 04STILL FAILING in 1 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147683/ [06:06:17] Project beta-scap-sync-world build #147684: 04STILL FAILING in 1 min 2 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147684/ [06:08:54] (03PS1) 10Ilias Sarantopoulos: ml-services: fix huggingface pipeline triggers [integration/config] - 10https://gerrit.wikimedia.org/r/1013161 (https://phabricator.wikimedia.org/T357986) [06:16:17] Project beta-scap-sync-world build #147685: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147685/ [06:26:12] Project beta-scap-sync-world build #147686: 04STILL FAILING in 1 min 0 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147686/ [06:36:15] Project beta-scap-sync-world build #147687: 04STILL FAILING in 1 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147687/ [06:46:19] Project beta-scap-sync-world build #147688: 04STILL FAILING in 1 min 5 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147688/ [06:56:16] Project beta-scap-sync-world build #147689: 04STILL FAILING in 1 min 5 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147689/ [07:06:18] Project beta-scap-sync-world build #147690: 04STILL FAILING in 1 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147690/ [07:16:33] Project beta-scap-sync-world build #147691: 04STILL FAILING in 1 min 17 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147691/ [07:26:17] Project beta-scap-sync-world build #147692: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147692/ [07:30:10] hashar, jnuche: thats scap crashing ^, has there been a new version on beta or something change last night? [07:36:18] Project beta-scap-sync-world build #147693: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147693/ [07:46:19] Project beta-scap-sync-world build #147694: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147694/ [07:56:14] Project beta-scap-sync-world build #147695: 04STILL FAILING in 1 min 4 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147695/ [08:10:33] Project beta-scap-sync-world build #147696: 04STILL FAILING in 4 min 43 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147696/ [08:19:18] RhinosF1: no idea [08:21:54] I am sure that is https://phabricator.wikimedia.org/T260667 [08:21:56] which never got fixed [08:22:02] Project beta-scap-sync-world build #147697: 04STILL FAILING in 6 min 13 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147697/ [08:26:51] Project beta-scap-sync-world build #147698: 04STILL FAILING in 1 min 15 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147698/ [08:31:11] 10Beta-Cluster-Infrastructure, 14Release-Engineering-Team (Doing), 06Discovery-Search, 10observability, 10Wikimedia-Logstash: 14scap on beta fails canary check: KeyError: 'aggregations' - 14https://phabricator.wikimedia.org/T260667#9648277 (10hashar) 05Resolvedβ†’03Declined [08:33:58] I don't know how scap forges the logstash check command :/ [08:34:13] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team: beta-scap-sync-world fails: logstash_checker.py: KeyError: 'aggregations' - https://phabricator.wikimedia.org/T360595 (10hashar) 03NEW [08:34:16] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team: beta-scap-sync-world fails: logstash_checker.py: KeyError: 'aggregations' - https://phabricator.wikimedia.org/T360595#9648296 (10hashar) p:05Triageβ†’03Unbreak! [08:35:39] Project beta-scap-sync-world build #147699: 15ABORTED in 23 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/147699/ [08:36:57] !log Disabled https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/ # T360595 [08:36:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:36:59] T360595: beta-scap-sync-world fails: logstash_checker.py: KeyError: 'aggregations' - https://phabricator.wikimedia.org/T360595 [08:40:15] RhinosF1, hashar: there was no new scap version of scap yesterday, and there haven't been any changes in scap around `logstash_checker.py` since February [08:40:34] the problem doesn't seem to be in scap, but we'll need someone who knows more about `logstash_checker.py` to take a look at the issue [08:41:15] it is not in scap yes [08:50:49] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team: beta-scap-sync-world fails: logstash_checker.py: KeyError: 'aggregations' - https://phabricator.wikimedia.org/T360595#9648336 (10hashar) Since I could not find how scap retrieve the `logstash_host` from configuration, notably all the bits I could find... [08:51:34] so that is logging-logstash-02.logging.eqiad1.wikimedia.cloud [08:58:17] 10GitLab, 10ChangeProp, 10MediaWiki-File-management, 06SRE, 10Platform Team Initiatives (API Gateway): Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596 (10akosiaris) 03NEW [08:59:56] 10GitLab, 10ChangeProp, 06Infrastructure-Foundations, 10MediaWiki-File-management, and 3 others: Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596#9648360 (10akosiaris) [09:00:58] 10GitLab, 10ChangeProp, 06Infrastructure-Foundations, 10MediaWiki-File-management, and 4 others: Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596#9648367 (10Peachey88) [09:01:17] jnuche: do we know who that is? [09:02:19] 10GitLab, 10ChangeProp, 06Infrastructure-Foundations, 10MediaWiki-File-management, and 4 others: Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596#9648371 (10MoritzMuehlenhoff) [09:05:51] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team: beta-scap-sync-world fails: logstash_checker.py: KeyError: 'aggregations' - https://phabricator.wikimedia.org/T360595#9648374 (10hashar) I don't know where the logs are actually sent or how to check them. The link I had bookmarked https://logstash-beta... [09:06:06] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 10observability, 06SRE Observability: beta-scap-sync-world fails: logstash_checker.py: KeyError: 'aggregations' - https://phabricator.wikimedia.org/T360595#9648375 (10hashar) [09:06:17] I have added SRE observability team to it [09:06:27] cause I can't find anything that might have changed on the beta side [09:06:35] and fun thing [09:06:48] logs are apparently not sent to the logging system [09:07:00] RhinosF1: I can see the most recent changes to the script were done by dancy and mutante, maybe they can at least point in the right direction once they get online [09:07:43] hashar: thanks for adding SRE to the task [09:08:54] 10GitLab (Pipeline Services Migration🐀), 06collaboration-services: move security.wikimedia.org to kubernetes - https://phabricator.wikimedia.org/T350796#9648378 (10Arnoldokoth) @sbassett @Mstyles I added you as maintainers to the gitlab repo > https://gitlab.wikimedia.org/repos/sre/miscweb/security-landing-pag... [09:09:06] I know nothing about elastic search :/ [09:11:05] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 10observability, 06SRE Observability: beta-scap-sync-world fails: logstash_checker.py: KeyError: 'aggregations' - https://phabricator.wikimedia.org/T360595#9648389 (10hashar) [09:24:48] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 10observability, 06SRE Observability: beta-scap-sync-world fails: logstash_checker.py: KeyError: 'aggregations' - https://phabricator.wikimedia.org/T360595#9648505 (10hashar) Eventually I have found the source for the logstash host checked by the l... [09:25:41] I have disabled https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/ [09:25:45] and well [09:25:49] I am going to start working now :] [09:45:53] I have poked their team IRC channel [09:46:26] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/EventLogging] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/1013093 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [09:46:34] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/MediaSearch] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/1013089 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [09:46:42] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/TranslationNotifications] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/1013092 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [09:46:48] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/WikiLove] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/1013091 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [09:48:13] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/EventLogging] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/1013093 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [09:48:16] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/MediaSearch] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/1013089 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [09:48:18] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/TranslationNotifications] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/1013092 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [09:48:22] (03CR) 10Hashar: [C:03+2] "AH my bad I apologize I should have caught it on review! :(" [integration/config] - 10https://gerrit.wikimedia.org/r/1013161 (https://phabricator.wikimedia.org/T357986) (owner: 10Ilias Sarantopoulos) [09:49:58] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/WikiLove] Enable Sonar Codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/1013091 (https://phabricator.wikimedia.org/T321837) (owner: 10Pwangai) [09:50:07] (03Merged) 10jenkins-bot: ml-services: fix huggingface pipeline triggers [integration/config] - 10https://gerrit.wikimedia.org/r/1013161 (https://phabricator.wikimedia.org/T357986) (owner: 10Ilias Sarantopoulos) [09:50:49] !log Reloaded Zuul to enable Sonar Codehealth on EventLogging, MediaSearch, TranslationNotifications and WikiLove # T321837 [09:50:52] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:50:52] T321837: Repositories integrated into Codehealth Pipeline (Production) - https://phabricator.wikimedia.org/T321837 [09:51:28] !log Reloaded Zuul for "ml-services: fix huggingface pipeline triggers" https://gerrit.wikimedia.org/r/1013161 | T357986 [09:51:30] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:51:31] T357986: Use Huggingface model server image for HF LLMs - https://phabricator.wikimedia.org/T357986 [09:51:49] (03CR) 10Hashar: [C:03+2] "Deployed!" [integration/config] - 10https://gerrit.wikimedia.org/r/1013161 (https://phabricator.wikimedia.org/T357986) (owner: 10Ilias Sarantopoulos) [10:14:51] 10Release-Engineering-Team (Now this 🫠), 06Security-Team, 13Patch-For-Review, 07SecTeam-Processed: Notify MediaWiki security tasks as soon as an uploaded patch fails to apply - https://phabricator.wikimedia.org/T350065#9648667 (10CodeReviewBot) jnuche opened https://gitlab.wikimedia.org/repos/releng/scap/-... [10:34:29] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 10observability, 06SRE Observability: beta-scap-sync-world fails: logstash_checker.py: KeyError: 'aggregations' - https://phabricator.wikimedia.org/T360595#9648743 (10hashar) [10:38:51] 10Continuous-Integration-Config, 10Projects-Cleanup, 10Wikidata, 10Wikidata Analytics, and 2 others: Archive WMDE analytics Gerrit repositories - https://phabricator.wikimedia.org/T357697#9648750 (10brouberol) All gerrit repos have been archived! I'm not sure what will happen w.r.t. the github repos. They... [10:53:11] 10GitLab, 10ChangeProp, 06Commons, 06Infrastructure-Foundations, and 7 others: Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596#9648786 (10taavi) [10:58:20] 10Continuous-Integration-Config, 10Projects-Cleanup, 10Wikidata, 10Wikidata Analytics, and 2 others: Archive WMDE analytics Gerrit repositories - https://phabricator.wikimedia.org/T357697#9648813 (10hashar) When archiving the Gerrit repositories, we delete the mirrors in GitHub. The replication replaces... [11:07:10] 10Continuous-Integration-Config, 10Projects-Cleanup, 10Wikidata, 10Wikimedia-GitHub, and 2 others: Archive WMDE analytics Gerrit repositories - https://phabricator.wikimedia.org/T357697#9648851 (10AndrewTavis_WMDE) [11:08:06] (03PS12) 10Jforrester: Zuul: Make PHP 8.2 voting for all(*) MediaWiki extensions and skins [integration/config] - 10https://gerrit.wikimedia.org/r/1010241 (https://phabricator.wikimedia.org/T352085) [11:08:14] (03CR) 10Jforrester: [C:03+2] Zuul: Make PHP 8.2 voting for all(*) MediaWiki extensions and skins [integration/config] - 10https://gerrit.wikimedia.org/r/1010241 (https://phabricator.wikimedia.org/T352085) (owner: 10Jforrester) [11:08:30] 10Continuous-Integration-Config, 10Projects-Cleanup, 10Wikidata, 10Wikimedia-GitHub, and 2 others: Archive WMDE analytics Gerrit repositories - https://phabricator.wikimedia.org/T357697#9648855 (10AndrewTavis_WMDE) Thank you both so much! Let me know when the GitHub repos have been deleted and I'll resolve... [11:08:54] (03Merged) 10jenkins-bot: Zuul: Make PHP 8.2 voting for all(*) MediaWiki extensions and skins [integration/config] - 10https://gerrit.wikimedia.org/r/1010241 (https://phabricator.wikimedia.org/T352085) (owner: 10Jforrester) [11:12:57] !log Zuul: Make PHP 8.2 voting for all(*) MediaWiki extensions and skins for T352085 [11:13:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:13:00] T352085: Make PHP 8.2 voting on development (master) branch of MW ecosystem (core, vendor, extensions, skins, libraries) - https://phabricator.wikimedia.org/T352085 [12:30:22] James_F: so wikibase fixed it ? :) [12:30:31] hashar: No. [12:30:41] hashar: We just exempted them from the rule. [12:30:44] :D [12:30:46] (Yay messiness.) [12:31:06] We can probably go to PHP 8.*3* testing for everything else, but I want 8.2 to bed down first. :-) [13:07:17] 10Continuous-Integration-Config, 13Patch-For-Review, 07PHP 8.2 support: Make PHP 8.2 voting on development (master) branch of MW ecosystem (core, vendor, extensions, skins, libraries) - https://phabricator.wikimedia.org/T352085#9648888 (10Jdforrester-WMF) [13:15:36] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Scap: find a way to systematically update the deployment server name across all repos - https://phabricator.wikimedia.org/T197470#9648980 (10kamila) For the record, I used `sudo bash -c 'find /srv/deployment -name DEPLOY_HEAD | xargs sed -i "s/git_server: depl... [13:16:40] 10GitLab, 10ChangeProp, 06Commons, 06Infrastructure-Foundations, and 7 others: Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596#9648979 (10Reedy) https://github.com/Snapchat/KeyDB already existed as a fork. https://github.com/Snapchat/KeyDB/... [13:22:10] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 06serviceops, 06SRE, and 2 others: Migrate internal traffic to k8s - https://phabricator.wikimedia.org/T333120#9649036 (10Clement_Goubert) [13:30:32] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 10observability, 06SRE Observability: beta-scap-sync-world fails: logstash_checker.py: KeyError: 'aggregations' - https://phabricator.wikimedia.org/T360595#9649145 (10hashar) @colewhite stated a disk is full and logs aren't being ingested, suspecti... [13:36:54] 10Continuous-Integration-Config, 10Projects-Cleanup, 10Wikidata, 10Wikimedia-GitHub, and 2 others: Archive WMDE analytics Gerrit repositories - https://phabricator.wikimedia.org/T357697#9649216 (10brouberol) Hi! I'm not an admin on our Github repos, meaning I can't do anything on that front. [13:43:17] (03PS2) 10Hashar: jjb: add to mediawiki-i18n-check-docker [integration/config] - 10https://gerrit.wikimedia.org/r/1013030 (https://phabricator.wikimedia.org/T360519) (owner: 10Amire80) [13:43:25] (03CR) 10Hashar: [C:03+2] jjb: add to mediawiki-i18n-check-docker [integration/config] - 10https://gerrit.wikimedia.org/r/1013030 (https://phabricator.wikimedia.org/T360519) (owner: 10Amire80) [13:43:49] (03Merged) 10jenkins-bot: jjb: add to mediawiki-i18n-check-docker [integration/config] - 10https://gerrit.wikimedia.org/r/1013030 (https://phabricator.wikimedia.org/T360519) (owner: 10Amire80) [13:47:29] 10Continuous-Integration-Config, 10CiteThisPage, 07affects-translatewiki.net, 07I18n, and 2 others: Automatically allow tag in message translations - https://phabricator.wikimedia.org/T360519#9649329 (10Amire80) [13:47:45] 10Continuous-Integration-Config, 10CiteThisPage, 07affects-translatewiki.net, 07I18n, and 2 others: Automatically allow tag in message translations - https://phabricator.wikimedia.org/T360519#9649330 (10Amire80) [13:55:50] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Scap, 13Patch-For-Review: find a way to systematically update the deployment server name across all repos - https://phabricator.wikimedia.org/T197470#9649416 (10CodeReviewBot) hashar opened https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/256... [14:06:17] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 05Release, 05Train Deployments: 1.42.0-wmf.24 deployment blockers - https://phabricator.wikimedia.org/T360156#9649534 (10DAlangi_WMF) ##### Risky Patch! πŸš‚πŸ”₯ * **Change**: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/955771 * **Summary**: Major refactor i... [14:13:27] 10Continuous-Integration-Config, 10Projects-Cleanup, 10Wikidata, 10Wikimedia-GitHub, and 2 others: Archive WMDE analytics Gerrit repositories - https://phabricator.wikimedia.org/T357697#9649641 (10hashar) I have deleted all the repositories from the search links at T357697#9648813 :) [14:17:53] 10GerritBot, 10Phabricator, 13Patch-For-Review: GerritBot comments for 7-digit Gerrit changes conflict with Diffusion commit hashes - https://phabricator.wikimedia.org/T358940#9649663 (10hashar) The change has been deployed and prefixing the change number with {key #} should fixes the issues for new comments. [14:25:01] !log github: deleted archived repos wikimedia/operations-software-matterircd and wikimedia/operations-software-mattermost # T343707 [14:25:03] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:25:03] T343707: Migrate SRE repositories to GitLab - Archiving unused Gerrit repositories - https://phabricator.wikimedia.org/T343707 [14:52:03] 10Continuous-Integration-Config, 10Projects-Cleanup, 10Wikidata, 10Wikimedia-GitHub, and 2 others: 14Archive WMDE analytics Gerrit repositories - 14https://phabricator.wikimedia.org/T357697#9649816 (10AndrewTavis_WMDE) 05Openβ†’03Resolved a:03AndrewTavis_WMDE 14Fantastic! Thank you both again for... [15:04:54] 10Gerrit (Gerrit 3.8), 06Release-Engineering-Team, 13Patch-For-Review: Upgrade to Gerrit 3.8 - https://phabricator.wikimedia.org/T354886#9649876 (10hashar) I have [[ https://wikitech.wikimedia.org/wiki/Deployments#deploycal-item-20240325T0900 | scheduled ]] the upgrade for March 25th at 9am UTC and [[ https:... [15:19:55] 10Phabricator (Upstream), 13Patch-For-Review, 07Upstream: 14Activity pane on front page no longer shows New Tasks by default after Phorge migration - 14https://phabricator.wikimedia.org/T344835#9649898 (10valerio.bozzolan) 05Openβ†’03Resolved a:03valerio.bozzolan 14Uh! Nice homepage! This means the... [15:21:03] 10GitLab, 10Release-Engineering-Team (Radar), 10ChangeProp, 06Commons, and 8 others: Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596#9649904 (10brennen) For GitLab: I //think// we currently run the bundled Redis in their Omnibus package. In... [15:29:43] 10Phabricator: Make sure anti-vandalism features are up to snuff - https://phabricator.wikimedia.org/T84#9649973 (10Aklapper) [15:29:45] 10Phabricator: Make sure anti-vandalism features are up to snuff - https://phabricator.wikimedia.org/T84#9649976 (10Aklapper) [15:29:52] 10Phabricator (Upstream), 07Upstream: 14"Unhandled Exception: Call to a member function getAppliedTransactionPHIDs() on bool" when viewing old Herald Transcript - 14https://phabricator.wikimedia.org/T294860#9649981 (10Aklapper) 05Stalledβ†’03Resolved 14This issue should be resolved in phabricator.wikime... [15:30:05] 10Phabricator (Upstream), 07Upstream: 14RuntimeException: Array to string conversion at [/src/error/PhutilErrorHandler.php:261] - 14https://phabricator.wikimedia.org/T347354#9649969 (10Aklapper) 05Stalledβ†’03Resolved 14This issue should be resolved in phabricator.wikimedia.org since 2024-03-1... [15:30:17] 10Phabricator (Upstream), 07CSS, 07Upstream: 14Upload popup overflows box with a long filename - 14https://phabricator.wikimedia.org/T316305#9649977 (10Aklapper) 05Stalledβ†’03Resolved 14This issue should be resolved in phabricator.wikimedia.org since 2024-03-19 thanks to brennen's deployment in `T358... [15:30:21] 10Phabricator (Upstream), 07CSS, 07Upstream: 14Incorrect height of class "phui-oi-frame" - 14https://phabricator.wikimedia.org/T307492#9649979 (10Aklapper) 05Stalledβ†’03Resolved 14This issue should be resolved in phabricator.wikimedia.org since 2024-03-19 thanks to brennen's deployment in `T358610`.... [15:30:28] 10Phabricator: Make sure anti-vandalism features are up to snuff - https://phabricator.wikimedia.org/T84#9649987 (10Aklapper) [15:30:36] 10Phabricator (Upstream), 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Upstream: 14Disable "Contact Numbers" settings panel (and SMS support functionality that we do not use) - 14https://phabricator.wikimedia.org/T240109#9649988 (10Aklapper) 05Stalledβ†’03Resolved 14This issue should be resolved... [15:31:00] 10Phabricator (Upstream), 07Upstream: 14After renaming a project in Phabricator, its /tag/projectname URL ends up as a 404 - 14https://phabricator.wikimedia.org/T224662#9649992 (10Aklapper) 05Stalledβ†’03Resolved 14This issue should be resolved in phabricator.wikimedia.org since 2024-03-19 thanks to bre... [15:31:16] 10Phabricator (Upstream), 07Upstream: 14Add link to atom feed in each main blog page - 14https://phabricator.wikimedia.org/T205181#9649990 (10Aklapper) 05Stalledβ†’03Resolved 14This issue should be resolved in phabricator.wikimedia.org since 2024-03-19 thanks to brennen's deployment in `T358610`. Please... [15:31:24] 10Phabricator (Upstream), 07Upstream: 14Unhandled Exception ("RuntimeException"): Undefined offset: 5 when trying to access T16235 - 14https://phabricator.wikimedia.org/T284397#9649983 (10Aklapper) 05Stalledβ†’03Resolved 14This issue should be resolved in phabricator.wikimedia.org since 2024-03-19 thank... [15:31:34] 10Phabricator (Upstream), 07Upstream: 14Give transparent embedded image files in Phabricator a checkered background - 14https://phabricator.wikimedia.org/T78644#9649998 (10Aklapper) 05Stalledβ†’03Resolved 14This issue should be resolved in phabricator.wikimedia.org since 2024-03-19 thanks to brennen's d... [15:31:42] 10Phabricator (Upstream), 07Upstream: 14Phabricator uses different monospace fonts for inline code snippets and block code snippets - 14https://phabricator.wikimedia.org/T141494#9650000 (10Aklapper) 05Openβ†’03Resolved a:03Aklapper 14This issue should be resolved in phabricator.wikimedia.org since 202... [15:45:58] 10Release-Engineering-Team (Seen), 10Scap: 14scap checks messages appears after they have completed - 14https://phabricator.wikimedia.org/T252534#9650083 (10hashar) 05Openβ†’03Resolved a:03dancy 14So years later, the explanation is checking logstash takes less than a second, so for sure you get the `E... [15:46:52] 10Phabricator (2024-01-16), 10Release-Engineering-Team (Priority Backlog πŸ“₯): 14Fix some simple performance issues in custom maniphest.project.task.transactions Conduit API - 14https://phabricator.wikimedia.org/T353042#9650088 (10Aklapper) 05Openβ†’03Resolved [15:52:56] 10Phabricator: Decide what to do with non-performant custom Conduit API maniphest.gettasktransactions - https://phabricator.wikimedia.org/T360654 (10Aklapper) 03NEW p:05Triageβ†’03Low [16:06:18] 10Phabricator (2024-03-19), 06Release-Engineering-Team, 06collaboration-services: Update wmf/stable to Phorge upstream's 2023.49 stable release - https://phabricator.wikimedia.org/T358610#9650261 (10Aklapper) [16:09:47] 10Phabricator, 07Regression: Tag links giving 404s - https://phabricator.wikimedia.org/T360663 (10Reedy) 03NEW [16:10:07] 10Phabricator, 07Regression: Tag links giving 404s - https://phabricator.wikimedia.org/T360663#9650342 (10Reedy) [16:10:23] 10Phabricator-Bot-Requests: Phab bot request for apps GitHub PRs - https://phabricator.wikimedia.org/T360568#9650343 (10Aklapper) Hej hej, per https://www.mediawiki.org/wiki/Phabricator/Bots#Acquiring_a_bot we need an email address :) [16:10:47] 10Phabricator, 07Regression: Tag links giving 404s - https://phabricator.wikimedia.org/T360663#9650348 (10Reedy) Similarly, #scribunto is not linking anymore, even though I selected it from the pop up... [16:11:32] brennen: ^^ [16:11:46] 10Phabricator (2024-03-19), 06Release-Engineering-Team, 06collaboration-services: Update wmf/stable to Phorge upstream's 2023.49 stable release - https://phabricator.wikimedia.org/T358610#9650357 (10Reedy) It seems {T360663} might be fallput from this upgrade? [16:11:48] thx Reedy [16:11:56] It's definitely not all tags... [16:12:08] 10GerritBot, 10Phabricator: GerritBot comments for 7-digit Gerrit changes conflict with Diffusion commit hashes - https://phabricator.wikimedia.org/T358940#9650351 (10Aklapper) p:05Triageβ†’03Medium a:03Aklapper [16:12:56] 10Phabricator, 07Regression: Tag links giving 404s - https://phabricator.wikimedia.org/T360663#9650361 (10Reedy) https://phabricator.wikimedia.org/project/manage/558/ seems to work fine [16:16:28] 10Phabricator-Bot-Requests: Phab bot request for apps GitHub PRs - https://phabricator.wikimedia.org/T360568#9650371 (10Dbrant) [16:16:39] 10Phabricator-Bot-Requests: Phab bot request for apps GitHub PRs - https://phabricator.wikimedia.org/T360568#9650372 (10Dbrant) >>! In T360568#9650343, @Aklapper wrote: > Hej hej, per https://www.mediawiki.org/wiki/Phabricator/Bots#Acquiring_a_bot we need an email address :) oops sorry, added! [16:17:13] 06Release-Engineering-Team, 06Data-Engineering, 06Data-Platform-SRE, 06Discovery-Search, and 3 others: Adapt gitlab pipelines for the new wmf-jvm-parent-pom - https://phabricator.wikimedia.org/T358841#9650379 (10Gehel) After discussion, we will deploy the parent pom to Maven Central, so we will not need an... [16:18:05] 10Phabricator, 06Release-Engineering-Team, 07Regression: Tag links giving 404s - https://phabricator.wikimedia.org/T360663#9650384 (10brennen) p:05Triageβ†’03High Hmm, yeah. As an example of one that works fine: https://phabricator.wikimedia.org/tag/gitlab/ [16:18:30] 06Release-Engineering-Team, 06Data-Engineering, 06Data-Platform-SRE, 06Discovery-Search, and 3 others: Adapt gitlab pipelines for the new wmf-jvm-parent-pom - https://phabricator.wikimedia.org/T358841#9650388 (10CodeReviewBot) gehel opened https://gitlab.wikimedia.org/repos/ci-tools/wmf-jvm-parent-pom/-/me... [16:19:56] 10Phabricator, 07Regression: EXCEPTION: (PhabricatorDataNotAttachedException) Attempting to access attached data on PhabricatorProject, but the data is not actually attached - https://phabricator.wikimedia.org/T360530#9650398 (10valerio.bozzolan) Maybe related? {T360663} [16:27:27] 10Phabricator, 06Release-Engineering-Team, 07Regression: Tag links giving 404s - https://phabricator.wikimedia.org/T360663#9650451 (10Aklapper) Note that #scribunto does not even autolink anymore and shows as plain text when writing it in a comment. Direct link to project tag (though absolutely not helpful h... [16:35:27] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: gitlab.wikimedia.org ssh host key should appear in wmf-known-host - https://phabricator.wikimedia.org/T337107#9650494 (10Dzahn) There was a question in the -gitlab IRC channel if the host key had changed with a reference to this ticket.... [16:38:06] (03PS1) 10Lucas Werkmeister (WMDE): Zuul: Also make PHP 8.2 non-voting for WikibaseLexeme [integration/config] - 10https://gerrit.wikimedia.org/r/1013338 [16:38:34] heh [16:39:31] (03CR) 10CI reject: [V:04-1] Zuul: Also make PHP 8.2 non-voting for WikibaseLexeme [integration/config] - 10https://gerrit.wikimedia.org/r/1013338 (owner: 10Lucas Werkmeister (WMDE)) [16:41:24] (03CR) 10Lucas Werkmeister (WMDE): "This change is kind of half-done; I’ve tested that WikibaseLexeme `check php` is broken (see I5cc37e9873), but I assume the same is true f" [integration/config] - 10https://gerrit.wikimedia.org/r/1013338 (owner: 10Lucas Werkmeister (WMDE)) [16:43:03] (03CR) 10Lucas Werkmeister (WMDE): "Okay, I don’t understand why the test is failing, sorry :/" [integration/config] - 10https://gerrit.wikimedia.org/r/1013338 (owner: 10Lucas Werkmeister (WMDE)) [16:48:31] Why do I end up having multiple tabs with https://integration.wikimedia.org/zuul/ open [16:51:35] judging by the number of duplicate tabs i have open, i think this is a specific case of a more general problem [16:53:43] browser tab de-duplication [17:21:09] 10Phabricator-Bot-Requests: Phab bot request for apps GitHub PRs - https://phabricator.wikimedia.org/T360568#9650808 (10Aklapper) p:05Triageβ†’03Medium a:03Aklapper Hi, please see P58883 [17:25:52] that's a thing: "Duplicate Tabs Closer – Get this Extension for Firefox (en-US)" [17:25:59] Hotkey: Alt+Shift+W to close all duplicate tabs [17:41:12] 10Phabricator, 06Release-Engineering-Team, 07Regression: Tag links giving 404s - https://phabricator.wikimedia.org/T360663#9651008 (10brennen) I tried a few changes to the `gitlab` properties to see if anything there would trigger this: Extra tags, setting repo URL, mentioning {rELUA} in the description. No... [17:50:27] 10Phabricator, 06Release-Engineering-Team, 07Regression: Tag links giving 404s - https://phabricator.wikimedia.org/T360663#9651078 (10Aklapper) Note we also deployed https://we.phorge.it/rP887e344c19af9ee6f4855227f8a215a491974794 for {T224662}. DB looks fine though when running `select * from phabricator_pro... [17:56:52] 10Phabricator, 06Release-Engineering-Team, 07Regression: Tag links giving 404s - https://phabricator.wikimedia.org/T360663#9651103 (10brennen) > Note we also deployed https://we.phorge.it/rP887e344c19af9ee6f4855227f8a215a491974794 for T224662: After renaming a project in Phabricator, its /tag/projectname URL... [17:58:53] ^ ok, so #scribunto seems to not 404 any more after adding it explicitly to the list of additional tags. i think this is just an instance of the T224662 that andre mentions. [17:58:54] T224662: After renaming a project in Phabricator, its /tag/projectname URL ends up as a 404 - https://phabricator.wikimedia.org/T224662 [17:59:26] (03PS2) 10Jforrester: Zuul: Also make PHP 8.2 non-voting for WikibaseLexeme [integration/config] - 10https://gerrit.wikimedia.org/r/1013338 (owner: 10Lucas Werkmeister (WMDE)) [17:59:53] after that my head hurts, so i am going to take a brief walk, then i will see about a revert patch to try for the other outstanding phab/phorge bug. :) [18:05:04] argh argh [18:05:14] yeah I should be around another two hours looking into this [18:05:44] I'll try to reproduce locally again [18:17:48] 10Phabricator, 06Release-Engineering-Team: Tag links giving 404s - https://phabricator.wikimedia.org/T360663#9651248 (10Aklapper) See the rename in Feb2024 in https://phabricator.wikimedia.org/project/manage/558/#104250 removing the original hashtag #scribunto before we deployed https://we.phorge.it/rP887e344c... [18:18:29] yeah I think now that ^ one is NOT a problem due to our deployment but existed before. Our deployment should have actually fixed it. Which leaves us again only with the other regression [18:19:26] 10Phabricator-Bot-Requests: Phab bot request for apps GitHub PRs - https://phabricator.wikimedia.org/T360568#9651270 (10Dbrant) πŸŽ‰ Tested and working! T360680#9651258 [18:20:19] 10GerritBot, 10Phabricator: 14GerritBot comments for 7-digit Gerrit changes conflict with Diffusion commit hashes - 14https://phabricator.wikimedia.org/T358940#9651267 (10matmarex) 05Openβ†’03Resolved 14Nice, thank you! [18:22:45] 10Phabricator-Bot-Requests: 14Phab bot request for apps GitHub PRs - 14https://phabricator.wikimedia.org/T360568#9651290 (10Aklapper) 05Openβ†’03Resolved 14Yay [18:27:13] 10Phabricator (Upstream), 07Upstream: Exception: Map returned by "newPagingMapFromCursorObject()" in class "ManiphestTaskQuery" omits required key - https://phabricator.wikimedia.org/T344241#9651295 (10Aklapper) Fix landed in upstream [18:27:52] (03CR) 10Jforrester: [C:03+2] "No worries – there are two tests, one of 'production' extensions and one of 'gate' extensions. Wikibase is in both. WikibaseLexeme is only" [integration/config] - 10https://gerrit.wikimedia.org/r/1013338 (owner: 10Lucas Werkmeister (WMDE)) [18:29:06] (03Merged) 10jenkins-bot: Zuul: Also make PHP 8.2 non-voting for WikibaseLexeme [integration/config] - 10https://gerrit.wikimedia.org/r/1013338 (owner: 10Lucas Werkmeister (WMDE)) [18:30:13] !log Zuul: Also make PHP 8.2 non-voting for WikibaseLexeme [18:30:14] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:35:02] 10Phabricator, 06Release-Engineering-Team: 14Requesting Phabricator admin rights for Urbanecm - 14https://phabricator.wikimedia.org/T359557#9651380 (10thcipriani) 05Openβ†’03Resolved a:03thcipriani 14Rationale makes sense to me (only admins can check MFA)β€” @Urbanecm I made you an administrator just n... [19:15:01] 10Phabricator (Upstream), 07Upstream: 14After renaming a project in Phabricator, its /tag/projectname URL ends up as a 404 - 14https://phabricator.wikimedia.org/T224662#9651554 (10brennen) [19:15:08] 10Phabricator, 06Release-Engineering-Team: 14Tag links giving 404s - 14https://phabricator.wikimedia.org/T360663#9651552 (10brennen) β†’14Duplicate dup:03T224662 [19:23:37] 06Release-Engineering-Team, 06Data-Engineering, 06Data-Platform-SRE, 06Discovery-Search, and 3 others: Adapt gitlab pipelines for the new wmf-jvm-parent-pom - https://phabricator.wikimedia.org/T358841#9651589 (10CodeReviewBot) gehel merged https://gitlab.wikimedia.org/repos/ci-tools/wmf-jvm-parent-pom/-/me... [19:37:44] 10Phabricator, 06Release-Engineering-Team: Maniphest search for tasks not in a tag may be broken for some tags after update to wmf/2024-03-11 - https://phabricator.wikimedia.org/T360701 (10brennen) 03NEW [19:38:38] 10Phabricator (2024-03-19), 06Release-Engineering-Team, 06collaboration-services: Update wmf/stable to Phorge upstream's 2023.49 stable release - https://phabricator.wikimedia.org/T358610#9651655 (10brennen) > @brennen: How to get the default wmf/stable branch to also display what's currently on the server?... [19:41:55] 10Phabricator, 07Regression: EXCEPTION: (PhabricatorDataNotAttachedException) Attempting to access attached data on PhabricatorProject, but the data is not actually attached - https://phabricator.wikimedia.org/T360530#9651712 (10brennen) > Maybe related? That turns out to be a pre-existing issue (T224662). [19:49:32] 10Phabricator, 06Release-Engineering-Team: Maniphest search for tasks not in a tag may be broken for some tags after update to wmf/2024-03-11 - https://phabricator.wikimedia.org/T360701#9651745 (10Aklapper) > I can get that now for some tags but not for others on People, please please please always provide sp... [20:00:48] Yay, I can reproduce T360530 locally [20:00:49] T360530: EXCEPTION: (PhabricatorDataNotAttachedException) Attempting to access attached data on PhabricatorProject, but the data is not actually attached - https://phabricator.wikimedia.org/T360530 [20:01:30] 10Phabricator, 07Regression: EXCEPTION: (PhabricatorDataNotAttachedException) Attempting to access attached data on PhabricatorProject, but the data is not actually attached - https://phabricator.wikimedia.org/T360530#9651803 (10Aklapper) Alright, so what we knew: * Issue must be about a user viewing (not: edi... [20:01:58] 10Phabricator, 07Regression: "EXCEPTION: (PhabricatorDataNotAttachedException) Attempting to access attached data on PhabricatorProject, but the data is not actually attached" when username mentioned in project description - https://phabricator.wikimedia.org/T360530#9651804 (10Aklapper) [20:03:19] Yeah, no reason to roll back. [20:04:14] 10Phabricator (Upstream), 07Upstream: 14RuntimeException: Array to string conversion at [/src/error/PhutilErrorHandler.php:261] - 14https://phabricator.wikimedia.org/T347354#9651807 (10Aklapper) [20:04:21] 10Phabricator (Upstream), 13Patch-For-Review, 07Upstream: 14Activity pane on front page no longer shows New Tasks by default after Phorge migration - 14https://phabricator.wikimedia.org/T344835#9651808 (10Aklapper) [20:04:25] 10Phabricator (Upstream), 07CSS, 07Upstream: 14Upload popup overflows box with a long filename - 14https://phabricator.wikimedia.org/T316305#9651811 (10Aklapper) [20:04:27] 10Phabricator (Upstream), 07CSS, 07Upstream: 14Incorrect height of class "phui-oi-frame" - 14https://phabricator.wikimedia.org/T307492#9651812 (10Aklapper) [20:04:30] 10Phabricator (Upstream), 07Upstream: 14"Unhandled Exception: Call to a member function getAppliedTransactionPHIDs() on bool" when viewing old Herald Transcript - 14https://phabricator.wikimedia.org/T294860#9651813 (10Aklapper) [20:04:38] 10Phabricator (Upstream), 07Upstream: 14Unhandled Exception ("RuntimeException"): Undefined offset: 5 when trying to access T16235 - 14https://phabricator.wikimedia.org/T284397#9651814 (10Aklapper) [20:04:46] 10Phabricator (2024-03-19), 06Release-Engineering-Team, 06collaboration-services: 14Update wmf/stable to Phorge upstream's 2023.49 stable release - 14https://phabricator.wikimedia.org/T358610#9651805 (10Aklapper) 05Openβ†’03Resolved 14Resolving this ticket as we won't roll back. [20:04:54] 10Phabricator (Upstream), 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Upstream: 14Disable "Contact Numbers" settings panel (and SMS support functionality that we do not use) - 14https://phabricator.wikimedia.org/T240109#9651816 (10Aklapper) [20:05:03] 10Phabricator (Upstream), 07Upstream: 14After renaming a project in Phabricator, its /tag/projectname URL ends up as a 404 - 14https://phabricator.wikimedia.org/T224662#9651817 (10Aklapper) [20:05:11] 10Phabricator (Upstream), 07Upstream: 14Add link to atom feed in each main blog page - 14https://phabricator.wikimedia.org/T205181#9651818 (10Aklapper) [20:05:19] 10Phabricator (Upstream), 07Upstream: 14Give transparent embedded image files in Phabricator a checkered background - 14https://phabricator.wikimedia.org/T78644#9651820 (10Aklapper) [20:05:27] 10Phabricator (Upstream), 07Upstream: 14Phabricator uses different monospace fonts for inline code snippets and block code snippets - 14https://phabricator.wikimedia.org/T141494#9651819 (10Aklapper) [20:08:14] andre: nice job on the repro [20:10:00] for F's sake, interesting one. Okay, added reproduction steps to https://phabricator.wikimedia.org/T360530 now [20:10:12] 10Phabricator (Upstream), 07Regression: "EXCEPTION: (PhabricatorDataNotAttachedException) Attempting to access attached data on PhabricatorProject, but the data is not actually attached" when username mentioned in project description - https://phabricator.wikimedia.org/T360530#9651845 (10Aklapper) [20:10:31] 10Continuous-Integration-Config, 10Math, 07PHP 8.2 support: Make PHP 8.2 voting on development (master) branch of Math extension - https://phabricator.wikimedia.org/T360709 (10Jdforrester-WMF) 03NEW [20:10:56] ...after trying with view-restricted tasks and view-restricted projects locally for an hour (all unrelated), sigh [20:16:12] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/Math] Disable PHP 8.2 testing for now [integration/config] - 10https://gerrit.wikimedia.org/r/1013399 (https://phabricator.wikimedia.org/T360709) [20:16:14] (03PS1) 10Jforrester: Zuul: [Wikibase*] Disable PHP 8.2 testing for more Wikibase repos [integration/config] - 10https://gerrit.wikimedia.org/r/1013400 (https://phabricator.wikimedia.org/T360560) [20:17:38] (03CR) 10CI reject: [V:04-1] Zuul: [Wikibase*] Disable PHP 8.2 testing for more Wikibase repos [integration/config] - 10https://gerrit.wikimedia.org/r/1013400 (https://phabricator.wikimedia.org/T360560) (owner: 10Jforrester) [20:18:00] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/Math] Disable PHP 8.2 testing for now [integration/config] - 10https://gerrit.wikimedia.org/r/1013399 (https://phabricator.wikimedia.org/T360709) (owner: 10Jforrester) [20:19:28] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/Math] Disable PHP 8.2 testing for now [integration/config] - 10https://gerrit.wikimedia.org/r/1013399 (https://phabricator.wikimedia.org/T360709) (owner: 10Jforrester) [20:20:03] !log Zuul: [mediawiki/extensions/Math] Disable PHP 8.2 testing for now, for T360709 [20:20:07] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:20:08] T360709: Make PHP 8.2 voting on development (master) branch of Math extension - https://phabricator.wikimedia.org/T360709 [20:20:36] (03PS2) 10Jforrester: Zuul: [Wikibase*] Disable PHP 8.2 testing for more Wikibase repos [integration/config] - 10https://gerrit.wikimedia.org/r/1013400 (https://phabricator.wikimedia.org/T360560) [20:20:54] 10Phabricator (Upstream), 07Regression, 07Upstream: PhabricatorDataNotAttachedException when rendering project hovercard with username mentioned in project description - https://phabricator.wikimedia.org/T360530#9651904 (10Aklapper) [20:21:10] 10Phabricator (Upstream), 07Regression, 07Upstream: PhabricatorDataNotAttachedException when rendering project hovercard with username mentioned in project description - https://phabricator.wikimedia.org/T360530#9651912 (10Aklapper) [20:21:39] (03CR) 10CI reject: [V:04-1] Zuul: [Wikibase*] Disable PHP 8.2 testing for more Wikibase repos [integration/config] - 10https://gerrit.wikimedia.org/r/1013400 (https://phabricator.wikimedia.org/T360560) (owner: 10Jforrester) [20:25:05] (03PS3) 10Jforrester: Zuul: [Wikibase*] Disable PHP 8.2 testing for more Wikibase repos [integration/config] - 10https://gerrit.wikimedia.org/r/1013400 (https://phabricator.wikimedia.org/T360560) [20:26:30] (03CR) 10Jforrester: [C:03+2] Zuul: [Wikibase*] Disable PHP 8.2 testing for more Wikibase repos [integration/config] - 10https://gerrit.wikimedia.org/r/1013400 (https://phabricator.wikimedia.org/T360560) (owner: 10Jforrester) [20:27:45] (03Merged) 10jenkins-bot: Zuul: [Wikibase*] Disable PHP 8.2 testing for more Wikibase repos [integration/config] - 10https://gerrit.wikimedia.org/r/1013400 (https://phabricator.wikimedia.org/T360560) (owner: 10Jforrester) [20:28:22] 10GitLab, 10Release-Engineering-Team (Radar), 10ChangeProp, 06Commons, and 9 others: Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596#9651946 (10Krinkle) [20:28:33] !log Zuul: [Wikibase*] Disable PHP 8.2 testing for more Wikibase repos, for T360560 [20:28:35] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:28:35] T360560: Make PHP 8.2 voting on development branch of Wikibase - https://phabricator.wikimedia.org/T360560 [20:36:00] 10GitLab, 10Release-Engineering-Team (Radar), 10ChangeProp, 06Infrastructure-Foundations, and 8 others: Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596#9651962 (10Peachey88) [20:38:24] 10Phabricator, 07Regression: PhabricatorDataNotAttachedException in PhabricatorRepositoryCommit (via getRepository()) - https://phabricator.wikimedia.org/T360714 (10Aklapper) 03NEW [21:09:02] 10GitLab (Pipeline Services Migration🐀), 06collaboration-services: move security.wikimedia.org to kubernetes - https://phabricator.wikimedia.org/T350796#9652078 (10sbassett) >>! In T350796#9648378, @Arnoldokoth wrote: > @sbassett @Mstyles I added you as maintainers to the gitlab repo > https://gitlab.wikimedia... [21:14:22] 10GitLab, 10Release-Engineering-Team (Radar), 10ChangeProp, 06Infrastructure-Foundations, and 8 others: Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596#9652082 (10Krinkle) In MediaWiki (as deployed at WMF), there exists 1 use of Redis, whic... [21:26:09] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 13Patch-For-Review, 05Release, 05Train Deployments: 1.42.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T354441#9652153 (10dancy) I'm rolling the train back to group1 while T360717 is being investigated. [21:35:10] 10GitLab-Application-Security-Pipeline, 06Security-Team, 07SecTeam-Processed, 07Security: Update the image used by the python-bandit appsec include - https://phabricator.wikimedia.org/T360721 (10sbassett) 03NEW [21:39:59] 10GerritBot, 10Phabricator: 14GerritBot comments for 7-digit Gerrit changes conflict with Diffusion commit hashes - 14https://phabricator.wikimedia.org/T358940#9652185 (10Umherirrender) 14QA - works good: * T360622#9649224 has linking, posted before the patch * T360622#9651718 has no linking for the same... [22:02:58] 10GitLab, 10Release-Engineering-Team (Radar), 10ChangeProp, 06Infrastructure-Foundations, and 8 others: Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596#9652274 (10Ladsgroup) >>! In T360596#9652082, @Krinkle wrote: > In MediaWiki (as deploye... [22:07:41] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 13Patch-For-Review, 05Release, 05Train Deployments: 1.42.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T354441#9652292 (10Ladsgroup) >>! In T354441#9652153, @dancy wrote: > I'm rolling the train back to group1 while T360717 is being invest... [23:32:49] 10GitLab, 10Release-Engineering-Team (Radar), 10ChangeProp, 06Infrastructure-Foundations, and 8 others: Figure out a plan to move forward with regarding Redis License changes - https://phabricator.wikimedia.org/T360596#9652472 (10bd808) [23:57:22] 10Scap: Ask for a commit/change summary in scap train if one not provided? - https://phabricator.wikimedia.org/T360729 (10Reedy) 03NEW [23:57:29] 10Scap: Ask for a commit/change summary in scap train if one not provided? - https://phabricator.wikimedia.org/T360729#9652500 (10Reedy) p:05Triageβ†’03Low