[06:44:41] 10GitLab: Internal Server Error when publishing a maven package from GitLab CI - https://phabricator.wikimedia.org/T434834#12224872 (10ABran-WMF) upon investigation, it seems you were hitting a `QuotaExceeded` limit on Apus: `api error QuotaExceeded: UnknownError"` [06:44:57] 10GitLab: Internal Server Error when publishing a maven package from GitLab CI - https://phabricator.wikimedia.org/T434834#12224873 (10ABran-WMF) [06:45:00] 10GitLab (Infrastructure), 06Collaboration-Services: Make sure GitLab does not exceed apus object storage quotas - https://phabricator.wikimedia.org/T413871#12224874 (10ABran-WMF) [06:45:11] 10GitLab, 06Collaboration-Services: Internal Server Error when publishing a maven package from GitLab CI - https://phabricator.wikimedia.org/T434834#12224876 (10ABran-WMF) [07:56:24] Gitlab needs a short maintenance in about 20 minutes. ( T435157 ) [08:27:01] Maintenance is complete. [09:07:54] 10GitLab, 06Collaboration-Services: Internal Server Error when publishing a maven package from GitLab CI - https://phabricator.wikimedia.org/T434834#12225358 (10Don-vip) Thanks, I have added a package prune mechanism in my pipeline, it solved the issue: https://gitlab.wikimedia.org/toolforge-repos/spacemedia/... [11:49:13] 10GitLab (Infrastructure), 06Collaboration-Services: Make sure GitLab does not exceed apus object storage quotas - https://phabricator.wikimedia.org/T413871#12225962 (10ABran-WMF) current quota usage: ` arnaudb@gitlab1004:~ $ s3cmd du -H s3://gitlab-packages/ 286G 5231 objects s3://gitlab-packages/ arnaudb... [11:49:29] 10GitLab (Infrastructure), 06Collaboration-Services: Make sure GitLab does not exceed apus object storage quotas - https://phabricator.wikimedia.org/T413871#12225963 (10ABran-WMF) [11:51:10] 10GitLab, 06Collaboration-Services: Internal Server Error when publishing a maven package from GitLab CI - https://phabricator.wikimedia.org/T434834#12225965 (10LSobanski) 05Open→03Resolved a:03ABran-WMF Good to see the quotas working, this looks OK to resolve. [11:57:09] 10GitLab, 06Collaboration-Services: Internal Server Error when publishing a maven package from GitLab CI - https://phabricator.wikimedia.org/T434834#12225983 (10ABran-WMF) Thanks @Don-vip, you might free up to 35GB in the package bucket: ` repos/machine-learning/ml-pipelines... [12:33:21] 10GitLab (Project Migration), 06Collaboration-Services, 10Observability-Alerting: Migrate SRE repositories to GitLab - operations/alerts - https://phabricator.wikimedia.org/T349626#12226143 (10LSobanski) 05Stalled→03Declined This work is no longer actively tracked. [12:33:29] 10GitLab (Project Migration), 06Collaboration-Services: Migrate SRE repositories to GitLab - operations/software - https://phabricator.wikimedia.org/T341504#12226149 (10LSobanski) 05Stalled→03Declined This work is no longer actively tracked. [12:34:10] 10GitLab (Project Migration), 06Collaboration-Services: Migrate SRE repositories to GitLab - https://phabricator.wikimedia.org/T341468#12226152 (10LSobanski) 05Stalled→03Declined This work is no longer actively tracked. [15:12:04] 10GitLab (CI & Job Runners), 13Patch-For-Review: gitlab-runners: WMCS fleet became ~6.5x slower at BuildKit filesystem operations overnight on 8/9 July 2026 - https://phabricator.wikimedia.org/T433505#12226965 (10Jdforrester-WMF) 05Open→03Resolved a:03dancy Update: Yes, the WCMS runners are fixed, th... [16:37:51] 10GitLab (Project Migration), 10CopyPatrol: Migrate CopyPatrol repo from GitHub to GitLab - https://phabricator.wikimedia.org/T395401#12227476 (10MusikAnimal)