[01:47:32] FIRING: PuppetAgentStaleLastRun: Last Puppet run was over 24 hours ago on instance deployment-echostore02 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentStaleLastRun [06:33:16] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Cloud-Services, 06collaboration-services: Zuul executor can not reach WMCS OpenStack API - https://phabricator.wikimedia.org/T430479 (10hashar) 03NEW The #Cloud-Services project tag is not intended to have any tasks. Please check the list on https:/... [06:34:46] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06cloud-services-team, 06collaboration-services, 10Data-Services: Zuul executor can not reach WMCS OpenStack API - https://phabricator.wikimedia.org/T430479#12063721 (10hashar) [06:35:16] 10Continuous-Integration-Infrastructure (Zuul upgrade): upload-logs-swift role fails due to ModuleNotFoundError: No module named 'requestsexceptions' - https://phabricator.wikimedia.org/T430458#12063723 (10hashar) 05Open→03Resolved Fixed after I have synced our integration/zuul-jobs fork with upstream.... [06:41:39] (03CR) 10Hashar: [C:03+2] "> PageImages is a soft runtime dependency (guarded by ExtensionRegistry)," [integration/config] - 10https://gerrit.wikimedia.org/r/1305998 (owner: 10Eric Gardner) [06:43:22] (03Merged) 10jenkins-bot: Add PageImages to MultimediaViewer's phan (and test) dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/1305998 (owner: 10Eric Gardner) [06:45:17] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/Chart] Add Scribunto dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1305909 (https://phabricator.wikimedia.org/T430324) (owner: 10Aude) [06:47:01] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/Chart] Add Scribunto dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1305909 (https://phabricator.wikimedia.org/T430324) (owner: 10Aude) [07:07:35] 06Project-Admins: Create project tag for Open Data Ivoir - https://phabricator.wikimedia.org/T430467#12063771 (10Aklapper) 05Open→03Declined Hi, I am not sure why you reopened this. Please see T430467#12063510 and create the project tag - thank you. :) Tu peux creer le Phab projet toi-meme; il y a pas d... [07:10:45] 10Phabricator, 06Release-Engineering-Team (Doing 😎): Rename "Nekomoto" Phabricator account to "Vulpomoto" - https://phabricator.wikimedia.org/T430472#12063781 (10Aklapper) 05Open→03Resolved a:03Aklapper Done. [07:30:34] 06Project-Admins, 10Reconciliation: Reconciliation project - Subproject to manage reconciliation service - https://phabricator.wikimedia.org/T426541#12063864 (10Aklapper) 05Stalled→03Open If you are really sure that [all current members of Reconciliation](https://phabricator.wikimedia.org/project/members/5... [07:42:15] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06cloud-services-team, 06collaboration-services, 10Data-Services: Zuul executor can not reach WMCS OpenStack API - https://phabricator.wikimedia.org/T430479#12063909 (10hashar) From zuul1002.eqiad.wmnet I can reproduce using curl. I have tried two app... [08:16:38] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06cloud-services-team, 06collaboration-services, 10Data-Services: Zuul executor can not reach WMCS OpenStack API - https://phabricator.wikimedia.org/T430479#12064055 (10hashar) If I try https://openstack.eqiad1.wikimediacloud.org:25000/ from contint10... [08:19:33] 06Release-Engineering-Team, 06collaboration-services: ProbeDown - https://phabricator.wikimedia.org/T430421#12064076 (10ABran-WMF) [08:19:41] 06Release-Engineering-Team, 06collaboration-services: ProbeDown - https://phabricator.wikimedia.org/T430421#12064081 (10ABran-WMF) [08:19:46] 06Release-Engineering-Team, 06collaboration-services: ProbeDown - https://phabricator.wikimedia.org/T430421#12064086 (10ABran-WMF) [08:20:50] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06cloud-services-team, 06collaboration-services, 10Data-Services: Zuul executor can not reach WMCS OpenStack API - https://phabricator.wikimedia.org/T430479#12064092 (10hashar) Using webproxy (rather than url-downloader), the tunneling is rejected: `... [08:20:52] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services: allow new zuul executor VMs in prod to talk to cloud VPS - https://phabricator.wikimedia.org/T394838#12064095 (10hashar) [08:20:57] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06cloud-services-team, 06collaboration-services, 10Data-Services: Zuul executor can not reach WMCS OpenStack API - https://phabricator.wikimedia.org/T430479#12064096 (10hashar) [08:27:13] 06Release-Engineering-Team, 06collaboration-services: ProbeDown - gerrit2003:29418 has failed probes (tcp_gerrit_ssh_ip4) - https://phabricator.wikimedia.org/T430421#12064124 (10ABran-WMF) [08:27:16] 06Release-Engineering-Team, 06collaboration-services: ProbeDown - gerrit2003 - https://phabricator.wikimedia.org/T430421#12064125 (10LSobanski) [08:29:44] 10Continuous-Integration-Infrastructure (Zuul upgrade): Artifact storage for updated Zuul CI - https://phabricator.wikimedia.org/T397385#12064130 (10hashar) [08:29:51] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06cloud-services-team, 06collaboration-services, 10Data-Services: Zuul executor can not reach WMCS OpenStack API - https://phabricator.wikimedia.org/T430479#12064131 (10hashar) [08:43:37] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06cloud-services-team, 06collaboration-services, 10Data-Services, 13Patch-For-Review: Zuul executor can not reach WMCS OpenStack API - https://phabricator.wikimedia.org/T430479#12064239 (10hashar) a:03hashar [08:51:43] 10Gerrit, 06collaboration-services, 13Patch-For-Review: Gerrit backups are growing - https://phabricator.wikimedia.org/T411583#12064312 (10ABran-WMF) I guess before reducing granularity, we could try and exclude [[ https://gerrit.wikimedia.org/r/1306164 | caches ]] and [[ https://gerrit.wikimedia.org/r/13061... [09:10:55] 10Phabricator, 06Release-Engineering-Team (Priority Backlog 📥), 07Essential-Work, 13Patch-For-Review: Remove "Deadline" task subtype - https://phabricator.wikimedia.org/T335395#12064387 (10Aklapper) [09:25:08] 06Release-Engineering-Team, 06collaboration-services: ProbeDown - gerrit2003 - https://phabricator.wikimedia.org/T430421#12064407 (10ABran-WMF) a:03ABran-WMF [09:28:29] 06Release-Engineering-Team, 06collaboration-services: ProbeDown - gerrit2003 - https://phabricator.wikimedia.org/T430421#12064417 (10ABran-WMF) Gerrit was [[ https://grafana-rw.wikimedia.org/d/nX8li17Sk/gerrit-overview-sre-collab?forceLogin=true&from=2026-06-28T01:41:45.088Z&orgId=1&refresh=1m&timezone=utc&to=...