[01:26:18] <wm-bot2>	 !log tools.cluebotng-monitoring Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41947942541 (https://github.com/cluebotng/component-configs/commits/fb51e734553eadc617de48a568da790fd164e106)
[01:26:20] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL
[02:39:50] <wikibugs>	 (03CR) 10AntiCompositeNumber: [C:03+2] "Looks like a reasonable initial implementation to test the feature." [labs/tools/stewardbots] - 10https://gerrit.wikimedia.org/r/1361544 (owner: 10Majavah)
[02:40:49] <wikibugs>	 (03CR) 10AntiCompositeNumber: [C:03+2] "Looks like a reasonable first implementation to test the feature." [labs/tools/stewardbots] - 10https://gerrit.wikimedia.org/r/1361543 (owner: 10Majavah)
[02:41:59] <wikibugs>	 (03Merged) 10jenkins-bot: StewardBot: Set bot mode [labs/tools/stewardbots] - 10https://gerrit.wikimedia.org/r/1361543 (owner: 10Majavah)
[02:42:24] <wikibugs>	 (03Merged) 10jenkins-bot: SULWatcher: Set bot mode [labs/tools/stewardbots] - 10https://gerrit.wikimedia.org/r/1361544 (owner: 10Majavah)
[02:57:11] <wikibugs>	 10Tool-k8s-status: k8s-status no healthy upstream - https://phabricator.wikimedia.org/T443750 (10AntiCompositeNumber) 03NEW
[03:28:37] <wm-bot2>	 !log tools.cluebotng-trainer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41961409257 (https://github.com/cluebotng/component-configs/commits/f392ff08eb9768ab93538b5d6ed5cdf5306b9e50)
[03:28:39] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL
[03:29:12] <wm-bot2>	 !log tools.cluebotng Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41961409161 (https://github.com/cluebotng/component-configs/commits/f392ff08eb9768ab93538b5d6ed5cdf5306b9e50)
[03:29:13] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL
[03:29:19] <wm-bot2>	 !log tools.cluebotng-review Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41961409152 (https://github.com/cluebotng/component-configs/commits/f392ff08eb9768ab93538b5d6ed5cdf5306b9e50)
[03:29:20] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL
[03:30:00] <jinxer-wm>	 FIRING: [2x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse
[04:00:00] <jinxer-wm>	 FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse
[04:55:31] <wm-bot2>	 !log tools.cluebotng Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41970293977 (https://github.com/cluebotng/component-configs/commits/249d7f9d3e64c31431d63973b2e535b32af4e788)
[04:55:33] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL
[04:57:12] <wm-bot2>	 !log tools.cluebotng-review Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41970294033 (https://github.com/cluebotng/component-configs/commits/249d7f9d3e64c31431d63973b2e535b32af4e788)
[04:57:13] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL
[05:30:12] <wm-bot2>	 !log tools.cluebotng-trainer Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41974300499 (https://github.com/cluebotng/component-configs/commits/62ff28a67ca66036c9cb17267fa8a2cff7a5b641)
[05:30:14] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL
[05:30:14] <wm-bot2>	 !log tools.cluebot3 Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41974300400 (https://github.com/cluebotng/component-configs/commits/62ff28a67ca66036c9cb17267fa8a2cff7a5b641)
[05:30:15] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot3/SAL
[05:30:23] <wm-bot2>	 !log tools.cluebotng-editsets Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41974300317 (https://github.com/cluebotng/component-configs/commits/62ff28a67ca66036c9cb17267fa8a2cff7a5b641)
[05:30:23] <wm-bot2>	 !log tools.cluebotng-monitoring Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41974300597 (https://github.com/cluebotng/component-configs/commits/62ff28a67ca66036c9cb17267fa8a2cff7a5b641)
[05:30:24] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-editsets/SAL
[05:30:24] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL
[05:32:10] <wm-bot2>	 !log tools.toolforge-functional-runner Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41974300533 (https://github.com/cluebotng/component-configs/commits/62ff28a67ca66036c9cb17267fa8a2cff7a5b641)
[05:32:10] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.toolforge-functional-runner/SAL
[05:32:37] <wm-bot2>	 !log tools.cluebot-syncer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41974300416 (https://github.com/cluebotng/component-configs/commits/62ff28a67ca66036c9cb17267fa8a2cff7a5b641)
[05:32:37] <wm-bot2>	 !log tools.cluebotng-staging Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41974300445 (https://github.com/cluebotng/component-configs/commits/62ff28a67ca66036c9cb17267fa8a2cff7a5b641)
[05:32:38] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot-syncer/SAL
[05:32:38] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-staging/SAL
[05:33:42] <wm-bot2>	 !log tools.cluebotng Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41974300376 (https://github.com/cluebotng/component-configs/commits/62ff28a67ca66036c9cb17267fa8a2cff7a5b641)
[05:33:43] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL
[05:34:38] <wm-bot2>	 !log tools.cluebotng-review Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41974300319 (https://github.com/cluebotng/component-configs/commits/62ff28a67ca66036c9cb17267fa8a2cff7a5b641)
[05:34:39] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL
[05:36:27] <wm-bot2>	 !log tools.cluebotng-staging Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41974543465 (https://github.com/cluebotng/component-configs/commits/7f5f130ad27cc6b58d18dc7db9fb6f17a994795f)
[05:36:29] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-staging/SAL
[05:37:28] <wm-bot2>	 !log tools.cluebotng-trainer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41974709611 (https://github.com/cluebotng/component-configs/commits/5efbb82f84986437a20d11414253c6eb8926c8d2)
[05:37:29] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL
[05:39:05] <wm-bot2>	 !log tools.cluebotng Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41974543482 (https://github.com/cluebotng/component-configs/commits/7f5f130ad27cc6b58d18dc7db9fb6f17a994795f)
[05:39:06] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL
[05:50:54] <wm-bot2>	 !log tools.cluebotng-review Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41976470115 (https://github.com/cluebotng/component-configs/commits/dfe8bea82a81a0c7ff97c707693c7eff5fc01658)
[05:50:56] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL
[05:52:56] <wm-bot2>	 !log tools.cluebotng-staging Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41976470140 (https://github.com/cluebotng/component-configs/commits/dfe8bea82a81a0c7ff97c707693c7eff5fc01658)
[05:52:57] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-staging/SAL
[05:54:15] <wm-bot2>	 !log tools.cluebotng Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/41976470110 (https://github.com/cluebotng/component-configs/commits/dfe8bea82a81a0c7ff97c707693c7eff5fc01658)
[05:54:16] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL
[05:54:53] <wikibugs>	 10Tool-campwiz-nxt: Migration of CampWiz NXT to toolforge - https://phabricator.wikimedia.org/T394515#12520980 (10Nokib_Sarkar) 05In progress→03Resolved
[06:02:27] <wm-bot2>	 !log tools.cluebotng-monitoring Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41977287826 (https://github.com/cluebotng/component-configs/commits/26aeb742e54365acb8d07e0d79767159175ed5ca)
[06:02:29] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL
[06:10:57] <wm-bot2>	 !log tools.cluebotng-monitoring Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41978117359 (https://github.com/cluebotng/component-configs/commits/445aed807c70e37fa8e4e9270743b99bd94a2c3c)
[06:10:59] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL
[06:14:12] <wm-bot2>	 !log tools.cluebotng-review Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41978491372 (https://github.com/cluebotng/component-configs/commits/c13b42c494b46b9b47c56144b4bfef621ce4246b)
[06:14:13] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL
[06:14:52] <wm-bot2>	 !log tools.cluebotng-monitoring Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41978385836 (https://github.com/cluebotng/component-configs/commits/518d18724ca549296a5ff755fa4e6a576a8c366c)
[06:14:53] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL
[06:40:20] <wm-bot2>	 !log tools.cluebotng-monitoring Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/41981558450 (https://github.com/cluebotng/component-configs/commits/23de2fc566aa0abfe228312ddd0b1e87b95990eb)
[06:40:22] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL
[07:19:14] <wikibugs>	 (03open) 10rluts: Add support for skipping "in use" pages with template-based detection [toolforge-repos/findsmith] - 10https://gitlab.wikimedia.org/toolforge-repos/findsmith/-/merge_requests/10
[07:41:38] <wmcs-alerts>	 FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[07:46:38] <wmcs-alerts>	 RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[08:00:00] <jinxer-wm>	 FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse
[08:13:22] <wikibugs>	 10Data-Services, 06tools-platform-team, 06DBA: Decommission clouddb1013-clouddb1020 - https://phabricator.wikimedia.org/T434048#12521130 (10Marostegui)
[08:20:09] <wikibugs>	 (03merge) 10rluts: Add support for skipping "in use" pages with template-based detection [toolforge-repos/findsmith] - 10https://gitlab.wikimedia.org/toolforge-repos/findsmith/-/merge_requests/10
[08:20:57] <wikibugs>	 (03open) 10rluts: Bump version to alpha.4 and update deps [toolforge-repos/findsmith] - 10https://gitlab.wikimedia.org/toolforge-repos/findsmith/-/merge_requests/11
[08:21:13] <wikibugs>	 (03merge) 10rluts: Bump version to alpha.4 and update deps [toolforge-repos/findsmith] - 10https://gitlab.wikimedia.org/toolforge-repos/findsmith/-/merge_requests/11
[08:42:14] <wikibugs>	 10Data-Services, 06tools-platform-team, 06DBA: Decommission clouddb1013-clouddb1020 - https://phabricator.wikimedia.org/T434048#12521222 (10Marostegui)
[08:42:22] <wikibugs>	 10Data-Services, 06tools-platform-team, 06DBA: Decommission clouddb1013-clouddb1020 - https://phabricator.wikimedia.org/T434048#12521225 (10Marostegui)
[09:01:52] <wikibugs>	 06tools-infrastructure-team, 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Test Platform: Add more CPUs to quibble jobs - https://phabricator.wikimedia.org/T439808#12521262 (10fgiunchedi) >>! In T439808#12414727, @Peter wrote: > Lets add the instances when T440670 is done so the ne...
[11:06:09] <wikibugs>	 10Tool-techcontribs: add some stats for mediawiki wiki and wikitech - https://phabricator.wikimedia.org/T443775 (10Novem_Linguae) 03NEW
[11:14:36] <wikibugs>	 10Tool-commons-delinquent: Tool-commons-delinquent: HTTP Code 500 error - https://phabricator.wikimedia.org/T437416#12521754 (10grin) 05Open→03Resolved a:03grin The webservice was running without running, possibly related to the php7.4 it's been using since before my time.  Since it magically seems to...
[11:18:05] <wikibugs>	 10Quarry, 06tools-platform-team: [quarry] started failing when results directory does not exist - https://phabricator.wikimedia.org/T443411#12521763 (10github-toolforge-bot) dhinus closed https://github.com/toolforge/quarry/pull/113
[11:53:20] <wikibugs>	 10Toolforge, 06tools-platform-team: [builds-api] latest image is missing from toolsbeta registry - https://phabricator.wikimedia.org/T443777 (10DamianZaremba) 03NEW
[12:00:00] <jinxer-wm>	 FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse
[12:01:10] <wikibugs>	 10Tool-techcontribs: add some stats for mediawiki wiki and wikitech - https://phabricator.wikimedia.org/T443775#12521854 (10KomalPandey4) Hi @Novem_Linguae! I would like to work on this task. Could you confirm if this is open to be picked up and point me to the codebase repository? Thanks
[12:07:29] <wikibugs>	 (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [labs/tools/commons-mass-description] - 10https://gerrit.wikimedia.org/r/1364265 (owner: 10L10n-bot)
[12:10:15] <wikibugs>	 (03open) 10raymond-ndibe: build: stop argument injection via source_url and ref [repos/cloud/toolforge/builds-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/merge_requests/171 (https://phabricator.wikimedia.org/T442299)
[12:12:12] <wikibugs>	 10Tool-techcontribs: add some stats for mediawiki wiki and wikitech - https://phabricator.wikimedia.org/T443775#12521879 (10Novem_Linguae) You can click on the Tools-techcontribs tag and visit the homepage of that tag to see a link to the repo. https://phabricator.wikimedia.org/project/profile/7381/  I'll let @C...
[12:14:30] <wikibugs>	 (03update) 10raymond-ndibe: build: stop argument injection via source_url and ref [repos/cloud/toolforge/builds-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/merge_requests/171 (https://phabricator.wikimedia.org/T442299)
[12:14:37] <wikibugs>	 (03approved) 10raymond-ndibe: build: stop argument injection via source_url and ref [repos/cloud/toolforge/builds-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/merge_requests/171 (https://phabricator.wikimedia.org/T442299)
[12:14:38] <wikibugs>	 (03update) 10raymond-ndibe: build: stop argument injection via source_url and ref [repos/cloud/toolforge/builds-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/merge_requests/171 (https://phabricator.wikimedia.org/T442299)
[12:15:03] <wikibugs>	 (03update) 10renovatebot: Lock file maintenance [toolforge-repos/namehistory] - 10https://gitlab.wikimedia.org/toolforge-repos/namehistory/-/merge_requests/39
[12:15:06] <wikibugs>	 (03update) 10renovatebot: Lock file maintenance [toolforge-repos/namehistory] - 10https://gitlab.wikimedia.org/toolforge-repos/namehistory/-/merge_requests/39
[12:15:16] <wikibugs>	 (03update) 10renovatebot: Lock file maintenance [toolforge-repos/allblocksever] - 10https://gitlab.wikimedia.org/toolforge-repos/allblocksever/-/merge_requests/48
[12:15:19] <wikibugs>	 (03update) 10renovatebot: Lock file maintenance [toolforge-repos/allblocksever] - 10https://gitlab.wikimedia.org/toolforge-repos/allblocksever/-/merge_requests/48
[12:15:28] <wikibugs>	 (03update) 10renovatebot: Lock file maintenance [toolforge-repos/checkusertools] - 10https://gitlab.wikimedia.org/toolforge-repos/checkusertools/-/merge_requests/46
[12:15:29] <wikibugs>	 (03update) 10renovatebot: Lock file maintenance [toolforge-repos/checkusertools] - 10https://gitlab.wikimedia.org/toolforge-repos/checkusertools/-/merge_requests/46
[12:16:38] <wikibugs>	 (03update) 10renovatebot: Lock file maintenance [toolforge-repos/rangetree] - 10https://gitlab.wikimedia.org/toolforge-repos/rangetree/-/merge_requests/76
[12:16:41] <wikibugs>	 (03update) 10renovatebot: Lock file maintenance [toolforge-repos/rangetree] - 10https://gitlab.wikimedia.org/toolforge-repos/rangetree/-/merge_requests/76
[12:16:48] <wikibugs>	 (03open) 10raymond-ndibe: runtime: stop argument injection via the build ref [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/214 (https://phabricator.wikimedia.org/T443554)
[12:16:55] <wikibugs>	 (03approved) 10raymond-ndibe: runtime: stop argument injection via the build ref [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/214 (https://phabricator.wikimedia.org/T443554)
[12:16:56] <wikibugs>	 (03update) 10raymond-ndibe: runtime: stop argument injection via the build ref [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/214 (https://phabricator.wikimedia.org/T443554)
[12:18:10] <wikibugs>	 10Toolforge, 06tools-platform-team: [builds-api] latest image is missing from toolsbeta registry - https://phabricator.wikimedia.org/T443777#12521898 (10Raymond_Ndibe) @DamianZaremba this is me deploying a security fix. It should be back up soon
[12:18:23] <wikibugs>	 10Toolforge, 06tools-platform-team: [builds-api] latest image is missing from toolsbeta registry - https://phabricator.wikimedia.org/T443777#12521899 (10Raymond_Ndibe) maybe 5 - 10mins
[12:18:38] <wikibugs>	 (03update) 10renovatebot: Lock file maintenance [toolforge-repos/wiki-mail-verify] - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-mail-verify/-/merge_requests/65
[12:18:41] <wikibugs>	 (03update) 10renovatebot: Lock file maintenance [toolforge-repos/wiki-mail-verify] - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-mail-verify/-/merge_requests/65
[12:20:21] <wikibugs>	 (03merge) 10raymond-ndibe: build: stop argument injection via source_url and ref [repos/cloud/toolforge/builds-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/merge_requests/171 (https://phabricator.wikimedia.org/T442299)
[12:21:24] <wikibugs>	 (03update) 10raymond-ndibe: runtime: stop argument injection via the build ref [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/214 (https://phabricator.wikimedia.org/T443554)
[12:24:17] <wikibugs>	 (03update) 10raymond-ndibe: runtime: stop argument injection via the build ref [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/214 (https://phabricator.wikimedia.org/T443554)
[12:24:31] <wikibugs>	 (03merge) 10raymond-ndibe: runtime: stop argument injection via the build ref [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/214 (https://phabricator.wikimedia.org/T443554)
[12:27:09] <wikibugs>	 (03open) 10group_203_bot_3c0afd0d9fd9529f3b7bc7e69a4a3bce: builds-api: bump to 0.0.226-20261105122032-2597669e [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1504 (https://phabricator.wikimedia.org/T442299)
[12:27:10] <wikibugs>	 (03update) 10group_203_bot_3c0afd0d9fd9529f3b7bc7e69a4a3bce: builds-api: bump to 0.0.226-20261105122032-2597669e [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1504 (https://phabricator.wikimedia.org/T442299)
[12:27:39] <wikibugs>	 (03open) 10group_203_bot_3c0afd0d9fd9529f3b7bc7e69a4a3bce: components-api: bump to 0.0.233-20261105122447-60c2d6ff [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1505 (https://phabricator.wikimedia.org/T443554)
[12:27:58] <logmsgbot_cloud>	 !log raymond-ndibe@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component builds-api
[12:29:34] <wikibugs>	 10Toolforge, 06tools-platform-team: [builds-api] latest image is missing from toolsbeta registry - https://phabricator.wikimedia.org/T443777#12521975 (10DamianZaremba) No worries, I'll come back to it after lunch... at least it is expected
[12:31:35] <logmsgbot_cloud>	 !log raymond-ndibe@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
[12:31:56] <logmsgbot_cloud>	 !log raymond-ndibe@cloudcumin1001 tools START - Cookbook wmcs.toolforge.component.deploy for component builds-api
[12:35:24] <jinxer-wm>	 FIRING: SystemdUnitFailed: security_group_ssh-from-restricted-bastion_to_project_trove.service on cloudcontrol1006:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[12:36:13] <logmsgbot_cloud>	 !log raymond-ndibe@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
[12:37:55] <logmsgbot_cloud>	 !log raymond-ndibe@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component components-api
[12:39:20] <wikibugs>	 (03approved) 10raymond-ndibe: builds-api: bump to 0.0.226-20261105122032-2597669e [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1504 (https://phabricator.wikimedia.org/T442299) (owner: 10group_203_bot_3c0afd0d9fd9529f3b7bc7e69a4a3bce)
[12:39:23] <wikibugs>	 10Toolforge, 06tools-platform-team: [builds-api] latest image is missing from toolsbeta registry - https://phabricator.wikimedia.org/T443777#12521988 (10Raymond_Ndibe) >>! In T443777#12521975, @DamianZaremba wrote: > No worries, I'll come back to it after lunch... at least it is expected  New image built. We c...
[12:39:24] <wikibugs>	 (03merge) 10raymond-ndibe: builds-api: bump to 0.0.226-20261105122032-2597669e [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1504 (https://phabricator.wikimedia.org/T442299) (owner: 10group_203_bot_3c0afd0d9fd9529f3b7bc7e69a4a3bce)
[12:41:03] <logmsgbot_cloud>	 !log raymond-ndibe@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
[12:44:09] <logmsgbot_cloud>	 !log raymond-ndibe@cloudcumin1001 tools START - Cookbook wmcs.toolforge.component.deploy for component components-api
[12:47:50] <logmsgbot_cloud>	 !log raymond-ndibe@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
[13:05:24] <jinxer-wm>	 RESOLVED: SystemdUnitFailed: security_group_ssh-from-restricted-bastion_to_project_trove.service on cloudcontrol1006:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[13:14:09] <wikibugs>	 (03update) 10renovatebot: Lock file maintenance [toolforge-repos/namehistory] - 10https://gitlab.wikimedia.org/toolforge-repos/namehistory/-/merge_requests/39
[13:14:15] <wikibugs>	 (03update) 10renovatebot: Lock file maintenance [toolforge-repos/allblocksever] - 10https://gitlab.wikimedia.org/toolforge-repos/allblocksever/-/merge_requests/48
[13:14:21] <wikibugs>	 (03update) 10renovatebot: Lock file maintenance [toolforge-repos/checkusertools] - 10https://gitlab.wikimedia.org/toolforge-repos/checkusertools/-/merge_requests/46
[13:15:11] <wikibugs>	 (03update) 10renovatebot: Lock file maintenance [toolforge-repos/rangetree] - 10https://gitlab.wikimedia.org/toolforge-repos/rangetree/-/merge_requests/76
[13:15:47] <wikibugs>	 (03update) 10renovatebot: Lock file maintenance [toolforge-repos/wiki-mail-verify] - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-mail-verify/-/merge_requests/65
[13:26:50] <wikibugs>	 (03update) 10damian: _check_for_matching_build - check the build's builder is current [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/213 (https://phabricator.wikimedia.org/T442364)
[13:30:44] <wikibugs>	 10Toolforge, 06tools-platform-team: [builds-api] latest image is missing from toolsbeta registry - https://phabricator.wikimedia.org/T443777#12522149 (10DamianZaremba) 05Open→03Resolved a:03DamianZaremba Thanks  ` damian@lima-kilo:~$ toolforge_deploy restore-all Restoring 2 components:     builds-api...
[13:33:49] <wikibugs>	 (03approved) 10raymond-ndibe: components-api: bump to 0.0.233-20261105122447-60c2d6ff [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1505 (https://phabricator.wikimedia.org/T443554) (owner: 10group_203_bot_3c0afd0d9fd9529f3b7bc7e69a4a3bce)
[13:33:57] <wikibugs>	 (03update) 10raymond-ndibe: components-api: bump to 0.0.233-20261105122447-60c2d6ff [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1505 (https://phabricator.wikimedia.org/T443554) (owner: 10group_203_bot_3c0afd0d9fd9529f3b7bc7e69a4a3bce)
[13:34:17] <wikibugs>	 (03update) 10raymond-ndibe: components-api: bump to 0.0.233-20261105122447-60c2d6ff [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1505 (https://phabricator.wikimedia.org/T443554) (owner: 10group_203_bot_3c0afd0d9fd9529f3b7bc7e69a4a3bce)
[13:34:33] <wikibugs>	 (03merge) 10raymond-ndibe: components-api: bump to 0.0.233-20261105122447-60c2d6ff [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1505 (https://phabricator.wikimedia.org/T443554) (owner: 10group_203_bot_3c0afd0d9fd9529f3b7bc7e69a4a3bce)
[13:48:55] <wikibugs>	 06cloud-services-team (Hardware), 06tools-infrastructure-team, 06DC-Ops, 06SRE, and 2 others: How to configure storage on new dumps servers - https://phabricator.wikimedia.org/T441202#12522241 (10xcollazo) Quick update: when I read doubling storage to 320TB I got *very* excited about the possibilities!  De...
[13:52:52] <wikibugs>	 10Toolforge, 06tools-platform-team, 07OKR-Work: [alerts-api] List all active alerts for a tool - https://phabricator.wikimedia.org/T441144#12522246 (10aputhin) p:05Triage→03High
[13:54:20] <wikibugs>	 10Tool-lingua-datasets, 10Toolforge (Quota-requests): Setup Lingua-data.toolforge.org with 400GB - https://phabricator.wikimedia.org/T440689#12522252 (10aputhin)
[14:03:46] <wikibugs>	 (03open) 10damian: Refactor outdated check [repos/cloud/toolforge/builds-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/merge_requests/172
[14:10:22] <wikibugs>	 10Toolforge, 06tools-platform-team: `config generate with some existing jobs works` failure - https://phabricator.wikimedia.org/T443680#12522313 (10Raymond_Ndibe) @taavi this is a flakiness issue in the test. I've experienced it too while trying to deploy things.  Maybe we can keep the task up though, so we ca...
[14:23:40] <wikibugs>	 (03PS1) 10Thibaut Le Page: build: Update dependencies for October 2026 security advisories [labs/striker] - 10https://gerrit.wikimedia.org/r/1364383 (https://phabricator.wikimedia.org/T443773)
[14:25:51] <wikibugs>	 (03PS2) 10Thibaut Le Page: build: Update dependencies for October 2026 security advisories [labs/striker] - 10https://gerrit.wikimedia.org/r/1364383 (https://phabricator.wikimedia.org/T443773)
[14:27:27] <wikibugs>	 (03PS3) 10Thibaut Le Page: build: Update dependencies for October 2026 security advisories [labs/striker] - 10https://gerrit.wikimedia.org/r/1364383 (https://phabricator.wikimedia.org/T443773)
[14:28:24] <wikibugs>	 (03PS4) 10Thibaut Le Page: build: Update dependencies for October 2026 security advisories [labs/striker] - 10https://gerrit.wikimedia.org/r/1364383 (https://phabricator.wikimedia.org/T443773)
[14:32:06] <wikibugs>	 (03CR) 10FNegri: [C:03+1] build: Update dependencies for October 2026 security advisories [labs/striker] - 10https://gerrit.wikimedia.org/r/1364383 (https://phabricator.wikimedia.org/T443773) (owner: 10Thibaut Le Page)
[14:33:34] <wikibugs>	 (03CR) 10Thibaut Le Page: [C:03+2] build: Update dependencies for October 2026 security advisories [labs/striker] - 10https://gerrit.wikimedia.org/r/1364383 (https://phabricator.wikimedia.org/T443773) (owner: 10Thibaut Le Page)
[14:35:21] <wikibugs>	 (03Merged) 10jenkins-bot: build: Update dependencies for October 2026 security advisories [labs/striker] - 10https://gerrit.wikimedia.org/r/1364383 (https://phabricator.wikimedia.org/T443773) (owner: 10Thibaut Le Page)
[14:58:45] <wikibugs>	 10superset.wmcloud.org: Superset WMCloud fails to access s[0-9] clusters - https://phabricator.wikimedia.org/T443585#12522612 (10Zache)  For my saved query Superset's SQL Lab  throws the error (target is `fiwiki_p` in `s2`)  username and ip address redacted. ` MySQL Error (MySQLdb._exceptions.OperationalError) (...
[14:58:47] <wikibugs>	 10Tools, 06Commons, 07Community-Wishlist: A set of tools for image editing directly in Wikimedia Commons - https://phabricator.wikimedia.org/T428062#12522611 (10mikez-WMF) Good catch! Makes sense to move under media actions for consistency. @Amdrel  Are there any other relevant changes we might be overlooking?
[14:59:26] <wm-bot2>	 !log tools.cluebotng-review Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42049573846 (https://github.com/cluebotng/component-configs/commits/120177aabd7236410755f7187b351195daceb172)
[14:59:28] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL
[15:15:45] <wm-bot2>	 !log tools.cluebotng-review Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42052229859 (https://github.com/cluebotng/component-configs/commits/4f5b065a67c9c1036d100bb6ae2816f193b11b90)
[15:15:47] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL
[15:52:56] <wikibugs>	 10wikitech.wikimedia.org: Remove LToscano (WMF)'s admin rights on Wikitech - https://phabricator.wikimedia.org/T443800 (10CptViraj) 03NEW
[16:00:00] <jinxer-wm>	 FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse
[16:01:42] <wikibugs>	 (03open) 10rluts: Make Save and next the primary review action [toolforge-repos/findsmith] - 10https://gitlab.wikimedia.org/toolforge-repos/findsmith/-/merge_requests/12
[16:04:06] <wikibugs>	 10wikitech.wikimedia.org: Remove LToscano (WMF)'s admin rights on Wikitech - https://phabricator.wikimedia.org/T443800#12523192 (10bd808) 05Open→03Resolved a:03bd808 https://wikitech.wikimedia.org/w/index.php?title=Special:Log&logid=1020805
[16:14:38] <wmcs-alerts>	 FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[16:19:38] <wmcs-alerts>	 RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[16:27:38] <wmcs-alerts>	 FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[16:37:38] <wmcs-alerts>	 RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[16:58:16] <wikibugs>	 10Tool-wikinewsie: Deprioritize lists - https://phabricator.wikimedia.org/T435844#12523620 (10Pharos) [[Deaths in 2026]] is inherently kind of a boring page; others like [[List of Turkish tornadoes in 2026]] are rather more interesting in my opinion. I think we want these defined on a portal-by-portal basis.  I...
[17:13:22] <wikibugs>	 10Tool-delintbot: Transform font tags with all sorts of color values properly - https://phabricator.wikimedia.org/T443813 (10Redmin) 03NEW
[17:13:49] <wikibugs>	 10Tool-delintbot: Transform font tags with all sorts of color values properly - https://phabricator.wikimedia.org/T443813#12523749 (10Redmin) p:05Triage→03Medium
[17:28:38] <wmcs-alerts>	 FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[17:32:25] <wikibugs>	 (03open) 10fnegri: Add pyproject.toml and uv.lock [repos/cloud/wmcs/utils] - 10https://gitlab.wikimedia.org/repos/cloud/wmcs/utils/-/merge_requests/14
[17:33:38] <wmcs-alerts>	 RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[17:47:33] <wikibugs>	 10Cloud-VPS (Project-requests): Request creation of deorph VPS project - https://phabricator.wikimedia.org/T442348#12523924 (10fnegri) @CristianCantoro I think that approach is viable, but I would give Toolforge a try, if the data processing fits in 8GB then I think a job in Toolforge that connects to Trove woul...
[18:00:38] <wmcs-alerts>	 FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[18:05:38] <wmcs-alerts>	 RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[18:09:38] <wmcs-alerts>	 FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[18:14:38] <wmcs-alerts>	 RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[18:37:13] <wikibugs>	 06tools-platform-team: Retire wikidata-geo-dumps - https://phabricator.wikimedia.org/T443820 (10BLiviero-WMF) 03NEW
[18:50:22] <wikibugs>	 (03open) 10project_5175_bot_1dc5cd2b4a7cadb6d1af8e29974ce1a7: Hakkında sayfası ve aracın işini gösteren diyagram [toolforge-repos/kategori] - 10https://gitlab.wikimedia.org/toolforge-repos/kategori/-/merge_requests/16
[18:53:10] <wikibugs>	 (03update) 10project_5175_bot_1dc5cd2b4a7cadb6d1af8e29974ce1a7: Hakkında sayfası ve aracın işini gösteren diyagram [toolforge-repos/kategori] - 10https://gitlab.wikimedia.org/toolforge-repos/kategori/-/merge_requests/16
[18:58:42] <wikibugs>	 (03update) 10project_5175_bot_1dc5cd2b4a7cadb6d1af8e29974ce1a7: Hakkında sayfası ve aracın işini gösteren diyagram [toolforge-repos/kategori] - 10https://gitlab.wikimedia.org/toolforge-repos/kategori/-/merge_requests/16
[19:06:31] <wikibugs>	 (03open) 10sarai-wmf: Replace Participation and Donation CTA pages with a Calls to action page [toolforge-repos/wikimedia-attribution] - 10https://gitlab.wikimedia.org/toolforge-repos/wikimedia-attribution/-/merge_requests/10
[19:20:38] <wmcs-alerts>	 FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[19:25:38] <wmcs-alerts>	 RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[19:26:38] <wmcs-alerts>	 FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[19:30:53] <wmcs-alerts>	 RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[19:55:38] <wmcs-alerts>	 FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[20:00:01] <jinxer-wm>	 FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse
[20:00:38] <wmcs-alerts>	 RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[20:02:24] <wm-bot2>	 !log tools.cluebotng-editsets Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42099048852 (https://github.com/cluebotng/component-configs/commits/c7e091e072725a2ef015a48313e08c2f2a378bdf)
[20:02:26] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-editsets/SAL
[20:03:38] <wmcs-alerts>	 FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[20:04:53] <wm-bot2>	 !log tools.cluebotng-trainer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42099049001 (https://github.com/cluebotng/component-configs/commits/c7e091e072725a2ef015a48313e08c2f2a378bdf)
[20:04:54] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL
[20:08:38] <wmcs-alerts>	 RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown
[20:15:01] <jinxer-wm>	 FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse
[20:57:25] <wikibugs>	 (03PS1) 10Lokal Profil: api: Fetch the fields the map formats read [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/1364771 (https://phabricator.wikimedia.org/T443832)
[21:25:01] <jinxer-wm>	 FIRING: [2x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse
[21:43:22] <wikibugs>	 10Cloud-VPS (Debian Bullseye Deprecation), 10Beta-Cluster-Infrastructure, 13Patch-For-Review: Replace Swift hosts: deployment-ms-(be0[78]|fe04) (Bullseye deprecation) - https://phabricator.wikimedia.org/T436476#12524853 (10Southparkfan) 05In progress→03Stalled Other than merging https://gerrit.wikimedia....
[22:47:20] <wikibugs>	 10Cloud-VPS (Project-requests): Request creation of deorph VPS project - https://phabricator.wikimedia.org/T442348#12525140 (10CristianCantoro) >>! In T442348#12523924, @fnegri wrote: > If you're at https://meta.wikimedia.org/wiki/ItWikiCon/2026 this weekend we can chat about this in person. :)  I am, I have jus...
[23:00:01] <jinxer-wm>	 RESOLVED: OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse
[23:01:02] <wikibugs>	 10Cloud-VPS (Debian Bullseye Deprecation), 10Beta-Cluster-Infrastructure: Replace deployment-echostore02 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436479#12525178 (10Southparkfan) `deployment-echostore02` has an interesting history. It looks like that back in 2024, I forgot to create that ins...
[23:35:54] <wikibugs>	 10Tool-athar: Impliment the Export function - https://phabricator.wikimedia.org/T443486#12525310 (10Bonnjalal) 05Open→03Resolved
[23:37:27] <wikibugs>	 10Tool-athar: Add link of the Activity page - https://phabricator.wikimedia.org/T443859 (10Bonnjalal) 03NEW
[23:46:22] <wikibugs>	 10Cloud-VPS, 06tools-infrastructure-team, 10Beta-Cluster-Infrastructure, 13Patch-For-Review, 07Puppet: Cannot rollout Trixie hosts in deployment-prep without rm -rf /var/lib/puppet/ssl - https://phabricator.wikimedia.org/T443373#12525336 (10Southparkfan) >As far as I know, agents on Bookworm hosts do not...