[00:02:24] 10Tools: templatetransclusioncheck displays "no healthy upstream" error - https://phabricator.wikimedia.org/T443148#12525379 (10Adithyak1997) I have only adopted the tool and not any changes were made in the same from my end. Currently, it is as part of the PHP vulnerability, the tool is still down. Inorder to r... [00:08:28] 10Cloud-VPS (Debian Bullseye Deprecation), 10Beta-Cluster-Infrastructure: Replace deployment-echostore02 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436479#12525383 (10Southparkfan) Fortunately, the process of setting up Cassandra is not as involved as in T436471#12519535 and T370461#10007736.... [00:21:24] 10Cloud-VPS (Debian Bullseye Deprecation), 10Beta-Cluster-Infrastructure, 13Patch-For-Review: Replace deployment-echostore02 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436479#12525401 (10Southparkfan) 05In progress→03Resolved [00:36:56] 10Tool-wikinewsie: New Wikipedia articles should be their own event type on Wikinewsie - https://phabricator.wikimedia.org/T431879#12525419 (10ZhaoFJx) 05Open→03In progress [00:48:23] 10Tool-wikinewsie: Wikinewsie portals page translatable - https://phabricator.wikimedia.org/T441192#12525441 (10ZhaoFJx) [00:48:26] 10Tool-wikinewsie, 07good first task: Wikinewsie i18n - https://phabricator.wikimedia.org/T432938#12525442 (10ZhaoFJx) [00:48:28] 10Tool-wikinewsie: Wikinewsie portals page translatable - https://phabricator.wikimedia.org/T441192#12525443 (10ZhaoFJx) p:05Triage→03Medium [00:50:33] 10Tool-wikinewsie: Required categories in Wikinewsie custom portals - https://phabricator.wikimedia.org/T440639#12525446 (10ZhaoFJx) p:05Triage→03Medium [00:52:12] 10Tool-wikinewsie: Wikinewsie language selector fails to show full language names on mobile - https://phabricator.wikimedia.org/T439939#12525457 (10ZhaoFJx) p:05Triage→03High [00:55:20] 10Tool-wikinewsie: Wikinewsie image captions - https://phabricator.wikimedia.org/T440058#12525461 (10ZhaoFJx) p:05Triage→03Medium [00:55:33] 10Tool-wikinewsie: WikiProject-based portals for Wikinewsie - https://phabricator.wikimedia.org/T440144#12525464 (10ZhaoFJx) p:05Triage→03Medium [00:56:05] 10Tool-wikinewsie: Wikinewsie feeds should let you explore the long tail - https://phabricator.wikimedia.org/T440617#12525467 (10ZhaoFJx) p:05Triage→03High [00:56:13] 10Tool-wikinewsie: Wikinewsie feeds should let you explore the long tail - https://phabricator.wikimedia.org/T440617#12525469 (10ZhaoFJx) p:05High→03Medium [01:00:00] FIRING: OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [01:14:32] (03update) 10renovatebot: Update pnpm to v12.17.0 [toolforge-repos/namehistory] - 10https://gitlab.wikimedia.org/toolforge-repos/namehistory/-/merge_requests/41 [01:14:34] (03open) 10renovatebot: Update pnpm to v12.17.0 [toolforge-repos/namehistory] - 10https://gitlab.wikimedia.org/toolforge-repos/namehistory/-/merge_requests/41 [01:14:46] (03update) 10renovatebot: Update pnpm to v12.17.0 [toolforge-repos/allblocksever] - 10https://gitlab.wikimedia.org/toolforge-repos/allblocksever/-/merge_requests/49 [01:14:48] (03open) 10renovatebot: Update pnpm to v12.17.0 [toolforge-repos/allblocksever] - 10https://gitlab.wikimedia.org/toolforge-repos/allblocksever/-/merge_requests/49 [01:15:00] (03update) 10renovatebot: Update pnpm to v12.17.0 [toolforge-repos/checkusertools] - 10https://gitlab.wikimedia.org/toolforge-repos/checkusertools/-/merge_requests/48 [01:15:03] (03open) 10renovatebot: Update pnpm to v12.17.0 [toolforge-repos/checkusertools] - 10https://gitlab.wikimedia.org/toolforge-repos/checkusertools/-/merge_requests/48 [01:16:00] (03update) 10renovatebot: Update pnpm to v12.17.0 [toolforge-repos/rangetree] - 10https://gitlab.wikimedia.org/toolforge-repos/rangetree/-/merge_requests/78 [01:16:07] (03open) 10renovatebot: Update pnpm to v12.17.0 [toolforge-repos/rangetree] - 10https://gitlab.wikimedia.org/toolforge-repos/rangetree/-/merge_requests/78 [01:16:54] (03update) 10renovatebot: Update pnpm to v12.17.0 [toolforge-repos/wiki-mail-verify] - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-mail-verify/-/merge_requests/67 [01:16:57] (03open) 10renovatebot: Update pnpm to v12.17.0 [toolforge-repos/wiki-mail-verify] - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-mail-verify/-/merge_requests/67 [02:05:00] FIRING: [2x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [03:18:12] !log tools.cluebotng Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42155618053 (https://github.com/cluebotng/component-configs/commits/4552b8960c5335ae823db495604178008b4a4985) [03:18:14] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [03:21:17] !log tools.cluebotng-trainer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42155618016 (https://github.com/cluebotng/component-configs/commits/4552b8960c5335ae823db495604178008b4a4985) [03:21:18] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [03:22:47] !log tools.cluebotng-review Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42155618011 (https://github.com/cluebotng/component-configs/commits/4552b8960c5335ae823db495604178008b4a4985) [03:22:48] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [03:53:29] 10Tool-k8s-status: k8s-status no healthy upstream - https://phabricator.wikimedia.org/T443750#12525746 (10AntiCompositeNumber) 05Open→03Resolved It's now working again. [04:00:00] FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [04:11:56] !log tools.cluebotng-monitoring Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42160643918 (https://github.com/cluebotng/component-configs/commits/d75eff3c0ab6904c739a0b88046627a0eb019bcc) [04:11:57] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [04:29:07] !log tools.cluebotng-staging Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42162423773 (https://github.com/cluebotng/component-configs/commits/f0d701fcec7061cd6d2bf7ea127a73895e4a005f) [04:29:09] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-staging/SAL [04:36:44] !log tools.cluebotng-review Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42163050761 (https://github.com/cluebotng/component-configs/commits/82355b13bf40529ca3af84e6749306f7cad42b92) [04:36:46] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [04:41:30] !log tools.cluebotng Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42162423731 (https://github.com/cluebotng/component-configs/commits/f0d701fcec7061cd6d2bf7ea127a73895e4a005f) [04:41:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [05:01:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [05:04:33] !log tools.cluebotng-monitoring Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42165716583 (https://github.com/cluebotng/component-configs/commits/73465623988d4a1316253b991ecba1de5f9f2902) [05:04:35] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [05:06:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [05:12:28] !log tools.cluebotng-monitoring Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42166166337 (https://github.com/cluebotng/component-configs/commits/533681769ce14b9dc06e88480d711dfd2be43926) [05:12:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [05:13:39] !log tools.cluebotng-review Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42166677986 (https://github.com/cluebotng/component-configs/commits/ceeb81f4d5e5d7b6a4947b95df020093bdea2903) [05:13:40] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [05:15:26] !log tools.cluebotng Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42166677976 (https://github.com/cluebotng/component-configs/commits/ceeb81f4d5e5d7b6a4947b95df020093bdea2903) [05:15:27] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [05:40:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [05:45:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [05:58:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [05:58:47] !log tools.cluebotng-monitoring Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42171126034 (https://github.com/cluebotng/component-configs/commits/99cc62d4318ecb79d0e5642eab91b3d01e3fe55c) [05:58:49] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [06:03:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [06:04:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [06:06:46] !log tools.cluebotng-trainer Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42172297634 (https://github.com/cluebotng/component-configs/commits/109e459ca496c932b43af3430299a707499c2d1b) [06:06:47] !log tools.cluebotng-review Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42172297558 (https://github.com/cluebotng/component-configs/commits/109e459ca496c932b43af3430299a707499c2d1b) [06:06:48] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [06:06:49] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [06:08:53] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [06:08:55] !log tools.cluebotng-editsets Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42172297595 (https://github.com/cluebotng/component-configs/commits/109e459ca496c932b43af3430299a707499c2d1b) [06:08:56] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-editsets/SAL [06:08:57] !log tools.cluebot-syncer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42172297537 (https://github.com/cluebotng/component-configs/commits/109e459ca496c932b43af3430299a707499c2d1b) [06:08:58] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot-syncer/SAL [06:08:59] !log tools.toolforge-functional-runner Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42172297576 (https://github.com/cluebotng/component-configs/commits/109e459ca496c932b43af3430299a707499c2d1b) [06:09:00] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.toolforge-functional-runner/SAL [06:09:00] !log tools.cluebot3 Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42172297593 (https://github.com/cluebotng/component-configs/commits/109e459ca496c932b43af3430299a707499c2d1b) [06:09:01] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot3/SAL [06:09:25] !log tools.cluebotng-staging Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42172297535 (https://github.com/cluebotng/component-configs/commits/109e459ca496c932b43af3430299a707499c2d1b) [06:09:26] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-staging/SAL [06:10:22] !log tools.cluebotng-monitoring Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42172297612 (https://github.com/cluebotng/component-configs/commits/109e459ca496c932b43af3430299a707499c2d1b) [06:10:23] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [06:10:38] !log tools.cluebotng Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42172297609 (https://github.com/cluebotng/component-configs/commits/109e459ca496c932b43af3430299a707499c2d1b) [06:10:39] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [06:14:15] !log tools.cluebotng-monitoring Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42172625760 (https://github.com/cluebotng/component-configs/commits/b4e427ee91fef96236b0b67b43d9e9465d70e2bc) [06:14:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [06:54:05] !log tools.cluebotng-review Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42177270550 (https://github.com/cluebotng/component-configs/commits/7dfced152f131f82242418b59d55e373d9ad8f77) [06:54:08] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [06:57:09] !log tools.cluebotng-trainer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42177270563 (https://github.com/cluebotng/component-configs/commits/7dfced152f131f82242418b59d55e373d9ad8f77) [06:57:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [06:57:33] !log tools.cluebotng Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42177270625 (https://github.com/cluebotng/component-configs/commits/7dfced152f131f82242418b59d55e373d9ad8f77) [06:57:34] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [07:01:24] !log tools.cluebotng-trainer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42177719434 (https://github.com/cluebotng/component-configs/commits/66d0b6ee6547ac11df039d6d7e288c9f0acac36b) [07:01:26] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [07:31:43] !log tools.cluebotng-monitoring Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42181039067 (https://github.com/cluebotng/component-configs/commits/6227b943f6ee18f3ac6046651a55ad11e53b6823) [07:31:45] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [07:39:34] !log tools.cluebotng-staging Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42181945089 (https://github.com/cluebotng/component-configs/commits/da424e45124170000e83bc261fc0ac034e51b840) [07:39:36] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-staging/SAL [07:40:23] !log tools.cluebotng Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42181945119 (https://github.com/cluebotng/component-configs/commits/da424e45124170000e83bc261fc0ac034e51b840) [07:40:24] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [07:41:33] !log tools.cluebotng-review Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42181945109 (https://github.com/cluebotng/component-configs/commits/da424e45124170000e83bc261fc0ac034e51b840) [07:41:34] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [08:00:00] FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [08:08:41] (03PS8) 10Arendpieter: Use IDP for authentication [labs/striker] - 10https://gerrit.wikimedia.org/r/1250537 (https://phabricator.wikimedia.org/T359554) [08:15:20] 10Tool-inteGraality: Persist per-cell counts in the database - https://phabricator.wikimedia.org/T443875 (10JeanFred) 03NEW [08:58:56] (03PS1) 10Wandji collins: Add the language-first landing page and API client. [labs/tools/wdaudiolex-fe] - 10https://gerrit.wikimedia.org/r/1365942 [09:05:04] (03update) 10yopanelly: feat: add interactive timeline chart with language filter [toolforge-repos/wsindex] - 10https://gitlab.wikimedia.org/toolforge-repos/wsindex/-/merge_requests/13 [09:06:01] (03PS2) 10Wandji collins: Add the language-first landing page and API client. [labs/tools/wdaudiolex-fe] - 10https://gerrit.wikimedia.org/r/1365942 [09:17:03] 10Tool-community-activity-alerts-system: Fix the community search API - https://phabricator.wikimedia.org/T443880 (10Swayam_Agrahari) 03NEW [09:17:34] (03update) 10renovatebot: Update dependency oauthlib to v4.0.1 [SECURITY] [toolforge-repos/dewiki-voterightsnotifier] - 10https://gitlab.wikimedia.org/toolforge-repos/dewiki-voterightsnotifier/-/merge_requests/54 [09:17:37] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/dewiki-voterightsnotifier] - 10https://gitlab.wikimedia.org/toolforge-repos/dewiki-voterightsnotifier/-/merge_requests/52 [09:17:38] (03open) 10renovatebot: Update dependency oauthlib to v4.0.1 [SECURITY] [toolforge-repos/dewiki-voterightsnotifier] - 10https://gitlab.wikimedia.org/toolforge-repos/dewiki-voterightsnotifier/-/merge_requests/54 [09:17:42] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/dewiki-voterightsnotifier] - 10https://gitlab.wikimedia.org/toolforge-repos/dewiki-voterightsnotifier/-/merge_requests/52 [09:18:00] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Fix the community search API - https://phabricator.wikimedia.org/T443880#12526163 (10Swayam_Agrahari) [09:18:17] (03update) 10renovatebot: Update dependency oauthlib to v4.0.1 [SECURITY] [toolforge-repos/flaggedrevspromotioncheck] - 10https://gitlab.wikimedia.org/toolforge-repos/flaggedrevspromotioncheck/-/merge_requests/42 [09:18:19] (03open) 10renovatebot: Update dependency oauthlib to v4.0.1 [SECURITY] [toolforge-repos/flaggedrevspromotioncheck] - 10https://gitlab.wikimedia.org/toolforge-repos/flaggedrevspromotioncheck/-/merge_requests/42 [09:18:20] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/flaggedrevspromotioncheck] - 10https://gitlab.wikimedia.org/toolforge-repos/flaggedrevspromotioncheck/-/merge_requests/40 [09:18:24] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/flaggedrevspromotioncheck] - 10https://gitlab.wikimedia.org/toolforge-repos/flaggedrevspromotioncheck/-/merge_requests/40 [09:18:33] (03update) 10renovatebot: Update dependency oauthlib to v4.0.1 [SECURITY] [toolforge-repos/dewikisignbot] - 10https://gitlab.wikimedia.org/toolforge-repos/dewikisignbot/-/merge_requests/47 [09:18:35] (03open) 10renovatebot: Update dependency oauthlib to v4.0.1 [SECURITY] [toolforge-repos/dewikisignbot] - 10https://gitlab.wikimedia.org/toolforge-repos/dewikisignbot/-/merge_requests/47 [09:18:37] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/dewikisignbot] - 10https://gitlab.wikimedia.org/toolforge-repos/dewikisignbot/-/merge_requests/45 [09:18:41] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/dewikisignbot] - 10https://gitlab.wikimedia.org/toolforge-repos/dewikisignbot/-/merge_requests/45 [09:22:30] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Fix activity API date validation - https://phabricator.wikimedia.org/T443881 (10Swayam_Agrahari) 03NEW [09:24:36] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Correct the Editor Counts chart labels - https://phabricator.wikimedia.org/T443882 (10Swayam_Agrahari) 03NEW [09:27:09] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Fix annotation editing modal information - https://phabricator.wikimedia.org/T443883 (10Swayam_Agrahari) 03NEW [09:28:53] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Fix date display inconsistencies - https://phabricator.wikimedia.org/T443884 (10Swayam_Agrahari) 03NEW [09:31:56] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Add data freshness information - https://phabricator.wikimedia.org/T443885 (10Swayam_Agrahari) 03NEW [09:33:47] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Allow users to edit watchlist notification preferences - https://phabricator.wikimedia.org/T443886 (10Swayam_Agrahari) 03NEW [09:34:13] (03update) 10sarai-wmf: Draft: Replace Participation and Donation CTA pages with a Calls to action page [toolforge-repos/wikimedia-attribution] - 10https://gitlab.wikimedia.org/toolforge-repos/wikimedia-attribution/-/merge_requests/10 [09:41:15] (03CR) 10Abijeet Patro: [V:03+2] Localisation updates from https://translatewiki.net. [labs/tools/commons-mass-description] - 10https://gerrit.wikimedia.org/r/1364265 (owner: 10L10n-bot) [09:41:52] (03Abandoned) 10Abijeet Patro: Localisation updates from https://translatewiki.net. [labs/tools/commons-mass-description] - 10https://gerrit.wikimedia.org/r/1361385 (owner: 10L10n-bot) [09:47:11] (03update) 10project_5175_bot_1dc5cd2b4a7cadb6d1af8e29974ce1a7: Hakkında sayfası ve aracın işini gösteren diyagram [toolforge-repos/kategori] - 10https://gitlab.wikimedia.org/toolforge-repos/kategori/-/merge_requests/16 [10:10:06] 10Tool-delintbot: Handle all sorts of invalid font size values properly - https://phabricator.wikimedia.org/T443890 (10Redmin) 03NEW [10:10:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [10:10:43] 10Tool-delintbot: Handle all sorts of invalid font size values properly - https://phabricator.wikimedia.org/T443890#12526368 (10Redmin) p:05Triage→03Medium [10:12:17] !log tools.cluebotng-staging Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42200307291 (https://github.com/cluebotng/component-configs/commits/255eec1d87ca571dcfd0f09253a9504e7d2b70f1) [10:12:19] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-staging/SAL [10:13:07] !log tools.cluebotng Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42200307462 (https://github.com/cluebotng/component-configs/commits/255eec1d87ca571dcfd0f09253a9504e7d2b70f1) [10:13:08] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [10:14:27] !log tools.cluebotng-monitoring Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42200488452 (https://github.com/cluebotng/component-configs/commits/3811ff050c237ab472542a64c46c558d7d159b6d) [10:14:28] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [10:16:55] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/dewiki-voterightsnotifier] - 10https://gitlab.wikimedia.org/toolforge-repos/dewiki-voterightsnotifier/-/merge_requests/52 [10:17:18] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/flaggedrevspromotioncheck] - 10https://gitlab.wikimedia.org/toolforge-repos/flaggedrevspromotioncheck/-/merge_requests/40 [10:17:25] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/dewikisignbot] - 10https://gitlab.wikimedia.org/toolforge-repos/dewikisignbot/-/merge_requests/45 [10:20:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [10:22:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [10:28:01] 10Tool-delintbot: Handle all sorts of invalid font size values properly - https://phabricator.wikimedia.org/T443890#12526426 (10KomalPandey4) Hi @Redmin, I would like to work on this task and prepare a patch to handle legacy font size values properly. [10:28:25] 10Tool-delintbot: Handle all sorts of invalid font size values properly - https://phabricator.wikimedia.org/T443890#12526429 (10KomalPandey4) a:03KomalPandey4 [10:32:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [10:39:18] 10Tools: templatetransclusioncheck displays "no healthy upstream" error - https://phabricator.wikimedia.org/T443148#12526485 (10Aklapper) @Adithyak1997: Thanks. Having more than one issue tracker leads to duplication and fragmentation of conversations. Could you please add info to https://toolsadmin.wikimedia.or... [11:11:34] 10Cloud-VPS (Debian Bullseye Deprecation), 10Continuous-Integration-Infrastructure: Re-build integration-cumin.integration.eqiad1.wikimedia.cloud on something newer than bullseye - https://phabricator.wikimedia.org/T433592#12526564 (10Jdforrester-WMF) a:03hashar [11:13:58] !log tools.cluebotng-monitoring Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42208073822 (https://github.com/cluebotng/component-configs/commits/6639b38d3900750530ebf6ef8d0a92218849b0f5) [11:14:00] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [11:28:38] !log tools.cluebotng-monitoring Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42210074274 (https://github.com/cluebotng/component-configs/commits/a4f327022b82328ec3d56e77ca3f3f984da6b964) [11:28:40] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [12:00:00] FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [12:09:18] 10Toolforge, 06tools-platform-team: [infra,k8s] Upgrade Toolforge Kubernetes to version 1.36 - https://phabricator.wikimedia.org/T443899 (10taavi) 03NEW [12:09:19] 10Toolforge, 06tools-platform-team: [infra,k8s] Upgrade Toolforge Kubernetes to version 1.36 - https://phabricator.wikimedia.org/T443899#12526752 (10taavi) [12:09:21] 06cloud-services-team, 10Toolforge: [infra,k8s] Upgrade Toolforge Kubernetes to version 1.35 - https://phabricator.wikimedia.org/T414596#12526753 (10taavi) [12:09:42] 10Toolforge, 06tools-platform-team: [infra,k8s] Upgrade Toolforge Kubernetes to version 1.37 - https://phabricator.wikimedia.org/T443900 (10taavi) 03NEW [12:09:46] 10Toolforge, 06tools-platform-team: [infra,k8s] Upgrade Toolforge Kubernetes to version 1.36 - https://phabricator.wikimedia.org/T443899#12526767 (10taavi) [12:09:47] 10Toolforge, 06tools-platform-team: [infra,k8s] Upgrade Toolforge Kubernetes to version 1.37 - https://phabricator.wikimedia.org/T443900#12526766 (10taavi) [12:15:10] 10Cloud-VPS, 10Toolforge, 06tools-infrastructure-team, 06tools-platform-team: 2026-10-21 Toolforge and Cloud VPS taken offline for "PHP2shell" security response - https://phabricator.wikimedia.org/T441908#12526785 (10Aklapper) [12:17:54] !log tools.cluebotng-trainer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42215994580 (https://github.com/cluebotng/component-configs/commits/e3ea44488c33b1bd4dd63a5c9d9c3ef4a58c3406) [12:17:56] !log tools.cluebotng-staging Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42216485410 (https://github.com/cluebotng/component-configs/commits/86fca53768dbc4df4fce7786bb9638ebe1a604e6) [12:17:56] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [12:17:57] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-staging/SAL [12:21:22] !log tools.cluebotng Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42216485438 (https://github.com/cluebotng/component-configs/commits/86fca53768dbc4df4fce7786bb9638ebe1a604e6) [12:21:24] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [12:21:46] !log tools.cluebotng-review Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42216485332 (https://github.com/cluebotng/component-configs/commits/86fca53768dbc4df4fce7786bb9638ebe1a604e6) [12:21:47] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [13:04:58] 10Tool-delintbot: Handle all sorts of invalid font size values properly - https://phabricator.wikimedia.org/T443890#12526973 (10Redmin) [13:06:16] !log tools.cluebotng-review Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42222102051 (https://github.com/cluebotng/component-configs/commits/5a9290a457af2e5700d955bc65fbd8d77cad712c) [13:06:18] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [13:07:05] !log tools.cluebotng Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42222101673 (https://github.com/cluebotng/component-configs/commits/5a9290a457af2e5700d955bc65fbd8d77cad712c) [13:07:06] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [13:09:19] 10Tool-wikinewsie: 'See also' mode for Wikinewsie - https://phabricator.wikimedia.org/T434468#12526989 (10Akash_A) Any updates guys? [13:11:03] 10Tool-delintbot: Handle all sorts of invalid font size values properly - https://phabricator.wikimedia.org/T443890#12526991 (10Redmin) Sounds good, @KomalPandey4. The code is hosted on [[ https://gitlab.wikimedia.org/toolforge-repos/delintbot | Wikimedia GitLab ]]; you may find [[ https://www.mediawiki.org/wiki... [13:17:20] !log tools.cluebotng Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42224188059 (https://github.com/cluebotng/component-configs/commits/63b10b4f2d7ce4aaf50e16dba7e59d0c415fb566) [13:17:22] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [13:17:25] !log tools.cluebot3 Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42224188028 (https://github.com/cluebotng/component-configs/commits/63b10b4f2d7ce4aaf50e16dba7e59d0c415fb566) [13:17:26] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot3/SAL [13:17:27] !log tools.cluebotng-trainer Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42224188055 (https://github.com/cluebotng/component-configs/commits/63b10b4f2d7ce4aaf50e16dba7e59d0c415fb566) [13:17:27] !log tools.cluebotng-review Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42223625514 (https://github.com/cluebotng/component-configs/commits/9b07be56cd997c4d5787c311dd3e89941f8e528a) [13:17:28] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [13:17:29] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [13:19:16] !log tools.toolforge-functional-runner Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42224188056 (https://github.com/cluebotng/component-configs/commits/63b10b4f2d7ce4aaf50e16dba7e59d0c415fb566) [13:19:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.toolforge-functional-runner/SAL [13:19:22] !log tools.cluebot-syncer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42224187995 (https://github.com/cluebotng/component-configs/commits/63b10b4f2d7ce4aaf50e16dba7e59d0c415fb566) [13:19:23] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot-syncer/SAL [13:19:34] !log tools.cluebotng-editsets Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42224188014 (https://github.com/cluebotng/component-configs/commits/63b10b4f2d7ce4aaf50e16dba7e59d0c415fb566) [13:19:35] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-editsets/SAL [13:20:15] !log tools.cluebotng-staging Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42224187997 (https://github.com/cluebotng/component-configs/commits/63b10b4f2d7ce4aaf50e16dba7e59d0c415fb566) [13:20:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-staging/SAL [13:20:44] !log tools.cluebotng-monitoring Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42224188048 (https://github.com/cluebotng/component-configs/commits/63b10b4f2d7ce4aaf50e16dba7e59d0c415fb566) [13:20:45] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [13:22:41] !log tools.cluebotng-review Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42224188041 (https://github.com/cluebotng/component-configs/commits/63b10b4f2d7ce4aaf50e16dba7e59d0c415fb566) [13:22:43] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [13:47:24] !log tools.cluebotng-monitoring Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42227653941 (https://github.com/cluebotng/component-configs/commits/e9ebfee649f69eaef20695673d0706925b51f232) [13:47:26] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [13:49:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [14:19:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [14:22:35] !log tools.cluebot-syncer Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42233248970 (https://github.com/cluebotng/component-configs/commits/79c2344352c2d5781360aaadce25e401270c4a47) [14:22:38] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot-syncer/SAL [14:22:39] !log tools.cluebotng-monitoring Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42233207209 (https://github.com/cluebotng/component-configs/commits/4f3156ee11ba46babbca95fca75b3194900de461) [14:22:40] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [14:22:43] !log tools.cluebotng-trainer Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42233249218 (https://github.com/cluebotng/component-configs/commits/79c2344352c2d5781360aaadce25e401270c4a47) [14:22:44] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [14:22:45] !log tools.cluebotng Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42233248920 (https://github.com/cluebotng/component-configs/commits/79c2344352c2d5781360aaadce25e401270c4a47) [14:22:46] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [14:24:32] !log tools.toolforge-functional-runner Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42233248952 (https://github.com/cluebotng/component-configs/commits/79c2344352c2d5781360aaadce25e401270c4a47) [14:24:33] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.toolforge-functional-runner/SAL [14:24:43] !log tools.cluebotng-editsets Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42233249040 (https://github.com/cluebotng/component-configs/commits/79c2344352c2d5781360aaadce25e401270c4a47) [14:24:44] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-editsets/SAL [14:25:03] !log tools.cluebot3 Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42233249044 (https://github.com/cluebotng/component-configs/commits/79c2344352c2d5781360aaadce25e401270c4a47) [14:25:04] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot3/SAL [14:25:14] !log tools.cluebotng-staging Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42233248999 (https://github.com/cluebotng/component-configs/commits/79c2344352c2d5781360aaadce25e401270c4a47) [14:25:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-staging/SAL [14:26:50] !log tools.cluebotng-monitoring Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42233248919 (https://github.com/cluebotng/component-configs/commits/79c2344352c2d5781360aaadce25e401270c4a47) [14:26:51] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [14:26:55] !log tools.cluebotng-trainer Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42233360609 (https://github.com/cluebotng/component-configs/commits/22a72d8f43f6861bbb9ac592cb9af892dd95f765) [14:26:56] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [14:27:05] !log tools.cluebotng-review Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42233248930 (https://github.com/cluebotng/component-configs/commits/79c2344352c2d5781360aaadce25e401270c4a47) [14:27:06] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [14:27:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [14:27:41] !log tools.cluebotng-editsets Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42233585957 (https://github.com/cluebotng/component-configs/commits/7bb201355741da2b451f50b4af56b8e395401843) [14:27:43] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-editsets/SAL [14:27:46] !log tools.cluebotng Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42233327546 (https://github.com/cluebotng/component-configs/commits/e9471c88503e10f4ba84992f33bfa015a2c6c887) [14:27:47] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [14:29:03] !log tools.cluebot-syncer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42233833391 (https://github.com/cluebotng/component-configs/commits/7145135cb4cf221ef0b405732373402fb9d9332d) [14:29:04] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot-syncer/SAL [14:29:05] !log tools.cluebot3 Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42233833480 (https://github.com/cluebotng/component-configs/commits/7145135cb4cf221ef0b405732373402fb9d9332d) [14:29:06] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot3/SAL [14:29:38] !log tools.cluebotng-staging Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42233691092 (https://github.com/cluebotng/component-configs/commits/b630be8dad4a568f64c5a4b612248b3a15b220a3) [14:29:39] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-staging/SAL [14:30:17] !log tools.cluebotng-trainer Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42233833462 (https://github.com/cluebotng/component-configs/commits/7145135cb4cf221ef0b405732373402fb9d9332d) [14:30:18] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [14:31:15] !log tools.cluebotng-editsets Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42233833484 (https://github.com/cluebotng/component-configs/commits/7145135cb4cf221ef0b405732373402fb9d9332d) [14:31:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-editsets/SAL [14:31:58] !log tools.cluebotng-monitoring Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42233881417 (https://github.com/cluebotng/component-configs/commits/13375ca9f2245b8713c6934705cc7b61659a7bf2) [14:31:59] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [14:32:34] !log tools.cluebot3 Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42233912183 (https://github.com/cluebotng/component-configs/commits/5de5c8fa4679db8df6d651ae192b27cc21927046) [14:32:35] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot3/SAL [14:32:38] !log tools.cluebotng-review Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42233833470 (https://github.com/cluebotng/component-configs/commits/7145135cb4cf221ef0b405732373402fb9d9332d) [14:32:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [14:32:39] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [14:32:39] !log tools.cluebot-syncer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42233853112 (https://github.com/cluebotng/component-configs/commits/20f17c32036d9fb435ac7633a2a983a4927d302d) [14:32:40] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot-syncer/SAL [14:33:18] !log tools.cluebotng Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42234031825 (https://github.com/cluebotng/component-configs/commits/15c352e208f001eee46b8462013282f7e93d9872) [14:33:21] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [14:33:41] !log tools.cluebotng-staging Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42233833553 (https://github.com/cluebotng/component-configs/commits/7145135cb4cf221ef0b405732373402fb9d9332d) [14:33:42] !log tools.cluebotng-trainer Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42234420334 (https://github.com/cluebotng/component-configs/commits/0c4dbb64c592db6c7c12afbcc69dec8082f6c60b) [14:33:42] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-staging/SAL [14:33:43] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [14:37:34] !log tools.cluebotng-review Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42234031860 (https://github.com/cluebotng/component-configs/commits/15c352e208f001eee46b8462013282f7e93d9872) [14:37:35] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [14:56:44] 10Tool-community-activity-alerts-system: Introduce internationalisation and localisation support - https://phabricator.wikimedia.org/T443923 (10Swayam_Agrahari) 03NEW [14:56:59] !log tools.cluebot-documentation Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42238039606 (https://github.com/cluebotng/component-configs/commits/3dd7fc0f72de4dcb4aded43e721032ff72f492a0) [14:57:01] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot-documentation/SAL [15:01:20] 10Tool-community-activity-alerts-system: Set up internationalisation infrastructure - https://phabricator.wikimedia.org/T443924 (10Swayam_Agrahari) 03NEW [15:02:55] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Add language selector to navbar - https://phabricator.wikimedia.org/T443925 (10Swayam_Agrahari) 03NEW [15:05:03] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Internationalise shared components - https://phabricator.wikimedia.org/T443926 (10Swayam_Agrahari) 03NEW [15:05:41] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Add language selector to navbar - https://phabricator.wikimedia.org/T443925#12527478 (10Swayam_Agrahari) [15:05:42] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Internationalise shared components - https://phabricator.wikimedia.org/T443926#12527477 (10Swayam_Agrahari) [15:05:43] 10Tool-community-activity-alerts-system: Set up internationalisation infrastructure - https://phabricator.wikimedia.org/T443924#12527479 (10Swayam_Agrahari) [15:06:45] 10Tool-community-activity-alerts-system: Introduce internationalisation and localisation support - https://phabricator.wikimedia.org/T443923#12527483 (10Swayam_Agrahari) [15:06:46] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Add language selector to navbar - https://phabricator.wikimedia.org/T443925#12527481 (10Swayam_Agrahari) [15:06:47] 10Tool-community-activity-alerts-system: Set up internationalisation infrastructure - https://phabricator.wikimedia.org/T443924#12527482 (10Swayam_Agrahari) [15:08:17] 10Tool-community-activity-alerts-system: Set up internationalisation infrastructure - https://phabricator.wikimedia.org/T443924#12527491 (10Swayam_Agrahari) [15:08:18] 10Tool-community-activity-alerts-system: Introduce internationalisation and localisation support - https://phabricator.wikimedia.org/T443923#12527492 (10Swayam_Agrahari) [15:08:19] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Internationalise shared components - https://phabricator.wikimedia.org/T443926#12527490 (10Swayam_Agrahari) [15:10:25] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Internationalise the homepage - https://phabricator.wikimedia.org/T443927 (10Swayam_Agrahari) 03NEW [15:11:40] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Internationalise Edit Counts and Editor Counts pages - https://phabricator.wikimedia.org/T443928 (10Swayam_Agrahari) 03NEW [15:11:53] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Internationalise Edit Counts and Editor Counts pages - https://phabricator.wikimedia.org/T443928#12527522 (10Swayam_Agrahari) [15:13:09] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Internationalise Watchlist and subscription features - https://phabricator.wikimedia.org/T443929 (10Swayam_Agrahari) 03NEW [15:14:11] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Internationalise charts and tables - https://phabricator.wikimedia.org/T443930 (10Swayam_Agrahari) 03NEW [15:14:19] !log tools.cluebotng-trainer Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42240676663 (https://github.com/cluebotng/component-configs/commits/4f4442799837e798ff4c86d87a85f93dd57a70e6) [15:14:21] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [15:14:41] 10Tool-community-activity-alerts-system: Internationalise charts and tables - https://phabricator.wikimedia.org/T443931 (10Swayam_Agrahari) 03NEW [15:14:53] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Internationalise charts and tables - https://phabricator.wikimedia.org/T443931#12527557 (10Swayam_Agrahari) [15:17:10] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Add the first additional language - https://phabricator.wikimedia.org/T443932 (10Swayam_Agrahari) 03NEW [15:18:11] !log tools.cluebotng-trainer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42240867017 (https://github.com/cluebotng/component-configs/commits/482c4765fb8b7f5b70651b620c2191c412c78234) [15:18:12] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [15:18:50] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Localise dates and numbers - https://phabricator.wikimedia.org/T443933 (10Swayam_Agrahari) 03NEW [15:31:59] FIRING: BackupCoverageDrop: volume backup coverage dropped on cloudbackup2004:9100 - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Backup#BackupCoverageDrop - https://grafana.wikimedia.org/d/fisr2p9/wmcs-backup?viewPanel=21&var-kind=volume&var-site=codfw - https://alerts.wikimedia.org/?q=alertname%3DBackupCoverageDrop [15:38:25] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Add a Help page with a report-a-problem form - https://phabricator.wikimedia.org/T443935 (10Swayam_Agrahari) 03NEW [15:46:25] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Add a How to Use This Tool / FAQ page - https://phabricator.wikimedia.org/T443936 (10Swayam_Agrahari) 03NEW [16:00:01] FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [16:06:32] !log tools.cluebotng-trainer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42248468132 (https://github.com/cluebotng/component-configs/commits/973b64f18651ed5b18cf908235604e70028861c8) [16:06:34] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [16:08:17] !log tools.cluebot3 Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42248627903 (https://github.com/cluebotng/component-configs/commits/a0ce1e9c453162937a059133fd92b52fde5d6083) [16:08:18] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot3/SAL [16:08:41] !log tools.cluebotng-review Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42248508387 (https://github.com/cluebotng/component-configs/commits/3acae96966e16f4f9f91683ad36f5e56d329753a) [16:08:42] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [16:09:05] !log tools.cluebot-syncer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42248689057 (https://github.com/cluebotng/component-configs/commits/e265779bb45845ef96baafd067c56b548b4cc7ea) [16:09:06] !log tools.cluebotng Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42248508521 (https://github.com/cluebotng/component-configs/commits/3acae96966e16f4f9f91683ad36f5e56d329753a) [16:09:06] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot-syncer/SAL [16:09:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [16:09:16] !log tools.cluebot-documentation Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42248723391 (https://github.com/cluebotng/component-configs/commits/2f79137abd5ecc2a7c38afe64d4081cb29378ad6) [16:09:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot-documentation/SAL [16:10:46] !log tools.cluebotng-monitoring Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42248666990 (https://github.com/cluebotng/component-configs/commits/e4b5844b01d67778506454fa3525d8c2d175c7bb) [16:10:47] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [16:11:20] !log tools.cluebotng-editsets Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42248723422 (https://github.com/cluebotng/component-configs/commits/2f79137abd5ecc2a7c38afe64d4081cb29378ad6) [16:11:21] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-editsets/SAL [16:11:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [16:12:31] 10Tool-community-activity-alerts-system: Introduce declining activity detection - https://phabricator.wikimedia.org/T443940 (10Swayam_Agrahari) 03NEW [16:16:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [16:17:25] !log tools.cluebotng-staging Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42248723213 (https://github.com/cluebotng/component-configs/commits/2f79137abd5ecc2a7c38afe64d4081cb29378ad6) [16:17:27] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-staging/SAL [16:20:12] !log tools.cluebot-documentation Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42249636361 (https://github.com/cluebotng/component-configs/commits/1eabcb440c70c78d5f010850ca7281a76cb4a0a5) [16:20:13] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot-documentation/SAL [16:20:24] !log tools.cluebotng-trainer Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42249188398 (https://github.com/cluebotng/component-configs/commits/276675d9fde45fb79a69d85827a61fea149951b7) [16:20:25] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [16:20:37] !log tools.toolforge-functional-runner Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42249636608 (https://github.com/cluebotng/component-configs/commits/1eabcb440c70c78d5f010850ca7281a76cb4a0a5) [16:20:38] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.toolforge-functional-runner/SAL [16:20:53] !log tools.cluebot3 Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42248723424 (https://github.com/cluebotng/component-configs/commits/2f79137abd5ecc2a7c38afe64d4081cb29378ad6) [16:20:53] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot3/SAL [16:21:20] !log tools.cluebot-syncer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42249636518 (https://github.com/cluebotng/component-configs/commits/1eabcb440c70c78d5f010850ca7281a76cb4a0a5) [16:21:20] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot-syncer/SAL [16:21:42] !log tools.cluebotng-editsets Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42249897407 (https://github.com/cluebotng/component-configs/commits/5dc6325f011fee8b3839d39da8c82b95d2894e99) [16:21:43] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-editsets/SAL [16:21:57] 10Tool-community-activity-alerts-system: Introduce declining activity detection - https://phabricator.wikimedia.org/T443940#12527901 (10Swayam_Agrahari) [16:22:24] !log tools.cluebotng-staging Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42250564899 (https://github.com/cluebotng/component-configs/commits/e3de70f78936ff777125f9357795dc9972758e87) [16:22:25] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-staging/SAL [16:22:42] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Implement declining activity calculation - https://phabricator.wikimedia.org/T443943 (10Swayam_Agrahari) 03NEW [16:22:47] !log tools.cluebotng-monitoring Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42249872552 (https://github.com/cluebotng/component-configs/commits/76be28c368c96ca51589616db44ce14830dbc7fe) [16:22:49] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [16:23:14] 10Tool-community-activity-alerts-system: Create API endpoint for declining activity - https://phabricator.wikimedia.org/T443944 (10Swayam_Agrahari) 03NEW [16:23:16] !log tools.cluebotng Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42249636648 (https://github.com/cluebotng/component-configs/commits/1eabcb440c70c78d5f010850ca7281a76cb4a0a5) [16:23:18] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [16:23:28] !log tools.cluebotng-review Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42249485663 (https://github.com/cluebotng/component-configs/commits/8c7f8b7d54d8637bd5f421ad2f493b96a7008504) [16:23:29] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [16:23:29] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Create API endpoint for declining activity - https://phabricator.wikimedia.org/T443944#12527937 (10Swayam_Agrahari) [16:23:37] !log tools.toolforge-functional-runner Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42250074402 (https://github.com/cluebotng/component-configs/commits/ab312d3c8f547dc101d3b112f047ff0cb4f3a2cf) [16:23:38] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.toolforge-functional-runner/SAL [16:23:39] !log tools.cluebot-documentation Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42250564862 (https://github.com/cluebotng/component-configs/commits/e3de70f78936ff777125f9357795dc9972758e87) [16:23:40] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot-documentation/SAL [16:24:00] 10Cloud-VPS, 06tools-infrastructure-team: Only 50 distinct volume objects are backed up on cloudbackup2004:9100 (codfw), below 90% of the level 24h ago. Some backups may have been lost. - https://phabricator.wikimedia.org/T443945 (10Andrew) 03NEW [16:24:13] 10Cloud-VPS, 06tools-infrastructure-team: Only 50 distinct volume objects are backed up on cloudbackup2004:9100 (codfw), below 90% of the level 24h ago. Some backups may have been lost. - https://phabricator.wikimedia.org/T443945#12527950 (10Andrew) [16:24:15] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Display activity increases and declines on the chart - https://phabricator.wikimedia.org/T443946 (10Swayam_Agrahari) 03NEW [16:24:30] !log tools.cluebotng-trainer Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42250695552 (https://github.com/cluebotng/component-configs/commits/820428026cf62c0b6d4d17e7d385728b4895c891) [16:24:31] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [16:24:55] !log tools.cluebot3 Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42250564857 (https://github.com/cluebotng/component-configs/commits/e3de70f78936ff777125f9357795dc9972758e87) [16:24:56] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot3/SAL [16:25:03] !log tools.cluebot-syncer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42250748898 (https://github.com/cluebotng/component-configs/commits/c43bf7a78f253b4a222b93c624e4d7f9a194e8f3) [16:25:04] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot-syncer/SAL [16:25:21] !log tools.cluebotng-editsets Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42250564952 (https://github.com/cluebotng/component-configs/commits/e3de70f78936ff777125f9357795dc9972758e87) [16:25:22] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-editsets/SAL [16:26:34] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Update the activity alerts table - https://phabricator.wikimedia.org/T443948 (10Swayam_Agrahari) 03NEW [16:27:45] !log tools.cluebotng-monitoring Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42250722611 (https://github.com/cluebotng/component-configs/commits/633d8f1771eebbfc0be6b8168219a96199a12fc9) [16:27:46] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-monitoring/SAL [16:28:50] !log tools.cluebotng Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42250695823 (https://github.com/cluebotng/component-configs/commits/820428026cf62c0b6d4d17e7d385728b4895c891) [16:28:52] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [16:29:12] !log tools.cluebotng-review Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/42250695395 (https://github.com/cluebotng/component-configs/commits/820428026cf62c0b6d4d17e7d385728b4895c891) [16:29:13] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [16:32:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [16:37:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [16:38:50] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Validate annotation links to allow only Wikimedia links - https://phabricator.wikimedia.org/T443949 (10Swayam_Agrahari) 03NEW [16:49:50] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Show percentage increase for detected peaks - https://phabricator.wikimedia.org/T443950 (10Swayam_Agrahari) 03NEW [16:51:30] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Add data export options - https://phabricator.wikimedia.org/T443951 (10Swayam_Agrahari) 03NEW [16:52:08] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Add tooltips to date range selection buttons - https://phabricator.wikimedia.org/T443952 (10Swayam_Agrahari) 03NEW [17:17:32] 10Tool-community-activity-alerts-system, 03WikiIndaba-Hackathon-2026: Translate Community Activity Alerts documentation - https://phabricator.wikimedia.org/T443957 (10Swayam_Agrahari) 03NEW [17:47:19] 10Tool-wiki-gender-stats: [TF] Integrate missing 2026 data from Danÿa’s local copy - https://phabricator.wikimedia.org/T431933#12528472 (10Danya) [18:05:34] 10Openstack-Magnum, 06tools-infrastructure-team: Replace all heat-based magnum clusters - https://phabricator.wikimedia.org/T437496#12528571 (10Andrew) [18:08:43] 10Openstack-Magnum, 06tools-infrastructure-team: Rebuild the cloud-runner-staging magnum cluster with the new magnum-cluster-api driver - https://phabricator.wikimedia.org/T437498#12528600 (10Andrew) [18:14:28] 10Openstack-Magnum, 06tools-infrastructure-team, 10Beta-Cluster-Infrastructure: Rebuild the deployment-wikikube-v127 magnum cluster with the new magnum-cluster-api driver and k8s v1.34 - https://phabricator.wikimedia.org/T437500#12528627 (10Andrew) [18:26:57] 10Openstack-Magnum, 06tools-infrastructure-team, 10Beta-Cluster-Infrastructure: Rebuild the deployment-wikikube-v127 magnum cluster with the new magnum-cluster-api driver and k8s v1.34 - https://phabricator.wikimedia.org/T437500#12528674 (10bd808) a:03bd808 [18:31:19] 10Openstack-Magnum, 06tools-infrastructure-team, 10Continuous-Integration-Infrastructure (Zuul upgrade), 06Release-Engineering-Team (Priority Backlog 📥): Rebuild the cloud-runner-staging magnum cluster with the new magnum-cluster-api driver - https://phabricator.wikimedia.org/T437498#12528687 (10bd808) [18:31:46] 10Openstack-Magnum, 06tools-infrastructure-team, 10Continuous-Integration-Infrastructure (Zuul upgrade), 06Release-Engineering-Team (Priority Backlog 📥): Rebuild the cloud-runner-staging magnum cluster in the zuul3 project with the new magnum-cluster-api d... - https://phabricator.wikimedia.org/T437498#12528689 [18:32:32] 10Openstack-Magnum, 06tools-infrastructure-team, 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team (Priority Backlog 📥): Rebuild the deployment-wikikube-v127 magnum cluster with the new magnum-cluster-api driver and k8s v1.34 - https://phabricator.wikimedia.org/T437500#12528690 (10bd808) [18:44:17] !log tools.cluebotng-trainer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42272751673 (https://github.com/cluebotng/component-configs/commits/c17542a324406771d842160c3b55a45fac0d1dae) [18:44:19] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [19:25:00] (03merge) 10rluts: Make Save and next the primary review action [toolforge-repos/findsmith] - 10https://gitlab.wikimedia.org/toolforge-repos/findsmith/-/merge_requests/12 [19:31:59] FIRING: BackupCoverageDrop: volume backup coverage dropped on cloudbackup2004:9100 - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Backup#BackupCoverageDrop - https://grafana.wikimedia.org/d/fisr2p9/wmcs-backup?viewPanel=21&var-kind=volume&var-site=codfw - https://alerts.wikimedia.org/?q=alertname%3DBackupCoverageDrop [19:40:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [19:50:58] 10Toolforge, 06tools-platform-team: Please include page_props on x4 - https://phabricator.wikimedia.org/T443973 (10Mike_Peel) 03NEW [19:57:13] 10Data-Services, 06tools-platform-team: Please include page_props on x4 - https://phabricator.wikimedia.org/T443973#12528913 (10JJMC89) [19:57:19] 10Cloud-VPS, 06tools-infrastructure-team: magnum-cluster-api: understand and back up tenant cluster state - https://phabricator.wikimedia.org/T440363#12528915 (10Andrew) Cannot make restores work at all. Started a support thread here: https://github.com/k3s-io/k3s/discussions/14969 [20:00:01] FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [20:24:34] 10Cloud-VPS, 06tools-infrastructure-team: magnum-cluster-api: understand and back up tenant cluster state - https://phabricator.wikimedia.org/T440363#12528933 (10Andrew) A lot of things have changed in recent years, but it's good to know that condescending answers to reasonable questions will always be with us. [20:30:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [20:30:48] 10Data-Services, 06tools-platform-team: Please include page_props on x4 - https://phabricator.wikimedia.org/T443973#12528954 (10Wargo) > Having all of the relevant info on the same servers seems like the most natural solution, unless anyone has other suggestions? This would cause to need to copy all other Medi... [21:46:56] 10Data-Services, 06Data-Persistence, 06DBA: Please include page_props on x4 - https://phabricator.wikimedia.org/T443973#12529113 (10bd808) [21:48:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [21:51:23] (03open) 10tlepage: Dummy API endpoints for /firing & /history [repos/cloud/toolforge/alerts-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/16 [21:55:01] FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [22:04:40] (03merge) 10sadrettin: Hakkında sayfası ve aracın işini gösteren diyagram [toolforge-repos/kategori] - 10https://gitlab.wikimedia.org/toolforge-repos/kategori/-/merge_requests/16 (owner: 10project_5175_bot_1dc5cd2b4a7cadb6d1af8e29974ce1a7) [22:05:14] (03open) 10ibrahemqasim: chore: HttpClient session support, toolinfo endpoint, and navbar refactoring [toolforge-repos/mdwiki] - 10https://gitlab.wikimedia.org/toolforge-repos/mdwiki/-/merge_requests/32 [22:06:00] (03update) 10ibrahemqasim: chore: HttpClient session support, toolinfo endpoint, and navbar refactoring [toolforge-repos/mdwiki] - 10https://gitlab.wikimedia.org/toolforge-repos/mdwiki/-/merge_requests/32 [22:06:41] !log tools.cluebot3 Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42301732341 (https://github.com/cluebotng/component-configs/commits/2359fb30db198ebb0e5c48ceade9726f34292e61) [22:06:43] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot3/SAL [22:08:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [22:10:01] FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [22:21:36] (03update) 10mercyjae: Included subpages views in books popularity counts [toolforge-repos/wsindex] - 10https://gitlab.wikimedia.org/toolforge-repos/wsindex/-/merge_requests/14 [22:25:07] 10Data-Services, 06Data-Persistence, 06DBA: Please include page_props on x4 - https://phabricator.wikimedia.org/T443973#12529170 (10bd808) I removed #tools-platform-team because as written this is a feature request for the DBAs to change the production database layout or figure out how to replicate some tabl... [22:26:16] (03merge) 10ibrahemqasim: chore: HttpClient session support, toolinfo endpoint, and navbar refactoring [toolforge-repos/mdwiki] - 10https://gitlab.wikimedia.org/toolforge-repos/mdwiki/-/merge_requests/32 [22:28:59] (03update) 10tlepage: Dummy API endpoints for /firing & /history [repos/cloud/toolforge/alerts-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/16 (https://phabricator.wikimedia.org/T441144 https://phabricator.wikimedia.org/T441145) [22:30:17] (03open) 10tlepage: blubber: build prod image without including poetry [repos/cloud/toolforge/alerts-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/17 [22:31:39] (03update) 10tlepage: blubber: build prod image without including poetry [repos/cloud/toolforge/alerts-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/17 [22:33:30] (03open) 10tlepage: http_session: move make_session out of the poller [repos/cloud/toolforge/alerts-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/18 [22:34:02] (03update) 10tlepage: http_session: move make_session out of the poller [repos/cloud/toolforge/alerts-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/18 (https://phabricator.wikimedia.org/T441144 https://phabricator.wikimedia.org/T441145) [22:34:39] (03update) 10tlepage: blubber: build prod image without including poetry [repos/cloud/toolforge/alerts-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/17 (https://phabricator.wikimedia.org/T441144 https://phabricator.wikimedia.org/T441145) [22:34:58] (03merge) 10pacmand: Stop charts growing on every resize; do not cache index.html [toolforge-repos/wsstats] - 10https://gitlab.wikimedia.org/toolforge-repos/wsstats/-/merge_requests/3 [22:35:01] RESOLVED: OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [22:49:25] (03open) 10tlepage: chart: move prometheusUrls to the top level of the values [repos/cloud/toolforge/alerts-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/19 (https://phabricator.wikimedia.org/T441144 https://phabricator.wikimedia.org/T441145) [23:00:15] 10Cloud-VPS (Debian Bullseye Deprecation), 10Beta-Cluster-Infrastructure, 07Epic, 13Patch-For-Review, 06Release-Engineering-Team (Priority Backlog 📥): Migrate deployment-prep away from Debian Bullseye to Bookworm/Trixie - https://phabricator.wikimedia.org/T401839#12529233 (10bd808) 05Open→03In progress [23:00:22] 10Cloud-VPS (Debian Bullseye Deprecation), 10Beta-Cluster-Infrastructure, 07Epic, 13Patch-For-Review, 06Release-Engineering-Team (Priority Backlog 📥): Migrate deployment-prep away from Debian Bullseye to Bookworm/Trixie - https://phabricator.wikimedia.org/T401839#12529235 (10bd808) p:05Triage→03High [23:02:18] 10Openstack-Magnum, 06tools-infrastructure-team, 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team (Priority Backlog 📥): Rebuild the deployment-wikikube-v127 magnum cluster with the new magnum-cluster-api driver and k8s v1.34 - https://phabricator.wikimedia.org/T437500#12529248 (10bd808) [23:03:59] (03open) 10tlepage: Reject non-api-gateway traffic via NetworkPolicy [repos/cloud/toolforge/alerts-api] (dummy-api-endpoints) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/20 [23:04:27] (03update) 10tlepage: Reject non-api-gateway traffic via NetworkPolicy [repos/cloud/toolforge/alerts-api] (dummy-api-endpoints) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/20 (https://phabricator.wikimedia.org/T441144 https://phabricator.wikimedia.org/T441145) [23:05:30] 10Openstack-Magnum, 06tools-infrastructure-team, 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team (Priority Backlog 📥): Rebuild the deployment-wikikube-v127 magnum cluster with the new magnum-cluster-api driver and k8s v1.36 - https://phabricator.wikimedia.org/T437500#12529251 (10bd808) [23:05:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [23:15:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [23:27:22] 10Toolforge, 06tools-platform-team, 07OKR-Work, 13Patch-For-Review: [alerts-api] List all active alerts for a tool - https://phabricator.wikimedia.org/T441144#12529294 (10TLepage-WMF) For an overview of how the various MRs attached to this task (and T441145) relate to each other: ` @ desired end state ├─╮... [23:31:59] FIRING: BackupCoverageDrop: volume backup coverage dropped on cloudbackup2004:9100 - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Backup#BackupCoverageDrop - https://grafana.wikimedia.org/d/fisr2p9/wmcs-backup?viewPanel=21&var-kind=volume&var-site=codfw - https://alerts.wikimedia.org/?q=alertname%3DBackupCoverageDrop [23:52:20] 10Toolforge, 06tools-platform-team, 07Epic, 07OKR-Work: ST5.4.3 Implement Toolforge UI MVP - https://phabricator.wikimedia.org/T440589#12529327 (10TLepage-WMF) Weekly Asana update: ### Summary **This week (phase //Design//):** - Submitted patches in alerts-api for review (T441144 and T441145). - Deployed...