[00:19:56] RESOLVED: [2x] ProbeDown: Service tools-k8s-haproxy-5:443 has failed probes (http_admin_toolforge_org_ip4) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [00:31:55] FIRING: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [00:58:21] (03open) 10ladsgroup: Add option to create the dns switchover patch automatically [toolforge-repos/switchmaster] - 10https://gitlab.wikimedia.org/toolforge-repos/switchmaster/-/merge_requests/13 (https://phabricator.wikimedia.org/T399891) [01:16:55] FIRING: [2x] ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 close to running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [01:21:55] FIRING: [2x] ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 close to running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [02:00:56] FIRING: ProbeDown: Service tools-k8s-haproxy-5:443 has failed probes (http_admin_toolforge_org_ip4) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [02:01:03] FIRING: [2x] ToolforgeKubernetesWorkerTooManyDProcesses: Node tools-k8s-worker-nfs-2 has at least 12 procs in D state, and may be having NFS/IO issues - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesWorkerTooManyDProcesses - https://grafana.wmcloud.org/d/3jhWxB8Vk/toolforge-general-overview - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesWorkerTooManyDProcesse [02:05:56] RESOLVED: ProbeDown: Service tools-k8s-haproxy-5:443 has failed probes (http_admin_toolforge_org_ip4) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [02:07:56] FIRING: ProbeDown: Service tools-k8s-haproxy-5:443 has failed probes (http_admin_toolforge_org_ip4) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [02:12:56] RESOLVED: [2x] ProbeDown: Service tools-k8s-haproxy-5:443 has failed probes (http_admin_toolforge_org_ip4) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [03:05:56] FIRING: ProbeDown: Service tools-k8s-haproxy-5:443 has failed probes (http_admin_toolforge_org_ip4) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [03:10:56] RESOLVED: ProbeDown: Service tools-k8s-haproxy-5:443 has failed probes (http_admin_toolforge_org_ip4) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [05:41:55] RESOLVED: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [05:42:25] FIRING: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [05:57:25] RESOLVED: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [06:31:55] FIRING: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [06:36:55] RESOLVED: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [09:22:23] (03PS1) 10Ologuie Arlette: arrange popular actors design and compare page design [labs/tools/WdTmCollab] - 10https://gerrit.wikimedia.org/r/1189950 [09:30:36] 10Tool-unwatchlist, 10MediaWiki-Watchlist, 06Moderator-Tools-Team, 10Technical-Tool-Request: Create a tool to unwatchlist large numbers of pages - https://phabricator.wikimedia.org/T401274#11198942 (10Samwalton9-WMF) I'm getting a Toolforge error when attempting to login. I click login, get taken to Meta t... [09:33:42] 10Tool-unwatchlist, 10MediaWiki-Watchlist, 06Moderator-Tools-Team, 10Technical-Tool-Request: Create a tool to unwatchlist large numbers of pages - https://phabricator.wikimedia.org/T401274#11198944 (10Novem_Linguae) Careful posting links like that in public tickets. I'm not 100% sure, but that URL is givin... [09:37:24] 10Tool-unwatchlist, 10MediaWiki-Watchlist, 06Moderator-Tools-Team, 10Technical-Tool-Request: Create a tool to unwatchlist large numbers of pages - https://phabricator.wikimedia.org/T401274#11198945 (10Samwalton9-WMF) >>! In T401274#11198944, @Novem_Linguae wrote: > Careful posting links like that in public... [09:40:48] (03CR) 10Ologuie Arlette: "waiting for review!" [labs/tools/WdTmCollab] - 10https://gerrit.wikimedia.org/r/1189950 (owner: 10Ologuie Arlette) [09:42:04] 10Tool-unwatchlist: Enable editors to unwatchlist user pages of blocked users - https://phabricator.wikimedia.org/T405140 (10Samwalton9) 03NEW [09:45:13] 10Tool-unwatchlist: Enable editors to unwatchlist pages by namespace - https://phabricator.wikimedia.org/T405141 (10Samwalton9) 03NEW [09:47:47] 10Tool-unwatchlist: Enable editors to unwatchlist deleted pages - https://phabricator.wikimedia.org/T405142 (10Samwalton9) 03NEW [09:48:42] 10Tool-unwatchlist: Enable editors to unwatchlist deleted pages - https://phabricator.wikimedia.org/T405142#11198984 (10Samwalton9) [09:50:19] 10Tool-unwatchlist: Enable editors to unwatchlist redirect pages - https://phabricator.wikimedia.org/T405143 (10Samwalton9) 03NEW [09:51:14] 10Tool-unwatchlist, 10MediaWiki-Watchlist, 06Moderator-Tools-Team, 10Technical-Tool-Request: Create a tool to unwatchlist large numbers of pages - https://phabricator.wikimedia.org/T401274#11198997 (10Samwalton9) [10:30:37] (03CR) 10Essa237: [C:03+1] arrange popular actors design and compare page design [labs/tools/WdTmCollab] - 10https://gerrit.wikimedia.org/r/1189950 (owner: 10Ologuie Arlette) [10:41:15] 10Tool-unwatchlist: Enable editors to unwatchlist pages by namespace - https://phabricator.wikimedia.org/T405141#11199026 (10Bugreporter2) This could be done as a more general case of anything matching a particular regex. [10:50:56] 10Tool-unwatchlist, 10MediaWiki-Redirects: Enable editors to unwatchlist redirect pages - https://phabricator.wikimedia.org/T405143#11199028 (10Bugreporter2) [11:35:33] 10Tool-unwatchlist, 10MediaWiki-Watchlist, 06Moderator-Tools-Team, 10Technical-Tool-Request: Create a tool to unwatchlist large numbers of pages - https://phabricator.wikimedia.org/T401274#11199040 (10Chlod) >>! In T401274#11198942, @Samwalton9-WMF wrote: > I'm getting a Toolforge error when attempting to... [12:31:55] FIRING: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [12:37:16] 10Tool-unwatchlist: Enable editors to unwatchlist deleted pages - https://phabricator.wikimedia.org/T405142#11199044 (10Bugreporter2) Sorry to be pedantic, but I guess this is probably meant to be "allow editors to unwatch non-existent pages." As in, the process will check whether the page exists, and remove it... [12:46:53] 10Toolforge (Toolforge iteration 24): [jobs-api] loki logs take really long to appear - https://phabricator.wikimedia.org/T404176#11199047 (10DamianZaremba) This may also (at least partly) explain T402736 [12:47:48] 06cloud-services-team, 10Toolforge: toolforge logs appears to suffer from intermittent latency - https://phabricator.wikimedia.org/T402736#11199049 (10DamianZaremba) Example occurrences: Timed out after 2min waiting for the log entries: ` 2025-08-24 16:54:38,388 [INFO] [coord-random-edits] Job succeeded 2025-... [12:54:23] 10Tool-unwatchlist, 10MediaWiki-Watchlist, 06Moderator-Tools-Team, 10Technical-Tool-Request: Create a tool to unwatchlist large numbers of pages - https://phabricator.wikimedia.org/T401274#11199050 (10Samwalton9) >>! In T401274#11199040, @Chlod wrote: >>>! In T401274#11198942, @Samwalton9-WMF wrote: >> I'm... [13:01:55] RESOLVED: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [13:04:36] 10Tool-unwatchlist, 10MediaWiki-Watchlist, 06Moderator-Tools-Team, 10Technical-Tool-Request: Create a tool to unwatchlist large numbers of pages - https://phabricator.wikimedia.org/T401274#11199051 (10Chlod) Yeah, I'm in the middle of trying different things out to see if the proxy is blocking something sp... [13:31:55] FIRING: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [13:47:52] 10Tool-unwatchlist, 10MediaWiki-Watchlist, 06Moderator-Tools-Team, 10Technical-Tool-Request: Create a tool to unwatchlist large numbers of pages - https://phabricator.wikimedia.org/T401274#11199055 (10Chlod) Should work now. It seems like Nginx just doesn't like super long cookies. [13:56:55] RESOLVED: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [14:31:55] FIRING: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [14:41:55] RESOLVED: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [14:47:21] 10Tool-unwatchlist, 10MediaWiki-Watchlist, 06Moderator-Tools-Team, 10Technical-Tool-Request: Create a tool to unwatchlist large numbers of pages - https://phabricator.wikimedia.org/T401274#11199072 (10Samwalton9) I tested with my main account (with confidence thanks to the numerous options to download list... [15:11:55] FIRING: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [15:27:46] 10Tool-unwatchlist, 10MediaWiki-Watchlist, 06Moderator-Tools-Team, 10Technical-Tool-Request: Create a tool to unwatchlist large numbers of pages - https://phabricator.wikimedia.org/T401274#11199091 (10Samwalton9) Results from some more thorough testing using a test account: * The randomly selected message... [15:31:49] 10Tool-unwatchlist: Enable editors to unwatchlist non-existent pages - https://phabricator.wikimedia.org/T405142#11199093 (10Samwalton9) [15:33:26] 10Tool-unwatchlist: Enable editors to unwatchlist non-existent pages - https://phabricator.wikimedia.org/T405142#11199095 (10Samwalton9) [15:37:06] 10Tool-unwatchlist: Enable editors to unwatchlist pages by namespace - https://phabricator.wikimedia.org/T405141#11199096 (10Samwalton9) >>! In T405141#11199026, @Bugreporter2 wrote: > This could be done as a more general case of anything matching a particular regex. It could, but given that this tool is target... [16:01:55] RESOLVED: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [16:31:55] FIRING: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [18:56:55] RESOLVED: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [19:27:22] 10Tool-unwatchlist, 10MediaWiki-Watchlist, 06Moderator-Tools-Team, 10Technical-Tool-Request: Create a tool to unwatchlist large numbers of pages - https://phabricator.wikimedia.org/T401274#11199182 (10Samwalton9) >>! In T401274#11199091, @Samwalton9 wrote: > * I noticed that the tool seems to be assessing... [19:31:55] FIRING: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [19:32:35] 10Tool-unwatchlist: Enable editors to unwatchlist non-existent pages - https://phabricator.wikimedia.org/T405142#11199185 (10Samwalton9) [20:01:55] RESOLVED: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [20:31:55] FIRING: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [20:33:21] (03CR) 10Jean-Frédéric: [C:03+2] handle missing wd_item values gracefully [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/1189582 (https://phabricator.wikimedia.org/T346681) (owner: 10Lokal Profil) [20:35:21] (03Merged) 10jenkins-bot: handle missing wd_item values gracefully [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/1189582 (https://phabricator.wikimedia.org/T346681) (owner: 10Lokal Profil) [21:33:26] 10Tool-gawa: [Code] Conception de la page ERREUR 404 - https://phabricator.wikimedia.org/T403592#11199225 (10poro26) 05Open→03Resolved Lien de la page ERREUR 404 : https://gawa.toolforge.org/erreur404 [21:34:46] 10Tool-gawa: [Code] Conception de la page A PROPOS - https://phabricator.wikimedia.org/T403582#11199228 (10poro26) 05Open→03Resolved Lien de la page A PROPOS : https://gawa.toolforge.org/about [21:35:58] 10Tool-gawa: [Documentation] Rédaction du contenu de la page A PROPOS - https://phabricator.wikimedia.org/T403589#11199231 (10poro26) 05Open→03Resolved Tâche effectuée : https://gawa.toolforge.org/about [21:41:19] 10Tool-gawa: [Documentation] Mise à jour du contenu de la page Méta-Wiki de l'outil GAWA - https://phabricator.wikimedia.org/T405148 (10poro26) 03NEW [21:56:55] RESOLVED: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [22:51:55] FIRING: ToolforgeKubernetesCapacity: Kubernetes cluster k8s.tools.eqiad1.wikimedia.cloud:6443 in risk of running out of cpu - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/ToolforgeKubernetesCapacity - https://grafana.wmcloud.org/d/8GiwHDL4k/kubernetes-cluster-overview?orgId=1 - https://prometheus-alerts.wmcloud.org/?q=alertname%3DToolforgeKubernetesCapacity [23:18:53] 10Tool-k8s-status: k8s-status can't show information about one tool - https://phabricator.wikimedia.org/T405150 (10Bewfip) 03NEW