[00:10:03] (InstanceDown) firing: Project tf-infra-test instance tf-infra-test is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [00:15:03] (InstanceDown) resolved: Project tf-infra-test instance tf-infra-test is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [01:22:20] 10Toolforge (Toolforge iteration 02), 10Toolforge Build Service (Beta release), 10User-Raymond_Ndibe, 10User-dcaro: Add a way to wait for a Toolforge build to finish - https://phabricator.wikimedia.org/T337043 (10bd808) {T352561} was closed as a dup of this task which made sense, but I'm not obviously seei... [01:27:08] 10Tool-bridgebot: Remove #wikimedia-cloud <-> chat.wmcloud.org bridge - https://phabricator.wikimedia.org/T352937 (10bd808) [01:39:24] 10Tool-bridgebot, 10User-bd808: Remove #wikimedia-cloud <-> chat.wmcloud.org bridge - https://phabricator.wikimedia.org/T352937 (10bd808) 05Open→03In progress p:05Triage→03Medium a:03bd808 [01:43:33] 10Tool-bridgebot: Remove italian-wikipedia mattermost bridge - https://phabricator.wikimedia.org/T352938 (10bd808) [01:55:03] (TfInfraTestApplyFailed) firing: Terraform failed to apply/create the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestApplyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestApplyFailed [01:55:03] (TfInfraTestDestroyFailed) firing: Terraform failed to destroy the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestDestroyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestDestroyFailed [02:01:56] 10Tool-bridgebot, 10User-bd808: Remove italian-wikipedia mattermost bridge - https://phabricator.wikimedia.org/T352938 (10bd808) 05Open→03Resolved p:05Triage→03Medium a:03bd808 [02:02:29] 10Tool-bridgebot, 10User-bd808: Remove #wikimedia-cloud <-> chat.wmcloud.org bridge - https://phabricator.wikimedia.org/T352937 (10bd808) 05In progress→03Resolved [04:42:08] !log taavi@cloudcumin1001 tools START - Cookbook wmcs.toolforge.remove_grid_node for tools-sgeweblight-10-26 [04:50:03] (InstanceDown) firing: Project tools instance tools-sgeweblight-10-26 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [04:55:03] (TfInfraTestDestroyFailed) firing: Terraform failed to destroy the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestDestroyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestDestroyFailed [04:55:03] (TfInfraTestApplyFailed) firing: Terraform failed to apply/create the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestApplyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestApplyFailed [04:55:03] (InstanceDown) resolved: Project tools instance tools-sgeweblight-10-26 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [05:37:45] (ProbeDown) firing: Service tools-k8s-haproxy-4:30000 has failed probes (http_admin_toolforge_org_ip4) - https://wikitech.wikimedia.org/wiki/Runbook#tools-k8s-haproxy-4:30000 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [05:42:45] (ProbeDown) resolved: Service tools-k8s-haproxy-4:30000 has failed probes (http_admin_toolforge_org_ip4) - https://wikitech.wikimedia.org/wiki/Runbook#tools-k8s-haproxy-4:30000 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [06:17:28] 10Toolforge Jobs framework: toolforge-jobs --wait will only wait 5 minutes - https://phabricator.wikimedia.org/T352945 (10Legoktm) [06:19:20] 10Toolforge Jobs framework: toolforge-jobs --wait will only wait 5 minutes - https://phabricator.wikimedia.org/T352945 (10Legoktm) The main workflow here is using the ~~grid~~ jobs framework to compile something with `&& webservice restart` [07:55:03] (TfInfraTestApplyFailed) firing: Terraform failed to apply/create the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestApplyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestApplyFailed [07:55:03] (TfInfraTestDestroyFailed) firing: Terraform failed to destroy the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestDestroyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestDestroyFailed [07:55:26] 10Toolforge Jobs framework: toolforge-jobs --wait will only wait 5 minutes - https://phabricator.wikimedia.org/T352945 (10JJMC89) It only waits for up to 5 minutes since {1cc8196} for {T285979}. [08:12:38] 10Toolforge (Toolforge iteration 02), 10Patch-For-Review: Add `toolforge build quota` command - https://phabricator.wikimedia.org/T341068 (10CodeReviewBot) sstefanova merged https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/merge_requests/62 api: add quota endpoint [08:13:35] 10Toolforge (Toolforge iteration 02), 10Patch-For-Review: Add `toolforge build quota` command - https://phabricator.wikimedia.org/T341068 (10CodeReviewBot) sstefanova merged https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-cli/-/merge_requests/28 cli: add quota command [09:29:23] 10Toolforge (Toolforge iteration 02), 10Patch-For-Review: Add `toolforge build quota` command - https://phabricator.wikimedia.org/T341068 (10CodeReviewBot) sstefanova opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/144 builds-api: bump to 0.0.112-20231207081230-2d86... [09:30:36] 10Toolforge (Toolforge iteration 02), 10Patch-For-Review, 10User-Raymond_Ndibe: [apis] nginx fails to reload on config change - https://phabricator.wikimedia.org/T350928 (10CodeReviewBot) sstefanova opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/144 builds-api: b... [09:38:16] 10Toolforge (Toolforge iteration 02), 10Patch-For-Review: Add `toolforge build quota` command - https://phabricator.wikimedia.org/T341068 (10CodeReviewBot) sstefanova opened https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-cli/-/merge_requests/30 d/changelog: bump to 0.0.7 [10:12:17] 10Toolforge (Toolforge iteration 02): [docs] Update Toolforge component README's - https://phabricator.wikimedia.org/T352964 (10Slst2020) [10:14:21] 10Toolforge (Toolforge iteration 02), 10Patch-For-Review: Add `toolforge build quota` command - https://phabricator.wikimedia.org/T341068 (10CodeReviewBot) sstefanova merged https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-cli/-/merge_requests/30 d/changelog: bump to 0.0.7 [10:44:31] 10Cloud-VPS, 10SRE, 10observability, 10Patch-For-Review, and 2 others: ossl rsyslog errors post-migration - https://phabricator.wikimedia.org/T351710 (10fgiunchedi) I've added some more visibility into how much we are writing to local files, compared to the amount of logs we are receiving. Turns out we re... [10:55:03] (TfInfraTestApplyFailed) firing: Terraform failed to apply/create the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestApplyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestApplyFailed [10:55:03] (TfInfraTestDestroyFailed) firing: Terraform failed to destroy the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestDestroyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestDestroyFailed [11:03:09] 10cloud-services-team: CephSlowOps Ceph cluster in eqiad has slow ops, which might be blocking some writes - https://phabricator.wikimedia.org/T352570 (10dcaro) [11:03:15] 10Cloud-VPS, 10cloud-services-team (FY2023/2024-Q1-Q2), 10DC-Ops, 10SRE, 10ops-eqiad: cloudcephosd1021-1034: hard drive sector errors increasing - https://phabricator.wikimedia.org/T348643 (10dcaro) [11:06:26] 10Cloud-VPS (Quota-requests), 10MinT, 10Language-Team (Language-2023-October-December): Increase quota to create large instance for MinT - https://phabricator.wikimedia.org/T352136 (10Pginer-WMF) [11:57:37] (03PS1) 10Muehlenhoff: Remove obsolete dummy certs [labs/private] - 10https://gerrit.wikimedia.org/r/981301 (https://phabricator.wikimedia.org/T350686) [12:09:19] (HAProxyBackendUnavailable) firing: HAProxy service neutron-api_backend backend cloudcontrol1005.private.eqiad.wikimedia.cloud is down - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable [12:14:19] (HAProxyBackendUnavailable) resolved: HAProxy service neutron-api_backend backend cloudcontrol1005.private.eqiad.wikimedia.cloud is down - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable [13:04:28] (03CR) 10Muehlenhoff: [V: 03+2 C: 03+2] Remove obsolete dummy certs [labs/private] - 10https://gerrit.wikimedia.org/r/981301 (https://phabricator.wikimedia.org/T350686) (owner: 10Muehlenhoff) [13:14:11] PROBLEM - Host cloudcephosd1001 is DOWN: PING CRITICAL - Packet loss = 100% [13:14:29] RECOVERY - Host cloudcephosd1001 is UP: PING OK - Packet loss = 0%, RTA = 0.25 ms [13:55:03] (TfInfraTestApplyFailed) firing: Terraform failed to apply/create the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestApplyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestApplyFailed [13:55:03] (TfInfraTestDestroyFailed) firing: Terraform failed to destroy the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestDestroyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestDestroyFailed [14:25:29] 10Toolforge (Toolforge iteration 02), 10Patch-For-Review, 10User-dcaro: [builds-builder] Investigate how to enable mono/dotnet/c# and implement the best one to unblock us to migrate tools - https://phabricator.wikimedia.org/T352774 (10dcaro) This is proving a bit more difficult than expected. I was able to... [14:28:57] 10Toolforge Jobs framework: toolforge-jobs --wait will only wait 5 minutes - https://phabricator.wikimedia.org/T352945 (10taavi) a:03taavi [14:29:50] 10Grid-Engine-to-K8s-Migration, 10Toolforge (Software install/update), 10Toolforge Jobs framework, 10WMCZ-General: Make it possible to run pandoc in Toolforge's jobs framework - https://phabricator.wikimedia.org/T345029 (10taavi) This seems like a good use case for a custom image built with the [[ https://... [14:46:32] !log sstefanova@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api [14:46:45] !log sstefanova@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api [14:52:05] 10Toolforge Jobs framework, 10Patch-For-Review: toolforge-jobs --wait will only wait 5 minutes - https://phabricator.wikimedia.org/T352945 (10CodeReviewBot) taavi opened https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/5 Allow specifying a timeout for --wait [14:54:11] 10Toolforge Jobs framework: Replace already completed one-off jobs when starting a new one - https://phabricator.wikimedia.org/T352989 (10taavi) [14:58:19] (HAProxyBackendUnavailable) firing: (3) HAProxy service radosgw-api_backend backend cloudcontrol1005.private.eqiad.wikimedia.cloud is down - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable [14:59:19] (HAProxyServiceUnavailable) firing: (2) HAProxy service radosgw-api_backend has no available backends on cloudlb1001:9900 - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyServiceUnavailable [14:59:24] 10cloud-services-team: HAProxyServiceUnavailable - https://phabricator.wikimedia.org/T352544 (10phaultfinder) [15:03:19] (HAProxyBackendUnavailable) resolved: (3) HAProxy service radosgw-api_backend backend cloudcontrol1005.private.eqiad.wikimedia.cloud is down - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable [15:04:19] (HAProxyServiceUnavailable) resolved: (2) HAProxy service radosgw-api_backend has no available backends on cloudlb1001:9900 - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyServiceUnavailable [15:17:25] 10Toolforge Jobs framework: toolforge jobs restart sometimes times out - https://phabricator.wikimedia.org/T352874 (10taavi) a:03taavi [15:17:59] 10Tool-bridgebot, 10User-bd808: Remove #wikimedia-cloud <-> chat.wmcloud.org bridge - https://phabricator.wikimedia.org/T352937 (10bd808) I accidentally introduced a regression. Fix inbound. `lang=irc [07:35] < wm-bb> [telegram]  bd808: is it intentional that the bridgebot messages here n... [15:18:58] 10Toolforge Jobs framework, 10Patch-For-Review: toolforge jobs restart sometimes times out - https://phabricator.wikimedia.org/T352874 (10CodeReviewBot) taavi opened https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/49 job: Specify a lower termination grace period [15:30:19] (HAProxyBackendUnavailable) firing: HAProxy service neutron-api_backend backend cloudcontrol1005.private.eqiad.wikimedia.cloud is down - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable [15:35:20] (HAProxyBackendUnavailable) resolved: HAProxy service neutron-api_backend backend cloudcontrol1005.private.eqiad.wikimedia.cloud is down - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable [15:50:26] 10Cloud-VPS, 10SRE, 10observability, 10Patch-For-Review, and 2 others: ossl rsyslog errors post-migration - https://phabricator.wikimedia.org/T351710 (10fgiunchedi) >>! In T351710#9385748, @Stashbot wrote: > {nav icon=file, name=Mentioned in SAL (#wikimedia-operations), href=https://sal.toolforge.org/log/I... [15:58:31] (03PS1) 10BryanDavis: check_username_create: Guard against missing response keys [labs/striker] - 10https://gerrit.wikimedia.org/r/981356 [15:59:56] 10cloud-services-team (FY2023/2024-Q1-Q2), 10Cloud-Services-Origin-Team, 10Cloud-Services-Worktype-Maintenance, 10Goal, and 2 others: ceph: upgrade hosts to bullseye - https://phabricator.wikimedia.org/T309789 (10dcaro) [16:06:13] 10Toolforge (Toolforge iteration 02), 10Patch-For-Review, 10User-dcaro: [builds-builder] Investigate how to enable mono/dotnet/c# and implement the best one to unblock us to migrate tools - https://phabricator.wikimedia.org/T352774 (10dcaro) I was able to migrate it to cnb, still have to test it works as exp... [16:11:57] 10Cloud-VPS: enable lists.wikimedia.org or wikimedia.org email addresses to receive dmarc reports for *.wmflabs.org - https://phabricator.wikimedia.org/T352902 (10jijiki) p:05Triage→03Medium [16:25:35] 10Grid-Engine-to-K8s-Migration: Migrate wikihistory from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T320157 (10Wurgl) I tried the following: Create an Aptfile like ` php ` and tried to start with toolforge jobs run --command 'php dewiki_update_new.php 1' --image mono6.8 -o... [16:55:03] (TfInfraTestApplyFailed) firing: Terraform failed to apply/create the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestApplyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestApplyFailed [16:55:03] (TfInfraTestDestroyFailed) firing: Terraform failed to destroy the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestDestroyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestDestroyFailed [17:03:10] 10cloud-services-team, 10Observability-Metrics: Deploy 'cloud' Prometheus instance to codfw - https://phabricator.wikimedia.org/T350010 (10fgiunchedi) [17:32:14] 10Cloud-VPS, 10cloud-services-team (FY2023/2024-Q1-Q2), 10Goal, 10Patch-For-Review: Support 'unmanaged' projects in cloud-vps - https://phabricator.wikimedia.org/T326818 (10Andrew) > Some questions arise (for us mostly, can be after starting the service): > * How will we monitor that the images uploaded ar... [17:50:00] 10Tool-ducttape, 10Abstract Wikipedia team, 10Patch-For-Review: Add horizontal scaling to 0 for inactive deployments - https://phabricator.wikimedia.org/T334247 (10Jdforrester-WMF) 05Open→03Resolved a:03SDunlap [17:52:22] 10Tool-ducttape, 10Abstract Wikipedia team: Add documentation for retriggering a failed pipeline to AW developer cheatsheet - https://phabricator.wikimedia.org/T333192 (10Jdforrester-WMF) a:05SDunlap→03Jdforrester-WMF [18:17:33] 10Horizon: Cannot view database instance logs in Horizon - https://phabricator.wikimedia.org/T353010 (10JJMC89) [18:21:56] 10Cloud-VPS: Cannot connect to database after restart - https://phabricator.wikimedia.org/T353011 (10JJMC89) [19:23:16] 10Cloud-VPS: Cannot connect to database after restart - https://phabricator.wikimedia.org/T353011 (10taavi) The node is out of disk: ` Filesystem Size Used Avail Use% Mounted on /dev/sdb 975M 924M 0 100% /var/lib/mysql ` Looks like Trove did not manage to resize the file system as required: the... [19:40:51] 10Cloud-VPS, 10cloud-services-team: Consider removing Postgres support from Trove - https://phabricator.wikimedia.org/T353018 (10Andrew) [19:47:03] 10Cloud-VPS, 10cloud-services-team, 10PostgreSQL: Consider removing Postgres support from Trove - https://phabricator.wikimedia.org/T353018 (10Aklapper) [19:51:38] 10Cloud-VPS: Cloud-VPS OpenTofu provider is not working on M1 Macs - https://phabricator.wikimedia.org/T353019 (10fnegri) [19:53:25] 10Grid-Engine-to-K8s-Migration: Migrate videoconvert from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T320126 (10Lokal_Profil) @komla Firstly. Don't expect a reply if you only ping the forst maintainer on the list. I'm a maintainer of this tool and was only made aware becaus... [19:55:03] (TfInfraTestApplyFailed) firing: Terraform failed to apply/create the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestApplyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestApplyFailed [19:55:03] (TfInfraTestDestroyFailed) firing: Terraform failed to destroy the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestDestroyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestDestroyFailed [19:57:07] 10Grid-Engine-to-K8s-Migration: Migrate heritage from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T319787 (10Lokal_Profil) Please extend the deadline for this tool past the winter holidays to give us a chance at figuring out the issues around the remaining cron-jobs [19:57:20] 10Cloud-VPS: Cannot connect to database after restart - https://phabricator.wikimedia.org/T353011 (10JJMC89) 05Open→03Resolved a:03Andrew [20:02:49] 10Grid-Engine-to-K8s-Migration: Migrate stockholm-mania from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T320063 (10Lokal_Profil) a:05Antoine2711→03Lokal_Profil @komla The third case of one of my tools that I dont get a ping about because you just picked the first maintai... [20:21:47] 10PAWS, 10OpenRefine: Upgrade OpenRefine on PAWS to 3.7.7. - https://phabricator.wikimedia.org/T353021 (10Spinster) [20:23:05] 10PAWS: Upgrade openrefine to 3.7.7 - https://phabricator.wikimedia.org/T352865 (10rook) [20:23:22] 10PAWS, 10OpenRefine: Upgrade OpenRefine on PAWS to 3.7.7. - https://phabricator.wikimedia.org/T353021 (10rook) [20:23:35] 10PAWS, 10OpenRefine: Upgrade OpenRefine on PAWS to 3.7.7. - https://phabricator.wikimedia.org/T353021 (10Spinster) 05Open→03Invalid Oops, duplicate of T352865 which I had failed to find here! [20:24:23] 10PAWS, 10OpenRefine: Upgrade OpenRefine on PAWS to 3.7.7. - https://phabricator.wikimedia.org/T353021 (10rook) 05Invalid→03Open [20:24:36] 10PAWS, 10OpenRefine: Upgrade OpenRefine on PAWS to 3.7.7. - https://phabricator.wikimedia.org/T353021 (10rook) >>! In T353021#9391731, @Spinster wrote: > Oops, duplicate of T352865 which I had failed to find here! C'est pas grave. Your ticket is better worded. [20:34:12] 10PAWS, 10OpenRefine: Upgrade OpenRefine on PAWS to 3.7.7. - https://phabricator.wikimedia.org/T353021 (10github-toolforge-bot) vivian-rook opened https://github.com/toolforge/paws/pull/358 [20:34:18] vivian-rook opened https://github.com/toolforge/paws/pull/358 [22:00:38] 10Grid-Engine-to-K8s-Migration: Migrate panoviewer from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T319953 (10Fuzheado) I kindly request that you extend the life of this tool to the February 2024 deadline. We will attempt to migrate it much earlier than that, preferably in D... [22:55:03] (TfInfraTestDestroyFailed) firing: Terraform failed to destroy the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestDestroyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestDestroyFailed [22:55:03] (TfInfraTestApplyFailed) firing: Terraform failed to apply/create the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestApplyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestApplyFailed [23:03:22] 10Grid-Engine-to-K8s-Migration, 10Pywikibot: Migrate pywikibot from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T319981 (10Xqt) @komla: I have no glue, what to do here. [23:17:19] 10Toolforge (Software install/update): Create a kubernetes container with mono and dotnet - https://phabricator.wikimedia.org/T311466 (10Hawkeye7) When I try to use the admin console (https://toolsadmin.wikimedia.org/tools/id/milhistbot/repos/create) to create a new repository it says: No GitLab accounts found... [23:55:39] 10Cloud-VPS: Cloud-VPS OpenTofu provider is not working on M1 Macs - https://phabricator.wikimedia.org/T353019 (10taavi) I don't think this is an Apple Silicon specific thing, I've only done builds for Linux amd64, there are no builds for Darwin at all.