[00:16:03] <wmcs-alerts>	 (InstanceDown) firing: Project tf-infra-test instance tf-infra-test is down   - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown
[00:21:03] <wmcs-alerts>	 (InstanceDown) resolved: Project tf-infra-test instance tf-infra-test is down   - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown
[02:13:49] <jinxer-wm>	 (SystemdUnitDown) firing: The systemd unit nova-fullstack.service on node cloudcontrol1006 has been failing for more than two hours. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/SystemdUnitDown - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=cloudcontrol1006 - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitDown
[03:33:34] <jinxer-wm>	 (SystemdUnitDown) resolved: The systemd unit nova-fullstack.service on node cloudcontrol1006 has been failing for more than two hours. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/SystemdUnitDown - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=cloudcontrol1006 - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitDown
[03:33:34] <jinxer-wm>	 (SystemdUnitDown) resolved: The service unit nova-fullstack.service is in failed status on host cloudcontrol1006. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/SystemdUnitDown - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=cloudcontrol1006 - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitDown
[04:01:40] <wikibugs>	 10Grid-Engine-to-K8s-Migration: Migrate mbh from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T319883 (10MBH) @nskaggs could you explain, step by step, how it can solve my problem?
[08:44:18] <wikibugs>	 10Grid-Engine-to-K8s-Migration: Migrate edgars from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T319717 (10Edgars2007) 05Open→03Resolved a:03Edgars2007
[09:08:06] <wikibugs>	 10Cloud-VPS, 10Community-Tech, 10WS Export: ssh: Connection closed by UNKNOWN port 65535 - https://phabricator.wikimedia.org/T353999 (10Samwilson)
[09:32:19] <jinxer-wm>	 (HAProxyBackendUnavailable) firing: HAProxy service neutron-api_backend backend cloudcontrol1005.private.eqiad.wikimedia.cloud is down - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable
[09:37:19] <jinxer-wm>	 (HAProxyBackendUnavailable) resolved: HAProxy service neutron-api_backend backend cloudcontrol1005.private.eqiad.wikimedia.cloud is down - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable
[11:04:44] <wikibugs>	 10Cloud-VPS, 10Community-Tech, 10WS Export: ssh: Connection closed by UNKNOWN port 65535 - https://phabricator.wikimedia.org/T353999 (10taavi) I see this in `auth.log`: ` 2023-12-24T09:05:34.853537+00:00 wsexport-prod02 sshd[55309]: pam_sss(sshd:account): Access denied for user samwilson: 4 (System error) `...
[12:54:19] <jinxer-wm>	 (HAProxyBackendUnavailable) firing: HAProxy service neutron-api_backend backend cloudcontrol1006.private.eqiad.wikimedia.cloud is down - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable
[12:59:19] <jinxer-wm>	 (HAProxyBackendUnavailable) resolved: HAProxy service neutron-api_backend backend cloudcontrol1006.private.eqiad.wikimedia.cloud is down - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable
[13:09:38] <wikibugs>	 10Grid-Engine-to-K8s-Migration: Migrate cdrbot from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T319620 (10Muhammed4IT) >>! In T319620#9424367, @Kizule wrote: > Umm, have you disabled the tool by any chance in toolsadmin? >  > https://wikitech.wikimedia.org/wiki/Help:Toolforg...
[14:24:20] <wikibugs>	 10Grid-Engine-to-K8s-Migration: Migrate unpatrollededitstats from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T320105 (10Stang) True, I //asked// for adoption but not adopted yet, so I don't think there's much thing I could do?
[14:26:06] <wikibugs>	 10Grid-Engine-to-K8s-Migration: Migrate satori from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T320029 (10Stang) Sorry too lazy to have this done for a really long time :( I've read the instruction like creating the new venv and something else, and I'm planning to do this in...
[14:37:22] <wikibugs>	 10Grid-Engine-to-K8s-Migration: Migrate satori from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T320029 (10Stang) p:05Triage→03High
[14:43:17] <wikibugs>	 10Grid-Engine-to-K8s-Migration: Migrate dibot from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T319676 (10MBH) I'm a ruwiki user MBH, like a Dmitry89, owner of this tool account. Both of us write our bots on C# and run them through mono, he also gave me source code of his bot...
[21:10:01] <wikibugs>	 10Tool-gitlab-account-approval, 10User-bd808: Consider adding Gerrit Trusted-Users group as source of trust - https://phabricator.wikimedia.org/T353914 (10bd808) 05Open→03In progress p:05Triage→03Medium a:03bd808
[21:13:25] <wikibugs>	 10Tool-gitlab-account-approval, 10Patch-For-Review, 10User-bd808: Consider adding Gerrit Trusted-Users group as source of trust - https://phabricator.wikimedia.org/T353914 (10CodeReviewBot) bd808 opened https://gitlab.wikimedia.org/toolforge-repos/gitlab-account-approval/-/merge_requests/8  Trust users trust...