[00:12:03] (TfInfraTestApplyFailed) resolved: Terraform failed to apply/create the resounces on tf-bastion - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TfInfraTestApplyFailed - https://prometheus-alerts.wmcloud.org/?q=alertname%3DTfInfraTestApplyFailed [01:53:59] (PuppetConstantChange) firing: Puppet performing a change on every puppet run on cloudweb2002-dev:9100 - https://puppetboard.wikimedia.org/nodes?status=changed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetConstantChange [06:06:29] (PuppetConstantChange) resolved: Puppet performing a change on every puppet run on cloudweb2002-dev:9100 - https://puppetboard.wikimedia.org/nodes?status=changed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetConstantChange [11:38:59] 10Grid-Engine-to-K8s-Migration: Migrate cdrbot from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T319620 (10Muhammed4IT) I cleared my job list, i ran the script, it worked. Then inserted it to the job list (`toolforge jobs run myjob --command ./ckbCAB.sh --image bullseye`), it... [12:30:14] 10Grid-Engine-to-K8s-Migration: Migrate cdrbot from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T319620 (10Kizule) Umm, have you disabled the tool by any chance in toolsadmin? https://wikitech.wikimedia.org/wiki/Help:Toolforge/Tool_Accounts#Delete_a_tool_account [18:21:07] 10Tool-bub2, 10Internet-Archive, 10Outreach-Programs-Projects, 10Outreachy (Round 27): For PDL, download and stream the PDF if available - https://phabricator.wikimedia.org/T348188 (10wassan.anmol117) 05Open→03Resolved This is done. [18:22:01] 10Tool-bub2: npm install not working - https://phabricator.wikimedia.org/T348833 (10wassan.anmol117) This is working now. All the peer dependencies have been resolved. [18:22:06] 10Tool-bub2: npm install not working - https://phabricator.wikimedia.org/T348833 (10wassan.anmol117) 05Open→03Resolved [18:23:07] 10Tool-bub2: Merge conflict in LICENSE.md - https://phabricator.wikimedia.org/T349950 (10wassan.anmol117) 05Open→03Resolved a:03wassan.anmol117 This is completed. [18:24:29] !log taavi@cloudcumin1001 tools START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api [18:24:41] 10Tool-bub2, 10Outreach-Programs-Projects, 10Outreachy (Round 27): Redesign HTML template as per Codex design - https://phabricator.wikimedia.org/T348413 (10wassan.anmol117) 05Open→03Invalid Since API:EmailUser doesn't support custom HTML, it cannot be integrated until the support for custom HTML is pres... [18:24:43] !log taavi@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api [18:26:11] 10Tool-bub2, 10Outreach-Programs-Projects, 10Outreachy (Round 27): Title not being shown and placeholder text is getting cut - https://phabricator.wikimedia.org/T348600 (10wassan.anmol117) 05Open→03Resolved [18:26:21] 10Tool-bub2, 10Internet-Archive, 10Outreach-Programs-Projects, 10Outreachy (Round 27): Fix preview for books with long description - https://phabricator.wikimedia.org/T348411 (10wassan.anmol117) 05Open→03Resolved This is completed. [18:26:51] 10Tool-bub2: QueueTable Footer Pagination inconsistent style - https://phabricator.wikimedia.org/T349567 (10wassan.anmol117) 05Open→03Resolved a:03wassan.anmol117 [18:27:15] 10Tool-bub2, 10Internet-Archive, 10Outreach-Programs-Projects, 10Outreachy (Round 27): Handle PDL library failures - https://phabricator.wikimedia.org/T348412 (10wassan.anmol117) 05Open→03Resolved [20:08:35] 10Grid-Engine-to-K8s-Migration: Migrate dplbot from Toolforge GridEngine to Toolforge Kubernetes - https://phabricator.wikimedia.org/T319701 (10russblau) I've got another problem. I'm running my webservice with "--backend kubernetes php7.4". I've rewritten a number of web tools that access ToolsDB to use the env... [20:18:33] (SystemdUnitDown) firing: The service unit nova-fullstack.service is in failed status on host cloudcontrol1006. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/SystemdUnitDown - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=cloudcontrol1006 - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitDown [22:13:33] (SystemdUnitDown) firing: The systemd unit nova-fullstack.service on node cloudcontrol1006 has been failing for more than two hours. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/SystemdUnitDown - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=cloudcontrol1006 - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitDown [22:13:48] 10cloud-services-team: SystemdUnitDown Unit nova-fullstack.service on node cloudcontrol1006 has been down for long. - https://phabricator.wikimedia.org/T353991 (10phaultfinder) [22:54:19] (HAProxyBackendUnavailable) firing: HAProxy service neutron-api_backend backend cloudcontrol1007.private.eqiad.wikimedia.cloud is down - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable [22:59:19] (HAProxyBackendUnavailable) resolved: HAProxy service neutron-api_backend backend cloudcontrol1007.private.eqiad.wikimedia.cloud is down - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable