[14:28:59] !log rebooting all k8s-nfs worker nodes for T385264 [14:28:59] andrewbogott: Unknown project "rebooting" [14:29:00] T385264: VM live migration failing for many/most VMs - https://phabricator.wikimedia.org/T385264 [14:29:06] !log tools rebooting all k8s-nfs worker nodes for T385264 [14:29:08] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [14:44:51] andrewbogott: the reboot k8s workers cookbook has a flag to act on all the NFS nodes btw :-) [14:49:39] taavi: I'm being compulsive and avoiding the (few) nodes that don't need a reboot [15:01:48] !log tools rebooting all k8s (non-nfs) worker nodes for T385264 [15:01:51] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [15:01:51] T385264: VM live migration failing for many/most VMs - https://phabricator.wikimedia.org/T385264 [15:14:39] !log toolsbeta hard rebooting all VMs for T385264 [15:14:42] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [15:14:43] T385264: VM live migration failing for many/most VMs - https://phabricator.wikimedia.org/T385264 [15:20:00] andrewbogott, I deleted parsing-qa-02 .. so, the temporary quota bump can be bumped down again. [15:22:38] great! Happen to have the task # handy? [15:50:53] * lucaswerkmeister checks in on bridgebot [15:52:19] !log lucaswerkmeister@tools-bastion-13 tools.bridgebot kubectl rollout restart deployment bridgebot # stopped bridging for no clear reason (kubectl logs show recent telegram/IRC messages), see if this helps [15:52:21] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.bridgebot/SAL [15:53:26] yay, it’s bridging again