[10:53:54] <dhinus>	 paws is down, looking into it
[10:54:39] <dhinus>	 one node is NotReady
[10:56:54] <dhinus>	 rebooted the node, https://hub-paws.wmcloud.org/ is still returning 503 Service Unavailable
[11:00:49] <dhinus>	 the hub pod is failing to start, with "Multi-Attach error for volume "pvc-287ed3a7-d043-4ad0-a233-727038d4b110" Volume is already used by pod(s) hub-5bf7498868-vbzd5"
[11:01:59] <dhinus>	 deleted the pod, now I'm getting "Multi-Attach error for volume "pvc-287ed3a7-d043-4ad0-a233-727038d4b110" Volume is already exclusively attached to one node and can't be attached to another"
[11:08:28] <dhinus>	 I created T413428
[11:08:29] <stashbot>	 T413428: PAWS failing to mount volumes - https://phabricator.wikimedia.org/T413428
[11:51:59] <dhinus>	 fixed, PAWS is back online!
[11:52:18] <dhinus>	 the mount to clouddumps1001 was failing, I had to "systemctl restart nfs-server" in the clouddumps host
[12:34:12] <dhinus>	 re: updatetools error emails, I created T413431
[12:34:13] <stashbot>	 T413431: updatetools is frequently timing out - https://phabricator.wikimedia.org/T413431