[19:17:30] <wikibugs>	 10Quarry, 10cloud-services-team (Kanban): quarry-nfs-1 went down; quarry is offline - https://phabricator.wikimedia.org/T302154 (10RhinosF1) p:05High→03Unbreak! Crashed again
[19:18:03] <wikibugs>	 10Quarry: Quarry has become inaccessible - https://phabricator.wikimedia.org/T302180 (10Huji)
[19:18:23] <wikibugs>	 10Quarry: Quarry has become inaccessible - https://phabricator.wikimedia.org/T302180 (10Huji) p:05Triage→03Unbreak!
[19:18:28] <wikibugs>	 10Quarry: Quarry has become inaccessible - https://phabricator.wikimedia.org/T302180 (10RhinosF1)
[19:18:32] <wikibugs>	 10Quarry, 10cloud-services-team (Kanban): quarry-nfs-1 went down; quarry is offline - https://phabricator.wikimedia.org/T302154 (10RhinosF1)
[19:19:05] <wikibugs>	 10Quarry: Quarry has become inaccessible ("502 Bad Gateway") - https://phabricator.wikimedia.org/T302180 (10Aklapper)
[19:21:06] <wikibugs>	 10Quarry, 10cloud-services-team (Kanban): quarry-nfs-1 went down; quarry is offline - https://phabricator.wikimedia.org/T302154 (10RhinosF1) For the record: > 19:02:04 <wmcs-alerts> (InstanceDown) firing: Project quarry instance quarry-nfs-1 is down   - https://prometheus-alerts.wmcloud.org > 19:19:04 <wmcs-al...
[19:21:33] <wikibugs>	 10Quarry, 10cloud-services-team (Kanban): quarry-nfs-1 went down; quarry is offline - https://phabricator.wikimedia.org/T302154 (10RhinosF1)
[23:35:40] <wikibugs>	 10Quarry, 10cloud-services-team (Kanban): quarry-nfs-1 went down; quarry is offline - https://phabricator.wikimedia.org/T302154 (10zhuyifei1999) kernel bug. RCU should not stall.
[23:49:13] <wikibugs>	 10Quarry, 10cloud-services-team (Kanban): quarry-nfs-1 went down; quarry is offline - https://phabricator.wikimedia.org/T302154 (10zhuyifei1999) > The VM was fully unresponsible, even the serial console didn't react at all.   Is it possible to send a [[https://en.wikipedia.org/wiki/Magic_SysRq_key|sysrq]]? If...