23:10 PST We are observing AC failure at the datacenter, which resulted in thermal shutdown of equipment.
According to the DC engineers, HVAC team is on site.
We will post updates as they come in.
00:18 PST So far we have not received any additional information; we are awaiting for a solution.
03:44 PST We are observing temperatures going down. We will be powering on our equipment in stages and we are in touch with the datacenter for more information.
03:57 PST We have received a confirmation from the datacenter that the AC system has been fixed. We are continuing powering up our equipment.
04:13 PST 3 nodes still powered off: v1109, v1119, v1120; we are working on restoring service for clients hosted on these nodes.
04:20 PST Nodes v1119, v1120 are online.
04:25 PST Node v1109 is online; all VMs on this node will start automatically within the next 5 minutes.
04:36 PST All services are fully operational. We are closing this issue at this time and we will continue monitoring the situation.