Here is what happened:
Massive Thunderstorm Hit.
Solar Battery Kicked in to Provide Whole of House Back Up.
Uninterruptible Power Supply on which the Witness node was on failed.
Power Was lost.
Here's what's fixed:
Swapped node to secondary UPS, tested, all working.
Picked up chain replay and reinstated price feed.
Back up and running now.
Future Enhancements / Redundancy:
A new UPS!
Primary Node and Back up Node to run on different UPS, on different circuits in the house.
An offsite backup:
Pending.
Downtime was around ~9 hours, mostly because I had a concert to go to and the power outage struck as I was leaving home, and I did not have time to move the server rack and play with electricity.
Fully restored now.
That sounds like 'fun'. Hive is resilient enough to handle the odd witness dropping out, so all is fun. Hope your systems are all okay.
!BEER
Yes, the hardware is fine, and thankfully the primary node had a graceful shutdown. Secondary node needs a full replay, so will be a little while before that is back online.
View or trade
BEER.Hey @holoz0r, here is a little bit of
BEERfrom @steevc for you. Enjoy it!Learn how to earn FREE BEER each day by staking your
BEER.