As one of the newer users around here (26 days according to Steemd), it was interesting to see what unfolded here yesterday. I'm still not sure that I understand exactly what happened, but I have been reading everything factual that I can find about it, and your post and video here were a great help to my understanding, @lukestokes.
What I am curious about is how this decentralised platform had a single point of failure that took down all of the Witness servers at once? I thought the idea of decentralisation is meant to prevent such things from happening?
My thanks to all the Witnesses, and to Steem Inc., for getting the blockchain back up and running. It sure looked like a mammoth task, coordinating everyone to update their servers like that.
Let's think about it this way:
Decentralized means many individual nodes, yes? If all those nodes are running the same software and they all have the same bug, then they will all respond in the same way. This is by design. All the nodes froze because the transaction that was being processed was invalid and should not have been allowed. So it's not a matter of more decentralization would have fixed this or not. The BitShares chain froze about a year ago also. Bitcoin had a chain fork in 2013 which I remember well. These things happen.
Thank you for taking the time to explain that, @lukestokes. I'm coming from a background of having used (as an end-user only) decentralised social networks like Diaspora* and GNUSocial/StatusNet (and now Mastodon), so I compare it to one of the "pods" going down, but not taking the entire network down with it.
But it seems that what you're saying here is that if all pods were running the same software, and there was a bug in that underlying software, then the whole lot could possibly fail, especially on a sync event, am I right? I guess the crypto side of things here (which neither D* nor GNUSocial etc have to worry about) means that all nodes in the Witness network for SteemIt must be running exactly the same software, and exactly the same version, huh?
Yep, that's essentially it.
Ideally we're all following the same protocol, but there could be multiple implementations of it in multiple programming languages. We're no where close that yet at this point though.