Ever since Bitcoin was released people have been experimenting with more energy efficient alternatives to the costly Proof of Work algorithm. All new algorithms must be compared against the “proven” model used by Bitcoin to see if they offer the same level of security that gives people confidence in Bitcoin. Security is a multidimensional concept and something that is difficult to measure. The true test of security is whether it successfully defends some other goal from attacks; therefore, it is difficult to measure security if we don’t clearly define the goal we are attempting to secure.
The ultimate goal of these algorithms is to reach an incorruptible universal consensus as quickly as possible. Account balances are considered secure once they are included in an incorruptible consensus. Another goal is to prevent censorship of transactions so that all users have the ability to use their balances. A final goal is to make it difficult to change the rules in any way that would hurt a minority holder.
We would consider anything that impacts those goals as an attack that should be mitigated by the blockchain and network protocols.
Security is relative to the Attack
Security is subjective and ultimately inseparable from the kinds of attacks it is designed to prevent. A mile high wall may keep out advancing armies, but is worthless against an air force. It is impossible to talk about security without addressing a particular kind of attack.
Robust security has defenses against as many different attack vectors as possible. In order to compare two different blockchain consensus algorithms you must compare how they each perform under a variety of different attack scenarios and then weight the results according to each individual's subjective opinion of which attacks are most likely and most harmful.
Decentralization
Decentralization is a buzzword that many have come to believe is the cure for everything. This belief is based upon the fact that decentralization does solve many problems, but like most things there exists a point of diminishing or negative returns. Each additional bit of decentralization adds a fixed cost, but produces less value than the previous bit. At some point the cost of an additional bit of decentralization exceeds the value it provides. This is ultimately a subjective value judgement as there is no objective measure of economic value.
To maximize decentralization it is critical to minimize the cost of each additional bit of decentralization. This will maximize the amount of decentralization that can be achieved before the cost of additional decentralization exceeds the value of additional decentralization.
What we can conclude from this is that the biggest gains from decentralization occur when you go from 1 person to 2 people. This is a 100% increase. Going from 2 people to 3 people is only a 50% increase. Going from 100 people to 101 people is a mere 1% increase.
Decentralization is a means toward achieving robust incorruptible consensus, censorship resistance, and difficulty in changing rules without general acceptance. Decentralization should not be viewed as an end in and of itself.
Why use Blockchains?
Everything that can be built on a blockchain can also be built as a traditional website, and every website can also be built as a blockchain. Both websites and blockchains take authenticated user actions and use them to update a database. The primary difference is that traditional websites do one-time server-side authentication that leaves no audit trail, while blockchains log self-authenticating actions that can be verified by anyone at anytime and cannot be modified without changing the entire dataset.
The decision to use a blockchain has high costs compared to a traditional LAMP (Linux, Apache, MySQL, PHP/Perl/Python) stack. Most of these costs are due to the immature infrastructure available for building and deploying new blockchain applications. Blockchains must offer some compelling advantages to justify the time and money required to deploy an application as a blockchain.
Decentralized Authentication
This is the most important benefit of a blockchain and is based entirely on the security gained by decentralizing the storage of user passwords and employing 256 passwords (private keys). Under this model there is no single server that can be hacked to gain access to everyone’s account. No account can be modified without being able to produce a verifiable audit trail of self-authenticating user actions that prove the validity of the account state.
This decentralization of authentication means the system as a whole has better security, but transfers most of the responsibility for security from the service provider on to the users. Individual users are less equipped to secure their own account than trained system administrators.
In aggregate decentralized authentication secures everyone better because all attacks are compartmentalized to individual users. Individual users now suffer from lost keys, irrecoverable accounts, and hacks on their own computers. If all user keys were secured to the same extent that a properly managed server would be, then the total cost of security would be much larger with decentralized authentication than centralized counterparts.
This kind of security is a good tradeoff when your platform as a whole is a target. This is mostly the case in platforms serving people that suffer from political persecution, such as alternative currency users or free speech. That said, with multi-signature transactions and other protocols it is possible to user security on the blockchain as easy as server-side security is today.
Incorruptible Audit Trail
The incorruptible audit trail means that anyone/everyone can mathematically prove that nothing has been tampered with and that the database is in the correct state. This means no server administrator can manipulate your account and bypass authentication. This means no one can impersonate you by simply modifying the database.
In order to have an incorruptible audit trail you need to have timestamped public record of the block headers. The more individuals that have a copy of these signed / timestamped headers the harder it becomes to forge an alternative ledger without getting caught. In principle, people do not even need to know the contents of the blocks in order for a company to prove to auditors that the block history has not been modified.
Any company could log all corporate actions in a private blockchain and to publish a hash of those actions that is signed, logged, and recorded by many different people and organizations. Even if the blockchain contents were completely centralized and only accessible by company employees, this system would be effective in preventing the company from cooking the books after-the-fact.
Replicated Database
The last aspect of blockchains is that they are heavily replicated with copies distributed around the globe. This replication protects them against localized natural and manmade disasters. The structure of a blockchain, an append-only log, is trivial to replicate reliably and directly contributes to preventing forgeries and alternative histories.
The level of replication required is mostly defined by the available political zones and the extent to which governments are willing to go in pursuit of control. Having a full replica in every country in the world is more than sufficient to ensure that no single country or coalition can take down every copy.
Those with a copy of the database do not need the ability to write to the database to serve the goal of protecting an incorruptible universal consensus. Each and every copy contains information within necessary to prove its accuracy, authenticity, and authority relative to all other copies. Massive replication is designed to protect integrity and prevent censorship of past transactions. It is not sufficient to prevent censorship of future transactions.
The Ultimate Proof of Consensus
The ultimate proof of consensus is for all parties to a system/network/blockchain sign every transaction to confirm that they acknowledge and accept its impact on the current state. In the case of bitcoin, it would be like having all other bitcoin users sign your transaction to confirm it.
The more parties involved the harder it becomes to rewrite history because you need exponentially increasing levels of unanimous agreement to produce an alternative history. This would involve convincing everyone to change history without having any pre-accepted rules for defining what the alternative history should be. Usually changes to history benefit a few at the expense of the many. This means that once a large enough number of people have committed to one version of history, it is next to impossible to get them to recant because they have no incentive to and every incentive not to.
It is clearly impractical to have millions of people directly confirm every action of every other user in real time; however, it is trivial for every user action to confirm all prior actions of other users by including the head block ID within the transaction. Over time all users eventually act and therefore form a consensus by direct confirmation all users.
On the Steem blockchain, 54% of all stake is active on a daily basis. 60% of all stake is active on a weekly basis. These numbers are stake-weighted and exclude activity of the founder’s account, @steemit, which controls about 45% of the network. When counting the @steemit account the weekly confirming stake up to 78% or more. No amount of proof-of-work can offer greater proof of consensus than 54% direct confirmation of stakeholders.
Bitcoin Case Study
Imagine for a moment that Bitcoin transactions were locked into a particular fork and could not be migrated. Under this situation active stake is similar to the “average Bitcoin days destroyed” metric. In the case of Bitcoin, there is an average of 100,000 unique BTC moving each day. This means the cost of forging the Bitcoin network is not the $1.5 million dollars worth of hash power, but the $77 million dollars worth of unique balances that move each day. Over the course of a week this could be $500 million dollars of balances. In percentage terms, the Bitcoin network would only be directly confirmed by less than 0.7% per day and less than 5% per week.
Bitcoin’s rate of confirmation would accelerate if people used Bitcoin as frequently as they use their credit card (daily). Unfortunately, it doesn’t look like Bitcoin will scale in that direction.
Stake Weighted vs Popular Vote
Stake-weighted activity is an objective measure that is protected against sybil attacks, but can be heavily biased toward the opinion of a few large holders. This is where account reputation and identity offer an alternative subjective metric: popular vote. Under this metric you weigh all accounts equally regardless of their stake. This process can be somewhat subjective because larger holders can fake it by dividing their funds up among multiple accounts. The purpose of this metric is simply to show that “the masses” can easily identify a collusive group and act accordingly. The longer the window of activity the greater the disparity between accounts used in a collusive group and the real blockchain.
Consensus by Web of Trust
When all accounts confirm the blockchain by their own actions it becomes possible to verify the blockchain by Web of Trust. If you trust a couple major exchanges and you know they transact regularly, then you can ignore any blockchain that doesn’t contain confirmations by those accounts. The exchanges don’t have to produce the blocks themselves, they simply have to transact like normal.
Each individual would have a different perspective, but with enough overlap, everyone would ultimately trust the same blockchain. Statistically, there is an average of 6 degrees of separation between any two people on the planet. This means that if everyone only trusts those they know, then through less than 6 trust links you can trust everyone on the planet.
When it comes to blockchains, the only information we are trusting people to report is “the current chain”. There is an implicit agreement to agree. Anyone who attempts to report a “fake” chain would not have their transaction propagated nor included in a block. In effect, all accounts that desire to transact can be trusted to report an honest opinion on the current chain.
Short Term Consensus vs Long Term Consensus
Hopefully by this point you are convinced that over time enough stake will directly confirm the blockchain and enough trusted accounts will have transacted on the chain that there is authoritative proof of supermajority consensus. How long this unambiguously direct confirmation takes is based upon the average activity of stakeholders. If the average user acts once per day, then it will take 24 hours on average to achieve 50% direct consensus. If they only act once per month then it may take 30 days.
It is obviously desirable to have transactions clear with near 99.99% certainty in a much shorter time. There is no need to collect votes from 50% of the holders to be 99.99% certain that you will eventually collect those votes. Even a small sampling of activity from your web of trust can yield a high degree of confidence.
The vast majority of users don’t directly confirm the state, but do so by reference to trusted off-chain communication. This means that a quick sampling of your trusted peers is sufficient to know which chain you are on. If an attacker isolates you in an effort to present a fake chain, then your transactions would not be valid on the real chain. This means that receiving transactions should be subject to validation by trusted peers. An attacker would then have to deceive all of your trusted peer network in order to attempt to present a forged chain.
Trusted Peers vs. Trusted Economics
Bitcoin operates on the assumption that there is no such thing as a trusted peer. Instead each node listens to all peers and draws a conclusion on which chain has consensus based only on the math involved. The only thing the math can “prove” is that someone spent a lot of money on electricity to solve a complex mathematical problem. This is proof of work. Stated another way, without reference to outside information a node can estimate the market cost associated with producing a block. A blockchain with a high market cost is most likely to be the one with greatest network effect.
Any attempt to commit fraud by producing a bad block must produce more profit than the block reward itself. In the case of Bitcoin, attempting to double-spend with 1 confirmation must net the attacker $10K or more.
On the surface this looks like most transactions would yield so little profit from double-spend attempts that it isn’t worth intentionally forging a blockchain. Rational economic actors are aligned such that waiting for 6 or more confirmations gives on mathematical certainty in irreversibility.
This trust in the economics has one giant hole. Anyone who stands to profit by censoring transactions is only losing the transaction fee. It also assumes that all market actors have equal economic incentives, namely selling the coins they produce at a profit. Some market actors realize profits from other sources and socialize their costs. Governments can afford to mine at a loss, while free market miners must make a profit. In fact, the taxes governments could extract from people profiting on Bitcoin can probably pay for enough hash power to censor the network.
When you rely on trusted peers it is much harder for governments to gain the upper hand. The entire free market is nothing more than peer to peer exchanges of goods, services, and information. If you cannot establish secure exchange of information, then you are not operating in a world where it is possible to exchange goods and services. A well connected social network can easily establish secure lines of communication to trusted peers. Anyone out-of-sync with the social network would be easily identified and avoided in business transactions.
Block Production and Censorship
While everyone has the ability to validate blocks, blocks and the transactions contained within, must be produced and processed in a deterministic order. When it comes to producing blocks it is critical that they include all legitimate transactions without applying censorship. The ability of a blockchain to resist censorship is a critical component of achieving our security goals.
There are two kinds of censorship:
- Universal - all transactions are blocked
- Targeted - only specific transactions are blocked
Universal Censorship
Under universal censorship all that is required is to take down the network. Under localized censorship the network continues to operate but only some users are impacted. It should be clear to all that universal censorship is the easiest to implement by attacking the public P2P protocol. Because the P2P protocol allows all nodes to be discovered, it is trivial to generate a list of IPs to block. Once an attack on the P2P protocol was executed, users of the blockchain would be required to create a dark-net of “trusted peers” which completely contradicts the arguments for Proof of Work.
Alternatively, the government could simply shut off power to as many miners as they can, seize as much mining hardware as they can, and then mine empty blocks. They could also flood the blockchain with transactions that crowd out everyone else. All of these attacks are profitable and chump-change for a government willing to spend millions of dollars to blow up a shack in the desert.
Targeted Censorship
Under targeted censorship the attacker must influence the consensus algorithm to allow some transactions through while blocking others. To achieve this the attacker must control the block production either directly or indirectly.
Under proof of work, such as Bitcoin, this means that the government can publicly subsidize block producers who produce blocks according to their censorship guidelines. Once 51% of block producers take the bribe, then the government can update the guidelines to require them to ignore blocks produced that violate the guidelines.
The initial subsidies would have to be quite high, but once 51% is achieved and the non-conforming block producers are pushed out-of-business then the subsidies can be lowered. Anyone who defected after 100% of block producers fell in line would get shunned. It would require the collusion of 51% of the miners in order to force the government to maintain subsidies.
In other words, there is a trap whereby miners have financial incentive to join an attack and once successful they have financial disincentive to break from the attack consensus.
Under Delegated Proof of Stake, such as Steem, this means bribing 75% of the elected witnesses to not only block certain financial transactions, but to also block attempts to vote for different witnesses. To be successful the government would have to approach these witnesses privately and establish a 51% majority in secret and then trust all of them not to defect from the coup.
Once the coup went public the minority witnesses would end up having to choose to join the majority or to start a resistance movement by forking. If they decide to fork, then the block production rate on the government chain would fall to 75% while the minority chain is at 25%. The masses of users would then have to decide whether to join the resistance chain or stay on the government chain. If they switch to the resistance chain then they can remove their votes from the collusive witnesses and elect new witnesses. Soon the resistance chain will be back to 100% participation and will quickly gain ground and overtake the government operating at just 75%.
Not only will the coup fail due to witness voting, but it will fail based upon direct user voting as well. Rather than a slow public hostile takeover of block production (like in Proof of Work), there would have to be a stealth takeover followed by rapid implementation with the obvious tell of rejecting transactions that vote for alternative witnesses.
Unlike Bitcoin, it would be trivial for a new fork to achieve direct consensus of majority stakeholders and within a couple of days the coup would be countered and the community would move on.
Delegated Proof of Stake
The combination of Delegated Proof of Stake (DPOS) and Transactions as Proof of Stake (TaPoS) means that the blockchain can reach rapid delegated consensus in just a couple of seconds and then confirm that delegated consensus with direct consensus within hours or at most days. DPOS and TaPoS leverage web-of-trust relationships to create a robust, self-healing, network that can ultimately prove unanimous consent via direct signature of all users.
Because consensus does not depend upon a capital-intensive process, it is trivial for resistance movements to adapt to attacks on any individual component. Proof of Work systems are ultimately vulnerable for the same reason that eGold and other digital gold-reserve currencies were vulnerable: they depend upon easily targeted physical assets. To be a successful long-term consensus algorithm it must be light, fast, and nearly invisible in order to out maneuver the slow heavy hand of government. Governments rely on brute force, consensus by brute force plays to their strengths and that is exactly what Proof of Work does.
Conclusion
Rather than assuming that Proof of Work is the gold-standard for decentralized consensus, I believe it is time that we revisit the goal (unanimous direct consent of all parties) and resistance to censorship. When measured against the ultimate goal of uncensorable, incorruptible, and provable consensus, it is clear that DPOS + TaPoS are the real “gold standard” by which all other consensus algorithms must be judged.
Helping Minnows Understand the Blockchain since 2016 LOL.
UV and RS so others can find this too! I am learning my way and that is like a Whitepaper in itself, this is a really valuable tool Dan, so TY very much for inVESTing your time in this for us - it helps everyone, as I see it -- not just Steemians.
No matter what -- you guys will always be the ones that built something friendly enough to get a lot of us started on the Blockchain and Crypto systems and you should be proud of that too.
It helps us dip our toes in the Crypto pool, dabble with currencies and the like, online wallets, and it has given a lot of us confidence to look at other things we can do Crypto/Next. You know?
Anyways, thanks. Some of us really appreciate who you guys are and what you do.
Sincerely one of your little Minnows.
Is this your way of saying Steem blockchain will be the gold standard?
Dan,
Thank you for your work to give all of us "little people" a more equal footing in a world of government monstrosities. 😄😇😄😇
I hope Steemit reaches this goal one day.
Hey Tuck, I agree. Good to see you BTW.
Until the power of the devs has a counterbalance we will see unilateral decisions made irrespective of what the popular consensus wants to do. This is a significant shortcoming of graphene, that it doesn't provide a tangible / provable way to poll popular opinion. One system of consensus that begins to do this is Dash, which uses a yes / no vote of the masternodes (> 4000 of them) to "poll" what that group wants to do. The masternodes are trusted entities, a much larger number than DPoS witnesses, but far short of a truly popular consensus where every user's vote directly contributes to a poll result.
If the claims of being the first censorship-resistant social media platform are to be trusted, they must be demonstrated. To do that requires decentralized control. It actually needs to be censorship-free, or at least censorship by community consensus which I don't believe steemit has yet achieved.
All blockchain projects start out as centralized, and until some tipping point is reached only the creators of it truly know what they're trying to achieve, so it's not unreasonable for them to control the direction of the project. What the threshold for that tipping point is depends on the project, but IMO it should be defined in the whitepaper, if not explicitly then at least via the metrics that will be used.
Projects conceived primarily from a profit seeking motivation may never want to relinquish a majority interest in the direction the project heads. That's not to say they ignore their users, as that is a sure way to guarantee failure.
When leadership transitions from the creators into the hands of the community is much more difficult to define for projects whose goals and motivation are more philosophical and broader than successfully earning a profit. Let me be clear - there is nothing wrong with earning a profit, in fact that is indeed a very important factor in measuring a project's success and ability to provide value to its' users.
Devs and investors who give birth to the project need to eat and also deserve compensation for the value they bring through their labors. It isn't necessarily unreasonable for them to want to retain a majority interest in how the project is managed. However, to achieve the lofty aspirations @dantheman has expressed requires control to become decentralized, and having only a few people that control it is a weakness that is easy to attack and counter-productve to those goals.
I have learned the most about the underlying blockchain tech from your posts, which i must say are very well thought out.
This is probably one of those post i would say deserves a payout, even if you are the dev.
I would invest in steem by just knowing your behind the project.
Thanks so much for your time. It is only one of few articles/ post I came across that explains the blockchain and the concepts in a way that personally I can understand. I have bookmarked your post.
Great post dan - I think you should be accepting payouts on things like this, you clearly have deep knowledge and a passion for these things and should be rewarded for your efforts.
I only ever had a problem with steemit staff accepting post rewards for announcements or info that only steemit staff/devs had access to ..
TLDR: You word good, accept payment please :)
/off/
I associate the pic of M.A. with this scene :D I cannot clear it from my mind.
/on/
Don't forget about the witness vote expiration, which is a good addition to the security. A witness cannot abuse a vote, he have got from a user who don't check the witness' works and intentions after the vote, more than 3 months. So active users can take the chain back easier from traitors than if a lot of inactive users' votes would stick to the corrupt chain.
wow, I actually read all that and followed most of it! I got lost a bit during Proof of Work details but then it got exciting with the Targeted Censorship discussion! Dan, you are such a thorough thinker, it's incredible. So, did I get this right that basically Steem blockchain is the real gold standard? awesome.
Under Delegated Proof of Stake, couldn't the attacker purchase a large a amount of stake (51%?) and elect new witnesses?
Yes, but if they start censoring they can be forked out by minority.
Amazingly, I think I actually get it this time. It's brilliant.
I think I understood the idea but not most of the systems. Or maybe I didn't get it:(
For some reason this: " To be a successful long-term consensus algorithm it must be light, fast, and nearly invisible in order to out maneuver the slow heavy hand of government."
This seems like witnessing a chapter of history that is yet to be written, where a certain generation builds a new system to replace the coercive one they lived under, and then the ruling class tries to use force as usual to crack on them, and then the revolutionaries adapt and they can never catch on to them.
That's how it seemed like to me, mainly due to my lacking technical knowledge.
That being said, I did learn something important today, when Dan said that most of think that decentralization is the answer to most problems, I am one of these people.
Although I never had expectations of imminent change, it is making me think.
Great read Dan, history has been written by the victors since the beginning of time, Now the truth will be on the blockchain!
I like that.
Glad TAPOS is making a comeback, I think almost every pos system would benefit greatly at almost no cost by incorporating it. Is this already implemented or on the roadmap?
Has been part of steem from day 1
Makes a little bit more sense now.
There's a lot in this post. A lot of which I've never heard about or think of.
It's good to see all the comments from people try to learn more about crytpocurrencies. It reminds me how few people know about them.
It kind of hard to make the most out of our complex world. Steem has got me into so much introspection. I love it.
Great article, but aren't you ignoring the ultimate tool for a resisting minority with Bitcoin style proof of work, that is an emergency change in the proof of work algorithm?
Any algorithm can be dominated the same way.
But that would happen "instantly" until the 75% of delegates would vote for more corrupted backup witnesses ... At least it is now possible with the power the majority of witnesses have... So SP distribution must first get distributed more equal until the written scenario could be realistic ... no ?
PS and how can you prevent that the majority of stake will not be controlled from a minority in the future? Th current elite group for example... they have infinite money to get the majority of steem stake, bitcoin stake and so on....
Why did you lump reputation and identity with a none stake weighted option. Reputation and identity can also be applied to the stake weighted option. In my opinion both stake weighted or not will be subjective. Calling stake weighted objective and then say it is biased towards a few did not make sense to me. Identity and Reputation can be a factor in both nonetheless, it's how this factor is accounted for. The only bug a coder can not fix in a code is the introduction of the human factor, that changes a lot from idea to application.
I did not understand this. An explanation of “average Bitcoin days destroyed” would be useful (google produced only this article), as well as what you mean by the first sentence. The language is ambiguous. By "locked in" aren't you referring to confirmation of the majority fork? By "migration" do you mean the minority fork?
OK, I'll admit this a nit: DPOS - the "O" = of, and is upper case and it is lower case in TaPoS. Consistency helps clarity.