HIVE isn't as decentralized as I previously thought!

in #witness • 2 months ago (edited)

Update/Edit: There are a few points in this article where I was wrong and Arcange corrected me in the comments. So please see their post. It's not AS bad as I originally stated but still could use a lot of work and more active nodes in my opinion.

If you have been around HIVE for any amount of time over a week than you are probably familiar with seeing comments or posts about somebody and their Witness Node and how they are asking for you to vote for their Witness Node. (Not hating on asking for votes at all)

Each Hive account can vote for 30 Witness Nodes and voting for those Nodes does not cost the account anything. So why not vote for the Nodes (and the people who run them) that you trust?

How to Vote?

The easiest way I have found is to head over to https://peakd.com/witnesses and just go down the list and click on the ones you want to vote for.

image.png

But why vote?

Good Question and the Good Answer is that because the top 20 Node Witnesses are the ones who determine the blocks that get produced. So if you have a couple bad actors in the top spots, well then bad things can happen at HIVE.

Let's ask AI why you should vote:

  • Top 20 Witnesses produce the majority of blocks and decide on upgrades to the Hive blockchain.
  • Backup Witnesses (21+ rank) fill in when needed and can eventually move up with more votes.
  • By voting for smaller, independent witnesses, you help decentralize power and prevent a few entities from controlling the network.

So yeah, if you are reading this then you probably have a HIVE account and you should vote for Witnesses. It's like your right to vote and if you don't vote, then you can't complain when things get approved or pushed through that you don't like.

But why isn't HIVE as decentralized as I thought?

Well I started to comb through the current list of Witness Nodes and something really scary jumped out to me.

There are 500 Witness Nodes listed on that page, and at face value that sounds pretty good. But once you start to actually look at the nodes you will see some glaring problems.

  • 84 Nodes - that have zero 'votes' for them (meaning no HP weight)
  • 76 Nodes - that have less than 1,000 HP Voting.
  • 221 Nodes - that have between 1 million and 1 thousand HP Voting.
  • 119 Nodes - That have over 1 Million in HP Voting

And I know what you are thinking... "That's not too bad! There are a lot of Nodes spread out semi evenly."

And sure that is one way to look at the data... but with every story there are 2 sides of the truth. So let's take another look at the Witness Nodes that run this blockchain we all know and love.

Out of the 500 Witness Nodes... There are 397 that are not online or are just severely out of date.

So we only have 103 Witness Nodes currently functioning and working on our Blockchain!

How I determined those numbers is pretty straight forward. Any Node that is behind on block number by at least 7 days to me is offline. And Honestly I should have made it any Node that is 3 days behind or more should be considered offline/dead (I think there is only 1 that is 6 days out of sync but there were a bunch of 7 days).

So that leaves us with 103 Witness Nodes that actually are in sync and can produce a block (in theory). Those Nodes are as follows (sorted by amount of votes by weight to each node).

RankWitnessNode VersionVotes(MHP)Last Block #Last Block Time
1arcange1.27.687.4m9398469211 minutes
2stoodkev1.27.686.4m9398468612 minutes
3gtg1.27.684.7m9398468312 minutes
4blocktrades1.27.682.6m9398469311 minutes
5yabapmatt1.27.582.2m9398470311 minutes
6ausbitbank1.27.681.8m9398469111 minutes
7good-karma1.27.379.9m9398468212 minutes
8guiltyparties1.27.677.2m9398470011 minutes
9steempeak by peakd1.27.376.6m9398470611 minutes
10ocd-witness by acidyo1.27.675.5m9398469411 minutes
11deathwing1.27.374.2m9398468912 minutes
12emrebeyler1.27.674.1m9398469611 minutes
13threespeak1.27.373.4m9398470711 minutes
14abit1.27.673.1m9398469911 minutes
15pharesim1.27.072.4m9398470411 minutes
16themarkymark1.27.571.6m9398468812 minutes
17therealwolf1.27.371.2m9398470511 minutes
18quochuy1.27.770.9m9398470211 minutes
19smooth.witness1.27.669.1m9398469811 minutes
20roelandp1.27.669.0m9398468512 minutes
21engrave1.27.667.0m9398463114 Minutes
22mahdiyari1.27.665.9m9398466913 Minutes
23howo1.27.665.6m9398451720 Minutes
24anyx1.27.458.6m9398454619 Minutes
25actifit1.27.649.6m9398440026 Minutes
26someguy1231.27.640.9m9398412040 Minutes
27techcoderx1.27.637.3m9398461415 Minutes
28jesta1.27.435.0m9398450321 Minutes
29leofinance1.27.630.7m9398428132 Minutes
30timcliff1.27.025.3m9398445623 Minutes
31liketu1.27.624.0m9398419136 minutes ago
32town-hall1.27.623.8m9398434329 minutes ago
33drakos1.27.723.2m9398459116 minutes ago
34oflyhigh1.27.622.7m9398436628 minutes ago
35brianoflondon1.27.622.4m9398410541 minutes ago
36louis.witness1.27.622.4m9398431630 minutes ago
37pfunk1.27.022.1m939837371 Hour
38pizza.witness1.27.620.5m939835331 Hour
39rishi5561.27.619.8m939837631 Hour
40ura-soul1.27.618.8m939834121 Hour
41hextech1.27.618.2m939837851 Hour
42dalz1.27.618.2m939834451 Hour
43neoxian1.27.617.8m939833801 Hour
44curie1.27.317.7m939839591 Hour
45vsc.network1.27.616.9m9398421135 Minutes
46aliento1.27.616.9m9398469511 minutes
47steemitboard1.27.615.8m939832291 Hour
48disregardfiat1.27.514.9m939840181 Hour
49solominer1.27.614.9m939834981 Hour
50sagarkothari881.27.614.2m939839371 Hour
51detlev.witness1.27.613.8m939837051 Hour
52hivecuba1.27.613.6m9398422635 Minutes
53r0nd0n1.27.613.4m939830342 Hours
54stayoutoftherz1.27.613.2m939836731 Hour
55empo.witness1.27.312.9m939829172 Hours
56ctrpch1.27.611.7m9398405243 Minutes
57reazuliqbal1.27.710.3m939830742 Hours
58c0ff33a1.27.310.1m939834281 Hour
59primersion1.27.69.1m939839911 Hour
60balaz1.27.48.8m9398424634 Minutes
61qurator1.27.68.6m9398417037 Minutes
62fbslo1.27.67.9m939822922 Hours
63patrice1.27.47.2m939817773 Hours
64bdcommunity1.27.47.1m939820342 Hours
65innerhive1.27.67.1m939838431 Hour
66thebbhproject1.27.67.0m939822202 Hours
67liotes1.27.66.5m939822022 Hours
68cadawg1.27.66.5m939827032 Hours
69wrestorgonline1.27.66.2m939830082 Hours
70crowdwitness1.27.65.3m9398403744 Minutes
71shmoogleosukami1.27.65.0m939801214 Hours
72hispapro1.27.65.0m939823532 Hours
73fernandosoder1.27.74.9m939815833 Hours
74hiq.witness1.27.64.8m938065246 Days
75klye1.27.84.6m939798634 Hours
76blue-witness1.27.64.6m939808063 Hours
77discovery-it1.27.64.5m9398448022 Minutes
78gwajnberg1.27.64.4m939806544 Hours
79jedigeiss1.27.44.4m939833581 Hour
80nuthman1.27.64.3m939826372 Hours
81la-colmena1.27.63.9m939825852 Hours
82condeas1.27.63.9m939835771 Hour
83mengao1.27.63.1m939802814 Hours
84synergy.witness1.27.83.1m9398444524 Minutes
85helpie-caster1.27.63.0m939802304 Hours
86atexoras.witness1.27.73.0m939788695 Hours
87keys-defender1.27.32.8m939776136 Hours
88whiterosecoffee1.27.32.8m939762517 Hours
89tazi1.27.62.8m939797634 Hours
90seattlea1.27.72.7m939776746 Hours
91cervisia1.27.62.6m939804364 Hours
92pishio1.27.62.1m939777076 Hours
93vimm.witness1.27.62.0m939751438 Hours
94chisdealhd1.27.62.0m939829982 Hours
95alloyxuast1.27.62.0m939742299 Hours
96perfilbrasil1.27.72.0m939836771 Hour
97earn.hive1.27.61.9m939765577 Hours
98isnochys1.27.61.9m939761107 Hours
99helo1.27.61.6m939773726 Hours
100lapingvino1.27.61.5m939800044 Hours
101mebonline1.27.61.5m939756428 Hours
102skatehive1.27.7905.5k9396444817 Hours
103crypto-shots1.27.4653.7k9397312710 Hours

These numbers were pulled on March 9th, 2025 at 11:30 EST. So they may be out of date by the time you are reading this article.

But why does that matter? There are over 100 Nodes ready to go if they are called up to the big leagues to produce a block right? NOPE!

CLARIFICATION TIME

I am going to be using names of accounts that host HIVE Witness Nodes. If I am speaking poorly about your Node, it is NOT personal at all. I am merely talking about things based on the numbers and facts. I truly appreciate all 103 of these accounts that are running Witness Nodes.

Let's say for example there is a network outage at a Amazon Datacenter or even a CloudFare outage and the top 20 Nodes are taken offline. Well Hive still needs to run so it is going to look to pull up some people from the B league to fill in. If you wanted to buy some tokens or send them or post/comment/edit articles on HIVE, would you want Node #102 by Skatehive to immediately step up and start producing blocks? I wouldn't, because it is 17 hours behind on blocks. That is scary.

The top 20 Nodes are the ones who produce blocks right, with #21 and #22 being on standby. What is more frightening then #102 being way out of sync is a #37 by pfunk being an hour behind. If something happens to the top few and #37 becomes #20 well that guy is going to be behind. You all hate when sites/services are down because of Nodes right? Well this could be 1 reason why those types of things happen!

So for my argument sake let's take a look at all the Nodes that are in sync within an hour or less okay?

That gives us these 45 Witness Nodes:

RankWitnessNode VersionVotes(MHP)Last Block #Last Block Time
1arcange1.27.687.4m9398469211 minutes
2stoodkev1.27.686.4m9398468612 minutes
3gtg1.27.684.7m9398468312 minutes
4blocktrades1.27.682.6m9398469311 minutes
5yabapmatt1.27.582.2m9398470311 minutes
6ausbitbank1.27.681.8m9398469111 minutes
7good-karma1.27.379.9m9398468212 minutes
8guiltyparties1.27.677.2m9398470011 minutes
9steempeak by peakd1.27.376.6m9398470611 minutes
10ocd-witness by acidyo1.27.675.5m9398469411 minutes
11deathwing1.27.374.2m9398468912 minutes
12emrebeyler1.27.674.1m9398469611 minutes
13threespeak1.27.373.4m9398470711 minutes
14abit1.27.673.1m9398469911 minutes
15pharesim1.27.072.4m9398470411 minutes
16themarkymark1.27.571.6m9398468812 minutes
17therealwolf1.27.371.2m9398470511 minutes
18quochuy1.27.770.9m9398470211 minutes
19smooth.witness1.27.669.1m9398469811 minutes
20roelandp1.27.669.0m9398468512 minutes
21engrave1.27.667.0m9398463114 Minutes
22mahdiyari1.27.665.9m9398466913 Minutes
23howo1.27.665.6m9398451720 Minutes
24anyx1.27.458.6m9398454619 Minutes
25actifit1.27.649.6m9398440026 Minutes
26someguy1231.27.640.9m9398412040 Minutes
27techcoderx1.27.637.3m9398461415 Minutes
28jesta1.27.435.0m9398450321 Minutes
29leofinance1.27.630.7m9398428132 Minutes
30timcliff1.27.025.3m9398445623 Minutes
31liketu1.27.624.0m9398419136 minutes
32town-hall1.27.623.8m9398434329 minutes
33drakos1.27.723.2m9398459116 minutes
34oflyhigh1.27.622.7m9398436628 minutes
35brianoflondon1.27.622.4m9398410541 minutes
36louis.witness1.27.622.4m9398431630 minutes
37vsc.network1.27.616.9m9398421135 Minutes
38aliento1.27.616.9m9398469511 Minutes
39hivecuba1.27.613.6m9398422635 Minutes
40ctrpch1.27.611.7m9398405243 Minutes
41balaz1.27.48.8m9398424634 Minutes
42qurator1.27.68.6m9398417037 Minutes
43crowdwitness1.27.65.3m9398403744 Minutes
44discovery-it1.27.64.5m9398448022 Minutes
45synergy.witness1.27.83.1m9398444524 Minutes

Also for simplicity we are not going to get into Node Versions either.

But that is 45 Nodes across the entire globe that can produce blocks fairly quickly if needed. That is scary to me. For as big as Hive the blockchain is and how much it is used across the globe, to only have 45 dependable Nodes is sketchy to me.

(Now this is not to knock Hive Blockchain in any way, it is just that I wish it was more decentralized than it is).

Now let's play a fun game together.

How much HP was voted to Nodes that are either offline or are out of sync for 1 hour or more?
So basically all the nodes except the 45 listed directly above?

The Answer

535038260
That is 535,038,260 worth of Hive Power voted to Nodes no longer working optimally.
Over half a million Hive Power guys....

Yeah it's scary lol

How did this happen?

I have a theory that people do not regularly check for which Nodes they are voting/voted for. So someone from 4 years ago probably voted for Witness Nodes and might have not checked it since and all the Nodes they voted for are now offline. That could very well be it.

But also I feel like when you create your first Hive account (at least for me) there is no real good onboarding or detailed information that you are pointed to.

Tell me where I can go (1 URL webpage) that has ALL of the following:

  • List out most/all of the various sites and services on Hive.
  • Explanations about Hive, Hive Power and HBD.
  • Explanation about Curation and posting payouts.
  • Explanation about Nodes and Witnesses (voting and creating one).
  • Explanation about Voting Power, Resource Credits, Weight of votes, etc.

I am willingly to bet a lot of HIVE users do not even know about voting for Witness Nodes and why they should AND that its completely free for them.

So I can't really blame people if they just never knew about Witness voting or if they just forgot about it because it's not really talked about (at least not in the circles that I follow).

What to do now? Your Call to Action!

This is your reminder that if you have a Hive account with ANY amount of Hive Power then you should want to vote for Witnesses and you should vote for witnesses.

This is not something you should have to do everyday or every week even. But every month or so go to: https://peakd.com/witnesses (or which ever site you are using) and check to see if the nodes you previously voted for are still up and running optimally, and if not remove your vote and vote for other ones.

#bbh #node #hive
2 months ago in #witness by
$2.77
  • Past Payouts $2.77
  • - Author $1.39
  • - Curators $1.38
62 votes
Reply 11
Sort:  
  • Trending
    • [-]
     2 months ago  

    Let's say for example there is a network outage at a Amazon Datacenter or even a CloudFare outage and the top 20 Nodes are taken offline. Well Hive still needs to run so it is going to look to pull up some people from the B league to fill in. If you wanted to buy some tokens or send them or post/comment/edit articles on HIVE, would you want Node #102 by Skatehive to immediately step up and start producing blocks? I wouldn't, because it is 17 hours behind on blocks. That is scary.

    Lots of misunderstandings about how block production works here. Let's tackle them.

    First, not all of the top 20 witnesses use Amazon Datacenter to host their nodes or rely on CloudFare services. Then, when you are a top witness, you often have a backup node that should be in a different physical location and ideally with a different internet provider if your witness infrastructure is properly designed. At least that's what I expect from them.

    But let's imagine that, like in your scenario, the worst happens, and the entire top 20 is out of service.

    First of all, the fact that the skatehive node #102 has not produced a block for 17 hours does not mean that it is 17 hours behind on the blockchain!
    All nodes that are running (properly) are constantly synchronized with the blockchain. They are simply waiting to be assigned to produce a block. This happens every minute for the top 20, and for backup witnesses, the higher you are in the ranking, the more regularly your node is called upon to produce a block. There is no delay in the time it takes for a backup witness to produce a block, regardless of its rank and regardless of how long it has been since it produced its last block.

    You can look at the block production process in real time on the page I created for: https://hive.arcange.eu/schedule

    So back to your catastrophic situation, what would happen?
    Every 63 seconds, a new round of block producers is scheduled, with the top 20 witnesses + 1 backup witness (the top 1 on the right pane list on the above-mentioned page). As all the top 20 nodes are down, only the backup witness would produce a block.

    The main effect would therefore be the slowing down of the blockchain to ~1 block per minute rather than 1 block every 3 seconds.
    Another effect would be transactions being dropped because there is no more room for them in the only block that is produced before they expire.

    To sum up, I don't find the state of the witnesses ranking and their node state scary at all and I don't think that block production isn't decentralized.
    On the other hand, I agree with you that participation in (good) governance is important and that we should all regularly review our witness votes. But I'm not going to go over again the reasons why this isn't always the case. You already did that in your post.

    [-]
     2 months ago  

    1st of all, thank you for kindly pointing out how my understanding was incorrect and thank you for explaining it!

    2nd of all, your site is way more detailed and easy to read then others. It is already bookmarked :)

    (also my analogy with AWS and CloudFare was too extreme but just wanted to come up with a worst case scenario type situation, but totally agree not everyone would be using those services if any, give IOPS and such for AWS)

    First of all, the fact that the skatehive node #102 has not produced a block for 17 hours does not mean that it is 17 hours behind on the blockchain!

    This was not how I interpreted the data I was seeing but now that you point this out I can see what you are saying and that makes more sense and is more comforting then I originally thought.

    So yes it's not as bad as I originally thought, however I guess my own assumptions at how big HIVE was and there by should have "thousands" of Nodes was ignorant. I do wish there was way more active Nodes then there are. Also would love to see the data on where Nodes are located even if it's just 'this many in this country, and that many in that country'. Time to google some more :)

    Thank you Arcange!

    $0.00
      Reply
      [-]
       2 months ago  


      Your comment is upvoted by @topcomment

      Info - Support - Discord

      image.png
      Curated by friendlymoose

      $0.00
        Reply
        [-]
         2 months ago  

        Hive consensus is OK, it would be much better if it followed BFT or GRANDPA principals

        $0.00
          Reply
          [-]
           2 months ago  

          Tbh I wasn't even aware of this or the importance of voting, thanks for explaining and bringing that up! 👌

          [-]
           2 months ago  

          I mean honestly... I wasn't either until I started to research what it would take to make my own node then was scared lol

          [-]
           2 months ago  

          From what I read in a post from @seattlea it takes 80Gb to run the newer version, double for the older, would you earn for running one?

          $0.00
            Reply
            [-]
             2 months ago  

            doing my research and such (again I am still new to this) its takes way more than 80GB. But who knows maybe there is an easier way then I found.

            The Blockchain file (basically a backup of past blocks is 497 GB). Then you gotta resync it all and verify blocks which takes HOURS...

            image.png

            More articles to come 😉

            $0.00
              Reply
              [-]
               2 months ago  

              Hmm maybe I just remembered the wrong number 🤔🤣

              $0.00
                Reply
                [-]
                 2 months ago  

                oh and the top 20 Nodes earn decent amount and it QUICKLY tanks from there so in theory I could earn a little bit but without a lot of votes to get up in the ranks not likely

                $0.00
                  Reply
                  [-]
                   2 months ago  

                  Congratulations @cherokee4life! You have completed the following achievement on the Hive blockchain And have been rewarded with New badge(s)

                  You distributed more than 1250 upvotes.
                  Your next target is to reach 1500 upvotes.

                  You can view your badges on your board and compare yourself to others in the Ranking
                  If you no longer want to receive notifications, reply to this comment with the word STOP

                  $0.00
                    Reply