Recently @blocktrades revealed the specifications of a computer capable of running an Hive API node which can handle ALL of Hive's current transactions.
This is the result of dramatic improvements in the efficiency of the core blockchain code (hived & hivemind).
I was astounded at how moderate the requirements were:
- CPU 4/8 cores/threads 4.0Ghz
- 64 Gb RAM
- 2 x 2 TB NVME SSD
I confirmed with him that similarly specced PCs such as my current witness node could also run ALL of Hive's current traffic.
I've been saying in various comments that one can purchase a PC meeting or exceeding those specs for around $1000 and thought I should actually price things out.
It turns out that the basic components for a Hive API node cost less than $750!
You'll need to add a power supply and case (around $50) if you don't have them already from an older PC.
You can run the Hive API node headless and SSH in from another machine or use another machine's monitor and keyboard / mouse to get things running.
So for as little as $750 outlay anyone with a bit of technical experience can run a Hive API node.
This surely makes Hive by far the lowest-cost content-storing blockchain to run.
This means Hive can offer the lowest cost transactions to DApp developers sick of Ethereum's crazy gas fees.
It means Hive can scale and become much more decentralised, at very little cost.
I call on all witnesses who have decent internet connections to consider running an API node.
- The more API nodes we have, the more resilient we are to attacks by enemies of free speech.
- The more API nodes we have the more refugees from The Purge we can onboard.
I also call on the amazing developers of Hive who have improved efficiency by orders of magnitude to properly and simply document the process of setting up an API node so that it is easy for people with average technical skills to do.
Hive is the Bitcoin of content, social media and DApps.
It can be as resiliant and decentralised as Bitcoin too.
Please vote for my Hive witness. (KeyChain or HiveSigner)
Hardware isn’t as much of a concern as Internet reliability is far more important. Home Internet is not adequate to provide a reliable witness or full node.
This is a gross generalisation and based on my experience is simply not true.
I have had absolutely no internet problems with my witness and plenty of other people are running reliable witnesses on home internet, even on 4G mobile.
Of course there are HUGE differences in quality of home internet and some home internet may be too unreliable.
Reading your comments below the main issue you seem to have is power outages (2-6 times every winter). That is terrible power reliability and is not typical.
External power outages where I live are so rare I'd say once in 3 years is typical.
My new backup witness will be a laptop with the ability to change internet connection to 4G, neighbour wifi or even city wifi in the event of a power outage.
Secondly, the new snapshot facility means you do not need to replay a node for every outage. I can generally have my witness up and running within 20 minutes of the machine going down (and I have auto-switching to a backup).
Thirdly, the whole point about decentralisation is that having lots of people run nodes means that individual nodes DO NOT NEED the ultra high uptime that data centers provide. If one API node is down front ends auto switch to others.
For an API node, I agree, for a witness node I disagree. Every time a witness misses a block, it has to be passed on to the next witness and puts the chain at risk.
Glad you agree on API nodes because that is actually what my post was about. :-)
Hive has enough witnesses (although more is always better) but does not have enough API nodes because people think they are very expensive to run.
The specs required to run a witness node are so low these days that you can run a backup witness on a 9 year old laptop (admittedly with 8Gb RAM and 500Gb NVME SSD and i7 4 core - my old Macbook Pro was a beast in its day).
A laptop is the poor man's uninteruptible power supply and 4/5G or alternate wifi (neighbours) can provide internet redundancy.
That's my bad, I know you run your witness on a workstation and assumed it was the same thing.
I disagree, we have far fewer witnesses than we did on Steem and very few who really know anything more than getting the witness up and running, which is just following some instructions and typing in a few commands.
Sure if you don't mind missing blocks, oh and I know the answer is going to be "But I haven't", performance of a top 100 witness is vastly difference than the requirements of a top 20, the higher you get the more demanding it is.
I agree that the level of knowledge about witnessing and running nodes needs to improve and broaden.
This is precisely why I am both calling for far better documentation by Hive devs and encouraging people to run nodes on owned equipment, physically located with them.
If you have a witness running beside you at your desk and you can see the logs running all the time (and immediately when they go down) you are much more connected to being a witness than having it run on some "cloud" server somewhere.
You are more likely to learn how it works. That is certainly what I am trying to do.
According to @someguy123 you can even run a witness on a 4Gb RAM machine.
8Gb RAM with a 4/8 core CPU should be plenty sufficient for a backup witness.
My main witness will have 24Gb & 6 core CPU and my API node 128Gb and 6/12 core CPU & 2x1Tb RAID0 NVME.
Why do you suggest that I would be missing blocks with this setup if I made it to a top20 witness slot? I can always add a second backup witness node to provide further redundancy if necessary. I've got heaps of PC parts lying around. :-)
I admire your persistence and patience (I can't even bear to continue reading more of that "wet blanket power" - don't have the patience or the time). You're definitely doing the right thing and thank you so much for your efforts. My "reality check" response fell on deaf ears, but was nonetheless very revealing and worth taking note of. In my opinion, what you are proposing is a key step forward that we must take, it's in our own best interests, and we must press forward in spite of whatever resistance that may arise, no matter how great (or small) it may be. A big thumb's up!
I disagree, I can see logs from anywhere in the world at similar efficiency. While I am not against having your own equipment what so ever, I do have concerns running critical servers on a home network. Granted not everyone has to deal with snow I won't run even simple services like Poshbot on my home network as I don't trust a home connection being available. This is just my own opinion.
I see no problem with that hardware, but you did mention a 9 year old laptop. The thing is, running a witness is a lot different than replaying one. While we do have the ability to do snapshots now, it can still take hours to transfer block logs and snapshots even with 1 Gbit Internet connection. The real problem comes during hard forks and emergency patches, you won't have a snapshot to rely on, and will need to do a full sync, witness nodes and API nodes.
Most witnesses are no where to be seen when critical shit happens, the witnesses that are and have the hardware, Internet, and knowledge to support agile recovery are who I want protecting the Hive blockchain. Unfortunately, this list is rather tiny and doesn't reflect in the witness rankings.
Posted Using LeoFinance Beta
Troll Alert: Warning, the following is strictly trolling now:
No! Your bad was your totally flippant and outright dismissing of the entire post!
Remember?
And then you continue by ending your comment with yet more negativity totally absent of possible solutions . . .
How apropos!
/end trolling
You are not even a good troll at that.
And that's what I think is THE point this person wants to avoid, or doesn't realize exists, and I'm not sure which is worse (even though lower he does talk about it with you).
Look at what it took me just to get him to talk about the number of witnesses! Amazing! I'm flabbergasted. And he is a top 3 witness.
(Well, maybe all that is not so surprising . . . depending on how suspecting of a mind one may have.)
Glad to see you got him focused. He finished with my by insulting me.
I totally disagree, unless you are referring to a "standard" home setup.
You can run what you want out of your home. What do you f@cking want from home? What do you want, a T1? lol
I know you get what I mean. You want a fixed IP, you contract it, you know what I mean? You can contract the bandwidth and redundancy you like, as long as you don't live in the middle of nowhere.
Or was that what you were referring to?
A T1 is slow and super expensive compared to standard home Internet. More importantly though is home Internet is not very reliable and any hiccup will cause a node to have to replay for a few days.
I cannot believe you're spouting this crap. Do you even know what a T1 is? Don't you get a joke? And then you come back and treat that like it's something serious to be debated! Obfuscation? Sorry that my joke lent itself to that - I should have known better. (But you could even get the job done with a T1, as long as your not living at the last mile, anyway, but let's forget about satire and try not to get distracted on the irrelevant and impertinent).
So anyhow, how about fibre optics? Have you heard of that where you live? What kind of "hiccups" are you referring to? The town where I come from gives me the same reliability that any data center could have, or are you saying that second line redundancy is actually necessary? Seriously? (My redundancy statement above was also a joke on that level.)
The idea is that you can do what you damned well please from home, all you have to do is contract it. Again, unless you live in the middle of nowhere . . .
My God, I'm appalled at your remarks, your negativism, and total lack of cooperative attitude. Simply stunned! Instead of snide, discouraging remarks, how about something constructive, like, for example, some specific specifications you might recommend? How about sharing the info on what you have? Or are you on a rented, centralized, out-of-your-control shared data server?
Marky smart smart smart smart smart.
Rent a server is cheaper anyway. Energy, computer power + connection are cheaper than ever :)
Posted Using LeoFinance Beta
Also much more centralized and haven't we learned what happens when you host on CENTRALIZED servers!?! Nothing good..
Nodes are generally not run on VPS units but dedicated servers, so they can't be centralized. Data centers can be though.
makes no sense. Sorry.
BLURT can run on much a less, in fact a Raspberry PI! A lot cheaper, and a lot smaller footprint!
With thousands of Raspberry Pi node users around the world, that would be a better idea for a more decentralized social platform. Can Hive upgrade to do this too I wonder?
No it can't. Not since it was spammed and is now over that threshold.
(There is a post about this by the Blurt dev, they are building a new chain now)
They also started the chain from scratch, if Hive didn't bring over state, it would be able to as well.
Posted Using LeoFinance Beta
Raspberry Pi's IO poor, especially to SD card.
Did Blurt ditch the Steem blockchain history?
The Hive blockchain is 330Gb - not something that a rPI can handle.
If Blurt is successful it will end up with a very large chain too.
What code changes did Blurt make to be able to run a node on an rPI?
Raspberry Pi's are atrocious at SD IO, the Pi 4 made a big improvement, but it is still bad, especially if you don't use a Samsung EVO+ (the best performing SD card by a long shot on a Raspberry Pi).
But anything serious I'd highly recommend just using an SSD drive, you can get one for about $30 and be 100x faster.
Some tests I did. Using something other than a Samsung EVO+ would have been much worse.
Maximum performance out of the Raspberry Pi 4 with an SSD drive
I'm planning to run the jussi for my API node on a Raspberry Pi 4 with 4Gb RAM and SD card.
Is this sufficient in your view or should I add a SSD on USB?
I have both the cable and SSD drive but understood jussi just used RAM for cache and didn't really do IO.
It will mostly be cpu/ram so I don't see a huge difference /w SD vs SSD outside of management time (and reliability on power outages as SD corrupt easier).
You can get a 250G SSD and cable for around $40.
That does sound VERY reasonable, but it also makes me wish we could spread out witness rewards a bit more widely to say 21-50, maybe even beyond - once you get passed 20 you hardly earn anything as I understand it!
Posted Using LeoFinance Beta
I agree. I think there should be a 22nd backup witness position that only backup witnesses that run API nodes are eligible for.
I don't know much about nodes/ API nodes or the difference but assuming it's a lot more expensive to run an API one, I think some kind of encouragement for witnesses lower down the order would certainly be good.
Posted Using LeoFinance Beta
Very interesting. What kind of traffic does the box see on a daily basis? I have been kicking around the idea of setting up a node, but I am not sure I would be able to handle the traffic. I think my connection is more than adequate, but I have a feeling my ISP might take offense to it.
Posted Using LeoFinance Beta
I don't have figures on how much internet traffic an API node takes as it depends on how popular it is, but I know that @techcoderx is running one on a fairly basic internet connection.
I wouldn't think it would use that much bandwidth at all because it is all text.
Images are handled by separate image servers and video is completely separate.
API calls are very small and the returned data is generally pretty small too (except for full account history). Even then it is just text.
A witness node is around 250GB-300GB/month, an API node much more (can be in TB if it is popular). Many home Internet connections have data caps and companies like Comcast are enforcing them pretty hard.
Posted Using LeoFinance Beta
Thanks for that useful data.
I had struggled to find it.
There is no capping on my fibre internet service and even monthly caps on 4G/5G mobile plans here are in the 250-500 GB range.
I'm a bit surprised how high you say the witness monthly bandwidth usage is.
I just checked both of my witnesses and one is averaging around 10% of "1 Mbps" (Windows) while the other is averaging around 40 "KiB/s" (Ubuntu).
Thus the Windows Witness is reporting 0.0125 Megabytes per second and the Ubuntu Witness is reporting 0.041 Megabytes per second.
Multiplying by 60 x 60 x 24 x 30 = 2,592,000 to get month usage this is 32,400 Megabytes per month for one and 106,272 Megabytes per month for the other.
I'm not sure why there is such a discrepancy but both measures are well below the 250Gb to 300Gb per month estimate you gave.
This is obviously excluding re-downloading the whole block_log which you should only have to do once.
I second that!
(Instead of happy-go-lucky, piss-on-your-parade comments.)
They don't program anymore on Hive.
There are more blockchain and app devs on Hive than Steem and Blurt combined.
Posted Using LeoFinance Beta
Are you running yours from home?
I was semi-contemplating running a node on my tiny little desktop "server" (it's actually a NUC) with more than enough specs to run my low traffic webservers and possibly the Minecraft server if I could be bothered moving that but looks like it's nowhere near enough to manage a node.
I am running two Hive witness nodes which is soon to become an API node and two Hive witness nodes. All from home.
For the API node I'll run jussi (reverse proxy server) on a Raspberry Pi 4 running on a different internet connection.
If your NUC has at least 8Gb RAM and at least 500 GB SSD and a 2/4 core fairly recent CPU you can manage a witness node though 24+Gb RAM and a better CPU with decent single core performance are recommended if you want to be able to replay.
How am I supposed to convince my gf I need a Threadripper now... lol
By telling her you can make money as a witness and buy her nice things with it. :-)
I am still thinking about running my own API node. This post may have encouraged me to actually do it after all :D The current constraint for me right now is still time. Once I find enough of that I will definitely do it.
Anyways, great to see that running an API node has such low requirements. I am looking forward to seeing a post about the actual performance of the node.
!gif hive
Via Tenor
I think all under 200$ a month for rent + host is super cheap API Node.
Longterm I think hive doesn't need to become cheaper, it needs more onchain applications. Like Smts and smart contracts. And if things slowly become more expensive, there becomes a need to get cheaper again. With the Top 20 Witness setup, we don't need a Home computer set up to Host Hive. That's why they earn something from inflation.
Compare to the price from one Asic for POW, hive is super cheap to run.
Posted Using LeoFinance Beta
already voted you as a witness, and not planning on removing any time soon.
Thanks.
!BEER
Just for the fun of it
I plan to see the possibility
to run a node one day.
Not too bad it show how decentralized
the system will be.
Posted Using LeoFinance Beta
Sorry, out of BEER, please retry later...
Congratulations @apshamilton! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s) :
You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOPoh, ok, go on then -...I'll vote for your witness thingy! lol
Thanks.
Good info but what incentive is there to run an API node - will I be rewarded with Hive?
Not directly but you are more likely to get votes for your witness if you do.
More witness votes gets you more Hive Power as Producer Rewards.
Also, if you run a DApp you are guaranteed better performance if you run your own API node.