Storage is still cheap

in #storage23 hours ago

How much space do you think it takes to store Hive? One SSD? Two SSDs? Well, it depends on the type of node you are running. For a seed node, it just takes ~540 GB. That's 1 SSD. A 1 TB Samsung 870 Evo "only" costs $200(and it's one of the better drives), https://www.amazon.com/SAMSUNG-Inch-Internal-MZ-77E1T0B-AM/dp/B08QBJ2YMG.

image.png

While the cost of drives is going up, our efficiency on using the space is getting better. Hive was larger than 550 GB a few years ago. Then compression of the block log got added in and guess what the size to store it went down.

Lets look at the size of storing the stuff necessary for Hive in multiple regions(please note, I run my shared memory in RAM so that's not accounted here, I know there are some users who choose to have that on disk).

My Chicago seed node is only taking up 536 gigs!

Chicago:$ du -h .hived
536G    .hived/blockchain
13M     .hived/logs/p2p
13M     .hived/logs
616K    .hived/p2p
536G    .hived

The one in San Fran is at 537 gigs. This one runs on Hive-In-A-Box by @someguy123 so that's why the output is different.

San Fransisco:$ du -h hive-docker/
84K     hive-docker/scripts
4.0K    hive-docker/.git/branches
68K     hive-docker/.git/hooks
12K     hive-docker/.git/logs/refs/remotes/origin
16K     hive-docker/.git/logs/refs/remotes
8.0K    hive-docker/.git/logs/refs/heads
28K     hive-docker/.git/logs/refs
36K     hive-docker/.git/logs
8.0K    hive-docker/.git/objects/c0
8.0K    hive-docker/.git/objects/2f
8.0K    hive-docker/.git/objects/90
12K     hive-docker/.git/objects/ac
8.0K    hive-docker/.git/objects/9b
8.0K    hive-docker/.git/objects/8b
8.0K    hive-docker/.git/objects/eb
4.0K    hive-docker/.git/objects/info
684K    hive-docker/.git/objects/pack
752K    hive-docker/.git/objects
12K     hive-docker/.git/refs/tags
12K     hive-docker/.git/refs/remotes/origin
16K     hive-docker/.git/refs/remotes
8.0K    hive-docker/.git/refs/heads
40K     hive-docker/.git/refs
8.0K    hive-docker/.git/info
940K    hive-docker/.git
8.0K    hive-docker/dkr
8.0K    hive-docker/.github
8.0K    hive-docker/data/witness_node_data_dir/snapshot
136K    hive-docker/data/witness_node_data_dir/p2p
14M     hive-docker/data/witness_node_data_dir/logs/p2p
14M     hive-docker/data/witness_node_data_dir/logs
537G    hive-docker/data/witness_node_data_dir/blockchain
537G    hive-docker/data/witness_node_data_dir
537G    hive-docker/data
8.0K    hive-docker/dkr_local
537G    hive-docker/

And in New York City, we are at 540 gigs.

NYC: $ du -h .hived
128K    .hived/blockchain/comments-rocksdb-storage
540G    .hived/blockchain
436K    .hived/p2p
3.2M    .hived/logs/p2p
3.2M    .hived/logs
540G    .hived

At the end of the day, storage space for text is not a concern. It compresses well, and there's not much of it.

Sort:  

I'm rather glad giving how prices are getting now that I brought 16TBs of NVMEs for about £667 back in 2023. My drives will probably wear out before HIVE fills them, unless I spin up a API node which was the original plan.

I wish I had more for sure too! But on hard drives! Nearly full on one of my storage servers and wanted to buy more but out of budget for now and my solution was to just slow down on things lol