You are viewing a single comment's thread from:

RE: August Hive Data - Word Count, Effort, Reward, Rambling

in Hive Statisticslast month

The two months of data I've got stored locally so far is about 5.7GB - its an enormous data set. I have to chunk the queries to avoid overloading HiveSQL at the moment, so quarterly would be another chunking exercise ;P

I know there are other solutions to grabbing the data - but at the moment, I really want to focus on my stuff, and not productionising data. If I manage to get my own hive witness node up and running and have the data available locally, then that might be another story all together :)

Looks like the entire history of the chain is something like 5-600GB :D Gonna take a while to sync!

Sort:  

Oh dang haha yeah that totally makes sense! Following your own motivation is definitely better than dealing with that!