Crypto Kitty Data Mining - What does it take?

in #cryptokitties7 years ago

I've been asked a few times what it would take to setup an application like mine. The short answer. A LOT.

Here's what I have done in the last 2 weeks.

Server Infrastructure

  • DB
  • Application
  • Cache
  • Message Broker
  • Cron

Application

  • Crypto Kitty API Parsing
  • Crypto Kitty Core Contract Parsing
  • Integrating Parsers with Message Broker (For durability)
  • Integrating Parsers with Cron (Set to sync every 5 minutes)
  • Build the Front-End
  • Build the DB

Why Do it?

I build large scale applications professionally, I know what it takes to make something like this work. The data is needed to build other Pawesome tools and that's what I really want to do!

How much time have I spent

Building the DNA data-set and interacting with the contract has taken me easily over 20 hours of actual work. The Server infrastructure to read over 500,000 kitty DNAs was extensive. At one point I had over 10 active "Consumer Applications" pulling from different ETH nodes to read the kitty DNA and even then I was only getting around 50-60 DNA responses /per second. The sheer volume of requests required a lot up front infrastructure costs as running servers with this much data was resulting in data corruption when the servers died from maxing out.

What did it make?

It made this! http://dna.cryptokittydata.info/ This is a Kitty DNA Profile tool. You can download your kitty DNA sequences (Dominant and Recessive genes). This is only the first phase of what I plan on building. There are more things coming soon.

What's next

Currently I need to take a few days break, I might need to sure up the site for the China Launch.

I think the next big thing is going to be a Gene Search tool.

How to support my development

What really helps is ETH so I can test new theories about how to make our kitties better!
Please send Kitties as well! 0xcecddbe88359f6ecebe90b42643b002543f27fe9

Sort:  

This is amazing. I’m working on something small https://steemkitties.com in my free time. I’d love your feedback. I feel I need to put in a lot more time to make the process smoother. Enter your cryptokitties address and it will load your kitties for use on Backgrounds. I’m still learning React. :(

Congratulations @geggleto! You have completed some achievement on Steemit and have been rewarded with new badge(s) :

<p dir="auto"><a href="http://steemitboard.com/@geggleto" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link"><img src="https://images.hive.blog/768x0/https://steemitimages.com/70x80/http://steemitboard.com/notifications/votes.png" srcset="https://images.hive.blog/768x0/https://steemitimages.com/70x80/http://steemitboard.com/notifications/votes.png 1x, https://images.hive.blog/1536x0/https://steemitimages.com/70x80/http://steemitboard.com/notifications/votes.png 2x" /> Award for the number of upvotes <p dir="auto">Click on any badge to view your own Board of Honor on SteemitBoard. <p dir="auto">To support your work, I also upvoted your post!<br /> For more information about SteemitBoard, click <a href="https://steemit.com/@steemitboard" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">here <p dir="auto">If you no longer want to receive notifications, reply to this comment with the word <code>STOP <blockquote> <p dir="auto">Upvote this notification to help all Steemit users. Learn why <a href="https://steemit.com/steemitboard/@steemitboard/http-i-cubeupload-com-7ciqeo-png" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">here!