Great work!
Question, syncing from scratch would still pull data from blockchain (hived) right?
Looking forward for more technical notes and source code changes...
You are viewing a single comment's thread from:
Great work!
Question, syncing from scratch would still pull data from blockchain (hived) right?
Looking forward for more technical notes and source code changes...
Yes, it still pulls data from hived, but it is able to pull the data much faster now, and the insert and updating of the database was also made much faster. A bunch of different techniques were used to achieve this speedup: we added an API function to hived that allows for getting a filtered set of virtual operations, we spawn multiple threads in hivemind to fetch the data, we temporarily drop some indexes and re-add them after sync is finished, etc.
Most of the sync speedups were done in this branch of hivemind: https://gitlab.syncad.com/hive/hivemind/-/commits/dk-issue-3-concurrent-block-query
A lot of the API-related speedups were done in this branch of hivemind: https://gitlab.syncad.com/hive/hivemind/-/commits/query-speedup