You are viewing a single comment's thread from:

RE: 11th update of 2022 on BlockTrades work on Hive software

in HiveDevs3 months ago

How did you learn the intricacies of the internal blockchain mechanics such as peer-to-peer layer, write queue, block handling, irreversibility and so on? I imagine this is extremely rare knowledge. And there are various approaches out there that various blockchains are trying that don't seem to result in much scalability.


I've been programming for about 40 years now, and most of that time was spent developing "high performance" code where speed and memory consumption was very important. Over time, that allowed me to develop a lot of techniques for estimating code performance, and also analyze and fixing unexpected performance problems.

This experience often gives me an advantage even against very skilled programmers when it comes to figuring out which algorithm will work best in practice in the real world. Knowing what's likely to work and what's likely to fail, without having to go down the wrong path first, is really beneficial when it comes to coding because it saves a lot of time, and time is always at a premium.

As far as blockchain coding goes, my relative experience is much shorter: I started blockchain coding back in 2013. But the fundamentals of high performance coding (e.g. selection of proper data structures, algorithms for working with parallel threads, efficient memory management) are applicable to most of blockchain-related programming , so the primary new things I had to learn were concepts associated with cryptography. And while the low-level math for this is complex, the higher level conceptual understanding needed to employ cryptographic algorithms as a blockchain programmer isn't that hard to gain.

Long story short: while there are certainly high-profile cases where blockchain projects have failed due to cryptographic mistakes, most of the scaling problems are due to a lack of experience in fundamentals of high performance computing. And while "fundamentals" may imply something that is easy to learn, it's not.

Thanks a lot for taking the time to share. Very interesting. So you have a huge amount of experience with developing high-performance applications which enables you to foresee the implications of design decisions in advance, so you don't get stuck in a local maximum. And you see the scaling problems in the blockchain world as mostly stemming from a lack of experience in high-performance computing?

I am reading the explanations in your blog posts with great interest. From this current one, it sounded like you pinpointed a few areas where computation on the same thing was done more than once, or unnecessarily expensive computation (e.g. accessing chainbase) was done for a whole class of things instead of selectively only when needed, or computation could be avoided if some block metadata (such as block ids) is stored. On a conceptual level, all these changes make very good sense, but I don't know the complexity of diagnosing and making them in practice. Still, taking full sync time from a week to a day, and then further to maybe 5 hours, sounds astonishing and perhaps something that previous blockchain devs could have seen if they had familiarity with the whole codebase. Or maybe you are explaining it in a simple way and you save us a lot of the complexity of what's involved.

Do any of these performance optimizations result in lessening of security?