The last few months have been busy for our blockchain team. We are making great progress and want to share what we have been doing with you.
HF 20
A little over a month ago we made the tough decision to postpone HF 20 implementation in favor of enhancing our application framework. We discussed this decision with the witnesses and they were supportive of this change. We are still planning on implementing and releasing HF 20 in the coming months.
Scaling the Blockchain
In the meantime, we have been working on some fundamental changes to help scale Steem. We currently run 7 steemd nodes in AWS to support steemit.com and will need to continue to run more and more to support our growth. The key problem is that Steemd is single threaded, however, we are well along in the process of making it multithreaded.
Imagine the steemd server program is a stove. Right now, it has a single burner and we need to make a meal. A few of the dishes are the peer to peer network, accepting and evaluating blocks, and responding to API requests for data clients like steemit.com. We are rapidly switching pots, pans, and skillets onto the single burner to make each component of the meal. The increased request volume from steemit.com is delaying the meal.
Thus far, we have been buying extra single burner stoves to distribute the load because this was how blockchains were originally designed. But this is not ideal because all of our stoves (our computers) actually have 8 burners (CPU cores) and we are only using 1 each. We have been making changes to the steemd server program so that we can use all 8 burners and not need as many stoves. That way we can put a skillet on one burner, a pot on another, pan on another, you get the idea.
Previously, we had a single application that handled the p2p code, the database, plugins, and APIs. Now, everything is a plugin and only needs to communicate with other plugins that they directly depend on. This increased level of modularity allows for quicker development, easier code review, and greater parallelism. It will also allows us to switch components in and out without breaking the greater whole. We are now developing with building blocks rather than specifically designed puzzle pieces.
FABRIC
Our early tests are showing a 5-fold increase in throughput for servicing API requests. This refactor also lays the foundation to scale steemd into the future and implement the Steem Blockchain Fabric.
There are some API changes on the steemd node that should make accessing data in the blockchain easier and allow upgrading of our APIs without breaking existing calls. We have added a condenser_api which exposes all existing APIs so that we do not break backwards compatibility with existing apps. We have also rewritten the jsonrpc parser and it is completely jsonrpc 2.0 compliant. We will announce when this is deployed to staging.
Most of the work we have been doing is highly technical and challenging, but we look forward to keeping the community abreast of our work and strategies, on much of which we look to the Witnesses and community for collaboration. We’re happy to announce that we are confident these solutions will provide a robust foundation for meeting all of our future scaling needs and enable us to turn our attention to the features you all want to see.
Team Steemit
I am a layman when it comes to understanding the depths of blockchain tech. Although I am interested in the overall application to Steemit since I am an investor in Steem and the platform of Steemit.
I had seen posts speaking about scaling issues that would be faced so it was nice to read about what you are working on in a format that is easily understood by me. Thank you!
@overkillcoin and I updated the graphic for you to use! Link to Post
Nice format you got. Was much easy to understand with your format.
Nice.. Thanks!
Hi is this the steemit networks work presentation on network
really eye opening I Love Steemit
as years back we where on MLM network but
we are now smart now we are on blockchain network
Steemit
Love
U
FTFY. Credit where credit's due.
I don't have the technical knowledge to understand it :) but glad it was useful for someone. That's what we were hoping for when we made it so thank you for letting us know and we appreciate the upvote as well!
Also the version in the blog post link should view better than this one as well if needed.
Yeah its very good
Speaking of layman's problems is not something to talk about.
The important thing in knowing is who wants then he will get it. So you have done something amazing that can improve the mindset of others and the spirit that can make steemit always in the hearts of the layman and the professional.
a very confusing statement indeed
Why did you confuse
"The important thing in knowing is who wants then he will get it."
I can't make heads or tails of that
Hahaha, what is your mean ??? Whats you want me to
This sentence is also incomprehensible, I'm sorry I don't want to be a grammar nazi, but Mabey use a translation program.
impressive content! No doubt that's great!
Of cour
hmmm.... i couldn't tell if you were being rude or making conversation... lol. I'll get to the bottom of it! Following :)
Haha lol. How about you if we have a personal room to chat ??
Well, now that you mention it! I do want to find some more people to interview on my tv show, we could chat via Zoom! Let me know if you would be interested! We could talk about your sense of humor ;) or your travels and art ;)
Great thats amazing, interesting words to read. So, whats I will have from the interview LOL ?
I am funny guy. I can make every girl falling in love with my tooth. Haha.
Thanks for that, that's great! The post has been updated.
Wow! really love the community here, and to see everyone working together! Thanks for the love we could witness @steemitadventure @justinw and @overkillcoin <3
Thank you @jacquelyne. Even though I'm relatively new here, it's great to be a very small part of the incredible effort and talent involved in Steemit. I'm sure @steemitadventure would agree.
Glad you liked and used it! If you ever come across anything you think would help Steemit feel free to let me know. We work on everything from infographics to explainer videos etc.
We have been pondering a few things but as of now lack the technical (blockchain) knowledge to compose it well.
Happy to assist :)
Created in error.
Safe to say I am a visual person, and its nice sometimes to have little pictures summarizing something up like that! Thanks
I am the same way, probably why I illustrate! You are very welcome and glad you found it useful.
Yes its easy get well lots of thanks to jerrybanfield
Nice
looks good!upvoted!
very good news ..
:)
Thanks for the impressive content, No doubt that's great!
Oh really, you can also check my blog updates
https://steemit.com/@tsandesh
hehehe you said abreast hehehehe
Hello @trafalgar! I am sorry to intrude like this on a comment of yours in another post. I saw you are voting sometimes for @ocd posts. I happen to be one of the curators working with acidyo and its team.
Recently I made a post where I talk about my story on steemit in order to participate in blueorgy´s Steefmest ticket giveaway, do you mind checking it out? Perhaps you like it and would like to vote on it, I am going to use all the rewards from that post to buy a plane ticket to Lisbon :)
This is the post and again, sorry to intrude here :) Thank you for your time!
hehehe... thx for your support on my article today. love steemit more by the day..
LOL, at your comment man.
--
It is late for me, but just stopping by again to say thanks for a vote and the support -- you have been a big help to me and I appreciate it every time.
u still alive?
WOW $22 on one comment! dude come back! where did u go! this was 2 months ago! we need you back! You make SO much on ur posts its so awesome!
I saw in an old post of mine 4 months ago you upvoted me and helped me make one of my higher earning posts of all times and i really really am grateful for this thank you so much! I know for you it was just one click but It meant a lot! Thank you! I will try to posty more about my ayahuasca plants and other tropical plants of mine!!
I think the right decision was made choosing blockchain performance over ease of unboarding. Both are very important, but there is no point in a ton of users if blockchain can't handle it.
Very glad that core team still developing blockchain. This means that STEEM alive a well. There is silence sometimes from devs and it is frightening a bit.
Would EOS not be the ultimate solution for scaling steem?
Thank you for the update.. keep on trucking
Awesome, multithreading is definitely a huge key to scaling. We will be well prepared for hundreds of millions of users soon.
This really is great news. You are tackling the issues which have been noticed lately as occasional hiccups of the frontend.
Great approach of making the backend software multithreaded and divided into individual dedicated workers.
If I assume correctly this development is not changing the structure of the database itself, isn't it? Maybe, sometime in the future, when even multithreaded approach won't suffice, a quantum leap into sharded blockchain will be necessary.
Are you thinking about this possibility already or is it too far in the future.
A side question - does this breaking of the code into plugins and apis means that community based nodes will be possible serving specific plugins only.
Thanks for your work. And thanks for the answers! Sorry if you'll find my questions badly formulated. I am not a backend / blockchain / daemon / node programmer :)
That's something we've been planning for a while - the fabric architecture. This is a step toward that future.
That architecture is already in place now, and different types of users can run different sets of plugins depending on their needs. :)
Great. Thanks a lot for your reply. Now all I need is a set of simple instructions, a tutorial, on how to put up a node. Hardware and software requirements, setup, options, maintenance, ... Basic stuff :)
Maybe I've missed it and it's already out there? I mean a recent one. Heck, there should be an officially sanctioned tutorial someplace on steem.io site.
It's right in the repo:
https://github.com/steemit/steem/blob/master/doc/quickstart.md
I remember this one was the post right before I started to buy steemit. Flash forward 6 months and we already are getting results in real code!
Thankyou for the update, please keep them coming - this sort of direct communication from Team Steemit is needed especially when multiple influential members are spreading doubt about Steem's scalability..
If there has been one constant complaint, it has been about communication. The @steemitdev team is in a tough spot. If they focus on development it doesn't much leave time for communication. When they want to communicate it is never enough. Like you said, this sort of direct communication is vital and very welcome.
Awesomeness!!👍👍👍
cool graphic
Thanks !!👍👍👍
Nice!
Thanks ! im glad you liked it!
cute graphic!! lol thanks for the fun vibes @karenmckersie
Glad you liked it !! Your very welcome!!!
HI Karen! Would you be interested in being interviewed on my tv show via Zoom? Would love to have someone knowledgeable about crypto currency come on! Let me know if you're open to it!
Oh no haha !! Im know very little about crypto !! Im just learning like a lot of us ! 👌✌👍😂
WoW Karenmckersie I like it its on Speed>>>
Haha thanks that was so long ago !!😂💕👌✌👍
Thanks for the update! As a non-coder the analogy made it pretty easy to understand. ^^
Yes as I didn't have to sit here scratching my head ya know
Stove and making meals, you guys really like the "steem" theme I see :p. Keep up the good work guys, I trust your team is working their asses off.
Sounds great! Love forward progress
Cool dude!!
With the various changes that have been done no other just to simplify the user and facilitate the development team in the face of the problems encountered. With the addition of the stove it will add visitors to enjoy the dish. Thanks for information sir.
Dood.. are you a bot?
I am notva bot. I am alive like you too. Haha
Ok LOL
I have seen bots that crafted replies based on posts before, and the way your comment picked bits of the post and crafted the body of text, well, it had me wondering =p
Haha. So whre are u come from
I was raised on an island in Alaska, and escaped into the hellish jungles of Oregon, and effete civilization.
And you?
Iam not a bot. Why you ask me that question ?
I was mistaken.
While I don't understand every single thing in the post(especially the bit about fabric) I certainly am very much excited that sterns is becoming multi-threaded. This platform is gaining ~10k users every week, and that kind of growth will absolutely need sofrware that runs on multiple threads and multiple hosts.
Also, very happy to see that you have put in the effort to maintain backwards compatibility! This is something that companies that have a lot more resources than you guys don't do!
Thanks for such an entertaining analogy. It made the more technical jargon easier to grasp. I'm confident that Steemit will only get better and better as time progresses. Can't wait to see what we'll be "served" next!
nice
THank you for the update, hope to see a nice price increase due to this!
When will we see the next hardfork?
Great post!
How do you do it? That's such an original and pithy comment. Makes me wanna learn linux!
Fascinating. I'm sure going from single-threaded to multi-threaded for steemd is no easy task. But from the sounds of it, you guys are doing great. We sincerely appreciate all your hard work @steemitdev ! :)
I'm still concerned that the interface is worse than geocities, the site's growth is capped and users will continue leaving until that's addressed.
Good job guys, thanks for the update!
"The fabric is coming!" :)
Great to hear about and look forward to the continued improvements.
The multi-thread-ness will be a great improvement and ability to scale more easily.
Keep up the great work!
Thanks!
Thank you very much @steemitdev ,It is huge what you are doing to help steemit to be more performing in the future .
We must also applaud this courageous initiative on your part. Sometimes one is obliged to approach subjects a little critical.
il faut aussi applaudir cette initiative courageuse de votre part. parfois on est obligé d'aborder des sujets un peu critiques.
This is encouraging in the medium-term.
Which AWS plan do you run your full RPC nodes on please?
I see it reduces the replay time massively, but what is the RAM usage difference between SKIP_BY_TX_ID=[OFF/ON] when compiling steem?
Thanks for the communication.
I believe Steemit is going to blow up!
@steemitdev Great work! I do have 1 question though. I am a witness in position #60 and you said;
" We discussed this decision with the witnesses and they were supportive of this change. We are still planning on implementing and releasing HF 20 in the coming months."
But I don't recall ever discussing this with anyone on the dev team ? Can you please elaborate? Thanks again.
Inquiring minds also want to know. Thanks for asking this question!
These kind of updates make me feel very confident in the platform and it would be great to read development news and plans on a regular basis. Thank you all for your great work!
This is a great start. Thank you for acknowledging the scaling issue. Thank you for working hard to deal with it. Please devise a strategy for maintaining and expanding on the information you present to the community. All the best.
Brilliant, just brilliant! Now can you direct your attention to a very pressing matter: Please program a CANCEL button for the ones that want to leave Steemit. If you can produce a REGISTER button, then you can also program a CANCEL button. As another user here stated: It is not possible to CANCEL on Steemit, because everything is written in stone on the blockchain. I agree. But as the old Romans already knew, EVERYTHING can be chiseled into stone, even a CANCEL procedure.
And while you are at it, maybe you can have a few thoughts in your young brains what happens if a member of Steemit dies or wants to sell his/her Steemit account. After all, this is an asset like every other asset and therefore I must be able to gift it, trade it or pass it on to my kids.
Thank you.
Well, you can gift your Steem, if you have time to power down.
I actually, to my surprise, agree that heirs, assigns, and survivors is an issue regarding power down and probate, at least.
Well, then I would appreciate it if somebody would attend to those issues. They will come up in future anyway and then there'll be big tears and legal cases.
Nice info, is there any place for proposals? New users really get bounced off by the fact they have to put an url for their avatar image, to complicated. Greetingz from fellow developper
Finally the change we expected :)
This is good news :) thanks
It's good to hear that work is progressing. There will be challenges is scaling up Steemit. When there are millions of users it will get really interesting. At least you can learn from what others have gone through.
Please keep us updated
I'm no expert but mightn't it also help if there was a no-javascript ultrabasic version of Steemit running as well that people on mobile and/or low-bandwidth connections can use?
Might lighten the load on both ends, is what I'm saying.
Its very good news)))
Thanks very much for this nice social network.
I mean its realy cool for connection people idea....
Where is the best place to mine STEEM?
@tipU - send tips by writing tip! in the comment and get share in service profit :)
By upvoting this comment you support the service - thanks!
tip
This is fantastic. Looks like Steem will be the first multi-threaded blockchain ahead of EOS.
I did not see I/O mentioned - this seems like a greater challenge currently. I assume you're working to optimize it, but it would be great if you mentioned specifics.
Naturally. Our team who actually conceived and designed multi threaded scaling with Chainbase and the rest of the steem blockchain stack have always and still work here. I'd be surprised if any group copying the strategy got there first.
Multithreaded? could you kindly explain more on this? is this the same concept in Computer Science e.g. multithreaded Programming?
Thanks, @justyy
I wasn't sure I had read this part of the post right. I'm glad you're confirming this part.
Thank you for sharing this info. I wonder can I get to know the things that will be implemented in the next hard fork? Or is the kept a secret till it is officially announced.
P.S: I post stuff on How to Become a Writer and make money with Steemit, please take a look at it.
Thank you.
Thanks for this. Very helpful indeed.
Thanks for the update.. i'm new here and i'm already addicted to steemit, let's make this network huge
Steem Blockchain update is awesome
The new format its easy to understand.
Thanks to @steemitadventure and @overkillcoin
Thanks for trying for all users, We hope goods in future
Awesome