Hello Steemians, welcome to our latest Steemit Engineering Update. The goal of these updates is to provide developers and users with greater insight into our engineering operations. You can view our last engineering update here.
Condenser/Wallet Split
We are nearing completion of our project to separate Condenser into separate social and wallet applications. We started by creating a copy of the Condenser app and designating one copy as the social app, and one as the wallet app. We then removed all of the social features from the wallet app and all of the financial features from the social app. That process is now complete, and what remains is user experience work in the wallet as well as some testing. Once that work is complete, the two dedicated apps will be more secure and cost much less to run than the old Condenser while enabling us to add new functionalities more rapidly.
Cost Reductions
Last week we completed an initial implementation of rate limiting for our public API. That enabled us to reduce our required number of running instances and our data transfer costs significantly. We worked with a number of prominent Steem based projects that rely on our public API to ensure that they continue running smoothly even with these changes. Those projects include busy.org, oracle-d, and SteemConnect. SteemConnect in particular is an application that many other applications (e.g. Partiko) within the ecosystem rely on.
We have settled on acceptable instance sizes for all of our applications and autoscaling has been fine-tuned to ensure an acceptable cost-to-performance ratio. This project is complete for the time being, but we will take another pass at it after MIRA is released as the required instance type for our Steem nodes may change.
Up next for the infrastructure cost reduction project will be planning a move to a different CDN (Content Delivery Network) for our image caching. While AWS's Cloudfront product is great and we will likely still use it for some other things, it can be rather expensive for a high volume of data transfer like that required for our images. We will begin by testing these changes in our dev environment throughout next week. When pushed to production, these changes should be transparent and unnoticeable to users.
Mira/RocksDB
We have successfully finished our first reindex in our dev environment. After deploying steemd to our dev environment we discovered a few intermittent problems when a node is live. We are currently debugging and fixing those problems and are hopeful to have a full stack test soon.
Be sure to follow @steemitblog if you would like to see more engineering updates like these!
The Steemit Team
@steemitblog,
What I have seen so far, after the recent fall of STEEMIT INC, you guys made a huge progress that could lead STEEM to top 10 position in CMC!
Hope to see SMT launch before the end of this year and 2020 will be ours!
Cheers~
@brockpierce should check this out and invest even MORE into steem!
More proof that steem is about to become more efficient and allow for a lot more to happen.... = more investors
That’s what I like to see, progress! 🚀🚀🌝🌝
Posted using Partiko iOS
Thanks guys for the regular updates!
It seems that you are getting back on track with the development again.
Good work.
Keep moving forward! Question about images do you reduce the size of them after people upload them. For example If I upload a 2MB image do you have something in place to reduce that image size down to a few KB instead? If not that honestly seems like a smart and right direction to go in terms of saving money via data costs.
You still hitting the dates in this table?
Would be really interesting to see how far they are with the progress regarding this table in general. I doubt though that there was any general progress with SMTs due to a lack of updates regarding the progress
Could be. They will eventually have to say something and that "eventually" is coming soon.
Posted using Partiko Android
We will know if there is a SMT lite test net in less than two weeks I guess. I'm very curious about the progress made with respect to these tokens.
Looking forward to the full roll out. @SteemitBlog
Posted using Partiko iOS
Why Rockdb ?
Thank you for these updates.
Sounds great. Thanks for the updates.
Posted using Partiko Android
More progress in the last couple of months than I've seen since starting here a year ago! Excellent to see!
Great update... by doing things like these actually produce more marketing mileage than spending ads. People become enthusiastic because we see movement and progress.
Excellent work steemit inc . Thank you for staying transparent here and keeping us updated. Steem is an incredible technology, that is many, many years ahead of it's time. It takes careful incubation. But, when the "mixture/recipe" is right, mass adoption is not only achievable, it is inevitable.
How can the image issue be solved with steem blockchain? There needs to be some kind of decentralised file storage solution for Steemit, so users can opt in to have their images stored on IPFS? or even steem tokens or something. I'm no expert just wondering, as don't want Steem to get left behind with the whole token revolution. Sites like MakersPlace, Digital Art Chain, OpenSea.io
Someone could fork Storj and re purpose it for Steem and use Steem Engine for the token platform...
Busy is using IPFS for it images.
If we are going to rely on token maybe it's better to wait for SMTs first
You just planted 0.10 tree(s)!
Thanks to @ucukertz
We have planted already
7718.69 trees
out of 1,000,000
Let's save and restore Abongphen Highland Forest
in Cameroonian village Kedjom-Keku!
Plant trees with @treeplanter and get paid for it!
My Steem Power = 26913.95
Thanks a lot!
@martin.mikes coordinator of @kedjom-keku
thank you for the update! Keep em coming, great to hear there is good progress being made.
Thanks for your transparency and the updates. We keep rolling and I love to see this :)
Thank you, steemit team. Your hard work and communication is appreciated.
Thanks for the update
Thanks for the report and keep up the the good Blockchain development work!
I hope some way will get implemented to interact with the rate limiting implementation from code. The rate limit currently pretty much killed the daily Flag-war stats visualization, and while it might be possible to fix it (not quite sure yet) by implementing rate-limit awareness in the underlying library, I will most definitely need a way to communicate about rate limiting metrics with the rate limiting implementation, that currently just starts sending HTTP 4xx responses.
A number of questions to help me get a handle on this thing:
I hope a good answer to these three questions might help me get things on track to get the @pibarabot daily flag war posts up and running once more.
Oh, just a thought: It might be a food idea to implement something like a ratelimit_api.get_current_ratelimit_window_size() API call and add it to the APPBASE API documentation. That way client libraries could easily be made rate-limit aware, and maybe the impact could be mitigated by such an API feature.
Great to see these results of your diligent effort. It's very encouraging, and I hope that soon the fruit of simplification and reductions of cost are magnified across the ecosystem.
Thanks!
Hell yeah! I'm loving the new vlog format that you folks have been doing lately. What this site was missing was some sort of clear and coherent direction coming from up top (no offense Ned). I always felt like this place was a random ball of chaos without much clear instruction on what was going on or happening behind the scenes.
How about SMT's? Any progress there?