Everything can be faster on Hive - hive-js & dhive new changes

in HiveDevs2 months ago (edited)

alien-hive.jpg

Explaining why there were problems with RPC nodes and what was the problem and why you must update your application.

Target audience

I explained everything as simple as I could. The main call is for the app owners to update their applications with the updated libraries.

After the recent timeout problems, it was necessary to take these steps and update the libraries.

I updated 2 major JS libraries, hive-js and dhive, to help resolve the timeout problems.

The "call" method

The "call" method was a way of talking to hived (the hive blockchain's application). This method got deprecated and removed from hived with the appbase update (HF19?).

So what has happened since then? Well, we were still using the "call" method for most of our applications. But the "call" method is not known to hived anymore, so the requests have to go through a proxy called "jussi". Jussi is installed in front of all the public RPC nodes.

Jussi takes the "call" method and translates it into the correct format then sends it to the hived. The problem is, you are adding an extra step for the API requests and it slows them down.

An example of the "call" method:

{"id":0,"jsonrpc":"2.0","method":"call","params":["database_api","get_dynamic_global_properties",[]]}

Notice "method":"call". So jussi takes that and translates it into the new format:

{"id":0,"jsonrpc":"2.0","method":"condenser_api.get_dynamic_global_properties","params": []}

See what happened? The user thinks the call was going to database_api but actually, jussi translates it into condenser_api.

To be honest, this is not the best way of doing things. The idea behind jussi was good but implementation was a mess.


What is the "new" format?

The "appbase" style calls are the calls that are supported directly by hived and don't need a translation.

It's not new and has been there for years but we didn't bother updating our libraries to use the supported methods.

With switching to the appbase calls, every request to the RPC nodes becomes faster.

This is the first change to both hive-js and dhive libraries. They both were using the deprecated "call" method.


You might not need jussi

Till now, hive-js wouldn't work with hived nodes until you added jussi in front of them. But with this update, hive-js will work with hived nodes. But remember, you will still need jussi for the hivemind calls. Also, there might be caching benefits in using jussi.


Broadcasting transactions

This is the important part. There are two methods for broadcasting a transaction.

  1. broadcast_transaction_synchronous
  2. broadcast_transaction

The first method sends the transaction to the RPC node, the RPC node validates the transaction, then waits for the transaction to be included in a block, then sends back a response including the transaction id and block number.

The second method sends the transaction to the RPC node, the RPC node validates the transaction, then sends an empty success response back.

In the second method, the library has to generate the transaction id locally. Which is way too much faster. In this method, the only way for a transaction to fail is expiration or micro forking maybe. So in most cases, it's good enough.

Of course, hive-js and dhive both were using the first method. When many people connect to the RPC node and wait there, it becomes problematic. The best way for handling huge traffic is to release a connection as soon as possible so other connections can be made.

Both libraries now use the second method for broadcasting transactions. You may already notice its huge impact on the speed of your votes and comments.

Peakd is already updated. Keychain is waiting for google approval and the mobile version will be soon updated. Some apps are catching up like Hivesigner, and Ecency. There was a problem updating hive.blog and that will be also updated.


Why bother updating

  • Everything becomes faster for the end-user
  • RPC nodes don't suffer overloading
  • Apps become future proof
  • It's easy as running a command

How to update your apps

hvie-js v2.0.1 - Release notes
Used in hive.blog, peakd, keychain, and many other apps.

npm install @hiveio/[email protected] --save

The minified JS for browsers:

https://cdn.jsdelivr.net/npm/@hiveio/[email protected]/dist/hive.min.js

dhive v1.0.0 - Release notes
Used in ecency (mobile & web) and many other apps.

npm install @hiveio/[email protected] --save

The minified JS for browsers:

https://cdn.jsdelivr.net/npm/@hiveio/[email protected]/dist/dhive.js

hive-tx v4.0.0 - Release notes
Used in keychain mobile and other apps.

hive-tx already is using the appbase methods and there is a function called .broadcastNoResult() for the fast transaction broadcasting. But I did update hive-tx so now the .broadcast() function is also updated.

npm install [email protected] --save

The minified JS for browsers:

https://cdn.jsdelivr.net/npm/[email protected]/dist/hive-tx.min.js

Note:
@foxon and @emrebeyler confirmed that beem doesn't need an update.


Scaling problems?

The recent problem doesn't seem to be a scaling problem. It's more related to the libraries and apps using the deprecated methods.

The recent library changes seem to be solving most of the problems. Although there are investigations still going on by @blocktrades regarding the timeout problems.

Hive blockchain in theory can handle much more activity than this. Some say like 1,000 transactions per second. But there might be minor problems because those scenarios are not actually tested.


Final words

I'm sure all the problems will be solved and we will see much better growth. I'm bullish on Hive and will continue my contributions as far as I can.

Thanks to the other devs who updated their applications on time to help with the problem.


Image source: pixabay.com

Sort:  

In general terms, everything is clear. Technical problems are being solved. This is good news. Everyone needs it. Thank you very much for your work.

@cherepin0512 you rock bro, you are correct. Everything is starting to flow better again.

untitled.gif

I am sure this informative post from @mahdiyari will benefit others and bring peace of mind to many who have been frustrated.

Thank you for being a solid, constant, and just overall amazing person on Hive, Mahdiyari!

untitled.gif

Thanks a lot for your work ! It's actually quite insane how much it improves UX

Already thought Hive is running into scaling problems, because of the success of Splinterlands and the growing number of transactions. Would be interesting to test, how much load the blockchain and api servers can actually handle.

@blocktrades said in a recent post that he thought his node alone could handle 12x more traffic. And i'm guessing he'll just add to it when it gets there.

There haven't been too many full blocks but if we get there I have also heard developers say they're willing to consider speeding up the block time. So if we go from 3 seconds to 1.5 then that seems like there is even more room for transactions.

As for peakd.com i've never heard the developer complain about us having too much traffic I guess i could ask at what point our servers would have issues with traffic. But as long as there isn't a crazy sudden spike i assume we'll be able to adapt in plenty of time.

But yes I would love to see some more specific tests and stats from people... i'm just using random things I've heard from others.

Thank you!

Does this mean @splinterlands can update their DHive and we can get our public nodes back now?

@yabapmatt promised to work on it. Might take a few days.

Thank you for the upgrades! I've reviewed all of my apps and made the corresponding upgrades for hive-js and hive-tx

Really appreciate your work on this update !!

I also want to thank you for the program for working on the blockchain Hive, Peakd. This is a really good job. I wish you success.

Peakd has been pretty solid. The automatic node switching function helped during this time. When it did not I was able to go to peakd settings and effortlessly manually switch to working nodes.

Also, I see I can view who is sending me HP delegation now on peakd, so thanks for the upgrade @peakd!

They are cool guys!

Good job man!

I really appreciate @mahdiyari and @blocktrades or all other hive blockchain developers work, few days ago I starts faceing some problems while using my @ecency app, but after updating it from Google play store, I feel the difference all problems are solved, speed of the app also increased, now I understand after reading your article what's happening in the background development, thanks again nice work keep it up

Thank you.

// By the way: it's one of those things that were supposed to be done by ned (may the Sun shine in his face) back in 2018. Well not done.

Thank you for the detailed info, learned something!
In a scenario like eg.: a scheduled post. Would it make sense to still use broadcast_transaction_synchronous and would a success response pretty much guarantee a success or would you recommend broadcast_transaction, followed by eg. an attempt to retrieve the post? Or is a transaction 'guaranteed' only after its block number is less or equal to 'last_irreversible_block_num'?

On a side node, I've recently checked the various implementations of client libraries and noticed that for broadcasting transactions some library chose 'ref_block_num' equal to value of 'head_block_number' and other library uses 'last_irreversible_block_num-1'. Is there a difference and does it matter which block a transaction uses as a 'ref_block_num'?

I'm asking in order to comprehend how things work so I can learn what the preferred usage is.

The synchronous method is deprecated and most likely will be dropped soon. So use the other one. The suggested API for checking transaction status is transaction_status_api.find_transaction. You can find it in the devportal. developers.hive.io

Yes, when the transaction is included in a irreversible block, there is no way for that to reverse or fail.

AFAIK for ref_block_num it doesn't matter. It can be any recent block.

Good job!

Good job !😍😍


The rewards earned on this comment will go directly to the person sharing the post on Twitter as long as they are registered with @poshtoken. Sign up at https://hiveposh.com.

Congratulations @mahdiyari! Your post has been a top performer on the Hive blockchain and you have been rewarded with the following badge:

Post with the highest payout of the day.

You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word STOP

Check out the last post from @hivebuzz:

Feedback from the August 1st Hive Power Up Day


Congratulations, @theguruasia You Successfully Shared 0.500 WINE With @mahdiyari.
You Earned 0.500 WINE As Curation Reward.
You Utilized 5/5 Successful Calls.

wine_logo


Contact Us : WINE Token Discord Channel
WINE Current Market Price : 0.270

This might be off-topic, but I wanted to ask you this question. Is it true that 100% of an upvote goes to the voter, the curator, after 5 minutes on Hive? Back on Steemit, it was set at 15 or 30 minutes, something like that. I recently read on Quora.com and I think other places too that it is at 5 minutes currently. I don't know but that may be the case.

The curation penalty was removed in HF25. Read posts by blocktrades and hiveio regarding the previous HF.

Just want to note that for any existing applications a simple npm update is probably not going to pick up the latest version because of the major version change. Need to do it manually.

!LUV

<><

<><

LUV

Connect

Trade


@mahdiyari, you've been given LUV from @donchate.

Check the LUV in your H-E wallet. (1/1)