We are delighted with the release of the Dawn 3.0 and have been experimenting to prove some of the hypothesis mentioned in release notes.
Our first goal is to find out if it is technically possible to achieve TPS published in <Dawn 3.0 Now Available> and Hong Kong Meetup presentation on April 6th. Unfortunately our results from the experiments have not met anywhere close to the TPS of worst cases published so far.
We believe that benchmarks of the EOSIO software have a very profound meaning to all members of EOS community and especially for block producers with a big interest. For the permanent stability of the EOS main chain, all block producers must fully understand the effects of their hardware and software configurations and then register their EOSIO parameters accordingly. In other words, through the benchmarks of EOSIO, it is imperative to explore extensively the limits of CPU, NET, RAM, and Storage to meet the needs of the situation where mainnet operates, and prepare hardwares as well as the operating practices accordingly. We will benchmark the operating limits of the EOSIO software from the CPU side and complete thoroughly by going one step at a time.
Our results will provide the block producers sufficient information and guide the EOS token holders about token staked necessary to their contracts. By sharing our results, we firmly believe that we are making significant contribution to the stability of the EOS mainnet.
Without further ado, please find the details of our experiments and hope to enlighten you as it certainly did it for us:
- Our TPS experiments by Sungmin Ma, Lead System Engineer at EOSeoul.
- Max sustainable(not peak) TPS is under 500 with multiple nodeos daemons on single
or multiple machines. - Detailed report (PDF)
- scripts : http://testnet01.eoseoul.io/script/bmt_client.tar.gz
- usage
- ./bmt.sh prepare
- ./bmt.sh run_job
- Max sustainable(not peak) TPS is under 500 with multiple nodeos daemons on single
We are trying to reproduce the results shown on Dawn 3.0 by Block.one, but seems to be far off the marker. In order to at least match the results, we requested the following information to Block.one.
- Test environments and those scenarios of block.one published in <Dawn 3.0 Now Available>
- if possible, with respective test scripts or unit test codes by block.one
- NET-, RAM-, Storage- bottlenecked TPS benchmarks by block.one
- Whether current config.hpp in libraries/chain/include/eosio/chain/ is go-to config or not
- EOSIO parameter guide for block producers
Any feedback on this would be much appreciated and please do not hesitate to contact EOSeoul for this or any other queries.
You can receive latest news of EOSeoul and EOS related technical insights if you join EOSeoul's telegram group.
Telegram (English) : http://t.me/eoseoul_en
Telegram (简体中文) : http://t.me/eoseoul_cn
Telegram (日本語) : http://t.me/eoseoul_jp
Telegram (General Talk, Korean) : https://t.me/eoseoul
Telegram (Developer Talk, Korean) : https://t.me/eoseoul_testnet
Steemit : https://steemit.com/@eoseoul
Github : https://github.com/eoseoul
Twitter : https://twitter.com/eoseoul_kor
Facebook : https://www.facebook.com/EOSeoul.kr
Our measurements were made using a transaction generator plugin and not via cleos. We will provide reproducible benchmark instructions and/or diagnose the slowdown.
Thank you for video and gist of BlockOne experiment setting. Video and gist post are very helpful and lead to reproduce 1000 TPS with txn_test_gen plugin on binaryen with full signature checking. Max sustainable TPS with BlockOne experiment setting is 1300+ on Intel i7-6700 @ 3.4GHz.
We also reproduced sustainable 2000+ TPS with txn_test_gen on wavm JIT with full signature checking. Producer CPU usage with JIT is about 40% on Intel i7-6700 @ 3.4GHz.
We plan to do a lot more tests under more realistic mainnet environment scenarios to share EOSeoul benchmark settings for block producers. When our next report is done, we will post the results again.
And we received email replies from BlockOne yesterday. It says that the guide on the block producing nodes will be included in the Developer Portal, BlockOne are working on the guide at the moment and will publish in May. We are glad to hear that news.
Thank you, @dan
We look forward to checking the instructions. Thank you for your comment, @dan
@eoseoul see the last comment on https://github.com/EOSIO/eos/issues/2078 for information on how to repeat the performance test at 1000 tps.
There is a video demonstration as well
I was about to share this video. :)
Thank you, @gleehokie. We will review your information.
No mention of any plugins activated.
What plugins were enabled?
Were the RPC APIs active on the block producing node?
Our test was done under the general configuration of the testnets such as Jungle and Arrowhead which is different to what Dan has done. And the active http plugin.
Thanks for your question and hope our feedback is helpful.
We plan to do a lot more tests under more realistic mainnet environment scenarios. Please do give us your thoughts on those reports as well.
Thanks.
Here is 1000 TPS achievement sample script. ""showing 1000TPS. Producer CPU usage is only ~48%"
Thank you for sharing this link, @j30231. We tested with BlockOne experiment setting using txn_test_gen plugin, and reproduced 1000 TPS on binaryen interpreter with full signature checking.