This tutorial is part of a series where different aspects of programming with EOS are explained, the full curriculum can be found below. This part will focus on building an unending stream with fallback api_endpoints and goes into more depth about the head_block and irreversible blocks.
Repository
What will I learn
- Head block & last irreversible block
- Unending stream
- Multiple api_endpoints and switching
Requirements
- Python3.6
Difficulty
- basic
Tutorial
Setup
Download the file from Github. There is 1 file stream_blocks.py. Running the file without any arguments will test the api_endpoints for their speed to decide what node is best and then start streaming from the current head_block. Adding the argument irreversible will perform the same test but only stream irreversible blocks.
Run scripts as following:
> python stream_blocks.py
Head block & last irreversible block
On the EOS Blockchain new blocks are produced by 21 block producers (BPs) every halve a second. As this is not enough time for the blocks to propagate over the entire network BPs are chosen to produce a set amount of blocks before another BP takes the role as a producer. The frequency in which the BPs are chosen is based on the amount of votes this BP has received.
This means that the current block producing BP is slightly ahead of the other BPs. For this reason there is a distinction between the head_block and the last_irreversible_block. Every block that has been verified by allBPs becomes an irreversible block. As it takes time for the blocks to propagate over the network and get verified by all BPs there is a delay before a block becomes irreversible.
eosnetworkmonitor
Depending of the type of application use of the head_block or last_irreversible_block can be preferred. For example dealing with financial transaction using the last_irreversible_block is recommended, while building a chat application speed is more important so the current head_block is preferred. Information about the head_block and last_irreversible_block is found by using the get_info API request.
def get_head_block(self, irreversible=None):
api_call = '/v1/chain/get_info'
address = self.url + api_call
request_json = self.s.get(url=address).json()
if not irreversible:
return request_json['head_block_num']
else:
return request_json['last_irreversible_block_num']
Unending stream
Knowing this a robust unending stream can be made. First the start_block has to be decided. If none is given it is taken from the current head_block or last_irreversible_block which is set in self.block. This is to keep track of which blocks have been processed in case the stream crashes and needs to restart, starting at self.block.
self.block = self.get_head_block(irreversible)
def stream_blocks(self,
start_block=None,
block_count=None,
irreversible=None):
if not start_block:
start_block = self.block
else:
self.block = start_block
When upgrading code for the application where it has to be shut down it can be preferred to start at the last processed block. In which case a start_block can be set.
The next block of codes takes the start_block and then ask for the current head_block / last_irreversible_block to create a range ofblocks. These blocks are requested and after the start_block is set to the last requested block. Repeating indefinitely.
The upside of this is that it becomes impossible to request blocks ahead of the current head_block / last_irreversible_block block. The downside is that it requires additional API requests to retrieve the information about the head_block / last_irreversible_block. In the case of using the last_irreversible_block this is the only way to make sure the blocks being requested are irreversible. As it is possible to requests blocks ahead of the last_irreversible_block, so no error message would be returned if that were the case.
while True:
head_block = self.get_head_block(irreversible)
for n in range(start_block, head_block + 1):
yield self.get_block(n)
self.block += 1
start_block = head_block + 1
In case something unexpected happens, such as losing internet connection the stream is put into a while loop with a try and except clause. This will keep trying to restart the stream.
while True:
try:
for block in self.stream_blocks(irreversible):
print(f'\nBlock {self.block}')
except Exception as error:
time.sleep(60)
print(repr(error))
Multiple api_endpoints and switching
Problems can also occur at the api_endpoint side. Proving additional api_endpoints to fall back to solves this problem.
self.nodes = ['http://mainnet.eoscanada.com',
'http://api-mainnet1.starteos.io',
'http://api.eosnewyork.io']
self.node_index = 0
self.url = self.nodes[self.node_index]
In the case the api_endpoint returns an unexpected error the node_index will be increased and adjusted to stay in the limits of the nodes list. The next api_endpoint is then selected and the session is renewed.
def reset_api_endpoint(self):
self.node_index = (self.node_index + 1) % len(self.nodes)
self.url = self.nodes[self.node_index]
self.s = requests.Session()
print(f'New api_endpoint: {self.url}')
Additionally when starting the stream all the nodes are tested for their response time. For each api_endpoint 100 blocks are retrieved and it is timed how long this process takes. The results are added to the results list as tuples (url, time). Then the results are ordered by their time sort(key=operator.itemgetter(1)). Eventually a sorted nodes list is returned by stripping only the urls from the each tuple in results.
def test_api_endpoints(self):
results = []
for url in self.nodes:
self.url = url
start_time = time.time()
print(f'\nTesting: {url}')
for block in self.stream_blocks(start_block=1, block_count=100):
print(f'{self.block}/100', end='\r')
end_time = time.time() - start_time
results.append((url, end_time))
print(f'Took {end_time:.4} seconds for completion')
results.sort(key=operator.itemgetter(1))
self.nodes = [node[0] for node in results]
print(self.nodes)
Running the code
The code can be run in two modes, either the current head_block mode or the last_irreversible_block mode. Regardless of the mode a test of each api_endpoint will first be conducted to sort the api_endpoints by their speed.
python stream_blocks.py irreversible
Testing: http://mainnet.eoscanada.com
Took 27.98 seconds for completion
Testing: http://api-mainnet1.starteos.io
Took 33.16 seconds for completion
Testing: http://api.eosnewyork.io
Took 29.68 seconds for completion
['http://mainnet.eoscanada.com', 'http://api.eosnewyork.io', 'http://api-mainnet1.starteos.io']
Mode: last_irreversible_block
Block 5998244
Curriculum
- Part 1: Using Python To Make API Requests To EOS Public API Endpoints
- Part 2: Streaming EOS Blocks With Python
The code for this tutorial can be found on GitHub!
This tutorial was written by @juliank.
Thank you for your contribution.
Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.
To view those questions and the relevant answers related to your post, click here.
Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]
Do you mean comments in the example code in the tutorial itself? As the code itself is commented.
Hey @steempytutorials
Thanks for contributing on Utopian.
We’re already looking forward to your next contribution!
Want to chat? Join us on Discord https://discord.gg/h52nFrV.
Vote for Utopian Witness!
It is very technical.....
Love @beingindian
https://steemit.com/bitcoin/@beingindian/coindelta-flux-p2p-is-it-a-game-changer-in-crypto-world-4befa313469ff
Upvote & comment & follow me for latest update.... ✌🏼
Congratulations @steempytutorials! You have completed the following achievement on Steemit and have been rewarded with new badge(s) :
Click on the badge to view your Board of Honor.
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
SteemitBoard World Cup Contest - Final results coming soon