How to follow all steem-engine token movements

in #steem-engine5 years ago (edited)

The steem-engine token balance of an account can be influenced by the following actions:

  • issue - can be performed by the token creator and increases the number of token
  • transfer - a token is transfered between two accounts
  • buy - when a buy order is fullfilled, the token is transfered into the account
  • sell - when a sell order is fullfilled the token is permamently removed from the account

All these operations are custom_json operation and they have to be accepted by the sidechain. Parsing only the custom_json is not sufficient for knowing who owns token.

Market operation

buy

A buy order allows an account to buy token from the market. The STEEMP that are needed in exchange for the tokens is locked into the market.

sell

A sell order allows an account to sell token to the market. The token that should be sold are locked into the market until the order is fullfiled or canceld.

Fullfilling a buy order

When there is already a matching sell order on the market, three internal transfers are generated:

  • STEEMP from the buyer is transfered to the market
  • The tokens are transfered from the market to the buyer
  • The STEEMP are transfered to the seller

When there is no matching sell order, only one transfer is generated:

  • STEEMP is transfered to the market

In both cases, the same custom_json is broadcasted. It is only possible to know, how much token each account has, when it is known if there is a matching sell order or not.

Fullfilling a sell order

When there is already a matching buy order on the market, three internal transfers are generated:

  • STEEMP from the buyer is transfered to the market
  • The tokens are transfered from the market to the buyer
  • The STEEMP are transfered to the seller

When there is no matching buy order, only one transfer is generated:

  • tokens are transfered to the market

In both cases, the same custom_json is broadcasted. It is only possible to know, how much token each account has, when it is known if there is a matching buy order or not.

If a order is fullfilled or not can be found out by using get_transaction_info(trx_id) from the steemengine python library. The transaction_id of the broadcasted buy/sell custom_json is needed as function input.

trx_info = api.get_transaction_info(b["trx_id"])
payload =json.loads(trx_info["payload"])
logs = json.loads(trx_info["logs"])
if "events" in logs:
    print(logs["events"])

This function returns a json with a logs field. When it has an events field, the order was sucessfully placed. When only one transfer is stored into logs["events"], there was no matching buy/sell order.
When there is more than 1 transfer, the buy/sell order was sucessfully.

Token movement tracker

The following script prints token movements. When a buy or a sell operation at the market is sucessfully, it is shown and all transfers are shown.
In the screen shot, bobby.madagascer bought 1 FREEX from dakeshi.
image.png

# This Python file uses the following encoding: utf-8
# (c) holger80
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from beem import Steem
from beem.comment import Comment
from beem.nodelist import NodeList
from beem.utils import formatTimeString, addTzInfo
from beem.blockchain import Blockchain
from steemengine.api import Api
import time
import json


if __name__ == "__main__":
    nodelist = NodeList()
    nodelist.update_nodes()
    stm = Steem(node=nodelist.get_nodes())
    
    api = Api()
    
    blockchain = Blockchain(mode="head", steem_instance=stm)

    start_block = blockchain.get_current_block_num()
    for b in blockchain.stream(start=start_block, opNames=["custom_json", "transfer"]):
        if b["type"] == "custom_json":
            if b["id"] != "ssc-mainnet1":
                continue
            trx_info = api.get_transaction_info(b["trx_id"])
            if trx_info is None:
                continue
            payload =json.loads(trx_info["payload"])
            logs = json.loads(trx_info["logs"])
            if "errors" in logs:
                continue            
            if trx_info["action"] == "issue":
                print("%s issued %s %s" % (trx_info["sender"], payload["quantity"], payload["symbol"]))
            elif trx_info["action"] == "transfer":
                print("%s transfered %s %s to %s" % (trx_info["sender"], payload["quantity"], payload["symbol"], payload["to"]))
            elif trx_info["action"] == "sell":

                if len(logs["events"]) == 1:
                    print("%s put sell order %s %s for %s" % (trx_info["sender"], payload["quantity"], payload["symbol"], payload["price"]))
                else:
                    print("%s sold %s %s for %s" % (trx_info["sender"], payload["quantity"], payload["symbol"], payload["price"]))
                    for transfer in logs["events"]:
                        print("    - %s transfers %s %s to %s" % (transfer["data"]["from"], transfer["data"]["quantity"], transfer["data"]["symbol"], transfer["data"]["to"]))                    
            elif trx_info["action"] == "buy":
                if len(logs["events"]) == 1:
                    print("%s put buy order  %s %s for %s" % (trx_info["sender"], payload["quantity"], payload["symbol"], payload["price"]))
                else:
                    print("%s bought %s %s for %s" % (trx_info["sender"], payload["quantity"], payload["symbol"], payload["price"]))
                    for transfer in logs["events"]:
                        print("    - %s transfers %s %s to %s" % (transfer["data"]["from"], transfer["data"]["quantity"], transfer["data"]["symbol"], transfer["data"]["to"]))
                    
            elif trx_info["action"] == "cancel":
                print("%s cancel %s order with %s" % (trx_info["sender"], payload["type"], payload["id"]))
            elif trx_info["action"] == "withdraw":
                print("%s withdraw %s STEEMP" % (trx_info["sender"], payload["quantity"]))
                
            elif trx_info["action"] == "updateMetadata":
                continue
            elif trx_info["action"] == "create":
                print("%s created the %s token" % (trx_info["sender"], payload["symbol"]))
            else:
                print(trx_info)
        elif b["type"] == "transfer":
            if "ssc-mainnet1" not in b["memo"]:
                continue
            trx_info = api.get_transaction_info(b["trx_id"])
            if trx_info is None:
                continue
            payload =json.loads(trx_info["payload"])
            if trx_info["action"] == "buy":
                print("%s bought STEEMP for %s" % (trx_info["sender"], payload["amountSTEEMSBD"]))
            elif trx_info["action"] == "removeWithdrawal":
                print("%s withdraw STEEMP for %s" % (payload["recipient"], payload["amountSTEEMSBD"]))
            else:
                print(trx_info)

In order to run the script, steemengine must be install by

pip install steemengine

It can than be run by storing the content into a py file and running the script by

python monitor_tokens.py
Sort:  

Wow you're all over the place lately, thanks for this tool. That's one of the things I'm worried about since you're practically able to create steem accounts without a trace and withdraw to a wallet only connected to an email so people should be careful with what coins they trust to buy. This might help make some things easier to spot.

I'm going to add a CLI extension for my steem engine python library, that will allow streaming and checking related trx_ids.

Posted using Partiko Android

Thank you very much @holger80 :)

I've got one question - what happens if the order is not fulfilled immediately? Because you wrote:

When there is already a matching buy order on the market

What happens if there's no matching order - how to track if the transaction was later completed? Would it be necessary to store all tx ids that we're looking for and check them occasionally to see if they were fulfilled?

It would be necessary to store all trx_ids of not completely fulfilled buy or sell orders and check once in a while.

Posted using Partiko Android

Thank you :)

When the order is not fullfilled immediately, only one transfer to the market is done. In this case,

len(logs["events"]) == 1

and the order is added to the buyBook or sellBook. When then a new ordre is placed which is able to fullfill it, this new order has more than 1 entry in events.

So, not fullfilled order are build up and the token are locked into the marked. When there is a sucessfully buy/sell, tokens are moved out the market. The movements can be tracked by checking all entries in log["events"]

Hi @holger80, thank you for working on many things.

I have a question. Is beem block stream function 100% reliable?

When I used official steem python libary's reliable_stream function, as opposed to its name, it actually misses some transaction from time to time.

ps. I asked this question before, but I didn't get the answer and then I also forgot (or gave up:) since I'm not using that function for critical purpose. But that was the main reason why I switched to beem :) But since I'm using it for non-critical purpose, I don't know wether it misses or not. But it seems much more reliable than the official one. Thanks!

ps. Not sure if it's already fixed, but steem-ua is leaving a comment twice for utopian posts, e.g., https://steemit.com/utopian-io/@blockchainstudio/steem-keychain-should-hide-private-keys-by-default and many others

Yes, there are several addional measures that garantee that the stream is reliable when using beem.

Thanks for your answer, btw, I made some suggestions and PR on beem as I already left on the beem post and GH.

The double utopian reply is fixed, thanks for saying it again.

Posted using Partiko Android

Neat. But I got this error when reading (sidechain) block 15850:

Traceback (most recent call last):
  File "monitor_tokens.py", line 32, in <module>
    payload =json.loads(trx_info["payload"])
TypeError: 'NoneType' object is not subscriptable

That error means that the trx_id is not known to the sidechain. I will check and fix.

Posted using Partiko Android

This post is supported by $41.61 @tipU upvote funded by @cardboard :)
@tipU voting service always profitable, instant upvotes | For investors.

Thank you so much for participating in the Partiko Delegation Plan Round 1! We really appreciate your support! As part of the delegation benefits, we just gave you a 3.00% upvote! Together, let’s change the world!

Hi @holger80!

Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your UA account score is currently 7.325 which ranks you at #64 across all Steem accounts.
Your rank has improved 1 places in the last three days (old rank 65).

In our last Algorithmic Curation Round, consisting of 182 contributions, your post is ranked at #165.

Evaluation of your UA score:
  • Your follower network is great!
  • The readers appreciate your great work!
  • Great user engagement! You rock!

Feel free to join our @steem-ua Discord server

Hi, @holger80!

You just got a 2.83% upvote from SteemPlus!
To get higher upvotes, earn more SteemPlus Points (SPP). On your Steemit wallet, check your SPP balance and click on "How to earn SPP?" to find out all the ways to earn.
If you're not using SteemPlus yet, please check our last posts in here to see the many ways in which SteemPlus can improve your Steem experience on Steemit and Busy.

This post has been included in the latest edition of SoS Daily News - a digest of all the latest news on the Steem blockchain.

Congratulations @holger80! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :

You made more than 800 comments. Your next target is to reach 900 comments.

Click here to view your Board
If you no longer want to receive notifications, reply to this comment with the word STOP

To support your work, I also upvoted your post!

Vote for @Steemitboard as a witness and get one more award and increased upvotes!