Post Promoter JavaScript Voting Bot - New Blacklist Options & Fix for Missed Transactions

in #utopian-io6 years ago

logo_small.png
New logo by the amazing @nateaguila!

Hello everyone! I’ve been really busy with some larger, long-term Steem projects but I’ve still made some time to make updates to the Post Promoter software here and there and thought it would be a good time to make a post about it - both to let the community know about the changes and to support Utopian while they recover from the recent attack.

The GitHub repo for the project can be found here: https://github.com/MattyIce/postpromoter

New Blacklist & Whitelist Options

In the never-ending war against spam/plagiarism/junk content on the Steem blockchain, it’s of vital importance that operators of content promotion services have the tools they need to prevent that type of content from being promoted without limiting the options of legitimate publishers.

With that in mind, I have beefed up the blacklisting options offered by the Post Promoter software and also added a whitelist option at the request of some members of the community.

All of the existing and new settings have been grouped together and moved into the “blacklist_settings” object in the config.json file as shown below. This change is backwards compatible so you don’t have to worry about breaking anything if you update your software before your config.

"blacklist_settings": {
    "flag_signal_accounts": ["spaminator", "cheetah", "steemcleaners", "mack-bot"],
    "blacklist_location": "blacklist.txt",
    "shared_blacklist_location": "http://sharedlist.com/list.txt",
    "whitelist_location": "whitelist.txt",
    "whitelist_only": false,
    "refund_blacklist": false,
    "blacklist_donation_account": "steemcleaners",
    "blacklisted_tags": ["nsfw"]
}

You may notice there are now a few new settings there. In addition to the existing “blacklist_location” setting, there is now a “shared_blacklist_location” setting. This allows you to both use a shared blacklist which can either be a local file or a URL and a bot-specific blacklist if you want to blacklist any accounts not included in the shared list.

There is also now a “whitelist_location” setting which allows you to specify a whitelist for your bot. The whitelist can serve a few different purposes. If you set the “whitelist_only” setting to “true”, then only accounts on the whitelist will be able to use the service.

If “whitelist_only” is set to “false” then the whitelist can be used to override accounts that are on the blacklist, if, for example, you’re using a shared blacklist maintained by a third party and want to still allow certain accounts on the list to use your service.

Separate max bid setting for whitelisted accounts

Now that there is a whitelist option, a new max bid setting has been added for whitelisted accounts called “max_bid_whitelist”. This allows you to set a smaller max_bid for the majority of users but allow certain pre-qualified users to purchase larger amounts of promotion.

"max_bid": 50,
"max_bid_whitelist": 200,

In the above example regular users may only bid up to as much as 50 SBD or STEEM (if accepted) but users on the whitelist may bid up to 200. Any bids that exceed the maximum allowed will be automatically refunded.

Fix for Missed Transactions

This is a pretty major change to how the software determines which transactions it has processed vs which are new. It was made to fix a bug that has been puzzling me for quite some time where certain transactions were just completely missed by the software. There were no errors, or log entries or anything, it’s as if the software never saw them at all.

To get the transactions, the software polls the account history for the bot account every 10 seconds using the getAccountHistory() function call. Each transaction returned by this call includes an account-specific index which starts at 1 for the first transaction ever on the account and increases sequentially with each new transaction.

So to identify which transactions the software has already processed vs which are new, it used to just save the index of the last transaction it saw and then on subsequent calls check for any transactions with an index greater than that.

I knew that since transactions were getting missed, something was wrong with that logic, and I was finally able to catch the issue as it happened. It turns out that, for whatever reason, sometimes transaction indexes can change between calls to getAccountHistory() as shown below:

[ 299611,
  { trx_id: 'db53d503c3a2804802a14c7aa7ea49e575e603e4',
    block: 22223537,
    trx_in_block: 20,
    op_in_trx: 0,
    virtual_op: 0,
    timestamp: '2018-05-07T14:20:15',
    op: [ 'transfer', [Object] ] } ]

[ 299616,
  { trx_id: 'db53d503c3a2804802a14c7aa7ea49e575e603e4',
    block: 22223537,
    trx_in_block: 20,
    op_in_trx: 0,
    virtual_op: 0,
    timestamp: '2018-05-07T14:20:15',
    op: [ 'transfer', [Object] ] } ]

What you are seeing in the snippet above is the same transaction record from two separate calls to the getAccountHistory() method. In the first call that transaction had index 299,611 and in the second call the same transaction had index 299,616.

In this case it would actually cause that transaction to be processed twice, which was also happening along with some transactions being missed.

So it was finally clear what was happening, and that I could no longer use the transaction index on the account as a bookmark for which transactions the software already processed and which were new. Instead I have updated the software to keep track of the “trx_id” for the last 50 or so transactions it has processed, and that way whenever it sees another transaction it can check if it has already processed it or if it’s new - regardless of the order in which they get returned by the getAccountHistory() call.

You can see the transaction IDs getting saved also in the state.json file so they can be restored if the software is ever stopped and restarted:

"transactions": [
    "cba86bcfbe1fa38bc9601026cc15b432187d5a03",
    "7f3e4d51b8396b049ccfb660c273d19b8f9a30b1",
    "ce94dfdf7ac1160aa6ffdb58b05460bdaf57c133",
    "e3dab712c97f3212a47e21a364299ded2706b092",
    "7be4b4800e0ab154423cb062e07cf13ce164dceb",
    "eaa915a273e1078bde886d83d07ceb99e37d6e4c",
    "5cd059662a18ffe148992df7cc6a04821ac239bd",
…

While it was somewhat rare, this issue seemed to happen in spurts and caused a lot of missed bids. I’m sorry it took so long but I’m happy to say that I believe it should be fixed for good now and hopefully we are closer to the goal of not missing a single transaction.

Thank you for your support!

As always I want to thank everyone who has helped and supported the development of this software, and to everyone who has submitted ideas, suggestions, bug reports, and contributions to the project!

Links to relevant commits:

Sort:  

Thanks for the contribution!

Some cool new features, and I really liked the way you explained everything! Keep it up!


Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]

Thank you @amosbastian! Great to see you guys keeping everything alive and well with Utopian while the issues are resolved!

Hey yapbapmatt,

As usual thanks up front for your continued efforts! The logo change is a lot more striking and certainly makes PP stand out more on the steambottracker.

I noticed the 50SBD limit recently and thought it was a great idea to prevent someone with a lot of SBD just consuming the entire vote.

Is PP currently running the whitelist model? If so do you have published criteria for how one would get access to it? I have found recently PP has been my go-to bid bot as I have a lot of experience with it and it just...well it just works, consistently.

Yes the @postpromoter account is using the whitelist model now. For the moment people can just contact me if they wish to be added to the whitelist and I will review.

There is no published criteria yet but it's just simply based on whether or not you are promoting good, original content that no one would reasonably have an issue with seeing on the trending page.

Okay great, nice and informal! Currently I don’t have more than 50 SBD to drop on a post so it’s not affecting me atm, I’ll drop you a message further down the line!

Great update. Only bot owners can fight against promoting spam on Steemit. This is good for everybody... Good job!

You got a 25.00% Upvote and Resteem from @ebargains, as well as upvotes from our curation trail followers!

If you are looking to earn a passive no hassle return on your Steem Power, delegate your SP to @ebargains by clicking on one of the ready to delegate links:
50SP | 100SP | 250SP | 500SP | 1000SP | 5000SP | Custom Amount

You will earn 80% of the voting service's earnings based on your delegated SP's prorated share of the service's SP pool daily! That is up to 38.5% APR! You can also undelegate at anytime.

We are also a very profitable curation trail leader on https://steemauto.com/. Follow @ebargains today and earn more on curation rewards!

@yabapmatt please review my profile for the whitelist category thanks i will be waiting for your response dear friend

Works like a charm. What other steem projects are you working on? can you you let us know?

wow good update. you are working very well

This comment has received a 0.15 % upvote from @speedvoter thanks to: @everything-4you.

glad you have fixed all the bugs and the new logo looks pretty cool...

I was first of the view that bots are really not needed. But with the blacklist options and such, they can work.
Great work my friend in reducing spam use of bots.

Thank you so much for continuing to update this code and add new features. Your work is greatly appreciated!

Thanks to guys like you, the platform will continue to be a good one. There are only two battle on steemit at the moment - reward pool manipulation by some accounts and the second is crap and copied content coming on the site. At least, we are addressing the second one.. :)

Hey @yabapmatt are you a witness..
You don't post much

You don't post much

I find this to be a humorous commentary on the state of Steem. I post about once a week which by most measures is quite often. Writing a good post takes a lot of time and you should only make a post when actually have something important or valuable to communicate. It's unfortunate that Steem encourages people to post multiple times / day just to try to earn rewards.

In English version We can discuss how we give SteemPower to the BOT.

En la versión en inglés podemos discutir cómo le damos SteemPower al BOT.

Como sabeis fair.zombie es un bot de apoyo a una comunidad que promociona la actividad constante, lo que es importante para Steemit y STEEM. Las pruebas han salido bien, siempre vota a todos, por orden de publicación y con el 100% de voto, no hay favoritismos ni arbitariedad, es por orden riguroso.

Necesitamos ser unos 20 miembros, de esta manera publicando cada 2 días obtenemos el voto equivalente de los 20 miembros. También hay que ver la manera de darle poder de voto (STEEMPOWER) delelgándole SteemPower o de alguna otra manera que se proponga. El bot puede alquilar SteemPower si tiene fondos.

Necesitamos más stemians. Si conoceis a alguien que quiera entrar en la comunidad, que lo diga y lo incluimos. Lo importante es que publique con frecuencia y con cierta calidad.

Se pueden leer las bases provisionales aquí

saludos
@rafestrape

Thank you for your support!
As always I want to thank everyone who has helped and supported the development of this software, and to everyone who has submitted ideas, suggestions, bug reports, and contributions to the project!

Thank you my friend for your great initiative, the supports aren’t possible if the primary idea from you isn’t there. Thanks all the way man. Thanks for the detailed update too

I just upvoted

Hey @yabapmatt
Thanks for contributing on Utopian.
We're already looking forward to your next contribution!

Contributing on Utopian
Learn how to contribute on our website or by watching this tutorial on Youtube.

Want to chat? Join us on Discord https://discord.gg/h52nFrV.

Vote for Utopian Witness!

Wow, how am I Just seeing this.. beautiful work..

Welldone

Good day Sir, please come to our rescue. Many steemians are crying about a bot not giving votes or refund and it looks like it is going to deaf ears. You are our last hope, please help.

https://steemit.com/steemit/@sneaky-ninja/2xuzdv-sneaky-ninja-weekly-update-blacklist-and-watch-list#@ajongcrypto/re-sneaky-ninja-2xuzdv-sneaky-ninja-weekly-update-blacklist-and-watch-list-20180509t123745647z

You got a 100.00% upvote from @upmewhale courtesy of @yabapmatt!

Earn 100% earning payout by delegating SP to @upmewhale. Visit http://www.upmewhale.com for details!

Congratulations @yabapmatt!
Your post was mentioned in the Steemit Hit Parade in the following category:

  • Pending payout - Ranked 4 with $ 517,24

It's a super bot ! I' ve just installed it on my server. How to add my bot on steembottracker ?

Hi sir @yabapmatt me send 10 Sbd for 2 time @postpromoter Becouse no my promblem Steem errore check plz Untitled.png

If you sent money and did not get an upvote, leave a msg on the postpromoter post and they will refund you.

Hi @yabapmatt,

promobot is showing -6%/none on steembottracker but i am looking at the vote value and it seems it is capped at around 6%.

@yabapmatt
Is it possible to make a report button on Steem Bot Tracker, when bots do not function as they should, are temporarly down or simply stopped upvoting doing no refunds and withdraw their SP like Speedvoter? I think this would be good, as many people lose money...

Yes, that is definitely a good idea...just not sure if i will be able to handle all the reports! For now best is just to contact me like you've done and I will look into the issue.

Ok thank you... so Speedvoter is an issue right now

Really nice. Good work

So to run it do I use JavaScript or Ubuntu