Steem Sincerity - Spam Filtering API and App Extensions

in #steemdev6 years ago (edited)



Steem Sincerity

This project will attempt to help the community address the spam problem by aiming to introduce the following:

  1. A fast API which provides detailed classification information about any Steem account.
  2. Chrome extension(s) for popular Steem web apps such as Condenser, which make this information easily available in these front-end interfaces, and allow the sorting and filtering of comments based on personalised criteria such as spam scores.
  3. Modifications to front-ends and (hopefully) apps to support this API directly.

Approach

A few months ago, I created an experimental account classifier. This uses machine learning and a small training set to determine patterns which identify comments as spam (or from bots). I would like to adapt this by increasing the training set of accounts, increasing the complexity of the feature set and making it performant enough to apply to the hundreds of thousands of Steem accounts in a scalable manner.

I'm aware that hiding spam does not directly stop it from bloating the blockchain, or being used for self-voting, which are significant concerns, but it should assist with content discovery, and by reducing the visibility of spam, somewhat undermine the economic incentive to produce it. In addition to this, it may reduce the number of comments which are upvoted and increase the number which are downvoted by providing instant insights into each account commenting on posts.

What's Available Now?

There is a minimally functional experimental endpoint at https://multi.tube/s/api/

This allows you to make HTTP GET calls like this:

https://multi.tube/s/api/get-classification-scores/account1,account2

...and which returns JSON responses like this:

{
    "account1": {
        "classification_bot_score": 0.285714285714286,
        "classification_human_score": 0.714285714285714,
        "classification_spammer_score": 0.0
    },
    "account2": {
        "classification_bot_score": 0.571428571428571,
        "classification_human_score": 0.428571428571429,
        "classification_spammer_score": 0.0
    }
} 

Up to 100 accounts can be queried at once, and the results may be easily used to modify/suppress comment rendering. If a requested account has not been classified, it will not be returned in the response.

This API is still using the same spam scoring formula as the Steem Reports accounts classifier, and this really needs to be improved and trained again to increase the accuracy of classification scores, but I think it is helpful in its current state, and serves as something to develop the other aspects of the project against.

When using this API to exclude spam, it is envisaged that we could also use followers information to augment the data. For example, somebody may want to allow accounts they already follow to evade the spam filtering process so they never miss something they would want to see, as no spam filter is perfect.

Training the Classifier Software

To improve the accuracy of the classifier I will need to collect three sets of account names:

  1. Spammers - generating significant spam, and little content of value
  2. Humans Content Creators - generating no (or very little) spam and valuable content
  3. Bots

I will announce a training date soon, and collect recent examples of each of these types of accounts. They need to be recent as the database will store a maximum of 14 days of comments.

Please let me know if you would be able to help by providing accounts in these categories when the time comes, so I can notify you.


I'm using my multi.tube domain/server in the short-term, but if the project gets enough support I'll give it its own.

Again, please let me know if you have any questions or would like to help with the project.

Sort:  

Thats a neat idea :) tipuvote! 0.1

I feel like we could be best friends!

I can definitely help with training and would like to see what other ways we could work together. I have a dataset of identified spammers as well as a whitelist of legitimate content creators, both of which continue to grow daily.

You can find me on steem.chat or catch me on Discord (duplibot#1884)

Great! I'll be in touch soon.

I really love the idea, But I want to ask how you intend to achieve this aim of checking for spams between thousands of messages from different users which is relevant to the post and at the same time has been used over and over again on the blockchain.
This might be a little bit difficult but I wish you luck.
💪

Initially this won't check for individual spam messages, but will evaluate accounts. So any comments from 'spammer' accounts would be hidden (or marked). It uses some machine learning approaches which are actually easier to apply than they are to explain - though it's not perfect.

I really like the idea of this. My hope is that spamming the Steem blockchain will be more effort than it's worth, and the spammers will move on to other platforms.

I checked my account and it says I'm human! So far so good!

interesting project. Looks a lot similar to what I tried with https://steemit.com/hello/@thefreebird/init-1

all the best.

PS: A query with invalid account generates internal server error. You may want to catch that exception :)

Yeah, thanks! I'm not always the most diligent when it comes to error handling.

happens to the best of us :)

Congratulations @andybets, this post is the forth most rewarded post (based on pending payouts) in the last 12 hours written by a User account holder (accounts that hold between 0.1 and 1.0 Mega Vests). The total number of posts by User account holders during this period was 3118 and the total pending payments to posts in this category was $6762.85. To see the full list of highest paid posts across all accounts categories, click here.

If you do not wish to receive these messages in future, please reply stop to this comment.

goodpost

GREAT! I wondered for a long time if Steemit would make it through all this spam. Your project gives me hope! Great work!