Steem Analyser is a post analyzer that allows a user to enter a post URL and be presented with a list of results relating to the analysis of that content. It uses a collection of machine learning APIs to analyze a user's post. This release provides the foundation for further integration with multiple machine learning providers and deeper integration into the steem blockchain.
Within this release I have created the following features:
- Ability to grab and display post
- Integration into DatumBox machine learning API's to grab and display results for adult content, spam, author gender, commercial detection, educational material & sentiment.
- Basic Styling
A screenshot of the overall experience which has currently been created can be seen below:
What & How was this created?
The first thing that I created within this release was the ability to grab a post and display it, this can be broken down into a few different parts such as:
- Ajax Request to grab content
- Define & Update
- Convert Body from Markdown to HTML
- Display Result & CSS
Firstly I created the following function which is used to grab the users post from the steem blockchain, in order to make a request I used the getContent call which requires you to pass in both author & permalink. In order to get the author a permalink I used the following regular expressions:
- Identify Permalink from URL: ()?([a-zA-Z0-9-]+)/?$/igm
- Identify Author from URL: /@[a-z][a-z0-9-]+[a-z0-9]/igm
For the request mentioned above, I also need to strip the '@' symbol from the match found with the above regular expression for the author. I did this by turning it into a string and then replacing the '@' with ''.
Following this, I then defined the items I need to display within my results and used them to create the HTML in index.html.
When grabbing the body of a post from the API it is displayed in markdown, in order to display this on my page correctly I needed to convert this to HTML, I did this using Remarkable.js using the following code:
var md = new Remarkable();
var body = md.render(body);
The function I used to grab the posts and generate the HTML for the post can be seen here:
Following that, I created multiple integrations into a machine learning provider called Datum Box (http://www.datumbox.com/machine-learning-api/). Within this release I created an integration for the following:
- Sentiment
- Commerical Detection
- Spam Detection
- Author Gender Detection
- Educational Detection
- Adult Content Detection
The method for each of these are the same, I simply passed the body of the post grabbed in getPosts into the function and was able to generate a result which I was able to pass into the HTML and display as a result for that post.
An example of one of the functions which are used to grab and create results for these integrations can be seen in the below screenshot:
A full list of the functions to generate the results from Databox are listed below:
https://github.com/tobias-g1/steem-post-insight/blob/master/js/adult.js
https://github.com/tobias-g1/steem-post-insight/blob/master/js/commerical.js
https://github.com/tobias-g1/steem-post-insight/blob/master/js/education.js
https://github.com/tobias-g1/steem-post-insight/blob/master/js/gender.js
https://github.com/tobias-g1/steem-post-insight/blob/master/js/sentiment.js
https://github.com/tobias-g1/steem-post-insight/blob/master/js/spam.js
Note: I'm aware that I could have done these in single function if I desired, however in future releases I believe I will begin to substitute some of the above for providers who are able to provide more detailed results.
Finally, I applied some basic styling to the page, I used a gridview in order to divide the page. This is a 12 column grid view provided by bootstrap, I do not use any other bootstrap styling within the project. I introduced some of the issues on reduced width windows/mobile when making adjustments to the overflow for the post columns vs the results and will resolve in a later release where I will design and implement a new responsive style.
Commits
The following commits are related to this release:
(Ordered Newest or Oldest)
https://github.com/tobias-g1/steem-post-insight/commit/8f2acf6de2ac92b99e72efa6cdaf53ef068eb5aa
https://github.com/tobias-g1/steem-post-insight/commit/0951578c730ea6e42c6424b4bebc3091366ec20a
https://github.com/tobias-g1/steem-post-insight/commit/a954469b7a571d51bb8713b9a2b01d8b5102e395
Current Limitations
- Limited Validation on post URL field
- Errors on reduced width windows & mobiles
- Limited Integrations & Limitations in detail of results returned from API
- Spelling Errors & Badly written copy
Roadmap
- Mobile Ready Design
- Animation and Custom CSS for results sets
- Deeper integrations with other opensource machine learning providers for things such as keyword/entity detection/extraction, details sentiment analysis using a more relevant data set, spelling and grammar.
- Ability to share results on steemit
- General Clean Up
Posted on Utopian.io - Rewarding Open Source Contributors
Thank you for the contribution. It has been approved.
Great post with a lot of possibilities.
Running the
index.htmlas suggested got me errors:Origin null is not allowed by Access-Control-Allow-Origin.So you have to start your browser with the--allow-file-access-from-filesfor it to work properly.I also got some interesting results whiles testing it
While the first two are ok verdict, the last one is not! Might there be a language bias?
You can contact us on Discord.
[utopian-moderator]
Your Post Has Been Featured on @Resteemable!
Feature any Steemit post using resteemit.com!
How It Works:
1. Take Any Steemit URL
2. Erase
https://3. Type
reGet Featured Instantly � Featured Posts are voted every 2.4hrs
Join the Curation Team Here | Vote Resteemable for Witness
Hey @tobias-g I am @utopian-io. I have just upvoted you!
Achievements
Community-Driven Witness!
I am the first and only Steem Community-Driven Witness. Participate on Discord. Lets GROW TOGETHER!
Up-vote this comment to grow my power and help Open Source contributions like this one. Want to chat? Join me on Discord https://discord.gg/Pc8HG9x
to @utopian-io
you should be proud of and worth the thumbs up with the achievements you achieve .. I support you and when you succeed .. marry to shareian-io
you should be proud of and worth the thumbs up with the achievements you achieve .. I support you and when you succeed .. marry to share
actually it is an opportunity to enter the contest .. but unfortunately my ability is still very weak. i need real help if you are willing ... thanks
what do you mean you need help?