So my buddy @btcmillionaire is super excited about the upcoming release of @Zappl, and earlier today he linked me a youtube livestream where one of the creators gave a quick run-through of the upcoming social app that's being built on the Steem blockchain.
As the demo unfolded however, a really interesting question arose around "app filtering" - specifically, how apps that post to, and read from, the same content database will maintain their own "content ecosystem" (my phrase), to prevent content created in one app from polluting the content viewed in another.
And with more apps in the works, it's a really important question, which I thought was worth having a discussion about
Quick Disclaimer: I've got a tech/compsci background but have not yet taken the time to sit down and really understand how Steem or blockchains in general work at a technical level. In writing this post I have done some reading around the proposed methods for app filtering, and taken a quick look at how this works at a technical level, but I may have some major lack of understanding that makes all my assumptions wrong. Hopefully I've got it right, but if I haven't then I'm quite confident you friendly Steemians will set me straight.
Challenge: How do we stop Steemit posts showing up in Zappl and vise-versa?
It was about 20 minutes into the demo that I heard the presenter raise the issue of filtering content within Zappl, and at this point I began to realise that because of the way apps store their content in the blockchain, whereby apps are essentially saving all of their data in the same database, there's an inherent challenge in ensuring that content from one app doesn't spill over into the other.
First of all, let's have a look at what @Thedegensloth had to say:
t=19m55s
(in response to user question)
"Umm, Zappl is more of a standalone... right now why you see the content on the site is because we're pulling data from the database because it's the only data that we have to use. But, after launch we will only be using Zappl data, so if it's posted on Zappl it'll show on Zappl and it'll post on every other site that pulls the database, but if you post on Steemit it will not show on Zappl"
And then a bit later he discusses it more.
t=21m38s
"Q: Have I had a conversation with Steemit team around how they will be filtering out zaps on their side."
"A: I'm not sure yet. I have mentioned it to a couple of developers but as for the steemit team, the front end developers, uhh, not yet.
I will have to have a chat with them of how... I just wanted to see how it would go on launch, and if it became a problem, well then now we know which... what we have to do, that app filtering is gonna have to be a thing.
But I feel that app filtering is going to have to be a thing, no matter what, because the more apps that we want to come on the network, well they're gonna want to post a lot too, and it's gonna just flood. So, sites like Steemit that are, I guess, more... long-form friendly, are gonna get just bombarded by short-form posts."
So there's two key take-aways here:
- Zappl will filter content so that only content posted from Zappl will appear in Zappl. And according to this comment from the team at Steepshot, they'll also be doing this
- Steemit.com does not currently filter content. If it's posted on the blockchain, it shows up on Steemit - this will include posts from other apps posting to the blockchain.
The first one makes a lot of sense, but the second one seems a little bit odd. After all, if we're going to keep bringing new apps onto the Steem blockchain, then Steemit.com is going to be flooded as those apps gain popularity, and will eventually become unusable.
But it starts to be a little less odd once you realise what Zappl and Steepshot are doing at the technical level to implement this app filtering.
So how does it work at the technical level?
Without going full nerd, every post and comment on the Steem blockchain is accompanied by a set of JSON metadata which gives you a little bit of extra information about the post itself. And, while at a technical level you could stick whatever parameters you want into this metadata set, in practice its usage is dictated by an open standard called Steem Script, put forward by @fabien in this post back in Nov 2016.
For comments, the Standard defines the metadata parameters with the following example:
{
"app": "steemit/1.23",
"format": "html",
"tags": ["steemit", "steem"],
"users": ["ned", "dan"],
"images": ["https://img.busy6.com/@ned"],
"videos": ["https://www.youtube.com/watch?v=rkQ7b-u8_6g", "https://www.youtube.com/watch?v=H399YZ0pv0o"],
"status": "archived",
"canonical": "http://blog.steem.io/steem/@ned/the-first-phase-of-the-steem-faq-and-wikee-consolidation-of-knowledge"
}
Now, I believe what @Zappl and @Steepshot will be doing to filter content for their apps is focusing on that "app" parameter. My understanding is that if the app parameter for a post/comment doesn't contain "zappl" or "steepshot", it won't appear in the relevant app. Seems pretty straight forward and logical, really.
But there's a bit of a problem here, in that the Steem Script standard doesn't actually have any usage guidelines for how the parameters in the comment JSON metadata should be used, and so while Zappl/Steepshot are looking to use the "app" parameter to filter content, its historic use has been exclusively for identifying the user agent that was used to post to the Steem blockchain.
To reiterate - the parameter these apps are most likely to use use to filter content tells which app the content came from, not which app the content is for. And that's sort of an important difference.
Why does this matter?
There are three key scenarios where I believe this sort of matters:
1. When developing 3rd party apps to interact with other apps' ecosystems
Starting with what is possibly the least impactful scenario, if the app parameter is used to identify the agent that is posting to the Steem blockchain, and 3rd party app creators adhere to this convention, then 3rd party apps will never be able to post into the ecosystem of apps like zappl and steepshot.
In otherwords, you will only ever be able to post to zappl from zappl.
Makes sense, right? Well.. not really. When I post an image on Instagram, I usually hit that share button and get it to send out to Facebook as well. In fact, many attribute this image sharing as one of the primary reasons Instagram became so big in the first place.
In fact, in this scenario, if the actual real Instagram app wanted to add a "Zappl" option to its sharing, it wouldn't be able to - because, if they followed convention, they would identify themselves as "instagram" in the blockchain's app parameter, and their posts would be filtered out of the Zappl ecosystem.
2. It creates a silo'd content ecosystem
The second concern, and it's very similar to the first, is that when we start filtering content in this way, we're beginning to create a silo'd ecosystem where content will only ever be able to appear in one app.
Now there are very solid reasons why Zappl would not want photos posted from Steepshot to Steepshot for Steepshot users showing up in the Zappl feed, but there's also lots of use cases where it makes sense for content to appear in multiple apps.
For example, outside of Steem it's common for people to post the same blog post on LinkedIn and then post it again on Medium. Particularly when we're attaching value to upvotes and sharing, and particularly when your followers across multiple apps are likely to be the same, it makes sense for this to be a single post to the Steem blockchain, rather than two posts.
However, if we're filtering based on the "app" parameter, a single-value field representing the app the content was posted from, then you'd need to post the same article twice, from two different apps, to make sure you reached the followers that tend to use one app over the other.
3. Alternative clients - the challenge of filtering Steemit.com
Lastly, we need to talk about what happens when you post to Steemit using a third party client like eSteem, whose "app" field looks something like this: "esteem/1.4.6".
It's not an issue right now, because Steemit just shows us whatever gets posted to the blockchain. But, if we get put in a position where we need to start filtering out micro-content (eg, a tweet, or a photo with a caption), then filtering on the "app" field to only include posts from "steemit" is going to make all the esteem content disappear.
So sure, Steemit.com can just filter on "steemit" and "esteem", problem solved.
But what if I want to create my own Steemit client, Rossteem? Do I have to convince the Steemit.com devs to add me to their filter? And the esteem developers too? What a pain!
Solution: We add an "ecosystem" parameter to the Steem Script Standard
The better solution seems, to me, to amend the Steem Script standard to include an "ecosystem" multi-value parameter which allows you to specify which app content ecosystem your post should appear in, with usage guidelines that state 'Steemit' will be the default ecosystem where no tag is supplied.
By doing it this way:
- All existing content will be preserved as is in the Steem ecosystem, without any need to modify the Steem blockchain
- Future apps can specify their own ecosystem attributes when posting from that app, and even allow sharing into other ecosystems
- 3rd party apps can add multiple ecosystems within their posting options
- Apps built on the Steem blockchain will have a standard convention for filtering content - at the moment they all could do it a different way. (eg, one could filter on "app", one could ignore the standard and use its own parameter fields)
I might also suggest a further optional parameter along the lines of "appkey" - if a post to a content ecosystem doesn't feature an approved public key, the post doesn't get included in the native app. This not only adds some security around the app filtering (currently, someone could fake posting from your app) but it also would give you the ability to approve third party apps to post into your ecosystem.
I have posted this suggestion as an issue to the github page for Steem Script.
Did I get it right? What do you think?
As mentioned at the start of this post, this is all based on my admittedly limited understanding of how the Steem blockchain works, and how I believe the upcoming apps plan to filter content within their ecosystems. Hopefully there's no fundamental misunderstandings that negate everything I've had to say - I actually feel like I learnt a fair bit about how it all fits together during the process of writing this post :)
Would be keen to hear what people think of my proposed solution, if they have any of their own, and whether they can offer any additional insight into the challenges of app filtering as we begin to see more and more apps come online.
Either way, it seems to me that now is the time that we need to do something. If we don't come up with a solution soon, app makers are going to have to find their own solutions, and by the time a standard approach is decided upon, we're going to have to reverse engineer something or otherwise find a way to integrate posts made by apps before the standard was agreed upon..
Great post! It would be cool to hear from @zappl on it too!
Me too! I think the real risk is that until there's a standard way to do app-based content filtering, app creators will all just develop their own approach... and by the time there's a standard, it'll sort of be too late. Definitely a countdown on this one
Yeah that is true. Sounds like there is a bit of a team behind @zappl and I am sure the other apps too.
Really looking forward to the launch in August and trying it out!
This post received a 2.4% upvote from @randowhale thanks to @rossdcurrie! For more information, click here!
Worth it!
Thanks for your deep dive!
"showing 3rd party app content in Steemit"
Did you heard smth about community namespaces feature as it mentioned in Steemit roadmap 2017? It is planned to be released soon (see my old visual guide to this roadmap https://steemit.com/marketing/@vitality/marketing-steemit-2017-roadmap-visually-explained).
This feature will enable a separate pools for the 3rd party apps content with ability to filtering by tag inside the pool.
It can solve the problem of content visibility.
"multi-value parameter which allows you to specify which app content ecosystem your post should appear in"
Don
t see it reasonable exactly as it is mentioned. You cant forbid apps to show smth or not from the blockchain._
Is it more clear now?
Well, by the way almost everyone here in steemit is considering how it would be realised. We`ll post some explanations from @steepshot team soon. And waiting for updates from Steemit!
That roadmap is really useful!
Haha, damn - wish I'd seen the roadmap and read up on the planned communities functionality before I posted
Interesting! My first read of the Roadmap makes me think it's more along the lines of a Facebook Group than something that caters well to apps, though I can see how you'd use it in that way. I suppose we'll wait and see how those features pan out.
Yeah, my thinking here was that as Steepshot you have no real control over what goes into the blockchain, but you do have the ability to control what gets shown in your app. So, if you wanted to prevent "just anyone" from posting into your ecosystem, you could filter based on appKey.
Anyone could post to the blockchain, but only the posts that provided a valid key would display in the app.
Eg, I've seen Zappl mention having a 250char limit on posts - well, if they were filtering on the "app" parameter, what would stop me just from posting longer content into the blockchain with a faked app parameter? How would their app handle that?
Yeah, I think that's where the real risk lies - until Steemit announce something, we're all left to come up with our own method to resolve it. Right now is a good time for a standard to be established - if we wait until there are more apps posting to the blockchain, then it's potentially too late.
Yes, app key exist. It's implemented already in Steemit platform. The problem is still to handle with the blockchain filtering.
Many devs are aware of what Steemit plan to do in future. There is one more huge step revealed in the roadmap is planned to come at the end of 2017/2018- "Fabric architecture - multi-chain parallelism"...
What do you mean by "app key"? Looking on Steemd for this post, the only reference to app is the "app" parameter in the JSON metadata that I discuss in my post.
Ayy thank you for this info as I am building a large ecosystem as one of my longer goals. This will sure come in handy thanks buddy :)