RE: Ingestron 0.0.2 released for every platform imaginable

[faddat]

Maybe?

Tell me more of what you'd like?

I haven't written a streaming API yet in go. Do you know if libs exist for this?

Put it in user-story format?

Comments

[alexpmorris]

this is the way the current steem api works, which is highly inefficient. Just "inspect" the market in Chrome and you'll see what I mean:

SteemApi exec 5 get_following ( Array[4] )
app.56d5f0d….js:5 SteemApi exec 6 get_ticker ( Array[0] )
app.56d5f0d….js:5 SteemApi exec 5 get_following ( Array[4] )
app.56d5f0d….js:5 SteemApi exec 0 get_order_book ( Array[1] )
app.56d5f0d….js:5 SteemApi exec 6 get_trade_history ( Array[3] )
app.56d5f0d….js:5 SteemApi exec 6 get_ticker ( Array[0] )
app.56d5f0d….js:5 SteemApi exec 0 get_order_book ( Array[1] )
app.56d5f0d….js:5 SteemApi exec 6 get_trade_history ( Array[3] )
app.56d5f0d….js:5 SteemApi exec 6 get_ticker ( Array[0] )
app.56d5f0d….js:5 SteemApi exec 0 get_order_book ( Array[1] )
app.56d5f0d….js:5 SteemApi exec 6 get_trade_history ( Array[3] )
app.56d5f0d….js:5 SteemApi exec 6 get_ticker ( Array[0] )
app.56d5f0d….js:5 SteemApi exec 0 get_order_book ( Array[1] )
app.56d5f0d….js:5 SteemApi exec 6 get_trade_history ( Array[3] )
app.56d5f0d….js:5 SteemApi exec 6 get_ticker ( Array[0] )

The way it should be is more like a pub/sub, ie. sub_order_book(10); // subscribe to receive and update 10 top levels

Instead of repeatedly requesting updates, it feeds them to you automatically as they come in. If it helps, here are a bunch of golang examples:

https://www.google.com/search?q=golang+pubsub+websocket

I guess the real question is can the current websocket implementations already do any of this natively, as opposed to building a new API that polls, which then in turn converts that to a pub/sub style API. (ie. caveat: it's possible I don't fully understand all the features of the current API)

[faddat]

Does it need to use a websocket or can pubsub be achieved by other means?

Pls know I'm a relative virgin with anything involving javascript, and prefer dart or gopherjs due to static typing heaven.

[alexpmorris]

websocket is just a layer over TCP, and you need it if you want a browser to interact with your API. It's how the current steemjs talks, which ironically means it should be able to stream pub/subs already. I would just imagine the front-end would already be using though if it were available. I built a websocket wrapper for delphi a while back. Maybe it'll help you understand better what it is: https://github.com/alexpmorris/WebSocketUpgrade

[faddat]

I know what websocket is, I just prefer to go with standards, and I know websocket is a standard, but.... have a look at this:

https://github.com/jcuga/golongpoll

Let me know. I can do this or websockets, or eventually both. I'm building this to get used, so the thing that is easiest for yall building front ends, is the thing that I'd like to include in ingestron.

[faddat]

Any idea what http long-polling is?

Generally I'd prefer to use http/2 instead of websockets, since well, ugh websockets.

[alexpmorris]

if you're gonna bother doing this, you're really looking at a websockets / tcp / json / protobuf type solution. long polling basically means you poll with a complete HTTP call each time, except without closing the connection. Stay away. websockets are cake for golang, dart, etc. though. It's practically native. what I did for delphi would best be described as torturous by comparison!

https://github.com/gorilla/websocket

[faddat]

There are a number of libs for websockets in go. I will start looking into them and choose one or another.

I'm also totally interested in any excuse to use grpc-web, cause..... I mean.... it's grpc-web, eh?

[alexpmorris]

grpc-web is almost exactly what I just described to you, though ironically it could add more complexity (because it's an extra framework that imposes the framework's way of doing things on you), but it should probably do everything we just talked about as well lol

http://www.grpc.io/blog/pubsub

or maybe not quite, since I just found this too:
https://github.com/tmc/grpc-websocket-proxy

[faddat]

Well, I've surely been looking for an excuse to put it to work. Are you also in the beta?

[alexpmorris]

the beta of grpc? nah, just looked it up after you mentioned. I tend to build my own stuff from lower-level pieces (like the websocket interface on my github, that links directly to zlib to handle inflate/deflate, etc). Probably a bit too low level for most people's taste. lol

But it does give you a certain level of control over some things that you may lose otherwise. As always, depends on the usage case and what you're trying to accomplish.

[faddat]

Goal:

Make it absolutley, utterly, 100% easy AF for people to build front end systems that connect to the steem blockchain, whose github repository I am banned from

....cause.... "reasons!"

[officialfuzzy]

Cause "promises" is a better way to say it.

[faddat]

@originalfuzzy are you saying I am banned from steemit's github because I haven't been able to finish what you hired me to do?

Or are you just making idle conjecture?

....and why do you think ingestron exists?

It exists because the current interfaces are hostile to the creation of applications like the one you want built. So I built a new API for steem.