Anyx, Cheetah, and Steemcleaners: witness update -- Running into limits!

in #witness6 years ago

IMAG1006.jpg
It's not just programs that crash! Photo by me, a few weeks ago in Vancouver. At least the bus is sorry?


Cheetah

Fun fact: Did you know Cheetah is approaching $300/day in operational costs? Fortunately, this is currently not a limit (due to witness pay and SBD price being high) but it's something to keep in mind.

During the Steem witness forum hosted by @aggroed, I briefly talked about current issues facing Cheetah. I'll outline the two biggest ones here.

The first issue is a bug I discovered that is an artifact of the current blockchain state design. I discovered this bug as when Cheetah was voting extremely often, votes would fail despite my calculation putting the vote weight at much higher than the dust limit.

After digging into the problem to find out why, and some discussion with @vandeberg, we discovered that there's currently a minimum voting power consumed that is used to make a vote due to precision. This is a bit of a bug, as the limit is artificial -- regardless of your SP, there is a maximum number of votes you can issue per day. This is counter-intuitive to the idea that your influence is based upon your SP, not the number of accounts you own. A GitHub issue was opened here, and you can see some of the more technical details of the issue.

The second issue I am running into is the 20 second comment limit. There are 86,400 seconds per day, which means you can comment only 4,320 times per day maximum. Guess what? Cheetah has hit that limit on popular days!

But the bigger issue is that transactions are somewhat "bursty". For example, during prime time USA there are a lot more transactions than at midnight. This imbalance allows her to "catch up" during the slow times, but she becomes behind during prime time. This can cause problems with external services that watch for a Cheetah downvote -- for example, a voting bot may refuse to vote on a post that was downvoted by Cheetah -- however, the downvote will not arrive on the post quick enough for the service to see it, and the bad actor can circumvent the services rule.

There is some support for removing the 20 second comment limit: a GitHub issue has been opened here. I am completely in favour, as it also goes against the idea that your influence should be based upon your SP, not the number of accounts you own.

Now, if you're a developer the obvious solution that you will tell me to use is to simply use multiple accounts. I know. But this isn't the point: your actions on the blockchain should be limited by your SP (e.g. by the bandwidth assigned to you)! I am repeating this over and over because it is fundamental to Steem's design. In the same way splitting your SP across two different accounts doesn't give you any advantage to voting, neither should commenting or voting from two different accounts. The goal is to make a Sybil attack ineffective. At the moment however, these two issues have yet to be addressed. If the popularity of Steem continues to rise, I will indeed be making multiple Cheetah accounts to circumvent these artificial limits, sadly...

I will be making an announcement if the move to multiple Cheetah accounts happens, so that service operators know what to expect if they are looking for specific Cheetah actions.


Steemcleaners

For the most part recently I have been working in an administrative and development role for Steemcleaners -- we now have several full time members working hard in the daily operational aspects.

In terms of development, I have built several bots and tools over the past few months in conjunction with @patrice, which you have probably already heard of if you follow the Steemcleaners posts. We have new tools to mark spammers, tools to precisely remove rewards without over-voting, and more!

We have also recently hired a Web+DB Developer to redesign our website, steemcleaners.org. With the SBD price being beneficial to us, we have had the funding to order some cool stuff, so I am very much looking forward to this. We have in the pipeline a much more automated procedure with user roles: we will be able to incorporate more helpers into our ecosystem, and give them controlled access to certain operations. For example, helpers can have assigned roles to be on the lookout for identity theft, and others to look for plagiarism. After issuing their finds to our website, it can be validated by a senior member and some actions can be taken automatically (such as nullifying rewards on the post).

This will greatly increase the scalability of Steemcleaners, and we will be on-boarding a lot more helpers. If you're interested in becoming a helper in the future (remember, it pays in steem/SBD!) feel free to reach out to myself or @patrice. We hope to see more curation groups working directly with us.


SBD Peg discussion

There has been a lot of discussion on this lately, so I'll take a moment to chime in. A TL;DR of the SBD peg issue is: Although SBD has a price "floor", being supported by conversion of SBD->Steem, SBD currently does not have a price "ceiling". This has allowed this run up of SBD valuation to go unchecked by active forces to push it back to the intended valuation. There are a lot of good arguments for adding a price "ceiling" (for example, by allowing Steem->SBD conversions), but there are also good arguments for allowing the SBD price to be free market determined (for example, the huge payouts we are seeing is incentivizing people to join).

I haven't weighed in on a lot of the discussion on the SBD pegging issues. Honestly it is mostly because I am on the fence. I have an interest to wanting the SBD price high (as it is leading to excellent funding for Steemcleaners, for example), but I also have an interest in having a stable token maintained by a peg, mostly for adoption an merchant reasons. I keep coming back to the argument that if we wanted a token to speculate value on, it should be steem: if we have two speculative tokens it's just more confusing for users. If SBD is left as an unpegged asset like Steem, we should just move to only using one token for simplicity. So I think if there is a movement to include enforcement of the SBD peg (e.g. with steem->SBD conversions) I would be in favour.


Scaling Steem Resources

Full RPC nodes hit the 256 GB barrier recently, and apparently a few had issues. Although I'm a little concerned in the short term for the health of having public RPC nodes, I am not overly concerned in the long run simply due to how many optimizations can be made to the Steem code. I work with HPC (High Performance Computing) systems, and the size of the Steem blockchain and state is still incredibly tiny compared to what I am used to, especially considering how un-optimized it is. After my PhD I might find myself working on scalability in the Steem and blockchain world, as it's pretty clear to me that basically no one here has any idea what they're doing when it comes to performance or scalability. (While it might sound pompous of me to say that... well, it's my field of PhD-candidate-level expertise.)

Anyways, a few times I have seen the idea that witnesses should offer both a seed node and a full RPC node for public usage. While I think this is a great goal (to have a large amount of RPC nodes available to aid developers), I don't have one made public at the moment. I have actually been running my own RPC node since I started as a witness, but it is private: the main reason for this is because my own programs already absolutely hammer it -- especially Cheetah. Opening this one to the public would open DoS attack vectors on Cheetah (which I do not want), so if I were to offer an RPC node publically I would have to launch a second one.

I would be curious to see if developers are interested in having another RPC node available, or if they believe there are currently enough to service their needs. So please let me know in the comments if that is something that would interest you, and let me know what you think of the current state of RPC nodes!



Like what I'm doing for Steem? If you want to see development of Cheetah and Steemcleaners continue, please vote for me as a witness here!

Sort:  
There are 2 pages
Pages

hey man,

pardon the off-topic comment…

I made a post today regarding a large-scale idea to advance Steem’s development, am aiming to get this in front of the audience who’d be in the position to do something with/about it, and it was recommended to share with the witnesses (hence, this share):


The $1 Billion Steem Development Fund: How Steemit Inc.'s Stake Could Be Best Allocated To Grow A Thriving Network Of Applications And Users...

would be cool if you could have a read, and IF you feel it’d be a great idea that’d serve the community, forward to anyone in particular you know who might be in a position of influence to advance the discussion.

either way, I thank you for your continued service to this community. 🙏

cheers,

Rok

you are the genious man. that's why i done my witness vote for you.

mmm, thanks. however, I'm not running for witness.

Loading...

Anything to keep steem blockchain safe, I am for it.
Anything to diminish abusers, I am for it.
Good job and your report is well versed.
Keep on steemin'

Hi, @anyx

You are taking very Importent step
but you have to take step to unusualy flager.

Appreciate the hard work @anyx, especially the support for initiatives that help clean up spam, plagiarism, etc. Having been part of many rewards-for-content sites in the past 20 years... this sort of stuff is the true "slow killer" of sites like Steemit, especially if it is allowed to run unchecked.

I would love to eventually see some sort of "easy reporting" feature for the average daily user to "ping" cheetah/steemcleaners/spaminator when you run across an obvious issue just while reading. I think there are lots of people who'd be willing to lend an eye/hand with reporting issues... maybe through a browser extension, or some kind of app.

Thanks for the support! We're actually working on the latter -- part of the reason we're bringing in a web dev :)

While not a browser extension there is the possibility of reporting to Steemcleaners here.

if I agree with you @denmarkguy, the work of cleaning plagiarism, spam, etc. it's a bit tedious, and really serious, well, it's a trip of some kind of application or something like that. @anyx

First of all, congratulations on your great work @anyx! Both when it comes to personal efforts and financial invest I think you and your team are definitely doing an exceptional job on this platform. Chapeau!

I've got a question regarding plagiarism, identity theft and copyright abuse: do you believe that a better education of new users could contribute to reduce steemcleaner's operational workload? Or in other words: do you generally believe that new users that join the platform could be better informed about the general rules in terms of content creation and distribution?

I figured maybe from the users' reactions on your warnings and advices you may deduce if there's an actual lack of briefings/instructions/tutorials.

In advance thanks for getting back :-)

Thanks! :)
I definitely believe plagiarism and copyright are things that can be taught and learned as new members get onboarded, and I think this should be better discussed with new members. (Identity theft doesn't really have an excuse -- that's an active con, no way it's a misunderstanding.)

We have to remember that not everyone here will have an education from a first world country, and they may not understand things like copyright. Education for these people would be ideal.

How to better educate: I'm not sure. We try our best with steemcleaners, but we're putting out fires rather than preventing them.

Thanks for getting back so quickly!
I supposed that you'd answer something like that. Even though I'm not behind them on a daily basis, my experience has been very similar to yours.

Let's keep this as a priority and promote education whenever we can.
Steem on!

Good job!

the 20 second comment limit is a pain even when I reply because sometimes I type too fast and at times I use voice type too. I have seen the issue raise to remove that which I am certainly all in favour😎

No doubt! It seems to be most annoying to regular users. Anyone who wants to spam is just going to create multiple accounts anyways. It should be limited via bandwidth (as a function of SP), not accounts.

That is true but unless the spam is to outside of steemit more likely the spam comes from one account at a time.

My son @exyle wrote about you this morning, so I went to your blog. Thanks for the great work you are doing for Steem. I will vote for you as a witness.

I just checked that out vlog, very cool! :)

Man, I remember when you created Cheetah. Unbelievable (in a really good way) that she is hitting comment limits per day! Pretty massive amount of growth on here in the past year, great to see someone as skilled as you starting to look into scalability :) Keep up the amazing work as always!!

I hadn't realized there were so many limitations with the current system. I was going to suggest splitting @cheetah into two bots until you talked about how the comments per day should be based on total SP. That makes sense as the people who are "higher level" should have more right to post when they want.

How can I help with Steemcleaners?

Right now, you can report abuse to us here. The more advanced helping will be coming shortly, stay tuned! :)

Mate, max respect to what you are doing here. I don't think anyone is doing more work to curb spam and plagiarism than you and your team.

If you ever need any support in this area let me know. I'm happy to get involved in whatever capacity I can.

did not know that you were after these wonderful projects very useful to keep in order the platform dear friend @anyx congratulations for the great work you are doing on the platform
Here you have my vote as a witness
I wish you a wonderful week

Your service is great, much needed and should be supported.

I would be curious to see if developers are interested in having another RPC node available, or if they believe there are currently enough to service their needs.

I'm interested to know the answer to this. I haven't seen anyone complain yet but there's so much happening it's hard to keep up. Hopefully we'll be able to catch this fairly quickly if that happen.

What do you think of designating a portion of the inflation to abuse mitigation similar to how witness pay and sp interest is paid?

The witnesses can decide how much, and to whom, should be paid out from the pool that should be allowed to build.

In this way all users of the platform share in the costs of abuse mitigation and not just the active authors as happens when upvotes are used to pay wages for work other than posting.

I would suggest some kind of white list for the 20 second comment rule, or maybe dropping it to 15 seconds.
Eliminating it will result in a spam attack from hell, imo.
Imagine how much more grumpy cat could selfvote.
And can we bring back the 4 post soft limit?

I would suggest some kind of white list for the 20 second comment rule, or maybe dropping it to 15 seconds.
Eliminating it will result in a spam attack from hell, imo.

Bad actors have already circumvented this in the past with multiple accounts. Many of those accounts using SP delegated from the faucet.

Ouch, maybe we should bring back the n2 and teach the minnows to dogpile abusers?

Maybe with a sigmoid function that keeps the whales from sucking it all up.

Having it as part of inflation is one option (and similar to "workers" from other coins like BTS) and I think it could be a good idea.

Eliminating it will result in a spam attack from hell, imo.

This is only true if the bandwidth algorithm remains broken. Imagine a newbie who spams 1000 comments a day. They can either do it once every 20 seconds, or they can do it all in one go: but then they are bandwidth limited and have to wait until the next day. End of the day, same total spam amount.

And can we bring back the 4 post soft limit?

I advocated for removal of this, but I wouldn't mind bringing a variant back in if it were somehow made intuitive.

I wasn't in favor of the 4 post limit. That was a little restrictive. It would be nice to see some variant that would stop the spam bot posters without having to flag them. As it is now, I'm not sure if I've stopped them or if they've just moved on to multiple accounts to do their spamming.

Ok, good to know.
Thanks for all your work.

about sbd peg, am also in favour of putting a ceiling to try to peg it, like you said it would be kind of confusing floating two speculative currencies.

Thanks for this important info.

As a Private Investigator, I would be interested in being a helper. I have already been going through introduceyourself posts and filing reports with Steemcleaners. My total SP is only around 4K too, but I have delegated 1K to Spaminator to help them. It would be an honor to assist your team further and help keep our community free from abuse.

I really appreciate your hardwork its all paid

Really bad weather conditions are causing too much trouble in transportation There are too many accidents Snowy weather is very dangerousimages (3).jpeg

OH SHIT! That is a hard crash! How can anything like this happen?

Now, if you're a developer the obvious solution that you will tell me to use is to simply use multiple accounts.

I am not a developer and that was apparent ;) Thanks for the hard work behind the scenes, people like you are what will take this to the next level.

@anyx, thanks for the update. This is very helpful. Is there a way to build Cheetah'ss algorithm into the steemit platform so that the search is performed prior to someone even posting?

Great video as always, u have my vote ;)

so cool bro lol

thats real true, cheetah got that problem really.

Done

wow....good post.... & excellent writing...thank for sharing.....dear @anyx...

nice and great article

impressive information to help us know how to differentiate one thing from another and be able to use it without being sanctioned.

Greetings from Venezuela a big hug for you I hope we can continue reading

wow nicr photo by you, thank you for a few weeks ago in Vancouver.

@anyx Oh thank you so much for helping me to understand this point,you are just love

Really informative but as a true author of content in Steemit.Its hard way to find voters.Question here is what it’s take to be successful at steem it

make good content and comments

Thanks for share this important info..... anyx (68)

Very important for your post i know about cheetah same police in Internet

thank you guys for today's knowledge about cheetah and steemcelaner, this is very useful for me and other steemit friends. hopefully we can be better friends, thank you.

Very nice post

Congratulations @anyx, this post is the most rewarded post (based on pending payouts) in the last 12 hours written by a Hero account holder (accounts that hold between 10 and 100 Mega Vests). The total number of posts by Hero account holders during this period was 394 and the total pending payments to posts in this category was $14184.22. To see the full list of highest paid posts across all accounts categories, click here.

If you do not wish to receive these messages in future, please reply stop to this comment.

that's awasome @anyx
thankyou so much for sharing this.

nice to know your blog,
iam a biginer steemian from Indonesia.

hope you will visit my blog, help me and coach me in improving my @steemitilmu account.

see you at
https://steemit.com/introduceyourself/@steemitilmu/introduce-yourself-1051fd8d2d835

Well a good start to your steemit career would be to stop spamming comments..

Spam is discouraged by the community, including comment spam.

Comment spam is currently defined by @steemcleaners as:

10 or more generic comments per day that are not in response to a question posed by the post or comment. “Thanks”, “Great Post!”, “Great Photo!”, “Good”, “Like”, “Nice”, “Wow”, “Cute”, “Follow me”, etc.

Commenting on old posts to hide self up-votes or a voting trail.

Copy/Paste comments that serve no purpose or the purpose is self promotion (including copying and pasting similar comments in rotating way).

Comments that inform the contributor that they have been up upvoted, resteemed, or flagged when the curator does not actually upvote, resteem, or flag the post.

Fishing/farming votes by using multiple accounts to comment on posts by known comment curators.

Unsolicited and off topic referral links.

Continued comment spamming may result in action from the cheetah bot.

Thanks for information and support, good job

You just got my witness vote for all of your good work.
I also agree that SBD should have a floor but not top price.

Congratulations @anyx!
Your post was mentioned in the hit parade in the following category:

  • Pending payout - Ranked 7 with $ 709,81

This is the first time reading your updates. So much work! Awesome.

Thanks for doing Steem – the Best. Good luck to you and Good.

Спасибо, что делаете Steem – Лучше. Удачи Вам и Добра.

please behind the follow

Perfect ad placement for that injury lawyer!

The following solution came to me a while ago, on a different issue about bots and how people (mis)use them.

If we could indicate that a user is a bot (similar to flagging) this could be determined through the feedback people give and it could be known in the system. Then there could be different rules for bots, for example no comment limit, no curation rewards etc. (few humans would need to comment more than every 20 seconds, unless they have serious hyperactivity or something)

@anyx, does cheetah make errors. ?

Cheetah does occasionally make an error, if you think the comment was a mistake and the link cheetah leaves has absolutely nothing to do with your post you can report it in the steemitabuse chat to have someone look into it.

Its about time

You can query rpc nodes with https://www.steemfiles.com/cgi-bin/getsteemnodes.cgi

It gives url and what witness is responsible for it as well as how well it is operating.

There are 2 pages
Pages