Blockchain Update 3: Hardfork 20 and Release 19.4 – AppBase, StatsD, and RocksDB

in #steem6 years ago

Today’s @steemitblog post is brought to you by Steemit’s Blockchain team. We have been making significant progress on our development projects, and wanted to share with you the details of what we have been working on. This post includes updates on Hardfork 20, as well as our 19.4 release which includes AppBase, the StatsD plugin, and RocksDB.

Hardfork 20 Updates

Our primary focus since our May update has been the completion of Hardfork 20. The recent changes we have been working on related to HF20 are listed below.

Resource Credits (RCs)

A major component of Hardfork 20 development is a complete overhaul of the bandwidth system, which is being replaced with a more advanced implementation of Resource Credits (RCs), as described in this post. Much of the work for this new system has now been developed, and we continue to make progress toward its completion. As we discussed in the post, the purpose of this update is to:

… create a system that 1) more efficiently allocates blockchain resources; 2) more accurately measures the true cost of running the blockchain; and 3) enables Steem developers to create more predictable user experiences.

Curation Updates

With the current blockchain rules, if authors vote for themselves right away, they get their author rewards, plus 100% of the curation rewards from their vote, plus a portion of the curation rewards coming from everyone who votes for the post after them. Any other curator voting at the same time as the author would get 0% of the curation rewards. This gives the author an unfair advantage over other curators, because the author can earn additional curation rewards through self-voting.

In order to eliminate this advantage, the unused portion of the curation rewards will be returned to the rewards pool instead of being awarded to the author, thereby increasing the overall percentage of rewards that will be paid to curators. This will better serve the original mission of the curation rewards budget: to ensure that the Steem blockchain distributes rewards to the most valuable content.

Expiration of Internal Market Orders

With the current implementation of the internal market, market orders are never forced to expire. This allows users to submit orders to the internal market which requires long-lived consensus bandwidth, degrading the overall performance of the blockchain. As part of Hardfork 20, we are including a change to require all market orders to expire after 28 days. This is consistent with the implementation of market orders on many of the major exchanges.

Update to Account Creation Fee

Since Hardfork 20 will no longer support the creation of accounts using a portion of delegated stake, there is no longer a need to take a multiple of the witnesses’ account creation fee in order to arrive at the “actual” fee for creating accounts. With Hardfork 20, we will start using the witnesses’ account creation fee value as the actual account creation fee.

In order to preserve the current account creation fee (which is currently the witnesses’ value multiplied by 30) at the time of the hardfork, there will be a one-time 30x modification of all the witnesses’ values.

Removal of 20 Second Comment Limit

We have completed the change to remove the 20-second limit on comments as part of Hardfork 20. It will now allow a new comment every block (once every three seconds), which is the same restriction that is placed on voting.

Implementation of Inventory Based Market for Account Creation

It is a highly technical change, but we have laid the foundation to implement a discounted account creation market based on a user’s held stake. More details about this change are in our original hardfork proposal post.

Discounted Account Creation Methods

In issue 1771 we implemented the operations to allow discounted account creation. The information that follows is somewhat technical (so feel free to skip over it), but we want to give the explanation here for those who are interested.

Creating an account normally requires matching the account creation fee. A discounted account will always have a fee of 0 STEEM. If an account is created in this manner, a discounted account creation token will be consumed, and the account creation will fail if no tokens remain in the creating account.

Any account creation that consumes the account creation fee or greater will not consume a discount token. Any account creation that has a fee less than the account creation fee will consume a token, and fail if a token does not exist. In this manner, accounts can be created cheaply, but not infinitely, and account creation is never prevented via the full fee.

The first operation will claim an account from the discount pool to the creator. This account can be created later at any time with the second operation. If the fee is greater than or equal to the account creation fee, then an account is claimed without taking from the discount pool. The second operation consumes a claimed account from the creator to create the account object, which is able to transact.

Combining these two operations works identically to the account_create_operation currently in use. However, because of the added extensions field, any potential changes to account creation will likely require this operation flow.

Removal of the Minimum SP Power Down Restriction

There is currently a rule that does not allow an account to power down until it has 10x the current account creation fee worth of SP. This was designed to prevent a new account from being created and immediately powered down.

HF 20 burns the account creation fee now instead of powering it up to the new account. Because of that, there is no danger in an account powering down as soon as it gets SP. For this reason, the 10x requirement has been removed.

Fix for Double Voting Exploits

Users reported two exploits: (issue 2428 and issue 2539), where an account could gain extra voting power by using up all of their voting power, and then delegating their SP to another account, or powering down and powering up again into another account.

We are including fixes for these exploits as part of Hardfork 20.
(These are also highly technical, so feel free to skip over if you’re not interested.)

  • The Steem blockchain will track remaining voting power as something called Voting Mana. Voting Mana will still generate to 100% * SP over five days. There is no impact on how votes are cast, only on how their weight is stored.
  • When STEEM is powered up into an account, 100% * STEEM Voting Mana will be added to the account. Currently, voting power remains unchanged and Voting Power% * STEEM is added to your account. The new system provides an increase in voting power compared to what happens today, and is 100% fair.
  • When STEEM powers down, the week before the power down it will not generate Mana for the account. The seven days needed for a chunk of STEEM to Power Down is greater than the five-day regeneration, which allows the STEEM to reach 100% power before it becomes liquid. This is why you can access the Mana as soon as you power up your STEEM.
  • When delegating SP, you have to have the excess Mana needed in order to transfer the SP; otherwise, the delegation will not be allowed. For example, if you have 1000 SP and 50% mana (500 Voting Mana), you will only be able to delegate 500 SP until your mana increases.
  • This will also let us reduce the delegation return period to five days (the mana regeneration period), to let the returning delegation power back up before returning the owner.

This will not negatively impact the user experience: it is a technical change designed to specifically address the double voting exploit.

Fix for End of Payout Period Downvote Abuse

In Hardfork 17, a change was implemented to prevent upvote abuse by creating a twelve-hour “lock out” period at the end of a post’s payout period. During this time, users are no longer allowed to upvote the post, but they can downvote.

This change opened an avenue for abuse, where a user could downvote a post during this period with no opportunity for stakeholders to negate the downvote by adding additional upvotes.

Hardfork 20 will change this “lock out” period to a “cool down.” During the last 12 hours of a post or comment payout period, upvotes and downvotes will still be allowed, but their strength (for the same amount of voting power) will decrease linearly from 100% to 0% over that 12-hour period. In other words, it will take twice as much voting power to have the same impact on a post’s payout if the vote is done with only six hours left on the payout window instead of twelve. A downvote cast in the last minute would have virtually no impact.

This change has not been implemented yet, but we have decided to include it in the hardfork, and are tracking the progress here.

Release 19.4 – AppBase, StatsD, and RocksDB

The 19.4 release, which includes AppBase, the StatsD tool for measuring statistics data in steemd, and the RocksDB plugin for account history, is nearing completion. At this point we have completed all of our planned development, and are just in the final stages of testing. Assuming no new issues are found, we plan to tag the latest master as release 19.4 very soon.

Miscellaneous Changes

In addition to the changes listed above, we have also been working with the Applications Team to make some necessary structural changes to better support Hivemind and communities. There have also been several minor bug fixes, and changes that are highly technical in nature.

The full details of all the changes we described here, as well as several that were not mentioned, can be found in our steem GitHub repository.

We will continue to keep you up to date on our progress and will let you know as soon as the 19.4 Release and Hardfork 20 are ready.

Steem on,

The Steemit Blockchain Team

Sort:  
There are 2 pages
Pages
Loading...

This is a good explanation of all the bits and bobs on the go for people who don't know how to go to GIT and track issues. This is a perfect example of how to marry the tech explanations to communication for the people who want to be informed here on platform in a more easily understandable way. The communication is improving markedly and it really is noticed. (I know I'm saying this what feels like daily now, but it can't be underlined enough that this is what has been missing for a long time, and your efforts to address it mean a lot to the full user base at all skill levels.)

Can't wait for the 19.4 master. I'm really looking forward to the possibility to provide more support with the same resources we're utilizing now, and to grow it from there.

Always appreciate the feedback! There's no such thing as "too much communication" and that goes both ways :)

Yes, there is such a thing called "too much communication "

Case in point

Nevermind..I am being overly critical

There's no such thing as "too much communication"

be still my beating heart... <3 ;)

Fix for End of Payout Period Downvote Abuse
In Hardfork 17, a change was implemented to prevent upvote abuse by creating a twelve-hour “lock out” period at the end of a post’s payout period. During this time, users are no longer allowed to upvote the post, but they can downvote.

This change opened an avenue for abuse, where a user could downvote a post during this period with no opportunity for stakeholders to negate the downvote by adding additional upvotes.

Hardfork 20 will change this “lock out” period to a “cool down.” During the last 12 hours of a post or comment payout period, upvotes and downvotes will still be allowed, but their strength (for the same amount of voting power) will decrease linearly from 100% to 0% over that 12-hour period. In other words, it will take twice as much voting power to have the same impact on a post’s payout if the vote is done with only six hours left on the payout window instead of twelve. A downvote cast in the last minute would have virtually no impact.

This change has not been implemented yet, but we have decided to include it in the hardfork, and are tracking the progress here.

With this change all you do is to fuck both human content uploaders and human spam adversaries and change the effective downvotability period from 7 days to 6 days and 12 hours, whereas you left the effective rewardability period on 6 days and 12 hours, whereas in platforms like Dailymotion and Youtube it is unlimited the way it should be.
This favors automated behavior even more and supports abuse.
Humans tend to miss content the shorter the period of effective votability gets.
The votability period here was 30 days initially, which was still too short.
You made this an instant messaging platform stored on a blockchain.
I wonder if the origin of this proposition is a mildly corrupt moron with no better way to spend her/his time, or a fully corrupt preminer or one of their cronies.
I also expect her/him to remain hidden behind a github identity due to the shame and guilt.

You got a 100.00% upvote from @luckyvotes courtesy of @stimialiti!

This comment has received a 76.92 % upvote from @steemdiffuser thanks to: @stimialiti.

Bids above 0.1 SBD may get additional upvotes from our trail members.

Get Upvotes, Join Our Trail, or Delegate Some SP

You got a 48.92% upvote from @sleeplesswhale courtesy of @stimialiti!

with 19.4 was it to allow us to edit our posts?

Yes, and it was allowed before it too.
Now is 19.5 and it is still possible.
As long as the post is less than 7 days old.

This comment has received a 90.91 % upvote from @steemdiffuser thanks to: @stimialiti.

Bids above 0.1 SBD may get additional upvotes from our trail members.

Get Upvotes, Join Our Trail, or Delegate Some SP

I have to say, the frequency of these updates is amazing compared to what it was last year. They really encourage me during the down market. I have a sense of what's happening and the changes always seem really fascinating. I still worry at times that the releases of new developments is proceeding at too slow of a pace.

Steemit could conceivably lose one of its primary advantages: being the first to roll out the concept of a social media blockchain. Of course many know about ONO, and more will follow. Many of the developments like Oracles sound incredible to my (nondeveloper's) mind.

As they say, though: the proof is in the pudding.
(And I can't wait to taste it)!

Awesome changes! I really like the cooldown for flags 12 hours before payout.

Btw, will the self-vote curation be changed to start at 15 minutes like previously discussed?

All in all, great work Steemit team!

The issue for that (along with quite a few others not mentioned in the post) is still listed in the HF20 project: https://github.com/steemit/steem/issues/1878

Looking forward to this. These forks are one reason why Steem outperforms all other decentralized networks on the planet. In my humble opinion.

Not judging by user retention it doesn't:

Snapchat has a retention rate of ~30%.
Instagram has a retention rate of ~39%
Facebook has a retention rate of ~69%

Steemit's retention rate is an embarrassing 13%.

My suggestion, which will be roundly ignored I'm sure, is to boot every single bot off this place - make "institutional" tools that handle spam and duping external content, and decouple the relationship between vote power and how many tokens one holds.

Otherwise, you just have a microcosm of the "outside world" where the deepest pockets win, and the little guy/gal just goes away.

But I'm sure they'll keep shuffling the deck chairs until there are no more users to care.

The platform is still young. Changes are being worked on to improve a lot of the issues that people have. Also, your comment is technically not relevant to @gray00's. He/she said "decentralized networks on the planet". None of the websites you cited are decentralized. Are there any blockchain social media sites that are beating Steem?

If Steemit can't keep people around, then what is the point?

A "perfect" system populated by bots?

The bots have been here since the beginning of the platform. Sites like Reddit have a lot of activity from bots too. Finding a way for bots and humans to co-exist should be part of the thought process, because it will never be possible to eliminate bots. They can also provide a lot of value too, depending on their programmed purpose.

Snapchat, Instagram, Facebook isnt decentralized.

Comparing Steemit to current mainstream social media platform is as unfair as comparing a horse to the first gasoline powered prototype automobile in the 1800s.. The horse wins, sure, early 1800s cars are shit. But no one is using horses on the AutoBahn today.

I have a feeling you don't understand what the Steem blockchain stands for and what it actually aims to achieve, It's more than just "content platform that pays users". "Institutionalization" is exactly what the steem blockchain is fighting against. "decoupling the relationship between vote power and how many tokens one holds" goes against stake-weighted voting that is the whole point of the steem blockchain.

Perhaps when SMTs come out people can issue SMT that works on 1A1V/ABV as there is use for such methods of voting. But the base layer steem reward distribution has to be stake-weighted to work. A spam account's vote cannot worth the same as an account of a person who worked and invested on the platform for years.

""decoupling the relationship between vote power and how many tokens one holds" goes against stake-weighted voting that is the whole point of the steem blockchain."

I disagree. Rewarding creators and curators is a completely separate issue from basing rewards on stake, and Steem, from my read of the white paper, was not intended to duplicate the issues stake weighting imposed on societies throughout history, but to revolutionize how economic rewards impacted society, using Steemit as a microcosm.

Indeed, as @ned has repeatedly pointed out recently, this is precisely one of the purposes of SMTs, as combining them with oracles and communities enables stake weighting to be avoided as it was not for Steem itself.

I expect that the user retention rate is a reflection of how unsatisfactory many have found stake weighting on Steemit. For the cohort of which I am a member, joining in May 2017, the retention rate is about half the 13% figure named, actually about 7.5% YOY.

I expect that communities that emphasize stake weighting will have even lower retention rates when alternatives become available, as they have not yet on Steemit.

The real point of censorship resistance is to empower society to reflect actual facts in their posts and comments, and pandering to stake is inherently degrading. The preferability of alternatives will be quickly revealed, and communities will thrive or fail as a result of their recognition of the vastly higher value of honest and factual creations to society than those merely better funded.

While the several social media platforms earlier mentioned weren't decentralized, few users of social media have much idea what effect decentralization has yet. The abysmal retention rate of Steemit in comparison reflects the satisfaction of users based on the metrics they find compelling, and the absence of stake weighting is possibly a significant factor, despite the utter lack of earnings on other platforms most users are availed that are potential on Steemit.

The 7.5% retention rate was calculated by @paulag, and you can verify the validity by checking out her blog, which I highly recommend.

You gave me alot to think about. Perhaps the emphasis on stake-weighted voting needs changing. I dont know for sure what is the best way to go about it.

But I can think of one problem if stakes on the blockchain are removed, specifically what then would incentivize users to gather steem and hold them as a Store of Value? You can surely agree that if there were no incentive to hold on to steem, every creators would drive views and clicks (possibly using click-farms) to their posts, get payout and immediately cashout.

As for user retention, tbh by now i can't quite care less (or perhaps i've changed my stance on "user retention/acquisition") because more than likely those who came here for quick profit alone came here for the wrong reasons and they will be the ones that leave here. You can call it maximalism, elitism whatever. But tbh i dont think we're ready for full mass adoption yet.

I don't think the steem blockchain will be the only blockchain that is relevant to the content web. Maybe another competitor will soon rise and rival against Steem's stake-based voting? Only the market will tell. Until then I'm sticking to the platform that gave me an opportunity that i otherwise would never have gotten anywere else.

I know in the crypto-verse every centralized service is the nail to every blockchain solution hammer -- but that isn't the point.

Those services know how to keep people around.

If Steemit can't get their act together, then what is the point?

Bot-topia, with automated crapposts and upvotes?

'bot-topia' love it love it love it.

The stats are not good, but there are many people on the ground trying to improve these numbers. The problem is, steemit inc is only supporting devs with smt possibilities and not people trying to help the folk here on Steemit. By doing this to me they are saying the do not believe in steemit.com. Steemit.com as it is now, if not taken care off will not be the front for the steem blockchain for long, Dlive and other apps will be

What do you suggest?

What is retention rate of Dlive ? Still a bit early to tell. What is the retention rate of YouTube Adsense creators? I think you have a valid complaint. But then again. How many accounts were scam accounts and what not? How did you get to 13% is what i'm asking? Not saying it's wrong just curious.

@paulag did a analysis - not only of the retention rate, but the theoretical userbase that is actually contributing to Steemit -- and it sure isn't 1 million accounts, more like 45,000 if we're lucky.

Doesn't bode well, is all I'm saying.

Regarding the Fix for End of Payout Period Downvote Abuse:
The 12 hour upvote lockout has been actually been useful for fighting abuse, and instances of it being abused have been minimal. I actually think this cooldown scheme will lead to more last-minute self voting abuse if the voters know that if it's not caught immediately, the vote power needed to counter it may not be available.

I also don't see the logic in adding yet another change to an already bloated feature and change list.

Agreed, if they make this change, they should at least make the period longer (like 24h instead op 12h)!

It makes me think of the little war that went on a few weeks ago (or maybe it's still going on? I haven't paid much attention) where numerous Haejin posts were downvoted on the last day. Uncertain if any of that was happening in the last 12hrs or not.

I understand that the initial 12hr window idea was an effort to allow correction of 'unjustified rewards', but this seems to make 'justice' (in whichever direction), more difficult to achieve. Seems to be another classic example of punishing 98% of people that aren't abusing things in response to the 2% that are (or whatever the percentage numbers may actually be).

but @pfunk, we have 7 flag-days per post, with a nice fade in and now a nice fade out, don't you dig it?

It's still ongoing. I see this change as at least a chance to move in a better direction. Right now, one person can invalidate the votes of 200+ other people and nothing at all can be done about it. At least the proposed approach allows others to disagree.

Very impressive.

It sounds like some updates are about ready to go live. This is an incredible development since so many are negative about the time it is taking (ironically I dont see the ones with programming/coding backgrounds doing the griping).

STEEM is going to really take off. It is great to see these developments to start rolling out as the second half of the year kicks off. A lot is going right for STEEM, it is exciting to be a part of all this.

A year from now we will all be very happy with how things progressed.

In order to eliminate this advantage, the unused portion of the curation rewards will be returned to the rewards pool instead of being awarded to the author, thereby increasing the overall percentage of rewards that will be paid to curators.

In light of this, can you explain why there's still an early voting penalty at all? It seems like if the goal is to reward curators it would be better served by making curation 25% from the beginning, rather than just making votes in the first fifteen minutes worth less than votes coming in later.

(I assume the change to 15 minutes remains and just wasn't mentioned.)

As I understand the original early voting penalty it was intended to allow voters to choose to reward the author more. If that functionality is going away I feel like there needs to be some new justification for keeping the penalty, or it should simply be scrapped.

The reason for the early voting penalty is to give "humans" fair footing against bots. Without the penalty, users who automate their votes via bots will be able to "snipe" curation rewards by getting their votes in before any humans have a chance.

Personally, I find that spurious - after all SteemAuto and SteemVoter votes can be set to come in at any specific time after a post is created. Nor am I convinced that this is actually a feature of significant value, even if it were effective.

But more so, I feel that anything that takes away from the idea that a vote has the same total value no matter when or where it is cast should be scrutinized extremely closely. Everything that does that makes it exponentially harder for new users to understand what's going on here, and gives a huge advantage to the sophisticated user, which seems like it's the opposite of what we want.

I agree with this - regardless of the method used there will be an "optimal" method of voting and bots will always be better at implementing that than humans.

regardless of the method

Mostly I agree with what you wrote but not this part. Simpler methods give bots less advantage or possibly no net advantage. More complicated methods calling for extreme precision like changing the percentage payout every 3 seconds definitely favors bots.

Making changes to the blockchain requires consensus. Even though you might feel one way, there are others who may disagree. As far as I know, there is not consensus among the stakeholders to get rid of it, as some do see value in having it there. Reducing it from 30 to 15 is at least heading in the direction you want it to go. It is just not getting rid of it entirely due to the people who still want it there.

Just to be clear, this is not "the direction I want it to go." Any change that reduces author rewards is a bad change as far as I'm concerned, because it favors existing stake over new users. I'd much rather things stayed as they are now. I'm just confused at the self-sabotaging character of the change your consensus has settled on.

In light of this, can you explain why there's still an early voting penalty at all? It seems like if the goal is to reward curators it would be better served by making curation 25% from the beginning, rather than just making votes in the first fifteen minutes worth less than votes coming in later.

I read this as "why not get rid of it".

I guess your reply though indicates why it was moved from 30 to 15 instead of getting rid of it entirely. It is a happy medium between people like you (who want to keep it, and have it high) and people who want to get rid of it / lower it.

I don't think you're grasping my argument at all. Moving it to 15 and getting rid of the additional rewards isn't in any way a happy medium. Moving it to 15 is irrelevant; the difference is between having all of the vote value given to the post, and having some of the vote value senselessly trashed.

A happy medium would be giving the additional rewards to the successful curators instead of the author.

If I make a run at this from a different direction: I have a fifteen-cent vote, and I want that vote to be worth fifteen cents to somebody when I cast it. Not fifteen cents but only if no one happened to vote on that post in the first fifteen minutes, otherwise less.

By trashing some of the curation, an early vote now makes my one-hour vote on that post worth less than the same vote cast on a post that didn't have early votes, because some of its value won't go to the post at all, it just disappears as if I had never cast that part of the vote. I still lose the voting power though.

How is that not completely insane?

Votes do not have the same value regardless of when or where they are cast. The curation rewards change depending on the presence and relative timing of other votes. The author's share of curation rewards changes based on the precise timing of the vote (within the first 30 minutes).

This change does not alter the fact that allocations shift due to the timing of the vote, it just changes how they shift.

If SBD were $1, a vote would currently have the same value no matter when or where it was cast. It doesn't go to the same people but a $0.15 vote is always worth $0.15 to someone involved with the post. This does change that, drastically.

(The broken SBD peg does make votes more valuable when cast early under the current system, though that's obviously much less of a big deal now.)

someone involved with the post

Okay, but I find that irrelevant. There are many different and independent voters (usually) and how the rewards are distributed between them changes, sometimes drastically. The sum of all the rewards going to a group of different independent people is a number without relevance.

If you are concerned about the case where the author is the only voter (or perhaps that all voters are colluding in some manner) and is rewarding themselves for something that no one else cares about, then you would have a point, in theory. Most would find that irrelevant at best and undesirable at worst.

The sum of all the rewards going to a group of different independent people is a number without relevance.

It is literally the value of the action of voting. It's incredibly relevant. In a larger sense it is the value of my stake, which is pretty much the most important thing here.

(speaking about vote penalty only, not about the effects on the author)
Actually, the change did something here. People were less motivated to set auto votes at exactly 0 minutes with this change, thinking from the perspective of curation reward maximizers. You have to act in accordance with how you believe similarly minded autovoted voters would, based on how much of the penalty they were willing to take.

Most users don't even get a chance to vote in this early window anyway, so that's why I don't really see this as a problem. And for those that do, that's where this reverse auction is useful. Otherwise you have this "which bot got their vote in first mechanic" that makes things completely chaotic.

But you're right. This system is probably too complicated. That square root curve for curators is pretty much what motivates this "bot needs to get in first" rush. But I don't really see a way out here. Without those rules, a large staked bot can take all the curation reward for themselves. With the window, others can sneak in before the bot potentially. And changing the curve to be less exponential might work but doesn't address the bot vote at exactly 0 minutes.

Upon looking at this thread I think it got jumbled because there are two talking points here: effect on the author and people that want their vote to go to the author, and why there's a penalty window to begin with (which in the previous change were related as well)

It was intended to give people a chance over bots.
With no penalty the bots get in line first.

So the account creation fee was not even now removed: Only modified. Sorry but not interested until that is solved: I see little hope for a service where you still need to pay (or get paid) to merely create an account 😠

There have never been plans to remove it. Allowing 100% free account creation would create an attack vector on the blockchain that would result in "name squatters" taking all of the good usernames that people would want.

I've dealt with people taking about this issue and for the love of god i was never able to explain to them as clearly as you just did. thanks for writing this comment.

IMO the majority has to understand why account creation cannot be free. Unlike other centralized platforms, accounts created on the steem blockchain is immutable.

PS. it's not only taking good names, an organized entity with enough time and resource can literally take any alphanumeric names from 1 to zzzzzzzzzzzzzzzz that hasn't been taken. Which considering the size and resource of the incumbent social media tech conglomerate has, isnt all that impossible.

I do have to say, this article does talk around the subject a bit?
Can we get an update with the actual plan for this, as far as inclusion in HF20?

I can see some positive things there but have to address this one :

the unused portion of the curation rewards will be returned to the rewards pool instead of being awarded to the author, thereby increasing the overall percentage of rewards that will be paid to curators.

This will definitely not ensure that the Steem blockchain distributes rewards to the most valuable content.

I appreciate that you're trying, but before you implement new problems to the system, consider how we're going to tackle them. What I can see coming is authors quickly learn that they don't get to keep their rewards by voting for themselves in the first 30 minutes; so they wait. And when they wait they reward the curators who voted before them. How lovely. Until the curators learn that they get better rewards for voting for self voters with more SP, ensuring that the whales get the most support while smaller accounts get the crumbs (curation) for voting for them. Unless you can add an antidote this is a recipe for implosion.

This change does not really open up a new form of abuse. What you described is possible today. If I am a large stakeholder today and I want to incentivize a large number of votes prior to my self-vote, I can just wait 30 minutes and vote then.

There will always be an economic incentive for curators to lead the votes of large stakeholders whose votes they can predict. In my view, if we are to address this problem we need to look more into ways to lower the rewards on content that is receiving a higher payout than it is worth (i.e. downvoting).

I think what @beanz may be suggesting is that with the 75/25 split, the author may gain MORE from waiting. But then again, they could do that now too. I don't see this change as adding any additional benefit to self-voters. It takes away an option to instantly vote. It doesn't add any new options.

It is already possible, but the incentive to do so is currently countered by the incentive to grab your own curation rewards. This counter incentive is being removed, and I don't disagree with that, but to suggest it will bring the best content to the top is utterly ridiculous. There will be increased incentive to vote for whales because whales have less incentive to vote themselves early. What we could do now and what we're incentivised to do are 2 different things.

The issue you are describing is that self-voting whales will now be waiting until the "early voting penalty period" is over, which will increase their rewards from self-voting because other users will be front-running their votes.

There are two possible scenarios:

  1. The amount they will get from this is more than the curation rewards they would currently receive by self-voting as soon as the post is created.
  2. The amount they will get from this is less than the curation rewards they would currently receive by self-voting as soon as the post is created.

If 1 is true (which is what you are asserting) then they can already do this today. This change is not enabling any abuse that is not already possible. Steemit could exclude the proposed change from the hardfork and all the self-voting whales could start doing this to increase their profit.

If 2 is true, then they will be making less under the new system, so there is no issue.

was there not to be the ability to edit posts with this update?

That should be part of the AppBase release. I think this post was just about the stuff they had specifically been working on. That change was technically completed a while ago, and is just waiting for the release to be deployed.

"If 2 is true, then they will be making less under the new system, so there is no issue."

Less of an issue.

This entire comment chain is exemplary of the problem that variably valuing votes, and rewards for both creators and curators, by timing and etc., introduces - IMHO unnecessarily.

Want curation to actually better reward content based on the value of the content? Get rid of all the timing and games that vary the value of votes on content.

Deal with bots separately. SOC (SMTs, oracles, and communities) will make that possible. Gamifying curation simply incentivizes playing games, rather than rewarding good content rationally.

I hope to see this realized at some time. I expect SOC will make it obviously a necessary solution to many that presently don't understand how gamifying curation degrades curation quality.

Thanks for clarifying this complexity as you have. I hope we can simplify curation significantly soon, as doing so will make good content - and the people that make it - more valuable than knowing how to manipulate the rewards system.

Get rid of all the timing and games that vary the value of votes on content.

There is talk of doing this, but no consensus yet.

Deal with bots separately. SOC (SMTs, oracles, and communities) will make that possible.

Curious how

I am happy to hear that folks you listen to are discussing simplifying curation.

As to how SOC can deal with bots, in his talk in Korea @ned (whom I believe would not make idle speculations on the matter) stated that oracles are able to ascertain whether an account is A) a human person, and B) even a unique user. Communities intent on being bot free can rely on such oracles to exclude bots and socks.

The actual detection of bots is complex, and I don't pretend to grasp the details. If @ned did not misspeak (and I did not misunderstand), and can be relied on, then oracles make it possible for communities to exclude bots.

They "problem" as you described it, sounds like the perfect / fair market to me...

Seems the plan may be to have SMT fix some of these issues; but you are right, it would be nice to SEE MUCH MORE OF A ROADMAP? on how these obvious problems will be solved... just makes sense to communicate this.
Please post the updated 2018 Roadmap along with this post.

Thanks for the update team!

My concern is with the removal of lock out period. There's no real need for the change. Maybe it's worth another round of discussion internally before you implement it into the fork.

Overall things are looking good and I'm excited to see where these changes will lead us. :)

Always love an update. Thanks :)
Any chance we could look at Editing after the payout date?
There are a couple of broken links in my blogs.
The story of when I popped the question ends with a gif of the big moment, except it doesn't.
Perhaps with a higher Resource Credit cost to deter misuse.

That one should actually be part of the 19.4 release. It is already merged into the master branch in GitHub.

WOAHHHHH

Fix for End of Payout Period Downvote Abuse
In Hardfork 17, a change was implemented to prevent upvote abuse by creating a twelve-hour “lock out” period at the end of a post’s payout period. During this time, users are no longer allowed to upvote the post, but they can downvote.
This change opened an avenue for abuse, where a user could downvote a post during this period with no opportunity for stakeholders to negate the downvote by adding additional upvotes.
Hardfork 20 will change this “lock out” period to a “cool down.” During the last 12 hours of a post or comment payout period, upvotes and downvotes will still be allowed, but their strength (for the same amount of voting power) will decrease linearly from 100% to 0% over that 12-hour period.
In other words, it will take twice as much voting power to have the same impact on a post’s payout if the vote is done with only six hours left on the payout window instead of twelve. A downvote cast in the last minute would have virtually no impact.
This change has not been implemented yet, but we have decided to include it in the hardfork, and are tracking the progress here

YES YES YES downvote abuse is over now flagging posts at the last 12 hours is now not going to work unless you REALLY want to spend money flagging hahahahaha YES

I hope this news goes out and more news about HF20 so we can get steem price up! we need steem whales to feel like they need to be buying more steem again!

DONT LET EOS TAKE THE MEAT FROM OUR MOUTH, DO NOT LET EOS DRAG US INTO THE JAWS OF DEFEAT,

STEEM STILL HAS A CHANCE just like COke and Pepsi both live, just like att and verizon both make money,

as @surfyogi said

I agree EOS will take the lead in many ways very quickly, and in price, has already done so.. but STEEM is application specific and good for people and communities, traditional blogging, vlogging, etc.
EOS is generic Enterprise Computing Operating System that will set the stage for a new era of "Blockchain does everything better".
The fact that STEEM won't migrate to EOS, shows it is application specific in design.

Glad to hear Zach agrees, or I would definitely "hear about it" ;-)

Well, I added issue 2428 intentionally not on steemit but github to prevent using this exploit.. And now it's all over public.

Hmm

If it is on GitHub then it is technically already public :)

Even though technically any user can exploit what is there, it is a fair amount of work to go through. It would likely only be worth it for an account with a large stake - and most of the people with that much stake are probably aware of what gets discussed in the GitHub repository.

Fantastic update! Thank you for the overview and the technical details for those who want them. I think that's a good approach to take to have everything in one place. Even if not everyone understands the details, they will at least understand that important work is being done.

there will be a one-time 30x modification of all the witnesses’ values.

Will this happen automatically or will the witnesses be asked to coordinate this change with a code rollout? If it could happen automatically... how?

We have completed the change to remove the 20-second limit on comments

Hurray! I've been using tools like fast-reply in the meantime.

Voting Mana and Voting Power kind of sound like elements of a video game. Do we get wizards soon? ;)

Fix for End of Payout Period Downvote Abuse

While we're talking about naming things, can we call this the berniesanders exploit fix? He seemed to enjoy using it (did so on me for about three months) :)

we plan to tag the latest master as release 19.4 very soon.

Nice! Do you recommend witnesses roll these changes out now to seed nodes to help with testing or just wait until the tag is cut? Should backup witnesses start testing the code now?

Thanks for the great work and the communication about what's going on. Much appreciated!

Hahah... Speaking of @bernisanders, looks like he's at it again:

225 upvotes from the community with 54 comments, but flagged by bernie in the last 12 hours because he didn't like something I said in chat. Now it will be hidden forever.

Keep an eye on my blog. I imagine many of them will go to zero soon because old bernie was trrrrrrriggered.

I hope ngc goes back to playing with steemmonsters. He's more fun that way.

Will this happen automatically or will the witnesses be asked to coordinate this change with a code rollout? If it could happen automatically... how?

The way it is described in the issue, it will happen automatically at the time of the HF.

When the hardfork hits, we need to modify the top 20 witness account creation fees by 30 (same effective fee) to prevent creating accounts at an extreme discount. Future witness updates will work as normal.

"we need to modify the top 20 witness account creation fees by 30 (same effective fee)"

huh?

Currently the cost to create an account is 30x the witness account creation fee. At the time of the hardfork, it will multiply all of the values by 30, but stop multiplying them by 30 to calculate the cost. It sounds confusing, but basically it will end up with the cost being exactly the same before and after the fork.

Zappl understands and agrees with everything other than the account creation fee. We commented in the top comment about why.

https://steemit.com/steem/@zappl/re-timcliff-re-fyrst-witness-re-steemitblog-blockchain-update-3-hardfork-20-and-release-19-4-appbase-statsd-and-rocksdb-20180629t190135595z

Curation

It was previously discussed to reduce the time limit on the initial voting penalty (currently 30 minutes) to make it less obtrusive to humans (rewarding intuitive "I like it" voting relatively more than strategic "should I wait another 20 minutes" voting). Is this still included?

A much shorter penalty would retain the disincentive for bots to flood the first block with instant votes but get out of the way of humans who don't want to set a timer to vote without being penalized. I personally think 30 seconds to a a minute is enough but the earlier discussion cenetered around 5-10 minutes.

The issue for that (along with quite a few others not mentioned in the post) is still listed in the HF20 project: https://github.com/steemit/steem/issues/1878. The current proposal has it being changed from 30 to 15. I'd be in favor of it being taken down to 5, but iirc they did not want to go too extreme due to wanting to quickly reach concensus on the set of changes in the HF.

IMO the 15-30 minutes is 'extreme' and human users of the platform (including both more- and less-informed ones) have been stating that for two years. But anyway, 30->15 minutes is a step in the right direction.

Is the expiration of market orders going to be a soft fork? Just have witnesses reject transactions having too long of a expiration time, if they’re producing a block. This would allow that change to be implemented immediately (before HF20).

Also, perhaps Condenser should default to (or always), setting the expiration to 28 days, right now (before HF20). Most people don’t want their orders to be long-lived anyways, so this wouldn’t be a big problem for users.

It could be done that way (soft fork) but there is really nothing urgent about this. The condenser change is a reasonable suggestion.

@steemitblog sir It does not seem to be a great advantage, because after the postponement of the steamout and steamboat, any post may have come at a specific time, so think that I think or think that a thing that takes away from the idea that total vote is not total When or when it is throwing it should be very closely scrutinized. UpVote

Thanks for the hard work.
I dont say that enough, i know.

Eagerly waiting for Hardfork 20 to be ready and launch...

we want the google authenticator feature. :)

We have completed the change to remove the 20-second limit on comments as part of Hardfork 20. It will now allow a new comment every block (once every three seconds), which is the same restriction that is placed on voting.

This allows quicker spam!?

Voting Mana

Feels like playing a video game 😀

"Bandwidth / Resource Credits should be the tool to combat spam."
To quote timcliff
Not sure, how it is different from now, but steemcleaners see a lot of spam.
And this will get worse, with the 3 second rule.
An account that has been phished will be able to spam now way faster with that implemented.
But yeah, that's no problem
:/

Loading...

Thanks for the update, love the little improvements that will make the steem experience much better. Looking forward to all the updates 💪🏼🙌🏼💯

I know most of the focus here has been on the HF20 changes, but I just want to say that I am extremely excited for AppBase/RocksDB. Anything that reduces the hardware requirements for running a full node will really help all of the sites and services that use the Steem blockchain!

Good to see RocksDB and HF20 getting close to completion. I notice you haven't left any dates - kudos, that's the sign of a maturing organization. Only leave release dates when you're absolutely certain you'll hit them. Of course, keep us updated while development is ongoing - exactly like this.

Also, I would suggest branding the RocksDB release 0.19.5 to separate it from current 0.19.4 RC doing the rounds. Some witnesses are using it on production nodes, no less, so there's potential for confusion. RocksDB is a pretty dramatic shift for the Steem blockchain (and blockchains in general), and needs its own point version, at least.

RocksDB is technically only for account history. (It won’t have an impact on witness nodes.)

Separating it out from the current “master” branch could introduce new risk into the code they have already been testing for weeks. It may be more trouble than it’s worth.

I also don’t know what real benefit it will provide since it sounds like all the node operators who are running account history will probably want to pick it up as soon as it is available.

Also FYI, some witnesses (myself included) have been running the RocksDB plug-in on some test nodes for a while now too.

Oh, I just meant purely a naming/branding change for the final release. Anyway, not an issue either way.

Thanks for the update!

Congrats, this wonderful post is featured on today's Joy Steemit Daily News! (in Chinese :D)

Keep on good work, Steemit team


Thanks for the update. I'll admit I don't understand all the technical details, but I trust the team. This is bleeding edge technology and you are creating a new economy. Getting it right is more important than doing it quickly, but I know the community is impatient to move forward. Have to find the right balance

Interesting!

Interesting changes. I can't wait to see how they play out, and the community reactions are always fun also.

Great update on the roadmap developments @steemitblog I know there is much anticipation in the community for HF20....great that the post was nicely balanced in not being too technical.

just cut the autovote reward already!! This would stop a good deal of abuse.

Removal of 20 Second Comment Limit

Did you test that the code can sustain that? Won't we just be DDOSed to death?

This can already be done by creating multiple accounts (with 7 accounts you can post every 3 seconds, the same as will be allowed for one account post-fork), or using transactions other than comments, which don't have the same time limit. The real protection against DDoS is the bandwidth limits (being improved) not the 20 second limit.

My question is twofold I guess:

  • on the blockchain, can you give an idea on how much bandwidth a comment take? or more like if I have 500 steems, how much comments will I make? btw was the limit on the UI or the blockchain itself?
  • off the blockchain, can the api/steemit.com sustain a hypothetical X20 the number of tx/s ?

In the current system bandwidth is mostly just measured in terms of the number of bytes. That's not very good because different operations use different resources.

As for capacity, it doesn't really work that way. Blocks have a set size (and in HF20 there will be limits on other resources too) and everyone is limited to a multiplier of their assigned bandwidth (according to SP). If blocks are full then the multiplier decreases and if they are not full then the multiplier increases.

So everyone already has multiple limits and it isn't possible for everyone to send 20x more transactions, even if the 20 second rule weren't there. The multiplier decreases and everyone gets more limited, according to available bandwidth.

The only change here is that once per 20 seconds is no longer an arbitrary limit for one account even if there is sufficient bandwidth available.

Thanks for the details. With all those limits, the only way to try to DDOS steem on-chain is to have a large amount SP, right?

So you have a bunch of limits on-chain. My second concern was that before those limits kick in, you still need to receive and process them. Did you test the steemd/the api can sustain a bigger load or do you assume there is no need? I mean it as someone sending the same tx again and again, that will be blocked by the blockchain when it gets there or more active users commenting live the world cup final.

It depends on the amount of data in the comment (text and metadata)

excellent update and as well I like the 3 sec. comment time now. Thanks

A major component of Hardfork 20 development is a complete overhaul of the bandwidth system,

Issuers of SMT’s would still need Steem Power for bandwidth? Or not still need it?

And, if they do, is there any way to quantify how much they’ll need?

Every operation will need SP for bandwidth (resource credits). Creating and managing SMTs will involve operations, which will require credits. The resource credit system should make it somewhat more transparent (if still somewhat complicated) to know how much is required as a function of the resources used by the operation and the availability of those resources on the system.

Also, it will remain the case (as with bandwidth today) that the resource credits needed will vary according to system usage. There is not a fixed amount, although in theory there should be a cap (very high).

Congrats Steemit team!
I like cooldown voting/downvoting in the last 12 hours and the removal of 20 seconds comment intervals.

Really great stuff - looking forward to 19.4 and HF20!

Is there any chance we will be getting an authentic Steemit App soon ??

Great job. More grease to yo elbows! I love this:

We have completed the change to remove the 20-second limit on comments as part of Hardfork 20. It will now allow a new comment every block (once every three seconds), which is the same restriction that is placed on voting.

When STEEM powers down, the week before the power down it will not generate Mana for the account.

Does this refer to the STEEM being unvested that week, or the entirety of the STEEM being powered down?

It will still be vested, but it will not be contributing to the “mana” i.e. what is currently called voting power.

Thank you so much for explaining these changes in such a way that anyone can understand! I appreciate that very much. Looks like there's some big things happening, especially the fix for end of payout post abuse, curation updates, and the change from 20 sec to 3 sec in between comments.

I'm resteeming this, but in the meantime, tagging a few folks to help spread the word a bit. Thanks again!

@davemccoy, @beeyou, @glenalbrethsen, @themanwithnoname, @thedarkhorse, @wolfhart, @abh12345 (I'm sure you're aware), @hitmeasap, @insideoutlet

You called? I'm checking it out now.

haha look how easy that was for me :) Thanks!

Just read through and it looks like they have some good changes in store. Looking forward to them being implemented. The curation thing won't affect us @lynncoyle1, not even on the one inadvertent upvote that gets removed. ;)

I just thought it might be something you're interested in, but yes, upon further reading, I see what you mean. I do like the other changes I mentioned though. Time will tell, right? :)

I think they're good changes. And I think it's a good idea to have the author's portion of the curation rewards burned for self-voters. It will decrease the reward they get from it. We'll see what happens.

Thanks looking it over. Thanks for the tag

You're welcome buddy...make sure you get some sleep before responding here lol

I'd have to agree with @themanwithnoname's general assessment. Of course, saying things will happen in x way because y was implemented is the intended purpose of making the change, but x has a way of morphing into z by the time users get a hold of it to find some way around it. I hope that's not the case, but it will be interesting to see what happens because of these changes.

It will be interesting, and right now I'm tired, so I'm coming back to this and your comment because at the moment, it's a bit like "who's on first?" to me haha

Thanks for the shoutout Lynn. I had several people mention to me too. A popular topic of discussion.

I thought I heard something about the 30 minute curation rule being changed into 15 minutes. Add I didn't read anything about this I guess I heard this incorrectly?

Excited about these changes, looks like they're going to make a lot of things better!

So, you did the first part from 30 minutes to 15, but not the second part.

restore 50/50 curation rewards

Why not fixed this one, if it works for Golos?

Sounds good, many thanks for the hard work you have been putting in with the whole team.

I'm getting quite impressed with the new communications with the community being undertaken. I also am getting more excited about the changes coming as I better understand them from these posts.

Thanks very much!

Also, just out of curiosity, what happens to curation rewards on a declined payout post? I'd appreciate any information folks that know could share.

Both author and curation rewards from the post are left in the rewards pool for other posts.

Thank you very much =)


Congrats, you made the #steemitminute for today!
Click the Image Below to see the Video!

There are 2 pages
Pages