This is just a quick update on some of our work on HF24 this week. As always, this is a fairly detailed report, so you can skip to the end if you just are interested in our updated expectations for the hardfork date.
Hived/Hivemind progress
Last week we continued testing and fixing discrepancies between output of the old hivemind and the eclipse version of hivemind.
One of the more time-consuming fixes required us to modify hived itself to report some post rewards information as virtual operations so that the hivemind indexer could store this information into its database.
We also found during testing that we needed to add an additional virtual operation to hived to inform hivemind when a comment was deleted. The related hived and hivemind changes for this are expected to be completed in the next couple of days.
Another “discovery” we made is that we now need to implement hot and trending tracking directly in hivemind (previously hivemind obtained these lists via realtime calls to hived). Our first attempt at this had some performance problems, but we’re planning to implement a faster solution tomorrow.
Until now, we’ve been working with a relatively small number of tests for hivemind (60 tests currently), which were sufficient to keep us busy fixing bugs initially, but we just recently added two new programmers to our hivemind team that will focus on creating a more comprehensive set of tests.
Investigating solutions to SQL view issue
In my previous post, I mentioned a performance problem that arose when we used a query that referenced a SQL view instead of inlining joins directly in the query. We traced this slowdown to a default limit in Postgres on how “deep” the Postgres planner will go when looking for an optimal query plan. Introducing the intermediary view exceeded the default join_collapse_limit. By increasing this value from the default value of 8 to 16, we were able to obtain the originally expected performance. So at this point we’re just investigating the best way to make this change.
Initial hivemind sync performance testing
We’ve started measuring the time it takes for an eclipse hivemind to sync. Our automated test system already tests this for 5 million blocks, and it takes around 40 minutes there. But when we tried this on another similarly configured machine that we were planning to use to measure the time to sync to the chain’s current head block, it took nearly twice as long to sync to 5 million blocks, so we need to figure out what’s causing the discrepancy.
At this point, we have a couple of possible reasons for the discrepancy: 1) the database instances are “tuned” differently, 2) the slower system may be overheating causing the CPU to be throttled, or 3) the slow system is configured with nvme drives rather than SSD drives (which should be faster, but possibly there is some driver problem).
Plans for upcoming week
While doing more testing of the new snapshot feature that allows fast saving/reloading of state information, we found that we weren’t storing the state information from plugins (we were only storing the state information in the core code), so we’ll be adding a mechanism for signaling plugins to save off their state data and implementing this functionality for the account history rocksdb plugin.
As previously mentioned, we’ll also be adding virtual operations for deletion of comments and creating more hivemind tests.
Micro-fork handling in hivemind
While working in hivemind, we found that hivemind’s current handling of forks is less than ideal. The forked blocks themselves are discarded from the hivemind database, but the effects of the forked-out blocks are retained in the database. With the movement of more data from hived to hivemind, this “solution” becomes ever more problematic since hived’s internal fork recovery mechanism would previously mask some of these problems, so we’re planning to look at a more robust method of handling forks in hivemind.
After a couple of days of discussion, we have at least two viable methods to investigate: 1) storing additional data into the tables to allow the effects of blocks to be rolled back up to the last irreversible block or 2) using a 2nd copy of the database that’s filled up to the last irreversible block that we can switch to in the case of a fork, and then play the fork blocks onto this second database (the mechanics of this are a bit more complicated, but that’s the general idea). We’re generally favoring method 1 right now, since it seems likely to be faster, and we currently think it can be done without too much difficulty and without requiring ongoing code maintenance to support it (the solution is generic to the types of data being saved and updated in the database).
We don’t plan to work on improved micro-fork handling until after we’ve released a fully tested hivemind, since it’s not a feature that will be needed by third party apps during their own testing work.
Updated release estimates (TL;DR)
I’d hope we would have a release candidate for hived out this last week, but we had several hivemind bugs that required hived changes AND we still need to make the fix to the snapshot feature, so we’re pushing out the expected date for the release candidate by another week (next Monday). We’re planning to release hivemind within a few days thereafter, barring any further major bugs.
As mentioned in my previous report, we plan to schedule the hardfork itself for 3 weeks after the date we publish the release candidate itself.
Wow, I just found out you fronted coins to my mom’s account @kdtkaren, that was a big surprise, thanks!! 🤙 🤙
HIVE to the mewn! 🚀
Early to mid September sounds like a good release time.
For several reasons, I think it would be great to get some updates from the community perspective on these things - a "what it means/What to expect" kind of thing - especially if it has any impact on the end user layer or experience. The reason I think it is important is that firstly, it creates a lot of discussion and while sometimes messy, I believe this is vital for "buy-in". Secondly, it allows for users who aren't necessarily following all of this to find the information that concerns them through their network here.
Thanks for all your work and these updates :)
As far as impacts on the end-user layer goes, there's been several prior posts that discuss the visible changes of HF24 (and some of the less visible ones), but I suppose it can't hurt to summarize those changes again when we publish the release candidate of hived (hopefully next week). These interim posts are only intended to record problems and low-level design decisions we've made during implementation of HF24 features.
The closer it gets the more apprehension some users will feel and the communication (even repetitive) goes a long way. It doesn't have to be you if course, but it would be nice to get some of the witness views on where it is headed and some "predictions" of outcomes. It is a great opportunity for engagement and the development of interest in Hive from the less technical side.
Last I heard, I think an announcement of that sort was planned using @hiveio account.
Let’s keep on BUIDLing and improving our Hive Ecosystem
Sounds like continuous scope increase. :)
Would it make sense to leave out anything non-essential in Hivemind for after the initial release? My understanding is that non-critical bugs, optimizations and such can be addressed at any point in time - potentially after the hardfork, as continuous iterative improvements (something which hasn't been feasible to do for the blockchain code).
I am concerned that as these optimizations are being made, we are losing out greatly by not spending that time on adding Hivemind features which could make a tremendous difference for the platform. Here are some examples of such community-building features, but I'm sure ideas can be sourced from existing communities to see what will help them grow and offer incredible value to people currently using other platforms.
There's been no scope increase. It's more discovery of bugs in current implementation.
Thank you very much for the detailed report and for the effort!
Hivemind sounds very complicated. I thought it simply was a second layer database, which doesn't sound like something that generates tons of bugs.
But then again, I am no dev.
Hivemind is responsible for responding to most of the API calls used by 2nd layer apps, so it's a fairly substantial sized service. And it has gained considerably more responsibilities as part of the eclipse upgrade. Calling it simply a database is probably the wrong way to look at it. It "has" a database, but then so do most web apps. The complexity is better measured in terms of the amount of functionality it provides.
I also don't want to imply that hivemind's design itself is "complicated" in a "difficult to understand" sense, it's not. It's just that there's a lot of queries and changes to the schema often requires those queries to be update. And those queries were also updated to make them faster. Between these two changes, it's a lot of changes to test.
i am having trouble writing posts on my phone after updating the top and bottom view of the review. I have previously asked about this on discord.
This is what it looks like when I open the submit post page on my cellphone. I am using a samsung j brand 5. is there a solution. thank you
Yes, it's a known issue, we should have a fix for it deployed soon.
Is it also known that if one is writing a post in HTML and later tries to edit it, the original HTML code isn't shown in the editor anymore (also not if one chooses "View HTML source": for example then there are added many "<br>", so that the post looks completely different after editing it)?
That means to edit posts written in HTML I always use PeakD (which shows the code exactly as it was).
I just tested it with my newest post again, and it still doesn't work correctly.
Another little thing is that in the description of communities embedded HTML links don't work (in PeakD it works fine!).
You can compare for example "https://hive.blog/created/hive-101587" with "https://peakd.com/c/hive-101587/created" to see the difference.
I wasn't aware of it. You can file an issue here for it: https://gitlab.syncad.com/hive/condenser/-/issues
Done.
Keep up the good work @blocktrades, that is all that I cans say, we just needed a robust blockchain that is capable for foing many things for the service of the end users:D
Congratulations @blocktrades! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s) :
You can view your badges on your board And compare to others on the Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOPTo support your work, I also upvoted your post!
I'm surprised to read Hivemind requires a new VOP for deleted comments as there is already a
delete_commentoperation. Is this not enough?No, because this operation doesn't always succeed. The new vop was to report cases where the operation fails. For more info, you can look at code here: https://gitlab.syncad.com/hive/hive/-/commit/6b944337a007e7738e518961d77822806156e738
Need help! I could not publish my post on iPhone6 S plus. There are still problems with the title section as I could not see the whole sentence on my screen.
The tag section also has a problem, typed tags kept disappearing once I press on Save post!
Thank you very much.
Cheers.
I've added an issue report for this: https://gitlab.syncad.com/hive/condenser/-/issues/83
Thank you very much. I forgot to say that this was happening when using the new Ecency dapp.
Thank you very much.
Ah, that's a very important correction. Please contact ecency developer directly about the bug or file an issue in their repository, and I'll remove the issue from condenser/hive.blog repo.
Incredible what you do.
Excellent my friend, I take this opportunity to thank you for your support in my Post's. I hope one day to be as good the Hive as you 🙏
This is gonna sound crazy but I honestly love Blocktrades, super easy to use and I'm so glad you are here on HIVE!
Doesn't sound crazy to me, and I'm glad you like our service. Cheers!
Love your fast exchange 💱 thank you for this💗
Just want to say thanks for all that you do. It seems like the Hard Fork went well from what I can tell so far!
woo!
woo!
Hi @blocktrades, I'm just curious but I discovered that you have delegated over 500K Hive Power to the account @usainvote. This account is also downvoting content on Hives Trending Page.
What is your purpose doing so? Since you are one of the most influential accounts here on Hive.
@blocktrades, I have been unable to post from your front end for 3 or 4 days! I keep getting Transaction broadcast error: 837. What does that mean and how do I fix it? I can post through peakd, but it takes much time for the posts to show up (2 hours today).
I'm familiar with your front end and don't enjoy having to switch to peakd in order to post.
It would be handy to have a list of ALL the front ends listed somewhere on the home page so that the community has options when one front end is malfunctioning. I'm only aware of the 2 front ends but I suspect there may be more.
Is there any chance that the (amazing) personal (decentralized) "blacklists" could make the "blacklist owner" invisible ("muted") to the accounts on their personal (decentralized) "blacklist" (in order to mitigate potential harassment)?
There's no really easy way to do this on the server side, given the current implementation. One possibility would be to encrypt the lists, then decrypt them on the client side, and let the client do the filtering. But it would be a big change.
But this could make more sense in terms of some of the things I'm planning for the reputation system, and we might be able to use that for filtering (again, it would imply the filtering is done on the client side, instead of hivemind).
Thank you for your consideration.
I think that a lot of people enjoy using fa.ceb.ook and other systems because they have a "friends only" option.
I've met a few very interesting people on HIVE who've since abandoned the platform specifically because they feel like they've been unfairly harassed with downvotes.
I was very happy to see the customizable decentralized "blacklists".
Do you know where I can find a good description of exactly what effect these individual "blacklists" have when someone is added to them?
Here's a description of how blacklists function: https://hive.blog/hive-102930/@blocktrades/hive-improvement-proposal-decentralize-blacklists-on-hive