Last week we continued work on post HF24 optimizations. Below is a summary of the work done last week and our plans for the upcoming week.
Hived work (blockchain node software)
We completed changes to the get_block_api plugin to speedup performance for the get_block API call. The old implementation used an overly pessimistic mutex locking scheme that severely degraded potential performance under loading. We’ve also added a new API call to fetch a consecutive set of blocks in one disk read operation.
https://gitlab.syncad.com/hive/hive/-/merge_requests/149
Unfortunately, due to delays related to several hash computations (including one associated with getting the signee key of the block) and conversion of the binary block data to fc::variants and later to json, these API calls are still too slow for the needs of hivemind initial sync.
To overcome this problem, we’re now creating a hived plugin that can directly write the needed data into hivemind’s database during hive reindexing and normal block reception. Most of the data being provided by get_block_api is of no interest to hivemind, so using this API to get the data was unnecessarily wasting CPU, in addition to slowing down hivemind. I expect that using the plugin approach will lead to significant speedup in the initial sync time for hivemind (my guess now is 2x at least) and it should also reduce normal hivemind live-sync write time.
Hived status
We’ve tagged a version v1.24.7 which contains the new get_block_range(starting_block_num, count) API call as well as the fix to make the legacy get_account_history API use 0-based operation index like the get_account_history_rocksdb plugin does.
v1.24.7 is a recommended upgrade for API node operators, but it doesn’t contain changes needed by witness nodes or exchanges. It doesn’t require a reindex from the previous v1.24.6, as it is strictly contains API-related changes.
Hivemind (2nd layer microservice for social media applications)
Most of our Hive devs continued to working on hivemind last week. Below are some of the merge requests incorporated into the develop branch of the hivemind repo:
performance optimizations:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/365
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/366
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/373
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/367
various bug fixes:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/354
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/362
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/371
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/364
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/337
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/370
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/382
new tests:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/351
test system improvement:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/376
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/374
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/375
Condenser + Condenser wallet (open-source code for hive.blog)
The bug editing comments was fixed:
https://gitlab.syncad.com/hive/condenser/-/merge_requests/151
Fix for wallet proposals page where non-burn proposals were being displayed as burn proposals:
https://gitlab.syncad.com/hive/wallet/-/merge_requests/75
We’re also testing and fixing bugs in the UI for creating decentralized lists:
https://gitlab.syncad.com/hive/condenser/-/merge_requests/155
https://gitlab.syncad.com/hive/condenser/-/merge_requests/156
Nearing completion of airdrop proposal re-payment
I’ve refunded the overage of Hive from HBD→Hive conversions to the hive.fund account (when I converted some of the HBD to Hive, there was a slight overage of Hive due to the unpredictable amount that the conversion will yield). Once the proposal completes, I will compute and refund the HBD overage back to the hive.fund account as well.
What’s the plan for next week?
- Finish hivemind and condenser muting changes. We’ve partially completed an additional change to condenser to allow a condenser operator to specify a “default observer” when a user isn’t logged in. This allows the condenser web site operator to specify their own initial list of spammers to be muted (for example, to protect their SEO results) that will function prior to a user’s login. For example, we’ve using the @hive.blog account as the default observer for the hive.blog site.
- Enable reputation calculation code.
- Continue creating hivemind tests.
- We’re prioritizing the speedup of hivemind full sync via a hived plugin as the slow sync time has a big impact on the speed of hivemind CI (which sets an upper limit on how fast we can validate changes).
- Still hoping to get out a post with a comprehensive list of planned changes for HF25 and for 2nd layer improvements soon (although the hived plugin for dumping directly to PostgreSQL is one of the planned major 2nd layer improvements: it’s just getting fast-tracked because it impacts our current work a lot).
A post with a comprehensive list of planned changes (and rough timelines?) for HF25 would be great. From my perspective, having a focused plan would go a long way, but I recognize one might exist and I just missed it.
Nope, you haven't missed it. We've just been so busy working on finishing up current work we haven't had time to write down the plan for things to work on next.
Is this the cause of the authorization errors we’ve been having lately?
LOL absolutely annoying, writing a comment about it and it happens.
No, it's completely unrelated. That error is generally associated with an improperly serialized transaction. What application are you using to create the comment?
I get it while using hive signer when I'm on PeakD (been meaning to figure out id keychain is available on the mobile version of brave) but it's also been happening on the Ecency app. I don't think it's happened on my laptop where I use keychain.
Good to know that it's not an inherent issue with the chain! It's annoying for me but I just want to make sure new people don't get it, get pissed and quit because they don't understand.
Good karma has looked at it on the Ecency side but I don't think they've had success finding a good cause for it yet, logs haven't been showing anything abnormal. Changing nodes usually fixes it but it took me a bit to figure out a node that would work.
Appreciate your update and work though! Thanks for the reply.
My best guess is that the node is timing out when doing the transaction and you're just getting an erroneous error, maybe because it's too loaded. But it's hard to say without analysis. First step would be to look at the network panel and capture the failing api call. Then you can do isolated testing on api nodes, and check the response times to see if it's a timeout or something else.
I knew it! BCH is one of the things that you need to update also... ^_^
When vacation?? Get some rest, you've never stopped working since the start of Hive. It might also give you some perspective on HF25 ideas.
Optimizations and bug fixing is a never ending job at some point we have to move on.
From what I've been told, "when vacation" is likely to be sometime in 2nd to 3rd quarter of next year, based on current travel issues.
Nice post! Thanks!
Congratulations @blocktrades! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s) :
You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @hivebuzz:
!tan
Congratulations, @asimo You Successfully Trended The Post Shared By @blocktrades.
You Utilized 2/3 Daily Summon Bot Calls.
TAN Current Market Price : 0.300 HIVE
Vote my post please
Congratulations @blocktrades, You Earned 0.682 TAN & Curators Made 0.477 TAN.
Join CORE / VAULT Token Discord Channel or Trade TANGENT Token
TAN Current Market Price : 0.320 HIVE
How is Dtube going? Long time no see.
Any plans to work on hive and maybe put a proposal up?
Lucky you! There's more in my state.