I don't really agree that there's not a real memory usage reduction. The memory is in fact allocated by the software, and hence isn't available to other programs, regardless of whether it is actively storing data or not. Further, I don't think it makes sense to compare it a snapshot load, because that is at best a temporary thing, not reflective of how the software will behave over time.
You are viewing a single comment's thread from:
What I was trying to say is that in optimized version hived requests to allocate more memory (because it rounds up to the size of whole pool, even if space might happen to never actually be used later), as opposed to next potential task that focuses on storing the same data in more compact form, thus leading to smaller index nodes and requests to allocate less memory.
Well, the underlying mechanism of why we get lower effective memory allocation is the same - objects are placed better in memory. In case of snapshot load it just happens because objects of the same class are allocated one after another, while pool allocator guarantees such behavior. That's why I said
except now the benefit persists instead of degrading over time.It is funny how the mind works. I wasn't consciously thinking about it, and yet I think I figured out why voting for old comments, when the data might need to be pulled from disk, might be faster than voting for fresh comments. Now it is kind of obvious, but since I expected reaching for archived comments to be two or three orders of magnitude slower than using multiindex, the results showing it was faster clouded the most likely reason.
When voting for archived comments all hived needs to do is to determine the target comment exists and then the work ends. On the other hand voting for fresh comments is actual voting, calculating mana to consume, strength of the vote, creation of related state object, updating data in multiple other objects, and then all that work is undone, redone once again during block production, undone again and performed yet again during block application. In case of archived comments it also needs to reach for it three times, but second and third time it reaches into cache. For a fair comparison we'd need to measure just access to comment in isolation from everything else.
If above turns out to be correct explanation of observed behavior, it would put me at ease, because it means there is no danger in using that optimization even on witness nodes, and even for saturated traffic with big blocks. I might still try to implement the in-memory version of archive if only to test whether Mariusz designed the interfaces correctly :o)