Hivesearcher vs Hivemind Search

in #hivelast month (edited)

In this post, I would like to talk about Elasticsearch and Postgresql search and their differences.

As you know, we have Elasticsearch implemented on Hive which is being rebranded as Hivesearcher (previously Esteem Search). System has been running for almost 2 years now without a glitch. As of this writing, there is no Hivemind or Postgresql based search engine on Hive yet, so this comparison is theoretical based on our limited tests/knowledge.

Hivesearcher_vs_Hivemind_search

Elasticsearch is based on Apache Lucene as a core, adds REST API and few other improvements. Comparison should really be Lucene vs Postgresql.

Lucene is built by Apache just for one purpose, to make scalable, high-performance indexing and full-featured text search engine.

Postgresql is robust, relational database with SQL compliance and focuses on extensibility.

You can also check this page to see their differences. As you can see, they both built for different reasons and for different goals.

When choose Postgresql

We have been using Postgresql on our backend way before Hivemind came to live. And were glad to see Hivemind being written in Python and Postgres which is exactly what we use on our backend stuff for Esteem/Ecency. So we use Postgres for almost everything but not for search. We considered Postgres when building search engine because our backend was in Postgres and Hivemind was also using Postgres, so you would think it would make sense to just use same technology. After careful consideration and long term goals, we choose to go with the best search engine tech which is Elasticsearch. However, if you are web or mobile developer, consider using Postgres, it is one of the best database engine out there for relational data. You won't regret it because of opensource, community size and tools, extensions, it is growing.

When choose Elasticsearch

Elasticsearch can also be used as datasource/database but it is not designed for that purpose and you will fail miserably if you use it as datasource. What Elasticsearch does best is building search engine and index large dataset, analyze queries. So perfect for large and long living content Hive produces.
It is #1 search engine for a reason.

Postgres Full-Text Search bring quite good improvements and can offer good alternative search. But it is very hard to scale and also I don't think, one can perform complex queries like "esteem surfer" -"monthly digest" -giveaway author:good-karma tag:esteem,wallet type:post. On the other hand, elasticsearch is easy to scale and you can perform any type of complex queries.

Query above searches for posts from @good-karma with two tags together #esteem and #wallet having exact match of "Esteem Surfer" phrase but excluding posts that include "monthly digests" or "giveaways". You cannot simply do this with Postgresql, you will end up creating indexes for very different use cases and end up with large and clunky performant data.

Creating search based on Postgres would require separated tables and indexes, clean body field because it includes html + markdown combine cleaned post body + post title to search a term in multiple fields at the same time.

Adding search into Hivemind is another story altogether because then we might be affecting performance of regular API queries. Postgres uses shared_buffers which means it tries to complete each task in shared buffers. We all know hivemind servers are quite busy with reorganizing active blockchain data and if search is added into it, those search results will take some place in shared buffers. In result, it will reduce hivemind's performance and rpc nodes.

So Postgresql doesn't offer a magic wand nor does Elasticsearch. But tools are used best for their designed purpose.

Support Hivesearcher proposal to help us continue to improve and provide open API to Elasticsearch instances which is used by one of your favorite tools on blockchain hive.blog, peakd and esteem/ecency.

Support proposal

Sort:  

Congrats for improving the platform!
What’s your opinion about the measures of our new normal?

Thanks! If you could be more specific

Last year we didn't wear masks.

  • What's different this year?
  • How many people ask questions?

Our leaders imposed some measures that reduce our liberties so

  • they have the burden of proof.
  • they have the responsibility to offer us the proof for their measures.

Without proof 1 theory is just 1 speculation.

What's the proof?

By the way: so far i didn't receive 1 answer to this question.
Everybody ignores it.
People are more or less creative in ignoring our reality.
i have a hunch: ignorance is not a virtue.
i ask this question in the idea that eventually somebody might enlighten me.

Hey there,

your question is oddly phrased and I am not even sure if I understand.

Everything starts from a theory, and with evidence, it becomes the truth.

The new normal is a fancy term for moving on, which is the meaning of growing up. I am not sure if I can comment on this. :)

Do you know what's the evidence?

The evidence of what? Sorry, but I can't seem to align myself with your thoughts.

People who create the measures of the new normal claim we have an invisible enemy. They make a scientific claim so they have the burden of proof.
Do you know what's the evidence of our invisible enemy?

In regards to the invisible enemy, are you referring to the coronavirus?

I've seen (and heard) many people in hospital camps during the peak death rate of March - April. There is obviously an issue and a lie to this scale is near-impossible to cover-up. I do not have any acquaintance that are researching this matter, but I do believe that the coronavirus exists. I doubt limiting our liberties will benefit the government and you can see that Trump is desperate to bring the US economy back on track in order to be elected this year.

These are my opinions on the subject

@good-karma , @esteemapp

I just noticed that the app is now showing some of the notifications like recent comments and replies.
I’ve try to change the saver and still the same.



106503807_3081346178567603_7135964424781606109_n.jpg

Thanks for sharing

Congratulations @good-karma! You received a personal badge!

Happy Hive Birthday! You are on the Hive blockchain for 4 years!

You can view your badges on your board And compare to others on the Ranking

Support the HiveBuzz project. Vote for our proposal!

I'm sure it is difficult to do search on the ever changing blockchain Database. For my personal use searching for old posts etc would be my main use I am sure it could be forgiven if search results had some sort of probability attached as the query refers to time sensitive info. It is really an interesting challenge. If I have not already voted proposal I will.

Oh yea that's great improve.

Hey why don't mirgethem