A few days ago @splinterlands opened the Chaos Legion edition general sale for everyone. Some players have already been able to purchase packs during the pre-sale, but the bulk of the purchases will happen during this phase.
As part of the general sale some players and a few guilds are expected to get a large amount of packs and expand their collection. So we decided to do some work on PeakMonsters and try to improve the performance of some pages to handle the increased amount of data that those accounts will have to manage.
🚨 Warning 🚨
The rest of this post focus on technical and development details. If you are not a developer you can just skip most of it and try out peakmonsters.com to enjoy a much more responsive experience than what used to be 😊
Or have a look at this post and the great comparison video recorded by @mozzie5: https://peakd.com/hive-126911/@peak-monsters/peakmonsters-my-cards-page-becomes-lightning-fast
In this post I'll share some details about PeakMonsters and the tech stack we use, the problem we faced managing a lot of data in the browser and how we solved it.
Brief Overview
Developers and engineers that are active on Hive already knows that the frontends for both PeakD.com and PeakMonsters.com are built using Javascript and Vue.js. I created those apps a few years ago and the codebase is quite large at this point so it's not easy for me to upgrade to Vue 3 and we are still using Vue 2. As many other projects using Vue we rely on a few additional libraries, the main ones are Vue Router and Vuex.
The backend for both apps is not very large and not relevant for this post, so let's skip it for now as I can share some more details in a later post.
PeakMonsters heavily rely on API provided by Splitnerlands itself and most of the data are saved in the Vuex stores and shared between multiple pages and components. Vuex is great for this use case and it's very easy to fetch data from the API, store everything once and read/update as the user interact with the website. And all changes are automatically reflected everywhere in the UI that is a nice perk. But this also means that to load pages for a player with thousands of cards the browser will have to do quite some work (I did a few tests in the Splinterlands QA environment with 500k+ cards and the network payload for some API response was about 5.5MB !!).
As we added more features to the website (sorting, filtering, ...) some actions started to feel slower. This is not very noticeable on normal accounts, but for larger ones the responsiveness was not always great.
With the Chaos Legion edition we realized some of those large players would increase their collection considerably, and some new investors or guild would also be in the condition to manage a very high number of cards. So I started testing a couple of possible solutions.
How much reactivity is too much?
The above benefits unfortunately are not for free. To provide reactivity Vue (and Vuex) have to attach observers to all those objects (mostly cards in case of PeakMonsters). This means that even 15k cards stored in the browser can generate quite some load and consume lot of memory because there will be observers attached to all of them. The overall impact is going to be even higher when the cards are filtered or sorted because multiple copies will be kept in memory.
After a few tests it was clear that the above behavior was not sustainable for very large card collections and was the cause of pages overall slowness and freezes.
Note that this is different in Vue 3, because the new paradigm is to declare reactivity explicitly: https://v3.vuejs.org/guide/reactivity-fundamentals.html#declaring-reactive-state
How to solve it?
Solution turned out to be simple, if reactivity is slowing down everything just remove it (or to say it better reduce) 😁
Reduce reactivity in Vuex stores does not require a lot of code. You just "freeze" the javascript objects so properties cannot be changed.
This means a classic store:
const state = {
items: []
}
const mutations = {
setItems(state, items) {
state.items = items
}
}
Can be freezed using the Object.freeze method:
const mutations = {
setItems(state, items) {
state.items = Object.freeze(items)
}
}
The above change will drastically reduce the observers Vue has to manage because only the items array will be watched, and not every single objects stored into the array itself. Suppose you have an array with 10k elements, it means going from 10,000 + 1 observers to just 1.
But of course there are a couple of problems to face after this change:
- Possible code changes in multiple parts of the application
- Find another way to propagate changes to the underlying data to the web pages
Good news here is that data fetched from the Splinterlands API are read frequently, but updated only after specific actions that modify the state of those cards (transfers, listing on the market, lock, ...).
To reflect such changes the code used to be something like this (in the following snippet we are setting the price for a card listed on the market):
setItemsOnMarket(state, {uid, price}) {
const item = state.items.find(i => i.uid === uid)
if (item) {
item.buy_price = price
}
}
}
But without reactivity on array items the whole array has to be overwritten to allow Vue to "see" the change and update the pages accordingly:
setItemsOnMarket(state, {uid, price}) {
state.items = state.items.map(i => i.uid === uid ? {...i, buy_price: price } : i)
}
Turned out that even for very large arrays creating a copy on the fly is pretty fast. And as all changes on data stored in Vuex are performed by mutations the changes required to the code are limited to the impacted stores without any change at all in the external components 🎉
This change has been live on PeakMonsters.com for a few days now and I've not seen any major drawbacks of this approach. And more important all feedback we got from users so far are great.
I may even try to apply the same to PeakD.com, also if I don't expect to see the same drastic improvements because PeakD does not store the same amount of data. But maybe it's still worth it in a few places ;)
That's all for now, see you next time 👋
If you like what we do and you would like to get in touch feel free to reach out in our Discord server or if you are a developer have a look at this post.
How do you handle rate limiting from the Splinterlands API servers? My understanding from this is that in the background each user would be fetching from the API from their own IP since the calls appear to be front-end, but at some point don't the Splinterlands API servers 429?
Yes, this is correct. Calls originate from the user browser and the standard rate limits apply.
This happens from time to time. We just try to not overwhelm the API with too many calls 😁
Congratulations @asgarth! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s):
Your next target is to reach 28000 upvotes.
You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOPCheck out the last post from @hivebuzz: