The get_account_history API call is based on a number for each transaction. So for my account with 130k transactions I can, for example, make a call for a group of transactions starting from number 130000 and going back to 129000 (it goes backwards).
However, these numbers can jump around. If this happens then when I call the next group of transactions (129000 to 128000) transaction 129000 may not be the same as in the previous call.
This could result in a history overlap (and duplication issues - less of a problem) or a gap (and missed transactions - more of a problem).
This is rare on the accounts that I've tested on Hive. On my account it doesn't happen at all. Similarly for accounts up to 500k transactions it doesn't seem to happen. It looks like it may be more of a problem on larger accounts, particularly very large accounts.
To prevent the loading issues described above I chain each batch of transactions together using the block number (plus more detailed transaction id info). So if transaction 129000 (the anchor transaction) is a different transaction in each of the two API calls the check fails.
At this point the load is stopped. My assumption has been that the jumping is temporary and would revert after a short delay, allowing the load to be restarted. However as the problem comes and goes it's difficult to test explicitly.
The issue is further complicated by 'empty' call returns which are caused by long periods with no financial transactions in the account history (lots of json for example). In this situation there is no anchor transaction and the 129000 number becomes more important. Jumping around at this point becomes more problematic.
It's something on my list to improve. I should be able to solve it eventually! @blocktrades team is currently working on the account_history side of things so this is going to lead to some changes that may make things easier.
Posted Using LeoFinance Beta
No thats inaccurate. Transaction numbers should be the same on every call. If you start at trx 12000, it should start at the same place every single time, since new transactions are added at higher indexes and you only go towards lower indexes.
Posted Using LeoFinance Beta
The issue is that in one call a transaction is being given the number 2484376:
But in another call that transaction is being given the number 2484378:
So I agree it's not a case of new transactions interspersing themselves inbetween the existing transactions since, as you say, new transactions are added as the end and the loading is backwards.
It's some kind of re-indexing issue. Perhaps indirectly caused by new transactions being added forcing a re-indexing? It needs to be something that mainly affects larger accounts (which typically have more frequent transactions).
Or the call being directed to a different source which has different numbers. A different node somehow? My knowledge of how the API returns are actually generated is sketchy.
Posted Using LeoFinance Beta
Oh thats actually really weird. Do you know if 2 different api nodes were used in there or if it was the one? There's a chance different nodes might have different actions at different indexes. Account history is being redone so it should hopefully be better in the future. On my call using @deathwing's node I got it at index id in the picture.
I'm just using 'https://api.hive.blog' for all calls.
It happens on Steem as well so I think it must be a pretty old issue.
Posted Using LeoFinance Beta
Try using https://api.deathwing.me for now. This should hopefully be fixed on launch of new account history.
OK, that's great advice thanks!
I used one of Roeland's nodes in some of my testing which may explain why I didn't see this error occurring as much previously.
Posted Using LeoFinance Beta