Update: Filtering of operations returned by the get_account_history API call.

in STEMGeeks3 months ago

gears_blockops_pink.png

OVERVIEW

A few weeks ago I posted about a hugely useful change to the Hive "get_account_history" API call made by the @blocktrades core coding team. As a reminder, this history API originally returned all operations made by a particular account. The change allowed for pre-filtering of the results by transaction type (vote, comment, transfer etc) thus drastically reducing the amount of data transfer if you are only interested in a certain subset of transactions.

You can see the original post here:
https://hive.blog/hive-139531/@miniature-tiger/filtering-of-operations-returned-by-the-getaccounthistory-api-call

A couple of updates have been made to this API since my original post. I've also been carrying out some testing of the API related to my Heystack portfolio tracker (which was my STEMGeek Hackathon Entry back in the spring). I'll cover some interesting points below.

LIMIT (NUMBER OF TRANSACTIONS) PARAMETER

In my last post I noted that you always get 1 more transaction than you ask for. This has now been corrected.

If we consider only the basic parameters:

  • account (string): the account name, e.g. "miniature-tiger";
  • start (int): the starting transaction number of the batch (which can be set to -1 for the most recent transaction); and
  • limit (int): the number of transactions in the batch (a max of 1000).

Previously ["miniature-tiger", -1, 0] would have returned my most recent single transaction.
Now ["miniature-tiger", -1, 1] will return my most recent single transaction.

This is much clearer, for me at least.

2000 SEARCH LIMIT AND ERROR HANDLING IF NO TRANSACTIONS FOUND

Something I didn't cover in my original post was the 2000 transactions search limit for filtered operations.

There is a limit of 1000 transactions for a single account history API call.

If you are requesting all operations then this is fairly straightforward. The prior 1000 operations from the start parameter are returned, ie:
["miniature-tiger", 99999, 1000] will return transactions from 99000 to 99999 (i.e. 1000 inclusive).

If you are using the filtering then up to 2000 prior transactions will be searched:
["miniature-tiger", 99999, 10, 2] will return the last 10 comment transactions (since the 2 parameter represents comments) counting back from 99999 with a maximum search back to 98000 (i.e. 2000 inclusive).

So you might get 10 comments from 99990 to 99999 (if you just made 10 comments!) or you may only get 5 comments returned if you have only made 5 comments out of your last 2000 transactions.

If you have made 0 comments out of the last 2000 transactions then you will get an error returned along the lines of:
"Could not find filtered operation in 2000 operations, to continue searching set start=98000".

Personally I could live without this error message, with just a return of an empty array. But perhaps it is useful for other users.

There is a fuller explanation here for anyone interested in the logic:
https://gitlab.syncad.com/hive/hive/-/issues/98

TESTING ISSUE: THE SAME API CALL CAN RETURN DIFFERENT NUMBERS OF TRANSACTIONS

The main reason I'm interested in the "get_account_history" API call is because I use it for the Heystack portfolio tracker that I've been developing. Essentially I use the API to extract all the relevant financial operations from Hive for an account and combine this with a user's trade data from exchanges to provide a view of a user's crypto trade history and holdings.

As a starting point on the Hive data I obtain the total number of Hive transactions for an account using the [account_name, -1, 1] call and, once the data is loaded, I can use this call again in future to see if new transactions need to be loaded.

However, as part of my testing, I noticed that for one account the total number of transactions was moving up and down each time I refreshed the call, flicking between 6413 and 6673 total transactions, even though I wan't changing the parameters or the node.

Through investigation I managed to obtain the full operations list for the account (using no filtering), once under the 6413 total operations count and once with the 6673 operations.

The issue appears to be that the history API is pulling through some blocks of transactions for an account twice (these transactions are not repeated in the blockchain - this is a different issue that can also occur).

For example here we can see two examples, both votes, that are pulled through twice in the API history. The "historyId" shows the transaction number provided by the history API call. A check of the block history shows these votes are not repeated in the blockchain.

repeatIssue2.png

So something to be aware of! If anyone knows why this issue occurs I would be interested. If you want to check the issue yourself the account I found the issue for is: austinsteinbart

That's about it! Hopefully useful info for anyone out there using this API!

Posted with STEMGeeks

Sort:  

This is indeed a big improvement.
@brianoflondon & I have been looking for a way to get account holder's balances on long ago dates because this will be needed to calculate damages claims in the Crypto Class Action against Facebook & Google.

Yes, historic account balances should be available from HeyStack for most accounts (although for the oldest accounts I haven't implemented things like mining - perhaps in the future).

Here's what it currently looks like for your account:

apshamilton account balance.png

  • Figures in USD based on @coingecko prices.
  • Dark blue is SP, light blue is Steem, grey-blue is SBD
  • Dark red is HP, light red is Hive, grey-red is HBD

That's really cool.
When will it be available to use?

There's a couple more weeks of work to do, although spread over time, so I would hope to have it out before Christmas.

Thanks for your work, it helped me to add this feature to beem. At the moment it is only used on https://api.hive.blog.

from beem import Hive
from beem.account import Account
hive = Hive("https://api.hive.blog")
account = Account("miniature-tiger", blockchain_instance=hive)
votes = list(account.history_reverse(stop=-1000, use_block_num=False, only_ops=["vote"]))

This returns all votes in the last 1000 account operations using operation_filter_low and operation_filter_high. The numbers are calculated in:

operation_filter_low, operation_filter_high = account._get_operation_filter(only_ops=["vote"])

Cool! Can you add more than one operation to the only_ops array? Like ["vote", "comment"]?

Yes this works also. Do you know when the update is applied to all API full nodes?