Peerplays_Blockchain/peerplays_migrated

Fork 0

Smartfee and libbitcoin #456

New issue

Closed

opened 2022-09-27 06:09:39 +00:00 by bobinson · 10 comments

bobinson commented

2022-09-27 06:09:39 +00:00

(Migrated from gitlab.com)

Bitcoin clients uses estimatesmartfee end point provided bitcoind (bitcoin-core) to set the fee rate while submitting the transactions. However the libbitcoin is missing this end point.

This means, we need to find a mechanism to estimate the fees while submitting transactions so that they are confirmed within acceptable time frame.

The best method seems to be implementing our own mechanism however to understand the scenario more, we are proceeding with the following steps

Proof of concept smart fee estimation using public nodes
testing of libbitcoin implementation and above proof of concept with public nodes to start ASAP !
Testing with Bitcoin MAINNET must be done using the libbitcoin and both hardcoded fees and proof of concept with public nodes
Other methods including using few hundred blocks or others must be explored & estimated
final decision on what needs to be deployed.

Bitcoin clients uses `estimatesmartfee` end point provided bitcoind (bitcoin-core) to set the fee rate while submitting the transactions. However the libbitcoin is missing this end point. This means, we need to find a mechanism to estimate the fees while submitting transactions so that they are confirmed within acceptable time frame. The best method seems to be implementing our own mechanism however to understand the scenario more, we are proceeding with the following steps 1. Proof of concept smart fee estimation using public nodes 2. testing of libbitcoin implementation and above proof of concept with public nodes to start ASAP ! 3. Testing with Bitcoin MAINNET must be done using the libbitcoin and both hardcoded fees and proof of concept with public nodes 3. Other methods including using few hundred blocks or others must be explored & estimated 4. final decision on what needs to be deployed.

bobinson commented

2022-09-27 06:09:39 +00:00

(Migrated from gitlab.com)

assigned to @hirunda

bobinson commented

2022-09-28 19:24:49 +00:00

(Migrated from gitlab.com)

assigned to @bobinson

hirunda commented

2022-10-10 11:53:09 +00:00

(Migrated from gitlab.com)

Proof of concept with public nodes to start ASAP is done (the code is there which consults external nodes and gives the estimate fee for the target block).

The status for the custom implementation of getting smart fee is as following:

Still waiting for the libbitcoin server to complete the sync and see does the approach of inspecting trx fees within few (~20) last blocks and finding some medium of fees will be close to the one which gives bitcoind and public nodes as a smart fee estimation.

Proof of concept with public nodes to start ASAP is done (the code is there which consults external nodes and gives the estimate fee for the target block). The status for the custom implementation of getting smart fee is as following: Still waiting for the libbitcoin server to complete the sync and see does the approach of inspecting trx fees within few (~20) last blocks and finding some medium of fees will be close to the one which gives bitcoind and public nodes as a smart fee estimation.

hirunda commented

2022-12-20 00:07:15 +00:00

(Migrated from gitlab.com)

First has to be finished #483 . In the meanwhile I will try to find some public server on which I can implement fallback estimate fee , by finding average fee for all transaction within last 20 blocks.
Note that we already have implemented finding estimate fee from external sources. If I find public server I will unblock ticket and start with the implementation.

First has to be finished #483 . In the meanwhile I will try to find some public server on which I can implement fallback estimate fee , by finding average fee for all transaction within last 20 blocks. Note that we already have implemented finding estimate fee from external sources. If I find public server I will unblock ticket and start with the implementation.

bobinson commented

2022-12-22 07:18:26 +00:00

(Migrated from gitlab.com)

mentioned in issue #498

hirunda commented

2023-01-16 12:46:28 +00:00

(Migrated from gitlab.com)

@bobinson on my machine I have the entire database. I already started with estimation fee by examining incoming blocks. For now I just added finding the average fee. For better metric I will introduce K-means algorithm by groping values around three centeroids. I am also going to provide data sheet for the results I get from external sources and internal (examining the blocks and applying K-Means)

hirunda commented

2023-02-01 10:38:04 +00:00

(Migrated from gitlab.com)

The final solution is done, which introduce listener for transaction events. Trxs which we receive before block are still not confirmed. We consider these transactions as transactions in memory pool. When block is received we take average from transactions which are in memory pool and which are included in block. This solution gives a good results which are almost aligned with recommended fees from external sources.
For more accurate results we could consider later solution which will fully implement smart fee estimation algorithm from bitcoin core.

The final solution is done, which introduce listener for transaction events. Trxs which we receive before block are still not confirmed. We consider these transactions as transactions in memory pool. When block is received we take average from transactions which are in memory pool and which are included in block. This solution gives a good results which are almost aligned with recommended fees from external sources. For more accurate results we could consider later solution which will fully implement smart fee estimation algorithm from bitcoin core.

bobinson commented

2023-02-01 10:45:11 +00:00

(Migrated from gitlab.com)

When you say almost aligned, can you give few examples ?

hirunda commented

2023-02-01 10:57:13 +00:00

(Migrated from gitlab.com)

As a external reference I used https://bitcoiner.live/api/fees/estimates/latest.

4000 from external source within one block
3600 from our algorithm

4000 from external source within one block
5200 from our algorithm

7000 from external source within one block
5000 from our algorithm

It's usually lower than external source ... But the value usually fits between 30 - 120 from the external source I gave above.

As a external reference I used https://bitcoiner.live/api/fees/estimates/latest. 4000 from external source within one block 3600 from our algorithm ------------------ 4000 from external source within one block 5200 from our algorithm ------------------- 7000 from external source within one block 5000 from our algorithm It's usually lower than external source ... But the value usually fits between 30 - 120 from the external source I gave above.

bobinson commented

2023-07-24 09:10:08 +00:00

(Migrated from gitlab.com)

implementation completed

bobinson (Migrated from gitlab.com) closed this issue

2023-08-08 12:11:33 +00:00