exploring connection pool for SONs

bobinson commented

2022-07-06 12:10:32 +00:00

(Migrated from gitlab.com)

current SON > external chain uses one API node
If the API node fails or slows down, this can have adverse impact on the sidechain functionality
Ideally there must be a connection pool and most healthy node must be selected

- current SON > external chain uses one API node - If the API node fails or slows down, this can have adverse impact on the sidechain functionality - Ideally there must be a connection pool and most healthy node must be selected

bobinson commented

2022-07-06 12:10:33 +00:00

(Migrated from gitlab.com)

assigned to @bobinson

serkixenos commented

2022-10-11 00:25:47 +00:00

(Migrated from gitlab.com)

assigned to @timur.5

timur.5 commented

2022-11-23 15:05:42 +00:00

(Migrated from gitlab.com)

!181

timur.5 commented

2022-12-24 20:57:39 +00:00

(Migrated from gitlab.com)

Testcase:

Use feature/SON-connection-pool branch until it's merged.
Follow "Requirements" section of README.md
to install python testing framework.
Change directory:

cd peerplays-utils/peerplays-qa-environment/e2e-tests

Stop QA environment:

./main.py --stop

Expected output:

=====                              Stopping all docker containers                              =====
-----                              All docker containers stopped                               -----

Start QA environment:

./main.py --start all --all-in-one

Expected output:

=====                            Starting Bitcoin docker container                             =====
-----                     Docker Bitcoin started, docker id is 440bd60de4                      -----


=====                               Initializing Bitcoin network                               =====
-----                               Bitcoin network initialized                                -----


=====                            Starting Ethereum docker container                            =====
-----                     Docker Ethereum started, docker id is e4c2c46c91                     -----


=====                              Initializing Ethereum network                               =====
-----                               Ethereum network initialized                               -----


=====            Starting Hive docker container and waiting for the Generated block            =====
-----                       Docker Hive started, docker id is 9fe165e1d0                       -----


=====                                Initializing Hive network                                 =====
-----                                 Hive network initialized                                 -----


=====         Starting Peerplays docker containers and waiting for the Generated block         =====
-----                    Docker Peerplays started, docker id is 66dac8c4f0                     -----


=====                              Initializing Peerplays network                              =====
-----                              Peerplays network initialized                               -----
...

Change to some empty directory, say 'mydir':

cd mydir

Copy 'witness_node', 'cli_wallet' executables, 'config.ini' and 'genesis.json' from container to 'mydir':

docker cp -L peerplays-qa-environment_peerplays-all-in-one_1:/home/peerplays/peerplays-network/witness_node ./
docker cp -L peerplays-qa-environment_peerplays-all-in-one_1:/home/peerplays/peerplays-network/cli_wallet   ./
docker cp    peerplays-qa-environment_peerplays-all-in-one_1:/home/peerplays/peerplays-network/genesis.json ./
docker cp    peerplays-qa-environment_peerplays-all-in-one_1:/home/peerplays/peerplays-network/witness_node_data_dir/config.ini ./

Change directory:

cd peerplays-utils/peerplays-qa-environment/e2e-tests

Stop QA environment:

./main.py --stop

Start QA environment except Peerplays container:

./main.py --start btc eth hive

Expected output:

=====                            Starting Bitcoin docker container                             =====
-----                     Docker Bitcoin started, docker id is 440bd60de4                      -----


=====                               Initializing Bitcoin network                               =====
-----                               Bitcoin network initialized                                -----


=====                            Starting Ethereum docker container                            =====
-----                     Docker Ethereum started, docker id is e4c2c46c91                     -----


=====                              Initializing Ethereum network                               =====
-----                               Ethereum network initialized                               -----


=====            Starting Hive docker container and waiting for the Generated block            =====
-----                       Docker Hive started, docker id is 9fe165e1d0                       -----


=====                                Initializing Hive network                                 =====
-----                                 Hive network initialized                                 -----

Change to 'mydir':

cd mydir

Run your copy of witness_node to have witness_node_data_dir created:

./witness_node

Wait a few seconds until red complaints, then Ctrl-C.

Copy docker-extracted config.ini to witness_node_data_dir:

cp config.ini ./witness_node_data_dir/

Edit mydir/witness_node_data_dir/config.ini:
uncomment
'# genesis-json ='
and set it to point to your genesis file:

genesis-json = mydir/genesis.json

Repeat URLs or IP addresses of sidechain RPC nodes:

bitcoin-node-ip = 10.11.12.201

==>

bitcoin-node-ip = 10.11.12.201
bitcoin-node-ip = 10.11.12.201

then

hive-node-rpc-url = 10.11.12.203:28090

==>

hive-node-rpc-url = 10.11.12.203:28090
hive-node-rpc-url = 10.11.12.203:28090
hive-node-rpc-url = 10.11.12.203:28090

and then

ethereum-node-rpc-url = 10.11.12.202:8545

==>

ethereum-node-rpc-url = 10.11.12.202:8545
ethereum-node-rpc-url = 10.11.12.202:8545

Enable RPC call debugging:

debug-rpc-calls = 1

After the previous line, add:

simulate-rpc-connection-reselection = 1

Delete blockchain data in mydir:

rm -rf mydir/witness_node_data_dir/blockchain

Now, start your copy of witness_node
to simulate switching between sidechain RPC nodes:

./witness_node

and observe its output.

There must be one

"### RPC connection reselection will be simulated"

line in the beginning.

Wait a few minutes.
There must arise lines like

### Ping hive node #0, 10.11.12.203:28090

with different node numbers after '#' sign,
for Bitcoin, Hive and Ethereum sidechains,
approximately every 10 seconds.
Check that there was at least one such line for each sidechain type.

Also, from time to time,

### Reselected hive node to #1, 10.11.12.203:28090

lines must appear - again, sidechain name, node number and URL will vary.
Check that there was at least one for each sidechain type.

Change directory:

cd peerplays-utils/peerplays-qa-environment/e2e-tests

Execute SON deposit/withdrawal tests:

pytest test_btc_son.py test_hive_son.py test_eth_son.py

All must pass
except, possibly, Ethereum,
which was failing intermittently a few weeks ago.

Testcase: 0) Use [feature/SON-connection-pool](https://gitlab.com/PBSA/peerplays/-/tree/feature/SON-connection-pool) branch until it's merged. 1) Follow ["Requirements" section of README.md](https://gitlab.com/PBSA/tools-libs/peerplays-utils/-/blob/master/peerplays-qa-environment/e2e-tests/README.md#requirements) to install python testing framework. 2) Change directory: ``` cd peerplays-utils/peerplays-qa-environment/e2e-tests ``` 3) Stop QA environment: ``` ./main.py --stop ``` Expected output: ``` ===== Stopping all docker containers ===== ----- All docker containers stopped ----- ``` 4) Start QA environment: ``` ./main.py --start all --all-in-one ``` Expected output: ``` ===== Starting Bitcoin docker container ===== ----- Docker Bitcoin started, docker id is 440bd60de4 ----- ===== Initializing Bitcoin network ===== ----- Bitcoin network initialized ----- ===== Starting Ethereum docker container ===== ----- Docker Ethereum started, docker id is e4c2c46c91 ----- ===== Initializing Ethereum network ===== ----- Ethereum network initialized ----- ===== Starting Hive docker container and waiting for the Generated block ===== ----- Docker Hive started, docker id is 9fe165e1d0 ----- ===== Initializing Hive network ===== ----- Hive network initialized ----- ===== Starting Peerplays docker containers and waiting for the Generated block ===== ----- Docker Peerplays started, docker id is 66dac8c4f0 ----- ===== Initializing Peerplays network ===== ----- Peerplays network initialized ----- ... ``` 4) Change to some empty directory, say 'mydir': ``` cd mydir ``` 5) Copy 'witness_node', 'cli_wallet' executables, 'config.ini' and 'genesis.json' from container to 'mydir': ``` docker cp -L peerplays-qa-environment_peerplays-all-in-one_1:/home/peerplays/peerplays-network/witness_node ./ docker cp -L peerplays-qa-environment_peerplays-all-in-one_1:/home/peerplays/peerplays-network/cli_wallet ./ docker cp peerplays-qa-environment_peerplays-all-in-one_1:/home/peerplays/peerplays-network/genesis.json ./ docker cp peerplays-qa-environment_peerplays-all-in-one_1:/home/peerplays/peerplays-network/witness_node_data_dir/config.ini ./ ``` 6) Change directory: ``` cd peerplays-utils/peerplays-qa-environment/e2e-tests ``` 7) Stop QA environment: ``` ./main.py --stop ``` 8) Start QA environment except Peerplays container: ``` ./main.py --start btc eth hive ``` Expected output: ``` ===== Starting Bitcoin docker container ===== ----- Docker Bitcoin started, docker id is 440bd60de4 ----- ===== Initializing Bitcoin network ===== ----- Bitcoin network initialized ----- ===== Starting Ethereum docker container ===== ----- Docker Ethereum started, docker id is e4c2c46c91 ----- ===== Initializing Ethereum network ===== ----- Ethereum network initialized ----- ===== Starting Hive docker container and waiting for the Generated block ===== ----- Docker Hive started, docker id is 9fe165e1d0 ----- ===== Initializing Hive network ===== ----- Hive network initialized ----- ``` 9) Change to 'mydir': ``` cd mydir ``` 10) Run your copy of witness_node to have witness_node_data_dir created: ``` ./witness_node ``` Wait a few seconds until red complaints, then Ctrl-C. 11) Copy docker-extracted config.ini to witness_node_data_dir: ``` cp config.ini ./witness_node_data_dir/ ``` 12) Edit mydir/witness_node_data_dir/config.ini: uncomment '# genesis-json =' and set it to point to your genesis file: ``` genesis-json = mydir/genesis.json ``` Repeat URLs or IP addresses of sidechain RPC nodes: ``` bitcoin-node-ip = 10.11.12.201 ``` ==> ``` bitcoin-node-ip = 10.11.12.201 bitcoin-node-ip = 10.11.12.201 ``` then ``` hive-node-rpc-url = 10.11.12.203:28090 ``` ==> ``` hive-node-rpc-url = 10.11.12.203:28090 hive-node-rpc-url = 10.11.12.203:28090 hive-node-rpc-url = 10.11.12.203:28090 ``` and then ``` ethereum-node-rpc-url = 10.11.12.202:8545 ``` ==> ``` ethereum-node-rpc-url = 10.11.12.202:8545 ethereum-node-rpc-url = 10.11.12.202:8545 ``` Enable RPC call debugging: ``` debug-rpc-calls = 1 ``` After the previous line, add: ``` simulate-rpc-connection-reselection = 1 ``` 13) Delete blockchain data in mydir: ``` rm -rf mydir/witness_node_data_dir/blockchain ``` 14) Now, start your copy of witness_node to simulate switching between sidechain RPC nodes: ``` ./witness_node ``` and observe its output. There must be one ``` "### RPC connection reselection will be simulated" ``` line in the beginning. Wait a few minutes. There must arise lines like ``` ### Ping hive node #0, 10.11.12.203:28090 ``` with different node numbers after '#' sign, for Bitcoin, Hive and Ethereum sidechains, approximately every 10 seconds. Check that there was at least one such line for each sidechain type. Also, from time to time, ``` ### Reselected hive node to #1, 10.11.12.203:28090 ``` lines must appear - again, sidechain name, node number and URL will vary. Check that there was at least one for each sidechain type. 15) Change directory: ``` cd peerplays-utils/peerplays-qa-environment/e2e-tests ``` 16) Execute SON deposit/withdrawal tests: ``` pytest test_btc_son.py test_hive_son.py test_eth_son.py ``` All must pass except, possibly, Ethereum, which was failing intermittently a few weeks ago.

timur.5 commented

2022-12-24 22:29:04 +00:00

(Migrated from gitlab.com)

assigned to @wsalloum and @prandnum

bobinson commented

2022-12-25 14:11:04 +00:00

(Migrated from gitlab.com)

Note to QA : What this feature means is that, now we can have more than one sidechain API node. To test, start with having multiple sidechain API nodes for each network and then shutdown them to see whether the sidechain transactions are still working with atleast one of the API node available.

timur.5 commented

2022-12-25 22:40:08 +00:00

(Migrated from gitlab.com)

OK, Bobinson described a real testcase.
Of course, this is much better,
I only didn't know how to perform it in QA environment.

In this case, 'simulate-rpc-connection-reselection' in config.ini won't be needed.

OK, Bobinson described a real testcase. Of course, this is much better, I only didn't know how to perform it in QA environment. In this case, 'simulate-rpc-connection-reselection' in config.ini won't be needed.

serkixenos commented

2022-12-28 07:50:36 +00:00

(Migrated from gitlab.com)

unassigned @wsalloum

timur.5 commented

2022-12-28 12:19:09 +00:00

(Migrated from gitlab.com)

mentioned in merge request !181

vampik commented

2023-03-01 06:02:29 +00:00

(Migrated from gitlab.com)

assigned to @vampik

vampik commented

2023-03-01 06:02:57 +00:00

(Migrated from gitlab.com)

Connection pool working fine both for HIVE and ETH

prandnum commented

2023-03-15 20:11:08 +00:00

(Migrated from gitlab.com)

Steps Executed:

Stopped peerplays02,03,04,05 - ~ 18:15:12
Transferred HIVE AND HBD(100 each) - ~ 18:17:57
Started peerplays02,03,04,05 - ~ 18:23:45

The transferred money was successfully debited from peerplays son-down-15032023.log

@bobinson @serkixenos let me know if this test is sufficient to validate this feature.

Steps Executed: 1) Stopped peerplays02,03,04,05 - ~ 18:15:12 2) Transferred HIVE AND HBD(100 each) - ~ 18:17:57 3) Started peerplays02,03,04,05 - ~ 18:23:45 The transferred money was successfully debited from peerplays [son-down-15032023.log](/uploads/0e56e05419c1eba5cf17cc365b13895f/son-down-15032023.log) @bobinson @serkixenos let me know if this test is sufficient to validate this feature.

prandnum commented

2023-03-28 18:30:26 +00:00

(Migrated from gitlab.com)

@serkixenos @bobinson can this be closed with the above details?

bobinson (Migrated from gitlab.com) closed this issue

2024-08-20 14:14:05 +00:00

exploring connection pool for SONs #398