witness process on SON node doesnot start if stop/started without "--replay-blockchain" option #77

Closed
opened 2021-04-18 18:40:09 +00:00 by prandnum · 8 comments
prandnum commented 2021-04-18 18:40:09 +00:00 (Migrated from gitlab.com)

The current witness service script has the following:

[Unit]
Description=Witness
After=network.target

[Service]
ExecStart=/var/opt/1018747577/witness_node --replay-blockchain
Restart=always
RestartSec=5
#StandardError=file:/var/opt/1018747577/son16/StdErr.log
#StandardOutput=file:/var/opt/1018747577/son16/StdOut.log
User=gladiator
WorkingDirectory=/var/opt/1018747577/son16

[Install]
WantedBy=multi-user.target

Question: 1) why do we need to replay every time when the SON witness nodes restart?

Since it is time-consuming to test with replay, I tried removing the option and started the witness service which failed with uniqueness violation error(see below)

Apr 18 18:29:26 gladiator16.peerplays.download systemd[1]: Stopped Witness.
Apr 18 18:29:26 gladiator16.peerplays.download systemd[1]: Started Witness.
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766529ms th_a       accounts_list_plugin.cpp:128  list_accounts        ] accounts list plugin:  list_accounts()
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766534ms th_a       bookie_plugin.cpp:468         plugin_initialize    ] bookie plugin: plugin_startup() begin
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766534ms th_a       db_management.cpp:289         force_slow_replays   ] enabling slow replays
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766535ms th_a       bookie_plugin.cpp:501         plugin_initialize    ] bookie plugin: plugin_startup() end
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766535ms th_a       peerplays_sidechain_plugin.cpp:655 plugin_initialize    ] peerplays sidechain plugin:  plugin_initialize() begin
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766536ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST8TCQFzyYDp3DPgWZ24261fMPSCzXxVyoF3miWeTj6JTi2DZdrL
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766537ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST82qv1LKFvwVKD9pg5JQf6qqwLcoeqUniQjWJ3wKTodyWa7gHUs
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766538ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST6xdp7MrEPnaNK9GuF3KTeTizgGN6JC5nPmxx81higFepSZ8N7r
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766538ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST55j32Up75gHCxJBPN18vEytL9anDgEVFtsaCii38keGQG71X22
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766539ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST68bX5bB16GkEAig6w2WTh9NbM9nHa66CemnDRx2njRY9bbWXhU
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766539ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST7mtTyM2rD18xDTtLTxWhq6W6zFgAgPFu9KHSFNsfWJXZNT8Wc8
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766540ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST7RMDnipLkFaQ4vtDyVvgyCedRoxyT9JWpAoM9mrE7rwVSyezoB
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766541ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST51nSJ2q1C9htnYWfTv73JxEc4nBWPNxJtUGPGpD4XwxeLzAd8t
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766542ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST8EmMMvQdAPzcnxymRUpbYdg8fArUY473QosCQpuPtWXxXtoNp4
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766542ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST5815xbKy73Bx1LJWW1jg7GshWSEFWub3uoiEFP7FtP6z4YZtkU
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766543ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST61qgG2v6JArygFiQCKypymxhBqg1wKmmbdkHeNkXhYDvkZmBtY
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766544ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST6z33kHxQxyGvFWfpAnL3X3MvLtPEBknkNeFFJyk63PvRtaN1Xo
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766544ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST55eCWenoKmZct5YvUYv7aphMmSVkroZTJZWFHGTVj8r8bKEPMd
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766545ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST5e4HXhA4yBEGzaXVyjuVabKhG1qGghi6rypvq5fLxAmU9XLRHT
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766545ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST89qMuZejYeeGvjr3bMEcGyvhD4dyUchmxbLNUBFbPodqJKJPZc
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766546ms th_a       peerplays_sidechain_plugin.cpp:157 plugin_initialize    ] Public Key: TEST6KRpHxYJSE5vXvoeVMLbKSYnVspt2nnGV2enncRzHgLQ9dez5v
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766546ms th_a       peerplays_sidechain_plugin.cpp:657 plugin_initialize    ] peerplays sidechain plugin:  plugin_initialize() end
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766546ms th_a       witness.cpp:91                plugin_initialize    ] witness plugin:  plugin_initialize() begin
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766547ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST6MRyAjQq8ud7hVNYcfnVPJqcVpscN5So8BhtHuGYqET5GDW5CV
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766547ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST5VEMezMNjQNs7jGsqrAYaf717KBxqgsJVW7crj1jyJD7f74WSz
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766548ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST8YgGjPfpnnK5rS1T21cd8CGtLxi8VvygpBCU2SvDZNBwkTApso
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766548ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST6bsLVw3PXzU5fpLbzc65zauPDr9Fr2Dam1ajJrg11rTtgd8kGv
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766549ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST7gt2dGGm6BBpqJ1B9hzPk7swu5afnPR5p6nxt6EwRMYQfPE7y6
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766550ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST7b71sb2eZyemoRLuD7r2kwhb3erxnm83U6VDiL7ArkJsDqTeZ1
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766550ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST7cycUsXEdXubwwv7XuJ15n7y4sHzx22tjijiUVBRK7DxdBcdia
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766551ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST6VeyMca1RPYwFf4TUhxwph4u51R5RurVtCLPoT7YiXE5DU17e1
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766552ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST6mi9oujSjWQWwwa9Wh828wVRnsM3mAbgenMabUT8qcbDakDnbH
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766552ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST5hwkj6FE6L3kgbyoGTbfVa2W1Dg4xjqBrnjGUHtBTyek6V4VxP
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766553ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST67sAReBFozV5uMGqTeeiWHKLoS7h1QXT13UyuHHKYPBX72BqqZ
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766553ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST5Jnq9LjreRbeMVZrx63h71R1pEL8tr7QiNHykH8qpFRh4y6ZCW
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766554ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST8AVAG4xdaQLi84wjibu2Zw7678gDEBPgZGYitBstLrtNHVBqzL
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766554ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST73Zi6sUtZrwNwNcmDjzLvgKVkpYwCnNaGeBiN5kbnBfUxZWHpZ
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766554ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST51a97eZtRv4Ekq1qZXaXfnQbZ1G3hmh9yva1NJTCdWNYrx3p24
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766555ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST6wbXLfEMkKY4aD9sqk3jNHo5wkxTQYgGHTfHVDUJCEQxWsfGDD
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766555ms th_a       witness.cpp:105               plugin_initialize    ] Public Key: TEST6HJpTxUzUePcRo2rLDjZwKpdBDWeHLXcYGSK9QVDPAn6yjXJBG
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766556ms th_a       witness.cpp:123               plugin_initialize    ] witness plugin:  plugin_initialize() end
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766556ms th_a       object_database.cpp:106       open                 ] Opening object database from /var/opt/1018747577/son16/witness_node_data_dir/blockchain ...
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766664ms th_a       db_management.cpp:239         open                 ] 10 assert_exception: Assert Exception
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: insert_result.second: Could not insert object, most likely a uniqueness constraint was violated
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  generic_index.hpp:53 insert
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  object_database.cpp:113 open
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766666ms th_a       db_management.cpp:239         open                 ] data_dir: /var/opt/1018747577/son16/witness_node_data_dir/blockchain
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766667ms th_a       application.cpp:394           startup              ] Caught exception 10 assert_exception: Assert Exception
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: insert_result.second: Could not insert object, most likely a uniqueness constraint was violated
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  generic_index.hpp:53 insert
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  object_database.cpp:113 open
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: rethrow
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  db_management.cpp:239 open in open(), you might want to force a replay
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766668ms th_a       application.cpp:439           startup              ] 10 assert_exception: Assert Exception
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: insert_result.second: Could not insert object, most likely a uniqueness constraint was violated
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  generic_index.hpp:53 insert
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  object_database.cpp:113 open
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: rethrow
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  db_management.cpp:239 open
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766668ms th_a       application.cpp:1019          startup              ] 10 assert_exception: Assert Exception
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: insert_result.second: Could not insert object, most likely a uniqueness constraint was violated
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  generic_index.hpp:53 insert
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  object_database.cpp:113 open
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: rethrow
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  db_management.cpp:239 open
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: rethrow
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  application.cpp:439 startup
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766668ms th_a       main.cpp:196                  main                 ] Exiting with error:
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 10 assert_exception: Assert Exception
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: insert_result.second: Could not insert object, most likely a uniqueness constraint was violated
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  generic_index.hpp:53 insert
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  object_database.cpp:113 open
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: rethrow
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  db_management.cpp:239 open
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: rethrow
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     {}
Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]:     th_a  application.cpp:439 startup
Apr 18 18:29:26 gladiator16.peerplays.download systemd[1]: witness.service: Main process exited, code=exited, status=1/FAILURE
Apr 18 18:29:26 gladiator16.peerplays.download systemd[1]: witness.service: Failed with result 'exit-code'.

 
The current witness service script has the following: ``` [Unit] Description=Witness After=network.target [Service] ExecStart=/var/opt/1018747577/witness_node --replay-blockchain Restart=always RestartSec=5 #StandardError=file:/var/opt/1018747577/son16/StdErr.log #StandardOutput=file:/var/opt/1018747577/son16/StdOut.log User=gladiator WorkingDirectory=/var/opt/1018747577/son16 [Install] WantedBy=multi-user.target ``` Question: 1) why do we need to replay every time when the SON witness nodes restart? Since it is time-consuming to test with replay, I tried removing the option and started the witness service which failed with uniqueness violation error(see below) ``` Apr 18 18:29:26 gladiator16.peerplays.download systemd[1]: Stopped Witness. Apr 18 18:29:26 gladiator16.peerplays.download systemd[1]: Started Witness. Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766529ms th_a accounts_list_plugin.cpp:128 list_accounts ] accounts list plugin: list_accounts() Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766534ms th_a bookie_plugin.cpp:468 plugin_initialize ] bookie plugin: plugin_startup() begin Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766534ms th_a db_management.cpp:289 force_slow_replays ] enabling slow replays Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766535ms th_a bookie_plugin.cpp:501 plugin_initialize ] bookie plugin: plugin_startup() end Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766535ms th_a peerplays_sidechain_plugin.cpp:655 plugin_initialize ] peerplays sidechain plugin: plugin_initialize() begin Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766536ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST8TCQFzyYDp3DPgWZ24261fMPSCzXxVyoF3miWeTj6JTi2DZdrL Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766537ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST82qv1LKFvwVKD9pg5JQf6qqwLcoeqUniQjWJ3wKTodyWa7gHUs Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766538ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST6xdp7MrEPnaNK9GuF3KTeTizgGN6JC5nPmxx81higFepSZ8N7r Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766538ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST55j32Up75gHCxJBPN18vEytL9anDgEVFtsaCii38keGQG71X22 Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766539ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST68bX5bB16GkEAig6w2WTh9NbM9nHa66CemnDRx2njRY9bbWXhU Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766539ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST7mtTyM2rD18xDTtLTxWhq6W6zFgAgPFu9KHSFNsfWJXZNT8Wc8 Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766540ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST7RMDnipLkFaQ4vtDyVvgyCedRoxyT9JWpAoM9mrE7rwVSyezoB Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766541ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST51nSJ2q1C9htnYWfTv73JxEc4nBWPNxJtUGPGpD4XwxeLzAd8t Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766542ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST8EmMMvQdAPzcnxymRUpbYdg8fArUY473QosCQpuPtWXxXtoNp4 Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766542ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST5815xbKy73Bx1LJWW1jg7GshWSEFWub3uoiEFP7FtP6z4YZtkU Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766543ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST61qgG2v6JArygFiQCKypymxhBqg1wKmmbdkHeNkXhYDvkZmBtY Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766544ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST6z33kHxQxyGvFWfpAnL3X3MvLtPEBknkNeFFJyk63PvRtaN1Xo Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766544ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST55eCWenoKmZct5YvUYv7aphMmSVkroZTJZWFHGTVj8r8bKEPMd Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766545ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST5e4HXhA4yBEGzaXVyjuVabKhG1qGghi6rypvq5fLxAmU9XLRHT Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766545ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST89qMuZejYeeGvjr3bMEcGyvhD4dyUchmxbLNUBFbPodqJKJPZc Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766546ms th_a peerplays_sidechain_plugin.cpp:157 plugin_initialize ] Public Key: TEST6KRpHxYJSE5vXvoeVMLbKSYnVspt2nnGV2enncRzHgLQ9dez5v Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766546ms th_a peerplays_sidechain_plugin.cpp:657 plugin_initialize ] peerplays sidechain plugin: plugin_initialize() end Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766546ms th_a witness.cpp:91 plugin_initialize ] witness plugin: plugin_initialize() begin Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766547ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST6MRyAjQq8ud7hVNYcfnVPJqcVpscN5So8BhtHuGYqET5GDW5CV Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766547ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST5VEMezMNjQNs7jGsqrAYaf717KBxqgsJVW7crj1jyJD7f74WSz Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766548ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST8YgGjPfpnnK5rS1T21cd8CGtLxi8VvygpBCU2SvDZNBwkTApso Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766548ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST6bsLVw3PXzU5fpLbzc65zauPDr9Fr2Dam1ajJrg11rTtgd8kGv Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766549ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST7gt2dGGm6BBpqJ1B9hzPk7swu5afnPR5p6nxt6EwRMYQfPE7y6 Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766550ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST7b71sb2eZyemoRLuD7r2kwhb3erxnm83U6VDiL7ArkJsDqTeZ1 Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766550ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST7cycUsXEdXubwwv7XuJ15n7y4sHzx22tjijiUVBRK7DxdBcdia Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766551ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST6VeyMca1RPYwFf4TUhxwph4u51R5RurVtCLPoT7YiXE5DU17e1 Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766552ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST6mi9oujSjWQWwwa9Wh828wVRnsM3mAbgenMabUT8qcbDakDnbH Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766552ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST5hwkj6FE6L3kgbyoGTbfVa2W1Dg4xjqBrnjGUHtBTyek6V4VxP Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766553ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST67sAReBFozV5uMGqTeeiWHKLoS7h1QXT13UyuHHKYPBX72BqqZ Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766553ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST5Jnq9LjreRbeMVZrx63h71R1pEL8tr7QiNHykH8qpFRh4y6ZCW Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766554ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST8AVAG4xdaQLi84wjibu2Zw7678gDEBPgZGYitBstLrtNHVBqzL Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766554ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST73Zi6sUtZrwNwNcmDjzLvgKVkpYwCnNaGeBiN5kbnBfUxZWHpZ Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766554ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST51a97eZtRv4Ekq1qZXaXfnQbZ1G3hmh9yva1NJTCdWNYrx3p24 Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766555ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST6wbXLfEMkKY4aD9sqk3jNHo5wkxTQYgGHTfHVDUJCEQxWsfGDD Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766555ms th_a witness.cpp:105 plugin_initialize ] Public Key: TEST6HJpTxUzUePcRo2rLDjZwKpdBDWeHLXcYGSK9QVDPAn6yjXJBG Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766556ms th_a witness.cpp:123 plugin_initialize ] witness plugin: plugin_initialize() end Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766556ms th_a object_database.cpp:106 open ] Opening object database from /var/opt/1018747577/son16/witness_node_data_dir/blockchain ... Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766664ms th_a db_management.cpp:239 open ] 10 assert_exception: Assert Exception Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: insert_result.second: Could not insert object, most likely a uniqueness constraint was violated Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a generic_index.hpp:53 insert Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a object_database.cpp:113 open Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766666ms th_a db_management.cpp:239 open ] data_dir: /var/opt/1018747577/son16/witness_node_data_dir/blockchain Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766667ms th_a application.cpp:394 startup ] Caught exception 10 assert_exception: Assert Exception Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: insert_result.second: Could not insert object, most likely a uniqueness constraint was violated Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a generic_index.hpp:53 insert Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a object_database.cpp:113 open Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: rethrow Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a db_management.cpp:239 open in open(), you might want to force a replay Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766668ms th_a application.cpp:439 startup ] 10 assert_exception: Assert Exception Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: insert_result.second: Could not insert object, most likely a uniqueness constraint was violated Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a generic_index.hpp:53 insert Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a object_database.cpp:113 open Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: rethrow Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a db_management.cpp:239 open Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766668ms th_a application.cpp:1019 startup ] 10 assert_exception: Assert Exception Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: insert_result.second: Could not insert object, most likely a uniqueness constraint was violated Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a generic_index.hpp:53 insert Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a object_database.cpp:113 open Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: rethrow Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a db_management.cpp:239 open Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: rethrow Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a application.cpp:439 startup Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 1766668ms th_a main.cpp:196 main ] Exiting with error: Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: 10 assert_exception: Assert Exception Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: insert_result.second: Could not insert object, most likely a uniqueness constraint was violated Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a generic_index.hpp:53 insert Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a object_database.cpp:113 open Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: rethrow Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {"data_dir":"/var/opt/1018747577/son16/witness_node_data_dir/blockchain"} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a db_management.cpp:239 open Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: rethrow Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: {} Apr 18 18:29:26 gladiator16.peerplays.download witness_node[8099]: th_a application.cpp:439 startup Apr 18 18:29:26 gladiator16.peerplays.download systemd[1]: witness.service: Main process exited, code=exited, status=1/FAILURE Apr 18 18:29:26 gladiator16.peerplays.download systemd[1]: witness.service: Failed with result 'exit-code'. ```
prandnum commented 2021-04-18 18:40:38 +00:00 (Migrated from gitlab.com)

@bobinson @serkixenos @sierra19XX

CC: @hbelakon

@bobinson @serkixenos @sierra19XX CC: @hbelakon
bobinson commented 2021-04-19 10:02:44 +00:00 (Migrated from gitlab.com)

assigned to @sierra19XX

assigned to @sierra19XX
sierra19XX commented 2021-04-20 11:24:57 +00:00 (Migrated from gitlab.com)

changed time estimate to 20h

changed time estimate to 20h
serkixenos commented 2021-04-21 12:00:22 +00:00 (Migrated from gitlab.com)

If you are not able to start witness node without replay blockchain, it means that the blockchain database is corrupted, and it needs to be resynced.

You can use replay-blockchain or resync-blockchain to fix it.

First one will just replay already downloaded data.

Second one will erase existing data, download data from the network again and replay it.

If you are not able to start witness node without replay blockchain, it means that the blockchain database is corrupted, and it needs to be resynced. You can use replay-blockchain or resync-blockchain to fix it. First one will just replay already downloaded data. Second one will erase existing data, download data from the network again and replay it.
prandnum commented 2021-04-21 14:57:54 +00:00 (Migrated from gitlab.com)

I had stopped the witness service using "sudo systemctl stop witness" command so it should not lead to a corruption issue.

I had stopped the witness service using "sudo systemctl stop witness" command so it should not lead to a corruption issue.
sierra19XX commented 2021-05-07 11:27:02 +00:00 (Migrated from gitlab.com)

This happens when the object data is corrupted.
It can be corrupted because of the service stoppage.
Moreover, there are no changes in this part of the code since the graphene blockchain is deployed.

This happens when the object data is corrupted. It can be corrupted because of the service stoppage. Moreover, there are no changes in this part of the code since the graphene blockchain is deployed.
sierra19XX commented 2021-05-07 11:27:22 +00:00 (Migrated from gitlab.com)

assigned to @prandnum and unassigned @sierra19XX

assigned to @prandnum and unassigned @sierra19XX
prandnum commented 2021-05-07 18:45:42 +00:00 (Migrated from gitlab.com)

@sierra19XX closing as per above comment as the issue looks to be because of the server used(gladiator)

@bobinson FYI

@sierra19XX closing as per above comment as the issue looks to be because of the server used(gladiator) @bobinson FYI
prandnum (Migrated from gitlab.com) closed this issue 2021-05-07 18:45:43 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Peerplays_Blockchain/peerplays_migrated#77
No description provided.