Graphene Updates and DApp Support #643

Closed
nathanielhourt wants to merge 84 commits from dapp-support into develop

84 commits

Author SHA1 Message Date
Nathaniel
f9fcffbb4d
Safety Check: Part 2 -- Implement and Integrate Checks
Implement a safety check mechanism on object_database, based on the
safety_check_policy abstract interface. Create two standard
implementations of the safety_check_policy interface, one
(null_safety_check) which allows all modifications unconditionally, and
the other (database_lock_safety_check) which allows modifications only
when unlocked.

Integrate these safety checks into chain::database and plugins, so that
the consensus databases are locked at all times except during core
consensus code pathways. Also ensures that databases are re-locked when
calling code outside of consensus pathways from consensus pathways.

To make this work, it was necessary to move two objects from the
consensus object spaces to a new API object space. The
operation_history_object and account_transaction_history_object were
moved to the API object space, as they are not actually used by
consensus and are maintained by a plugin (which can no longer modify the
consensus object spaces, due to the safety checks).

Finally, add a mechanism to application and chain::database, which
allows the chain to start in "unit testing mode" and allows unchecked
actions upon the database within delimited scopes. This was necessary
because many tests edit the database directly to set up the environment
for their respective tests. This mode is activated by database_fixture so
tests can utilize it conveniently, but it is architecturally difficult to
enable this mode in production, i.e. from a plugin.
2022-03-12 14:04:08 -06:00
Nathaniel
e8b432c19f
Safety Check: Part 1 -- Evaluator Tagging
Pursuant to the requested safety checks on the database, to ensure that
plugin code (i.e., third party code) cannot modify the database, we
implement evaluator tagging so the chain can distinguish between
consensus evaluators and third party evaluators. Also, define a new kind
of evaluator base class, third_party_evaluator, so that fees are not
charged multiple times for operations with multiple evaluators.

Next step, implement the actual safety check mechanism on the database.
2022-03-07 16:37:53 -06:00
Nathaniel
659d135b9b
Merge remote-tracking branch 'glu/develop' into dapp-support 2022-02-13 14:25:06 -06:00
Nathaniel
2da369453e
Merge branch 'develop' into dapp-support 2022-02-08 13:15:07 -06:00
Nathaniel
d2ced50bbf
Bump FC 2022-02-08 11:27:26 -06:00
serkixenos
c456d4ec6b
Update README.md 2022-02-08 11:26:34 -06:00
serkixenos
f46a223dce
Update README for Ubuntu 20.04 2022-02-08 11:26:34 -06:00
serkixenos
7c7f768ce4
Replace vulnerable XML library 2022-02-08 11:26:34 -06:00
Vlad Dobromyslov
3a7187baba
bug #245 exception seen in witness logs 2022-02-08 11:26:34 -06:00
Vlad Dobromyslov
45e501b916
Resolve "port ES changes from Bitshares" 2022-02-08 11:26:34 -06:00
Vlad Dobromyslov
62a553ab5f
bug #267 Fix error in chain_test in gitlab autobuild 2022-02-08 11:26:34 -06:00
serkixenos
8c402d2e70
Fix list_active_son command output on deregistered SONs 2022-02-08 11:26:34 -06:00
serkixenos
dcdf406a2d
Fix cli wallet memo displaying 2022-02-08 11:26:34 -06:00
Nathaniel Hourt
ee0d2b21e0
Fix build
In some environments, build fails here due to `curl` dependency not being linked in time for `utilities` to see it.
2022-01-29 16:21:13 -06:00
Nathaniel Hourt
2a37d8a0a1
Fix build
In some environments, this fails to build without this header.
2022-01-27 15:29:23 -06:00
Nathaniel
f667fda2e4
Merge remote-tracking branch 'origin/beatrice' into to-upstream 2022-01-26 13:16:02 -06:00
Nathaniel
d17eb5ec72
Add wallet command for custom_operation
Create a new cli_wallet command, run_custom_operation, which makes it
convenient to run custom_operation transactions which invoke third party
contracts (i.e., dapps)
2022-01-14 20:10:16 -06:00
Nathaniel
de87e1b82c
Canonicalize chain ID calculation
When using an external genesis file, it doesn't make sense to calculate
the chain ID as the literal text contents of the file, which are quite
volatile. Rather, it should be calculated based on the logical content
of the file. Serialize the genesis object to get a canonical
representation, and calculate the chain ID off of that.
2022-01-12 16:55:45 -06:00
Nathaniel
c833ca646f
Set CMAKE_INSTALL_RPATH for dynamic builds
When building dynamic libraries, set the RPATH so that binaries know
where to find their libraries.
2022-01-12 16:55:01 -06:00
Nathaniel
fe02a13685
Replace count_objects_in_space with inspect_all_indexes
The count_objects_in_space function, while providing the necessary
functionality, was clumsy and inconvenient, and was not idiomatic.
Replace it with inspect_all_indexes which resolves these shortcomings.
2022-01-05 17:27:52 -06:00
Nathaniel Hourt
25faf9b084
Merge pull request #2 from MichelSantos/graphene-dapp-support
Adjustments for snapshot validation
2022-01-04 13:06:42 -06:00
Michel Santos
4fea001586 Fix Bookie plugin to provide a consistent data state
even when the node is started in replay mode
2022-01-03 10:24:32 -05:00
Michel Santos
8ec87b404f Enhance secondary_index
Enhance secondary_index by distinguishing between
previous objects loaded from persistence versus
new objects created during the session
2022-01-03 09:49:44 -05:00
Michel Santos
8325b25d0e Harmonize the activation date of GPOS 2022-01-03 09:49:44 -05:00
Michel Santos
c2675b4423 Re-allow build of programs for compatibility with Commit 082df7 2022-01-03 09:49:44 -05:00
Nathaniel
5fe3408893
Allow querying number of objects in object space
Previously, there was no good way to determine how many objects are in
an object space. Add a way to do so.
2022-01-02 14:50:17 -06:00
Nathaniel
d2dedbc4e4
Remove database unity build
Upstream dropped it and it was causing problems, so let it go!
2022-01-02 14:32:56 -06:00
Nathaniel
8df442a65f
Add secondary_index concerns
Add a way to create a secondary index with the space/type ID of the
object rather than the compile-time type, which may not always be
available.

Also, add a way to delete a secondary index (whether by compile-time
type or by object space/type ID).
2022-01-02 14:22:03 -06:00
Michel Santos
a5a78dacf1 Update fc to merger of latest-fc into dapp-support-fc 2021-11-13 08:42:42 -05:00
Michel Santos
c8cfaeb529 Fix test by reverting to the original timestamp used for the genesis 2021-11-11 18:48:49 -05:00
Michel Santos
ec66970d78 Add missing header 2021-11-11 18:48:43 -05:00
Nathan Hourt
906ffa6351 Fix tests better
The solution being used for initializing the boost test modules is
now deprecated, so update it to use the global test fixture instead
2021-11-11 14:38:36 -05:00
Nathan Hourt
8a6bae6006 Fix tests
These tests did not properly define their BOOST_TEST_MODULE macro,
causing linking to fail in some environments.
2021-11-11 14:35:53 -05:00
Nathan Hourt
0c320fdc71 Add missing header 2021-11-11 14:35:53 -05:00
Nathan Hourt
0d18523d8f Fix dynamic build
In some environments, dynamic linking fails if Boost is static
2021-11-11 14:32:47 -05:00
Nathan Hourt
95afb342c3 Fix to dynamic builds
FC wasn't always getting the memo that it should build dynamically
if Graphene is.
2021-11-11 14:31:47 -05:00
Nathan Hourt
e168d2d830 Fix install of hardfork.hpp 2021-11-11 14:29:34 -05:00
Nathan Hourt
3f50447013 Fix dynamic libs build 2021-11-11 14:29:34 -05:00
Nathan Hourt
c76e1f3157 Toward dynamically reloading contracts...
Add the ability to dynamically unload an auxiliary evaluator (any
evaluator for a given operation type except the first one registered for
that operation type), and as an added convenience, add an idiomatically
consistent way to check if a given index is already registered or not.
2021-11-11 14:27:50 -05:00
Nathan Hourt
082df7ab4a Allow build of dynamic libraries
In order to support dynamically linked nodes based on Graphene, add
support for building dynamic libraries for the core Graphene
modules: chain, db, protocol, net, and utilities.
2021-11-11 14:20:40 -05:00
Nathan Hourt
95d4cbdf4a Add build/IDE artifacts to gitignore 2021-11-11 11:25:47 -05:00
Nathan Hourt
ab113b2fc4 Add support for multiple evaluators per operation type
This commit adds the ability to register multiple evaluator types for a
given operation type. This will be used to allow plug-ins to define their
own evaluators for various operation types (most notably, custom
operations) so that they may be used to carry app-specific data which can
be parsed by a plug-in.
2021-11-11 11:25:47 -05:00
Nathan Hourt
4d836dacb9 Ref !3/#376: Graphene Updates
This adds the most important updates to Graphene from BitShares. Most notably,
https://github.com/bitshares/bitshares-core/issues/1506

Second most notably, it updates Peerplays' FC to be in sync with BitShares FC.

This is a squash commit of several subcommits. The subcommit messages are
reproduced below:

Replace fc::uint128 with boost::multiprecision::uint128_t

replace smart_ref with shared_ptr

Fixes/Remove Unused

Remove NTP time

Remove old macro

This macro is now in FC, so no need to define it here anymore

Replaced fc::array with std::array

Separate exception declaration and implementation

Adapted to fc promise changes

Fixes

Add back in some of Peter's fixes that got lost in the cherry pick

_hash endianness fixes

Remove all uses of fc/smart_ref

It's gone, can't use it anymore

Replace improper static_variant operator overloads with comparators

Fixes

Remove boost::signals from build system; it's header-only so it's not
listed in cmake anymore.

Also remove some unused hashing code

Impl. pack/unpack functions for extension class

Ref #1506: Isolate chain/protocol to its own library

Ref #1506: Add object_downcast_t

Allows the more concise expression `object_downcast_t<xyz>` instead of
the old `typename object_downcast<xyz>::type`

Ref #1506: Move ID types from db to protocol

The ID types, object_id and object_id_type, were defined in the db
library, and the protocol library depends on db to get these types.
Technically, the ID types are defined by the protocol and used by the
database, and not vice versa. Therefore these types should be in the
protocol library, and db should depend on protocol to get them.

This commit makes it so.

Ref #1506: Isolate chain/protocol to its own library

Remove commented-out index code

Wrap overlength line

Remove unused key types

Probably fix Docker build

Fix build after rebase

Ref #1506/#1737: Some requested changes

Ref #1506/#1737: Macro-fy ID type definitions

Define macros to fully de-boilerplate ID type definitions.

Externalities:
 - Rename transaction_object -> transaction_history_object
 - Rename impl_asset_dynamic_data_type ->
impl_asset_dynamic_data_object_type
 - Rename impl_asset_bitasset_data_type ->
impl_asset_bitasset_data_object_type

The first is to avoid a naming collision on transaction_id_type, and the
other two are to maintain consistency with the naming of the other
types.

Ref #1506/#1737: Fix clean_name()

Ref #1506/#1737: Oops

Fix .gitignore

Externalized serialization in protocol library

Fix compile sets

Delete a couple of ghost files that were in the tree but not part
of the project (I accidentally added them to CMakeLists while
merging, but they're broken and not part of the Peerplays code), and
add several files that got dropped from the build during merge.

General fixes

Fix warnings, build issues, unused code, etc.

Fix #1772 by decprecating cli_wallet -H

More fixes

Fix errors and warnings and generally coax it to build

Fix test

I'm pretty sure this didn't break from what I did... But I can't build
the original code, so I can't tell. Anyways, this one now passes...
Others still fail...

Small fix

Fix crash in auth checks

Final fixes

Last round of fixes following the rebase to Beatrice

Rename project in CMakeLists.txt

The CMakeLists.txt declared this project as BitShares and not Peerplays,
which makes it confusing in IDEs. Rename it to be clear which project is
open.

Resolve #374

Replace all object refs in macros with IDs, and fix affected tests to look
up objects by ID rather than using invalidated refs.

A full audit of all tests should be performed to eliminate any further
usage of invalidated object references.

Resolve #373: Add object notifiers

Various fixes

Fixes to various issues, primarily reflections, that cropped up
during merge conflict resolution

Fix startup bug in Bookie plugin

Bookie plugin was preventing the node from starting up because it
registered its secondary indexes to create objects in its own primary
indexes to track objects being created in other primary indexes, and did
so during its `initialize()` step, which is to say, before the database
was loaded from disk at startup. This caused the secondary indexes to
create tracker objects when the observed indexes were loading objects
from disk. This then caused a failure when these tracker indexes were
later loaded from disk, and the first object IDs collided.

This is fixed by refraining from defining secondary indexes until the
`startup()` stage rather than the `initialize()` stage. Primary indexes
are registered in `initialize()`, secondary indexes are registered in
`startup()`.

This also involved adding a new method, "add_secondary_index()", to
`object_database`, as before there was no way to do this because you
couldn't get a non-const index from a non-const database.

I have no idea how this was working before I got here...

Fix egenesis install

Fixes after updates

Rebase on updated develop branch and fix conflicts
2021-11-11 11:25:47 -05:00
Michel Santos
111ac16e92 Merge branch master into graphene-updates 2021-11-09 16:24:26 -05:00
Nathan Hourt
ed7b8b60e0
Resolve #373: Add object notifiers 2020-08-27 20:07:36 -05:00
Nathan Hourt
f2d4bce003
Resolve #374
Replace all object refs in macros with IDs, and fix affected tests to look
up objects by ID rather than using invalidated refs.

A full audit of all tests should be performed to eliminate any further
usage of invalidated object references.
2020-08-27 18:55:37 -05:00
Nathan Hourt
6b59f8269f
Rename project in CMakeLists.txt
The CMakeLists.txt declared this project as BitShares and not Peerplays,
which makes it confusing in IDEs. Rename it to be clear which project is
open.
2020-08-25 13:02:45 -05:00
Nathan Hourt
4e3e0e010a
Final fixes
Last round of fixes following the rebase to Beatrice
2020-08-25 13:01:51 -05:00
Nathan Hourt
a9135cbdd2
Fix crash in auth checks 2020-08-23 19:31:38 -05:00
Nathan Hourt
3dd78de312
Small fix 2020-08-23 14:51:47 -05:00
Nathan Hourt
ce8caae327
Fix test
I'm pretty sure this didn't break from what I did... But I can't build
the original code, so I can't tell. Anyways, this one now passes...
Others still fail...
2020-08-23 14:51:47 -05:00
Nathan Hourt
b8c3e63f1b
More fixes
Fix errors and warnings and generally coax it to build
2020-08-23 14:51:47 -05:00
Peter Conrad
a31e56f531
Fix #1772 by decprecating cli_wallet -H 2020-08-23 14:51:47 -05:00
Nathan Hourt
1c0de74aa7
General fixes
Fix warnings, build issues, unused code, etc.
2020-08-23 14:51:45 -05:00
Nathan Hourt
2af062e7cf
Fix compile sets
Delete a couple of ghost files that were in the tree but not part
of the project (I accidentally added them to CMakeLists while
merging, but they're broken and not part of the Peerplays code), and
add several files that got dropped from the build during merge.
2020-08-23 14:43:21 -05:00
Peter Conrad
9ae796c0a2
Externalized serialization in protocol library 2020-08-23 14:43:21 -05:00
Nathan Hourt
841c6319f9
Fix .gitignore 2020-08-23 14:43:21 -05:00
Nathan Hourt
d02bea9ef0
Ref #1506/#1737: Oops 2020-08-23 14:43:21 -05:00
Nathan Hourt
f2814a451c
Ref #1506/#1737: Fix clean_name() 2020-08-23 14:43:21 -05:00
Nathan Hourt
c94e46f451
Ref #1506/#1737: Macro-fy ID type definitions
Define macros to fully de-boilerplate ID type definitions.

Externalities:
 - Rename transaction_object -> transaction_history_object
 - Rename impl_asset_dynamic_data_type ->
impl_asset_dynamic_data_object_type
 - Rename impl_asset_bitasset_data_type ->
impl_asset_bitasset_data_object_type

The first is to avoid a naming collision on transaction_id_type, and the
other two are to maintain consistency with the naming of the other
types.
2020-08-23 14:43:19 -05:00
Nathan Hourt
6074749813
Ref #1506/#1737: Some requested changes 2020-08-23 14:37:35 -05:00
Nathan Hourt
ad670ecde1
Fix build after rebase 2020-08-23 14:37:02 -05:00
Nathan Hourt
f57dc27f55
Probably fix Docker build 2020-08-23 14:37:02 -05:00
Nathan Hourt
35d864e269
Remove unused key types 2020-08-23 14:37:02 -05:00
Nathan Hourt
3a15577617
Wrap overlength line 2020-08-23 14:37:02 -05:00
Nathan Hourt
9a52c29023
Remove commented-out index code 2020-08-23 14:37:02 -05:00
Nathan Hourt
9c71579d6e
Ref #1506: Isolate chain/protocol to its own library 2020-08-23 14:37:02 -05:00
Nathan Hourt
f076bb2586
Ref #1506: Move ID types from db to protocol
The ID types, object_id and object_id_type, were defined in the db
library, and the protocol library depends on db to get these types.
Technically, the ID types are defined by the protocol and used by the
database, and not vice versa. Therefore these types should be in the
protocol library, and db should depend on protocol to get them.

This commit makes it so.
2020-08-23 14:37:02 -05:00
Nathan Hourt
9bada13735
Ref #1506: Add object_downcast_t
Allows the more concise expression `object_downcast_t<xyz>` instead of
the old `typename object_downcast<xyz>::type`
2020-08-23 14:37:02 -05:00
Nathan Hourt
14f627c014
Ref #1506: Isolate chain/protocol to its own library 2020-08-23 14:37:00 -05:00
abitmore
d54cc47a4f
Impl. pack/unpack functions for extension class 2020-08-23 14:18:59 -05:00
Nathan Hourt
9ac63ce0b3
Fixes
Remove boost::signals from build system; it's header-only so it's not
listed in cmake anymore.

Also remove some unused hashing code
2020-08-23 14:18:59 -05:00
Nathan Hourt
13a76d25ac
Replace improper static_variant operator overloads with comparators 2020-08-23 14:18:59 -05:00
Nathan Hourt
54ce371a4a
Remove all uses of fc/smart_ref
It's gone, can't use it anymore
2020-08-23 14:18:59 -05:00
Peter Conrad
68c57a7414
_hash endianness fixes 2020-08-23 14:18:59 -05:00
Nathan Hourt
cc77a08891
Fixes
Add back in some of Peter's fixes that got lost in the cherry pick
2020-08-23 14:18:59 -05:00
Peter Conrad
7eb3729cc0
Adapted to fc promise changes 2020-08-23 14:18:59 -05:00
Peter Conrad
784221986a
Separate exception declaration and implementation 2020-08-23 14:18:59 -05:00
Peter Conrad
0569efc068
Replaced fc::array with std::array 2020-08-23 14:18:59 -05:00
Nathan Hourt
ced4380664
Remove old macro
This macro is now in FC, so no need to define it here anymore
2020-08-23 14:18:55 -05:00
Vikram Rajkumar
10adf05881
Remove NTP time 2020-08-23 14:18:24 -05:00
Nathan Hourt
953a1314a3
Fixes/Remove Unused 2020-08-23 14:18:24 -05:00
John Jones
a24a6bfc68
replace smart_ref with shared_ptr 2020-08-23 14:18:22 -05:00
Peter Conrad
f25c4f6ae6
Replace fc::uint128 with boost::multiprecision::uint128_t 2020-08-23 14:16:41 -05:00