From b7436cffce0236539ed1c58876501afd86bc52c7 Mon Sep 17 00:00:00 2001 From: Vikram Rajkumar Date: Wed, 10 Jun 2015 16:31:07 -0400 Subject: [PATCH] Move docs to submodule for wiki https://github.com/cryptonomex/graphene/wiki --- .gitmodules | 15 ++-- DEV_README.md | 29 -------- build-ubuntu.md | 21 ------ doc/draft-license/HEADER.draft | 14 ---- doc/draft-license/LICENSE.draft | 19 ----- doc/witness-rng.md | 37 --------- doc/witness-scheduler.md | 128 -------------------------------- docs | 1 + 8 files changed, 10 insertions(+), 254 deletions(-) delete mode 100644 DEV_README.md delete mode 100644 build-ubuntu.md delete mode 100644 doc/draft-license/HEADER.draft delete mode 100644 doc/draft-license/LICENSE.draft delete mode 100644 doc/witness-rng.md delete mode 100644 doc/witness-scheduler.md create mode 160000 docs diff --git a/.gitmodules b/.gitmodules index 3e39da4a..c78c21f3 100644 --- a/.gitmodules +++ b/.gitmodules @@ -1,8 +1,11 @@ +[submodule "docs"] + path = docs + url = git@github.com:cryptonomex/graphene.wiki.git [submodule "libraries/fc"] - path = libraries/fc - url = git@github.com:cryptonomex/fc - ignore = dirty + path = libraries/fc + url = git@github.com:cryptonomex/fc.git + ignore = dirty [submodule "libraries/leveldb"] - path = libraries/leveldb - url = https://github.com/bitcoin/leveldb.git - ignore = dirty + path = libraries/leveldb + url = git@github.com:bitcoin/leveldb.git + ignore = dirty diff --git a/DEV_README.md b/DEV_README.md deleted file mode 100644 index b8133fea..00000000 --- a/DEV_README.md +++ /dev/null @@ -1,29 +0,0 @@ - -How to use fc async to do recurring tasks ------------------------------------------ - - _my_task = fc::async( callable, "My Task" ); - _my_task = fc::schedule( callable, "My Task 2", exec_time ); - -Stuff to know about the code ----------------------------- - -`static_variant` is a *union type* which says "this variable may be either t1 or t2." It is serializable if t1 and t2 are both serializable. - -The file `operations.hpp` documents the available operations, and `database_fixture.hpp` is a good reference for building and submitting transactions for processing. - -Tests also show the way to do many things, but are often cluttered with code that generates corner cases to try to break things in every possible way. - -Visitors are at the end of `operations.hpp` after the large typedef for `operation` as a `static_variant`. TODO: They should be refactored into a separate header. - -Downcasting stuff ------------------ - -- You have an `object_id_type` and want to downcast it to a `key_id_type` : `key_id_type( object_id )` -- You have an `operation_result` and want to downcast it to an `object_id_type` : `op_result.get()` -- Since `operation_result` is a `static_variant`, the above is also how you downcast `static_variant` - -Debugging FC exceptions with GDB --------------------------------- - -- `catch throw` diff --git a/build-ubuntu.md b/build-ubuntu.md deleted file mode 100644 index 5f02ff82..00000000 --- a/build-ubuntu.md +++ /dev/null @@ -1,21 +0,0 @@ - -The Boost which ships with Ubuntu 14.04 LTS is too old. You need to download the Boost tarball for Boost 1.57.0 -(Note, 1.58.0 requires C++14 and will not build on Ubuntu LTS; this requirement was an accident, see ). Build Boost as follows: - - # tarball available at http://sourceforge.net/projects/boost/files/boost/1.57.0/boost_1_57_0.tar.bz2/download - # sha256sum is 910c8c022a33ccec7f088bd65d4f14b466588dda94ba2124e78b8c57db264967 - - BOOST_ROOT=$(echo ~/opt/boost_1_57_0) - - # build Boost from source - cd ~/src/boost_1_57_0 - ./bootstrap.sh --prefix=$BOOST_ROOT - ./b2 link=static variant=debug threading=multi stage - ./b2 link=static variant=debug threading=multi install - -Then we need to tell `cmake` to use the Boost we just built, instead of using the system-wide Boost: - - cd ~/src/graphene - [ -e ./doc/build-ubuntu.md ] && sh -c 'cmake -DBOOST_ROOT="$BOOST_ROOT" -DCMAKE_BUILD_TYPE=Debug .' - -If all goes well, you should see the correct Boost version in the output messages to the above command. diff --git a/doc/draft-license/HEADER.draft b/doc/draft-license/HEADER.draft deleted file mode 100644 index b8e211bd..00000000 --- a/doc/draft-license/HEADER.draft +++ /dev/null @@ -1,14 +0,0 @@ -/* - * This is draft license text currently under development. This license does not apply to anything right now. - * - * Copyright (c) 2015, Cryptonomex, Inc - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: - * - * 1. No distribution of binaries compiled from modified versions of the code - * - * 2. Any modifications become property of Cryptonomex, Inc and must be licensed under these same terms - * - * 3. Any hosting provider is authorized to delete any version of this code at the request Cryptonomex, Inc. - */ diff --git a/doc/draft-license/LICENSE.draft b/doc/draft-license/LICENSE.draft deleted file mode 100644 index 90c85fd5..00000000 --- a/doc/draft-license/LICENSE.draft +++ /dev/null @@ -1,19 +0,0 @@ -This is draft license text currently under development. This license does not apply to anything right now. - -Copyright (c) 2015, Cryptonomex, Inc -All rights reserved. - -The graphene source code is proprietary and may not be used for any purpose without written permission from Cryptonomex, Inc. The source code is -provided publicly for the sole purpose of allowing users to audit the code and compile their own executables. Blockchain technology depends upon -a strong consensus on the rules of an open protocol; this source code is provided to completely document the protocol rules (bugs and all). - -Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: - -1. No distribution of binaries compiled from modified versions of the code - -2. Any modifications made to the code become property of Cryptonomex, Inc and must be licensed under these same terms - -3. The software may be forked/cloned on github for the purpose of submitting bug fixes and pull requests. All forked repositories are property of Cryptonomex and by -forking you authorize Github.com to delete your fork at the request Cryptonomex, Inc. - -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/doc/witness-rng.md b/doc/witness-rng.md deleted file mode 100644 index 672adad2..00000000 --- a/doc/witness-rng.md +++ /dev/null @@ -1,37 +0,0 @@ - -Witness scheduler RNG ---------------------- - -The witness scheduler RNG is a random number generator which uses the -blockchain random number generator state as its seed. - -The witness scheduler RNG creates an infinite stream of random bytes -by computing `sha256( sha256( seed ) + little_endian_64bit(i) )`, increasing -`i` from 0 to 1 to 2, etc. The RNG only runs during a scheduling block, -and `i` starts from `0` in each scheduling block (relying on different -seeds to produce different results). - -This infinite stream of random bytes is equivalent to an infinite -stream of random bits in little bit-endian order. Given a bound `B`, -the bitstream can be used to produce a random number uniformly -distributed in the range `[0, B)` using a sample-and-reject algorithm: - -- Let `n` be the smallest integer such that `2^n >= B`. -- Let `x` be the next `n` bits from the bitstream, interpreted as an integer in little bit-endian order. -- If `x <= B`, return `x`. Otherwise, throw `x` away and repeat. - -The worst-case running time is unbounded, but each iteration has a -termination probability greater than one half. Thus the average-case -running time is `2` iterations, and a running time of more than `N` -iterations will occur (on average) at most once every `2^N` -RNG queries (assuming a worst-case choice of e.g. `B = 2^63+1` for all -queries). Since each RNG query schedules a witness, the query rate -is (over the long term) equal to the block production rate (although -in practice many queries are all performed at once in scheduling -blocks). So while it is, in theory, possible for the algorithm to -require more than 1000 iterations, in practice this will occur on average -only once every `2^1000` blocks (again assuming all queries have -worst-case `B`). - -The sample-and-reject algorithm is totally unbiased; every `x` value -has equal probability. diff --git a/doc/witness-scheduler.md b/doc/witness-scheduler.md deleted file mode 100644 index 3308a393..00000000 --- a/doc/witness-scheduler.md +++ /dev/null @@ -1,128 +0,0 @@ - -Turn/Token witness scheduling algorithm ---------------------------------------- - -The algorithm which determines the order of witnesses is referred -to as the *witness scheduling algorithm*. - -This was designed by a community bounty in thread -https://bitsharestalk.org/index.php/topic,15547.0 -however, Graphene has an additional requirement which -is not taken into account by the solutions in the thread: - -The membership and length of the list of witnesses may change over -time. - -So in this article I'll describe my solution. - -Turns and tokens ----------------- - -The solution is based on terms of *turns* and *tokens*. - -- Newly inserted witnesses start out with a turn and a token. -- In order for a witness to be scheduled, it must have a turn and a token. -- The scheduler maintains a FIFO of witnesses without tokens. -- If no witness has a turn, then the scheduler gives a turn to all witnesses. This is called "emitting a turn." -- While less than half of the witnesses have tokens, give a token to the first witness in the FIFO and remove it from the FIFO. -- Schedule a witness by picking randomly from all witnesses with both a turn and token. -- When a witness is scheduled, it loses its turn and token. - -The generic scheduler ---------------------- - -The generic scheduler implements turns and tokens. It only depends -on the C++11 stdlib and boost (not even using fc). Types provided -by Graphene are template parameters. - -The generic far future scheduler --------------------------------- - -The far future scheduler is implemented with the following rules: - -- Run until you emit a turn. -- Record all witnesses produced. -- Run until you emit a second turn. -- The witnesses produced between the emission of the first turn (exclusive) -and emission of the second turn (inclusive) are called the *far future schedule*. - -Then the schedule for the rest of time is determined by repeating -the future schedule indefinitely. The far future scheduler is required -to give the scheduling algorithm bounded runtime and memory usage even -in chains involving very long gaps. - -Slots ------ - -Due to dynamic block interval, we must carefully keep in mind -the difference between schedule slots and timestamps. A -*schedule slot number* is a positive integer. A slot number of `n` -represents the `n`th next block-interval-aligned timestamp after -the head block. - -Note that the mapping between slot numbers and timestamps will change -if the block interval changes. - -Scheduling blocks ------------------ - -When each block is produced, the blockchain must determine whether -the scheduler needs to be run. If fewer than `num_witnesses` are -scheduled, the scheduler will run until `2*num_witnesses` are scheduled. -A block in which the scheduler runs is called a *scheduling block*. - -Changes in the set of active witnesses do not modify the existing -schedule. Rather, they will be incorporated into new schedule entries -when the scheduler runs in the next scheduling block. Thus, a witness -that has lost an election may still produce 1-2 blocks. Such a witness -is called a *lame duck*. - -Near vs. far schedule ---------------------- - -From a particular chain state, it must be possible to specify a -mapping from slots to witnesses, called the *total witness schedule*. -The total witness schedule is partitioned into a prefix, called the -*near schedule*; the remainder is the *far schedule*. - -When a block occurs, `n` entries are *drained* (removed) from the head -of the total schedule, where `n` is the slot number of the new block -according to its parent block. - -If the block is a scheduling block, the total schedule is further -transformed. The new near schedule contains `2*num_witnesses` entries, -with the previous near schedule as a prefix. The rest of the near -schedule is determined by the current blockchain RNG. - -The new far schedule is determined by running the far future scheduler, -as described above. The far future scheduler also obtains entropy -from the current blockchain RNG. - -As an optimization, the implementation does not run the far future -scheduler until a far-future slot is actually queried. With this -optimization, the only circumstance under which validating nodes must -run the far future scheduler is when a block gap longer than `num_witnesses` -occurs (an extremely rare condition). - -Minimizing impact of selective dropout --------------------------------------- - -The ability of any single malicious witness to affect the results of the -shuffle algorithm is limited because the RNG is based on bit commitment -of the witnesses. However, a malicious witness *is* able to -refuse to produce a block. A run of `m` consecutively scheduled -malicious witnesses can independently make `m` independent choices -of whether to refuse to produce a block. Basically they are able to -control `m` bits of entropy in the shuffle algorithm's output. - -It is difficult-to-impossible to entirely eliminate "the last person -being evil" problem in trustless distributed RNG's. But we can at least -mitigate this vector by rate-limiting changes to the total witness -schedule to a very slow rate. - -If every block schedules a witness, our adversary with `m` malicious -witnesses gets `m` chances per round to selectively drop out in order -to manipulate the shuffle order, allowing `m` attacks per round. -If witnesses are only scheduled once per round, -a selective dropout requires the malicious witness to produce the -scheduling block, limiting the probability to `m/n` attacks per round. diff --git a/docs b/docs new file mode 160000 index 00000000..1b53a8ec --- /dev/null +++ b/docs @@ -0,0 +1 @@ +Subproject commit 1b53a8eca77783d073ce7cc95991447c3f34b927