This document is deprecated. Please reference: (deprecated) RSpace 0.1 Specification
1. Introduction
The purpose of this document is to provide a specification of the RChain storage layer that is sufficient to allow an engineer to write a package that satisfies the available requirements, to the extent that the requirements are known. The areas where the requirements are not available are noted.
...
The important RChain document for the storage layer is the “RChain Platform Architecture,” found at http://rchain-architecture.readthedocs.io/en/latest/index.html. The other sources that are referenced in this document are listed in the Bibliography.
2. RChain Architecture Overview
RChain is a “decentralized, economic, censorship-resistant, public compute infrastructure and blockchain. It will host and execute programs popularly referred to as ‘smart contracts’. It will be trustworthy, scalable, concurrent, with proof-of-stake consensus and content delivery.” (Arch Doc)
...
Since the validator functions are executed by the RhoVM, it is considered a part of the RhoVM for the purposes of the storage layer.
2.1 Execution Layer
RhoVM operates against a key-value data store. A state change of RhoVM is realized by an operation that modifies which key maps to what value. (p. 20)
2.1.2 RhoVM Execution Environment
The RhoVM Execution Environment is a Scala implementation of the Rosette VM. Rosette is a reflective, object-oriented language that achieves concurrency via actor semantics. (Arch Doc)
2.1.3 Rholang and Contracts
2.1.3.1 Continuations
Token contract uses continuations.
...
Kent says that there is no way to detect that a continuation won't be executed again. This is unfortunate because it would shrink the database to prune continuations that aren't needed.
2.1.3.2 Destructure of Keys, Tuplespace, Namespaces
Example of destructured assignment to key
...
Keys in tuplespace map to storage.
2.2 Storage Layer
A variety of data is supported, including public unencrypted json, encrypted BLOBs, or a mix. This data can also be simple pointers or content-hashes referencing off-platform data stored in private, public, or consortium locations and formats. (page 26, Arch Doc)
...
- The node first queries its in-memory cache for the requested data. Then if it is not found it,
- queries its local store, and, if it is not found, stores a delimited continuation at that location, and
- queries the network. If and when the network returns the appropriate data, the delimited continuation is brought back in scope with the retrieved data as its parameter.”
(page 27, Arch Doc)
2.2.1 Requirements of the Execution Layer
2.2.1.1 Prolog and Unification
In the full implementation of RChain, keys will be Prolog terms. For Mercury, the keys will be flat and of the form rholang-X, where X is a hexadecimal number. TODO I didn’t ask for the length of the ID.
...
A key will either be associated with a set of data or a set of continuations because data and continuations are noever on the same channel.
2.2.1.2 Requirements of the RhoVM
Does VM have registers that need to be stored? No.
...
The Rholang primitive types are Booleans, Integers, Double, and Strings as described in rchain/rholang/src/main/bnfc/rholang.cf.
2.2.1.3 Blocks
Blocks are not just a list of transactions. They are little programs, which extend out to the affected transactions list. The programs that have consensus around them, they have the most 'weight' behind the proposal - this is stake weight. The 'fork choice' rule says that when you take a fork, it's the one that has the most stake behind it.
...
Does the blockchain require an index? TBD
2.2.1.3.1 A Block can have multiple parents
2.2.1.3.2 Open questions
https://rchain.atlassian.net/wiki/spaces/CORE/pages/10223617/Open+questions says that the following are not specified:
...
Encrypted blobs (page 26 of RChain Arch)
2.2.1.3.3 To Be Done
Logging
Monitoring
2.2.2 Requirements of the Communications Layer
Although the requirements that the communications layer has of the storage layer are not confirmed yet, the following requirements appear likely.
...
- ID
- IP address
- last know reputation
- black-list status
3. Design Considerations
3.1 Design Requirements
In general, the requirements of the storage layer are to:
- provide query unification and key-value storage for the RhoVM
- provide key-value storage for the communications layer
- store all the data necessary for the node to reconstruct its state after a crash, up to the most recently committed transaction
3.1.1. Unifier Requirements
Greg says we need arrays and structures for Mercury.
...
Keys from RhoVM are strings of the form of "rholang-" followed by a string representation of a numeric ID.
3.1.2 Node and Lmdbjava Requirements
While the hardware and software requirements to run a node are not defined yet, the hardware and software requirements of the storage layer are the same as those to run lmdbjava, the key-values store that underlies the storage layer.
...
Roscala is implementing serialization so the storage layer doesn't need to.
3.1.3 Limitation on size of value
In a conversation on September 11, 2017, Mike asked Greg how big the blogs stored as values in the storage layer would be. He asked “Kilobytes? Megabytes? Gigabytes?” Greg answered, “Megabytes.” During the week of November 6, 2017, I spoke with Kent multiple times about if there was a maximum size on blobs or continuations. Kent had conferred with Greg and confirmed that there is no limit. I told Kent that the storage layer will have to set a limit on the size of a value.
...
- Allow keys and values to have a structure like Prolog terms. Call this structure a “Prolog structure.”
- Support Prolog unification such that a key that has Prolog structure can be unified with keys or values in the database that have Prolog structure
3.2 Performance Requirements
To become a blockchain solution with industrial-scale utility, RChain must provide content delivery at the scale of Facebook and support transactions at the speed of Visa. website
“The most that Bitcoin can do on a good day is 10 transactions a second,” says Greg Heuss, an adviser helping RChain with its fundraising. “Visa can do 40,000 transactions in the same time.”
https://www.geekwire.com/2017/seattle-based-rchain-takes-bitcoin-ethereum-new-blockchain-tech/
4. System Architecture
The storage layer has two parts: a unifier and two Lmdbjava databases. The diagram below indicates that each lmdbjava interface wraps an lmdb instance, which is written in C. Further, each lmdb instance makes use of two database files: data.mdb and lock.mdb.
4.1 Unifier Subsystem
The implementation of unification in the storage layer is a restricted form of Prolog. Specifically, the storage layer includes Prolog facts but not rules.
...
Specify unification algorithm
4.1.1 Use Cases
4.2 Lmdbjava Subsystem
4.2.1 Lmdb and lmdbjava
Lmdb is written in C and located at https://symas.com/lmdb/.
...
Most methods in this package will throw a standard Java exception for failing preconditions (eg NullPointerException if a mandatory argument was missing) or a subclass of LmdbException for precondition or LMDB C failures. The majority of LMDB exceptions indicate an API usage or Env configuration issues, and as such are typically unrecoverable.
http://www.javadoc.io/doc/org.lmdbjava/lmdbjava/0.6.0
4.2.1 Lmdbjava does not support unsigned types
Lmdbjava is a Java package and Java does not have native unsigned types and as a consequence the storage layer does not offer unsigned types in its interface. There are Java packages that simulate unsigned types that we can add to address this deficiency.
4.2.2 Lmdb contrains the size of a key
The biggest key that lmdb accepts is 511 bytes.
4.2.3 Lmdb constrains the size of MDB_DUPSORT values
If lmdb is configured with MDB_DUPSORT then the size of a value associated with a key cannot exceed 511 bytes. See lmdb C code in mdb.c.
4.2.4 A key can be associated with multiple large keys
Greg, Mike, and I discussed the storage layer requirements in a call on September 11, 2017. Greg mentioned that each key may be associated with multiple blobs. Neither Mike nor I asked Greg if this requirement is a necessity. Further, on November 7, 2017, Kent confirmed that Rholang contracts require that a key be associated with multiple keys.
...
“LMDBJAVA SMALL-VALUES” is keyDb.“LMDBJAVA BIG-VALUE” is valueKeyDb
“SMALL-VALUES” is plural because… while “BIG-VALUE” is singular because …
4.2.5 Uses Cases
Example of use of unification in Rholang code from Kent
...
anything!([1, 2, 3 ])
|
for (rholang-0@[ rholang-1, rholang-2, rholang-3 ] <- anything) {
print(rholang-1, rholang-2, rholang-3)
}
|
for (rholang-0@[ rholang-1, rholang-2, rholang-3 ] <- anything) {
print(rholang-1, rholang-2, rholang-3)
}
|
for (rholang-0@[ rholang-1, rholang-2, rholang-3 ] <- anything) {
print(rholang-1, rholang-2, rholang-3)
}
4.3 Unification and Lmdbjava
Working together
Note that this diagram abstracts out the 2 lmdbjava databases.
5 Bibliography
RChain Overview
...