Block Storage Specification (draft)
Introduction
Purpose:
This document aims at providing a specification for a persistent storage of blocks for the BlockDAG use case.
References:
Jira Issue: https://rchain.atlassian.net/browse/CORE-754
Definitions:
Scope
Objectives & goals:
separate subproject in the rchain repository
persistent
backed by LMDB
"MTL-style" of code
explicit context bounds (via implicit params)
reports metrics
identify useful typeclasses
the storage should use block identifiers (blockHash)
Non-goals:
refactor RSpace so that the projects share code.
To Be Determined
should the lmdb instance be shared with RSpace? (one file) ANSWER: NO
should the library support multiple block stores? ANSWER: NOT FOR THIS VERSION
should the Store depend on BlockHash and BlockMessage or should it be abstract K, V? ANSWER: PROBABLY NOT
what is the level of transactions?
- 1 Introduction
- 1.1 Purpose:
- 1.2 References:
- 1.3 Definitions:
- 2 Scope
- 2.1 Objectives & goals:
- 2.2 Non-goals:
- 2.3 To Be Determined
- 3 Use Cases
- 4 Design Considerations
- 5 Interfaces
- 5.1 System Interface
- 5.2 Hardware Interface
- 5.3 Software Interface
- 5.3.1 BlockStore v1
- 5.3.2 BlockStore v1 object
- 5.3.3 BlockStore usage
- 5.4 User Interface
- 5.5 Communications Interface
- 6 System Overview
- 7 Limitations
- 8 Assumptions and Dependencies
- 9 Architectural Strategies
- 10 System Architecture
Use Cases
storing blocks (put)
retrieving blocks (get)
lookup of blocks (lookup). seems to be same as 2.
view the whole db asMap
Design Considerations
Interfaces
System Interface
Hardware Interface
The LMDB instance needs to be able to read and write to a file.
Software Interface
BlockStore v1
trait BlockStore[F[_]] {
def put(blockHash: BlockHash, blockMessage: BlockMessage): F[Unit]
def get(blockHash: BlockHash): F[Option[BlockMessage]]
def put(f: => (BlockHash, BlockMessage)): F[Unit]
def asMap(): F[Map[BlockHash, BlockMessage]]
}And instance:
BlockStore v1 object
class InMemBlockStore[F[_], E] private ()(implicit
monadF: Monad[F],
refF: Ref[F, Map[BlockHash, BlockMessage]],
metricsF: Metrics[F])
extends BlockStore[F] {
def get(blockHash: BlockHash): F[Option[BlockMessage]] =
for {
_ <- metricsF.incrementCounter("block-store-get")
state <- refF.get
} yield state.get(blockHash)
@deprecated(message = "to be removed when casper code no longer needs the whole DB in memmory",
since = "0.5")
def asMap(): F[Map[BlockHash, BlockMessage]] =
for {
_ <- metricsF.incrementCounter("block-store-as-map")
state <- refF.get
} yield state
def put(f: => (BlockHash, BlockMessage)): F[Unit] =
for {
_ <- metricsF.incrementCounter("block-store-put")
_ <- refF.update { state =>
val (hash, message) = f
state.updated(hash, message)
}
} yield ()
}Usage example:
BlockStore usage
BlockStore[F].put {
awaitingJustificationToChild -= block.blockHash
_blockDag.update(bd => {
val hash = block.blockHash
val newChildMap = parents(block).foldLeft(bd.childMap) {
case (acc, p) =>
val currChildren = acc.getOrElse(p, HashSet.empty[BlockHash])
acc.updated(p, currChildren + hash)
}
val newSeqNum = bd.currentSeqNum.updated(block.sender, block.seqNum)
bd.copy(
childMap = newChildMap,
currentSeqNum = newSeqNum
)
})
(block.blockHash, block)
}User Interface
None. We are backend developers. We hail the matrix, and the matrix speaks to us.
Communications Interface
System Overview
Provide a description of the software system, including its functionality and matters relating to the overall system and design. Feel free to split this up into subsections, or use data flow diagrams or process flow diagrams.