/
Block Storage Specification (draft)

Block Storage Specification (draft)

Jul 23, 2018

 

Introduction

Purpose:

This document aims at providing a specification for a persistent storage of blocks for the BlockDAG use case.

References:

Jira Issue: https://rchain.atlassian.net/browse/CORE-754

Definitions:

 

Scope

Objectives & goals:

separate subproject in the rchain repository
persistent
backed by LMDB
"MTL-style" of code
explicit context bounds (via implicit params)
reports metrics
identify useful typeclasses
the storage should use block identifiers (blockHash)

Non-goals:

  • refactor RSpace so that the projects share code.

 

To Be Determined

should the lmdb instance be shared with RSpace? (one file) ANSWER: NO
should the library support multiple block stores? ANSWER: NOT FOR THIS VERSION
should the Store depend on BlockHash and BlockMessage or should it be abstract K, V? ANSWER: PROBABLY NOT
what is the level of transactions?

 

Use Cases

  1. storing blocks (put)

  2. retrieving blocks (get)

  3. lookup of blocks (lookup). seems to be same as 2.

  4. view the whole db asMap

Design Considerations

Interfaces  

System Interface

 

Hardware Interface

The LMDB instance needs to be able to read and write to a file.

Software Interface

trait BlockStore[F[_]] { def put(blockHash: BlockHash, blockMessage: BlockMessage): F[Unit] def get(blockHash: BlockHash): F[Option[BlockMessage]] def put(f: => (BlockHash, BlockMessage)): F[Unit] def asMap(): F[Map[BlockHash, BlockMessage]] }

 

And instance:

class InMemBlockStore[F[_], E] private ()(implicit monadF: Monad[F], refF: Ref[F, Map[BlockHash, BlockMessage]], metricsF: Metrics[F]) extends BlockStore[F] { def get(blockHash: BlockHash): F[Option[BlockMessage]] = for { _ <- metricsF.incrementCounter("block-store-get") state <- refF.get } yield state.get(blockHash) @deprecated(message = "to be removed when casper code no longer needs the whole DB in memmory", since = "0.5") def asMap(): F[Map[BlockHash, BlockMessage]] = for { _ <- metricsF.incrementCounter("block-store-as-map") state <- refF.get } yield state def put(f: => (BlockHash, BlockMessage)): F[Unit] = for { _ <- metricsF.incrementCounter("block-store-put") _ <- refF.update { state => val (hash, message) = f state.updated(hash, message) } } yield () }

Usage example:

BlockStore[F].put { awaitingJustificationToChild -= block.blockHash _blockDag.update(bd => { val hash = block.blockHash val newChildMap = parents(block).foldLeft(bd.childMap) { case (acc, p) => val currChildren = acc.getOrElse(p, HashSet.empty[BlockHash]) acc.updated(p, currChildren + hash) } val newSeqNum = bd.currentSeqNum.updated(block.sender, block.seqNum) bd.copy( childMap = newChildMap, currentSeqNum = newSeqNum ) }) (block.blockHash, block) }

 

User Interface

None. We are backend developers. We hail the matrix, and the matrix speaks to us.

Communications Interface

System Overview

Provide a description of the software system, including its functionality and matters relating to the overall system and design.  Feel free to split this up into subsections, or use data flow diagrams or process flow diagrams.  

Limitations

Assumptions and Dependencies

Architectural Strategies

Describe any design decisions and/or strategies that affect the overall organization of the system and its higher-level structures. These strategies should provide insight into the key abstractions and mechanisms used in the system architecture. Describe the reasoning employed for each decision and/or strategy (possibly referring to previously stated design goals and principles) and how any design goals or priorities were balanced or traded-off. Such decisions might concern (but are not limited to) things like the following:

  • Use of a particular type of product (programming language, database, library, etc. ...)

  • Reuse of existing software components to implement various parts/features of the system

  • Future plans for extending or enhancing the software

  • User interface paradigms (or system input and output models)

  • Hardware and/or software interface paradigms

  • Error detection and recovery

  • Memory management policies

  • External databases and/or data storage management and persistence

  • Distributed data or control over a network

  • Generalized approaches to control

  • Concurrency and synchronization

  • Communication mechanisms

  • Management of other resources

 

System Architecture