2018-05-31 Meeting notes: RSpace team meeting

2018-05-31 Meeting notes: RSpace team meeting

Date

May 31, 2018

Attendees

  • @Henry Till

  • @Łukasz Gołębiewski (Unlicensed)

  • @Dominik Zajkowski

  • @Yaraslau Levashkevich

  • @Kelly Foster

  • @Medha Parlikar (Unlicensed)

Goals

  • Discuss where things are.

  • Discuss what comes next.

  • Cut some placeholder tickets for next Sprint.

Action items

@Łukasz Gołębiewski (Unlicensed) and @Dominik Zajkowski look for ticket on log batch pruning 
@Łukasz Gołębiewski (Unlicensed) and @Dominik Zajkowski look for tickets for skip block work
@Yaraslau Levashkevich look for tickets related to metrics
@Medha Parlikar (Unlicensed) confirm what the expectations are for the performance of a single node

Discussion items

Item

Who

Notes

Item

Who

Notes

Where we are: History & Rollback

Henry

What needs to happen: History & Rollback

Henry + Lukasz + Dom

  • Let's talk about the TrieUpdate log

    • Demo of the current implementation of getCheckpoint

    • IDEA - Sort by hash, if matching and the last is a delete, then don't need to put into the Trie

      • Don't want to put log of everything in the Trie

      • TICKET - batch log pruning (Łukasz)

    • There will be logging at two levels: batch and trace

  • Structure of PointerBlock "cells" and minimizing unnecessary db access in deleteLeaf 

    • Issue - when there are two children and you know one, you have to go to the store to find out who the other child is, and the info available in the cell is limited.

      • IDEA

    • Discussion about node child and leaf child

  • Skip blocks

    • Ethereum calls them "extension nodes"

    • TICKET - assign creation of skip block to Łukasz and Dom

  • Transfer 

MDB_NOTLS & Concurrency Tests

Henry + Yaraslau

  • Metrics - discussion about Prometheus and costs of calculating averages inside RSpace

    • IDEA there are independent libraries that can push counters to Prometheus

    • DECISION - Let's review how we are doing this and come up with a better way to avoid calculations in RSpace. Goal - RSpace emits metrics

    • TICKET - revise how metrics are emitted and then calculated externally

    • TICKET - calculation of comm events per second for Sprint 10

    • TICKET - investigate reads and writes to LMDB for metrics

  • Discussion of MDB_NOTLS

    • Tickets created and assigned to Yaraslau

    • There must be a locking mechanism at the level of produce and consume

    • IDEA expose memory and write to LMDB

    • Unclear if we can use namespaces. Sharding and namespaces come with RNode v0.6

Execution traces

Henry + Yaraslau

  • Henry and Michael Birch are working together at this point. Will bring Yaraslau in later after metrics and concurrency work in Sprint 10.

What about: inmem implementation

Dom

  • How much can we afford to keep inMemoryStore?

  • Discussion of hot vs cold storage and how it applies to cost model

    • This may not be a part of Mercury. Rent may be.

    • IDEA - have separate APIs to support transfer from hot to cold or a single API

      • Will need a process

      • Will need to understand costs

      • Will need garbage collection

        • Two types: 1) language level and 2) higher level consensus

  • Henry to review https://github.com/rchain/rchain/pull/861 by EOD

  • Discussion about transaction work Dom is doing

    • Could it be done in Sprint 10?

      • Depends on fancy or simple implementation

      • DECISION simple model of LMDB transactions can be done in Sprint 10

sealed trait Child case class NodeChild(hash: Blake2b256Hash) extends Child case class LeafChild(hash: Blake2b256Hash) extends Child case object Nothing extends Child