Project Planning July 25, 2018

Risk in the project - LMDB

@Henry Till

@Michael Birch (Unlicensed)

• Suspect that it is in the Java layer - worth performing testing in LMDB to ensure that all exceptions are propagated

• Observed that the log being provided to CASPER not always complete.  Have not yet isolated the log.

• It stands to reason if anything in LMDB can fail without throwing an exception or error.  The C layer has it's kind of error handling, and the Java layer will attempt to handle this by passing up Java exceptions. 

• Steps:

  • Comb through their issue tracker → @Medha Parlikar (Unlicensed) - see if there are any issues  Assign to one of the RSpace team members.

  • Use Michael's test case, reduce the test case further. 

  • We should consider creating more tests for other areas in LMDB.

  • 2 pronged approach - 1 person goes and ferrets out the map size error and where it is happening.

  • 1 person goes and hunts for other sources of non-determinism.

    • Run Jeremy's perf test with some very high 10GB map size limit.

  • Nothing in the RSpace design is coupled to LMDB - FYI.  It would take approx a sprint to swap it out for another KVS. 

  • What is the status of the in-memory RSpace implementation?

    • It has been tested in the context of RSpace.  There are no tests for Casper /Rholang.  There are not higher level integration tests for it.  

    • Dom has been keeping it up to date with the current one.  It is available to us as well.

    • Definitely a good intermediate step before using another KVS.

    • No in memory history implementation.

    • Create a ticket to run with the in memory implementation & run Jeremy's script.

      • Swap out the normal store for RSpace in memory

      • If the test proves good

        • Create in memory trie store.  Lower priority.

  • Next steps

    • Run with Jeremy's test with an high map size, 10GB. Attempt to create anything that would non-determinism.

    • Task RSpace team to switch to the in memory store

    • Look at LMDB issue log

    • Audit projects that consume rspace to ensure that they are correctly handling exceptions coming from RSpace and below.

Risk - Consensus

@Michael Birch (Unlicensed)

• These things in flight or are not started and are at risk due to lack of resources

  • Non-Determinism bugs are taking time to work through

  • Slashing work not started

  • Challenge/response protocol work not started.

  • Needs help from Rholang team if possible.

    • Rholang team will help starting next sprint.  Need to get some more convenience methods done.

Risk- Rholang

@Kyle Butt

5 min

Communications Risks

@Pawel Szulc (Unlicensed)

• Clean up circular references in Communications between Kademlia & RChain protocol & transport

• Possible impacts to communications which will impede consensus testing.

• Work off of feature branch & test? 

• Signing messages with validator private key

  • Discussion done with Kyle earlier

  • Pawel to write up findings

• Concern: how many connections should node consistently keep running?

  • Raised with Nash. Waiting for response.

  • Pawel and Sebastian to write up to make a decision for how to understand this risk related to testnet and mainnet

5 mins

Risk- Sharding

@Michael Stay (Unlicensed)

• Mike doing work while Ovidiu is on vacation.

• Writing up last week's and this week's research today and then shifting to doing the Rholang work.

• Concern: what is the state of the documentation and need to share with the community

  • Mike to update as he works on the Rholang contracts

• API page: Concern: need to update page on the wiki to get details on how to interact with the gRPC api for Rholang and later Scala code that interacts with contracts and supports signatures this is required in order to support cross shard interactions.

• Still working through some design issues

• Pseudocode being authored

• Question: if I listen to a message on a parent shard, can I deploy something on a child shard?
  
  

  • This would need API #2 to show finalization in the parent

  • This would need API #1 to deploy something on the child

  • All of this is in Scala (could be any language, but scala's reasonable)

  • The Client part is separate from the server part. It would be nice if we could ask 'who are your peers' and just ask them.

• Question - Comms and Sharding? @Pawel Szulc (Unlicensed)

  • Would be good to use the existing knowledge of the peers to communicate with them.

  • First implementation, Comms would not need to know about sharding.

  • The blocks for all shards are not being passed around the network (Birch) 

• Read only nodes cannot make blocks.  Not running a CASPER instance with a validator identiy.

• They receive blocks but cannot produce blocks.  They cannot make their own blocks.

• Gossiping: Mike and Medha have discussed, and work is not in scope at this time