Community Update 190

Owned by Rao Bhamidipati
Sept 22, 2021

General

Sprint 96 in progress

Main Focus is to prepare the block merge Hard fork (hf2), specifically (a) merging REV balances directly - in process (b) leaderless block merge - work ongoing (c) a lot of testing + bug fixing and (d) performance improvements. Other work items include resolving any identified bugs,  hardening the network.  Current PR list is at https://github.com/rchain/rchain/pulls .  

Block merge Hard Fork (hf2)

  1. There was a slashed node on the main net due to tuple space error, something we run into once in several months. We are not able to easily replicate the error in our tests. We will likely put in a fix to replay one more time in case of such an error, to see if the node recovers from the error and proceeds. We will monitor behavior after this fix and take any additional action needed.
  2. Nutzipper is identifying and resolving logical inconsistencies / bugs in the leaderless state merge changes (based on a fringe/boundary of previous last finalized states). 
  3. Tomislav temporarily switched to trouble shooting the stalling of linear scaling after several number of nodes. He will be presenting his performance characterization work and results today. While the reason for the stalled performance scaling is still inconclusive, we are making good progress in ruling out some parts of the code as the source of the problem. While this is slow-going and frustrating, we need to look at the performance data from multiple angles and characterize it, to both identify the prioritized areas for investigation, as well as understand the causes of slow performance. Will also has been working in this area with flame graphs, to identify suspect modules, whether our own code or the libraries or machinery we use.
  4. Tomislav will continue work on rejecting double spends in the merging of REV balances directly. https://github.com/rchain/rchain/issues/3513 .
  5. Ongoing work to refactor any necessary tests including Casper tests, run and troubleshoot any identified issues.
  6. Denis' PR for RadixTree implementation of History is in https://github.com/rchain/rchain/pull/3515  with performance data for read, delete, insert operations.  Full impact of his improvements will be known only after we merge this into the main code base, which has more steps to go through.
  7. We may add one more resource to the team soon if we can work out reasonable terms.
  8. Status of various Block merge tasks is at
    1. https://github.com/rchain/rchain/projects/4

Release Plan for the rest of the year and beyond - for reference

Currently planned near term releases are

  1. Block merge release hard fork 2 plus any configuration changes that do not need extensive work - Leaderful and Leaderless block merge
  2. Configuration changes release,
  3. Performance improvements releases (one or more),
  4. Hard Fork 3 with PoS changes and other items,
  5. External validators process validation with trusted community validators (aka Friends & Family) on the test net
  6. Friends & Family validation on the main net
  7. External validators among extended ecosystem (e.g. exchanges, if they're willing)
  8. External validators in the wild.


Explanation of the different Hard Forks (for reference)

Proceeding further with block merge requires changes to the per validator vault to make it a per deploy vault/purse etc. We are currently finalizing the design for this. Actual code and test will take some time. We are potentially looking at this scenario:
1. First hard fork for balances and removing the slashed validator
2. Second hard fork for block merge - This is the fastest way to get block merge on the main net without holding it until the third hard fork. This includes REV balance merging (aka per deploy vault), attestation messages and the Tree depth being increased to 4.
3. Third hard fork with all other changes.  Third hard fork includes implementation of a lot of changes including the PoS contract changes, Soft fork process etc.  Design for soft fork process is complete and is being reviewed by the team.

(Hard Fork) TEST NET 2 ADVISORY: (for reference)

To test the version of block merge with data changes that need a hard fork, we are creating a separate feature branch and a TEST NET 2 as the testbed for this feature branch. This content will be left here until we are close to Hard Fork 2 implementation on the main net. TestNet 2 is now available for the community to test their contracts on.

  1. There will be no guarantees of data storage format compatibility on Test Net 2, as the data storage format changes are incrementally implemented. The anticipated changes are mostly storage level format changes visible only to node operators. We encourage the community to start using Test Net 2 in addition to the current test net, so that we can quickly identify and resolve any issues with the upcoming hard forks. All new development should be targeted to TEST NET 2 and all current code MUST BE tested against TEST NET 2 to ensure future compatibility. 

On all the hard forks on the main net, we will be starting from an empty state, with REV balances only. On Test Net 2, this 'loosing the state' will be repeated multiple times.

Tech-Governance meetings on Fridays 9:30 AM Eastern, 6:30 Am Pacific 

Mercury requirements and acceptance criteria

Details on the acceptance criteria: Mercury acceptance criteria

Please see the documentation at https://github.com/rchain/rchain/blob/dev/docs/features.md. 

Developer website

https://developer.rchain.coop 

Date