Community Update 177

Owned by Rao Bhamidipati
Jun 23, 2021

General

  • Status of the networks and Releases
    • Main net nodes are running the Last Finalized State version 0.10.2
    • Current version running on Test net is 0.11.0.alpha-2.  Current test net info is at RChain public testnet information
    • We discovered a bug with the additional block merge test net meant to be available to the community. For now, it is available but not really usable by the community because it crashes after the first rev transfer.

Sprint 88 in progress

Main Focus is to continue Block merge work and improvements, resolve any identified bugs, prepare for hard fork 1 to eliminate the slashed validator, harden the main net, improve performance.  Current PR list is at https://github.com/rchain/rchain/pulls .  

Significant work is ongoing in the following streams simultaneously
(1) Leaderful block merge targeted for the current main net and Leaderless block merge changes that will get on the main net, even though may not become immediately operational: 

Lot of testing has been done on the the release candidate 1. Completed bug fixes are at https://github.com/rchain/rchain/pulls?q=is%3Apr+is%3Aclosed. The focus is on thoroughly testing this block merge version for the main net. More tests are being written, to continue through next week.

One of the bugs discovered exposed an issue in the old code that would not work with the block merge changes. Fixing this requires changes to the PoS contract. This and other required changes have led to us modifying the sequence of milestones/deliverables. 
We will now focus on getting the Hard Fork 1 out with balances and this one fix, so that further testing and updates can be done.


(2) Block merge tasks that can be on the main net only after Hard fork 2
No new activity here but we will create a hard fork 2 test net for this branch after leaderful block merge is complete. This hf2-testnet will get updates as various parts of hard fork 2 changes are worked on. It's not intended for any testing that relies upon long running state.

(3) Hard fork 1 for balances
The coding work is mostly complete. Will's PR to provide a transaction API for both user and system deploys https://github.com/rchain/rchain/pull/3433 is under review. This PR enables each node to provide transaction API services without having to go through the transaction server.

We now need to focus on documenting the processes, testing the genesis, providing the community the balances and sufficient time to review, etc. The tasks are listed in the secure repository.


(4) Planning for Hard fork 2 - identifying all items/tasks necessary, analyzing impact and interactions, specifying and reviewing the designs etc.
Ongoing.  

The main identified categories for Hard Fork 2 items are
4.1 Hard Fork changes needed for performant leaderless block merge
4.2 PoS changes to eliminate slashing errors and other shard configuration changes
4.3 Safe Soft-fork process: an approach to create a safe process for soft forks, such that all changes need not go in as part of the hard fork.
4.4 Revvault.rho changes.
4.5 Replacing Protobufs with Cap'nProto - 
4.6 Bug fixes: The 'end of quarantine' bug we discussed last week, rewards calculation (dependent on BigInt) and other bugs needs to be fixed as part of hard fork 2 also.
4.7 Additional optimizations for performance - Significant progress here. Tomislav will discuss a 10x improvement opportunity with new history implementation.

All the above contract changes will be implemented in rholang v1.1 


(5) Implementing Hard fork 2 changes 
Isaac is working on improvements to the revvault.rho PR https://github.com/rchain/rchain/pull/3418
Significant progress on Safe Soft-fork process: An elegant initial design spec is agreed on and is being documented. Greg will present an abbreviated version of that today.
Tomislav and Denis will be working closely on the replacement of protobufs with Cap'nProto. Quite a lot of work remains to be done in this area. Once completed, this may allow us to upgrade to Blake3 from Blake2 subject to the availability of high quality libraries.
Further analysis of channels map issues revealed that the LMDB writes is the area that could use performance improvements. We will look into possible optimizations here.

(6) Full realization of performance improvements on the main net will require substantial hardware budget. We will be publishing the performance data at various number of node configurations. What is optimal to run on the main net given the transaction volumes will be determined. We're looking at how best to get to fast finalization as the main performance indicator, measured against per time, per cost, per power utilization etc.


-- END OF NEW UPDATES --


(6) Performance tests and performance improvements
Ongoing - no change this week. This section is left here for reference until we have updates.

VERY Preliminary performance numbers on the leaderful, 0.99 synchrony constraint (unoptimized) block merge version that will be eventually promoted to the main net:

7x to 10x improvement (currently 41 sec per block on the main net vs 4 to 6 secs per block with block merge) over what we see on the main net currently, with 5 nodes on testnet 2, with the 0.11.0.alpha-2 version.  

Numbers likely to change as we move to leaderless 0.67 synchrony constraint, remove channels map, vary the number of nodes, vary the number of blocks for tests, vary machine resources, vary network distribution etc.. 

(Hard Fork) TEST NET 2 ADVISORY:  To test the version of block merge with data changes that need a hard fork, we are creating a separate feature branch and a TEST NET 2 as the testbed for this feature branch. This content will be left here until we are close to Hard Fork 2 implementation on the main net. 

  1. There will be no guarantees of data storage format compatibility on Test Net 2, as the data storage format changes are incrementally implemented. The anticipated changes are mostly storage level format changes visible only to node operators. We encourage the community to start using Test Net 2 in addition to the current test net, so that we can quickly identify and resolve any issues with the upcoming hard forks. All new development should be targeted to TEST NET 2 and all current code MUST BE tested against TEST NET 2 to ensure future compatibility. 
  2. Network id for Testnet 2 is testnet20210604.

When we do hard fork 1 on the main net, we will be starting from an empty state, with REV balances only. On Test Net 2, this 'loosing the state' will be repeated multiple times.

Tech-Governance meetings on Thursdays 10 AM Eastern, 7 Am Pacific 

Mercury requirements and acceptance criteria

Details on the acceptance criteria: Mercury acceptance criteria

Please see the documentation at https://github.com/rchain/rchain/blob/dev/docs/features.md. 

Developer website

https://developer.rchain.coop 

Date