Community Update 143

Owned by Rao Bhamidipati
Oct 28, 2020

General

  • Release
    • All the main net observer nodes are running release 0.9.25.6, which is a bug fix release https://github.com/rchain/rchain/releases/tag/v0.9.25.6 Main net validators are currently running 0.9.25.5.
    • After much testing, Tomislav is preparing the final Rnode 0.9.26 release for the main net, which includes the Last Finalized State (LFS) work, which is a major milestone.  Additionally, he has been working on updating scalapb, grpc and netty. This was a bit involved because of the custom work done around the past deficiencies of scalapb. Tomislav has been simplifying how scalapb is used. We expect to be able to release all of this some time next week.  Tomislav also created a fix for the data corruption error discussed last week.
    • Test net is running release 0.9.25.6 on all the nodes (validator as well as observers).  
    • With 0.9.26-lfs-Pre-release, node catch up is now much improved. This release also includes many bug fixes including for the slashing /tuple space mismatch bug reported earlier.  gRPC behaviour is improved with per validator grpc channel.   
    • Main net has 20 nodes running plus observer nodes and exchange nodes.
    • Main difference between testing on Observer nodes vs. Validator nodes: When we test on the observer node, a large part of the code base is executed. The Block creation code runs only on the validator nodes. As a part of that, the portion of the Runtime Manager that plays the deploy is run on the validator node but not on the observer node.
  • Sprint 69 in progress
    • Main Focus is work on Block merge in a couple of streams explained below, Releasing and enabling 0.9.26 on the main net, Resolve recently identified bugs in hashing and concurrency (we suspect one of these is responsible for slashing errors), Continue to update project dependencies to the latest feasible library versions, hardening the main net, improve performance.  Current PR list is at https://github.com/rchain/rchain/pulls
    • Current Work In Progress
      • Good progress - Significant work completed in Block Merge and conflict resolution for block merge. latest changes are being merged on to the Block merge branch.  Will is working on tests, runtime manager refactoring and combining his code for conflict resolution with nutzipper's block merge code changes.  nutzipper is working on additional refactoring needed for block merge.
      • Delayed by a week due to fixing scalapb updates and the data corruption bug - Tomislav is preparing the final version of Release 0.9.26-rc.  Tomislav has made further improvements to block store. He will demonstrate the improvements next week, along with the conclusion of the Last Finalized state work on the main net.
      • Fixed for now. Tomislav changed how the direct buffers are used in the code. - We have noticed a bug where the data occasionally gets corrupted during write to the database. This seems to be correlated to network hangs. There were several recent changes and we are trying to investigate which might be causing these.  Potential sources of these are recent lmdb version upgrade, recent IBM network changes, possibly our changes to the codebase, other dependency updates. Currently, Tomislav is backing out the lmdb version changes and see if this resolves it. Also, today, after conclusion of the IBM cloud infrastructure updates, the system behavior has been better. We should be able to resolve this soon. Netty updates may help the issue (see below).
      • Will continues to work the logic to identify and resolve conflicts.  We will demo once the block merge branch is ready.
      • Dependencies: Significant updates to the dependencies are completed, more underway.  Scalapb and protobufs is the one of the main remaining dependency updates - but this will take quite a bit of work. Updating netty is tied up with Scalapb update. Update to Scala 2.13 may be the last dependency update, which may also take quite a bit of work.
      • Ongoing - Fix additional identified bugs
      • API and usability improvements - in conjunction with the community
        • In Process - Will has started on OpenAPI/Swagger documentation of the API https://github.com/rchain/rchain/pull/3128  Familiarity with Swagger and the available swagger tools should make it easy for developers to use our API.
        • In Process - Identify and implement further 'quick hit' improvements to developer experience, in preparation for the hackathon. 
        • Future - Also in the thinking stage is a graph representation of payments, with perhaps date+time and amount on the arcs.
      • Release 0.9.27 and beyond
        • Near term road map is last finalized state (next week), block merge, updating PoS contract, sharding, name spaces
        • Creating a design spec and the code for how to update contracts on the main net.  This will enable addressing some of the pressing issues such as faster epoch times, and other issues listed below
        • In process: Finish dependency updates and plugin updates
        • In process: Improve error logging, debugging and monitoring infrastructure. This will help improve developer productivity, both on the core team as well as dApp developers
        • Ongoing - Refactor Run Time Manager. This includes many changes including https://github.com/rchain/rchain/pull/2946.  We will be testing this with non-conflicting deploys. We're getting nice clean DAG after the refactoring the runtime to remove most of the locks and allow multiple runtimes to run in parallel. We may need more refactoring in future, but we want to do this incrementally, as needed. 
        • Protection from 'too much free work' attack, policy to reject deploys when user doesn't provide sufficient phlo or does not have sufficient REV in the wallet to cover the phlo etc - https://github.com/rchain/rchain/issues/2997  and https://github.com/rchain/rchain/issues/2976 
        • Implement On-chain configuration and Shard configuration  https://github.com/rchain/rchain/pull/2986
      • Release 0.9.28 - Block merge
      • Explanation for last finalized state (released in 0.9.26): The scope of this work enables (a) faster catchup by new nodes - you can start from the last finalized state - this is a differentiator for RChain (b) offloading older data, differentiated storage and retrieval strategies for the same (c) allows for a leaner / less bloated node. This change touches most parts of the code base. Having to pick between refactoring and work-arounds in various parts. Tomislav implemented a mostly modular and future-beneficial approach.
      • Hackathon planning and execution is in full swing. Close to 150 registrations so far. Need more. Please promote the much improved hackathon page at https://rchain.coop/hackathon.html to all the groups you are part of. Hackathon dates are Nov 7-8, with qualifying test on Oct 31,  to accommodate student exam schedules and to give us more time to market and prepare for it. 
        • In process - We need to generate a bank of test questions, Pull Request tasks on Github, and Problem Statements, brief sessions on tips and tricks during the hackathon, and office hours during the hackathon.
        • In process - We need to improve documentation, learning links and examples to help developers new to Rholang get up to speed. The community is requested to contribute to any or all of these.
        • Ongoing - First meeting with the developer community was held last week. Dan has substantially improved the Rholang learning resources. Current plan is to make https://rholang.github.io/ the hub of all learning materials. dckc has been making substantial improvements to the learning materials as well as the API tools.
      • CI/CD:
        • Ongoing - We are slowly moving from Jira to Github for the development team, started publishing release notes in Github with release 0.9.25.  New issues are entered only in github. For a while, we will maintain in both Jira and Github. 
      • TechGovernance:
        • Thursday 10 AM Eastern. Meets alternate weeks.  Last meeting focused on how to implement and manage shard configuration. Meeting notes are at https://docs.google.com/document/d/1fvxMC6Bt5XwbVaLzYPy6ZPB8KzJvASO2sKRC6ZCPwpI  Discussions and work on a model of economics to ensure network security is proceeding. Most recent discussion was about ensuring some amount of stake beyond a 4 month window, spreading out stake over a longer time and how to incentivize that, while ensuring that everyone pays for network security.
        • ToDo - There were questions from the community regarding enabling privacy and zero knowledge proofs on the chain - this is in active discussion. Greg will create a position statement on this and likely discuss the current state of the art, use cases and potential approaches in an RCast session.  Same with sharding and cross-chain operation. 
      • In progress - Excel Model for 'economics of sustainable network security and decentralization' to enable community understanding of the inter-relationships and community conversation around these issues.  Will be releasing the initial version soon. May have multiple releases.
      • dApp Development:
        • Substantial progress on RVote including user testing of UI and improvements resulting from that.
        • Looking for developers to help make progress on the RChat effort using zulip.  SRT and possibly Will to add 'Rchain to zulip' functionality. Invite any community members with the expertise to help in this.
      • Current Backlog (partial)
        • Improve merging in system deploys
        • Improve Casper by enabling more tests and resolving identified code issues
        • Improve BlockMerge 
        • Improve multi-parent Casper enablement
        • Resolve slashing bugs, including additional refactoring of RunTimeManager
        • Implement sharding capabilities
        • Improve logging to be able to learn what API calls are being used, so they can be related to resource use and performance etc
        • Rholang 1.1 to improve syntax and user experience / learning curve
      • Tuesday TeachOuts by Tomislav (Tuesday 10 AM Eastern, 7 Am pacific) in Jimscarver's zoom room https://zoom.us/j/6853551826 

Tech-Governance meetings on Thursdays 10 AM Eastern, 7 Am Pacific 

Mercury requirements and acceptance criteria

Details on the acceptance criteria: Mercury acceptance criteria

Please see the documentation at https://github.com/rchain/rchain/blob/dev/docs/features.md. 

Testnet status

Please see RChain public testnet information to learn more about public testnet as well as a FAQ.

Tech Governance + Community testing

Thursdays at 14:00 UTC. Please see RChain community RNode testing for more information.

Blockers to Mainnet

NA

Risks to code completion for Mercury


Developer website

https://developer.rchain.coop 

Date