(deprecated) Measurement of RChain Network Performance

Goals

  • The RChain network in aggregate must process 40,000 COMM events / second.

Definitions:

  • Transaction: The deployment of a contract, or the invocation (running) of a contract.  
  • COMM Event: An event where a computation is performed because a new value is observed at a key.  In terms of the tuplespace (storage layer), a COMM event takes place when a produce() and consume() 'match' and a join event takes place.

Background and strategic fit

If a blockchain system is ever to replace Point of Sale, the performance of the network must come close to Visa's transaction speeds.   

Assumptions

  • A reasonable approximation of a transaction is a transfer of token from one entity to another.  For the test, the Rev system contract will be used.
  • A test harness will be used to execute transactions repeatedly against multiple nodes, to simulate load on the network. 
  • Transactions are not the same thing as COMM events.  A transaction can comprise of many COMM events.  The simplest transaction will contain a single COMM event.

Requirements

#TitleUser StoryImportanceNotesTimeframe to Demo
1Raw system performance
  • Run the following Rholang:
contract @"dupe"(@depth) = {
  if (depth <= 0) { Nil } else { @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) }
} |
@"dupe"!(5)
Must Have
  • This contract generates a large number of COMM events.
Testnet
2Measure Deploys
  • Run the following Rholang:  x!(Nil) | for (_ ← x) Nil
  • Creates 1 COMM event / deploy
Must Have
  • Demonstrates the rate of deploys possible on the system.  Each deploy of this contract generates 1 COMM event. 
  • Exclude from the demo
Testnet
3Contract Deployments

Deploy a set of contracts with names, so that they can be called in a subsequent deploy.  Obtain the address for each contract deployed on the blockchain.  Then run a series of transactions that reference the name of that contract.

Must Have
  • Contract deployment is a specific type of transaction.  Running a contract from 'cold storage' will have a different transaction speed than running a contract from 'hot' storage.  
    • Deploy the mint - create wallets that contain Rev.  Test how fast you move money from 1 wallet to another. 
    • Propose a single transfer and observe it in Grafana
    • Have all the wallets in place in advance.
Testnet
4Token transactions

Call the Rev contract..  Alice sends Bob 1 token.   Requires that a wallet be implemented first.

See https://gist.github.com/birchmd/f25578323b249ae03a45499a8165ab9d for scala code which can generate rholang for this test.

Must Have
  • Demonstrates how Rev will be sent on the platform.  Obtain the performance for an in-shard Rev transfer.
    • Propose blocks of monetary transactions and see how many you can get through.
Testnet
5Cross-Shard transactionsMeasure the performance of sending Rev across shards.  Repeat across many shards.  Must Have
  • Set of transactions that are specifically cross-shard transactions.  For simplicity, we should send Rev across shards, as this is a likely use case.
Pre-Main Net
6Run a  contract that stores a binary fileCall a contract that stores a binary file on the blockchain.  Run this test for a variety of file sizes (1MB, 5MB, 10 MB)Must Have
  • Measure the performance of storing binary data on the blockchain.  
  • Measure the performance of retrieving binary data from the blockchain.
TestNet

*5 & *6 is deferred for Test Net. 

Test Results

Below is an attempt to track performance as a result of changes made.

TestDateGitHashResultGrafana
Dupe!(5)10/3/2018

107f213368c4c155e165cbc57d3b89a9eaae2f2e

(Includes fine grained locking in RSpace

Base performance of ~ 1K COMM / sec, peak of ~ 2K / sec

loop.sh10/25/18

RChain Node 0.7.1 (5fa4b3fc49e8019929fb8e634f9925e2d8490088)

(includes removal of SyncVar + InMem Hot store)

Observation: Increased throughput on replay RSpace.






Not Doing