(deprecated) Measurement of RChain Network Performance
Goals
- The RChain network in aggregate must process 40,000 COMM events / second.
Definitions:
- Transaction: The deployment of a contract, or the invocation (running) of a contract.
- COMM Event: An event where a computation is performed because a new value is observed at a key. In terms of the tuplespace (storage layer), a COMM event takes place when a produce() and consume() 'match' and a join event takes place.
Background and strategic fit
If a blockchain system is ever to replace Point of Sale, the performance of the network must come close to Visa's transaction speeds.
Assumptions
- A reasonable approximation of a transaction is a transfer of token from one entity to another. For the test, the Rev system contract will be used.
- A test harness will be used to execute transactions repeatedly against multiple nodes, to simulate load on the network.
- Performance harness - The harness that will be used to run these tests.
- RNode Integration Testing Tool
- Transactions are not the same thing as COMM events. A transaction can comprise of many COMM events. The simplest transaction will contain a single COMM event.
Requirements
# | Title | User Story | Importance | Notes | Timeframe to Demo |
---|---|---|---|---|---|
1 | Raw system performance |
contract @"dupe"(@depth) = { if (depth <= 0) { Nil } else { @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) | @"dupe"!(depth-1) } } | @"dupe"!(5) | Must Have |
| Testnet |
2 | Measure Deploys |
| Must Have |
| Testnet |
3 | Contract Deployments | Deploy a set of contracts with names, so that they can be called in a subsequent deploy. Obtain the address for each contract deployed on the blockchain. Then run a series of transactions that reference the name of that contract. | Must Have |
| Testnet |
4 | Token transactions | Call the Rev contract.. Alice sends Bob 1 token. Requires that a wallet be implemented first. See https://gist.github.com/birchmd/f25578323b249ae03a45499a8165ab9d for scala code which can generate rholang for this test. | Must Have |
| Testnet |
5 | Cross-Shard transactions | Measure the performance of sending Rev across shards. Repeat across many shards. | Must Have |
| Pre-Main Net |
6 | Run a contract that stores a binary file | Call a contract that stores a binary file on the blockchain. Run this test for a variety of file sizes (1MB, 5MB, 10 MB) | Must Have |
| TestNet |
*5 & *6 is deferred for Test Net.
Test Results
Below is an attempt to track performance as a result of changes made.
Test | Date | GitHash | Result | Grafana |
---|---|---|---|---|
Dupe!(5) | 10/3/2018 | 107f213368c4c155e165cbc57d3b89a9eaae2f2e (Includes fine grained locking in RSpace | Base performance of ~ 1K COMM / sec, peak of ~ 2K / sec | |
loop.sh | 10/25/18 | RChain Node 0.7.1 (5fa4b3fc49e8019929fb8e634f9925e2d8490088) (includes removal of SyncVar + InMem Hot store) | Observation: Increased throughput on replay RSpace. | |