Demo Standups Notes
Tickets we will track:
Network Launch - https://rchain.atlassian.net/browse/RHOL-414
Performance: https://rchain.atlassian.net/browse/CORE-659
Demonstrate Performance of the Network: https://rchain.atlassian.net/browse/CORE-659
Immersion Demo - No ticket.
Tuesday 08/21/18
Network Launch - https://rchain.atlassian.net/browse/RHOL-414
Michael & Mateusz have identified the source of the error, will capture details in https://rchain.atlassian.net/browse/RHOL-485
Performance: https://rchain.atlassian.net/browse/CORE-659
Mike is meeting with folks on https://rchain.atlassian.net/browse/CORE-1033 this afternoon. Medha will take notes and action items
Medha to Invite Tomas to standups for status on Java tuning & production engineering for a status
Demonstrate Performance of the Network: https://rchain.atlassian.net/browse/CORE-659
Dominik: Working to set up Grafana on a public IP and connect to external IPS - https://rchain.atlassian.net/browse/CORE-1037
Immersion Demo
Kayvan needs to know how to interact with the contracts on the blockchain. Mike to walk him through it.
Alex will cut a PR for the Client API today or tomorrow morning.
Dry run demos tomorrow as scheduled.
Wednesday 08/22/18
Network Launch - https://rchain.atlassian.net/browse/RHOL-414
Found the issue, ran a test https://rchain.atlassian.net/browse/RHOL-485
PR cut for the RSpace issue (Clear issue)
@Medha Parlikar (Unlicensed): @Kelly Foster: For a Friday demo of the launch, Michael would like to have a script for the presentation aspects of the demo.
Performance: https://rchain.atlassian.net/browse/CORE-659
Outcome of meeting - we will use the InMemoryStore as a waypoint towards a permanent fix. @Dominik Zajkowski & @Łukasz Gołębiewski (Unlicensed) have details. https://rchain.atlassian.net/browse/CORE-1033
Team discussed https://rchain.atlassian.net/browse/RHOL-616 and created issue https://rchain.atlassian.net/browse/CORE-1055
@Medha Parlikar (Unlicensed) Create a script of the demo.
Demonstrate Performance of the Network: https://rchain.atlassian.net/browse/CORE-659
Dominik: Working to set up Grafana on a public IP and connect to external IPS - https://rchain.atlassian.net/browse/CORE-1037
Immersion Demo
Kayvan needs to know how to interact with the contracts on the blockchain. Mike to walk him through it.
In his integration test, he deploys the immersion contract. He turns around and performs a propose, and the propose fails. However if he waits, it succeeds. He will reach out to Medha to reproduce.
Alex is testing the client api's locally.
His plan is to work on the user data, understand how the gRPC API works.
Couple of days for the demo. Friday morning for an internal demo.
Open Action Items:
@Michael Birch (Unlicensed): gist from(deprecated) Measurement of RChain Network Performance needs to be revised.
Once updated, need the output integrated into the performance harness.
Stable performance - Dupe(5) contract - updates to the scheduler (CORE-1033) will address.
What is the hardware configuration we need to recommend? Do we need people focusing on this?
We know we need at least 4 GB of memory
How many CPU's?
Approach - What does it take for a node to hit 1K COMM / second? We send a dupe(3) out 1X /second. Have versions of dupe(3) that write on different channels. - The above is in progress by @Medha Parlikar (Unlicensed)
Create a version of dupe that makes a new name. https://rchain.atlassian.net/browse/RHOL-645
How are validators deciding when to propose? Gatling is also making a propose.
We will need to determine what the interval of propose will be & block size (as a result). We can tune the number of proposes per deploy.
Thursday 08/23/18
Network Launch
@Medha Parlikar (Unlicensed): @Kelly Foster: For a Friday demo of the launch, Michael would like to have a script for the presentation aspects of the demo. Kelly is booking time on the calendar for today.
Completed a community test against dev of the Genesis launch - received the requisite number of signatures, despite having more peers. Several folks hit a comms issue, which has been filed.
Discovered a stack overflow error when including the full 8K records for the genesis block - ticket filed.
@Medha Parlikar (Unlicensed) - working to move RHOC to a paper wallet to test the Rev issuance.
Performance
Filed an issue: https://rchain.atlassian.net/browse/RHOL-650 @Dominik Zajkowski is investigating.
When a block is added to the CASPER state, all blocks have to go through the replay the state. There is an optimization possible where a node that proposed the block will not replay the block.
@Medha Parlikar (Unlicensed): Need to script the demo.
@Dominik Zajkowski: Update on CORE-1033 - have a version in dev in a branch. The default build is using LMDB. They are testing. Switching to mixed mode is a build switch.
If we are fine with the tradeoffs, we can commit a change that will make mixed mode the default mode. No changes needed to the build process.
Immersion Demo:
@Kayvan Kazeminejad: In his integration test, he deploys the immersion contract. He turns around and performs a propose, and the propose fails. However if he waits, it succeeds. He will reach out to Medha to reproduce.
Demo tomorrow that shows how he will store and fetch some data from the blockchain.
Need a script for the demo.
Client API's - Alex is testing locally. Medha to get an update.
Friday 08/24/18
Network Launch
Ticket to modify the wallet to accept the public key https://rchain.atlassian.net/browse/RHOL-661 - assigned to @Michael Stay (Unlicensed) - needs to be done by Monday.
@Medha Parlikar (Unlicensed): Recover public key for her ETH wallet.
Team discussed what needs to be done in order to support the Rev issuance, option to 'just do the same thing that Ethereum does, and offer the Rev issued via a ledger, where the withdrawal returns a purse.
Stack overflow error when including the full 8K records for the genesis block - ticket filed. https://rchain.atlassian.net/browse/RHOL-652
Schedule a test for Monday with @Medha Parlikar (Unlicensed) to test the issuance.
Performance
https://rchain.atlassian.net/browse/CORE-1062 - there was an issue with the Grafana dashboard, Replay looks better, Medha to verify.
Observed that Replay is still slower. Filed issue: https://rchain.atlassian.net/browse/CORE-1064
Found a new issue - Kent is working on it as a high priority: https://rchain.atlassian.net/browse/RHOL-653
Plan to test how a node handles receiving an invalid block, it should ignore it and continuing processing.
@Łukasz Gołębiewski (Unlicensed): Update on CORE-1033 - have a version in dev in a branch. The default build is using LMDB. They are testing. Switching to mixed mode is a build switch.
Discovered that the mixed mode didn't deliver the boost on performance that was hoped. Turns out that threads are blocked on a sync var.
Plan B - Fine grained transactions in the in-memory store
Refactor the in memory store and use cats mvar with green threads.
@Łukasz Gołębiewski (Unlicensed): Showed the grafana dashboard for a bare metal test.
Immersion Demo:
@Kayvan Kazeminejad: My demo is not ready yet. to use the GRPC Rholang implicits form the Rchain.model, i need to reference that project wich brought in a new cats.effect library that is incompatable with http4s. the issue is resolved now and I also opened a gira issue with http4s. today i've been working with Kent to complete the GRPC stuff contract updats/retrieval completed.
Tuesday 08/28/18
Network Launch
Ticket to modify the wallet to accept the public key https://rchain.atlassian.net/browse/RHOL-661 - assigned to @Michael Stay (Unlicensed) - needs to be done by Monday.
@Medha Parlikar (Unlicensed): Recover public key for her ETH wallet.
Worked with Michael to attempt to recover key - will test his solution today.
We can recover the public key from the private key. We may need to add new Rholang features in order to make this work seamlessly.
Need to consider where the wallets end up → Depends on the name registry. → we are fine for the issuance.
Need to consider how the wallets are used in the end. Wallets will be unlocked by providing the public key. → we don't need this for the demo.
The public key method in our crypto library has an extra byte in it.
Stack overflow error when including the full 8K records for the genesis block - ticket filed. https://rchain.atlassian.net/browse/RHOL-652 - Fixed
Schedule a test for Tuesday with @Medha Parlikar (Unlicensed) to test the issuance. -
Performance
https://rchain.atlassian.net/browse/CORE-1062 - there was an issue with the Grafana dashboard, Replay looks better, Medha to verify.
Observed that Replay is still slower. Filed issue: https://rchain.atlassian.net/browse/CORE-1064
Found a new issue - Kent is working on it as a high priority: https://rchain.atlassian.net/browse/RHOL-653
Plan to test how a node handles receiving an invalid block, it should ignore it and continuing processing.
Issues affecting performance found in Aug. 28 node testing
@Medha Parlikar (Unlicensed): testing Kent's latest PR - Observing Bonds cache errors & Neglected invalid block - ticket filed
Discovered an issue with wide contract in interpreter.
@Michael Stay (Unlicensed): testing contracts in the repl & checking performance
wide in the repl is about 1.5X slower in the node - 2K COMM events / 6 second.
same contract took 5 seconds in CLI
@Dominik Zajkowski: Why does wide(5000) not fit into 16 GB of memory? See if we can engage with @Mateusz Gorski (Unlicensed) to run the contract on the interpeter + node to see what is going on.
See if we can engage
Immersion Demo
Kayvan is running a bit behind - he was having a hard time getting data out of the gRPC call. Storing song data is working. Focus is on retrieving data from the blockchain.
Dan Connoley has completed the Rholang contract. He has removed then number of plays from the Rholang contract.
There is an immersion contract, and there is a part where upon providing a userid, he gets a play count. Kayvan needs to point to a contract that Dan Connolly has authored.
@Michael Stay (Unlicensed) to look at the code to observe what is going on and get Kayvan unblocked.
Able to store song data. Focusing on the retrieval
The contracts are in /src/main/resources/rho
Performance - Demo
Mike to provide an updated wide loop contract.
Will run dupe against stress-docker
Have a network ready to do this
Will run wide against stress-docker
Have a network ready to do this, with the contract set up.
Will run token transfer contract against test net
Will speak to our roadmap for performance
No silver bullet for performance. It's a lot of little fixes.
Created a performance harness, will become part of our test criteria.
Re-engineer appends in RSpace - Issue is reading something out of the DB, checking for a match, and then putting it back.
to append, requires that a blob needs to be deserialized, checked for a match and then appended to (no match for a produce)
Customize write locks for channels that are disjoint.
Examine the performance of the matcher.
Examine the performance of block storage.
Examine the performance of cryptographic functions.
Optimize how we store blocks in Casper, presently there is a good amount of duplicated data
Optimize replay (proposing node also replays the block, which is not necessary)
Optimize how we pass blocks around then network
Optimize read /write locks and threading in transaction handling code.
Implement peek in Rholang + RSpace
Will speak to how long we have been performance testing
When we are feature complete
What we have built.
Question about the token transfer contract code - per @Michael Stay (Unlicensed) the code appears correct, not sure what the 0, 11 represent.
Run the token transfer contract and see how the system performs.
Run with Kent's PR to observe the Neglected invalid block + bonds cache issue