...
% of total | |
---|---|
leaf | 17% |
leaf:data | 6% |
leaf:continuation | 9% |
node | 81% |
skip | 2% |
Thoughts
Thought experiment 1
Let's assume that:
- the system is operating at 1500 COMM/s
- it takes 50 COMM events to transfer X from A to B
- the random state is the only thing we track in history
one random state takes up 100 bytes
we are able to process 30 rev txns/s (1500 / 50)
This results in 10MB of random state per hour, 1.7GB in a week.
Thought experiment 2
Let's assume that:
- there are 20000 active rev vaults being used
- each rev vault lives on a dedicated channel
- the keys are evenly distributed
each key weighs 32 bytes
each node can hold up to 256 keys
To be able to track 20k unique channels it takes a trie that:
- has a filled 0 level node (256 * 32 bytes)
- the 1 level is able to index 65 536 keys
- we use 20k of those keys
- the rest of the structure needs a leaf and a skip per key - 64 bytes each
The structure that tracks 20k active accounts weighs:
- level 0: 8 192
- level 1: 640 000
- skip + leaf: 1 280 000
So to track a block that has 20k rev transactions we need a total of 1 928 192 bytes on disk.
Thought experiment 3 (1 + 2)
Assumptions 1 & 2 hold.
we can have 20k rev transactions in 667 seconds, 12 hours.
Let's assume that we only create a block when we have 20k rev txns accumulated.
The lower limit of the cost on disk is: 12*10MB + 2MB = 122MB per block.
The cost of the above will go up with the frequency of block creation (only whole nodes can be shared).
Takeaways/next steps
- Work through checkpoint creation (1 step instead of 13 steps) which would reduce the amount of stored tries (less data)
Jira Legacy server System JIRA serverId 50130123-f232-3df4-bccb-c16e7d83cd3e key RCHAIN-3415 - estimated gain:
- store of top level pointerblock (4416 + 4512 + 128) instead of 44992 (80%)
- note: this calculation is very imprecise as the change will happen (is possible) in next rspace
- Address EmptyPointer stored in Node pointer block.
Jira Legacy server System JIRA serverId 50130123-f232-3df4-bccb-c16e7d83cd3e key RCHAIN-3226 - encode pointer type in 2 bits, not in 8 bits
- explore gain from storing sparse vectors
- estimated gain:
- Continue analysis of rholang AST
- we can address the source size (which is constant but it constitutes a 10% or 15% of the data stored)
- there is no way of addressing the random state
...