Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...


% of total
leaf17%
leaf:data6%

leaf:continuation

9%
node81%
skip2%


Thoughts

Thought experiment 1

Let's assume that:

  1. the system is operating at 1500 COMM/s
  2. it takes 50 COMM events to transfer X from A to B
  3. the random state is the only thing we track in history

one random state takes up 100 bytes

we are able to process 30 rev txns/s (1500 / 50)

This results in 10MB of random state per hour, 1.7GB in a week.

Thought experiment 2

Let's assume that:

  1. there are 20000 active rev vaults being used
  2. each rev vault lives on a dedicated channel
  3. the keys are evenly distributed

each key weighs 32 bytes

each node can hold up to 256 keys

To be able to track 20k unique channels it takes a trie that:

  • has a filled 0 level node (256 * 32 bytes)
  • the 1 level is able to index 65 536 keys
  • we use 20k of those keys
  • the rest of the structure needs a leaf and a skip per key - 64 bytes each

The structure that tracks 20k active accounts weighs:

  1. level 0: 8 192 
  2. level 1: 640 000
  3. skip + leaf: 1 280 000

So to track a block that has 20k rev transactions we need a total of 1 928 192 bytes on disk.

Thought experiment 3 (1 + 2)

Assumptions 1 & 2 hold.

we can have 20k rev transactions in 667 seconds, 12 hours.

Let's assume that we only create a block when we have 20k rev txns accumulated.

The lower limit of the cost on disk is: 12*10MB + 2MB = 122MB per block.

The cost of the above will go up with the frequency of block creation (only whole nodes can be shared).

Takeaways/next steps

  1. Work through checkpoint creation (1 step instead of 13 steps) which would reduce the amount of stored tries (less data) 
    Jira Legacy
    serverSystem JIRA
    serverId50130123-f232-3df4-bccb-c16e7d83cd3e
    keyRCHAIN-3415
    1. estimated gain:
      1. store of top level pointerblock (4416 + 4512 + 128) instead of 44992 (80%)
      2. note: this calculation is very imprecise as the change will happen (is possible) in next rspace
    2. Address EmptyPointer stored in Node pointer block. 
      Jira Legacy
      serverSystem JIRA
      serverId50130123-f232-3df4-bccb-c16e7d83cd3e
      keyRCHAIN-3226
      1. encode pointer type in 2 bits, not in 8 bits
      2. explore gain from storing sparse vectors
  2. Continue analysis of rholang AST
    1. we can address the source size (which is constant but it constitutes a 10% or 15% of the data stored)
    2. there is no way of addressing the random state

...