Creating infrastructure

Make sure that you have gcloud installed, terraform installed and configured.

Git clone https://github.com/rchain/rshard

Initialise git-crypt inside the repo. This will be used to encrypt .pem files for SSL while transferring them to validator node.

git-crypt init

Export git-crypt key. Keep it safe!

git-crypt export-key ~/rshard-git-crypt-secret.key

Generate network files

./scripts/generate-network-files network-files root-shard.mainnet.rchain.coop mainnet 9

This script will create necessary files for all nodes without private and public keys filled in.

Add and commit changes

git add -A
git commit -m "Nodes added"

Set paths to wallets an bonds files inside ./start.sh

Add and commit changes

git add -A
git commit -m "Wallets and bonds files added"

Now deploy the network infrastructure using Google Cloud

cd terraform
terraform init
terraform apply

# For var.rshard-git-deploy-key input path to ssh deploy key that 
# has read access to https://github.com/rchain/rshard repository
# to check if key works GIT_SSH_COMMAND='ssh -i <path_to_key>'
# git clone https://github.com/rchain/rshard

# For var.rshard-secret-key input path to the git-crypt key exported previously
# This will be uploaded to /root/rshard-git-crypt-secret.key of the nodes servers
# to decrypt validator keys on the server

.........
Plan: 33 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Wait till network infrastructure is installed.

Apply complete! Resources: 36 added, 0 changed, 0 destroyed.

Starting the nodes

Prepare 9 key pairs for validators. Adjust the following config files.

network-files/node{0-8}/rnode.conf.d/90-node-private.conf - put here private keys (9)
network-files/node{0-8}/rnode.conf.d/80-node.conf - put here public keys (9)

Replace existing config files on servers containing dummy keys with the real ones.

for num in {0..8}; do scp -r ./network-files/node$num/rnode.conf.d/* root@node$num.root-shard.mainnet.rchain.coop:/var/lib/rnode-static/rnode.conf.d/; done

Start the nodes

for num in {0..8}; do ssh -o StrictHostKeyChecking=no root@node$num.root-shard.mainnet.rchain.coop "chmod +x /opt/rshard/start.sh;/opt/rshard/start.sh"; done

Pull configuration files used to run the nodes to your local machine. These files are required to restart RNode and should be kept secret as they contain private keys.

for num in {0..8}; do scp root@node$num.root-shard.mainnet.rchain.coop:/var/lib/rnode/rnode.conf ./node$num.rnode.conf; done

Delete config files containing private keys from the remote servers.

for num in {0..8}; do ssh root@node$num.root-shard.mainnet.rchain.coop rm /var/lib/rnode/rnode.conf; done

Pull .pem keys and certificates from servers. Leave them on servers.

for num in {0..8}; do scp root@node$num.root-shard.mainnet.rchain.coop:/var/lib/rnode-static/node.key.pem ./node$num.node.key.pem; done
for num in {0..8}; do scp root@node$num.root-shard.mainnet.rchain.coop:/var/lib/rnode-static/node.certificate.pem ./node$num.node.certificate.pem; done 

Ceremony master node0.root-shard.mainnet.rchain.coop

Peers node{1-8}.root-shard.mainnet.rchain.coop

To restart the node operator should place config file that corresponds this particular node back and use the following command. Substitute rchain/rnode:v0.9.22 with required release version. After node is started - remove config file again.

docker stop rnode && docker rm rnode

# Copy config file
# e.g. scp ~/node.conf root@node0.root-shard.mainnet.rchain.coop:/var/lib/rnode/rnode.conf

docker run -d --name=rnode --network=host -v /var/lib/rnode:/var/lib/rnode \
-v /var/lib/rnode-diag/current:/var/lib/rnode-diag/current \
-v /var/lib/rnode-static:/var/lib/rnode-static:ro rchain/rnode:v0.9.22 \
-XX:+HeapDumpOnOutOfMemoryError \
-XX:HeapDumpPath=/var/lib/rnode-diag/current/heapdump_OOM.hprof \
-XX:+ExitOnOutOfMemoryError -XX:ErrorFile=/var/lib/rnode-diag/current/hs_err.log \
-XX:MaxJavaStackTraceDepth=100000 -Dlogback.configurationFile=/var/lib/rnode-static/logback.xml \
-c /var/lib/rnode/rnode.conf '-J-Xms26g' '-J-Xmx26g' \
run --network mainnet --bootstrap 'rnode://2bf7856958a3b36030381d5f0465147a2a0a1547@node0.root-shard.mainnet.rchain.coop?protocol=40400&discovery=40404' 

# Delete config file 
# ssh root@node0.root-shard.mainnet.rchain.coop -c "rm /var/lib/rnode/rnode.conf"