Use persistent working directory for rchain/rnode Docker containers


When we run the rnode Docker image created by sbt, the process has /opt/docker as current working directory by default. Rnode files are in this directory (e.g. /opt/docker/bin/rnode).

When a node crashes, for instance on event like this

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007f2ee24dbea3, pid=1, tid=351 # # JRE version: OpenJDK Runtime Environment (11.0.1+13) (build 11.0.1+13-Debian-2) # Java VM: OpenJDK 64-Bit Server VM (11.0.1+13-Debian-2, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64) # Problematic frame: # C [] mdb_txn_commit+0xca3 # # Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport %p %s %c %d %P" (or dumping to /opt/docker/core.1) # # An error report file with more information is saved as: # /opt/docker/hs_err_pid1.log # # If you would like to submit a bug report, please visit: # # The crash happened outside the Java Virtual Machine in native code. # See problematic frame for where to report the bug. # [error occurred during error reporting (), id 0xb, SIGSEGV (0xb) at pc=0x00007f2f9f6fc3e4]

interesting files are put in the current working directory and these files are lost when container stops.

We should modify the created image to run rnode from a designated directory so that it'll be easy to bind-mount it into host filesystem. We cannot just bind-mount /opt/docker because the image has rnode files there.

Let's use either /var/lib/rnode or /root/.rnode because people are already used to bind mount one of these two to persist node logs. Let's also make /var/lib/rnode symlink to /root/.rnode in Docker image so that there's no confusion where does the rnode actually store logs.

We can't force people to do it but we definitely need to persist logs and everything else that gets dumped into current working directory in all the nodes we run, including bootstrap ones.

The problem could be that /opt/docker/bin/rnode shell script could be changing directory. This script and the corresponding Docker image are generated by sbt-assembly-plugin. It may not be that straightforward to bend the plugin but it should be investigated. In the worst case, the rnode Docker image could be created by extending Docker image generated by sbt-assembly-plugin.





Tomáš Virtus


Story Points


Epic Link