HashMap in BlockDagFileStorage is a bottleneck.

Description

https://files.rchain-dev.tk/heapdumps/testnet.rchain-dev.tk/2019-10-06T20-04-12.testnet2-1.node0/ap/finalization.svg

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 def load[F[_]: Sync: Log: RaiseIOError, K: Codec, V: Codec]( logPath: Path, crcPath: Path, corruptedError: => StorageError ): F[PersistentIndex[F, K, V]] = { val keyValueCodec = ("key" | Codec[K]) ~ ("value" | Codec[V]) for { _ <- createNewFile[F](logPath) dataList <- readDataList(logPath, keyValueCodec, corruptedError) readCrc <- readCrc(crcPath) result <- RandomAccessIO.openResource(logPath, RandomAccessIO.ReadWrite).use { randomAccessIO => validate(randomAccessIO, readCrc, dataList, keyValueCodec, corruptedError) } (resultList, resultCrc) = result logOutputStream <- FileOutputStreamIO.open[F](logPath, append = true) } yield new PersistentIndex( logPath, crcPath, new AtomicMonadState[F, Map[K, V]](AtomicAny(resultList.toMap)), resultCrc, keyValueCodec, logOutputStream ) }

 

Status

Assignee

Marcin Rzeźnicki

Reporter

Dominik Zajkowski

Components

None

Story Points

None

Epic Link

None

Labels

None

Priority

Medium
Configure