2018-08-14 Testing Strategies Meeting notes

Date

Attendees

Goals

  • Get aligned on approaches, tools and goals with  respect to automated testing.

Discussion items

TimeItemWhoNotes
15 minPerformance testing     
  • Walks through what they are building for performance testing
 5 minIntegration testing 
  •  His tests are for correctness (integration tests)
  • Jeremy's tool Creates a docker network, bootstrap node, configures peers.  Main issue is that Pawel does his changes for comms, and he needs help from someone else to test his changes.
  • Ovidiu asserts that it was not easy for developers to add tests to the integration harness.  He believes that Python is a good approach for the problem.  It's a good tool for integration testing.  
  • We don't have small components that are easy for someone to build their tests.
    • He wants to refactor the p2p test tool.
      • We are doing our own test result aggregation, which is overkill.
      • We are missing a few features - which are available in pytest.
      • We can organize the tests by features and components
      • It offers 'discovery'
      • It has very good management of the log files.  Logs display only in the event the test fails.
      • We can implement a few primitives
        • Implement a network, fully connected, partially connected, image used, etc...
  • We can improve the performance of the tests by using Docker images.  
  • We are using sleeps, and we could reduce the sleeps.

Action items

  • DECISION: The performance suite will be checked in under the RChain Organization in GitHub, but in a separate Repository from the rchain repo.
  • DECISION: We will implement pytest & add new tests to it.
  • DECISION: We will NOT implement another client for tests that focus on correctness.  We will use the Java gRPC client.