Test-plan
Test Plan
What is the purpose of testing? 1. to ensure high quality code 2. to find and eliminate bugs
This test plan is designed to accomplish those goals.
Principle: Write Tests For things that Make sense
Testing works well for certain components and doesn't work so well for others. Most tutorials focus on pure functions with simple input and output, and ignore more complex state problems.
The ideal testing situation is for a function or program which must handle a wide variety of inputs, such as a parser, or compiler. In these cases the quality of the program can be greatly improved by regularly running against a large data set.
Guideline
- Create tests for all tools that follow a unix stdin stdout pattern. We expect these tools to be the foundation of our project.
- Create tests for pure functions.
- Do not unit test trivial state changes.
- Document the purpose of a test. If the test does not have a clear purpose it probably is just affirming the developer's original assumptions.
Principle: KISS
A lot of time can be spent fiddling with test frameworks. Tests also increase the amount of time to adapt to new business logic.
Guideline
- Avoid using a large testing library. Prefer simple functions.
- Avoid complex mock scaffoldings.
- Use assertions heavily directly in the code.
Principle: Prefer Correctness
Program testing can be used to show the presence of bugs, but never to show their absence - Dijsktra
Tests can only identify bugs for particular inputs, while a proof can very for the correctness for all inputs. When possible prefer to verify correctness.
A good example, was Alpheus and Justin working on the panel sizing system. They framed the problem mathematically and proved a valid solution. This mathematical notation was documented in the code, and was used to write several other helpful functions using the same ideas.
Guideline
- Describe problems in mathematical language when possible. (This usually simplifies the code and the problem.)
- Prove algorithms mathematically and document them in the code, using latex syntax.
- Identify and document invariants with assertions.
References
https://medium.com/welldone-software/an-overview-of-javascript-testing-in-2018-f68950900bc3
https://philipwalton.com/articles/how-to-unit-test-private-functions-in-javascript/