100% Test Coverage

Some software developers like testing, others try to avoid it. I use it since the year 2000 in different ways. Test Driven Development seems to work best for me. It gives me a clear goal and I know when I finished a peace of softeware. But I often encounter discussions at work where developers use testing as a currency and try to bargain with the product owner: "If you want to have the feature by the end of next week I can only have 20% test coverage."

I see high test coverage as an integral part of good software and not something optional. So I decided to take it to the next level. Is it possible to have 100% test coverage all the time.

Definition

We need some definitions first. What do I mean when I use the words "test" and "coverage" and "all the time" and "100%"?

Software can be tested with unit tests, integration tests and some other forms of tests. It turned out that the best way for my project is a mixture of unit tests and system test. Then I tried to cover every requirement and every line of code with at least one test. This is my definition of 100%. And I am not allowed to commit my changes until I reached my desired test coverage.

Select a project

• Not too small - it should need to take a couple of days or weeks to implement to see if my goal can be kept over time.
• Not too large - it should not be discouragingly complex for one programmer.
• A real world problem - it should not be too abstract or overly theoretical to prove that the goal can be achieved in my everyday work.
• Well-defined - I do not want to spend much time on requirements engineering.

Tools

I used C# and Visual Studio for Mac for the implementation. The packages xUnit and FluentAssertions where used for the tests. The test coverage was measured using coverlet.collector. This package generates just one number for the test coverage of my project. This is exactly what I needed. And I used the pacages SimpleTCP and Stateless to make my implementation easier.

Process

I know the project that I chose very well. So I was able to skip the investigation to make myself familliar with the topic. The first step was to break the whole work down into smaller pieces. Every piece should take about 2 to 6 hours to implement. This size is usable for me because I do it as a side project. I can spend 2 hours in the evening during the week and probably up to 6 hours during a weekend. Then I prioritized the small tasks and created a new repository on GitHub. Then I implemented the software according to my priority list. After one task was finished I reviewed my plan and made adjustments.

Results

Summary: It is possible to achieve and maintain 100% test coverage with the setup that I chose. Here is a list of things that I learned during the implementation.

• Create a software design before you start to type and identify the parts that you need to mock.
• Develop the application by implementing one feature after the other, not one layer after the other. This will make it less likely to rewrite tests and write mocks that you will not need in the end.
• Write tests for your error handling. Programmers often live in a universe where the software always works. But error handling is a part of the software that must be designed and tested as well.
• Write the system test first. These tests are the representation of the functional requirements in your source code. Then add your implementation and then add the code for the mocks. And do not add code that is not required for the test. It's tempting because you might think that you need it for the next test anyway. But we want to have your 100% coverage and as little code as possible.
• Do you have a part of your software system that is only used by other parts but does not depend on anything? Then write unit tests instead of system tests.
• My usual reaction to line of code without a test was: "Ok, I need to write another test.". But this changed to "Am I missing a requirement? Do I really need this code? Did I write it out of habit?"

Final Thoughts

The most important requirement to work with 100% test coverage is the commitment of all people who are involved. Especially the software developers need to see the tests as a value, not a burden.