What is your code coverage?

Code coverage is a metric telling how many lines of code are executed when testing procedure is run. It tells what part of the code is covered with tests (thus code coverage).

Like every metric, it can be useful if we know what it means and how to interpret it.

How about road coverage?

Let's take a step back and imagine that you were given a task to check if all roads from point A to point B are driveable. Since those two points are just across the river from one another you just have to check all the bridges (and there are only 3). You take your bicycle for a ride through all 3 bridges in both directions. Everything goes smooth, all "tests are green". Your special device showing road coverage blinks green, showing 100%, all possbile roads between point A and B were checked, life is good. You reported the results to your boss - "Good, we will include the roads in our regular transportation schedule".

Next day, you got a call from your boss - "We just lost our most valuable vehicle! You said the roads are fine, you are fired!".

Wait, what went wrong? Well, in fact, one of the bridges is (or rather was, before it collapsed) a wooden bridge, easy to pass on a bicycle. Who may have thought, that a freaking, 20 tons heavy digger would drive there next day.

I hope you get the analogy, code coverage is like checking all the possible roads (all code execution branches), and that's it. It doesn't matter how it was tested (what type of input) as long as the code was executed. For example, the code may pass with Integer value and it will have 100% coverage but when String is passed it will throw an error and the whole application/program will fail. This means that for the exact same code, 100% code coverage may not be the same as 100% code coverage 😉 It won't tell you anything about your tests quality.

Pros of code coverage

• Tells which code needs to be still tested.
• Shows dead or unused code.
• Useful when refactoring (especially big pieces of code which already have 100% coverage).
• Helps to keep a good amount of tests and stimulates thinking and testing edge cases (which are covered in code but may not be tested yet).

Cons of code coverage

• Just another metric, it's a good addition but should not be the only source of truth when it comes to code quality.
• 100% coverage generated by unit tests is not the same as 100% coverage generated by integration or e2e test. It needs to be used and interpret wisely.
• May make one lazy and less cautious (100% coverage does not mean the code is 100% valid and will work for every case).

Expand your testing toolbox

When it comes to measuring code and tests quality, there are multiple metrics/tools which can be used in parallel with code coverage:

• Cyclomatic Complexity,
• Nesting Depth,
• Relational Cohesion,
• Lack of Cohesion of Methods (LCOM),
• Static analysis in general (for finding security issues, common code flaws, and similar stuff).

Metrics are nice, but first things first, so focusing on proper testing itself is really important. There are multiple types of tests and approaches and it's super wide topic. I would advise looking on Testing Pyramid (for example 1, 2, 3) and then dig deeper.

And I will leave you with that you can even test your tests to see if they test what they should be testing 😃 For example, mutation testing is one of the approaches.

May your tests be green, always! 🤞