A bug is a missing test case

I came across this quote a few years ago:

"Treat bugs as proof of a missing test case. When a bug is found, figure out a way to implement a test case." -Jeremy Dunck*

 

What's the first thing you have to do to fix a bug? Reproduce it! And what's the last thing you have to do before you push the fix? Test it!

We can feed two birds with one scone by adopting the mantra "a bug is a missing test case."

Next time you have to fix a bug, write a test to reproduce it. Ideally this would be an automated unit test, but this advice is true regardless of what level your tests are at or whether or not they are automated. If a repository or file has zero unit tests, a bug fix is a great opportunity for you to write its first unit test.

 

Write this test case before you change any code to fix the bug. Follow the standard "red-green-refactor" steps: Based on the steps to reproduce the bug, write a test that fails (goes red), and run it to confirm the defect. Change the code to make the test pass (go green). Then refactor the code. Before you commit the fix, run regression tests to make sure you didn't break existing functionality.

 

Your new test should serve as a regression test to make sure that the fixed bug will never reoccur.

 

In addition to writing the test and fixing the bug, ask yourself how you could have prevented the bug in the first place. Was it an edge case or an exceptional scenario and an oversight when writing the initial tests? Did you think it wasn't risky enough to test? Was it too hard to test initially? Is the code base too complex, tightly coupled to dependencies, and untestable? Is there a coding lesson to be learned? Is there any process you could change to catch it next time? Could it have been caught in a review of either the code or the tests? Human error is rarely a satisfactory root cause. Don't keep these lessons to yourself.

 

Is it possible that similar bugs are lurking elsewhere? Do testing gaps exist? Will you find out before your users do?

 

Writing tests after you find a bug is a wonderful idea, but it's better if you can write tests and learn to improve your code so you can prevent bugs before they happen (see my previous post about building quality in). Even 100 percent code coverage is not a panacea -- you still need careful thought to come up with good test cases and do comprehensive assertions.

 

* I'm not sure the exact source of this quote. If anyone knows, please share! The above quote was the earliest version I found, from the old Joel on Software forum from 2004.