I Hate It When My Tests Pass

I’ve been writing a bunch of unit tests recently, and I came across an odd behavior. I was writing the tests for existing code and whenever I would write a test, as soon as it passed, I wouldn’t trust it. Nothing ever works the first time around. It simply can’t be. So I found myself repeating quite an odd ritual:

1. Write a test.
2. Run it. It passed. There’s no way that really works…
3. What to do, what to do? I know! Let’s change the expected value on the asserts and see what happens!
4. The test fails! Yay!
5. Change it back. Now I know it works!

At the moment I realized I was doing something weird, I immediately saw all the unit tests I ever wrote flash before my eyes. I was always doing this ritual*. So I’m thinking - what can be done to give me more confidence in my tests? One solution I can think of is a somewhat exaggerated form of verbosity. Take the unittest module for example. Its assert method all take a msg argument, which is printed in case the assert aborts. Well, how about adding a success_msg argument? Imagine working on a test, setting the verbosity to a maximum and see the following:

Asserting that result_value equals 5... success!
Asserting that result_error is empty... success!
Asserting that the correct exception was thrown... success!

These messages can even be automated, to some degree. 

Does this happen to you as well? Do you have a solution to this problem?

*Actually, there is a subset of my tests in which I have very high confidence. Those are the tests I developed with TDD. This post isn’t the right scope for that, but this is one of the advantages of TDD.

Similar Posts

• How to pick a starter project that'll make someone quit
• Scrum Retrospective Meeting is Necessary and Sufficient
• Write Explicit Tests
• Debugging with Humility
• The Pass-Once Bug
• Let Your Process Go