With and without unit tests (C++)

Once upon a time I was tasked with writing a parser.

The parser was meant to consume a file and make specific values from that file available to a C++ program.

Because this task was so complex, with a lot of cases to support, the only reason the project was still compiling after weeks of my work was the test suite I attached to it.

Every time after a non-trivial change to the program, I would re-run tens of specific test cases and check that the output still met my expectations.

I ran the test suite after even small changes. If one test unexpectedly failed, I knew exactly what caused the failure.

This was a very different experience than the one I had during my studies in Computer Science.

Back then, I would focus on writing as much as possible in one burst.

I would test just a single case at a time, and forget about other ones for a while.

This meant after servicing 5 test cases, once in a while I would try the first case to check if it works, and it wouldn't.

Trying to fix that, I would make changes that undermined my confidence that the remaining 3 cases are serviced as they should.

There I was, oscillating between 5 cases, when I already could've been implementing the 6th, 7th and 8th test case.

Those days are over, thankfully.

At present I pay attention to always have a suite of tests that lets me quickly check if each and every test case is serviced correctly.

In C++, the package that enables me to do that is doctest.

It is so fast that I can run most tests instantly.

What about the situation where you already have a large, untested C++ program? Do you need to rewrite it in a "testable" manner?

The answer comes from Robert C. Martin. He says large, successful rewrites are a myth. Sure, they sometimes succeed, but consider the following scenario.

Let's say you are determined to do a complete rewrite of your program.

You create a copy of its source, and call it "Version 2".

You extract a function from version 1 and add unit tests to it.

You extract another function, add unit tests there as well.

Then your customer emails you and says they discovered a bug in version 1. You fix that bug, commit the code to version 1, and attempt to merge changes to version 2.

If it goes fine, you're lucky and nothing else is needed – you can happily return to your main job of rewriting the program.

If some changes to version 1 happened in the same place where the "refactors" in version 2 were, you now have a bunch of code to merge by hand. It's not trivial, because in the process you need to remember both about the details of the refactor, and of the bugfix.

Because of that, it's likely going to be something only you will able to carry out, imagining you usually work in a team.

Let's say either way the merge is now done. Both versions work and you can return to preparing for version 2.

Some time has passed, and you've successfully refactored about 40% of the code.

You receive a note from the customer that one of the features cannot wait any longer and needs to be implemented quickly.

You implement that in version 1, because that's the only version the customer sees. We would call it the only one "in production".

You know that feature will have to end up also in version 2. You would like to merge it right away, but again a large part of new feature code depends on interfaces that have since changed in the course of the refactor.

That sounds like good 2 days of work to update the new code and test the solution, and you really wish you could spend that time doing something else.

Let's say despite this you have successfully refactored 80% of the functionality you think users need on a daily basis.

You feel it is time to let some users test the new program.

You set up another server, call it "Version 2 Staging", and ask some users to use it instead of the previous, stable version. What happens?

Users click through some of the new functionality. They find a bug in a feature critical to them, they let you know about it, and they promptly switch to version 1 to get any of their work done at all.

Meanwhile, bug reports for version 1 haven't stopped coming. Now you have two projects to maintain in parallel. Only a small share of users have so far been able to click through your staging version 2, so the bugs in that version are still mostly undiscovered.

Those bugs surely are there – if not because you omitted an important case, then possibly because something you thought you "fixed" was a behavior your users have long adapted to, treating it as the new normal ever since.

It's hard to imagine this story ending well.

For this reason, it's much better to abandon the dreams about "a great refactor" right away, and focus on small changes to the existing program, adding tests with those changes as you go.

Implementing feature #1148? Good! Make sure the new code has unit tests alongside it. Check those tests in together with the code so you and others could easily retest that behavior when needed.

If you can't keep the tests with the code for practical reasons, save them in a separate repository, but still keep them. Your future self will thank you for that.

What could be a reason to not keep tests alongside the code? I can only think of a situation where other developers think that would clutter the project because "Who needs those unit tests anyway?".

You won't convince everyone right away. You don't need to, to write tests and keep them.

Observe the number of bugs being found in your features over time, and let that speak for itself.

If your current team doesn't value the fact all important behaviours of your code can be tested in the blink of an eye, perhaps later you will meet one that will cherish that.

At the least, your family will cherish the fact that when you finish your work for the day, you can run a single command to check all important cases are still supported as they should, and log off knowing right away that they are.