James Golick (a Rails developer living in Quebec!) writes about unit testing and probably the most common argument used by people who don’t do them: “we just don’t have the time.” He goes on to analyze that claim, and it’s pretty clear that writing unit tests is more efficient than manual testing. For pretty much every Django site and application I write, I try to test as much as I can. With tests, I can say “this code works as I expect it to.”
We also maintain older projects written in PHP at work, and those have exactly zero unit tests (I should mention that I did not write the original code for those applications.) Making modifications on these projects always takes longer than expected and sometimes puts us in the embarrassing position of deploying buggy code to the client. Because the code was not written with automatic testing in mind, the code is not properly modularized and compartmentalized; fixing a bug in one place or adding a feature can possibly break another part of the program and we have no way to know it. We test the feature we fixed/added and we do a quick overview of the system afterwards, but nothing exhaustive. We deploy the code, cross our fingers and of course two days later we get the dreaded “something that used to work is broken now” email from the client.
Our Django applications are much more solid, I test all the Python code and the templates. Before I deploy to the client, I run the tests to make sure everything is doing what it should be doing. And once deployed, I run the test on the client’s server to make sure that a change of environment does not break functionality.
I feel very uncomfortable writing code without tests, I don’t have the assurance that I didn’t screw up and that makes me very uneasy.
October 4, 2007 at 10:26 am |
This sounds terribly familiar to me. About 20% of my time right now is spent maintaining an application I didn’t write that’s byzantine in its (mostly unnecessary) complexity. It’s got two separate code bases (one written in PHP and one written in Perl) that access the same database – leading to the necessity of making fixes to the same bug in multiple places in two different languages. No unit tests goes without saying, and the code is mostly too entwined to really think about writing any sort of effective unit tests.
So I just started on regression tests. A large portion of the app is reporting and I’ve got a little python test suite that downloads various generated reports for various data ranges and compares them to previously downloaded and stored reports. This at least lets me know if my changes broke any calculations being performed by the app. It’s also key to my ongoing improvement strategy – I’m taking one perl page at a time and rewriting it in PHP, than using mod_rewrite to map the old url to the new one. The initial goal is just to have the new page exactly the same as the old page – and of course visually confirming that for multiple pages of numbers is tricky. The regression testing is giving me the confidence to make sweeping changes without worrying about testing for breakage…
October 4, 2007 at 3:51 pm |
Unit testing is a must when you get to that state. But how do you avoid it in the first place?
I say that code reviews are a must. If you don’t let bad code into your main branch in the first place, then you are not going to have nearly as many problems down the road.
October 5, 2007 at 4:34 am |
Yes, testing is good and we all should make time for it.
October 7, 2007 at 1:41 am |
Fred Brooks in the Mythical Man Month writes about how important a proper test scaffolding is to an engineering project. This was written in the 1970’s based on development work he managed in the 60’s for IBM. It’s sad that we’re just now getting back to where a stable of component tests is even contemplated much less used for every project.