Objections to Unit Testing

13:05 Writing unit tests slows development and costs too much.

“This is the most frequent one that I hear.”

The number of conditions to test increases with the complexity of the code you write. As you add code you have to add tests to cover changes to the codebase. The original tests won’t cover the new code. If the new code interacts with existing code it can introduce errors to previously tested code.

Testing changes how code is written. Code not written with testing in mind is difficult to test. Most likely the code will need to be refactored to create testable units. If that’s not possible your tests will likely be overly complex to account for the code. Dependency injection is a prime example of this.

“You didn’t actually find less bugs you just perceived you found less bugs.”

Testing reduces the overall development time by finding faults early. Maintenance is the most expensive part of software life cycle. Unit tests are a major part of making code maintainable. Without them the maintenance team will not know if what they added broke existing functionality. When adding changes to existing code if the original code was covered those test will identify breaking changes. In the short term it does increase the amount of time in development. However it does reduce QA and UAT time (if you have that) because well written unit tests find most bugs. Development is more than just writing the code. It also reduces the amount of errors found by users.

19:25 Writing unit tests is too difficult.

This usually means that the codebase needs to be refactored. Spaghetti or tightly coupled code can be near impossible to test. You’ll need to split the code into smaller testable chunks. Remember that each method should only do one thing.

“You’re going to have to pull that crap out and test it.”

Some areas of code are difficult to test. Code involved in IO or relying on side effects can be hard to predict. You can mock up a lot of things but at some point you have to test connections to things like the database. You want to be able to test the connection but then not break if the data changes. Making your code testable adds complexity. Writing unit tests requires thinking about what can go wrong rather than how to make something work.

The more unit tests and testable code you write the easier it becomes. It will become second nature to write your code in testable units. The more tests you write the more you think about it.

25:05 Tests halt new development.

The tests may not pass after changes in business logic. Changes in business requirements may affect the underlying code. When the code is modified or outputs changed the tests may not pass because they are testing the wrong logic.

“This is treating testing like a second class citizen.”

Improperly written unit tests cause the code to be too rigid. Internal services and APIs your code is calling may change based on issues or new business requirements. If external code changes break previously passing tests the code is too rigid. You may be testing the service rather than your code.

Unit tests are living documents and need to change with your code. Unit tests can also be refactored and changed based on the needs of the business. If built properly you should only need to change the tests covering the units of code related to that business need.

29:05 We’re too busy write unit tests now, but will write them later.

This typically comes from management. It indicates that deadlines are unrealistic. This usually requires sacrificing code quality for quantity over time. Managers with this attitude do more harm than good.

Later will not ever come in these situations. Developers will never have time to write unit tests. There is not likely to be a break between projects. The next one will be waiting on the team as soon as they are finished.

“It’s a snowball pattern that leads to burn out and company failure.”

This is hard to fix as you have to change the structure of the company. It will require convincing management of the long term costs of not writing unit tests. If the company has been around for a while you can point out that historically you haven’t had time to write tests “later”. This is a great place for new junior developers to shine. As a last resort you may want to start looking for another job.

34:05 This class is so trivial that doesn’t need to be tested.

These are small classes with little involvement in the application. They may only have a few methods that are rarely used. The class itself may not be used often enough in the code. It’s functionality could be trivial to the business needs.

No class it too trivial to test. While it may not be necessary for the business needs other classes may rely on it. As changes are made in the code that class may become necessary. The class is likely to grow over time as functionality is added.

Only methods without business logic can be skipped in testing. These would be things like getter and setter methods. They cannot contain any logic. If logic is added to these methods tests need to be created for them.

37:05 Only integration tests “really” test the code.

This assumes that the only way to test code is with “real world” cases. Testing end-to-end functionality is considered the only “true” test. The idea is that in order to completely test a feature black box testing of various situations needs to occur. Each integration test would use all the functionality of the path in the app. This tests both the method logic and the functionality of the infrastructure.

Unit testing creates many more test cases than a single integration test. Integration tests have to be created for every time a component is used. However these unit tests only need to be created once. They test components regardless of where and how often they are used in the app.

“It’s like testing a rocket by shotting it off.”

You’ll need to create a single test for each possible place that an app could fail. Infrastructure tests verify connections to services like email servers and databases. They ensures that the cause of system failures can be pinpointed. This happens without going through code that is actually working. Unit tests are used to verify that your code connecting to the services is working. They also ensure that classes and methods containing business logic are functioning.

Integration tests may become redundant when unit test are implemented correctly. You are already testing each component of the app. Integration testing only tests how the components or units interact with each other.

41:45 We should only test behavior not implementation.

“Ask you manager if this seems like a reasonable way to test employee productivity.”

Tests should assert behavior independent of implementation. Even if the internals of a method change but not the output the tests shouldn’t change. Looking at unit tests with lots of mocks the code is similar to the code in the method itself. This could be easily confused as a test that is asserting implementation as opposed to behavior. If you use a model of mocking dependencies the test follows this pattern.

There may be times when changing the internals but not the output affects the tests. This may be due to assertions of the test being too strong. Making more calls than the ones using the mocking system could cause assertion failure. These failure wouldn’t be caught in integration testing.

Tests with mock dependencies may appear to be testing implementation but are testing behavior. Unit tests catch duplicate calls. They also will find places where the dependency injection is not properly implemented.

44:55 Automated unit tests will not replace QA testing.

QA is necessary for testing user experience and integration. Less experienced developers are more likely to not test their code if going through QA. They will assume that bugs will be found by QA testers.

“That’s not the point of it all.”

Evidence suggests that QA tests do not measure the integrity of the software. Manual testing doesn’t include load or scalability testing. It requires human testers which are mistake prone.

“QA is user acceptance testing before it gets to the user.”

This doesn’t include UAT. UAT is an essential piece of the development lifecycle. It can be tested by internal staff or external customers. UAT testing is for usability including integration testing.

48:45 Improperly written tests can lead to a false feeling of security.

Tests can be written to pass without catching any issues. They only test the “happy path” that doesn’t look at unexpected errors. Sometimes the assert statements are no better than assert that true equals true. Other times developers don’t look for ways to break their code.

“There’s a reason dude’s like us get called into consulting positions.”

Some developers will comment out old tests when adding new functionality. This means they won’t know if or how the new code breaks past functionality. It’s a lazy way of coding that avoids updating tests with code updates.

Tests should be updated with updates to the code. They are living documents. When the code changes the tests need to be updated to reflect the changes in the code. Code changes need to take into account creating testable units of code.