One of the most common questions I get about Nullables is, “How is that any different than a mock?” The short answer is that Nullables result in sociable, state-based tests, and mocks (and spies) result in solitary, interaction-based tests. This has two major benefits:

1. Nullables catch bugs that mocks don’t.
2. Nullables don’t break when you refactor.

Let’s dig deeper.

1. Why They’re Different
2. Nullables Catch More Bugs
3. Nullables Don’t Break When You Refactor
4. Conclusion

Why They’re Different

Imagine you have a class named HomePageController. It has a dependency, Rot13Client, that it uses to make calls to an external service. Rot13Client in turn depends on HttpClient to make the actual HTTP call to the service.

A mock-based test of HomePageController will inject MockRot13Client in place of the real Rot13Client. It validates HomePageController by checking that the correct methods were called on the MockRot13Client.

This mock-based test is a “solitary, interaction-based test.” It’s solitary because the HomePageController is isolated from its real dependencies, and it’s interaction-based because the test checks how HomePageController interacts with its dependencies.

In contrast, a Nullable-based test of HomePageController will inject a real Rot13Client. The Rot13Client will be “nulled”—it’s configured not to talk to external systems—but other than that, it’s the exact same code that runs in production. The test validates HomePageController by checking its state and return values.

This is a “sociable, state-based test.” It’s sociable because the HomePageController talks to its real dependencies, and they talk to their real dependencies, and so on, all the way to the edge of the system. It’s state-based because the test checks HomePageController’s state and return values, not its interactions.

Nullables Catch More Bugs

Bugs tend to live in the boundaries. Imagine that someone intentionally changes the behavior of Rot13Client, not realizing that HomePageController relies on the old behavior. Now HomePageController doesn’t work properly. A well-meaning change to Rot13Client has introduced a bug in HomePageController.

Solitary tests, such as mock-based tests, can’t catch that bug. HomePageController’s tests don’t run the real Rot13Client, so they don’t see that the behavior is changed. The tests continue to pass, even though the code has a bug.

Sociable tests, including Nullable-based tests, do catch that bug. That’s because HomePageController’s tests run the real Rot13Client. When its behavior changes, so do the tests results. The tests fail, revealing the bug.

Nullables Don’t Break When You Refactor

Imagine that you need to change the Rot13Client API to support cancelling requests. You change its API, and when you do, you also update HomePageController to use the new API.

Interaction-based tests, such as mock-based tests, will break when you make this change. They’re expecting HomePageController to call the old API, and now it calls the new API.¹

¹Automated refactoring tools can prevent this problem, but not in every case.

State-based tests, in contrast, won’t break when you refactor a dependency. The test checks the output of the HomePageController, not the methods it calls. As long as the code continues to return the correct value, the test will continue to pass.

Conclusion

Although Nullables and mocks seem similar at first glance, they take opposite approaches to testing. Nullables are sociable and state-based; mocks are solitary and interaction-based. This allows Nullable-based tests to catch more bugs and support more refactorings.

“Testing Without Mocks” Training

Jun

6

2023

I’m offering live online training for this material in four 3-hour sessions from June 6th to 15th. Register here.

To be notified about future “Testing Without Mocks” training courses, join the mailing list here (requires Google login).

For private training, contact me directly.