How Are Nullables Different From Mocks?

One of the most common questions I get about Nullables is, “How is that any different than a mock?” The short answer is that Nullables result in sociable, state-based tests, and mocks (and spies) result in solitary, interaction-based tests. This has two major benefits:

  1. Nullables catch bugs that mocks don’t.
  2. Nullables don’t break when you refactor.

Let’s dig deeper.

  1. Why They’re Different
  2. Nullables Catch More Bugs
  3. Nullables Don’t Break When You Refactor
  4. Conclusion

Why They’re Different

Imagine you have a class named HomePageController. It has a dependency, Rot13Client, that it uses to make calls to an external service. Rot13Client in turn depends on HttpClient to make the actual HTTP call to the service.

A class diagram for the example. HomePageController has an arrow pointing to Rot13Client, which has an arrow pointing to HttpClient. HttpClient has a jagged arrow pointing to Rot13Server. A class diagram for the example. HomePageController has an arrow pointing to Rot13Client, which has an arrow pointing to HttpClient. HttpClient has a jagged arrow pointing to Rot13Server.

Example design

A mock-based test of HomePageController will inject MockRot13Client in place of the real Rot13Client. It validates HomePageController by checking that the correct methods were called on the MockRot13Client.

The example design has been expanded with a test class pointing at HomePageController. The connection to Rot13Client has been x’d out and replaced with a connection to MockRot13Client. Rot13Client and all its dependencies are greyed out.The example design has been expanded with a test class pointing at HomePageController. The connection to Rot13Client has been x’d out and replaced with a connection to MockRot13Client. Rot13Client and all its dependencies are greyed out.

A mock-based test

This mock-based test is a “solitary, interaction-based test.” It’s solitary because the HomePageController is isolated from its real dependencies, and it’s interaction-based because the test checks how HomePageController interacts with its dependencies.

In contrast, a Nullable-based test of HomePageController will inject a real Rot13Client. The Rot13Client will be “nulled”—it’s configured not to talk to external systems—but other than that, it’s the exact same code that runs in production. The test validates HomePageController by checking its state and return values.

The example design has been expanded with a test class pointing at HomePageController. There is no mock class; instead, HomePageController depends on Rot13Client, which depends on HttpClient. Each of these connections is marked “nulled.” The jagged connection between HttpClient and Rot13Service has been x’d out. Rot13Service is greyed out.The example design has been expanded with a test class pointing at HomePageController. There is no mock class; instead, HomePageController depends on Rot13Client, which depends on HttpClient. Each of these connections is marked “nulled.” The jagged connection between HttpClient and Rot13Service has been x’d out. Rot13Service is greyed out.

A Nullable-based test

This is a “sociable, state-based test.” It’s sociable because the HomePageController talks to its real dependencies, and they talk to their real dependencies, and so on, all the way to the edge of the system. It’s state-based because the test checks HomePageController’s state and return values, not its interactions.

Nullables Catch More Bugs

Bugs tend to live in the boundaries. Imagine that someone intentionally changes the behavior of Rot13Client, not realizing that HomePageController relies on the old behavior. Now HomePageController doesn’t work properly. A well-meaning change to Rot13Client has introduced a bug in HomePageController.

Solitary tests, such as mock-based tests, can’t catch that bug. HomePageController’s tests don’t run the real Rot13Client, so they don’t see that the behavior is changed. The tests continue to pass, even though the code has a bug.

The “mock-based test” diagram has been annotated. It says, “A change here (Rot13Client) has an unexpected side effect here (HomePageController) and the mock (MockRot13Client) hides it. (Crying face emoji.)”The “mock-based test” diagram has been annotated. It says, “A change here (Rot13Client) has an unexpected side effect here (HomePageController) and the mock (MockRot13Client) hides it. (Crying face emoji.)”

How mocks hide bugs

Sociable tests, including Nullable-based tests, do catch that bug. That’s because HomePageController’s tests run the real Rot13Client. When its behavior changes, so do the tests results. The tests fail, revealing the bug.

The “Nullables-based test” diagram has been annotated. It says, “A change here (Rot13Client) has an unexpected side effect here (HomePageController) and it’s caught here (the test). (Celebration emoji.)”The “Nullables-based test” diagram has been annotated. It says, “A change here (Rot13Client) has an unexpected side effect here (HomePageController) and it’s caught here (the test). (Celebration emoji.)”

How Nullables reveal bugs

Nullables Don’t Break When You Refactor

Imagine that you need to change the Rot13Client API to support cancelling requests. You change its API, and when you do, you also update HomePageController to use the new API.

Interaction-based tests, such as mock-based tests, will break when you make this change. They’re expecting HomePageController to call the old API, and now it calls the new API.1

1Automated refactoring tools can prevent this problem, but not in every case.

The “mock-based test” diagram has been annotated. It says, “A design change here (Rot13Client) causes a failure here (the test) until the change is duplicated here (MockRot13Client).”The “mock-based test” diagram has been annotated. It says, “A design change here (Rot13Client) causes a failure here (the test) until the change is duplicated here (MockRot13Client).”

How mocks prevent refactoring

State-based tests, in contrast, won’t break when you refactor a dependency. The test checks the output of the HomePageController, not the methods it calls. As long as the code continues to return the correct value, the test will continue to pass.

The “mock-based test” diagram has been annotated. It says, “A design change here (Rot13Client) causes a failure here (the test) until the change is duplicated here (MockRot13Client).”The “mock-based test” diagram has been annotated. It says, “A design change here (Rot13Client) causes a failure here (the test) until the change is duplicated here (MockRot13Client).”

How Nullables support refactoring

Conclusion

Although Nullables and mocks seem similar at first glance, they take opposite approaches to testing. Nullables are sociable and state-based; mocks are solitary and interaction-based. This allows Nullable-based tests to catch more bugs and support more refactorings.

“Testing Without Mocks” Training

To be notified about future “Testing Without Mocks” training courses, join the mailing list here (requires Google login).

For private training, contact me directly.

If you liked this entry, check out my best writing and presentations, and consider subscribing to updates by email or RSS.