Nullables Livestream #8: Nullable Application

In this weekly livestream series, I pair up with Ted M. Young, aka jitterted, to look at Nullables and A-Frame Architecture as an alternative to Mocks, Spies, and Hexagonal Architecture. Each episode combines a healthy dose of architecture and design discussion with hands-on coding.

Watch us live every Tuesday! For details, see the event page. For more episode recordings, see the episode archive.

In this episode...

Now that we’ve finished making our adapters nullable, we’re ready to introduce them to our application code. We start by making GameService nullable, then clean up all the existing tests to use GameService.createNull() rather than creating and injecting dependencies manually. That leaves plenty of time for design discussion, and a start to our next feature: persistence! We design our database schema and decide how to tackle the problem.

Visit the episode archive for more.

Jan
30
2023
Livestream: Nullables and A-Frame Architecture with Ted M. Young

The schedule has changed! We’re now streaming on Mondays from 1-4pm Pacific.

Mondays from 1-4pm Pacific, I’m livecoding on Twitch with Ted M. Young, aka jitterted. We’re looking at Nullables and A-Frame Architecture as an alternative to Mocks, Spies, and Hexagonal Architecture. It’s a good time. Join us if you like to get down and dirty with technical details.

Next Stream: January 30

Databases! With our initial implementation of nullables complete, it’s time to start on a new feature. Persistence to a database is the plan. We’ll start by making a GameRepository adapter that will be responsible for saving and loading our Game object.

Episode Archive

Find recordings of past episodes here.

Nullables Livestream #7: High-Level Adapters

In this weekly livestream series, I pair up with Ted M. Young, aka jitterted, to look at Nullables and A-Frame Architecture as an alternative to Mocks, Spies, and Hexagonal Architecture. Each episode combines a healthy dose of architecture and design discussion with hands-on coding.

Watch us live every Tuesday! For details, see the event page. For more episode recordings, see the episode archive.

In this episode...

We take advantage of our new JsonHttpClient infrastructure to simplify our ScoreCategoryNotifier and AverageScoreFetcher adapters. Now they can be tested easily and no longer need clunky embedded stubs. Along the way, we have lots of conversations about design—test design, input validation, anti-corruption layers, and more.

Visit the episode archive for more.

“Testing Without Mocks” Training Survey

I'm putting together a training course for my "Testing Without Mocks" / Nullables material. If you're interested, can you fill out a short survey? It will help me decide what to include.

The survey is closed. Thank you for your help!

Nullables Livestream #6: Finish HttpClient

In this weekly livestream series, I pair up with Ted M. Young, aka jitterted, to look at Nullables and A-Frame Architecture as an alternative to Mocks, Spies, and Hexagonal Architecture. Each episode combines a healthy dose of architecture and design discussion with hands-on coding.

Watch us live every Tuesday! For details, see the event page. For more episode recordings, see the episode archive.

In this episode...

We finish off our Nullable JsonHttpClient adapter by adding support for multiple responses per endpoint, output tracking, and the post() method. Now we’re ready to use it in the rest of our code. In the final few minutes of the episode, we demonstrate how it works by using it in HttpScoreCategoryNotifier.

Visit the episode archive for more.

Back to the Office With the Agile Fluency® Game

With the holidays firmly in the rear-view mirror, it’s time to focus on the work ahead. That means getting back to the office—whether it’s in-person or remote—and working together as effective teams.

And what better way to refresh that teamwork than with the Agile Fluency Game?

A picture of the Agile Fluency Game box.

I’m happy to announce that I am now offering in-person and remote workshops featuring the Agile Fluency Game. Participants will play the game and learn how to apply its lessons to their real-world work. It’s great as a standalone workshop, an icebreaker for an internal conference, or as part of larger work process design discussion. Any number of players can be accommodated, with each group of 3-6 people playing a separate copy of the game. The minimum session length is two hours.

Remote workshops start at $4,000 and in-person workshops start at $5,000. All workshops are customized to your specific needs. Contact me for a free consultation.

In-person workshops include a physical copy of the game that’s yours to keep!

About the Agile Fluency Game

The Agile Fluency Game is a fun and thought-provoking exercise for team members, managers, and executives. It accurately simulates the first 2½ years of a newly-formed software development team. The challenge in the game is to decide how and when to invest in developing the team’s capability, and when to invest in delivering features instead.

A picture of the Agile Fluency Game in progress.

Photo courtesy of Bart den Haak

The game is inspired by modern board game experiences such as Settlers of Catan, Ticket to Ride, and Pandemic. It’s a genuinely fun experience, and a challenge, too! Many players are excited to play again to try for a better score.

The game was designed by James Shore and Arlo Belshee. Both are Agile Alliance Gordon Pask Award recipients, software developers, and avid game players.

What You’ll Learn

“Saved us money on our agile transition effort by getting a preview in 90 minutes!”

—Director, Agile Center of Excellence

“Makes the need for best practices come to life.”

—Manager

“Played first game and beautifully painted ourselves into a corner through the same mistakes real teams make. How’d that happen? Powerful teaching.”

—Senior Learning & Development Manager

“[Our leadership team] experienced 20 years of agile learning in 4 hours!”

—Director, R & D

“The shift from playing the game gave new focus to our discussion of leadership challenges.”

—VP, Product Management

The game uses experiential learning to bring software development concepts to life. One session, which includes a retrospective on your play, may be all you need to stimulate actionable insights. Yet the game challenges prior assumptions, and most players enjoy the opportunity to try for a better score. Each time brings greater understandings of the benefits, tradeoffs, and investments you need to succeed.

The game focuses on the role software development practices play in a team’s success. As they play, participants will make choices about how to invest their time. They’ll choose between practices such as Continuous Integration, User Stories, Retrospectives, Test-Driven Development, Exploratory Testing, and much more. They’ll experience the costs and benefits of these tradeoffs: how they take time away from feature development, how much time they take to master, and how maintenance costs change the equation.

The end result is a powerful new understanding of how individual practices fit into the big picture of software development, and what is needed for success. After playing the game, participants will retrospect on what they’ve learned and integrate it into their reality of their day-to-day work. They’ll come out of it with new ideas for improving software development in your organization.

The Agile Fluency Game workshop is available for both remote and in-person sessions. Contact James Shore to arrange a session at your company.

Nullables Livestream #5: get() HttpClient

In this weekly livestream series, I pair up with Ted M. Young, aka jitterted, to look at Nullables and A-Frame Architecture as an alternative to Mocks, Spies, and Hexagonal Architecture. Each episode combines a healthy dose of architecture and design discussion with hands-on coding.

Watch us live every Tuesday! For details, see the event page. For more episode recordings, see the episode archive.

In this episode...

In the last few episodes, we made the adapters for Ted’s scoring service Nullable. These are the “out” or “secondary” adapters in the codebase’s Hexagonal architecture.

Now that the adapters are Nullable, we see an opportunity create a low-level HttpClient adapter that the scoring service adapters can use. This should simplify the Nullability logic in those adapters and open up new opportunities for testing. We start with the get() method and the Nullable’s configurable responses.

Visit the episode archive for more.

The Problem With Dependency Injection Frameworks

Over on his “Shade Tree Developer” blog, Jeremy Miller threw some shade1 my way after praising my recent update to the Testing Without Mocks: A Pattern Language article.

1See what I did there? Eh? Eh? …Yeah.

Of course, I’m going to ignore the nice things he said and focus on the criticism. He wrote:

Some time over the holidays Jim Shore released an updated version of his excellent paper Testing Without Mocks: A Pattern Language. He also posted this truly massive thread with some provocative opinions about test automation strategies:

[Mastodon thread]

I think it’s a great thread over all, and the paper is chock full of provocative thoughts about designing for testability. Moreover, some of the older content in that paper is influencing the direction of my own work with Wolverine. I’ve also made it recommended reading for the developers in my own company.

All that being said, I strongly disagree with approach the approach he describes for integration testing with “nullable infrastructure” and eschewing DI/IoC for composition in favor of just willy nilly hard coding things because “DI [i]s scary” or whatever.

—Jeremy Miller, “Automating Integration Tests using the ‘Critter Stack’

“DI is scary or whatever?” DI IS SCARY? Oh, it’s on.

I Ain’t Afraid of No Ghosts

Let me be clear: I don’t have a problem with dependency injection. One of my most popular articles is “Dependency Injection Demystified”, and if you look at the Testing Without Mocks examples, you’ll see that dependency injection is used nearly everywhere.

No, my problem isn’t with dependency injection. My problem is with dependency injection frameworks.

The Cost of Third-Party Code

People think of “build vs. buy” as a simple decision: either I pay to build and maintain something myself, or I pay someone else to build and maintain it for me. Easy! Now I just need to worry about whether the dependency is part of my core competency or not. If it is, I build, because I need to be able to make changes easily. For everything else, I buy, so I can focus my time where it matters.

This view of build vs. buy is clean, pleasant, and 100% wrong.

The common view of build vs. buy is clean, pleasant, and 100% wrong.

You’re not comparing the cost of building and maintaining it yourself to the cost of buying it from someone else. You’re comparing it to the cost of buying it, learning it, working around not-quite-right behavior, keeping up with updates, and dealing with incompatibilities.

Every line of code in your system adds to your maintenance burden, and third-party code adds more to your maintenance burden than well-designed and tested2 code your company builds itself.

2Ay, there’s the rub. I’m assuming competence. That includes having more than one person who understands the code. (If your company isn’t competent, well, you know what you need to do.)

  • Your code is designed to handle the specific narrow case you care about.3 Third-party code is generic, making it harder to understand and use. Often, much harder.

  • Your code is updated on your schedule, and is built to integrate with the rest of your code. Third-party code is updated on others’ whims, and can break your existing code. Security-critical upgrades can result in “dependency hell:” cascading incompatibilities requiring you to drop everything as you scramble to make everything work together again.

  • There’s usually someone you can ask about how your code works. Third-party maintainers may not be interested in answering support requests, and you’ll be lucky if the documentation covers every obscure edge case.

  • Your code is maintained by full-time professionals. It’s as high-quality and secure as your company cares to invest.4 Third-party code is often maintained by part-time volunteers and students. It’s subject to catastrophic security holes, fits of pique, and supply-chain attacks.

3If it’s not, you’re doing it wrong. Build exactly what you need, and no more. Then evolve it as your needs change.

4There’s that rub again. But if your company isn’t willing to pay for security of your code, there’s no way they’re going to pay to properly secure third-party code.

Sometimes these costs are worth it! But it’s not the clean, simple, “only build core competencies” people would have you believe. Third-party code is expensive. Even when it’s free.

Magic Frameworks Are Bad

Frameworks have the problems of third-party code in spades. They tend to be these massive kitchen-sink things that have loads of edge cases and undocumented behavior. Because they’re so big, they have lots of security risks, and their reliance on plug-ins means that you’re sure to run face-first into dependency hell sooner or later. Not to mention the mutual finger-pointing that occurs when a set of plug-ins won’t play nice together.

But magic… magic makes it worse. “Magic” is when a framework does something automatically, in a cool way. It uses esoteric language features like monkey-patching and reflection to make you say, “wow!” It’s fun to use and makes for a great demo.

And then you do something wrong and your code doesn’t work and now the deadline is here and it still isn’t working and you have no idea Why. It. Won’t. Just. Work!

That’s magic. Sorta cool when it works. Absolute hell when it doesn’t.

That’s magic. Sorta cool when it works. Absolute hell when it doesn’t.

I need to know everything that’s going on in my code. I need simple, straightforward function calls. Nothing else! I want to be able to start at main() and trace through the code. I want to look at callers and find where every parameter came from. I want to get a stack trace and see the line where I made a mistake. Reading code is hard enough already. Magic frameworks make it harder.

But again, sometimes these costs are worth it. Building magical frameworks is a favored pastime of senior developers. Chances are good that the “best” way to solve a common problem in your language involves a framework. Just… for the love of Bob… only use a magic framework if you don’t have another choice.

Dependency Injection Frameworks Encourage Bad Design

Now we get to dependency injection frameworks. Third-party code, check. Magic framework, check. And for what? To move object instantiation from one part of the code to another? From the place where it’s used to a distant, opaque blob? A blob that might not even be real code?

No, that’s uncharitable. Hard-coded dependencies are bad. Microsoft says:

Hard-coded dependencies, such as in the previous example [of a Worker class that depends on a MessageWriter instance], are problematic and should be avoided for the following reasons:

  • To replace MessageWriter with a different implementation, the Worker class must be modified.

  • If MessageWriter has dependencies, they must also be configured by the Worker class. In a large project with multiple classes depending on MessageWriter, the configuration code becomes scattered across the app.

  • This implementation is difficult to unit test. The app should use a mock or stub MessageWriter class, which isn't possible with this approach.

—Microsoft, “Dependency injection in .NET

Yep, that sure is terrible! Except it’s all a crock. Let’s dismiss the first and last points quickly:

  • “The Worker class must be modified.” As compared to what, changing some other file? Yeah, I’m fine with that. It’s called cohesion. Put the code where it’s used, not in some far off mess. (Seriously, have you seen the example code? Search for CreateHostBuilder. Who reads this and thinks, “Oh yeah, this is so much better than calling new?”)

  • “This implementation is difficult to unit test.” Horsepucky. You can still have dependency injection without a framework. Make a constructor that takes the dependency as an optional parameter. Done. Applause. Early lunch.

The big one is the second one. “If MessageWriter has dependencies… the configuration code becomes scattered across the app.” You can’t just say new Worker(). You have to say new Worker(new MessageWriter(myConfig)). The deeper your dependency chain, the worse it gets.

Or does it?

When your code is hard to work with like this, it’s telling you something. It’s telling you that your design is bad and needs to be improved. The problems dependency injection frameworks solve are problems of bad design.

The problems dependency injection frameworks solve are problems of bad design.
  • If you have to construct massive dependency chains, you have poor encapsulation. Each class should unpack the configuration its given and pass just what is needed to the next level down. Callers should only worry about their direct dependencies.

  • If you have to pass the same dependency to every method, you have poor class responsibility design or poor application architecture. You shouldn’t need logging everywhere, for example. Your logic code should be side-effect free, and most of the rest of your code should be throwing detailed exceptions or returning errors rather than writing to the log and returning null.

  • If you have huge lists of dependencies to pass in to a method, you have a data clump, and possibly the other problems too. Identify which dependencies belong together and combine them into a single class. Figure out which dependencies you don’t need and improve your design so you don’t have to pass them around.

Furthermore, dependency injection frameworks encourage you to think in terms of globals. That’s what they inject! A globally-configured instance of a class. Think about it. If you one day want two different instances of an injected variable, you’ll need an impact driver to express just how screwed you are.5 This has all kinds of knock-on effects in terms of reducing encapsulation and separating state from behavior.

5Yeah, yeah, yeah. You have a fancy DI framework. You can make named instances or whatever. But at what complexity cost? And how is that better than just passing a parameter?

Sure, Go Ahead

But that’s just like, my opinion, man.6

6Duuuude.

I get it. Good design is hard, and sometimes we’re stuck with code that’s, um, Not So Great. Engineering is tradeoffs, and there’s no one right answer when it comes to design. DI frameworks solve design problems at the cost of increased maintenance costs and lower-quality design. Manual dependency injection solves those problems, but at the cost of spending more time passing around variables and thinking about how to structure your dependencies.

Look, I’m a professional. I get the job done. Pretty much every client I work with uses a DI framework. I’m not going to do that passive-aggressive programmer thing where I sniff dismissively and refuse to work on any code that involves a DI framework. I know how to work with DI frameworks, I do it, and I’ll even admit they make life easier. The Testing Without Mocks patterns are designed to let you not have a DI framework, but they work just as well if you do.

But I also think they’re a crutch. Next time you start a new codebase from scratch, try it without a DI framework. You’ll have to spend a lot more time thinking about good design, and that’s a good thing. You might be surprised where it takes you.

As Brian Marick likes to say, “An Example Would Be Handy Right About Now.” A complete example exists, but documentation and cleanup is still in progress. See the “complex example” link in the Testing Without Mocks Examples. That link will be updated to point to the final example when it’s ready.

“Testing Without Mocks” Training

My Testing Without Mocks article has been a big hit! I’m planning to offer a public training course on the material. If you’d like to be notified when the course is available:

That mailing list will only be used to announce “Testing Without Mocks” training courses. If you’d like to be informed of all new material I post, I have more options here.

Colophon

Every so often, I describe what I do to make this site a reality. For the four of you who like to watch me gaze at my navel, I have good news: today is one of those days. For the rest of you, I’m sorry. It’s one of those days. You can move on. I won’t be offended.

Okay, here goes.

Production

I use a 14” M1 MacBook Pro with a large external monitor to compose all my essays. I love this machine. Years ago, my Windows laptop died and I decided to see if life was better on the other side. I’ve come to like MacOS, after some initial frustrations, and now I have a hard time imagining going back.

I write my essays in hand-crafted HTML using Webstorm. The files are served using a custom content management engine (CME) I originally wrote for my Let’s Code JavaScript screencast. It’s a polyglot engine, so I could theoretically use other markup languages, such as Pug, Markdown, or AsciiDoc, but I’m used to HTML. I do have a very hacked-together parser for AsciiDoc that I wrote to support my book, which is written in AsciiDoc, but I use HTML everywhere else.

Everything required to make the site go, including all the content, is stored as static files in a git repository. I run my CME locally for testing, then push to my web host when I’m ready to go live. I’ve got a really nice local build script that validates my changes and restarts the server faster than I can switch windows.

I’m paranoid about backups and revision control. There’s decades of work here. That’s why my essays are written in a text editor and stored in git. Most blogging engines save your work in the cloud. Easy, but not fault tolerant. I’ve been writing for nearly two decades and plan to continue writing for several more. My stuff has to be easily scripted, version-controlled, backed up, and trivial to deploy to another web host when (not if) my host goes under... which it already has. Twice.

My site is backed up many times over. First, in the git repo on my laptop. Second, in a copy of the git repo on my web host. The computer itself is backed up to two redundant drives with Time Machine every hour. I also make a bootable whole-disk backup using SuperDuper! every night. Once per quarter, I rotate one of my backup drives to an off-site location. I also continuously backup to the cloud using Backblaze.

Yeah... I might be a bit paranoid. But in 20 years, I’ve never lost data... at least, not that I couldn’t recover.

Design

The site was originally designed in 2005. For 15 years, it had a very... “classic” look, and it didn’t work well on mobile. In 2020, I finally redesigned the site. I’m very happy with it, and particularly proud of its dark mode support and custom print styling.1

1Try it out! Click the “Print” button at the top of this page, or just tell your browser to print the page.

I’m not much of a graphic designer, so I did what great artists do: I stole whatever I could. Dave Liepmann’s Tufte CSS, based on the work of Edward Tufte, was my principal inspiration. The home page was inspired by Michael “GeePaw” Hill’s website,2 although he’s since changed his design.

2GeePaw Hill is super smart and thoughtful about Agile engineering practices. Go read his stuff.

The site’s fairly monochromatic, but there’s bits of color here and there. The colors are based on my Let’s Code JavaScript site, which was professionally designed by the talented folks at Primate.

This is a JavaScript-free site, almost, which is a bit ironic considering that I had a subscription screencast about JavaScript for many years. I guess you could say I know it well enough to know when not to use it. The one exception is syntax highlighting, which is provided by Prism. It’s all progressive, though, so the site works fine if JavaScript is disabled.

Icons3 are in SVG, which I want to love, and probably will someday. After they make it better. I purchased my icons from The Noun Project, except for the Mastodon and Twitter logos, which I got from their respective brand toolkits. I also use emojis as icons4 in a few spots, mostly because they’re so much easier to work with.

3 Subscribe icon Mastodon icon Twitter icon Search icon Print icon

4 ⭐️ 📖 🎙

Fonts come from Adobe. I’m using Utopia for my main serif font, Cronos for my sans serif buttons and some links, and good old Courier for monospace—or Menlo, if you happen to have it installed.

Finally, lots of elbow grease got me to the result I have today. MDN, the Mozilla Developer Network, was invaluable for figuring out the style sheet. CSS has improved a lot since 2005, and I think I might even... like it? Well, almost.

Nearly all of the original design has been wiped away, but one I did keep one of my favorite aspects: printed URLs. I learned that from Eric Meyer’s “Going to Print” article.

Hosting

The site is hosted by Heroku and runs on a custom content management engine I wrote in Node.js. Logging is handled by Papertrail, which also sends me email and text message alerts when things go wrong. I use Pingdom as a backup alerting system. TierraNet makes sure that my domains point to the right place. I use DuckDuckGo for my site search and Google Groups for update emails.

Also, I’ve finally dumped Google Analytics. Woot! No cookies or tracking of any sort, in fact, unless you visit a page with an embedded YouTube video. YouTube’s behavior is out of my control, unfortunately.

The combination of custom source code and outside services gives me total control over the code behind the site while outsourcing a lot of the complexities of modern software stacks. It’s a nice tradeoff that allows me to focus my attention on more important things. The code is very well tested, so it almost never fails. Because there’s no database and minimal service dependencies, my uptime is the same as Heroku’s, and requires no effort from me.

Having custom source code has allowed me to tune the site into a well-oiled machine. I don’t get a ton of traffic—I’m lucky to average 15 requests per second—but a single low-end Heroku dyno is all I’ve needed to handle everything the Internet has thrown at me. Typical service time is around 5ms, and I’m confident enough in its performance that I start getting warning emails if responses take longer than 100ms.

History

In the very beginning, the site was rendered by a very simple static site generator I wrote in Ruby. But for 15 years, from February 2005 to July 2020, the site was rendered by the ultra-minimalistic Blosxom. That’s when I started blogging in earnest; you can see my very first entry here.

Back in the day, Blosxom was the only blogging software I could find that actually allowed me to store entries locally, in files that I could back up and put in version control, rather than on a database on the server. (Nowadays, of course, static site generators are a dime a dozen.)

Blosxom ran on Perl 5 in Apache on NearlyFreeSpeech.NET. It did the job, but my volume of content caused performance problems, and it had a lot of idiosyncracies. It was low maintenance, but also constrained what I could do. My new codebase is even lower maintenance, and more powerful. I’m happy to have moved on.

Colophonem adidi.

Testing Without Mocks: A Pattern Language

This is a draft update of my original 2018 article. The final version will be published at this URL in early 2023. In the meantime, please send me your feedback:

Emailjshore@jamesshore.com
Mastodon@jamesshore@mastodon.online
DiscordInvite link

Automated tests are important. Without them, programmers waste a huge amount of time manually checking and fixing their code.

Unfortunately, many automated tests also waste a huge amount of time. The easy, obvious way to write tests is to make broad tests that are automated versions of manual tests. But they’re flaky and slow.

Folks in the know use mocks and spies (I say “mocks” for short in this article) to write isolated interaction-based tests. Their tests are reliable and fast, but they tend to “lock in” implementation, making refactoring difficult, and they have to be supplemented with broad tests. It’s also easy to make poor-quality tests that are hard to read, or end up only testing themselves.

Bad tests are a sign of bad design, so some people use techniques such as Hexagonal Architecture and functional core, imperative shell to separate logic from infrastructure. (Infrastructure is code that involves external systems or state.) It fixes the problem... for logic. But infrastructure is often left untested, and it requires architectural changes that are out of reach for people with existing code.

This pattern language1 describes a fourth option. It avoids all the above problems: it doesn’t use broad tests, doesn’t use mocks, doesn’t ignore infrastructure, and doesn’t require architectural changes. It has the speed, reliability, and maintainability of unit tests and the power of broad tests. But it’s not without tradeoffs of its own.

1The structure of this article was inspired by Ward Cunningham’s CHECKS Pattern Language of Information Integrity, which is a model of clarity and usefulness.

The patterns combine sociable, state-based tests with a novel infrastructure technique called “Nullables.” At first glance, Nullables look like test doubles, but they're actually production code with an “off” switch. And that’s the tradeoff: do you want that in your production code? Your answer determines whether this pattern language is for you.

The rest of the article goes into detail. Don’t be intimidated by its size. It’s broken up into bite-sized pieces with lots of code examples.

“Testing Without Mocks” Training

Due to this article’s popularity, I’m planning to offer a public training course on this material. To be notified when it’s available, send an email with the subject “Subscribe” to nullables-training+subscribe@googlegroups.com or join the mailing list here (requires Google login).

For private company training, contact me directly.

Contents:

Examples

Here’s an example of testing a simple command line application. The application reads a string from the command line, encodes it using ROT-13, and outputs the result.

The production code uses the optional A-Frame Architecture pattern. App is the application entry point. It depends on Rot13, a Logic class, and CommandLine, an Infrastructure class. Additional patterns are mentioned in the source code.

// Example production code (JavaScript + Node.js)
import { CommandLine } from "./infrastructure/command_line";  // Infrastructure Wrapper
import { Rot13 } from "./logic/rot13";

export class App {
  constructor(commandLine = CommandLine.create()) {   // Parameterless Instantiation
    this._commandLine = commandLine;
  }

  run() {
    const args = this._commandLine.args();

    if (args.length === 0) {    // Tested by Test #2
      this._commandLine.writeOutput("Usage: run text_to_transform\n");
      return;
    }
    if (args.length !== 1) {    // Tested by Test #3
      this._commandLine.writeOutput("too many arguments\n");
      return;
    }

    // Tested by Test #1
    const input = args[0];                          // Logic Sandwich
    const output = Rot13.transform(input);
    this._commandLine.writeOutput(output + "\n");
  }
};

The tests of App look like end-to-end integration tests, but they’re actually unit tests. Technically, they’re Narrow, Sociable tests, which means they’re unit tests that execute code in dependencies.

As narrow tests, the tests only care about testing App.run(). Each of the dependencies is expected to have tests of their own, which they do.

The tests use a Nullable CommandLine to throw away stdout and Configurable Responses to provide pre-configured command-line arguments. They also uses Output Tracking to see what would have been written to stdout.

// Example tests (JavaScript + Node.js)
import { assert } from "assert";
import { CommandLine } from "./infrastructure/command_line";
import { Rot13 } from "./logic/rot13";
import { App } from "./app";

describe("App", () => {
  // Test #1
  it("reads command-line argument, transform it with ROT-13, and writes result", () => {
    const { output } = run({ args: [ "my input" ] });     // Signature Shielding, Configurable Responses
    assert.deepEqual(output.data, [ "zl vachg\n" ];       // Output Tracking
  });

  // Test #2
  it("writes usage when no argument provided", () => {
    const { output } = run({ args: [] });                                 // Signature Shielding, Configurable Responses
    assert.deepEqual(output.data, [ "Usage: run text_to_transform\n" ]);  // Output Tracking
  });

  // Test #3
  it("complains when too many command-line arguments provided", () => {
    const { output } = run({ args: [ "a", "b" ] });                       // Signature Shielding, Configurable Responses
    assert.deepEqual(output.data, [ "too many arguments\n" ]);            // Output Tracking
  });

  function run({ args = [] } = {}) {                      // Signature Shielding
    const commandLine = CommandLine.createNull({ args }); // Nullable, Infrastructure Wrapper, Configurable Responses
    const output = commandLine.trackOutput();             // Output Tracking

    const app = new App(commandLine);
    app.run();

    return { output };                                    // Signature Shielding
  }
});

If you’re familiar with mocks, you might assume CommandLine is a test double. But it’s actually production code with an “off” switch and the ability to monitor its output.

// Example Nullable infrastructure wrapper (JavaScript + Node.js)
import { EventEmitter } from "node:events";
import { OutputTracker } from "output_tracker";

const OUTPUT_EVENT = "output";

export class CommandLine {
  static create() {
    return new CommandLine(process);                  // 'process' is a Node.js global
  }

  static createNull({ args = [] } = {}) {             // Parameterless Instantiation, Configurable Responses
    return new CommandLine(new StubbedProcess(args)); // Embedded Stub
  }

  constructor(proc) {
    this._process = proc;
    this._emitter = new EventEmitter();               // Output Tracking
  }

  args() {
    return this._process.argv.slice(2);
  }

  writeOutput(text) {
    this._process.stdout.write(text);
    this._emitter.emit(OUTPUT_EVENT, text);           // Output Tracking
  }

  trackOutput() {                                     // Output Tracking
    return OutputTracker.create(this._emitter, OUTPUT_EVENT);
  }
};

// Embedded Stub
class StubbedProcess {
  constructor(args) {
    this._args = args;                                // Configurable Responses
  }

  get argv() {
    return [ "nulled_process_node", "nulled_process_script.js", ...this._args ];
  }

  get stdout() {
    return {
      write() {}
    };
  }
}

The patterns shine in more complex code that has multiple layers of dependencies. Find more examples here:

  • Simple example. The complete source code for the above example. (JavaScript and Node.js)

  • Complex example. The blinged-out version of the above example. A web application and microservice that performs ROT-13 encoding. Production-grade code with error handling, logging, timeouts, and request cancellation. (JavaScript and Node.js; work in progress)

  • TDD Lunch & Learn Screencast. A series of one-hour webinars that demonstrate how to use the patterns. (JavaScript and Node.js)

  • Nullables Livestream. A series of three-hour livestreams with James Shore and Ted M. Young. They pair on applying the patterns to an existing web application. (Java and Spring Boot)

Contents

Goals

This pattern language was created to satisfy these goals:

  • No broad tests required. The test suite consists entirely of “narrow” tests that are focused on specific concepts. Although broad integration tests can be added as a safety net, their failure indicates a gap in the main test suite.

  • Easy refactoring. Object interactions are considered implementation to be encapsulated, not behavior to be tested. Although the consequences of object interactions are tested, the specific method calls aren’t. This allows structural refactorings to be made without breaking tests.

  • Readable tests. Tests follow a straightforward “arrange, act, assert” structure. They describe the externally-visible behavior of the unit under test, not its implementation. They can act as documentation for the unit under test.

  • No magic. Tools that automatically remove busywork, such as dependency-injection frameworks and auto-mocking frameworks, are not required.

  • Fast and deterministic. The test suite only executes “slow” code, such as network calls or file system requests, when that behavior is explicitly part of the unit under test. Such tests are organized so they produce the same results on every test run.

Experience has revealed these additional benefits:

  • Faster than mocking frameworks. In a head-to-head comparison, tests using these patterns were 2–3 orders of magnitude faster than tests using a mocking framework. (Comparison code here.)

  • Simple test setup. Test setup is straightforward and easy to encapsulate in helper methods.

  • High reusability. The most complicated code needed for these patterns is also the most generic and reusable.

  • In-memory infrastructure testing. High-level infrastructure wrappers, such as a client for a specific web service, can be tested without network calls or complicated setup. (Example test.)

  • Edge case support. It’s easy to test complex edge cases, such as error conditions and timeouts. (Example tests.)

  • Legacy code compatibility. The patterns are completely compatible with mocks and other test doubles, and can even be used together in the same test. Legacy code can be converted incrementally without impacting existing code.

Contents

Tradeoffs

Nothing’s perfect. These are the downsides of using this pattern language:

  • Changes to production code. The patterns require you to modify your production code, particularly for infrastructure classes. Although the modifications are usable in production, and have production use cases, many of the changes will only be used by tests.

  • Hand-written stub code. Some third-party infrastructure code has to be mimicked with hand-written stub code. It can’t be auto-generated and takes extra time to write. However, the results are highly reusable.

  • Multiple test failures. Although tests are written to focus on specific concepts, the units under test execute code in their dependencies. (Jay Fields coined the term “sociable tests” for this behavior.) This can result in multiple tests failing when a bug is introduced.

Contents

Foundational Patterns

Start here. These patterns establish the ground rules.

Narrow Tests

Broad tests, such as end-to-end tests, tend to be slow and brittle. They’re complicated to read and write, often fail randomly, and take a long time to run. Therefore:

Instead of using broad tests, use narrow tests. Narrow tests check a specific function or behavior, not the system as a whole. Unit tests are a common type of narrow tests.

When testing infrastructure, use Narrow Integration Tests. When testing pure logic, use the Logic Patterns. When testing code that has infrastructure dependencies, use Nullables.

To ensure your code works as a whole, use State-Based Tests and Overlapping Sociable Tests.

Contents

State-Based Tests

Mocks and spies result in “interaction-based” tests that check how the code under test uses its dependencies. However, they can be hard to read, and they tend to “lock in” your dependencies, which makes structural refactorings difficult. Therefore:

Use state-based tests instead of interaction-based tests. A state-based test checks the output or state of the code under test, without any awareness of its implementation. For example, given the following production code:

// Production code to describe phase of moon (JavaScript)
import { Moon } from "astronomy";
import { format } from "date_formatter";

export describeMoonPhase(date) {
  const visibility = Moon.getPercentOccluded(date);
  const phase = Moon.describePhase(visibility);
  const formattedDate = format(date);
  return `The moon is ${phase} on ${formattedDate}.`;
}

A state-based test would pass in a date and check the result, like this:

// State-based test of describeMoonPhase() (JavaScript)
import { describeMoonPhase } from "describe_phase";

it("describes phase of moon", () => {
  const dateOfFullMoon = new Date("8 Dec 2022");    // a date when the moon was actually full
  const description = describeMoonPhase(dateOfFullMoon);
  assert.equal(description, "The moon is full on December 8th, 2022.";
};

In contrast, an interaction-based test would check how each dependency was used, like this:

// Interaction-based test of describeMoonPhase() (JavaScript and fictional mocking framework)
const { Moon } = mocker.mockImport("astronomy");
const { format } = mocker.mockImport("date_formatter");
const { describeMoonPhase } = mocker.importWithMocks("describe_phase");

it("describes phase of moon", () => {
  const date = new Date();    // specific date doesn't matter

  mocker.expect(Moon.getPercentOccluded).toBeCalledWith(date).thenReturn(999);
  mocker.expect(Moon.describePhase).toBeCalledWith(999).thenReturn("PHASE");
  mocker.expect(format).toBeCalledWith(date).thenReturn("DATE");

  const description = describeMoonPhase(date);
  mocker.verify();
  assert.equal(description, "The moon is PHASE on DATE");
};

State-based tests naturally result in Overlapping Sociable Tests. To use state-based tests on code with infrastructure dependencies, use the Nullability Patterns.

Contents

Overlapping Sociable Tests

Tests using mocks and other test doubles isolate the code under test by replacing its dependencies. This requires broad tests to confirm that the system works as a whole, but we don’t want to use broad tests. Therefore:

When testing the interactions between an object and its dependencies, use the code under test’s real dependencies. Don’t test the dependencies’ behavior, but do test that the code under test uses its dependencies correctly. This happens naturally when using State-Based Tests.

For example, the following test checks that describeMoonPhase uses its Moon and format dependencies correctly. If they don’t work the way describeMoonPhase thinks they do, the test will fail.

// Example of sociable tests (JavaScript)

// Test code
it("describes phase of moon", () => {
  const dateOfFullMoon = new Date("8 Dec 2022");
  const description = describeMoonPhase(dateOfFullMoon);
  assert.equal(description, "The moon is full on December 8th, 2022.";
};

// Production code
describeMoonPhase(date) {
  const visibility = Moon.getPercentOccluded(date);
  const phase = Moon.describePhase(visibility);
  const formattedDate = format(date);
  return `The moon is ${phase} on ${formattedDate}.`;
}

Write Narrow Tests that are focused on the behavior of the code under test, not the behavior of its dependencies. Each dependency should have its own thorough set of Narrow Tests. For example, don’t test all the phases of the moon in your describeMoonPhase() tests, but do test them in your Moon tests. Similarly, don’t check the intricacies of date formatting in your describeMoonPhase tests, but do test them in your format(date) tests.

In addition to checking how your code uses its dependencies, sociable tests also protect you against future breaking changes. Each test overlaps with dependencies’ tests and dependents’ tests, creating a strong linked chain of tests. This gives you the coverage of broad tests without their speed and reliability problems.

For example, imagine the dependency chain LoginControllerAuth0ClientHttpClient:

  • The LoginController tests checks how LoginController uses Auth0Client. Auth0Client in turn runs HttpClient, but that isn’t explicitly checked by the LoginController tests.

  • The Auth0Client tests check how Auth0Client uses HttpClient.

  • Together, they ensure the whole chain is checked. Changing HttpClient’s behavior would break the Auth0Client tests (and possibly the LoginController tests), and changing Auth0Client’s behavior would break the LoginController tests.

In contrast, if the LoginController tests stubbed or mocked out Auth0Client, the chain would be broken. Changing Auth0Client’s behavior would not break the LoginController tests, because nothing would check how LoginController used the real Auth0Client.

To avoid manually constructing the entire dependency chain, use Parameterless Instantiation with Zero-Impact Instantiation. To isolate tests from changes in dependencies’ behavior, use Collaborator-Based Isolation. To prevent your tests from interacting with external systems and state, use Nullables. To catch breaking changes in external systems, use Paranoic Telemetry. For a safety net, use Smoke Tests.

Contents

Smoke Tests

Overlapping Sociable Tests are supposed to cover your entire system. But nobody’s perfect, and mistakes happen. Therefore:

Write one or two end-to-end tests that make sure your code starts up and runs a common workflow. For example, if you’re coding a web site, check that you can get an important page.

Don’t rely on smoke tests to catch errors. Your real test suite should consist of Narrow, Sociable tests. If the smoke tests catch something the rest of your tests don’t, fill the gap with more narrow tests.

Contents

Zero-Impact Instantiation

Overlapping Sociable Tests instantiate their dependencies, which in turn instantiate their dependencies, and so forth. If instantiating this web of dependencies takes too long or causes side effects, the tests could be slow, difficult to set up, or fail unpredictably. Therefore:

Don’t do significant work in constructors. Don’t connect to external systems, start services, or perform long calculations. For code that needs to connect to an external system or start a service, provide a connect() or start() method. For code that needs to perform a long calculation, consider lazy initialization. (But even complex calculations aren’t likely to be a problem, so profile before optimizing.)

Contents

Parameterless Instantiation

Overlapping Sociable Tests require your whole dependency tree to be instantiated, but multi-level dependency chains are difficult to set up in tests. Dependency injection (DI) frameworks work around the problem, but we don’t want to require such magic. Therefore:

Ensure that all classes have a factory or constructor that doesn’t take any parameters. In practice, this means that most objects instantiate their own dependencies, although they may also accept them as optional parameters.

For some code, particularly Value Objects, a parameterless constructor won’t make any sense. For example, an immutable “Address” class would be constructed with its street, city, and so forth. In this case, provide a test-only factory method. The factory method should provide overridable defaults for mandatory parameters. (If your language doesn’t support default parameters, use an Options object, as shown in the Signature Shielding pattern.)

The factory method is easiest to maintain if it’s located in the production code next to the real constructors. It should be marked as test-specific and should be simple enough to not need tests of its own.

// Test-specific factory using named, optional parameters (JavaScript)

class Address {
  // Production constructor
  constructor(street, city, state, country, postalCode) {
    this._street = street;
    this._city = city;
    //...etc...
  }

  // Test-specific factory
  static createTestInstance({
    street = "Address test street",
    city = "Address test city",
    state = State.createTestInstance(),
    country = Country.createTestInstance(),
    postalCode = PostalCode.createTestInstance(),
  } = {}) {
    return new Address(street, city, state, country, postalCode);
  }
}

If you don’t want test-specific code in production, you can put the factory in a test-only helper module instead.

Contents

Signature Shielding

As you refactor your application, method signatures will change. If your code is well-designed, this won’t be a problem for production code, because most methods will only be used in a few places. But tests can have many duplicated method and constructor calls. When you change those methods or constructors, you’ll have a lot of busywork to update the tests. Therefore:

Provide helper functions to instantiate classes and call methods. Have these helper functions perform any setup your tests need rather than using your test framework’s before() or setup() functions.

Make the helper functions take optional parameters for customizing your setup and execution. If convenient in your programming language, return multiple optional values as well. This will allow you to expand your helper functions without breaking existing tests.

// Optional parameters and multiple return values (JavaScript)

// Example test
it("uses hosted page for authentication", () => {
  const { url } = getLoginUrl({       // Use the helper function
    host: "my.host",
    clientId: "my_client_id",
    callbackUrl: "my_callback_url"
  });

  assert.equal(url, "https://my.host/authorize?response_type=code&client_id=my_client_id&callback_url=my_callback_url");
});
  
// Example helper function
function getLoginUrl({
  host = "irrelevant.host",           // Optional parameters
  clientId = "irrelevant_client_id",
  clientSecret = "irrelevant_secret",
  connection = "irrelevant_connection"
  username = "irrelevant_username",
  callbackUrl = "irrelevant_url",
} = {}) {
  const client = new LoginClient(host, clientId, clientSecret, connection);
  const url = client.getLoginUrl(username, callbackUrl);

  return { client, url };             // Multiple return values
}

If you’re using a language without support for optional parameters, use method overloading or an “Options” object with the Builder pattern. If you’re using a language without support for multiple return values, you can return a simple data structure.

// Optional parameters and multiple return values (Java)

// Example tests
@Test
public void usesHostedPageForAuthentication() {
  GetLoginUrlResult actual = getLoginUrl(new GetLoginUrlOptions()   // Use the helper function and Options object
    .withHost("my.host")
    .withClientId("my_client_id")
    .withCallbackUrl("my_callback_url")
  );
  assert.equal(actual.url, "https://my.host/authorize?response_type=code&client_id=my_client_id&callback_url=my_callback_url");
}

// Helper function using Options object and return data structure
private GetLoginUrlResult getLoginUrl(GetLoginUrlOptions options) {
  LoginClient client = new LoginClient(options.host, options.clientId, options.secret, options.connection);
  String url = client.getLoginUrl(options.username, options.callbackUrl);
  return new GetLoginUrlResult(client, url);
}

// Options object using Builder pattern
private static final class GetLoginUrlOptions {
  public String host = "irrelevant.host";
  public String clientId = "irrelevant_client_id";
  public String clientSecret = "irrelevant_secret";
  public String connection = "irrelevant_connection";
  public String username = "irrelevant_username";
  public String callbackUrl = "irrelevant_url";

  GetLoginUrlOptions withHost(String host) {
    this.host = host;
    return this;
  }

  GetLoginUrlOptions withClientId(String clientId) {
    this.clientId = clientId;
    return this;
  }

  GetLoginUrlOptions withCallbackUrl(String url) {
    this.callbackUrl = url;
    return this;
  }
}
  
// Return data structure
private static final class GetLoginUrlResult {
  public LoginClient client;
  public String url;
  
  public GetLoginUrlResult(LoginClient client, String url) {
    this.client = client;
    this.url = url;
  }
}

Contents

Architectural Patterns

Testing works best when you pay careful attention to the dependencies in your codebase. These architectural patterns help you do so. They aren’t required, but they’re useful.

A-Frame Architecture

Code without infrastructure dependencies is much easier to test than code that has infrastructure dependencies. However, a normal layered architecture puts infrastructure at the bottom of the dependency chain:

Application/UI
      |
      V
    Logic
      |
      V
Infrastructure

Therefore:

Structure your application so that infrastructure and logic are peers under the application layer, with no dependencies between Infrastructure and Logic. Coordinate between them at the Application layer with a Logic Sandwich or Traffic Cop. Use Value Objects to pass data between the Logic and Infrastructure layers.

   Application/UI     Values
   /            \
  V              V
Logic   Infrastructure

Build the Logic and Values layers using Logic Patterns. Build the Infrastructure layer using Infrastructure Patterns. Build the Application/UI layer with a Logic Sandwich or Traffic Cop, and use Nullables to test it.

Although A-Frame Architecture is a nice way to simplify application dependencies, it’s entirely optional. This pattern language will work without it.

To build a new application using A-Frame Architecture, Grow Evolutionary Seeds. To convert an existing codebase, Descend the Ladder.

Contents

Logic Sandwich

When using an A-Frame Architecture, the infrastructure and logic layers aren’t allowed to communicate with each other. But the logic layer needs to read and write data controlled by the infrastructure layer. Therefore:

Implement the Application layer code as a “logic sandwich,” which reads data using the Infrastructure layer, processes it using the Logic layer, then writes it using the Infrastructure layer. Repeat as needed. Each layer can then be tested independently.

// JavaScript
const input = infrastructure.readData();
const output = logic.processInput(input);
infrastructure.writeData(output);

This simple algorithm can handle sophisticated needs if put into a stateful loop. In some cases, your Application layer might need a bit of logic of its own, or you might need multiple sandwiches.

For applications that respond to events, use a Traffic Cop instead.

Contents

Traffic Cop

The Logic Sandwich boils infrastructure down into simple infrastructure.readData() and infrastructure.writeData() abstractions. But some applications need to respond to changes instigated by the infrastructure and logic layers. Therefore:

Program the application layer to use the Observer pattern to listen for events from the infrastructure and logic layers. For each event, implement a Logic Sandwich.

// Traffic Cop example (JavaScript)

server.onPost("/login", (formData) => {               // event from infrastructure layer
  const loginData = processLoginForm(formData);           // application logic
  const userData = userService.logInUser(loginData);      // infrastructure layer
  this._user = new User(userData);                        // logic layer

  const userIsValid = this._user.isValid();               // logic layer
  if (userIsValid) {                                      // application logic
    const sessionData = user.sessionData;                 // logic layer
    sessionServer.createSession(sessionData);             // infrastructure layer
    return redirect(loginData.postLoginUrl);              // application logic
  }
  else {
    return redirect(LOGIN_FAILED_URL);                    // application logic
  }
});

this._user.onChange((userData) => {                   // event from logic layer
  userService.updateUser(userData);                       // infrastructure layer
});

Be careful not to let your Traffic Cop turn into a God Class. If it gets complicated, better infrastructure abstractions might help. Sometimes taking a less “pure” approach and moving some Logic code into the Infrastructure layer can simplify the overall design. In other cases, splitting the application layer into multiple classes or modules, each with its own Logic Sandwich or simple Traffic Cop, can help.

Contents

Grow Evolutionary Seeds

One popular design technique is outside-in design, in which an application is programmed by starting with the externally-visible behavior of the application, then working your way in to the details.

This is typically done by writing a broad integration test to describe the externally-visible behavior, then using interaction tests to build higher-level functions before lower-level functions. But we want to use Narrow Tests, not broad tests, and State-Based Tests, not interaction tests. Therefore:

Use evolutionary design to grow your application from a single file. Choose a simple end-to-end behavior as a starting point, then test-drive a single class to implement a trivial version of that behavior. Hardcode one value that would normally come from the Infrastructure layer, don’t implement any significant logic, and return the result to your tests rather than displaying it in a UI. This class forms the seed of your Application layer.

// Simplest possible Application seed (JavaScript)

// Test code
it("renders user name", () => {
  const app = new MyApplication();
  assert.equal("Hello, Sarah", app.render());
});

// Production code
class MyApplication {
  render() {
    return "Hello, Sarah";
  }
}

Next, implement a barebones Infrastructure Wrapper for the one infrastructure value you hardcoded. Test-drive it with Narrow Integration Tests and code just enough to provide one real result that your Application class needs. Don’t worry about making it robust or reliable yet. This Infrastructure Wrapper class forms the seed of your Infrastructure layer.

Before integrating your new Infrastructure class into your Application class, use the Nullability Patterns to make the Infrastructure class testable from your application layer. Then modify your Application class to use the Infrastructure class, injecting the Nulled version in your tests.

// Application + read from infrastructure (JavaScript)

// Test code
it("renders user name", async () => {
  const usernameService = UsernameService.createNull({ username: "my_username" });  // Nullable with Configurable Responses
  const app = new MyApplication({ usernameService });
  assert.equal("Hello, my_username", await app.renderAsync());
});

// Production code
class MyApplication {
  constructor({
    usernameService = UsernameService.create()    // Optional parameter (Parameterless Instantiation)
  } = {}) {
    this._usernameService = usernameService;
  }

  async renderAsync() {
    const username = await this._usernameService.getUsername();
    return `Hello, ${username}`;
  }
}

Next, do the same for your UI. Choose one simple output mechanism that your application will use (such as rendering to the console, the DOM, or responding to a network request) and implement a barebones Infrastructure Wrapper for it. Make it Nullable and modify your Application layer tests and code to use it.

// Application + read/write to Infrastructure (JavaScript)

// Test code
it("renders user name", () => {
  const usernameService = UsernameService.createNull({ username: "my_username" });  // Nullable with Configurable Responses
  const uiService = UiService.createNull();       // Nullable
  const uiOutput = uiService.trackOutput();       // Output Tracking

  const app = new MyApplication({ usernameService, uiService });

  app.render();
  assert.deepEqual(uiOutput.data, [ "Hello, my_username "]);
});

// Production code
class MyApplication {
  constructor({                                   // Parameterless Instantiation
    usernameService = UsernameService.create(),
    uiService = UiService.create(),
  } = {}) {
    this._usernameService = usernameService;
    this._uiService = uiService;
  }

  async render() {
    const username = await this._usernameService.getUsername();
    await uiService.render(`Hello, ${username}`);
  }
}

Now your application tests serve the same purpose as broad end-to-end tests: they document and test the externally-visible behavior of the application. They’re Narrow Tests, because they’re focused on the behavior of the Application class, and because they use Nullable dependencies, they don’t communicate with external systems. That makes them fast and reliable. But because they’re also Overlapping Sociable Tests, they provide the same safety net that broad tests do.

At this point, you have the beginnings of a walking skeleton: an application that works end-to-end, but is far from complete. You can evolve that skeleton to support more features. Choose some aspect of your code that’s obviously incomplete and test-drive a slightly better solution. Repeat forever.

// Application + read/write to Infrastructure + respond to UI events (JavaScript)

// Test code
it("renders user name", async () => {
  const usernameService = UsernameService.createNull({ username: "my_username" });  // Nullable with Configurable Responses
  const uiService = UiService.createNull();     // Nullable
  const uiOutput = uiService.trackOutput();     // Output Tracking

  const app = new MyApplication({ usernameService, uiService });
  await app.startAsync();

  uiService.simulateRequest("greeting");        // Behavior Simulation
  assert.deepEqual(uiOutput.data, [ "Hello, my_username" ]);
});

// Production code
class MyApplication {
  constructor({                                 // Parameterless Instantiation
    usernameService = UsernameService.create(),
    uiService = UiService.create(),
  } = {}) {
    this._usernameService = usernameService;
    this._uiService = uiService;
  }

  async startAsync() {
    this._uiService.on("greeting", () => {
      const username = await this._usernameService.getUsername();
      await uiService.render(`Hello, ${username}`);
    });
  }
}

At some point, probably fairly early, your Application layer class will start feeling messy. When it does, look for a concept that can be factored into its own class. This forms the seed of your Logic layer. As your application continues to grow, continue refactoring so that class collaborations are easy to understand and responsibilities are clearly defined.

When working with existing code, use the Legacy Code Patterns instead.

Contents

Logic Patterns

Logic code is pure computation. To qualify, code can’t involve external systems or state. That means it can’t talk to a database, communicate across a network, touch the file system, read the date and time, look at environment variables, or use most random number generators. It can’t depend on any code that does these things, either.

Pure computation is easy to test. The following patterns make it even easier.

Easily-Visible Behavior

Logic computation can only be tested by State-Based Tests if the results of the computation are visible to tests. Therefore:

Prefer pure functions where possible. Pure functions’ return values are determined only by their input parameters.

// JavaScript
function add(a, b) {
  return a + b;
}

When pure functions aren’t possible, prefer immutable objects. The state of immutable objects is determined when the object is constructed, and never changes afterwards.

// JavaScript
class Value {
  constructor(initialValue) {
    this._value = initialValue;
  }

  plus(addend) {
    return new Value(this._value + addend);
  }
}

For methods that change object state, provide a way for the change in state to be observed, either with a getter method or an event.

// JavaScript
class RunningTotal {
  constructor(initialValue) {
    this._total = initialValue;
  }

  add(addend) {
    this._total += addend;
  }

  getTotal() {
    return this._total;
  }
}

In all cases, avoid writing code that explicitly depends on (or changes) the state of dependencies more than one level deep. That makes test setup difficult, and it’s a sign of poor design anyway. Instead, design dependencies so they completely encapsulate their next-level-down dependencies.

Contents

Testable Libraries

Third-party code doesn’t always have Easily-Visible Behavior. It also tends to introduce breaking API changes with new releases, or simply stop being maintained. Therefore:

Wrap third-party code in code you control. Ensure your application’s use of the third-party code is mediated through your wrapper. Write your wrapper’s API to match the needs of your application, not the third-party code, and add methods as needed to provide Easily-Visible Behavior. (This will typically involve writing getter methods to expose deeply-buried state.) When the third-party code introduces a breaking change, or needs to be replaced, modify the wrapper so no other code is affected.

Frameworks and libraries with sprawling APIs are more difficult to wrap, so prefer libraries that have a narrowly-defined purpose and a simple API.

If the third-party code interfaces with an external system or state, use an Infrastructure Wrapper instead.

Contents

Collaborator-Based Isolation

Overlapping Sociable Tests ensure your tests will fail if your code’s behavior changes, no matter how far down the dependency chain those changes may be. On the one hand, this is nice, because you’ll learn when you accidentally break something. On the other hand, this could make feature changes terribly expensive. We don’t want a change in the formatting of addresses to break hundreds of unrelated reports’ tests. Therefore:

When a dependency’s behavior isn’t relevant to the code under test, use the dependency to help define test expectations. For example, if you’re testing a report that includes an address in its header, don’t hardcode “123 Main St.” as your expectation. Instead, ask the address how it would render itself, and use that as part of your test expectation.

Be careful not to write tests that are a copy of the code under test. Collaborator-Based Isolation is for writing Narrow Tests that ignore irrelevant details. For example, in the following code, the test is checking the special case of a report with a single address, not the behavior of address rendering. Address rendering is expected to have its own Narrow Tests.

// JavaScript

// Example test
it("includes the address in the header when reporting on one address", () => {
  // Instantiate the unit under test and its dependency
  const address = Address.createTestInstance();                 // Parameterless Instantiation
  const report = new InventoryReport(Inventory.create(), [ address ]);

  // Define the expected result using the dependency
  const expected = "Inventory Report for " + address.renderAsOneLine();

  // Run the production code and make the assertion
  assert.equal(report.renderHeader(), expected);
});

// Example production code
class InventoryReport {
  constructor(inventory, addresses) {
    this._inventory = inventory;
    this._addresses = addresses;
  }

  renderHeader() {
    let result = "Inventory Report";
    if (this._addresses.length === 1) {
      result += " for " + this._address[0].renderAsOneLine();
    }
    return result;
  }
}

This provides the best of both worlds: Overlapping Sociable Tests ensure that your application is wired together correctly and Collaborator-Based Isolation allows you to change behavior without breaking a lot of tests. However, it also ties the tests more tightly to the production code’s implementation, so it should be used sparingly.

Contents

Infrastructure Patterns

Infrastructure code is for communicating with the outside world. Although it may contain some logic, that logic should be focused on making infrastructure easier to work with. Everything else belongs in Application or Logic code.

Infrastructure code is unreliable and difficult to test because of its dependencies on external systems and state. The following patterns work around those problems.

Infrastructure Wrappers

Infrastructure code is complicated to write, hard to test, and often difficult to understand. Therefore:

Isolate your Infrastructure code. For each external system—service, database, file system, or even environment variables—create one wrapper class that’s solely responsible for interfacing with that system. Design your wrappers to provide a crisp, clean view of the messy outside world, in whatever format is most useful to the rest of your code.

Avoid creating complex webs of dependencies. In some cases, high-level Infrastructure classes may depend on generic, low-level classes. For example, LoginClient might depend HttpClient. In other cases, high-level infrastructure classes might unify multiple low-level classes, such as a DataStore class that depends on a RelationalDb class and a NoSqlDb class. Other than these sorts of simple one-way dependency chains, design your Infrastructure classes to stand alone.

Test your Infrastructure Wrappers with Narrow Integration Tests and Paranoic Telemetry. Make them testable with the Nullability Patterns.

Infrastructure Wrappers are also called “Gateways” or “Adapters.”

Contents

Narrow Integration Tests

Ultimately, Infrastructure code talks over a network, interacts with a file system, or involves some other communication with external systems or state. It’s easy to make a mistake. Therefore:

Test your external communication for real. For file system code, read and write real files. For databases, access a real database. Make sure that your test systems use the same configuration as your production environment. Otherwise your code will fail in production when it encounters subtle incompatibilities.

Run your narrow integration tests against test systems that are reserved exclusively for one machine’s use. It’s best if they run locally on your development machine, and are started and stopped by your tests or build script. Otherwise, you could experience unpredictable test failures when multiple people run the tests at the same time.

If you have multiple external systems that use the same technology, such as multiple web services, create a generic, low-level infrastructure wrapper for the underlying technology. Then create higher-level infrastructure wrappers for each system. The high-level wrappers don’t need Narrow Integration Tests. Instead, you can Fake It Once You Make It by delegating to the low-level wrapper.

For example, you could create a high-level LoginClient that depended on a low-level HttpClient. The LoginClient would Fake It Once You Make It and the HttpClient would be tested with Narrow Integration Tests.

// Example of narrow integration tests for low-level HTTP client wrapper (JavaScript + Node.js)
import { http } from "http";
import { HttpClient } from "./http_client";

const HOST = "localhost";
const PORT = 5001;

// Tests
describe("HTTP Client", () => {
  let server;

  before(async () => {
    server = new TestServer();
    await server.startAsync();
  });

  after(async () => {
    await server.stopAsync();
  });

  beforeEach(function() {
    server.reset();
  });

  it("performs request", async () => {
    await requestAsync({
      host: HOST,
      port: PORT,
      method: "POST",
      path: "/my/path",
      headers: { myRequestHeader: "myRequestValue" },
      body: "my request body"
    });

    assert.deepEqual(server.lastRequest, {
      method: "POST",
      path: "/my/path",
      headers: { myrequestheader: "myRequestValue" },
      body: "my request body"
    });
  });

  it("returns response", async () => {
    server.setResponse({
      status: 999,
      headers: { myResponseHeader: "myResponseValue" },
      body: "my response",
    });

    const response = await requestAsync();
    assert.deepEqual(response, {
      status: 999,
      headers: { myresponseheader: "myResponseValue" },
      body: "my response",
    });
  });

  async function requestAsync(options = {
    host: HOST,
    port: PORT,
    method: "GET",
    path: "/irrelevant/path",
  }) {
    const client = HttpClient.create();
    return client.requestAsync(options);
  }

});

// Localhost HTTP server
class TestServer {
  constructor() {
    this.reset();
  }

  reset() {
    this._lastRequest = null;
    this._nextResponse = {
      status: 500,
      headers: {},
      body: "response not specified",
    };
  }

  startAsync() {
    return new Promise((resolve, reject) => {
      this._server = http.createServer();
      this._server.once("listening", resolve);
      this._server.once("error", reject);
      this._server.on("request", this.#handleRequest.bind(this));
      this._server.listen(PORT);
    });
  }

  stopAsync() {
    return new Promise((resolve, reject) => {
      this._server.once("close", resolve);
      this._server.close();
    });
  }

  setResponse(response) {
    this._nextResponse = response;
  }

  get lastRequest() {
    return this._lastRequest;
  }

  // In JavaScript, methods that start with "#" are private.
  #handleRequest(request, response) {
    let body = "";
    request.on("data", (chunk) => {
      body += chunk;
    });
    request.on("end", () => {
      this.#storeRequest(request, body);
      this.#sendResponse(response);
    });
  }

  #storeRequest(request, body) {
    const headers = { ...request.headers };
    delete headers.connection;
    delete headers["content-length"];
    delete headers.host;

    this._lastRequest = {
      method: request.method,
      path: request.url,
      headers,
      body,
    };
  }

  #sendResponse(response) {
    response.statusCode = this._nextResponse.status;
    Object.entries(this._nextResponse.headers).forEach(([key, value]) => {
      response.setHeader(key, value);
    });

    response.end(this._nextResponse.body);
  }
}

Ensure your code works in production with Paranoic Telemetry.

Contents

Paranoic Telemetry

External systems are unreliable. The only thing that’s certain is their eventual failure. File systems lose data and become unwritable. Services return error codes, suddenly change their specifications, and refuse to terminate connections. Therefore:

Assume they really are out to get you, and instrument your code accordingly. Expect that everything will break eventually. Test that every failure case either logs an error and sends an alert, or throws an exception that ultimately logs an error and sends an alert. Remember to test your code’s ability to handle requests that hang, too.

All these failure cases are expensive to support and maintain. Whenever possible, use Testable Libraries rather than external services.

An alternative to Paranoic Telemetry is Contract Tests, but they’re not paranoid enough to catch changes that happen between test runs.

Contents

Nullability Patterns

Sociable Tests run real code. That’s good for catching errors, but if the dependency chain includes external systems or state, they become hard to manage. The following patterns allow you to “turn off” external dependencies while retaining the benefits of sociable and state-based testing.

Nullables

Narrow Integration Tests are slow and difficult to set up. Although they’re useful for ensuring that low-level Infrastructure Wrappers work in practice, they’re overkill for code that depends on those wrappers. Therefore:

Program code that includes infrastructure in its dependency chain to have a createNull() factory method. The factory should create a “Nulled” instance that disables all external communication, but behaves normally in every other respect.2 Make sure it supports Parameterless Instantiation.

2Nullables were originally inspired by the Null Object pattern, but have evolved to be completely different.

For example, calling LoginClient.createNull().getUserInfo(...) should return a default response without actually talking to the third-party login service.

Nullables are production code and should be tested accordingly. Although Nulled instances are often used by tests, they’re also useful whenever you want the ability to “turn off” behavior in your application. For example, you could use Nullables implement a “dry run” option in a command-line application.

// Example of using Nullables to implement "dry run" option (JavaScript + Node.js)
async initializeGitWriter(config) {
  if (config.dryRun) {
    return GitWriter.createNull();
  }
  else {
    return GitWriter.create();
  }
}

As another example, you can use Nullables in a web server to cache popular URLs when the server starts up:

// Example of using Nullables to implement cache warming (JavaScript + Node.js)
async warmCacheAsync(popularUrls, log) {
  for await (const url of popularUrls) {
    await this.routeAsync(HttpRequest.createNull({ url, log });
  }
}

Implement low-level infrastructure wrappers with Embedded Stubs. For everything else, Fake It Once You Make It. To make existing code Nullable, see the Legacy Code Patterns.

If your Nullable reads data from external systems or state, or any of its dependencies do, implement Configurable Responses. If it or its dependencies write data, implement Output Tracking. If they respond to events, implement Behavior Simulation.

Contents

Embedded Stub

Nullables need to disable access to external systems and state while running everything else normally. The obvious approach is to surround any code that accesses the external system with an “if” statement, but that’s a recipe for spaghetti. Therefore:

When making code Nullable, don’t change your code. Instead, stub out the third-party code that accesses external systems.

In your stub, implement the bare minimum needed to make your code run. Ensure you don’t overbuild the stub by test-driving it through your code’s public interface. Put the stub in the same file as the rest of your code so it’s easy to remember and update when your code changes.

Write a stub of the third-party code, not your code, so your Sociable Tests test how your code will really work in production. Be careful to have your stub mimic the behavior of the third-party code exactly. To help you do so, write Narrow Integration Tests that document the behavior of the real code, paying particular attention to edge cases such as error handling and asynchronous code. Then write additional tests of the Nulled instance that will fail if your stub doesn’t have the same behavior.

Here’s a simple example of stubbing out JavaScript’s Math library:

// An Infrastructure Wrapper for a random die roller. (JavaScript)

// Infrastructure Wrapper
export class DieRoller {

  // Normal factory
  static create() {
    return new DieRoller(Math);    // "Math" is a built-in JavaScript global
  }

  // Null factory
  static createNull() {
    return new DieRoller(new StubbedMath());
  }

  // Shared initialization code
  constructor(math) {
    this._math = math;
  }

  // Infrastructure wrapper implementation.
  // This is the same code you would write without a stub.
  roll(amount) {
    const randomNumber = this._math.random();
    return Math.trunc((randomNumber * 6) + 1);    // There's no need to stub Math.trunc, so we use the real Math library here
  }
};

// Embedded Stub. Note that we only stub the function we use.
class StubbedMath {
  random() {
    return 0;
  }
}

Here’s a more complicated example. It stubs out Node.js’s http library:

// An infrastructure wrapper for a generic HTTP client. (JavaScript + Node.js)

import { http } from "node:http";
import { EventEmitter } from "node:events";

export class HttpClient {

  // Normal factory
  static create() {
    return new HttpClient(http);
  }

  // Null factory
  static createNull() {
    return new HttpClient(new StubbedHttp());
  }

  // Shared initialization code
  constructor(http) {
    this._http = http;
  }

  // Infrastructure wrapper implementation.
  // This is the same code you would write without a stub.
  async requestAsync({ host, port, method, path, headers = {}, body = "" }) {
    if (method === "GET" && body !== "") throw new Error("Don't include body with GET requests; Node won't send it");

    const httpOptions = { host, port, method, path, headers };
    const request = this.#sendRequest(httpOptions, body);

    await new Promise((resolve, reject) => {
      this.#handleResponse(request, resolve);
      this.#handleError(request, reject);
    });
  }

  // In JavaScript, methods that start with "#" are private.
  #sendRequest(httpOptions, body) {
    const request = this._http.request(httpOptions);
    request.end(body);
    return request;
  }

  #handleResponse(request, resolve) {
    request.once("response", (response) => {
      let body = "";
      response.on("data", (chunk) => {
        body += chunk;
      });
      response.on("end", () => {
        resolve({
          status: response.statusCode,
          headers,
          body,
        });
      });
    });
  }

  #handleError(request, reject) {
    request.once("error", reject);
  }
};
  
// Embedded Stub. Note that it’s built exactly for the needs of the infrastructure code, nothing more.
class StubbedHttp {
  request() {
    return new StubbedRequest();
  }
}

class StubbedRequest extends EventEmitter {
  end() {
    // setImmediate() is used to make the emit() call asynchronous, duplicating the behavior of real code
    setImmediate(() => this.emit("response", new StubbedResponse()));
  }
}

class StubbedResponse extends EventEmitter {
  constructor() {
    super();
    setImmediate(() => {
      this.emit("data", "Nulled HttpClient response");
      this.emit("end");
    });
  }
  get statusCode() {
    return 200;
  }
  get headers() {
    return {};
  }
}

Configure the embedded stub’s return values with Configurable Responses. If your language requires it, as with Java or C#, create a Thin Wrapper.

Nullables are production code, and despite appearances, the Embedded Stub is too. It must be tested accordingly. If you don’t like the idea of having stubs in your production code, you can put the Embedded Stub in a separate test-only file instead. However, this will make dependency management more complicated, and it will prevent you from using Nulled instances in production, which can be useful.

Contents

Thin Wrapper

Languages such as Java and C# will require your Embedded Stub to share an interface with the real dependency. Often, there won’t be an interface you can use, or it will be bigger than you need. Therefore:

Create a custom interface for your third-party dependency. Match the signature of the dependency exactly, but only include the methods your production code actually uses. Provide two implementations of the interface: a real version that only forwards calls to the third-party dependency, and an Embedded Stub.

// A simple Infrastructure Wrapper for a random die roller. (Java)
// It has an embedded stub for Java's standard “Random” library.
// Based on an example created with Ted M. Young in his Yacht codebase.
public class DieRoller {
  private final RandomWrapper random;

  // Normal factory
  public static DieRoller create() {
    return new DieRoller(new RealRandom());
  }

  // Null factory
  public static DieRoller createNull() {
    return new DieRoller(new StubbedRandom());
  }

  // Private constructor with shared initialization code
  private DieRoller(RandomWrapper random) {
    this.random = random;
  }

  // Infrastructure wrapper implementation.
  // This is the same code you would write without a stub.
  public int roll() {
    return random.nextInt(6) + 1;
  }

  // Interface for Thin Wrapper. Note that we match the real code's interface exactly,
  // and we only include the function we use.
  private interface RandomWrapper {
    int nextInt(int bound);
  }

  // Real implementation of Thin Wrapper
  private static class RealRandom implements RandomWrapper {
    private final Random random = new Random();

    @Override
    public int nextInt(int bound) {
      return this.random.nextInt(bound);
    }
  }

  // Embedded Stub implementation of Thin Wrapper
  private static class StubbedRandom implements RandomWrapper {
    @Override
    public int nextInt(int bound) {
      return 0;
    }
  }
}

If the third-party code returns custom types, you’ll need to wrap those return types as well. Remember to match the third-party code’s signatures exactly.

// Infrastructure Wrapper for an HTTP request. (Java + Spring Boot's RestTemplate)
// Based on an example created with Ted M. Young in his Yacht codebase.
public class AverageScoreFetcher {
  private static final String YACHT_AVERAGE_API_URI = "http://localhost:8080/api/averages?scoreCategory={scoreCategory}";

  private final RestTemplateWrapper restTemplate;

  // Normal factory
  public static AverageScoreFetcher create() {
    return new AverageScoreFetcher(new RealRestTemplate());
  }

  // Null factory
  public static AverageScoreFetcher createNull() {
    return new AverageScoreFetcher(new StubbedRestTemplate());
  }

  // Private constructor with shared initialization code
  private AverageScoreFetcher(RestTemplateWrapper restTemplate) {
    this.restTemplate = restTemplate;
  }

  // Infrastructure wrapper implementation
  public double averageFor(ScoreCategory scoreCategory) {
    ResponseEntityWrapper<CategoryAverage> entity = restTemplate.getForEntity(
      YACHT_AVERAGE_API_URI,
      CategoryAverage.class,
      scoreCategory.toString()
    );
    return entity.getBody().getAverage();
  }

  // Interfaces for Thin Wrapper. Note that we only include the functions we use.
  interface RestTemplateWrapper {
    <T> ResponseEntityWrapper<T> getForEntity(String url, Class<T> responseType, Object... uriVariables);
  }

  interface ResponseEntityWrapper<T> {
    T getBody();
  }

  // Real implementations of Thin Wrapper
  private static class RealRestTemplate implements RestTemplateWrapper {
    private final RestTemplate restTemplate = new RestTemplate();

    public <T> ResponseEntityWrapper<T> getForEntity(String url, Class<T> responseType, Object... uriVariables) {
      return new RealResponseEntity<T>(restTemplate.getForEntity(url, responseType, uriVariables));
    }
  }

  private static class RealResponseEntity<T> implements ResponseEntityWrapper<T> {
    private ResponseEntity<T> entity;

    RealResponseEntity(ResponseEntity<T> entity) {
      this.entity = entity;
    }

    public T getBody() {
      return this.entity.getBody();
    }
  }

  // Stubbed implementations of Thin Wrapper
  private static class StubbedRestTemplate implements RestTemplateWrapper {
    @Override
    public <T> ResponseEntityWrapper<T> getForEntity(String url, Class<T> responseType, Object... uriVariables) {
      return new StubbedResponseEntity<>();
    }
  }

  private static class StubbedResponseEntity<T> implements ResponseEntityWrapper<T> {
    @Override
    public T getBody() {
      return (T) new CategoryAverage("Nulled AverageScoreFetcher category", 42.0);
    }
  }

}

Contents

Configurable Responses

State-based tests of code with infrastructure dependencies needs to set up the infrastructure’s state, but setting up external systems is complicated and slow. Therefore:

Program the infrastructure dependencies to return the response you need. Make them Nullable and program the createNull() factory to take your desired response as an optional parameter. Define the responses from the perspective of the dependency’s externally-visible behavior, not its implementation.

If the Nullable dependency has multiple types of responses that can be configured, give each one its own configuration parameter. Use named and optional parameters so tests only need to configure the data they care about. If your language doesn’t support optional parameters, use an Options object, as shown in the Signature Shielding pattern.

For example, the following test is for a LoginController that depends on a Nullable LoginClient. Although the LoginClient is used to make HTTP requests, its Configurable Responses aren’t about HTTP. Instead, they’re about the logged-in user’s email address and verification status, which is the behavior LoginController and its tests care about.

// Example of configuring multiple types of responses. (JavaScript)
it("logs successful login", async () => {
  // Configure login client dependency
  const loginClient = LoginClient.createNull(
    email: "my_authenticated_email",  // configure email address
    emailVerified: true,              // configure whether email is verified
  );

  // Run production code
  const { logOutput } = await performLogin({ loginClient }));  // Signature Shielding

  // Check results
  assert.deepEqual(logOutput.data, [ "Login: my_authenticated_email (verified)" ]);   // Output Tracking
});

If it makes sense for your class respond differently each time it’s called, configure the responses with an array or list. It’s often helpful to support two data types: a list of values, that results in a different response each time, and causes an exception when it runs out; and a single value, that returns the same response every time, and never runs out.

For example, the following test configures a Nullable DieRoller with a set of expected die rolls:

// Example of a single type of response with multiple return values. (JavaScript)
// Inspired by an example created with Ted M. Young in his Yacht codebase.
it("rolls a hand of dice", async () => {
  // Configure die rolls
  const dieRoller = DieRoller.createNull([ 1, 2, 3, 4, 5 ]);

  // Run production code
  const game = new Game(dieRoller);
  const hand = game.roll();

  // Check results
  assert.deepEqual(hand, HandOfDice.create(1, 2, 3, 4, 5));
});

If your Nullable uses an Embedded Stub, implement the responses in the stub. Otherwise, Fake It Once You Make It. Either way, decompose the responses down to the next level.

The following example uses an Embedded Stub to make a random die roller. It’s configured at the level its callers care about: dice roll results. In the Embedded Stub, those configured values are decomposed to the level DieRoller operates at: random floating point numbers from between zero and one. For example, a configured roll of 6 is turned into the floating point number 0.83333.

// Example of implementing Configurable Responses in an Embedded Stub. (JavaScript)

// Infrastructure Wrapper
export class DieRoller {

  static create() {
    return new DieRoller(Math);    // "Math" is a built-in JavaScript global
  }

  // Null factory with Configurable Responses
  // If a number is provided, it always returns that number.
  // If an array is provided, it returns exactly the numbers provided, then throws an error when it runs out.
  // If nothing is provided, it defaults to returning ones.
  static createNull(rolls = 1) {                    // set default to 1
    return new DieRoller(new StubbedMath(rolls));   // pass configuration to Embedded Stub
  }

  constructor(math) {
    this._math = math;
  }

  roll(amount) {
    const randomNumber = this._math.random();
    return Math.trunc((randomNumber * 6) + 1);      // There's no need to stub Math.trunc, so we use the real Math global
  }
};

// Embedded Stub with Configurable Responses
class StubbedMath {

  constructor(rolls) {
    // Store configured responses
    this._rolls = rolls;
  }

  random() {
    // Use configured responses
    const roll = this.#nextRoll();    // Get configuration to use
    return (roll - 1) / 6;            // Convert to float to match behavior of real Math.random()
  }

  // Retrieve configured response
  #nextRoll() {
    if (Array.isArray(this._rolls)) {
      // Configuration is an array, so return the next roll in the array
      const roll = this._rolls.shift();
      if (roll === undefined) throw new Error("No more rolls configured in nulled DieRoller");
      return roll;
    }
    else {
      // Configuration is a number, so always return that number
      return this._rolls;
    }
  }
}

The above code can be simplified by factoring #nextRoll() into a generic helper class. The result looks like this:

// Example of implementing an embedded stub with a ConfigurableResponses helper class (JavaScript)
class StubbedMath {
  constructor(rolls) {
    this._rolls = ConfigurableResponses.create(rolls);
  }

  random() {
    return (this._rolls.next() - 1) / 6;
  }
}

This is a JavaScript implementation of ConfigurableResponses you can use in your own code:

// Copyright 2023 Titanium I.T. LLC. MIT License.
export default class ConfigurableResponses {

  // Create a list of responses (by providing an array),
  // or a single repeating response (by providing any other type).
  // 'Name' is optional and used in error messages.
  static create(responses, name) {
    return new ConfigurableResponses(responses, name);
  }

  // Convert all properties in an object into ConfigurableResponse instances.
  // For example, { a: 1 } becomes { a: ConfigurableResponses.create(1) }.
  // 'Name' is optional and used in error messages.
  static mapObject(responseObject, name) {
    const entries = Object.entries(responseObject);
    const translatedEntries = entries.map(([ key, value ]) => {
      const translatedName = name === undefined ? undefined : `${name}: ${key}`;
      return [ key, ConfigurableResponses.create(value, translatedName )];
    });
    return Object.fromEntries(translatedEntries);
  }

  constructor(responses, name) {
    this._description = name === undefined ? "" : ` in ${name}` ;
    this._responses = Array.isArray(responses)
      ? [ ...responses ]
      : responses;
  }

  // Get next configured response. Throws an error when configured with a list
  // of responses and no more responses remain.
  next() {
    const response = Array.isArray(this._responses)
      ? this._responses.shift()
      : this._responses;
    if (response === undefined) throw new Error(`No more responses configured${this._description}`);

    return response;
  }

};

To test code with dependencies that write to infrastructure, use Output Tracking. To test code with dependencies that respond to events, use Behavior Simulation.

Contents

Output Tracking

State-based tests of code with dependencies that write to external systems need to check whether the writes were performed, but setting up external systems is complicated and slow. Therefore:

Program each dependency with a tested, production-grade trackXxx() method that tracks the otherwise-invisible writes. Have it do so regardless of whether the object is Nulled or not.

Track the writes in terms of the behavior your callers care about, not the underlying implementation of your code. For example, a structured logger might write strings to stdout, but its callers care about the structured data that’s being written. Its Output Tracking would track the data, not the string.

One way to implement Output Tracking is to have trackXxx() return an OutputTracker that listens for events emitted by your production code. The following example shows how this works, including implementations of OutputTracker in JavaScript and Java that you can use in your own code. It starts with a test of LoginPage, which writes to a structured Log when the user logs in.

// Example of using Output Tracking (JavaScript)

// Application layer test
it("writes to log when user logs in", async () => {
  // Set up a log and track its output
  const log = Log.createNull();
  const logOutput = log.trackOutput();
  
  // Instantiate the code under test
  const loginPage = new LoginPage(log);

  // Run the code
  const formData = // code to set up "my_email" login here
  await loginPage.postAsync(formData);

  // Check the log output
  assert.deepEqual(logOutput.data, [{
    alert: "info",
    message: "User login",
    email: "my_email",
  }]);
});

// Application layer code
class LoginPage {
  constructor(log) {
    this._log = log;
  }

  async postAsync(formData) {
    const email = // code to parse formData and verify login goes here

    // Code under test
    this.log.info({
      message: "User login",
      email,
    });
  }
}

// High-level "Log" infrastructure wrapper used by the code under test 
import { Clock } from "clock";                    // Low-level infrastructure wrappers
import { Stdout } from "stdout";
import { EventEmitter } from "node:events";       // Standard Node.js event library
import { OutputTracker } from "output_tracker";   // Output tracking library

const OUTPUT_EVENT = "output";                    // Event to emit when output occurs

class Log {
  static create() {
    return new Log(Clock.create(), Stdout.create());
  }

  static createNull({
    clock = Clock.createNull(),                   // Fake It Once You Make It
    stdout = Stdout.createNull(),
  } = {}) {
    return new Log(clock, stdout);
  }

  constructor(clock, stdout) {
    this._clock = clock;
    this._stdout = stdout;

    this._emitter = new EventEmitter();           // Event emitter used when output occurs
  }

  // Output tracker
  trackOutput() {
    return OutputTracker.create(this._emitter, OUTPUT_EVENT);
  }

  // The method called by the code under test
  info(data) {
    data.alert = "info";

    // Write the log
    const now = this._clock.formattedTimestamp();
    const dataJson = JSON.stringify(data);
    this._stdout.write(`${now} ${dataJson}`);
  
    // Emit the event. This is received by the OutputTracker.
    this._emitter.emit(OUTPUT_EVENT, data);
  }
}

At first glance, Output Tracking might look like the same thing as a spy (a type of test double), but there’s an important difference. Output Tracking records behavior, and spies record function calls. Output Trackers should write objects that represent the action that was performed, not just the function that was called to perform it. That way, when you refactor, you can change your functions without changing your Output Trackers or the tests that depend on them.

This is a JavaScript version of the OutputTracker class you can use in your own projects:

// Copyright 2020-2022 Titanium I.T. LLC. MIT License.
export default class OutputTracker {

  static create(emitter, event) {
    return new OutputTracker(emitter, event);
  }

  constructor(emitter, event) {
    this._emitter = emitter;
    this._event = event;
    this._data = [];

    this._trackerFn = (text) => this._data.push(text);
    this._emitter.on(this._event, this._trackerFn);
  }

  get data() {
    return this._data;
  }

  clear() {
    const result = [ ...this._data ];
    this._data.length = 0;
    return result;
  }

  stop() {
    this._emitter.off(this._event, this._trackerFn);
  }

}

Below, I’ve also included a Java version of the OutputTracker library that I created with Ted M. Young. Because Java doesn’t have a built-in event emitter, it’s used slightly differently. Here’s an example of using it in the Log infrastructure wrapper from the earlier example:

// Example of Output Tracking in Java
public class Log {
  // Instantiate the event emitter
  private final OutputListener<LogData> outputListener = new OutputListener<>();

  public static Log create...
  public static Log createNull...
  private Log...

  // Create the output tracker
  public OutputTracker<LogData> trackOutput() {
    return outputListener.createTracker();
  }

  public void info(Map<String, LogData> data) {
    // ...

    // Emit the event
    outputListener.track(data);
  }
}

This is the Java version of OutputTracker. Split it into two files.

---- OutputListener.java ----
// Copyright 2022 Titanium I.T. LLC and Ted M. Young. MIT License.
package com.jamesshore.output_tracker;

import java.util.ArrayList;
import java.util.List;

public class OutputListener<T> {
  private final List<OutputTracker<T>> listeners = new ArrayList<>();

  public void track(T data) {
    listeners.forEach(tracker -> tracker.add(data));
  }

  public OutputTracker<T> createTracker() {
    OutputTracker<T> tracker = new OutputTracker<>(this);
    listeners.add(tracker);
    return tracker;
  }

  void remove(OutputTracker<T> outputTracker) {
    listeners.remove(outputTracker);
  }
}

---- OutputTracker.java ----
// Copyright 2022 Titanium I.T. LLC and Ted M. Young. MIT License.
package com.jamesshore.output_tracker;

import java.util.ArrayList;
import java.util.List;

public class OutputTracker<T> {
  private final List<T> output = new ArrayList<>();
  private final OutputListener<T> outputListener;

  public OutputTracker(OutputListener<T> outputListener) {
    this.outputListener = outputListener;
  }

  void add(T data) {
    output.add(data);
  }

  public List<T> data() {
    return List.copyOf(output);
  }

  public List<T> clear() {
    List<T> data = this.data();
    output.clear();
    return data;
  }

  public void stop() {
    outputListener.remove(this);
  }
}

To test code with dependencies that read from infrastructure, use Configurable Responses. To test code with dependencies that emit events, use Behavior Simulation.

Contents

Behavior Simulation

Some external systems will push data to you rather than waiting for you to ask for it. Code that depends on those systems need a way to test what happens when their infrastructure dependencies generate those events, but setting up infrastructure to send events is complicated and slow. Therefore:

Add methods to your dependencies that simulate receiving an event from an external system. Share as much code as possible with the code that handles real external events. Write it as tested, production-grade code.

For example, the following Application layer code uses Socket.IO (a WebSocket library) to copy messages from one client to all others. The test checks that it does so when it receives a message.

// Example of using Behavior Simulation (JavaScript)

// Application layer test
it("broadcasts messages from one client to all others", async () => {
  // Set up test data
  const clientId = "my_client_id";
  const message = new TestClientMessage("my_message");

  // Set up the infrastructure wrapper and the code under test
  const network = WebSocketServer.createNull();   // Create the infrastructure wrapper
  const sentMessages = network.trackMessages();   // Track messages sent by infrastructure wrapper (Output Tracking)
  const server = new MessageServer(network);      // Instantiate the application code under test
  await server.startAsync();                      // Start listening for messages (Zero-Impact Instantiation)

  // Simulate a client connecting
  network.simulateConnection(clientId);

  // Simulate the client sending a message
  network.simulateMessage(clientId, message);

  // Check that the message was broadcast (Output Tracking)
  assert.deepEqual(sentMessages.data(), [{
    type: "broadcast",
    excludedClient: clientId,
    message
  }]);
});

// Application layer code
class MessageServer {
  constructor(webSocketServer) {
    this._webSocketServer = webSocketServer;
  }

  async startAsync() {
    // Code under test
    this._webSocketServer.onMessage((clientId, message) => {
      this._webSocketServer.broadcastToAllClientsExcept(clientId, message);
    });

    await this._webSocketServer.startAsync();
  }

  //...
}

// Infrastructure wrapper used by the code under test 
import { Server } from "socket.io";               // Socket.IO
import { EventEmitter } from "node:events";       // Standard Node.js event library
import { OutputTracker } from "output_tracker";   // Output tracking library

const CLIENT_MESSAGE_EVENT = "client_message";    // Event constants
const SERVER_MESSAGE_EVENT = "server_message";

class WebSocketServer {
  static create(port) {
    return new WebSocketServer(io, port);
  }

  static createNull() {
    return new WebSocketServer(StubbedServer, 42);
  }

  constructor(server, port) {
    this._server = server;
    this._port = port;
    this._emitter = new EventEmitter();
    this._connectedSockets = {};
  }

  // Real Socket.IO event handler
  async startAsync() {
    this._io.on("connection", (socket) => {
      this.#handleConnection(socket);
      socket.onAny((event, ...args) => {
        const message = this.#deserializeMessage(event, args);
        this.#handleMessage(socket.id, message);
      });
      socket.on("disconnect", () => {
        this.#handleDisconnection(socket.id));
      });
    });
  }

  // Behavior Simulation
  simulateConnection(clientId) {
    this.#handleConnection(new StubbedSocket(clientId));
  }

  simulateMessage(clientId, message) {
    this.#handleMessage(clientId, message);
  }

  simulateDisconnection(clientId) {
    this.#handleDisconnection(clientId);
  }

  // Shared by event handler and behavior simulation
  #handleConnection(socket) {
    this._connectedSockets[socket.id] = socket;
  )

  #handleMessage(clientId, message) {
    this._emitter.emit(CLIENT_MESSAGE_EVENT, { clientId, message });
  }

  #handleDisconnection(clientId) {
    delete this._connectedSockets(clientId);
  }

  // Methods called by the code under test
  onMessage(fn) {
    this._emitter.on(CLIENT_MESSAGE_EVENT, ({ clientId, message }) => {
      fn(clientId, message));
    });
  }

  broadcastToAllClientsExcept(clientId, message) {
    const socket = this._connectedSockets[clientId];
    socket.broadcast.emit(message.name, message.payload);

    this._emitter.emit(SERVER_MESSAGE_EVENT, {    // Output Tracking
      type: "broadcast",
      excludedClient: clientId,
      message,
    });
  });

  // Output Tracking
  trackMessages() {
    return OutputTracker.create(this._emitter, SERVER_MESSAGE_EVENT);
  }

  //...
}

To test code with dependencies that read from infrastructure, use Configurable Responses. To test code with dependencies that write to infrastructure, use Output Tracking.

Contents

Fake It Once You Make It

Narrow Integration Tests are slow and difficult to set up. Similarly, Embedded Stubs can be difficult to create. Although they’re needed for low-level infrastructure wrappers, they’re overkill for code that doesn’t have direct dependencies on third-party infrastructure code. Therefore:

Program your dependencies to be Nullable. (If your code has a third-party infrastructure dependency, extract it into a low-level Infrastructure Wrapper.) Use Nulled instances in your tests and Configurable Responses.

In your tests, create Nulled instances of your code under test’s dependencies. Inject the Nulled instances into the code under test, then use Configurable Responses, Output Tracking, and Behavior Simulation to write your tests. Because Nullables are real production code, your tests will still test your code’s real behavior. They’ll be Sociable Tests that execute all the way to the edges of your system.

For example, the following code tests LoginClient, which depends on a low-level HttpClient. The LoginClient tests use a Nulled version of HttpClient.

// Example of a using Fake It Once You Make It in a test (JavaScript)

it("performs network request", async () => {
  // Set up the low-level HTTP client (Configurable Response)
  const httpClient = HttpClient.createNull({
    "/oauth/token": [{                        // The Auth0 endpoint our code will call.
      status: VALID_STATUS,                   // Status, headers, and body Auth0 could really return.
      headers: VALID_HEADERS,
      body: VALID_BODY,
    }],
  });

  // Track requests made with the HTTP client (Output Tracking)
  const httpRequests = httpClient.trackRequests();

  // Instantiate the code under test, injecting the Nulled httpClient
  const client = new LoginClient(httpClient, "my_client_id", "my_client_secret", "my_management_api_token");

  // Run the code under test
  await client.validateLoginAsync("my_login_code", "my_callback_url");

  // Assert that the correct HTTP request was made (Output Tracking)
  assert.deepEqual(httpRequests.data, [{
    host: HOST,
    port: PORT,
    method: "post",
    path: "/oauth/token",
    headers: {
      authorization: "Bearer my_management_api_token",
      "content-type": "application/json; charset=utf-8",
    },
    body: JSON.stringify({
      client_id: "my_client_id",
      client_secret: "my_client_secret",
      code: "my_login_code",
      redirect_uri: "my_callback_url",
      grant_type: "authorization_code"
    }),
  }]);
});

If your code has Configurable Responses, program your createNull() method to delegate to Nulled instances of its dependencies. Decompose your code’s Configurable Responses down to the lower-level responses needed by its dependencies.

For example, LoginClient uses HttpClient to make requests of the Auth0 service. The LoginClient’s Configurable Responses involve the user’s email address and validation status. In LoginClient.createNull(), those responses are decomposed into the actual HTTP responses Auth0 would return.

// Example of a using Fake It Once You Make It to make a class Nullable (JavaScript)
class LoginClient {

  // Normal factory
  static create(clientId, clientSecret, managementApiToken) {
    const httpClient = HttpClient.create();
    return new LoginClient(httpClient, clientId, clientSecret, managementApiToken);
  }

  // Null factory with Configurable Responses
  static createNull({
    validateLogin = {
      email = "null_login_email",   // The email address associated with the login
      emailVerified = true,         // True if the email address has been verified
      forbidden = undefined,        // Set to a string to simulate an Auth0 "forbidden" response
    },
    clientId = "null_client_id",
    clientSecret = "null_client_secret",
    managementApiToken = "null_management_api_token",
  } = {}) {
    // Convert LoginClient's Configurable Response into the response Auth0 would actually return
    const auth0Response = nullValidateLoginResponse(validateLogin);

    // Create a Nulled HttpClient that's configured to return the Auth0 response
    const httpClient = HttpClient.createNull({
      [VALIDATE_LOGIN_ENDPOINT]: auth0Response;
    });

    // Instantiate the LoginClient
    return new LoginClient(httpClient, clientId, clientSecret, managementApiToken);
  }

  // Shared initialization
  constructor(httpClient, clientId, clientSecret, managementApiToken) {
    this._httpClient = httpClient;
    this._clientId = clientId;
    this._clientSecret = clientSecret;
    this._authHeader = {
      authorization: `Bearer ${managementApiToken}`,
    };
  }

  // Shared production code
  async validateLoginAsync(code, callbackUrl) {
    const response = await this._httpClient.postAsync(
      VALIDATE_LOGIN_ENDPOINT,              // URL
      this._authHeader,                     // headers
      {                                     // body
        client_id: this._clientId,
        client_secret: this._clientSecret,
        code,
        redirect_uri: callbackUrl,
        grant_type: "authorization_code"
      }
    );

    const decodedToken = /* code to validate and decode response here */

    return {
      email: decodedToken.email,
      emailVerified: decodedToken.email_verified
    };
  }

  // Configurable Responses translation code
  // This function decomposes the responses passed to LoginClient.createNull() down
  // into responses for HttpClient.createNull(). HttpClient.createNull() is configured
  // with a status, optional headers, and a body.
  function nullValidateLoginResponse({ email, emailVerified, forbidden }) {
    // If the "forbidden" response is set, return a 403 (Forbidden) response.
    if (forbidden) return { status: STATUS.FORBIDDEN_403, body: forbidden };

    // Otherwise, create a JSON Web Token, because that's what Auth0 returns
    const response = { email, email_verified: emailVerified };
    const id_token = jwt.sign(response, "irrelevant_secret", { noTimestamp: true });

    // Return the JWT in a 200 (OK) response
    return {
      status: STATUS.OK_200,
      body: JSON.stringify({ id_token }),
    };
  }
}

Implement Output Tracking and Behavior Simulation normally, without regard to whether the dependencies are Nulled or not.

To make your dependencies Nullable, either Descend the Ladder or Climb the Ladder.

Contents

Legacy Code Patterns

If you’d like to convert your existing code and tests to use Nullables, the patterns in this section will help you do so.

Work incrementally. You can mix Nullables with your current approach in the same codebase, and even in the same test, so there’s no need to convert everything at once. Similarly, focus your efforts on code where testing with Nullables will have noticeable benefit. Don’t waste time converting code that’s already easy to maintain, regardless of how it’s tested.

Descend the Ladder

Complex codebases have a lot of dependencies, and it isn’t feasible to improve all the tests at once. Instead, you’ll need to make progress incrementally. Therefore:

When converting a module or class, convert the code and its direct dependencies, but nothing more. Work your way down through the rest of the dependency tree gradually, when time allows.

If the code you’re converting is a low-level Infrastructure Wrapper with third-party dependencies, test it with Narrow Integration Tests, then make it Nullable with an Embedded Stub. Otherwise, apply one of the following options to each of its direct dependencies:

After you’ve updated the dependencies, Fake It Once You Make It. (If your code has a Throwaway Stub, replace it.) Replace Mocks with Nullables and add tests as needed.

When you’re done, the code you’re converting will be Nullable and tested. Its dependencies will be Nullable, but not tested. You can move on to other work. When you’re ready to convert another class or module, Descend the Ladder again. Over time, you’ll gradually convert the entire codebase.

For example, imagine you had the dependency chain RouterLoginControllerAuth0ClientHttpClient, where HttpClient is a low-level Infrastructure Wrapper. If you wanted to convert Router, you would follow these steps:

  1. Router’s direct dependency is LoginController, which has a mix of logic and infrastructure in its dependency chain. Make LoginController Nullable with a Throwaway Stub.

  2. Make Router Nullable with Fake It Once You Make It.

  3. Convert Router’s tests with Replace Mocks with Nullables.

Later, if you wanted to convert Auth0Client, you would follow these steps:

  1. Auth0Client’s direct dependency is HttpClient, which is a low-level Infrastructure Wrapper. Make HttpClient Nullable by introducing an Embedded Stub.

  2. Make Auth0Client Nullable with Fake It Once You Make It.

  3. Convert Auth0Client’s tests with Replace Mocks with Nullables.

When you wanted to convert LoginController, you would follow these steps:

  1. LoginController’s direct dependency is Auth0Client, which was previously converted, so it’s already Nullable.

  2. LoginController has a Throwaway Stub from when Router was converted. Now that Auth0Client is Nullable, replace the stub with Fake It Once You Make It.

  3. Convert LoginController’s tests with Replace Mocks with Nullables.

Finally, when you were ready to convert HttpClient, you would follow these steps:

  1. HttpClient is a low-level Infrastructure Wrapper, and it was made Nullable when Auth0Client was converted, so it only needs to be tested.

  2. Test HttpClient with Narrow Integration Tests.

Code that’s been converted can be refactored without breaking its tests. Once you’ve converted enough code, you can refactor it to use A-Frame Architecture or any other architecture you like.

Descend the Ladder is for code with large dependency trees. If the code you’re converting has a small dependency tree, Climb the Ladder instead.

Contents

Climb the Ladder

Descending the Ladder is a careful, methodical approach to improving existing code. However, it involves creating Throwaway Stubs, which is wasteful, and it takes a long time. Simple dependency trees don’t need so much care. Therefore:

Convert all of your code’s dependencies at once. Start by graphing out a dependency tree for the code you want to convert, ignoring third-party dependencies. Then convert each node from the bottom of the tree up. (A post-order depth-first traversal). Apply one of the following options to each node:

When you’re done, the entire dependency tree will be tested and Nullable. You can then refactor it toward A-Frame Architecture or any other architecture you like.

For example, imagine you had the dependency chain RouterLoginControllerAuth0ClientHttpClient, where HttpClient is a low-level Infrastructure Wrapper. If you wanted to convert Router, you would follow these steps:

  1. HttpClient is a low-level Infrastructure Wrapper. Make it Nullable by introducing an Embedded Stub.
  2. Test HttpClient with Narrow Integration Tests.
  3. Make Auth0Client Nullable with Fake It Once You Make It.
  4. Convert Auth0Client’s tests with Replace Mocks with Nullables.
  5. Make LoginController Nullable with Fake It Once You Make It.
  6. Convert LoginController’s tests with Replace Mocks with Nullables.
  7. Make Router Nullable with Fake It Once You Make It.
  8. Convert Router’s tests with Replace Mocks with Nullables.

Climb the Ladder works best when you have a small dependency tree. If you have a large dependency tree, Descend the Ladder instead.

Contents

Replace Mocks with Nullables

Existing code is often tested with mocks, spies, and other test doubles. Some of those tests will get in your way. They might be hard to understand and maintain, or they might make refactoring difficult. Therefore:

When an existing test gets in your way, use Nullables in place of the existing test doubles. Depending on the quality of the existing tests, it might be easiest to inline any setup blocks or helper methods prior to starting. Then apply the following options to each mock, spy, or other test double in the tests you want to convert:

  • Start by replacing the test double with a Nulled version of the real dependency.

  • If the test double is configured to return specific values, replace the configuration with Configurable Responses.

  • If the test double is configured to emit events, replace the configuration with Behavior Simulation.

  • If the test checks how a test double is called, replace its assertions with Output Tracking. Convert these test doubles last, after test doubles with only configuration have been replaced.

For example, here’s a controller for a web page. When the user posts to the page, it uses the rot13Client infrastructure wrapper to call a web service, then renders the result.

// Example web page controller (JavaScript + Node.js)

import { HomePageView } from "home_page_view";
import { Rot13Client } from "rot13_client";
import { HttpRequest } from "http_request";
import { WwwConfig } from "www_config";

export class HomePageController {
  constructor(rot13Client) {
    this._rot13Client = rot13Client;
  }

  // 'request' is an HttpRequest instance
  // 'config' is a WwwConfig instance
  async postAsync(request, config) {
    // Parse the 'text' field from the request's JSON body
    const body = await request.readBodyAsync();
    const formData = new URLSearchParams(body); 
    const textFields = formData.getAll("text");
    const userInput = textFields[0];

    // Call the web service
    const output = await this._rot13Client.transformAsync(config.rot13ServicePort, userInput);

    // Render the page
    return HomePageView.homePage(output);
  }
};

The following test uses spies to check that the above code calls the web service. It’s an interaction-based test that checks whether the dependency’s methods are called correctly.

// Example of spy-based test (JavaScript + testdouble.js)
  
it("POST asks ROT-13 service to transform text", async () => {
  // Create spies
  const rot13Client = td.instance(Rot13Client);
  const request = td.instance(HttpRequest);
  const config = td.instance(WwwConfig);

  // Configure spies
  config.rot13ServicePort = 999;      // rot13ServicePort is a getter, but testdouble.js can’t configure getters’ responses, so we just set the property directly
  td.when(request.readBodyAsync()).thenResolve("text=hello%20world");

  // Run the code under test
  const controller = new HomePageController(rot13Client);
  await controller.postAsync(request, config);

  // Check that the web service’s wrapper was called correctly
  td.verify(rot13Client.transformAsync(999, "hello world"));
});

This test can be converted one spy at a time. First, we replace the HttpRequest spy with a Configurable Response.

// Replace HttpRequest spy (JavaScript + testdouble.js)

it("POST asks ROT-13 service to transform text", async () => {
  const rot13Client = td.instance(Rot13Client);

  // Replace the HttpRequest spy with a real HttpRequest. (Nullable with Configurable Responses)
  const request = td.instance(HttpRequest);
  const request = HttpRequest.createNull({ body: "text=hello%20world" });

  const config = td.instance(WwwConfig);

  config.rot13ServicePort = 999;
  td.when(request.readBodyAsync()).thenResolve("text=hello%20world");    // Old configuration no longer needed

  const controller = new HomePageController(rot13Client);
  await controller.postAsync(request, config);

  td.verify(rot13Client.transformAsync(999, "hello world"));
});

Because Nullables can coexist with test doubles, the tests still pass after this change is made. Next, we replace the WwwConfig spy:

// Replace WwwConfig spy (JavaScript + testdouble.js)

it("POST asks ROT-13 service to transform text", async () => {
  const rot13Client = td.instance(Rot13Client);
  const request = HttpRequest.createNull({ body: "text=hello%20world" });

  // Replace the WwwConfig spy with a real WwwConfig. (Nullable with Configurable Responses)
  const config = td.instance(WwwConfig);
  const config = WwwConfig.createNull({ rot13ServicePort: 999 });

  config.rot13ServicePort = 999;     // Old configuration no longer needed (and real WwwConfig doesn't allow property to be set)

  const controller = new HomePageController(rot13Client);
  await controller.postAsync(request, config);

  td.verify(rot13Client.transformAsync(999, "hello world"));
});

The tests continue to pass. Finally, we replace the Rot13Client spy:

// Replace Rot13Client spy (JavaScript + testdouble.js)

it("POST asks ROT-13 service to transform text", async () => {
  // Replace the Rot13Client spy with a real Rot13Client. (Nullable)
  const rot13Client = td.instance(Rot13Client);
  const rot13Client = Rot13Client.createNull();

  // Track the requests made by the Rot13Client. (Output Tracking)
  const rot13Requests = rot13Client.trackRequests();

  const request = HttpRequest.createNull({ body: "text=hello%20world" });
  const config = WwwConfig.createNull({ rot13ServicePort: 999 });

  const controller = new HomePageController(rot13Client);
  await controller.postAsync(request, config);

  // Replace the method call check with a state-based output check. (Output Tracking)
  td.verify(rot13Client.transformAsync(999, "hello world"));
  assert.deepEqual(rot13Requests, [{                        
    port: 999,            // The port of the ROT-13 service 
    text: "hello world",  // The text sent to the service   
  });                                                       
});

Here’s a side-by-side comparison of the two tests.

// Side-by-side comparison of spy-based test and Nullables-based test (JavaScript + testdouble.js)

// Interaction-based test using spies
it("POST asks ROT-13 service to transform text", async () => {
  // Create dependencies
  const rot13Client = td.instance(Rot13Client);
  const request = td.instance(HttpRequest);
  const config = td.instance(WwwConfig);

  // Configure dependencies
  config.rot13ServicePort = 999;
  td.when(request.readBodyAsync()).thenResolve("text=hello%20world");

  // Run code under test
  const controller = new HomePageController(rot13Client);
  await controller.postAsync(request, config);

  // Check that rot13Client was called
  td.verify(rot13Client.transformAsync(999, "hello world"));
});

// State-based test using Nullables
it("POST asks ROT-13 service to transform text", async () => {
  // Create and configure dependencies
  const rot13Client = Rot13Client.createNull();
  const rot13Requests = rot13Client.trackRequests();

  const request = HttpRequest.createNull({ body: "text=hello%20world" });
  const config = WwwConfig.createNull({ rot13ServicePort: 999 });

  // Run code under test
  const controller = new HomePageController(rot13Client);
  await controller.postAsync(request, config);

  // Check that rot13Client made the correct request
  assert.deepEqual(rot13Requests, [{
    port: 999,            // The port of the ROT-13 service
    text: "hello world",  // The text sent to the service
  }]);
});

To make a dependency Nullable, either Descend the Ladder or Climb the Ladder.

Contents

Throwaway Stub

Making a dependency Nullable requires making all of its infrastructure dependencies Nullable, too. Sometimes, that’s too much work to tackle all at once. Therefore:

In the code you’re making Nullable, create Embedded Stubs for any dependencies you don’t want to make Nullable. This will break the chain of Overlapping Sociable Tests, leaving you vulnerable to behavioral changes in the dependencies, so throw away the stub and replace it with Fake It Once You Make It as soon as the dependency is Nullable.

To avoid writing throwaway code, Climb the Ladder.

Contents

Conclusion

These patterns are an effective way of writing code that is easy to test, easy to refactor, and doesn’t require broad tests.

“Who Is Agile” Interview

Yves Hanoulle interviewed me for his “Who Is Agile” series. The result is a wonderful, wide-ranging interview covering everything from how Dungeons & Dragons has influenced my coaching style, to perfectionism and types of value, to making change stick in an organization, to coordinating large numbers of people, and much more. It’s worth watching.

The video is embedded below and you can also find it on YouTube.

Moved to Mastodon

I'm moving from Twitter to Mastodon. I’ll still be posting announcements to Twitter, but for conversations, find me on Mastodon.

Nullables Livestream #4: State-Based Notifier

In this weekly livestream series, I pair up with Ted M. Young, aka jitterted, to look at Nullables and A-Frame Architecture as an alternative to Mocks, Spies, and Hexagonal Architecture. Each episode combines a healthy dose of architecture and design discussion with hands-on coding.

Watch us live every Tuesday! For details, see the event page. For more episode recordings, see the episode archive.

In this episode...

We look at how to make the HttpScoreCategoryNotifier class nullable. This adapter stores data to a separate “Yacht Tracker” service. We demonstrate how to make the adapter expose its state by creating a reusable OutputTracker class, then demonstrate how easy making an adapter Nullable can be.

Visit the episode archive for more.

Nullables Livestream #3: Fetcher Nullability

In this weekly livestream series, I pair up with Ted M. Young, aka jitterted, to look at Nullables and A-Frame Architecture as an alternative to Mocks, Spies, and Hexagonal Architecture. Each episode combines a healthy dose of architecture and design discussion with hands-on coding.

Watch us live every Tuesday! For details, see the event page. For more episode recordings, see the episode archive.

In this episode...

We start our work on the HttpAverageScoreFetcher and HttpScoreCategoryNotifier classes. These adapters are part of Ted’s implementation of Hexagonal Architecture: they retrieve and store data from a separate “Yacht Tracker” service. We spend some time understanding how hexagonal architecture works, then make HttpAverageScoreFetcher nullable. Along the way, we demonstrate the plusses—and minuses!—of using an embedded stub.

Visit the episode archive for more.