Presentations

I’ve presented at conferences around the world and appeared on many podcasts. Many of my presentations are no longer online (or were never recorded in the first place) but I’ve collected the ones that are still available below.

You might also be interested in:

Top Five

The Agile Fluency® Model Explained24 Jul, 2018
A ten-minute overview of the Agile Fluency Model.
Agile Without Dedicated QA3 Nov, 2019
How Agile teams achieve above-average quality without traditional QA.
Estimates or No Estimates?30 Aug, 2016
How and when to estimate, and why you might not want to.
The Reward30 Aug, 2016
A frank discussion about what it takes to succeed with Agile.
Thinking in Tests3 Nov, 2019
Why bother with automated testing? A look at what you get and what comes next.

All

Find all my presentations and podcast appearances below, starting with the most recent.

Coding Over Cocktails Podcast

I appeared on the Coding Over Cocktails podcast earlier this month. It’s a nice, free-ranging conversation covering topics from large-scale Agile development, to complexity, to organizational investment.

Check out the excerpt in the video below, or listen to the whole podcast at one of these links:

Evolutionary Design Animated

Modern software development welcomes changing requirements, even late in the process, but how can we write our software so that those changes don’t create a mess? Evolutionary design is the key. It’s a technique that emerges from Extreme Programming, the method that brought us test-driven development, merciless refactoring, and continuous integration.

I first encountered Extreme Programming and evolutionary design nearly 20 years ago. Although I was initially skeptical, I’ve explored its boundaries ever since. In this invited talk for the December 2019 YOW! conference series in Australia, I share what I learned through in-depth animations of real software projects. You’ll see how designs evolve over time and you’ll learn how and when to use evolutionary design for your own projects.

Agile Without Dedicated QA

In the early days of Agile, methods such as Extreme Programming advocated for shipping without a QA phase. In fact, they often didn’t have dedicated testers—or even bug-tracking systems. And yet there are stories of these teams producing an order of magnitude fewer defects than normal. What did these teams do, and why did it work? And what role does that leave for QA? In an age where Agile is interpreted to mean “sprints” and “story points,” the technical side of Agile is often forgotten. In this invited talk for the October 2019 Pacific Northwest Software Conference, I spoke about the technical underpinnings of Agile and how they lead to true business agility.

Thinking In Tests

Assert(js) is a conference about testing. Specifically, automated testing. But why bother? Why have automated testing? In my short and sweet keynote session, I looked at what you get and what comes next.

The Agile Fluency Model Explained

The Agile Fluency® Project has put out a new video overview of the Agile Fluency Model. Diana Larsen provided the words and I did the animation. It's on YouTube, so it's easy to share with others and use in your presentations.

I'm particularly proud of how this one came out. Here's the direct link.

ScrumMaster Toolbox Podcast on Agile Fluency Model

Diana Larsen and I were interviewed on the Scrum Master Toolbox Podcast earlier this year. We shared our latest thoughts on the Agile Fluency® Model, including why it's different from a maturity model (and where the idea of "maturity" might fit into the model), our "bus zone" metaphor, and how to use it in your work.

Listen to it here.

Evolution of Agile

I guest-starred on the Ruby Rogues podcast in August 2016. We had a wide-ranging talk discussing how Agile has changed over time, evolutionary design, large-scale Agile, and more. It's one of my favorite podcast appearances to date. Well worth the listen.

Listen to it here.

Two Talks on Scaling Agile

I've been doing a lot of work with multi-team development projects recently, and this has resulted in two good talks on large-scale Agile.

Scaling Beyond the Enterprise

My first talk was a keynote for Agile India in March 2016. It provides a good overview of the issues that come up, some of the solutions, and discusses my approach is different from existing approaches to scaling.

Scaling Beyond the Enterprise

The brilliance of early Agile methods was their non-conformity. They rejected conventional wisdom about how software should be created and substituted a new reality: one where collaboration, adaptation, and continuous improvement were more important than rigid processes and plans. At first, many people rejected these innovations, but Agile stood the test of time. Now it's won the day.

When people talk about scaling Agile, they forget those insurrectionary roots. They focus on what's palatable to the "enterprise:" how to make Agile safe, non-threatening, and acceptable--how to make it more conventional and conformist. In doing so, they risk losing the innovations that make Agile work so well.

What if we stopped worrying about what's safe and acceptable? What if we went back to those innovative roots? What would Agile look like if we scaled beyond the enterprise?

Come find out.

Rethinking Scaling

At the I T.A.K.E. conference in Romania, in May 2016, I keynoted on this topic again. This was an audience of developers, so I took a deeper look at the architectural and team structure considerations. There's a bit of overlap, but it goes into more detail with more examples. I'm particularly pleased with how this talk came out: it's covers a very solid list of things to think about as your company grows.

Rethinking Scaling

That feeling of a successful startup. A handful of people in a room, getting shi...ny things done. Everybody working together, all cylinders firing. It's intoxicating.

That feeling of a great XP team. A cross-functional team, all in a room, getting shi...pping done. Everybody working together, sharing responsibility, creating great code. It's impossible to forget.

But what do you do when the startup IPOs, and the 12-person company is now a 1000-person behemoth? What do you do when the XP team grows, and you have 100 people working on a product, not ten? How do you keep those great small-team dynamics in a big organization?

When people talk about scaling Agile, they focus on what's palatable to the "enterprise:" how to make Agile safe, non-threatening, and acceptable. But what if we aren't in that kind of company? What if we know what it's like to be great, but we're too big to do it the way we used to?

Let's set aside the brand names, consulting companies, and enterprise certifications. Let's look at the possibilities of large-scale Agile at its best.

Estimates or No Estimates?

The Agile community has been arguing about whether estimates are a good idea or not for a while now. In this talk at Øredev in November 2015, I think I did a pretty good job of threading the needle. I talk about how and when to estimate, and why you might not want to.

Estimates or No Estimates?

There's a debate raging in the Agile world: should we estimate, or not? Lost in the noise are more important questions: When should we estimate, and why? When should we not estimate, and why not?

As with so many Agile questions, the answer to "should we estimate?" isn't a clear-cut "yes" or "no." Instead, the answer depends on what your team is capable of, what your organization needs, and how to best balance the two.

We'll take a deep look at how estimates work, why they work, and when and why to discard them. Along the way, we'll encounter surprising insights about the nature of Agile development and the teams that use it.

ESTIMATES OR NO ESTIMATES? from Øredev Conference on Vimeo.

Agile Engineering for the Web

I spoke on Agile engineering at Øredev in November 2015. I'm particularly proud of how well this talk came out--it's livecoding, which can be hard to do well, and it does a great job of showing how to test JavaScript and CSS. It's basically a summary of what I've learned from my Let's Code JavaScript screencast.

Agile Engineering for the Web

Test-driven development, refactoring, evolutionary design… these Agile engineering techniques have long been established for back-end code. But what about the front-end? For too many teams, it's dismissed with a "JavaScript sucks!" and unreliable, brittle code.

In this session, we look at what it takes to bring the best of Agile development to front-end code. Test-drive your JavaScript and refactor your CSS.

AGILE ENGINEERING FOR THE WEB from Øredev Conference on Vimeo.

2015 Sydney Keynote

I keynoted at Agile Australia in June 2015 and InfoQ recorded it. If you know somebody who's doing "Agile in name only," they should watch this video. Here's the blurb:

The Reward

For an approach that emphasizes simplicity, success with Agile is surprisingly difficult. Some teams see amazing results. They're able to respond to changing needs, deliver new releases every day, and do so with nearly zero defects. Other teams complain that Agile makes things worse, not better. They get less done and spend more time in pointless meetings.

What is about Agile that creates such different experiences? Let's take a step back and look at what makes Agile work. What's the point of these Sprints, stories, Scrums, and other practices? What leads to success and what leads to struggle? It's time for a frank discussion about what it takes to succeed with Agile.

See the presentation and transcript here.

Reddit "Ask Me Anything"

The Agile subreddit on Reddit invited me to do an "Ask Me Anything" (AMA) earlier this year. We had a great turnout and discussed many interesting topics.

Read it here.

2013's Conference Videos

I spoke at several conferences in 2013 and I've finally taken the time to track down all the conference videos. Here you go:

Agile Fluency™ Model at NDC 2013

The promise of Agile is simple and compelling: a team that effortlessly surfs the wave of business possibility, changing direction to meet the needs of a changing market. So why do so few teams achieve that ideal? Lack of fluency. Agile may be simple, but it's far from easy, and it takes years of practice to do well. We'll look at four phases of Agile fluency, what you can expect from each phase, and how to increase your team's fluency so you can achieve what Agile promises.

The Agile Fluency Model is a tool Diana Larsen and I created to help teams understand what's possible with Agile and how to increase their fluent proficiency. A lot of people have praised it for its accurate representation of how teams grow, and for its usefulness in discussing Agile with managers and executives.

If you like the model, Diana and I are launching the Agile Fluency Project next week. If you're interested in improving the state of Agile in your company, you owe it to yourself to sign up to learn more.

The NDC 2013 video is here.

I also gave a keynote on this topic at XP 2013. As a keynote, it spends more time on understanding the "why" than the "how," and I work harder to be entertaining. I'll let you decide how successful I was.

The XP 2013 video is here. (Scroll to the bottom of the page and use a browser that supports .mp4.)

Leading Agile Engineering at Agile 2013

You're leading a team as it becomes agile. You've got the planning process under control. You're realizing the benefits of improved transparency and your teams are communicating better with the customer. But you know that many of the key advantages--faster release cadence, improved quality, reduced cost of change--come from changing the core development activity.

Some of these changes, such as basic TDD, are obvious. But even they can be difficult to measure. Beyond the basics, the technical practices become difficult even to describe. Your teams give you transparency into what they do. How can you get similar transparency into how they do it? What should you realistically expect a team to do, and how can you tell whether it is doing that?

This talk targets managers and executives for agile teams. We discuss helping people see what is possible, creating feedback systems so that everyone knows where they are at, and ways to choose practices that align with your business needs.

Arlo Belshee and I make a great team, and we've done some crazy stuff together. This talk is on the serious side, but it's still really good stuff. We talk about how to create a great engineering team and introduce Arlo's "subway map" of Agile engineering practices (influenced in part by the Agile Fluency Model).

This is really Arlo's stuff, much of it inspired by his work at Microsoft, and good stuff it is. I'm honored to have been part of it.

The video is here. (Note: requires a free login.) The sound's a bit low, but headphones help.

Professional JavaScript at NDC 2013

Today's applications run on the web, and they're only getting more complicated. The old "toss a few lines of JavaScript on a page and pray it doesn't break" approach just doesn't cut it any more. To be successful developing today's rich web applications, we need rigorous, professional software development techniques. From test-driven development, to continuous integration, to automated cross-browser testing, this session covers what it takes to build JavaScript that lasts.

I gave several talks at NDC 2013. This one is about taking a rigorous approach to JavaScript and minimizing the cost of maintenance. It's a broad overview of the same material that you'll find in the this screencast, but condensed down and focused on technique. It covers build automation, linting, continuous integration, cross-browser testing, and front-end TDD.

The NDC 2013 video is here.

I keynoted on this topic at ScotlandJS as well. Again, as a keynote, it's more about why than how.

The ScotlandJS video is here.

Test-Driven JavaScript Master Class at NDC 2013

You know JavaScript, but do you know how to test-drive it? In this session, we cover the hard problems. DOM testing, mouse events, touch events, SVG, and more--all test-driven, all against multiple real browsers, and all without mocks or other test doubles. Come dive deep.

Note: This session assumes familiarity with JavaScript. We'll introduce the basics of test-driven development briefly, but prior exposure to TDD is also helpful.

My third session at NDC 2013 picked up where the previous session left off. It's a deep dive into the hard problems of test-driven JavaScript development. If you're a subscriber, it covers the same ground as LL9: Front-End Unit Testing in a Nutshell, but it's more conversational. I think it came out well.

The video is here.

Agile Fluency Podcast

At Agile 2012 this year, Bob Payne interviewed Diana Larsen and me for his long-running Agile Toolkit Podcast. In this thirty-minute session, we talk about our Agile Fluency® model.

It's a great interview. Listen to it here.

"Evolutionary Design Illustrated" Video

At this year's Norwegian Development Conference, I gave a new presentation on evolutionary design, and NDC was good enough to record it and put the video online. It's a highly visual look at how evolutionary design has worked in practice on three projects I've been involved in. It's good to watch if you've ever wondered how evolutionary design plays out in practice. Here's the blurb:

In an agile environment, programmers must deliver working software in the first iteration. Requirements may change at any time, so there's no way to design the software in advance. Instead, you must design your software based on its current needs, and evolve the software design as the requirements change. This process is called evolutionary design. (It's also called continuous design, or iterative and incremental design.)

But how does it work? How can evolutionary design result in high-quality code? In this visual and example-filled session, James Shore will demonstrate how evolutionary design works in practice, using real-world examples culled from his decade of Agile development experience. You'll see how designs evolved in response to external forces, and how responding to those forces yielded in designs that were clean, flexible, and maintainable.

James Shore -- Evolutionary Design Illustrated from Matteo Vaccari on Vimeo.

Certification Debate with Alistair Cockburn

On Tuesday, Alistair Cockburn and I debated the merits and limitations of certification in a webcast hosted by the PMI. We had an interesting and cordial discussion and the PMI has graciously put up their recording for anyone to hear.

Listen to the debate here.

Agile Release Planning Video

My recent Øredev presentation, "Agile Release Planning from Top to Bottom," was recorded and the video is now available online. This presentation was very well received and I think it's one of the best I've given recently.

Agile Release Planning from Top to Bottom - James Shore from Øredev Conference on Vimeo.

Video of Bloody Stupid Johnson Now Online

The Agile Alliance has put up their recording of my presentation with Arlo Belshee, Bloody Stupid Johnson Teaches Agile. This presentation was the highest rated session of the Agile 2010 conference. You'll need to be a member of the Agile Alliance to view it.

If you'd like to see this presentation live, Arlo and I will be reprising it on December 15th (2010) for Agile PDX. This is a free event hosted by the Portland Agile community. Watch the event page or my Twitter feed for the location.

Single-Piece Flow in Kanban (Video)

Arlo Belshee and I presented "Single-Piece Flow in Kanban: A How-To" at the recent Lean Software and Systems 2010 conference in Atlanta. I've embedded a recording of it below. The speaker introducing us at the beginning is David Anderson, a well-known Kanban proponent and one of the founders of the conference.

“Is Fit Dead?” Conversation Summary

Eric Lefevre-Ardant has an excellent transcript of a discussion about Fit that I participated in on Twitter a few weeks ago. Here's an excerpt:

It was kicked off by an interview of Ward Cunningham and James Shore on Hanselminutes where Ward & James were asked whether Fit was dead.

Bob Martin first reacted on Twitter by pointing out that, at any rate, “FitNesse is thriving”, along with Slim, a new system that can be used by Fitnesse as an engine to run the test tables.

Michael Feathers replied that, in his view, Fit was more appropriate as a seed for other works: “Take it, grow it locally, and never commit back.” This seems confirmed by James Shore, a former leader of the Fit project (and a successor to Ward in that role): “Fit core was intentionally resistant to change [...] from an organizational perspective”

Interestingly, James believes that both Fit and Fitnesse have “similar flaws, which could be solved by another approach”:

  • Fit flaw #1: maintenance. HTML is hard to maintain and refactor.”
  • Fit flaw #2: Customers. Customers don’t generate the documents, and that was the whole idea.”
  • Fit flaw #3: Programmers. Fit loves domain models and Whole Value. Most programmers don’t. Impedance mismatch.”

If you're interested in Fit, FitNesse, or similar tools, it's well worth the read.

Read the whole transcript here.

“Maximizing Value with Agile Release Planning” Recording

"Maximizing Value with Agile Release Planning" is one of my favorite presentations to give. It's a thorough look at five ways to improve value on your agile projects:

  1. Work on One Project at a Time
  2. Release Early and Often
  3. Adapt Your Plans
  4. Keep Your Options Open
  5. Plan at the Last Responsible Moment

I delivered this presentation at the Portland IIBA meeting (International Institute of Business Analysts) last week, which reminded me that there's a recording of this presentation online at the Product Management View blog. I thought you might like to see it.

Watch it here.

Interview on Unbound Developers Show

The Unbound Developers Show just posted a podcast of an interview they did with me a few weeks ago.

The interview came out well. We discussed a wide range of topics, including Scrum, Lean, Kanban, how to adopt Agile, resistance to change, distributed teams, and more. It's worth a listen.

Hear it here.

“Absolutely No Machete Juggling” Review

Rod Hilton's Absolutely No Machete Juggling blog has a review of The Art of Agile Development. Here's an excerpt:

That said, this is an excellent book on XP. All of XP's practices are covered in just the right amount of detail - giving you enough information that you can adopt the practice, but requiring that you do additional research about the ones that interest or challenge you the most. This allows the book to be a pretty quick read in general, which is beneficial for a team wishing to adopt XP practices soon. This level of detail does have one disadvantage, though: by staying largely in the realm of hypotheticals and ideals, I often found myself thinking that the authors were being naive and idealistic about how easy some practices were to adopt. This made me somewhat more skeptical of the book than I would have been if more detail had been offered, but ultimately I felt the level of detail was right; I did my own research online about some of the things that challenged me.

Read the full review here.

Fit Essay in MundoJava

I have a short essay in the September issue of MundoJava, which was described to me as "Brazil's best software development magazine."

The editors at MundoJava translated my essay to Portuguese, but kindly gave me permission to post my original here:

What is Fit and why does it matter? It's part of your agile bug-prevention arsenal. Unlike the traditional approaches to quality, which focuses on finding and removing defects, agile approaches focus on preventing bugs from being created in the first place. This leads to some pretty amazing results. Experienced agile teams only produce a handful of new bugs per month.

The "agile engineering practices" introduced by Extreme Programming (XP) are part of the reason. These techniques--such as test-driven development, pair programming, collective code ownership, simple design, and energized work--prevent most programming errors. They ensure that the code does exactly what the programmers intended it to do.

Eliminating programming errors is a big step, but it doesn't prevent all bugs. Sometimes programmers misunderstand what is needed. Agile's idea of an involved product owner who sits with the team--an "on-site customer," in XP terms--goes a long way to preventing programmer misunderstandings. They can easily ask questions of the product owner, and she can see the team's progress and make corrections as they go.

These are great techniques, but some bugs still sneak by. Complex problem domains, such as finance or law, have a lot of nit-picky scenarios that have to programmed exactly right. In many cases, even the experts don't agree about the correct answers. In order for the programmers to create the software without bugs, they have to understand all of the possible cases and what the correct answers are. Surprisingly, even the experts haven't thought through all of the possibilities, and they often won't agree which answers are correct!

It's a recipe for bugs, and that's where Fit comes in. More than anything else, Fit is a tool for improving communication and collaboration around business requirements. Using Fit, teams discuss real-world examples of business rules in practice, and Fit automatically checks that the software does what the examples say. This conversation flushes out misunderstanding, errors, and disagreements. The automatic checks force the conversation to be rigorous, and provide documentation and ongoing confirmation.

This ability to enhance communication is Fit's most important attribute. Teams often think of Fit as a tool for testers. Nothing could be further from the truth! Fit's strength is its ability to get programmers and business experts talking to each other, disagreeing with each other, and reaching consensus together. Testers play a role in this process--often bringing up edge cases nobody had considered--but Fit is not a testing tool. It's a tool for collaboration... that happens to automatically check your results.

What is Fit and why does it matter? It's a tool for eliminating defects that arise from the confusion and miscommunication that often surround complex business requirements. It's an important part of your agile bug-prevention arsenal.

"Art of Agile Development" Interview with O'Reilly

O'Reilly conducted an interview with Shane and me at OSCON in June. It came out well, and this is one of the rare times that Shane and I have appeared together to discuss the book.

InfoQ Video Interview

InfoQ just posted a video interview with me. Deborah Hartmann interviewed me at Agile 2007 last year just before the book came out. It came out great, and InfoQ also included a transcript for those of you who prefer reading to watching.

This interview joins a few other good recent interviews: Bob Payne's Agile Toolkit interview, also conducted at Agile 2007, and Clarke Ching's Agile Thinkers interview.

Interview on Agile Thinkers

Clarke Ching has a new website called Agile Thinkers. It's devoted to interviews with people in the agile community and it looks like it will have some great stuff. (Clarke's already proven the model with TOC Thinkers, about the Theory of Constraints.)

Shane and I are the first ones up. Clarke is putting the interviews up one question at a time--you can either visit the front page and see everything as it comes, or you can jump straight my interview or Shane's interview.

To give you a taste, here's my response to Clarke's first question. There's more on the site.

Q1. Hi Jim, I'm really enjoying your book. I honestly think it is the best agile book I've read - and I've read a good few of them. Good job! Can you tell me a bit about yourself - both personally and professionally? What got you to the point where you could produce such a wonderful book?

Thank you! I should start by recognizing my coauthor, Shane Warden. Shane and I collaborated closely on the book, so that's part of the reason it turned out so well. Another reason would be the sheer amount of time and effort we put into it--I put nearly a year of full-time work into it myself, and that's not counting Shane's effort or the effort of all the people at O'Reilly. I think our open review process helped a lot, too. We put nearly every section of the book online and we got tons of feedback--over 1200 messages were posted to the reviewer list. Our best work started as drafts that got brutal beatings.

So: blood, sweat, and tears. You know, the usual.

As for how I personally got here... you know, it's hard to say. I've always liked reading and writing, even though I was born a hardcore geek. (I placed in my first programming contest at age ten or eleven. I think I would have come in first if my program had actually loaded off the tape drive.) I actually did better on the English portion of the SATs than the Math portion. I also wasted a huge amount of time on Fidonet and Usenet, talking about programming and (in retrospect) being a general pain in the ass. I could spend hours composing careful explanations of why OS/2's Worksplace Shell was better than the Windows desktop. I expended far more effort than the topics were worth, except for two things: I had fun, and I had a lot of practice writing and seeing how my writing was received.

So that's how I learned to write. On the software side, the really core experience was when I had an AppleSoft BASIC program dissolve on me. Applesoft BASIC was old-school: line numbers, two-letter variables, GOTO's, the works. When I was in high school, the complexity of one of the programs I was writing overwhelmed my capacity to keep track of it all. The program just fell apart in my brain. Ever since, I've been fascinated with the human side of programming: if programming is partly an exercise in making the unbelievably complex comprehensible, how do we do that? That led me to structured programming, then OOP, then software engineering texts, then Ward's Wiki, then XP and agile development, and now to questioning what makes software successful. Along the way, I did the same thing I did with writing: I spent hours every day just playing with the ideas, writing programs, writing about programming, and so on. (As well as my professional experience.)

So really, the answer is that, for most of my life, I've had a pretty single-minded dedication to writing, programming, and software development. (The word you're looking for: Nerd.) It's amazing I ever married, let alone reproduced.

“Art of Agile” Interview on Agile Toolkit Podcast

Bob Payne has a great interview with me up on his "Agile Toolkit" podcast. It's a nice, wide-ranging interview. We start out talking about the book, migrate to a discussion of learning styles, then Bob's Mano a Mano project at Agile 2007, before wrapping up with a discussion of agile tools and CardMeeting.

It's a fun interview. Check it out.

Advanced NUnitAsp Video

I've just finished something that I think is really cool. I've created a video for advanced users of NUnitAsp that goes into NUnitAsp's architecture and provides some tips for overcoming common challenges. It's a neat presentation, very compact, and well illustrated with animation and figures, if I do say so myself.

Watch it here.

This is the first time I've done something like this and I'm very happy with how it turned out.

So how did I do it? It took me about eight hours. I worked off of some PowerPoint material that I had already created for my upcoming Test-Driven ASP.NET course, although it turned out that several of the really complicated pages from that material needed rewriting. (PowerPoint presentations usually suck, but I guarantee that this one doesn't. It's very dynamic, with animated illustrations rather than bullet points.) I wrote a script and practiced it several times, making notes about changes and what didn't flow well. Then I used Camtasia Studio to record the video as I narrated the PowerPoint deck "live."

The first take came out pretty well, but I felt that I sounded a little robotic. I recorded another take--the video is only eight minutes--and I'm much happier with it. I used Camtasia Studio to edit out a few of my worst stutters and then produced a Flash video. It turned out great!

I learned a few lessons from this. First, having an interesting set of PowerPoint slides to narrate worked very well. Second, a written script is critical. (Thanks to Joshua Kerievsky for that insight.) I returned to the script over and over again, marked up where I stuttered, etc. Third, when you make a mistake, just pause and repeat the bumbled line from the start. You can edit out the mistake easily afterwards. Finally, a good microphone is essential. I have a fairly cheap microphone and it shows.

Producing this video took me a while, but not as long as I feared. I'll certainly do it again. Hopefully it will get faster as I go. This one took about an hour of production per minute of actual video. :-)

Using Fit for Agile Requirements Collaboration

The content from the first half of the presentation, about Fit, can be found in my "Fit Workflow" document on the Fit website.

The content from the second half, about agile requirements collaboration, can be found in my "Beyond Story Cards" write-up.

References

Software by Numbers is a great book that talks about how you can increase your return on investment just by changing the order in which you deliver features. Kent Beck has released a new edition of Extreme Programming Explained. It's a good book and reflects a lot of experience in helping teams understand XP. Finally, Lean Software Development is my favorite book for understanding the principles that underlie agile methods. I highly recommend it.

Beyond Story Cards: Agile Requirements Collaboration

I gave a presentation at the Rose City Software Process Improvement Network (SPIN) on March 10th, 2005. This is approximately what I said:

Introduction

[A picture of a story card]

This is a story card. It's a great tool for planning and a lousy tool for communicating requirements. If you've done much work with Extreme Programming (XP), then you may have heard people say, "story cards are a marker for future conversation." But what does that mean? Where do the story cards come from, and what do you do with them when you have them?

In this presentation, I'll be talking about what happens before you create the story cards and what happens after you create them. The one thing I won't be talking about much is the story cards themselves.

Beyond Story Cards: Agile Requirements Collaboration

[A picture of a ripped up story card]

My name is Jim Shore. I've been coaching teams in adopting agile methods since 1999, when we called them "lightweight methods." I started out leading a team using Feature-Driven Development. In 2000, I tried XP, was very successful with it, and I've been helping teams adopt XP ever since. I've worked with a wide variety of companies, and this has given me perspective on what works in XP and what doesn't.

There are a lot of ways to do requirements, and I'm sure there are a lot of ways to do agile requirements. What I'll be presenting today is based on my experience in implementing XP over the last several years. It's a complement to existing requirements engineering techniques and you can continue to use whatever techniques you're currently using for gather requirements.

Why Collaborate?

Lack of user involvement traditionally has been the No. 1 reason for project failure. Conversely, it has been the leading contributor to project success. Even when delivered on time and on budget, a project can fail it it doesn't meet user needs or expectations.

"Project Management: The Criteria for Success"

The title of my presentation is "Agile Requirements Collaboration." I chose that title very carefully. I could have said "Agile Requirements Gathering" or "Agile Requirements Engineering." I chose "collaboration" because user involvement is a key success factor in software projects... in all software projects, agile or not. Agile development values collaboration because it increases your chance of success. I want to emphasize that the techniques I'm going to be talking about are tools for enhancing collaboration, not for creating requirements and throwing them over the wall.

Where do stories come from?

[A picture of a person holding a card to his head]

So where do stories come from in the first place? Well... somebody makes them up. Somebody has to have a vision of what the product is going to be. On an XP project, that person is usually called the "on-site customer." You'll also hear terms like "lead customer" and "product manager." I'm going to call him (or her) a "product manager" in this presentation, because it's more in line with common industry terms.

The product manager needs to have a strong, clear vision of the product, but that doesn't mean he or she is solely responsible for it. The product manager will work with interaction designers, business analysts, real customers, marketers, users... Ultimately, though, somebody has to coalesce all these conflicting views and come up with a clear, unifying product vision. That's the role of the product manager.

The product manager's job isn't easy. He or she (I'll just say "he") has to have a strong sense of vision, to tie all the diverse threads together into a strong, usable product. He has to know the market well and understand what it needs. He has to have the political savvy to juggle all of the project stakeholders' demands, and he has to have the authority to make his decisions about priorities stick.

People like this are very valuable to their companies and it can be hard to get their time. One of the biggest challenges I see with XP teams is difficulty getting a good product manager. To that, I have to say: Yes! Good product managers are expensive. But let's remember that their involvement is a key success factor. Developing software is even more expensive than a good product manager. If your product isn't worth the cost of a good product manager's time, maybe you should re-evaluate whether your software is valuable enough to be worth doing in the first place.

Audience question: What if you don't have anyone with a vision of the product?

I haven't encountered that. What I have seen is that people sometimes aren't sure what they want. They think they know, but when they actually see it running, they realize that they wanted something else. I've also seen people having trouble articulating their vision. I think the frequent iteration-by-iteration deployment of agile methods are an important way to help people understand on what they really want, because they get to see the software actually running.

On the other hand, I was on a team where we were working with a customer and we were showing him the product every two weeks. We weren't sure that we were really delivering what he wanted. Every time we said, "this is really what we're going to deploy!" And he said, "sure, sure." And then, about four weeks from release, we said that again and he said, "wait! That's what we're going to deliver? No, no, it's all wrong."

But it only took us one two-week iteration to change the software to what he wanted. Most of the changes were just user interface changes.

Audience question: I think something that's more common is to have too many people with product vision. How do you balance this?

That's the product manager's role, to balance the competing visions of all the different stakeholders. You need someone with political savvy to do that. I've tried a number of different options, such as allocating each group a portion of the iteration budget, having people justify the value of their stories, and they didn't work well. We spent too much time doing bits and pieces of too many things and never got anything completely done. Ultimately what worked best was just having a product manager make the call.

The "latest responsible moment"

[A picture of a wall of story cards]

The product manager, with the assistance of his team--business analysts, customers, marketers, etc.--creates the stories. But watch out for "story card hell."

Story card hell is when you have 300 story cards and you have to keep track of them all. This doesn't work. Story cards aren't meant to capture all of the details of your product requirements. They're only supposed to be a sentence or two. At most, a paragraph. When you have 300 cards, you lose the ability to understand what they mean. Let's say you have one story card that says "make toast" and another that says "cook bread." Is that the same story? Or are they different requirements for completely different parts of the system? You can't tell, and you lose track of the overall vision because you're surrounded by trees.

A good sign that you're in story card hell is when you start hearing the question, "can we put these in a database?" Every customer I've ever worked with wanted to put story cards in a database. That misses the point of story cards. If you're worried about losing story cards, you're doing too much with them.

That brings us to the idea of "latest responsible moment." "Latest responsible moment" is a core agile idea. The term comes from Lean Software Development. The idea is that you want to put off decisions as long as you can: to the latest responsible moment. But it's the latest responsible moment, not the "last possible" moment. That wouldn't be responsible.

The reason we do this is because putting off decisions is cheaper and results in better decisions. Money we spend tomorrow is cheaper than money we spend today, and between now and tomorrow, we'll learn more things about our environment and the decisions we need to make. Plus, things could change so much that the decision isn't relevant any more. If we've spent the time and effort to make the decision, that money is wasted. So we wait as long as we responsibly can.

Focus on more detail over time

[A picture of a funnel showing increasing detail over time.]

Using this idea of "latest responsible moment," we postpone creating our detailed requirements specification. In fact, we can do the specification for a story at the same time as we actually implement the story. (I'll be talking about that more in a little bit.) We don't need to know every detail about a feature ahead of time. Going into the level of detail is expensive, and if (or when) our plans change, that's wasted effort.

In this graph here (points to slide) I show different levels of detail according to how far away a feature is from being implemented. In most agile methods, we like to deliver a release--at least an internal release--in less than three months. So when you're more than three months away from delivery, all you really need is an overall vision for the release. What market is the software going to serve, what kind of value is it going to provide, what are the basic things the software is going to do. You may have a product vision document or a mission statement.

Once you get closer than three months to delivery, you need to be able to plan the specific features you're going to deliver. So go into more detail. Some people call these "release-level stories," "high-level stories," or "motherhood stories." Either way, you define the major features that your product is going to contain. You can write them on cards.

Six weeks is a good threshold for detailed planning. It's about as far out as you can reasonably predict in detail. Story cards give you the ability to do that detailed planning, so once a feature is less than six weeks from delivery, go into more detail and create stories. If your market and product requirements are particularly stable, you may want to plan a longer timeframe. If it's more chaotic or poorly understood, perhaps less.

And of course, once you've started implementing the story--in this graph, I'm assuming two-week iterations, so that's two weeks before delivery--you need all of the details about the story. That's when you create the specification.

This is an idealized view of the world. In practice, we don't proceed smoothly from vision, to features, to stories, to specification, on the clear, crisp timeframe described here. Reality is messier and it can be hard to know when to go into more detail. But the overall point is valid: you don't need all of the details in advance. You can avoid story card hell by going into more detail only when you need it--at the latest responsible moment.

Audience question: How do you define a "feature?"

You can use whatever definition you're currently using. The important point here is increasing detail over time. I like the definition from Software by Numbers (I'll have a reference at the end), which uses "minimum marketable features:" the smallest set of functionality that has value to your market. You can also think of features as anything the product manager wants the team to do.

Audience question: Can you define the product vision more than three months in advance?

Absolutely. Three months is when I'd start going into more detail and defining features. If you're doing a long term project, with one or two year time frames, then it's a good idea to have an overall vision for the whole project. And perhaps a more detailed vision for the next release, and specific features when you cross the three month mark.

The miracle of collaboration

If you have your product manager and his team working directly with your programmers throughout this process, something amazing happens. I call it "the miracle of collaboration." And it really is a miracle, because time appears out of nowhere.

Now, this isn't easy to achieve. It takes a team that works well together. When your product manager works with your programmers, and the programmers give an estimate, the product manager asks a question that causes every programmer's teeth to grind:

"Why does it cost so much?"

The instictive reaction to this question is to get defensive. "It costs so much because software development is hard, damn it! Why are you questioning me!?"

But there's another approach. It's not easy. But you can take this question, "why does it cost so much?" and think of it in another way. You can think of it as an honest request for information: "Why is this expensive?" And you can answer by talking about what's easy and what's hard.

For example, if the product manager asks for the toaster to automatically pop up the toast when it's done, programmers might say that's really expensive. And when the product manager asks why, the programmers can say, "Well, popping up the toast is easy; that's just a spring. But detecting when the toast is done--that's new. We'll need an optical sensor and some custom brownness-detecting software."

This gives the product manager an opportunity to ask, "But what about all those other toasters out there? How do they know when the toast is done?"

The programmers respond: "They use a timer. But that doesn't really detect when the toast is done. It's just a kludge."

To which the product manager says, "That's okay! Our customers don't want a super toaster. They just want a regular toaster. Use a timer just like everyone else."

And the programmers say, "oh, okay. Well, that won't be expensive at all."

When you do this, when you have honest and open dialog between product managers and programmers, the miracle of collaboration occurs and extra time appears out of nowhere. This happens because product managers don't really know what's easy and what's hard. They might think they do, they might guess, but they're wrong often enough for it to be interesting.

Similarly, programmers don't really know what product managers feel is important. They might think they do, they might guess, but they're wrong often enough for it to be interesting.

Sometimes--not all the time, but often enough to be valuable--these two cases intersect. The product manager asks for something hard that's not super important. He asks for it because he doesn't know it's hard. If the programmers say it's hard, and offer easy alternatives, the product manager can change direction and save time. This doesn't happen on every story, but it does happen often enough to add up. Time appears out of nowhere. It's the miracle of collaboration.

Audience question: What about when your programmers can't give you reliable estimates?

I've found that programmers are pretty good at estimating scope, but not time. Their estimates may be wrong, but they're consistent, so you can tell when something is twice as difficult as something else. This is particularly true when estimating stories rather than features or overall releases.

If your programmers aren't able to estimate consistently, it's probably due to bigger problems. I'd guess they're facing a lot of design debt. Design debt occurs when you sacrifice code quality to meet a deadline. You're essentially taking a loan out on your code. You get some free time, but afterwards, everything takes a little more time to implement, because the code quality isn't there. You're paying interest--high, credit card interest--on your time loan. The only way out is to pay back your principal: spend time to fix the shortcuts.

What happens, though, is that people don't realize they're taking on design debt. So it piles up, and code quality deteriorates. Things start taking longer and longer. Everything takes longer than expected and programmers lose their ability to estimate because they're always running into unexpected problems.

If your programmers can't give you reliable estimates, you probably have a problem with design debt. You need to spend time refactoring and pay back those loans. But don't do it all at once--combine refactoring with implementing new features.

Audience question: When does it make sense to just rewrite rather than refactoring?

I'm reluctant to rewrite. It risky and usually takes longer than expected. You've got a big chunk of time where you're not producing anything of value to your customers because you're re-implementing existing functionality.

On the other hand, software can be so bad that it's cheaper to rewrite than refactor. I'm not sure where the boundary lies. The better I understand refactoring, the more I'm inclined to refactor rather than rewrite. One rule of thumb to keep in mind, from Luke Hohmann, is that rewriting software costs 1/2 the total man-hours of creating and maintaining the software from inception to now. That's often a bigger cost than you might expect.

The requirements fallacy

[A picture of a densely-populated story card]
Image courtesy of xprogramming.com

I've talked a lot about what where story cards come from. Now let's talk about where they go.

A common mistake is to think of story cards as little 3x5 requirements documents. I call this "the requirements fallacy." It's a fallacy because story cards make lousy documentation! You end up in story card hell, with lots of little pieces of paper everywhere. It's miserable.

A good sign you're in story card hell is when you're afraid of losing the story cards. As I mentioned before, if you want to put the cards in a database, you're doing too much with them. Another sign is lots of teeny-tiny writing. Another sign is wanting to use bigger cards.

In fairness, this card here (points to picture) is actually taken from the first XP project. But on the page I got it from, there's big red letters saying "don't do this!"

If you don't write requirements on story cards, then where do they go? Actually, XP has had an answer for this from the beginning. Most people forget about it.

Customer Acceptance Tests

Any program feature without an automated test simply doesn't exist. ...Customers write functional tests so that their confidence in the operation of the program can become part of the program, too.

Extreme Programming Explained, p57

Customer tests are how XP expects you to keep track of your stories' details. People don't do this, though, because it's been really hard to do. We have xUnit for programmer tests and test-driven development, but there was no equivalent tool for customer tests.

Some teams created their own tools, but there's a second part of customer tests that's even harder to achieve. Customers are supposed to write and own their tests. So not only do we need a tool for customer tests, we need it to work in a way that customers--that is, our product manager and his team--can understand and use.

In 2002, Ward Cunningham created a tool that enables this. It's called Fit.

Fit

Fit is a tool for encouraging collaboration on the details of a story. It's also a testing tool. I focus on the collaboration part, though, because I think people often get too caught up in the testing aspect of Fit. You're not supposed to write your requirements using Fit and hand them to your programmers. You're supposed to work with your programmers to understand all of the details of the story, create tests, and create lasting specifications that you can refer to in the future.

This portion of my presentation was closely based on my Fit documentation trail, particularly "Introduction to Fit" and "Fit Workflow." Please refer to those documents for the remainder of the presentation.

Conclusion

To summarize, you need a visionary product manager to work with your team. If you have one, you don't need to define all the details of your requirements in advance: you can wait until the last responsible moment. If a release is more than three months away, all you may need is a vision for that release. Up until six weeks before implementing a feature, a basic description of the feature on a card may be enough. You'll need stories for planning about six weeks before implementing, and once you start implementing a feature, you should specify it in detail. Fit is a great tool for using word processors to create customer acceptance tests, and combined with some prose descriptions of the tests, a Fit document can also work as your specification.

References

  • Jim Shore - http://www.jamesshore.com
  • Fit - http://fit.c2.com
  • "Offing the Offsite Customer"

  • Software by Numbers - Denne & Cleland-Huang
  • Extreme Programming Explained, 2nd Ed. - Beck, Andres
  • Lean Software Development - Poppendieck

I maintain a web site, jamesshore.com, that contains articles about agile development and a blog. You can learn more about Fit at the Fit website, fit.c2.com.

I talked about the importance of having a visionary product manager to work with your team. It can be hard to get people to realize the value of this. I've created a simulation that helps people realize how important working directly with the team is. The simulation is called "Offing the Offsite Customer" and you can find a full set of presenter materials for it on my website. It's a lot of fun.

Software by Numbers is a great book that talks about how you can increase your return on investment just by changing the order in which you deliver features. Kent Beck has released a new edition of Extreme Programming Explained. It's a good book and reflects a lot of experience in helping teams understand XP. Finally, Lean Software Development is my favorite book for understanding the principles that underlie agile methods. I highly recommend it.

I'll be available for additional questions afterwards. Good night.