It's entirely possible to make predictions with Agile. They're just as good as predictions made with other methods, and with XP practices, they can be much better. Agile leaders talk about embracing change because that has more potential value than making predictions.
Software is inherently unpredictable. So is the weather. Forecasts (predictions) are possible in both situations, given sufficient rigor. How your team approaches predictions depends on what level of fluency they have.
One-star teams adapt their plans and work in terms of business results. However, they don't have rigorous engineering practices, which means their predictions have wide error bars, on par with typical non-Agile teams (for 90% reliability, need 4x estimate*). They believe predictions are impossible in Agile.
Two-star teams use rigorous engineering practices such as test-driven development, continuous integration, and the other good stuff in XP. They can make predictions with reasonable precision (for 90% reliability, need 1.8x estimate*) They can and do provide reliable predictions.
Three- and four-star teams conduct experiments and change direction depending on market opportunities. They can make predictions just as well as two-star teams can, but estimating and predicting has a cost, and those predictions often have no real value in the market. They often choose not to incur the waste of making predictions.
So if a company were to talk to me about improving predictability, I would talk to them about what sort of fluency they wanted to achieve, why, and the investments they need to make to get there. For some organizations, *3 fluency isn't desired. It's too big of a cultural shift. In those cases, a *2 team is a great fit, and can provide the predictability the organization wants.
I've heard people say TDD automatically creates good designs. More recently, I've heard David Hansson say it creates design damage. Who's right?
Neither. TDD doesn't create design. You do.
TDD Can Lead to Better Design
There are a few ways I've seen TDD lead to better design:
A good test suite allows you to refactor, which allows you to improve your design over time.
The TDD cycle is very detail-oriented and requires you to make some design decisions when writing tests, rather than when writing production code. I find this helps me think about design issues more deeply.
TDD makes some design problems more obvious.
None of these force you to create better design, but if you're working hard to create good design, I find that these things make it easier to get there.
TDD Can Lead to Worse Design
There are a few ways I've seen TDD lead to worse design:
TDD works best with fast tests and rapid feedback. In search of speed, some people use mocks in a way that locks their production code in place. Ironically, this makes refactoring very difficult, which prevents designs from being improved.
Also in search of speed, some people make very elaborate dependency-injection tools and structures, as well as unnecessary interfaces or classes, just so they can mock out dependencies for testing. This leads to overly complex, hard to understand code.
TDD activates people's desire to get a "high score" by having a lot of tests. This can push people to write worthless or silly tests, or use multiple tests where a single test would do.
None of these are required by TDD, but they're still common. The first two are obvious solutions to the sorts of design problems TDD exposes.
They're also very poor solutions, and you can (and should) choose not to do these things. It is possible to create fast tests and rapid feedback without these mistakes, and you can see us take that approach in the screencast.
So Do Your Job
TDD doesn't create good design. You do. TDD can help expose design smells. You have to pay attention and fix them. TDD can push you toward facile solutions. You have to be careful not to make your design worse just to make testing better.
So pay attention. Think about design. And if TDD is pushing you in a direction that makes your code worse, stop. Take a walk. Talk to a colleague. And look for a better way.
I got involved with Extreme Programming in 2000. Loved it. Best thing since sliced bread, yadda yadda. I was completely spoiled for other kinds of work.
So when that contract ended, I went looking for other opportunities to do XP. But guess what? In 2001, there weren't any. So I started teaching people how to do it. Bam! I'm a consultant.
Several lean years later (I don't mean Lean, I mean ramen), I'm figuring out this consulting thing. I've got a network, I've got business entity, people actually call me, and oh, oh, and I make a real damn difference.
Then Agile starts getting really popular. Certification starts picking up. Scrum's the new hotness, XP's too "unrealistic." I start noticing some of my friends in the biz are dropping out, going back to start companies or lead teams or something real. But I stick with it. I'm thinking, "Sure, there's some bottom feeders creeping in, but Agile's still based on a core of people who really care about doing good work. Besides, if we all leave, what will keep Agile on track?"
It gets worse. Now I'm noticing that there are certain clients that simply won't be successful. I can tell in a phone screen. And it's not Scrum's fault, or certification, or anything. It's the clients. They want easy. I start getting picky, turning them down, refusing to do lucrative but ineffective short-term training.
Beck writes XP Explained, 2nd edition. People talk about Agile "crossing the chasm." I start working on the 2nd edition XP Pocket Guide with chromatic and it turns into The Art of Agile Development. We try to write it for the early majority--the pragmatics, not the innovators and early adopters that were originally attracted to Agile and are now moving on to other things. It's a big success, still is.
It gets worse. The slapdash implementations of Agile now outnumber the good ones by a huge margin. You can find two-day Scrum training everywhere. Everybody wants to get in on the certification money train. Why? Clients won't send people to anything else. The remaining idealists are either fleeing, founding new brands, or becoming Certified Scrum Trainers.
Believe it or not, things haven't really gotten worse since then. Actually, they've gotten a bit better. See, 2-5 years is about how long a not-really-Agile Agile team can survive before things shudder to a complete halt. But not-quite-Agile was Actually. So. Much. Better. (I know! Who could believe it?) than what these terribly dysfunctional organizations were doing before that they're interested in making Agile work. So they're finally investing in learning how to do Agile well. Those shallow training sessions and certifications I decried? They opened the door.
And so here we are, 2014. People are complaining about the state of Agile, saying it's dying. I disagree. I see these "Agile is Dying" threads as a good thing. Because they mean that the word is getting out about Agile-in-name-only. Because every time this comes up, you have a horde of commenters saying "Yeah! Agile sucks!" But... BUT... there's also a few people who say, "No, you don't understand, I've seen Agile work, and it was glorious." That's amazing. Truly. I've come to believe that no movement survives contact with the masses. After 20 years, to still have people who get it? Who are benefiting? Whose lives are being changed?
That means we have a shot.
And as for me... I found that opportunity, so I get to be even more picky about where I consult. But I continue to fight the good fight. Diana and I produced the Agile Fluency™ model, a way of understanding and talking about the investments needed, and we're launching the Agile Fluency Project later this month. We've already released the model, permissive license, for everyone to use. Use it.
Because Agile has no definition, just a manifesto. It is what the community says it is. It always has been. Speak up.
Object Playground in action
Martin Fowler recently asked me via email if I thought there might be a relationship between Agile Fluency and how teams approach estimation. This is my response:
I definitely see a relationship between fluency and estimation. I can't say it's clear cut or that I have real data on it, but this is my gut feel:
"Focus on Value" teams tend to fall into two camps: either "estimation is bad Agile" or "we're terrible at estimating." These statements are the same boy dressed by different parents. One star teams can't provide reliable estimates because their iterations are unpredictable, typically with stories drifting into later iterations (making velocity meaningless), and they have a lot of technical debt (so even if they took a rigorous approach to "done done" iterations, there would be wide variance in velocity from iteration to iteration, so their predictions' error bars would be too wide to be useful).
"Deliver Value" teams tend to take a "we serve our customer" attitude. They're very good at delivering what the customer asks for (if not necessarily what he wants). Their velocity is predictable, so they can make quite accurate predictions about how long it will take to get their current backlog done. Variance primarily comes from changes to the backlog and difficulty discussing needs with customers (leading to changes down the road), but those are manageable with error bars. Some two-star teams retain the "estimation is bad Agile" philosophy, but any two-star team with a reasonably stable backlog should be capable of making useful predictions.
"Optimize Value" teams are more concerned with meeting business needs than delivering a particular backlog. Although they can make predictions about when work will be done, especially if they've had a lot of practice at it during their two-star phase, they're more likely to focus on releasing the next thing as soon as possible by reducing scope and collaboratively creating clever shortcuts. (E.g., "I know you said you wanted a subscriber database, but we think we can meet our first goal faster if we just make REST calls to our credit card processor as needed. That has ramifications x, y, z; what do you think?"). They may make predictions to share with stakeholders, but those stakeholders are higher-level and more willing to accept "we're working on business goal X" rather wanting than a detailed timeline.
I'm not sure how this would play out with "Optimize for System" teams. I imagine it would be the same as three-star fluency, but with a different emphasis.
Let's Code TDJS isn't exactly a startup--I already have a business as an independent Agile consultant--but I'm working on the series full-time. I've effectively "quit my day job" to work on it. I'm doing this solo.
(By the way, launching on Kickstarter was fantastic. I mean, sure, it was nailbiting and hectic and scary, and running the KS campaign took *way* more work than I ever expected, but the results were fantastic. As a platform for confirming the viability of an idea while simultaneously giving you seed funding, I can't imagine anything better for the bootstrapper.)
Anyway, the Kickstarter got a reasonable amount of attention. I tossed up a signup page for people who missed it, and by the time I was ready to release the series to the public, I had a little over 1500 people on my mailing list.
I announced the series' availability on February 4th. First on Sunday via Twitter, and then on Monday morning (recipient's time) via the mailing list. That's about 4,200 Twitter followers and 1,500 mailing list recipients.
Before we get into the numbers, I should tell you that I don't use Google Analytics. I don't track visitors, uniques, pageviews, none of that. I'm not sure what I would do with it. What I do track is 1) number of sales and 2) conversions/retention. That's it.
So, I announced on the weekend of the fourth. There was a corresponding surge in sales. Here's how many new subscriptions I got each day, with Monday the 4th counting as "100x." (So if I got 100,000 subscriptions on Monday--which, since you don't see me posting from a solid gold throne, you can assume I didn't--then I would have gotten 48,000 on Sunday.)
82% of these subscribers converted to paying customers. 15% cancelled the trial and 3% just didn't pay. Although I collect credit card information at signup, those subscribers' credit card didn't process successfully.
A week and a half later, just after midnight my time (PST) on Wednesday the 13th, the series was posted to Hacker News by Shirish Padalkar. It was on the front page for about a day, and was near the top of the front page for the critical morning hours of every major European and US time zone. It peaked at #7. That led to a shorter, sharper surge.
(Total: 163x, or 79% of the email's surge.)
83% of these subscribers converted from the free trial. 14% cancelled and 4% didn't pay. The only real difference between the two surges was that a lot more of the HN subscribers cancelled at the last moment. About half actually cancelled after their credit card failed to process and they got a dunning email. It makes me wonder if HN tire-kickers are more inclined to "hack the system" by somehow putting in a credit card that will can be authorized but cannot be charged. A debit card with insufficient funds would do the trick.
Another interesting data point is that "background" subscriptions--that is, the steady flow of subscriptions since February 2nd, other than traffic surges--averages 10x per day. (That is, on an average day, I get one tenth the subscriptions I got on Monday the 4th). However, the conversion rate for those "background" subscriptions is 95%. I'm not sure why it's so much higher. Perhaps those subscriptions are the result of word-of-mouth recommendations? That would make sense, since I'm not advertising or doing any real traffic generation yet.
For this service, a mention on HN is about equivalent to a mailing list of 1,200 interested potential customers and 3,330 Twitter followers. That's actually less than I would have expected.
The conversion behavior of HN'ers is about the same as the mailing list. I would have expected the mailing list to convert at a higher rate, since they've already expressed interest. But it's essentially the same.
Both surges led to significantly less conversions than word-of-mouth subscribers. Don't get me wrong--from my research, I'm led to believe that ~83% is excellent. But 95% is frikkin' amazing.
HN'ers are more likely to cancel a free trial at the last moment and use credit cards that authorize but cannot be charged.
Finally--thanks to everyone who subscribed! I hope this was interesting. You can discuss this post on Hacker News. I'm available to answer questions.
I've been working on this project for over nine months. Over a thousand people have had early access, and reactions have been overwhelmingly positive. Viewers are saying things like "truly phenomenal training" (Josh Adam, via email), "highly recommended" (Theo Andersen, via Twitter), and "*the* goto reference" (anonymous, via viewer survey).
Up until last week, the show was only available to Kickstarter backers. Now I've opened it up to everyone. Come check it out! There's a demo video below.
CITCON, the Continuous Integration and Testing Conference, is coming up this weekend in Portland, and I'm going to be there and recording episodes of Let's Play TDD! I'm looking for people to pair with during the conference.
If you're interested, check out the current source code and watch of a few of the most recent videos. Then, at the conference, come to my "Let's Play TDD" session and volunteer to pair! It should be a lot of fun.
There are still a few slots open at the conference, so if you haven't registered, there's still time. I hope to see you there!
I think "acceptance" is actually a nuanced problem that is fuzzy, social, and negotiable. Using tests to mediate this problem is a bad idea, in my opinion. I'd rather see "acceptance" be done through face-to-face conversations before, after, and during development of code, centering around whiteboard sketches (earlier) and manual demonstrations (later) rather than automated tests.
That said, we still need to test the behavior of the software. But this is a programmer concern, and can be done with programmer tests. Tools like Cucumber shift the burden of "the software being right" to the customer, which I feel is a mistake. It's our job as programmers to (a) work with the customer, on the customer's terms, so we build the right thing, and (b) make sure it actually works, and keeps working in the future. TDD helps us do it, and it's our responsibility, not the customer's.
I don't know if this is very clear. To rephrase: "acceptance" should be a conversation, and it's one that we should allow to grow and change as the customer sees the software and refines her understanding of what she wants. Testing is too limited, and too rigid. Asking customers to read and write acceptance tests is a poor use of their time, skill, and inclinations.
The gist of the article appears to be that we can effect organizational improvement in a large company by driving change from the level of individual software development teams.
He spends the rest of his article elaborating on this point, and particularly making the point that most software is built in IT (rather than product) organizations, where software teams have little ability to drive change.
It's a well-made argument, and I agree with it. Organizational change does require top-down support, and software teams in IT do have little ability to drive bottom-up change.
Just one problem: I don't see why Dave presents this as a criticism. Our article isn't about using software teams to drive organizational change.
Our essay describes how teams progress through stages of Agile fluency. It's based on what we've seen in 13 years of applying Agile and observing others' Agile efforts. We've developed the fluency model over the last year and a half. Along the way, we've reviewed it with dozens of people in all sorts of roles--team members, managers, executives, and consultants. The feedback has been clear: the model reflects their experiences. I doubt it's perfect, but the fluency model reflects reality to the best of our ability.
This model isn't about how organizations grow. It's about how teams grow. (There's probably room for an article about organizational Agile fluency, but that's for another time.) And this is what we see:
1. Teams learning Agile first get good at focusing on the business perspective. User stories and the like. (That's not to say that every team using user stories is successfully focusing on the business perspective!) This requires some cultural changes at the team level.
2. If the team has been working on improving their development skills, they get good at this next. It takes longer than the first stage because there's a lot to learn. I'm talking TDD, refactoring, and so forth. Some teams don't bother.
3. Typically, by this point, the team has been feeling the pain of poor business involvement for a while. Sometimes, in organizations that support it, the team gets good at the business side of things next. It takes longer than the previous stage because "getting good at it" means involving people outside the team. Most organizations are set up with "business" and "software" in different parts of the org chart, and this "third star" of fluency typically (but not always) requires them to be merged into cross-functional teams.
We don't say how to make this happen, just that it's a prerequisite for three-star fluency, and that these sorts of changes to organizational structure are difficult and require spending organizational capital. It can be top-down or bottom-up. (Really, it has to be both.) We also say that it may not be worth making this investment. Two-star fluency could be enough.
4. Finally, in a few companies, the team's focus extends beyond its product/projects to helping to optimize the overall system. This requires an organizational culture that likes its teams to advise on whole-company issues. Again, we don't say how to achieve this, just that it's a prerequisite for four-star fluency, and that we've only seen it happen in small companies, and typically companies that have this as an organizational value from the beginning. We again emphasize that more stars aren't necessarily worth the investment.
So, to recap: Dave argues that the individual software teams in IT cannot drive bottom-up organizational change. I agree. Organizational change must occur if you want to achieve three- or four-star fluency, but our article doesn't describe how to do so. It just says it's necessary.
Agile methods are solidly in the mainstream, but that popularity hasn't been without its problems. Organizational leaders are complaining that they're not getting the benefits from Agile they expected. This article presents a model of Agile fluency that will help you achieve Agile's benefits. Fluency evolves through four distinct stages, each with its own benefits, costs of adoption, and key metrics.
Your Path through Agile Fluency
A Brief Guide to Success with Agile
by Diana Larsen and James Shore
For over twelve years, we’ve been leading and helping teams transition to Agile. The industry has changed a lot in that time. When we started in 1999, methods with names like Scrum, Extreme Programming, and Crystal were gaining visibility under the banner “lightweight methods.” Programmers looking for faster, simpler, and more effective ways of working were the primary drivers.
Throughout the next decade, Agile grew. In 2001, prominent members of the “lightweight methods” community met in Utah, coined the term “Agile” and created the Agile Manifesto. In 2005, the XP/Agile Universe and Agile Development conferences merged to form the Agile Alliance’s “Big” Agile conference.
The community grew, too. From a programmer-centric, Extreme Programming focus in the early days, to a more inclusive approach in the mid-2000s, to a project management and Scrum focus in more recent years. What was once a grassroots effort among early adopters is now solidly in the mainstream.
Growth hasn’t been without its problems. Programmers, once the drivers of Agile adoption, are increasingly turning away from what they see as a bloated, ineffective project management methodology. Agile luminaries are posting articles such as Martin Fowler’s “Flaccid Scrum” (2009). Organizational leaders are complaining that they’re not getting the benefits from Agile that they expected.
We’ve been helping teams transition to Agile since the beginning. We’ve learned a lot over the years about what it takes to achieve the benefits promised by Agile. In this paper, we share what we’ve learned.
I work with people who want to be great. People who are willing to take risks, rock the boat, and change their environment to maximize their productivity, throughput, and value. If that's you—particularly if you're in a product-focused, entrepreneurial environment—I want to hear from you. We can do great things together.
James Shore is a thought leader in the Agile software development community. He helps development teams worldwide achieve high throughput, market focus, shrinking costs, and joyful work. James is the co-author of The Art of Agile Development (O'Reilly, 2007). You can find more of his writing on his Art of Agile blog at http://jamesshore.com.