Beyond Defects

I was once involved with a product that was delivered on time, on budget, exactly to specification, and with an extremely low defect rate. It was an enormous technical success... but it misjudged its market and nobody used it.

Another company judged its market well and delivered a first product that was a big hit. Over time, though, the software became increasingly difficult to modify. Now they're struggling.

These examples show that quality comes in several forms. Although many people think of quality as "number of defects," defects are really only a symptom. In this newsletter, I'm going to talk about how to find and solve the quality problems that underlie defects. One is lack of market quality.

Your product's market quality is judged by your customers: the better it meets their needs, the higher its quality--and the more it will be used. The first example above was a solid product from a technical perspective, but it didn't do what the market needed. It had design quality but lacked market quality.

In order to achieve products with high market quality, you need a product manager who has a strong vision for the product to be created. This person needs to understand the market intimately and be able to make tough trade-offs between delivery date and scope. You also need to make sure that the product manager's vision is reflected in the software. Requirements documents are the typical way to do so, but they're insufficient. For something as complex as software, nothing beats face-to-face communication. As much as possible, your product manager should work full time with the development team, communicating his or her vision and answering questions about details. The time your product manager spends with your team will have a direct impact on the market quality of your software.

Market quality is crucial for a product's acceptance. Design quality is equally vital because it translates directly to costs. The product in my second example did well initially but eventually became too expensive to maintain. It had market quality but lacked design quality.

I've discussed techniques for increasing design quality before, so I'll just summarize them this time: make developers aware of the importance of design quality, train them in continuous design techniques such as test-driven development and refactoring, and hire technical leaders with passion for high-quality design.

Defects are an easy way to judge quality, but they're a late indicator. Products with poor design often have a lot of defects. Teams sometimes react by increasing the amount of testing, but that's a costly solution that doesn't attack underlying design quality problems. In addition, defects more often speak to design quality than market quality.

However, defects are a handy diagnostic tool, and you can use your existing defect list to start attacking underlying quality problems today. Find the common thread:

  • Customer complaints about properly functioning features indicate a market quality problem. Improve your market understanding.
  • Features that work properly but aren't what you expected indicate a market quality problem. Improve communication between developers and product management.
  • Software that is prone to defects ("buggy") indicates a design quality problem. Improve continuous design efforts.

It's best to solve quality problems before they result in defects, but that's a subject for another time. For now, ask yourself which of the above problems are most common in your organization and use the above list to start making changes that will have a lasting effect on quality.

If you liked this entry, check out my best writing and presentations, and consider subscribing to updates by email or RSS.