Make Testing and Evaluation Your Best Friends
by Adele Sommers
Once your company designs its products and services, how should you oversee whether your offerings are actually performing the way they're supposed to? Do you have a systematic method for ensuring that what you develop results in a smoothly functioning product or service?
To begin with, you would want to use a specification to describe what your product or service is intended to do, and have a way to continually compare your product or service against that specification to determine whether it actually 1) does what it's supposed to do, 2) does it correctly, and 3) as advertised. This article explores why testing and evaluation are so critical, and dispels two pervasive myths about testing.
The Importance of Testing and Evaluating Your Offerings
A great product launch can take an undesirable turn if there's no process to ensure that your product does exactly what you and your customers expect. Are the features muddy and maze-like? Do they require users to go around in circles trying to follow the instructions? Are there bugs and inconsistencies throughout? These are signs of trouble! Remember that your customers will have demanding expectations -- just as you should have as a consumer.
 Related inquiries you could make include: Are your products and services confusion-free -- even if they carry out complicated tasks? Also, how well do they perform the actions they're supposed to perform?
This is where testing and evaluation techniques can become your best friends and your "secret sauce." The earlier in the life cycle this process can begin -- specifically, in the design stage, when the initial concepts are still on the drawing board -- the more successful your products or services will be at satisfying customer needs and desires. An early starting point lets you build quality incrementally into your offerings, instead of trying to add it as an afterthought, the way your competitors might.
Testing is no trivial undertaking, however. Below appear two myths about testing and evaluation that highlight the pitfalls of underestimating these efforts.
Myth #1: Testing Complex Products Is Very Straightforward
Just the opposite is true -- the more complex a product is, not only is it more difficult to use, it can actually become too complicated to test.
The reason for that phenomenon is: The more features you have, the more likely that there will be too many combinations of features to test. When something is too complicated to test, no quality assurance program in the world can manage it.
Are you wondering why that matters? Perhaps you're thinking, "Who cares how many features there are? If we just make sure that the main features work, the customers ought to be happy, right?"
To provide more insight into this issue, I recommend a business novel called "Necessary But Not Sufficient" by Eli Goldratt. In it, the fictional characters explore the problem of complex systems and the pros and cons of how to design, test, and maintain them.
 One main theme of this book exposes why an exceedingly competent software development team suddenly cannot figure out how to continue to maintain a highly successful but extremely complex software product. It was the direct result of customers asking for more and more features, and the development team accommodating them.
Every added feature set increased the combinations of functions and interactions within the system exponentially, rather than in a linear fashion. Eventually, no one could identify, much less test, all of the feature permutations.
That's the problem with complex systems. You can reach a point where there are too many sets of variables to test in your lifetime, much less within the time available to release the product!
But let's assume that you are dealing with something less complicated. In any case, if you believe that your offerings ought to be as robust and error-free as possible, then you should consider testing early in the game -- as early as the design phase -- to evaluate the degree to which the functions and features are simple, elegant, intuitive, and correct.
Myth #2: "But It's Fine to Wait Until the End!"
Really? As you can imagine, waiting to catch all of the problems at the end is sort of like hoping to count all of the water droplets that flow over Niagara Falls. The effort can be overwhelming! Testing is lot more about prevention than about cure, which means doing much more in the early stages of the life cycle when most people don't even want to think about it. Instead, they're hoping to test everything just prior to releasing the system, which is like trying to put icing on a bug-riddled cake.
 By the time the end rolls around, the chances of finding -- and successfully fixing -- all buried defects are remote. The expense of fixing a bug, for example, grows dramatically with any delay in either discovering it or correcting it. By many estimates, the costs are 10 to 100 or more times greater after deployment than if the team had found and resolved it much earlier in the life cycle.
Contributing to this price includes redoing anything that was mistakenly undone as a result of trying to fix the original problem. What does that mean? Well, frequently, any attempt to tweak one area of a system can accidentally "break" something else that was working correctly. This happens so often that there's a special category of testing designed just for detecting such mishaps, called regression testing.
In conclusion, please don't underhandedly transfer to your customers the primary burden of finding and reporting system problems. And whenever someone does report a bug, ask yourself the following:
- How could we have automatically detected this defect?
- Better yet, how could we have prevented it?
By making testing and evaluation your partners and allies, you will give your brand and offerings enormous competitive advantages. Related articles on this topic will explore specific techniques that you can use to achieve outstanding results.
Copyright 2007 Adele Sommers
|
Today's issue is about the quality of your products and services, and a recognition that as you increase quality, you also increase the value your customers perceive in your offerings. Perceived value ranks highly among the factors that people use to make buying decisions.
Isn't it interesting how our perceptions rule our beliefs and actions? So much of the brain research today seems to support the idea that our perceptions define our reality. Is the glass half empty or half full? The definition resides in your customers' eyes! 
"
