Beta Testing, Anyone? by Adele Sommers

In a hurry? Learn in small bites! Sign up for my free, 7-day e-mail mini-course, and I'll dish up the basics in just one week!

Take the business prosperity Treasure Hunt, then claim your free gift!

Beta Testing, Anyone? 10 Potent Strategies for Achieving Success
by Adele Sommers, Ph.D.

Successful beta testing starts in the earliest stages of project planning. Why is that? Because beta testing is meant to involve a methodical prove-in of a carefully planned system. It's not meant to be a hit-or-miss, cross-your-fingers-and-hope-everything's-OK Band-Aid that you can apply at the last minute. This article (Part 3 in a series; see Part 1 and Part 2 here) discusses ten tips for performing this crucial activity. But first of all, why is it so important?

We've all seen examples of products — even some from well-known, respectable companies — that arrive on our desktops barely breathing. They seem to be so full of bugs that they cause us more grief than they help us carry out work. Or we try to use a Web site that looks great, but we can't get from the shopping cart to the order page. Or we buy a new device, yet even using the instruction booklet, we can't carry out critical functions the way we're supposed to.

Are you anxious to frustrate your customers this way?

Of course not! That's why beta testing involves such a systematic, tedious, yet indispensable sequence of activities.

But without a method to the madness, you are only randomly banging on the system to see if by chance you can find a way to break it.

What should you be striving to accomplish? Using the checklist that appears below, you will want to determine whether your product or system will:

Function without bugs or snags. The test procedure should validate the features embodied in the system against those called for in the specification. It also should call for testing extreme data as well as typical data, and test the features in combination with one another in many different sequences. These permutations represent how actual customers will behave — unpredictably! A system should either prevent, or smoothly respond to, awkward situations.

Guide people to achieve their primary aims. Beta testing can examine the "big picture" aspects as well as the detailed minutiae. Can users successfully complete their high-level goals using the onscreen guidance and user guides?

Handle extreme circumstances gracefully. Your product or service should deliver helpful, intelligent responses in any extreme conditions, such as when people might be stranded, working alone, or battling the elements. At the very least, the system should cause no confusion and inform users of their options.

Beta Testing Checklist

Below are 10 strategies for achieving success with planning the beta testing effort, carrying out the process, and keeping the testers happy.

1.	Design test scenarios. What's a "test scenario"? Each test scenario should be mirror image of a "use scenario" that's been guiding a team to design and develop the system. A use scenario describes one typical interaction a customer has with the system. For instance, for an automated teller machine (ATM), one scenario could involve a customer inserting a card in order to withdraw some cash. In another scenario, a customer makes a deposit. In another, he or she verifies the balance. Scenarios must represent any plausible ways in which users could interact with the system, including *unusual* and *unintended* actions. So both use scenarios and test scenarios should account for possible error conditions such as jammed cards, cancelled transactions, or overdrawn accounts.
2.	Write a test procedure. A test procedure specifies how testers will exercise the test scenarios, including the order to follow, as well as running several tests in random order. It should also explain what results to expect in each case. You will want the procedure to test all new system features or changes. The procedure should also specify testing features in various combinations. For example, you might specify 1) withdrawing cash, then 2) verifying balance information, and then 3) making a deposit. Be sure to vary the order, and test error conditions.
3.	Determine what data you need. If your system stores values in a database, you will need to pre-load some typical data to test the scenarios. In the ATM example, values would include account balances -- for testing withdrawal limits and giving balance information. Create the sample data sets and pre-load the systems to be tested. Don't forget to include extremely high and low values!
4.	Plan specific roles for testers. Schedule each tester to focus on specific test scenarios and related data sets. If there are enough testers, assign more than one to cover each test scenario. Each person will approach it differently.
5.	Create a bug reporting system. It could be form-based, a database, an e-mail method, or a combination. Have testers submit bug reports as they find errors in each round of testing.
6.	Establish a test schedule. The schedule should allow for several iterations of beta testing. Be sure to clear the schedules of testers for each round in which they will be participating.
7.	Get all materials ready for testing. The following items should be ready for the kickoff: A new or updated system, lists or descriptions of any bugs fixed, new or updated documentation, test scenarios and procedures, and so on.
8.	Set a start date and schedule progress milestones. After the kickoff meeting, if testers find numerous bugs — or especially critical ones — before reaching a given milestone, stop testing, fix the bugs and/or documentation, and return to Step 1. Ask before restarting: Are new test scenarios or data sets needed?
9.	Perform a new round of testing for each new test baseline. This means starting the complete test *from scratch* after each round of fixes. You can't sidestep this requirement, because each time something is fixed, it can "break" something else. Stop the cycles of testing only when no new bugs are evident.
10.	Plan a reward for a job well done. Testing is very tedious -- so testers need a special incentive to keep them focused on the goal. Although they're helping to produce a high-quality system, a post-testing party wouldn't hurt morale!

Thorough beta testing is essential for producing quality systems. However, if you discover errors you can't fix in time, you could decide to release a system with known defects (documented in your "Read-me" notes). The stakes can be high, so weigh this option carefully before proceeding.

To download the related checklist, click here.

~~~~~~~~~~~
About the Author

Adele Sommers, Ph.D. is author of “Straight Talk on Boosting Business Performance: 12 Ways to Profit from Hidden Potential.” To learn more about her book and sign up for more free tips like these, visit her site at www.LearnShareProsper.com

This article may be distributed freely on your Web site, as long as this entire article, including the links and full “About the Author” section, are unchanged. Please send a copy of, or link to, your “reprint” to Adele@LearnShareProsper.com.

1,023 words

Return to the Free Articles index

Adele Sommers

Adele Sommers, Ph.D.

Receive “just in time” tips for tuning up your business performance, sent to your in-box every 2 weeks!

Yes! Send FREE Boosting Business Performance advice!

Explore the award-winning “Straight Talk” series by Adele Sommers, Ph.D.

Contact Information