Exploring Some Analogies for Testing

Testing is Kind of Like Cooking

Testing is an activity. Anybody can test. Sometimes testing is just using a product.

But testing is also a complex domain requiring deep knowledge of tools, techniques, and analytical skill. Testing is often used as a tactic to achieve goals in an overarching strategy driven by company goals and business priorities. Ok, it just got serious; let's back up a bit.

Let's say that you can cook (me, my signature dish is chili, which my family says they like). Say that you are pretty good at keeping your family happy with reasonably appetizing dinners. You've been cooking for years, even; you extend your repertoire by adding new dishes that you picked up from cookbooks, cooking shows, and maybe that you deconstructed from dishes at restaurants. Let's say that you enjoy cooking, it's a hobby; you devote time and thought to cooking. You know your ingredients and have a well-stocked pantry, and you know your tools and hand-sharpened knives and state-of-the art cooking appliances.

Cooking is an activity, albeit one where some planning and skill of execution is necessary. Your goals when cooking are focused on providing meals that are nutritious and taste good and accommodate your family's dietary needs.

Let's expand the scope: given that you can cook dinner for your spouse and kids, could you run a restaurant? Could you plan weekly menus, source the supplies, manage freshness and supplies turnover, set up a cooking line, design the menus, train staff, set up and manage front and back of house teams, etc., and deliver deliver deliver. For your restaurant, you will need to define goals and strategy for service, marketing, expenses, and revenue. You will have to use the activity of cooking in some highly-formalized ways to help implement tasks that support your strategy; cooking the way you cook your family dinner just won't cut it, no pun intended.

In fact, cooking is just one activity involved in the domain of doing restaurant stuff — I want to call it "restauranting", but that's not a word, right? Ok, "restauranting" involves a hierarchy of roles, each with their own core activities. Some of these activities may be cooking, but maybe with a different focus or emphasis.

Testing is Kind of Like Wrenching

I'm a bicycle mechanic, self-trained before going through formal training and certification as a Master Bicycle Technician; wrenching is what we call the activity of bicycle mechanic-ing (because it's a little easier on the tongue).

Wrenching requires a good understanding of the mechanical aspects of bicycles, for example drivetrains, braking systems, and contact surfaces. Some things are straightforward enough for most people to learn, like changing tires and adjusting the drivetrain shifting. It's possible for anyone to learn many of the necessary techniques to keep their own bicycle in reasonable tune.

However, things get more complicated very quickly if you need to deal with more bikes, new technology and materials, or the more complicated parts of the bike (wheel building and hydraulics, I'm looking at you).

Let's expand the scope: given that you can wrench on your personal collection of bikes (say you have a road bike and a mountain bike), could you wrench in a bike shop? Are you able to diagnose and fix problems on a range of used bikes that customers bring to the shop? Are you able to assemble and tune new bikes from their boxed parts? The bike shop has to establish goals and strategy for service, supplies, marketing, and revenue, and it's difficult to make a profit on the labor of wrenching, so you need to be accurate and fast. Time is money, but the bicycles have to, well, work.

Wrenching is one activity involved in the business of running a bike shop; you also have fitting (the customer to the bike), parts and stock ordering, repairs management, merchandising, and you get the picture. There's a hierarchy of roles, along with role-specific activities, of which wrenching is a important activity.

Quality Control and Quality Assurance

A quality control program involves inspecting the product and evaluating against some set of quality criteria.

In our cooking analogy, this would mean examining the presentation of the meal, and seeing if it tastes correct (assuming it tastes good, too). A quality assurance program would be looking into changes in the kitchen and the food preparation process to correct problems with the dishes' presentation or taste, AND finding ways to improve it even if it is already pretty good.

In our wrenching analogy, quality control means verifying that the bike builds were done correctly. If good quality controls are in place, procedures like bike builds follow a checklist with specified tasks in a specified order. Some steps will tell the builder what lubricant to apply between surfaces or what torque setting to use when tightening a bolt. The quality of the build is measurable and is tied to the correctness of the execution of the defined steps, and is finished off with a test ride to ensure that the whole system of the bike works correctly.

It was a quality assurance program that determined the build steps and order, surface treatments, torque settings, etc., because QA looks to eliminate problems and improve the process through which products are created. It's a QC program that says whether the finished product is good enough, and what good enough means.

Quality assurance is a focus on measuring and improving the quality of some product. QA looks at the product itself and the processes and systems that produce that product. QA does use tests and testing as part of its toolbox, but it takes a high-level strategic look at the problem of testing that product in that domain in order to identify what could be tested and to define what will be tested. Testing is an activity, while quality assurance is a program that uses testing strategically, after planning how and what to test, wait for it… strategically.

Back to Testing

Ok, testing is unlike cooking and wrenching because testing at its base is about learning about software; the direct product of testing isn't something you can eat or ride. But the relationship of the activity to the complicated process that employs that activity to support goals and strategy does fit.

Testing is an activity, but the value of testing increases as expertise, experience, patterns awareness, and strategic planning are employed around testing. Professional testers bring to the activity a frame of reference and adeptness beyond what a hobbyist tester can bring to the game. In software development, any testing is better than no testing, but professional testing is probably better than unfocused testing.

In a software development process, testing is used to evaluate whether the software works (this is a painful simplification, on the order of saying that cooking is used to make sure that the food is "done" — there's a lot more that needs to be said here, but not now). Software quality control looks to the product's quality by identifying defects; software quality assurance employs quality control to look for those defects, but also looks to eliminate defects, or at least reduce them, by improving the software development process. Testing is an activity used in pursuit of the bigger picture of quality.