Missing the forest for the trees

Understanding customer needs for being able to do good testing sounds like commonplace that doesn’t need a special discussion. And testing projects often omit to discuss them indeed. Testing teams often start their project involvement with reading functional (or non-functional) specifications, skipping the customer view. I’m still mentioning it on the first place in the introduction of approaches to

tackle testing complexity

.

Imagine that you need to test the database that we discussed in the same

article

. You got to make choices, which of the possible combinations you test and which of them not, given you cannot realistically cover all of them. Imagine that you know that 98% of your customers are small and mid enterprises with up to 1.000 employees. Naturally, you might conclude that the majority of your customers will prefer:

Given these requirements, you might conclude that your testing should focus on 1) quality of small scale deployments, 2) superb installation experience and on 3) ease of serviceability. This is of course an oversimplification of all the factors that one needs to consider in the real world, but I hope it is good enough to make the point.

At the same time, you might not know that the minority of your customers (2% represented by big enterprises) are generating 98% of your revenues. Such a ratio isn’t a rare case in the real world. Now what? This is completely changing the game. High-availability likely becomes a priority. Usability of installation is nice, but perhaps not that important as one would think, because those big customers prefer to employ external vendors, who are experts in doing deployments and who may prefer interfaces that enable automation of large scale deployment over an easy to use graphical, but manual interface. Those customers are also likely to engage an external service vendor (most probably the software vendor itself) to provide support services, so they don’t need to waste time on it. Consequently, testing will focus on completely different targets than you might have thought based on the initial information.

Understanding your customer can make a huge difference, how good or bad you are in testing. You may easily miss the forest for the trees, when jumping right away on analyzing functional specifications or, God forbid, on test automation without understanding your customer. The trick is that it is often difficult to find reliable and usable customer data. Anyway, try to do your best to find them. It’s worth the effort. Try to talk to marketing, support, professional services or any other applicable departments, if data are not readily available. They will not necessarily provide you with hard customer data, they may still be able to give you some soft metrics and a useful perspective on the way your customers use your product.