Give your Hypothesis the “Silent Treatment”


Dan Adams In many areas of life, there’s the “old way” and the “new way.” Same with product development. Does your company still develop “hypotheses” in internal ideation sessions, and then meet with customers to validate them? If so, are your sales reps still wearing fedoras? Just kidding… but let’s consider why this is more than a style issue.

You’ve just had your performance review with your boss. He said positive things about your work in 7 areas, but had some criticism—sorry, make that “opportunities for improvement” in 3 areas. Did you spend much time challenging him on the 7 favorable areas? Why not? Well, because your boss got that part right, of course.

This is called confirmation bias, defined by Wikipedia as the “tendency for people to favor information that confirms their preconceptions or hypotheses regardless of whether the information is true.” We all do it, and rarely notice. But any form of bias should concern us. By definition, bias prevents unprejudiced consideration and promotes systematic distortion of data. Now that can’t be very helpful, can it?

Confirmation Bias

If you experience confirmation bias trying on a new pair of jeans, I’m thinking we can all live with the consequences. But consider the new products your company develops for customers—the lifeblood of its future existence. What if you’re preventing unprejudiced consideration here? What if your company is systematically distorting data on customer needs?

What if your company is systematically distorting data on customer needs?

This is precisely what your company is doing if it develops products the “old way.” Your organization has lots of company. Most suppliers have a stage-and-gate process… which can bring a helpful level of discipline. Many of these processes start with an “idea generation” stage… perhaps accompanied by a light bulb picture.

Try asking these companies, “So, whose idea is being generated… yours, or the customers?” The answer is nearly always, “Ours.” When AIM begins working with new clients, virtually all of them—regardless of industry or region of the world—say they focus on their own solutions before they fully understand customer needs. Further, when asked—OK, nudged at bit—they admit their best understanding of market needs comes when they launch their product and see if anyone buys it.

This is a problem. Your new product development process should start where it ends: with the customer. When you take your “pride and joy” hypothesis to customers and ask for their opinion, two very bad things happen: 1) They tell you what you want to hear. 2) You hear what you want to hear.

Your new product development process should start where it ends: with the customer

“Oh, no,” you say. “We’re careful to avoid confirmation bias.” Really? How long have you had that fedora? It fits nicely. My suggestion: Start believing your “hypothesis validation” data when you begin challenging your boss’s compliments during performance reviews.

The Silent Treatment

So what’s the “new way”? Throw out any ideas you have for customers? Ignore that promising new technology quivering on your lab bench? Treat your new product concepts as evil thoughts to be purged?

Nope. I love it when our clients have cool technology and clever ideas. But don’t tell customers about your ideas during interviews. From the customer’s perspective, a Discovery (qualitative) interview should look exactly the same whether or not you’ve got a great hypothesis. Give your hypothesis the silent treatment for now. Just listen to the customer.

What happens during interviews using this new approach? The customer senses you are more interested in their needs than your ideas… and gives you many “outcomes” or desired end-results. During the interview, you might notice these outcomes are a perfect match with that hypothesis you’ve been suppressing in your prefrontal cerebral cortex… or perhaps not.

If you have a great outcome-to-solution match, you’ve got unbiased confirmation you’re on the right track. But play it cool; you can do a victory dance in the parking lot later. If the customer is asking for different outcomes than your solution delivers, you must look for a different solution. Take heart: You probably saved your company a fortune in otherwise-squandered R&D.

Leave a Reply