Our Blog

Bad Customer Development Questions and How to Avoid My Mistakes

Although Customer Development can give us tremendous insight into market problems, it takes a lot of time – time that’s wasted if we do it incorrectly. Worse yet, poorly worded questions can cause us to reach wrong conclusions about what people want.

The best questions don’t require customers to speculate about their behavior. Kevin Dewalt shares real examples of his bad questions and mistakes and offers some better alternatives.

If you’re starting Customer Development you’re getting ready to talk to a lot of potential customers. You started with an an idea, wrote down your key assumptions, and started flipping through contacts to see who you can interview.

Awesome! You’re off to a great start. Talking face-to-face with customers brings us insight we can’t get from surveys and clicks.

Unfortunately, conducting face-to-face customer development interviews is a skill that takes practice. I’ve been doing it for 5 years and I’m still learning. So many times I’ve asked the wrong questions and later realized I was wasting time or – worse yet – coming to incorrect conclusions and building the wrong products.

“Learn Nothing” Questions

I call “Learn Nothing” questions those that don’t result in any learning, just wasted time.

“What do you think of my idea?”

I LOVE talking about startup ideas. To quote my friend Patrick Smith , “talking about startups is entertainment like sports”. Fun, yes. Validated learning? No. What do you think of my startup idea? is useless because if my idea is great people will like it. If it sucks, surely some people will still like it.

“If you could wave a magic wand…”

I know Steve Blank calls this the “IPO question”. I call it the “sit back and watch people ramble about things I have no chance of building” question. I’ve asked this question at the end of meetings and – after some funny looks – watched somebody ramble. It just hasn’t worked for me.

Too Broad Questions“Can you tell me about your problems with medical bills?”

I asked this question a few times before realizing that asking people to talk about problems just results in venting and no learning. This is particularly true with complex, personal, emotionally charged problems like those in health care. “Well there was this one time….and then…but what really got me mad…”

“False Positive” Questions

Wrong conclusions are worse than no learning at all, and I call “False Positive” questions those designed to get customers to tell us what we want to hear.

All of us want our product ideas to be right – I want it, you want it, Steve Jobs wanted it. While our enthusiasm is our biggest asset, it is dangerous in customer development because most people don’t want to disappoint us.

Leading Questions

I’m building a product to help people manage medical bills. Can you tell me…”

Any question that starts with the solution already biases people’s expectations. Someone hearing this question might focus on a minor problem they had with medical billing in the past and convince us that it is a major source of pain.

Better: Skip the “I’m building a…” intro.

Questions that Put People on the Defensive

“How do you reconcile your HSA account with your bills, receipts, and statements to make sure you’re optimizing future tax savings?”

The “How do you do something complex to achieve results” questions can put people on the defensive. You can imagine someone thinking, “You mean I’m supposed to be doing something with that paperwork to save money on taxes? Oh no, I’m such an idiot, what am I doing wrong?”

Of course customers will be biased to tell us they need help with something after we cause them to doubt their own competence.

Better: “What do you do with that HSA paperwork?”

Questions Leading to Answers you Want to Hear

“Would you take a picture of a medical bill with your iPhone?”

This question sets up the prospective customer to tell us what we obviously want to hear.

Better questions would give us insight into how they currently work and whether the extra steps and inconvenience of using our products gives a promise of a big return.

Better: “Do you scan or file medical bills you get in the mail?”

This question is better because it gives someone the opportunity to disappoint us without realizing it, what Rob Fitzpatrick http://vimeo.com/40192415 calls the “mom test”.

In this case, suppose your key assumption is that managing bills via in iPhone app saves time over filing or scanning. Should they respond, “Scan them? I’m too busy for that, I don’t even open them,” you’ve just invalidated your whole company vision with one question.

The Key to Effective Customer Development

In retrospect, the root of my bad questions was that I only had an idea – I didn’t have specific hypothesis I was trying to validate. I tried to replace the hard work of documenting and testing assumptions with meetings and simply wasted time.

Always remember that Customer Development is a big commitment that takes a huge amount of time, time best spent on something else if you’re not doing it right.

Before you line up a bunch of meetings and spend weeks talking to people, take some time to carefully consider what you’re going to ask and why. The key is to make sure your questions are designed to test your assumptions. I write them down and you should too.

“Bad Customer Development Questions and How to Avoid My Mistakes” was originally published by KevinDewalt.com on January 21, 2013.
By: Kevin Dewalt, 20 years as a startup hacker, founder, & investor.