Most Market Research Is Horse Shit.
Why customers couldn't have told you they wanted a Dyson vacuum.
When you google market research you’ll find things like focus groups, online surveys, and phone interviews.
If you listen to your MBA professor he’ll tell you about trend reports, market statistics, sales data and industry content.
Most market research is about as useful as a spoon with a hole in it.
The main reason you think it is, is because firms do this for a living.
So it doesn’t matter if it doesn’t work.
As long as you pay for it and you BELIEVE it’s useful, they’ll continue to make money with this sophisticated con.
And because so much of this is subjective (even though it’s positioned by those firms as objective) it can be difficult to find out it’s useless.
(This reminds me of when all of the hired consultants from prestigious firms such as McKinsey, did market research for Howard Schultz and informed him that opening the first ever international Starbucks in Japan would be a massive failure. He ignored them, launched, and before they officialy opened there was a line around the block. Oh and they were selling hot coffee… during a heatwave. So much for market research. You can listen to the story here.)
One of our first clues that most market research is BS is to look at businesses that spend a lot of time and money on it vs. businesses that use something like a design sprint.
The second clue is that so many companies who’ve invested and relied on market research fell flat on their ass, like Microsoft’s Steve Ballmer during the smartphone era.
And we all know how important Microsoft has been as a player in the smartphone market since we’ve all owned a windows phone.
Just kidding.. that was a jab for the lulz.
They lost billions on their windows phones and shut down their smartphone business 5 yrs after they bought Nokia for 7.6 billion dollars.
Fortunately though, they did better with their mp4 player The Zune.
By better I mean they lost a few billions less.
.They launched a stunning 5 years after the iPod and lost over a quarter of a billion on it.
If you wait for market research, at best you’ll never be at the bleeding edge, at worst, it’ll lead you astray.*
The biggest problem with market research is that it all comes from the same place: the past.
Rory Sutherland (Vice Chairman, Ogilvy)
What happens, when we incentivize market research so it’s not asymmetrical anymore (meaning you get rewarded regardless of outcome)?
What if, if your research fails, you fail (in your wallet) and if it succeeds, you win (in your wallet)?
That way, people advising you on market research would have as, Nassim Taleb so often stresses, ‘‘skin in the game’’.
That’s exactly what Google Ventures (GV) did. They invested in a ton of startups, so they were incentivized to help them succeed as opposed to market research firms who only care about extracting your hard-earned money.
They quickly learned how useless market research is and as a result, over time shrunk down the industry standard from many months to what is now known as a 4-day design sprint.
ONLY 2 days of coming up with hypotheses, 1 day of building experiments and 1 day of testing it out.
Market Research For Startups
One of the biggest mistakes market research firms, professors and basically everyone makes is to think that a startup is just a small company and should, therefore, do what a large company does albeit at a smaller scale.
They don’t realize that they do market research by gathering data not because they want to but because they don’t have a choice.
They can’t do the most effective things anymore because they’ve outgrown them.
You can talk to all your users, they can’t.
In the early days, you can talk to your entire market.
You show them what you’ve got, get feedback, iterate and go back.
To this day, this is still how most startups start and this is what you should do as well.
There’s such a temptation to copy big companies, partly because that’s what everyone tells you to do and partly because it’s scary to be vulnerable and present your product to users.
Market Research In Sophisticated Markets
But the problem is that as you become more successful and the market becomes more sophisticated, this becomes harder.
It can tell you what your customers think of something you present to them, or what they want, based on what they already know.
But they can’t predict something they want that they’re currently unaware of.
Said another way, your customers during the time horses where the main form of transport, wouldn’t have told you they wanted a car.
Just like a doctor wouldn’t view himself as a pill prescribing machine and expect the patient to come up with his own diagnosis, we shouldn’t expect our customers to do our job.
(Matter of fact, oftentimes customers will laugh at your innovation at first. The early adopters use it. It crosses the chasm to the mainstream and finally the late adopters. During this process, no one seems to realize that they were idiots so cognitive dissonance and the accompanying embarrassment never ensues. Although it’s fun with Twitter etc. that one can now hold such people accountable when they so overconfidently call things dumb.)
Customers couldn’t have predicted the Dyson.
Market research would’ve led you to a better hoover vacuum.
Why? Because those companies didn’t give a shit about making a better vacuum. Only to the degree, it allowed them to make more money.
(Don’t believe me? Listen to James Dyson’s story about what happened when he tried to license his invention to them. Spoiler alert: they laughed him out of the room because a user constantly in need of a new, expensive bags is quite the business model.)
Our customers aren’t trained in innovation. They’re engineers, professors, teachers, nurses, plumbers etc.
They have busy lives. So when you ask them that question.. they’ve likely never given it much thought before.
So now they’re devoting a few moments to think about a better vacuum.
So what does the lazy human brain do? It goes to what it knows (the vacuum) and makes some tiny tweaks.
Market Research can tell you what people don’t like, and it can be helpful for small incremental jumps.
Not massive leaps.
I’ll close today’s essay with Steve Jobs’ take on market research:
*There’s one exception though. You can wait until the market has developped a bit more, but when you launch you have launch something people will love. Apple hasn’t always been the first mover. Yes they were first with the notch and many other things that people laughed at and other companies now copy.
(Sidenote in the middle of my sidenote: Samsung ran ads poking fun of Apple’s notch and lack of a headphone jack etc. and now that they’re cloning all those features, they’ve quietly deleted all those ads off of Youtube.)
But they were late with both the Apple watch (Pebble launched 2 yrs earlier), as well as their wireless airpods (Bragi Dash launched a year earlier).
However, when they released it, they shipped something that was better. Customers wanted it. This is something that Microsoft didn’t do. They not only were late, but they also shipped something people didn’t love. That’s a recipe for disaster. In a sense Dyson was late to the vacuum market, but his vacuum was so much better in the eyes of the users that they wanted it.
I wrote about the fallacy of the first mover advantage in Resisting The Siren's Song.