Why Representative Samples Really Matter

If you want market research that matters, make sure the sample of people in your survey matches the audience you want to reach with your product or message.

If you want market research that matters, make sure the sample of people in your survey matches the audience you want to reach with your product or message.

A favorite story involves meeting with a client interested in promoting first-time homeownership. I mentioned the need for market research. No problem, the client said, we have that covered. I was handed the research summary and, as a matter of habit, jumped to the page about the telephone survey sample. It was very revealing. 

More than 50 percent of the respondents were 65 years or older. They were the majority of people who answered the phone and were willing to spend 15 or 20 minutes talking to a stranger about owning a home. Unfortunately, they weren’t the people the client had in mind as first-time homebuyers. 

Survey data is worthless unless the sample of who you interview reflects the audience you seek to reach. The sample in my client’s survey would have been terrific if the subject was reverse mortgages. It stunk as a reflection of who to address potential first-time homebuyers. 

Conversations between clients and research professionals must start with who to interview. If you have the wrong sample, the answers you get from the questions you pore over won’t matter a lick. 

Too often, the question of who to interview is glossed over. Sometimes the most obvious sample goes overlooked. When I was a lobbyist, a client hired me to “fix” his message that wasn’t gaining any traction with legislators. I started by interviewing about a third of the legislature, including virtually all of the lawmakers on the committees that were most engaged on my client’s issue. 

The interviews produced a wealth of insight. My client’s issue had latent support, but needed to be explained and demonstrated in a far different way. Lawmakers basically wrote the script my client and I used to lobby them. And it worked. 

Representative samples are harder to achieve for a mix of reasons. For example, increasing numbers of people don’t have landline phones and, if they do, they shield themselves from unsolicited calls with Caller ID. It takes a lot more calls, at greater expense, to collect a representative sample. Market research must cope with growing segmentation, which adds extra layers of complexity in selecting the right group of people to survey. 

The value of representative samples goes beyond quantitative research. Focus groups must be representative, too. And why would you do a customer satisfaction intercept survey for Nordstrom by interviewing people coming out of a rival department store? Representative samples matter in public opinion polling. A poll of New York voters wouldn’t be all that useful in projecting election results in Indiana. 

Despite the difficulty, solid research is grounded on good samples. Who you talk to matters if you want findings that mean something for your marketing.  

Gary Conkling is president and co-founder of CFM Strategic Communications, and he leads the firm's PR practice, specializing in crisis communications. He is a former journalist, who later worked on Capitol Hill and represented a major Oregon company. But most importantly, he’s a die-hard Ducks fan. You can reach Gary at garyc@cfmpdx.com and you can follow him on Twitter at @GaryConkling.