When conducted correctly, online research can accurately collect opinions among voters, the general population and consumers.
Recently, New York Times reporter Nate Silver took to task an online voter survey conducted in South Carolina. Before Citing a Poll, Read the Fine Print.
Yes, caveat emptor (let the buyer beware) is an appropriate cautionary note for any research. But what factors should decision-makers look for to determine if an online survey is valid or not.
CFM has found the following are key elements to conducting successful and accurate online studies.
- Cast a wide net for emails. For general population surveys we collect emails addresses from a variety of valid sources, such as e-billing, e-newsletters, website registrations and existing online panels. Using multiple sources for email addresses makes the final email list representative of the community.
- Diversify sources. If appropriate, use several online panels for community surveys. Each panel has its strengths and weaknesses. Using several sources helps avoid potential biases inherent in all commercial panels.
- More is better. Sample size for online surveys are significantly larger than those used for telephone surveys. By interviewing thousands, rather than hundreds, participants tend to be representative of the community.
- Bless the Census. The U.S, Census website, American Fact Finder, is a treasure trove of information. Once an online survey is completed, we compare demographic characteristics of participants with those of the general population. Matching results confirms we have interviewed a representative sample.
- Compare results. To validate online results, CFM will frequently conduct a parallel telephone survey, using the same questionnaire. That way we can compare findings to increase client confidence in the results.
- Sweat the details. Review survey results closely for discrepancies. If, for example, there are significantly more women than men participating in a voter survey, you have a problem. Using sample quotas can help avoid the problem of biases results.
CFM has been conducting online research for nearly a decade. We now conduct as many online surveys and focus groups as we do traditional phone and live focus groups. It is important to understand the strengths and weaknesses of any research method. CFM does and is transparent in explaining those to clients.