Scrubbing Unintended Bias from Research Surveys

 Unintended bias can undermine survey research, rendering findings as unreliable. Best practices, starting with clear and objective questions, is the best way to ensure survey findings are useful and actionable.

Unintended bias can undermine survey research, rendering findings as unreliable. Best practices, starting with clear and objective questions, is the best way to ensure survey findings are useful and actionable.

“Bias is the mortal enemy of all surveys,” says SurveyMonkey. It turns out bias is a sly enemy that can sabotage meaningful research findings.

SurveyMonkey offers tips on how to “promote honest answers” from surveys, which all depend on the good intentions and sound practices of the survey creator.

“One of the leading causes of misleading survey data is researcher bias that comes directly from the survey writer,” according to SurveyMonkey. “This bias is sneaky. It’s caused by survey creators who innocently influence the results to reach an outcome they hope or expect to reach. It’s sneaky because survey creators are typically unaware it’s happening.”

Bias is reflected in the wording of questions. Just as attorneys are taught not to lead witnesses, researchers should avoid leading questions in surveys. This applies to all types of research from online surveys, telephone polls and stakeholder interviews.

 SurveyMonkey says unintended bias can be sneaky and sabotage research findings.

SurveyMonkey says unintended bias can be sneaky and sabotage research findings.

Unintended bias also can occur by asking the wrong or incomplete questions, SurveyMonkey says. If you ask respondents to name their favorite kind of pizza, then list a few options, you may skew the results by appearing to limit the range of choice. An open-ended question would be better that allows respondents to list their choice, whether it was pepperoni or pineapple.

Another survey flaw is interviewing the wrong people. An unrepresentative sample can generate findings that don’t reflect the views of the audience you are targeting.

Related to that is excluding a significant cohort from your sample. This can happen when surveys are conducted at times or places where some people can’t participate. For example, a telephone poll that relies only on randomly selected landline phone numbers is bound to underrepresent young people, minorities and lower-income households. A focus group only works for a random sample of people in the immediate area of where the focus group is held.

Bias can rear its head by misreading survey data. “Bias can come into play when a survey creator gets excited about a finding that meets their hypothesis, but overlooks that the survey result is only based on a handful of respondents,” SurveyMonkey says. A common mistake is trying to quantify findings from qualitative research such as focus groups or stakeholder interviews.

The key takeaway is that bias can creep into research at just about every phase of survey work. Tools such as SurveyMonkey make online surveys broadly available to anyone who wants answers. However, researching best practices are essential to ensure you get useful, actionable answers.

Best practices start with clear, objective questions and include a representative sample and a faithful reading of results.

“By remaining true to your survey’s purpose and having a firm understanding of the topics of your research,” SurveyMonkey says, “ you’ll be well on your way to eliminating researcher bias from your survey.”