public affairs

Clear, Fair Questions Key to Reliable Survey Results

Reliable survey research depends on many factors, but it all starts with questions that are asked clearly and fairly.

Reliable survey research depends on many factors, but it all starts with questions that are asked clearly and fairly.

A lot of things need to be done right to deliver reliable and useful survey data. At the top of the list is asking relevant questions clearly and fairly.

Fuzzy questions produce fuzzy answers. Skewed questions produce skewed results. Fuzzy answers and skewed results aren’t a solid foundation for successful advertising and advocacy campaigns or for smart business decision-making.

Clarity in the wording of questions is essential to avoid confusing respondents. You want to make sure respondents have a common comprehension of the question so you can reliably measure their responses. A best practice in public opinion polling and market research is to test questions to ensure they are as clear and commonly understood as possible

Question clarity is becoming more important because of increased cultural diversity and audiences that include English-as-a-second-language speakers. Question testing needs to include this variable as well.

Along with clarity, questions should explore a single idea, issue or product feature. If you mix in multiple ideas, issues or features, you won’t necessarily know which one is the factor that fetches a  “like” or “dislike” answer. Sarah Taylor, in a blog post about survey questions, provides this example:

For example, asking "how valuable and organized was the Ebook?" is mixing two issues together: value and organization. Maybe the respondent really enjoyed the content in the Ebook, but thought it was organized poorly. If you ask them that question, the response may be for either issue and you won't really have a sense of what needs to be changed for your next Ebook. Instead, ask two questions: “How valuable was the e-book?" and “Rate the organization of the Ebook" to get better quality data.

The vocabulary of questions should be basic. Respondents will run the gamut of education and backgrounds. If your subject matter is technical, take special care to avoid jargon or phrases that may go over the head of some respondents. In some cases, you may need to preface a question so there is a common framework for all respondents. Special care is required to make sure any explanation is fair and factual.

Which brings us to skewed questions. If you are in the propaganda business, skewed questions and push polls may be tools of your trade. But for everyone else, skewed questions aren’t helpful and can be disastrous if you mistakenly base million-dollar campaigns on their findings.

Finding out what you need to know – as opposed to what you want to know – is the proper attitude to bring to any kind of research, whether  a quantitative poll or qualitative focus group. Curiosity, not a fixed mind set, is the right frame of mind.

Credible pollsters and market researchers can readily identify and will avoid loaded words or leading sequences of questions. But sometimes skew still slips in, as it did in this famous question, “Would you vote for a woman for president if she were qualified in every other way?” In addition to being confusing, the question implies being a woman is a qualification as opposed to a gender. Subtle, yes, but doesn’t mean it can’t have a significant effect.

The order of questions can influence responses. Pollsters and market researchers will often rotate the order of questions to minimize the possibility of skewing results.

Give respondents a full range of choices in evaluating an idea, issue or product feature. You also might give them a chance to comment through an open-ended question, which allows you to capture their words, not just their answers.

One of the most common forms of skewing is the omission of a key word or concept. If you were polling the congressional Republican proposals to replace Obamacare, it would be a mistake not to test the concept of lower health insurance premiums in connection with the cost of keeping premiums low for patients with pre-existing conditions. Who wouldn’t want to see lower health insurance premiums, but at which price and paid by whom? Installing residential rooftop solar panels will be enticing as a way to cut electricity bills, but the financial equation is incomplete with exploring how long it would take to break even after making the investment in solar panels.

Many other factors determine the accuracy and reliability of survey findings, such as representative samples, sample sizes, length of surveys, when surveys are conducted and the type of survey used. Surveys can generate rich, meaningful and actionable data if done right. A good place to start is to get the survey questions right.

Metaphorically Speaking, Showing and Thinking

Metaphors, like this one about gay marriage, touch the familiar and trigger emotions that can persuade, explain or entertain. Some of the most powerful metaphors are visuals that elegantly make a point with few or no words.

Metaphors, like this one about gay marriage, touch the familiar and trigger emotions that can persuade, explain or entertain. Some of the most powerful metaphors are visuals that elegantly make a point with few or no words.

Getting noticed is getting harder. With a lot of money, you can pummel your audience with advertising, assuming they are still tuning in where the advertising is placed. Without a lot of money, the best course is to penetrate the brains of your intended audience wherever they are.

Metaphors are a proven path into people’s brains. By piggybacking onto something familiar and that you can sense, your message has a better chance to get noticed, triggering a memory and evoking an emotion. Neuroscientists have found that emotional responses are accompanied by physical reactions, which are key to actual decision-making.

Journalist and writer James Geary said in a TED Talk that “metaphor lives a secret life all around us. We utter six metaphors a minute. Metaphorical thinking is essential to how we understand ourselves and others, how we communicate, learn, discover and invent. Metaphor is a way of thought before it is a way with words.”

Linguist Adele Goldberg says a familiar line such as “that was a sweet comment" can activate human taste centers and the portion of the brain linked to fear or pleasure. The phrase touches emotions and memory. More importantly, it sticks because our subconsciousness tends to be literal.

Chip and Dan Heath in their book Made to Stick point out the value of “concrete” references to create mental “stickiness.” Something is concrete, according to the Heath brothers, when it can be “described or detected by human senses.” One example – a V-8 engine is a concrete reference contrasted to a high-performance engine, which is more abstract.

Metaphors add concrete to what you say. Instead of noting a box of movie popcorn contains 20 grams of fat, you could more persuasively say the box contains more fat than a bacon and eggs breakfast, Big Mac lunch and steak dinner combined. People know what fat is, but they can taste, see and smell bacon and eggs, a hamburger and a steak.

Writing for ragan.com, Nick Morgan said metaphors reach the senses with “sweet lines, loud opinions, beautiful phrases, soft poetry and smelly scenes.” Put another way, metaphors put abstract concepts into concrete – and more familiar and digestible – terms.

While we commonly think of metaphors as words, pictures and symbols are often more powerful metaphors. Icons are a great example. We see a light bulb icon and our minds associate it with a “bright idea” or “innovation.” Familiar shapes or visual devices serve as handy metaphors, such as faces of clocks, luggage tags and party invites. Their shape sends a message our minds receive.

Visual metaphors help propel the eye through visual explanations and infographics. Metaphors also can take the form of familiar formats like a flipchart or a website with easy-to-find navigation that enhance user experience and lessen frustration over finding what they want. Pattern recognition can be a key to people’s willingness to explore or engage.

And there is such a thing as an anti-metaphor, which psychologists call cognitive dissonance. Writers refer to it as a “man-bites-dog” statement that startles a listener. Geary’s example: “Some jobs are jails.” The juxtaposition isn’t familiar, but the imagery is concrete and the meaning is clear.

Metaphors can help you get noticed, make your point and earn valuable media coverage. Hillary Clinton showed how in her speech this week taking aim at Donald Trump’s business record. “He's written a lot of books about business,” she said. “They all seem to end at Chapter 11."

Gary Conkling is president and co-founder of CFM Strategic Communications, and he leads the firm's PR practice, specializing in crisis communications. He is a former journalist, who later worked on Capitol Hill and represented a major Oregon company. But most importantly, he’s a die-hard Ducks fan. You can reach Gary at  garyc@cfmpdx.com and you can follow him on Twitter at @GaryConkling.

​A Humming Economy with Trouble Below the Surface

On the surface, the Portland metropolitan area's economy seems to be humming. Just under the surface, things don't look quite so rosy.

The Portland region's economy continues to grow jobs, but full-time jobs and per capita income continue to lag other comparable metro areas.

The Portland region's economy continues to grow jobs, but full-time jobs and per capita income continue to lag other comparable metro areas.

According to the sixth annual Value of Jobs economic check-up released last week, job growth in the Portland region continues at a strong pace for the seventh consecutive year, at a rate outpacing most other peer metro regions across the nation. Portland added 35,800 jobs in the last year and now boasts 70,000 more jobs than its pre-recession peak in 2007.

However, the per capita rate of full-time jobs in the Portland region falls below the U.S. metro average and has actually declined since 2000 to only 47.9 percent of total employment.

The decline of full-time employment in Portland parallels – and may be the cause of – a troubling shrinkage of middle income households, which has serious implications for affordability in the region's hot housing market. In 1980, the study says, middle-income jobs represented 69 percent of the Portland region's total employment, but that number has fallen to 57 percent and continues to drop.

The study reaffirms Portland's outsized economic reliance on manufacturing and exports. It also points out those solid numbers rely heavily on jobs and exports associated with semiconductor manufacturing. Without Intel's output, the Portland region's manufacturing sector would be just on par with other peer metro areas. Likewise, semiconductor products represent 41 percent of the Portland region's exports.

The Value of Jobs study says the decline in the Portland region's per capita income began at least a decade before the last recession and fell below the national average as early as 2007, when the national economy peaked.

A key goal of the Oregon Business Plan is to boost per capita income, but that hasn't happened. The Portland region is now $1,821 below the $47,615 national average for metro areas. Real per capita income in Portland has only increased by $740 since 2000.

The study for the third year measured median household income, which yielded a brighter note for the Portland region. Even though median household income has remained stagnant in Portland, it still surpasses the national metro area average by $3,700. Portland checks in higher than peer metro areas such as St. Louis, Cincinnati and Sacramento, but trails Salt Lake City, Seattle and Denver, and it falls short of aspirational levels expressed by local business leaders.

As Oregon politicians and voters wrangle with a higher minimum wage, the Value of Jobs data indicates that Portland's lowest wage-earners (the bottom 20 percent) earn 19 percent more than the national average, which could be attributed to Oregon's relatively high minimum wage. It is the top end of the wage scale that lags behind other regions and pulls down the Portland region's overall per capita income.

The economic check-up is part of the Value of Jobs Campaign launched in 2010 to examine the Portland region's job market. The coalition that supports the effort includes the Portland Business Alliance, Associated Oregon Industries, Oregon Business Association, Oregon Business Council and the Port of Portland.

Moving the Needle Is What Counts

Public affairs and public relations campaigns must do more than rack up statistics. They need to move the needle.

Measuring success can be highly variable in the public affairs and public relations world. If you are handling communications for a ballot measure campaign, success is easy to mark — you either win or lose at the polls. But most of the time, success comes in the form of incremental movement.

Just because the advance is incremental doesn't mean you can blow off measuring it. For example, coming up with an incentive to encourage loyal customers to come to your restaurant one more time each month may not seem like a big deal, but the return on investment — a discount or free food item — can be enormous because you are communicating with people that already love your restaurant.