A “tier one airline” recently sent out a customer satisfaction survey. The survey was seriously flawed, leaving respondents confused, frustrated, offended, and exhausted. Not surprisingly, the survey responses were just as poor. In the words of one respondent, “I can’t say I was too sincere."
Unfortunately, this is not an isolated incident. Countless organizations send out surveys riddled with mistakes, greatly reducing both quality and value of the end results. And if it can happen to them, it can happen to you! So be sure to follow these tips for writing effective survey questions:
1. Be as Clear as Possible
A respondent can only accurately answer a question if they understand what the question is asking. For this reason, use simple and consistent sentence structures, proper spelling and grammar, and basic vocabulary. Steer clear of jargon, acronyms, and “grandiloquent” words (like grandiloquent) that may be foreign to your audience. Also, clarify any potentially ambiguous terms: replace “regularly” with “daily”, “income” with “annual household income before tax”, etc.
2. Minimize Bias
Do you want the respondent to provide your desired answer or their true answer? If the latter, take some time to identify and mitigate bias in your survey questions. Bias can arise in many ways. Explaining the goal of a survey (also known as framing) can lead respondents to answer questions in a way that meets those goals, rather than giving their genuine responses. Presenting certain information before a question (also known as anchoring) can create an artificial—and influential—reference point for respondents. Moreover, using emotional, sensitive, politically-charged, or absolute wording can intensify respondents’ answers…if they answer at all. To avoid these pitfalls, keep your language neutral and omit extraneous information.
3. Ask One Question at a Time
Let’s be honest, nobody likes super long surveys. But surveys with vague or multi-part questions are even worse! Questions like “Do you like this car?” leave the respondent wondering what to focus on (the car’s look, feel, price?). Additionally, double-barreled questions like “Do you brush and floss your teeth daily?” limit the respondent to a single answer for two distinct questions. To detect this flaw, scan your questions for broad concepts and conjunctions like “and” or “or”. Then break them down into several, focused questions.
4. Don’t Neglect the Answer Options
After you’ve perfected your questions, turn your attention to the possible answers—which are just as important! Ensure that the answer options reflect all reasonable possibilities, including “neither agree nor disagree”, “don’t know,” “not applicable”, “other”, or “prefer not to answer”. Next, check that the answer options don’t overlap—a common blunder when options are numerical (like 0 – 10, 10 – 20, 20 – 30). Finally, consider options that allow for more nuance than just “yes/no”. Two strong alternatives are multiple choice (with no more than 7 options) or Likert scales (with an equal number of positive and negative responses).
If you’ve followed these tips, it’s time for just one more: beta test your survey with a few people in the intended audience. Ask them to flag any confusing, biased, or overly complex questions and to brainstorm answer choices you may have missed.
Op4G’s operational staff has over 100 years of combined market research experience and provides survey design, optimization, and consultation upon request. At Op4G, we strongly believe that a well-written survey with clearly outlined objectives leads to error-free and efficient programming. Most importantly, it provides our market research clients value by ensuring their research is fundamentally sound and captures all information necessary to achieve their objectives.
For more help or information on our growing survey consultation services, don’t hesitate to contact your account contact or email@example.com. Also, stay tuned for future blogs on other elements of effective surveys including data quality, the survey length, order, format, device compatibility and more.