Picture this: A tier one airline recently sent out a customer satisfaction survey. The survey was seriously flawed, leaving respondents confused, frustrated, offended, and exhausted. Not surprisingly, the survey responses were just as poor. In the words of one respondent, “I can’t say I was too sincere."
Unfortunately, this is not an isolated incident. Countless organizations send out surveys riddled with mistakes, greatly reducing both the quality and value of end results. And if it can happen to them, it can happen to you! So be sure to follow these tips for writing effective survey questions:
1. Be as Clear as Possible
A respondent can only accurately answer a question if they understand what the question is asking. For this reason, use simple and consistent sentence structures, proper spelling and grammar, and basic vocabulary. Steer clear of jargon, acronyms, and “grandiloquent” words (like grandiloquent) that may be foreign to your audience. Also, clarify any potentially ambiguous terms: replace “regularly” with “daily”, “income” with “annual household income before tax”, etc.
2. Minimize Bias
Do you want the respondent to provide your desired answer or their true answer? If the latter, take some time to identify and mitigate bias in your survey questions. Bias can arise in many ways. Explaining the goal of a survey (also known as framing) can lead respondents to answer questions in a way that meets those goals, rather than giving their genuine responses. Presenting certain information before a question (also known as anchoring) can create an artificial—and influential—reference point for respondents. Moreover, using emotional, sensitive, politically charged, or absolute wording can intensify respondents’ answers…if they answer at all. To avoid these pitfalls, keep your language neutral and omit extraneous information.
3. Ask One Question at a Time
Let’s be honest, nobody likes super long surveys. But surveys with vague or multi-part questions are even worse! Questions like “Do you like this car?” leave the respondent wondering what to focus on (the car’s look, feel, price?). Additionally, double-barreled questions like “Do you brush and floss your teeth daily?” limit the respondent to a single answer for two distinct questions. To detect this flaw, scan your questions for broad concepts and conjunctions like “and” or “or”. Then break them down into several, focused questions.
4. Don’t Neglect the Answer Options
After you’ve perfected your questions, turn your attention to the possible answers—which are just as important! Ensure that the answer options reflect all reasonable possibilities, including “neither agree nor disagree”, “don’t know,” “not applicable”, “other”, or “prefer not to answer”. Next, check that the answer options don’t overlap—a common blunder when options are numerical (like 0 – 10, 10 – 20, 20 – 30). Finally, consider options that allow for more nuance than just “yes/no”. Two strong alternatives are multiple-choice (with no more than 7 options) or Likert scales (with an equal number of positive and negative responses).
If you’ve followed these tips, it’s time for just one more: beta-test your survey with a few people in the intended audience. Ask them to flag any confusing, biased, or overly complex questions and to brainstorm answer choices you may have missed.
At Op4G, we believe a well-written survey with clearly outlined objectives is the start of good sound research that captures all information necessary to achieve a client's objectives.