Everyone makes mistakes, including in the field of market research! Over the years, we’ve seen dozens of missteps related to research design and delivery. Unfortunately, some can seriously jeopardize the quality and utility of research results. Below is our list of the most common market research pitfalls and how your organization can avoid them. After all, why make the same mistakes when you can learn from others?
- Unclear Goals: The objectives of a research project should inform all elements of research design, from the type of study to the questions asked. Yet, many organizations neglect to set goals or flesh them out, leaving researchers aimlessly searching for meaning in the data. Organizations can avoid this fate by outlining specific aims for the research project, including the knowledge they are hoping to gain. To facilitate the process, consider the background for the project, why it was proposed, and how the results will be used.
- Sampling Errors: Market research can predict the views or behaviors of a target population—but only if the sample is truly representative. Here, many researchers fall short; they poll too few people or an unrepresentative subset. Sometimes, researchers haven’t even defined the target population that they are trying to match. To control and reduce such errors, take the time needed to define your target population and carefully design sample groups. Also, take advantage of online sample size calculators.
-
Arbitrary Quotas: In the quest to achieve a representative sample, some establish quotas for particular demographic groups. Often, these quotas mirror the US population (e.g. 49% of responses from males and 51% from females). But they don’t necessarily correspond to the target population. For example, say you are surveying makeup product customers and they are actually 70% female. Quotas are also subject to self-reporting bias and, when applied to hard-to-reach groups, "risk an increase in field time and reduction in feasibility." If quotas aren't the best fit for a particular study, try sending survey invites according to your demographic targets. Alternatively, use weighting after data collection to help correct any under- or over-representation of demographic groups.
- Low Incentives: As noted in #2 above, polling a sufficiently large sample group is critical. But it can be exceedingly difficult if incentives for participation are too low. In some cases, the solution is simply to offer participants more money. We recommend paying at least $1 per minute in order to properly compensate panelists for their time. Other cases demand more creativity. For example, C-suite executives aren’t motivated by $5, $20, or even $100. So instead, appeal to their altruism by donating survey incentives back to nonprofits on their behalf.
- Poor Wording: Words have power. In the context of market research, question wording can baffle respondents or—just as problematically—guide them towards a particular response. To mitigate these problems, steer clear of jargon, acronyms, and sophisticated language. Also, strive to keep language-neutral and to omit any extraneous (or potentially influential) wording. To learn more, check out our blog on effective survey question writing.
- Question Order: As important as question wording is the question order. Boring or sensitive questions can cause respondents to abandon a survey completely. Thus, leave such questions to later in the survey and avoid more than two in a row. Additionally, some questions can have a “priming” effect, shaping how respondents answer subsequent questions. For instance, if a survey asks about the most pressing issues facing America after a question on crime, crime is likely to rank higher. To prevent such priming, try randomizing the order of survey questions or pages. For more tips, read our blog on question sequencing.
- Excessive Length: Asking too many questions, or overly long questions, can trigger respondent fatigue. Some cope by exiting the survey, thus lowering completion rates. Others race through the remaining questions, reducing data quality. In fact, according to one study, the per question response time decreases steadily from 75 seconds for question #1 to only 19 seconds for questions #26 – 30. So, remove any questions that don’t advance your research goals (see #1 above). Then, pilot the survey—and take it yourself—to ensure a reasonable length.