When the Data Has Other Ideas: Decoding Unexpected Results

Market research can be a powerful tool, but what happens when the data you receive doesn’t align with your expectations?

While at times it can be exciting, if the results are substantially different to what you expected it can be tempting to question their validity. However, before you dismiss the findings entirely, it’s important to investigate the potential reasons behind them.

This process can lead to a deeper understanding of your target audience, the survey itself, and provide valuable insights you might’ve otherwise missed.

Here are a few things you can check for:

Who took part in the study?

• Audience alignment: The first step is to ensure the right people participated in your study and that they accurately represent your target audience. Carefully review the demographics, business activity, purchase intentions, job titles, and other relevant criteria to confirm their suitability – as a mismatch here can significantly skew your results.
• Sample size and statistical significance: A representative sample size is crucial for reliable data. Consider the size of your target audience and determine if your sample was large enough to provide statistically significant results. For quantitative data, ensure any differences between subgroups were tested for significance. This gives you confidence that observed differences reflect genuine trends and aren’t down to chance.

Analyse the fieldwork

• Timing is key: Consider the timeframe of your data collection. Did any significant events occur during this period that might have influenced respondents’ views? For instance, a major economic shift, political movement, or a competitor’s product launch could drastically alter your consumer’s behaviour.
• Seasonality: If you’re surveying businesses or groups affected by seasonal trends, factor this into your analysis. Data collected during peak season might paint a very different picture than data collected during a lull.

Evaluate the questions

• Clear and unambiguous: Ambiguous or poorly worded questions can lead to misinterpretations and skewed responses. Review your questions to ensure they were clear, concise, and easy to understand.
• Bias and leading questions: Identify any potential biases in your questions that may have led respondents towards a particular answer. Questions that ask multiple things at once should also be avoided, as this can confuse respondents and make it difficult to interpret their answers.

Assess the overall coherence

• Consistency throughout: Do the results of the full survey form a coherent and logical narrative? Inconsistent, contradictory, or seemingly random answer patterns can indicate problems with data quality, respondent comprehension, or even translation issues in international studies.

Re-examine your expectations

• Underlying assumptions: Think back to what your original expectations were based on. Sometimes preconceived notions can cloud our judgment, as we can be too close to a product, customer segment, or existing narrative to see the bigger picture.
• The value of external perspective: This is where professional market research can be invaluable; providing an objective, data-driven perspective that helps you to recalibrate your understanding of the market.

Embrace the unexpected

Any professional market research agency will pay attention to the above when designing and conducting your survey. They should be able to answer all of these questions for you and provide insights from a broader market view to help interpret any findings that may not make immediate sense.

If you have a positive answer to all of the above, be excited about your results because they can offer you that rare and valuable thing: new insights to enable successful product and service development, add impact to your corporate and customer communications, or facilitate a more differentiated brand positioning.

If you would like to know more or run your own piece of market research, talk to us. We’d love to help get you the insights that can make a positive difference to your business.