Concept testing is an early-stage market research method that maximises the chances of launching a product or service that people want to buy. The concept in question can be a few different things - It might be a new set of product ideas that no one has ever seen, or it could be a redesign or rebrand. Conducting concept tests allows you to develop ideas in a safe and controlled space, exploring the viability of the product or service early on and improving development based on feedback. 

The tips and insights shared in this article draw upon Milieu’s market research expertise in crafting and analyzing concept test studies from a wide range of clients every day. 

#1 - Pay close attention to your respondents’ survey experience

You may be surprised by how the response quality of respondents can be hugely impacted by a multitude of factors: the complexity of your concepts, the order in which they are shown, or even the survey length. Based on Milieu Insight’s own extensive experiments, a typical respondent's attention span lasts no longer than 10 minutes.‍

To optimise response quality, it is important to consider the respondent's experience when taking your survey. For instance, when dealing with intricate concepts, limiting exposure to 1-2 concepts per respondent (such as adopting the Monadic model) often proves more effective in sustaining respondents' attention and, hence, translates to better response quality.

‍

#2 - Ask questions around concepts and values that your brand is willing to adhere to

While standardised questionnaire formats are undeniably convenient, the potential pitfalls of such an approach become apparent when results are misaligned with a brand's capabilities or intentions. Our advice? Customise questions to align with what your brand can afford and is willing to do.

For instance, if pricing is non-negotiable, omitting it from the survey prevents respondents from overly fixating on it, and allows you to steer their focus on aspects where their feedback is required. Conversely, if pricing is negotiable, measuring concept preferences at different price points by employing models like Van Westerndorp can yield more insightful results.

#3 - Never underestimate the impact of a brand name on concept appeal

The inclusion of a brand name in concept testing introduces a layer of bias that demands careful consideration.  You may then ask: If bias is usually undesirable and as much as possible, removed from the research process, should you always exclude your brand from the concept?

Not always - in certain contexts, embracing it can actually provide valuable insights. Assessing brand impact involves strategic testing, such as comparing debranded and branded concepts to glean insights into customer preferences. For instance:

• The results of the debranded and branded concepts are closely similar: This may suggest a strong affinity for the concept irrespective of brand
• The debranded concept is preferred over the branded one: This could point to a mismatch in concept versus how people perceive the brand
• The branded concept is preferred over the debranded one: The brand could possibly be elevating the concept, which may be worth a reconsideration of pricing.

#4 - Explore data with an open mind

Approaching data with as many hypotheses as possible ensures that you are not limiting your analysis based on what you already know. A common mistake of data analysis is cherry-picking data that feeds into existing biases or perceptions, defeating the purpose of a research study. Results should be interpreted through multiple lenses, such as customer segments, and holistically by looking at the big picture, beyond comparing single data point to data point. This enriches the analysis, providing a more nuanced understanding of preferences beyond straightforward comparisons.

For example, if concept A’s likeability is similar to that of concept B, what will be the one decisive factor to help you reach a conclusion? This is where it is helpful to gather insights from different approaches, such as comparing the likeability of the respective concepts to a summarising question that directly asks for respondents’ preferences of concept A versus concept B.

Alternatively, you can also look at other data within the survey that may not address likeability directly but tells you the elements of each concept that contribute to their appeal. These elements should ideally also align with your branding.

#5 - Dive into a gold mine of insights with customer segmentation

Customer segmentation offers much more granularity to understanding the often complex and diverse preferences or habits of your customers. For instance, a Gen Z male can have very different preferences for health supplements from a Gen X male. These differences can even be further branched out to their incomes, jobs or parental status.

To define what segmentations you should be looking at, start with your brand’s vision for the concept that you are launching, and who your target audience is. At the same time, keep in mind that the goal is not simply to obtain immediate answers, but to also uncover potential customer groups that may have been overlooked initially. Keeping your eyes wide for these insights could open your brand up to many more opportunities.

Optimising your respondents’ survey experiences is crucial for reliable insights. From designing a questionnaire with careful consideration of respondents’ experience to approaching data with an open mind, a successful concept test study covers all bases, ensuring the depth and reliability of insights.

‍