Concept testing questions to ask your respondents

Why are good concept testing questions so important?

One of the biggest bets a company can make is to launch a new product. It takes incredible planning, timing, work, investment, and even faith when the payoff is uncertain. Some of the smartest global marketers have failed at launching new products.

The upfront market research you conduct on new concepts is like an insurance policy on future events. Concept testing helps to reduce the risk of launches by spotting problems early.

Concept testing is how people, without prompting, interpret an idea for a new product or service (Smith and Albaum, Fundamentals of Marketing Research, Sage Publications, 2006). Concept testing is most often used in concept development to test the success of an idea for a new product, service or experience before it is marketed.

Capture real-time feedback early with our Concept Testing Program

Before you write your concept testing questionnaire

Concept testing can be carried out in a range of different scenarios. You may be heading into a new market, launching into a fresh product category, even starting a brand new company or sub-brand. Concept testing always happens at the earliest stages of development – that’s what distinguishes it from product testing, where researchers assess the target audience’s response to existing products or new product prototypes.

Concept testing is very much a ‘drawing board’ form of research, done far in advance of a successful product launch. It brings potential consumers into the service or product development process at the earliest stage. For that reason, it’s important to make sure respondents understand that what they’re looking at is an idea, not a finished item.

You may need to direct them carefully in the introduction to your survey to make sure they’re aware of the context, and shape the question wording around the fact that they’re evaluating a product idea, not a product per se.

Types of concept test surveys

Concept testing methods that use a survey generally fall into one of three categories:

Monadic testing

Monadic testing presents a single concept to your potential customers for evaluation. It allows you to go relatively in-depth with your market research, since you don’t have to split your time and resources between multiple product concept tests.

Sequential monadic testing

With sequential monadic testing, the respondent sees at least two different concepts within the same concept survey. After seeing the concepts, they are asked the same range of questions about each one. This kind of concept survey allows you to use the same group for multiple tests, so it’s quite cost effective. However, it may end up being longer and more complex for the participant to complete, which might introduce response rates and completion rates. This method often includes a preference question to determine which concept is preferred. Although it’s not a replacement for later stages in the development process, it can give you a useful steer, especially if you have been careful to select the right audience

Concept optimisation using conjoint analysis

Comparative testing differs from the other two concept testing methods mentioned, as it invites respondents to evaluate possible bundles of features. This is effective for the researcher to understand the significance and appeal of different bundle options. It can be particularly useful when trying to determine the optimal bundle of features, at a specific price point, that would be most desirable.

Concept testing survey templates

One option for compiling a questionnaire for concept testing is to use a concept testing survey template as a starting point. The advantage of a survey template is that it will have been developed by experts who have run any number of concept tests and understand the requirements and potential pitfalls to avoid. It might save some time and effort composing your own survey questionnaire from scratch.

A template for concept tests may be available as a stand-alone product or download, or it might be bundled with a concept research program or platform as part of an external provider’s service.

Although a concept testing survey template may save you some time and work, you will need to adjust and adapt the questionnaire to suit your requirements, since every concept test is different. Your product or service, your business goals and your specific audience will all play a role in influencing your choice of questions and wording.

Types of concept testing survey questions

Your survey questionnaire can be divided into a few different sections.

1. Introduction
2. Screening questions
3. Concept validation questions

Introduction to your concept testing survey

The introduction helps set expectations for your survey respondents. It lets them know what you expect them to do, and the limitations of your expectations too – for example you can assure them that questions are not compulsory. Use the introduction to explain briefly how their data will be used and link them to the privacy policy associated with your study or your organisation.

The introduction is also an opportunity to introduce participants to your research goals so that they have a better idea of the kind of information you’re looking for. As we’ve noted previously, make sure they understand that they are looking at a concept, not a finished experience, service or product, and not even a prototype.

Screening questions

Your survey’s screening questions are used to filter out anyone who doesn’t fit the parameters of your study. You may well have intentionally selected only people who are part of your target audience during your respondent recruitment process. But screening questions will help make sure that if anyone comes across your survey link who doesn’t fit the bill, they won’t waste their time completing your survey, and their answers won’t confuse your concept test results.

Use earlier stages of your research design to create some filtering questions that will eliminate people who are not part of your target audience.

For example, if your concept test is for a lawnmower, a screening question might ask the respondent how often they take care of a garden. If the answer is ‘very infrequently’ or ‘never’, you may want to exclude them and end their survey questionnaire at this point.

Concept validation survey questions

This section can be considered the ‘meat’ of your concept testing survey. These questions introduce the concept you’re refining to its target audience, and investigate their responses to it.

Questions might include these topics:

Liking

Ask the respondent how much they like the concept

Follow-up questions

Use follow-up questions to contextualise and explain the ‘liking’ response. What did they like or not like about it? Was there something they felt was missing? Or something that set the concept apart from other products or services they had seen before?

Likelihood to purchase (aka purchase intent)

How likely is it that the respondent would buy the product or service represented by the concept you’ve shown them?

Frequency of use

If the respondent said they would buy your concept, you’ll likely want to find out what sort of role it would play in their life. Would it be something they made use of daily, weekly, or less frequently?

Replacement or addition

Would the concept be an addition to the respondent’s life, or would it supersede and replace another solution that they currently use?

Product attribute assessment

In product concept testing, these questions will explore the target market’s reaction to different product concept features.

Question formats for concept testing questionnaires

As well as considering your question topics, you’ll need to think about the format of your questions, since these choices will change the way your respondents answer.

Likert scale question

A Likert scale question provides the respondent with an odd number of options, typically 5, which range from ‘strongly agree’ to ‘agree somewhat’ to ‘neither agree nor disagree’ to ‘disagree somewhat’ to ‘strongly disagree’. Likert scales produce quantitative data, despite dealing with qualitative assessments. The Likert scale is widely used in market research as well as in other disciplines.

Ranking question

A ranking question asks the participant to arrange a predefined list in order of preference. There is no standard list length, but it’s better to make it shorter rather than longer, as this will make it easier for participants to consider the question and their answers will be more meaningful – and ultimately more useful in evaluating your product concept. Ranking questions can help you with comparative testing, weighing your product concept against the products of your competitors. They can also be used for comparative testing of possible product concept key features with your target market.

Rating question

Like the ranking question, rating questions ask your target audience to express their preferences on a range of specific examples. However, the ratings do not have to be mutually exclusive – the respondent can rate multiple options as 1, 2, and so on rather than having to choose just one option for each rank.

Open field question

Open field questions, or open-ended questions, allow your respondents to answer in their own words. Using them successfully in your concept test requires the ability to process natural language data – either manually, if your target market sample is small, or using AI tools. Open field questions may be useful for follow-up questions, when you ask your future customers to explain why they gave the answer they did.