MaxDiff Overview

Our customers are constantly making choices, some conscious, but most unconscious. Regular surveys questions with traditional question scales are unable to get to the core of why a consumer makes his or her decision. Most of these scales don’t simulate real-life conditions, which always include some type of real trade-offs.

To simulate real life consumer decision making, the most precise approach is through either a Conjoint Analysis or MaxDiff Analysis.

MaxDiff is a type of survey questioning approach in which respondents are presented with lists of items and are asked to indicate which in each list they like the most and which they like the least. It is also known as best-worst scaling, max diff, maximum difference scaling, and Max Differential Scale Data. An example of a MaxDiff question is shown below.

Typically respondents are asked to complete multiple similar tasks, where the options shown in each task varies according to an experimental design.

Understanding preferences:

• Preferences bebetween new products
• Preferences for existing brands
• Message testing

Segmentation:

• Identify groups of people that value different attributes

General-purpose measurement

• Since MaxDiff forces trade-offs, it can be a good way to understand simple choices. Very often in multiple likert scale items, everything is important, but maybe you want to understand what is more important given a set of choices.

A MaxDiff analysis is conducted by showing participants subsets of items from a list and having the respondent identify the best and worst or most and least preferred options from said list. The reason we take this approach is that it can be challenging for a respondent to rank order 7 or more items in a survey experience. So what MaxDiff leverages is our ability to identify the polars (best and worst) from a list, and simplifies the task in a more digestible number of items at a time.

A respondent would typically see around 5 to 15 questions where 3 to 5 items are shown, and they are asked to indicate the best and worst from the list. This yields very accurate data, since it is a more comprehensible task for survey takers than presenting the entire list.

The steps for running a MaxDiff analysis are:

1. Determine the attributes to be tested in the MaxDiff analysis.
2. Generate the experimental design.
3. Program the survey that hosts the MaxDiff tasks.
4. Collect responses.
5. Analyze the MaxDiff results.
6. Report the findings.

Each of these builds upon the previous action in working toward the end goal of understanding the preferences of the customer base.

1. Try and limit the number of alternatives in each set to 5 or less. More generally, any set containing more than seven is highly unlikely to be useful, as respondents find it difficult to compare so many alternatives and tend to take short-cuts (e.g., reading only half the alternatives).
2. Specify the number of alternatives in each set to be no more than half the number of alternatives in the entire study.
3. Ensure that each alternative is shown at least three times (unless anchored MaxDiff is used; see below).
4. Where the focus is only on comparing the alternatives (e.g., identifying the best from a series of product concepts), it is a good idea to create multiple versions of the design so as to reduce the effect of order and context effects. Sawtooth Software suggest that if having multiple versions, 10 is sufficient to minimize order and context effects, although there is no good reason not to have a separate design for each respondent. Where the goal of the study is to compare different people, such as when performing segmentation studies, it is often appropriate to use a single version (as if you you have multiple designs, this is a source of variation between respondents, and may influence the segmentation)

Where there are a large number of alternatives to be researched, and the resulting incomplete block design is infeasibly large, two solutions are:

1. Randomly allocate blocks. For example, if a design is created with 20 blocks (i.e., questions) each respondent may be randomly allocated 10).
2. Use an incomplete block design to allocate alternatives to respondents and then use an incomplete block design for each set of alternatives. For example, if there are 100 alternatives, you could:
   • Create a design with 100 alternatives, 20 blocks and 10 alternatives per block (Design A).
   • Create a second design with 10 alternatives, 6 blocks and 5 alternatives per block (Design B).
   • Randomly allocate each respondent to one of the 20 blocks in Design A (i.e., so that the respondent only sees the 10 alternatives in this block).
   • Use Design B to create the questions to be shown to the respondent (where the alternatives used are dictated by the respondent’s block in Design A).

Strategies for addressing prohibitions include:

• Use commercial software specifically design for prohibitions (e.g., Sawtooth’s MaxDiff software).
• Randomizing across respondents. For example, if two alternatives are never to be shown together, create the experimental design with only one alternative in place of these two and then show one of the alternatives to half the respondents and the other to the other half.
• Rotational splitting of alternatives. For example, if two alternatives are never to be shown together, create the experimental design with only one alternative in place of these two and then randomly assign half of the appearances of the alternative to the first of the alternatives and the other half to the second.
• Random removal of alternatives. Wherever a prohibited combination appears, manipulate the design in such a way to remove the prohibition (e.g., replace an alternative with another alternative).
• Create sets by randomly selecting alternatives (e.g., using R’s sample function) and discard any sets that contain the prohibited features. Note that this approach is only sensible if different respondents see different sets of alternatives.

There are a variety of alternative forms of anchored MaxDiff. The most generally useful proceeds as follows:

1. Get respondents to rate the appeal of all of the alternatives (e.g., using a 7-point rating scale).
2. (Optionally) Get respondents to rank any alternatives that receive the same rating.
3. Select a subset of the alternatives for inclusion in the MaxDiff experiment.