Analyzing MaxDiffs
- A general guide on how MaxDiff works
- A practical example in R
- An in-depth case study
-
A guide to MaxDiff in R with the
MaxDiffpackage
At Gradient, we’ve developed our own package to analyze MaxDiff experiments. Our approach uses [#Standard Logit Models] to estimate individual respondent utilities. A simple workflow on how to use our analysis package:
This package contains a set of functions to help you analyze MaxDiff experiments. All of the functions are designed to work together, and will be hard to use seperately. For more detail, please refer to the vignette. A simple overview of the workflow is presented below.
md_data <- md_read_data(data)
Read MaxDiff data and return a list with Design and Answers data frames, suitable for the rest of the md_ functions family. Check ?md_read_data for more information.
md_data_asserted <- md_assert_data(md_data)
This function takes the md_read_data() object as input and will assert if the data is correct. If it’s not, it will tell you where changes need to be made to the initial dataframe, or make the changes automatically. E.g. filtering respondents and checking if all design elements are correct. Check ?md_assert_data for more information.
md_data_tidied <- md_tidy_data(md_data_asserted)
This function takes the output from md_assert_data() and tidies the design and answers data frame so that we can use it for modeling. This is the most important piece, and is also likely the cause of any issues if modeling doesn’t work as intented. Check ?md_tidy_data for more information.
md_data_model <- md_mlogit_model(md_data_tidied)
This function takes the output from md_tidy_data() and puts the model_data into the mlogit model.
This package uses the mlogit package to estimate the parameters. mlogit stand for Multinomial Logit, and is specifically designed to deal with discrete choice models.
md_stats <- md_mlogit_stats(md_data_model)
What we care about. This functionn calculates the statistics for each attribute in the MaxDiff design.
Most simple, but often incomplete analysis.
http://www.surveyanalysis.org/wiki/Counts_Analysis_of_Max-Diff_Data
Hierarchical Bayes (HB) estimation is an iterative process. It encompasses a lower level model that estimates the individual’s relative utilities for the tested attributes, as well as an upper-level model that predicts the population’s preference. These two work together until the analysis convergences on the coefficients that represent the value of each attribute for each individual.
In a sense, HB estimation allows for borrowing information from other responses to gain even better and more stable individual-level results. It is very robust and allows us to get great insight into customer preferences, even while presenting less tasks to the respondent.
The technique is deemed “hierarchical” because of the higher and lower-level models. This approach estimates the average preferences (higher level) and then gauges how different each respondent is from that distribution to derive their specific utilities (lower level). The process repeats over a number of iterations to ultimately help us hone in on the probability of a specific concept being selected based on its utility (thus a Multinomial Logistical Regression model).
The outcome of the Bayesian model are preference scores that represent the utility that the individuals perceive with each variable. These scores are frequently called partworth utilities, and are the basis of all summary metrics produced from the MaxDiff study.
The utility file would have a row for each respondent included in the MaxDiff analysis, and a column for each unique level tested within the study. In modeling the preferences of each respondent, the utilities help us predict what selections respondents would make when faced with different lineups.
The utilities are ordinal in nature and tell us the rank order of list of variables.
Latent class analysis is like cluster analysis. You put in a whole lot of data, and tell it how many classes (i.e., clusters) you want. The table below shows the results for five classes (i.e., segments). The results for each class are shown in the columns. The size of the class is shown at the top. Beneath is theProbability %, also known as a preference share (i.e., the estimated probability that an person in the segment will prefer an alternative from all the alternatives in the study).
The Multinomial Logit Model and its various mixture generalizations are designed to predict a choice from amongst a set of alternatives. These models can be ‘tricked’ to analyze max-diff data by treating each set (i.e., block) in the Max-Diff experiment as two separate sets. In the first set the dependent variable is the Best choice. In the second set, the Worst choice is the dependent variable and the independent variables are multiplied by -1. See this page for examples