Can Marketing Managers Trust Traditional Market Research?, By Mike T. Davis, SCI - Computer Intuition
One of the most common objectives of market research is to find the customers wants and wishes, or their hot buttons. But what if traditional market research identifies the wrong hot buttons? What if conventional market research singles out hot buttons that freeze the user's fingers? What if standard market research uses malfunctioning thermometers? A recent study by Professors Dan Horsky, Paul Nelson, and Steven S. Posavac published in the Journal of Consumer Psychology examined this possibility.
The study compared the attractiveness of five sporty car attributes calculated using answers provided in a market research study (what people say), and the attractiveness of the same five attributes derived from the actual buying behavior of the car buyers (what people do). The five attributes were Performance, Dependability, Comfort, Prestige, and Exterior Styling.
The relative attractiveness of the performance, dependability, comfort, prestige, and exterior styling attributes, calculated using the answers in the market research study, were 0.13, 0.22, 0.13, 0.16, and 0.20. The relative attractiveness of the same five attributes, calculated using the real behavior in the marketplace, were 0.24, 0.21, 0.13, 0.00, and 0.19 (note the change in values of the first and forth numbers).
According to the authors: “a rather dramatic change in the ordering of the average weights occurs ... Specifically, the tangible attribute Performance, previously one of the least important attributes on average, is now the most important to sporty sedan buyers. … In contrast, the weight of Prestige, an intangible attribute, falls dramatically and becomes the least important attribute. The remaining attributes change little.”
This “dramatic change” has dramatic implications. “The implication of our findings is that stated preferences may not be highly predictive of actual consumer decisions because the relative importance of attributes differs in value elicitation and choice. This finding is troubling because of the reliance of marketing practitioners on research data pertaining to attitudes, purchase intentions, and attribute importance rankings. If predictions based on stated preferences are markedly different from reality, marketers’ decisions (e.g., product positioning, advertising emphasis) made based on the stated preference data may be suboptimal.” In other words, “forecasts of choice based on stated attribute importances would have been erroneous.”
So, can marketing managers trust traditional market research? Not really. Traditional market research assumes foresight, that is, that study participants are aware, and are willing to describe their own future state of mind, emotions, and specifically behavior. As the observations above suggest, this assumption is most likely wrong. Participants are either unaware, or are unwilling to share these sentiments. And since the assumption is wrong, so are the results produced by all traditional market research tools.
So what's next? If participants have no foresight, is there another way to predict their behavior, except asking them to do it for us?
This article is based on a report in the series Science on Decision Making. The reports in this series analyze papers from scientific journals that include interesting findings on the relationship between analysis of data and managerial decision making. More reports are available at http://www.computerintuition.com/Reports.htm
 Horsky D., Nelson P., Posavac SS. Stating Preference for the Ethereal but Choosing the Concrete: How the Tangibility of Attributes Affects Attribute Weighting in Value Elicitation and Choice. Journal of Consumer Psychology, 2004, Vol. 14, No. 1&2, Pages 132-140
About the Author:
Mike T. Davis, Ph.D., SCI, Rochester NY
We are the inventors of Computer Intuition™, a psycholinguistics based program that analyzes the language that people use to describe themselves and their environment. The result of the Computer Intuition analysis is an Emotional X-Ray™ image. The image shows the emotional bones™ that define the person’s character and direct his behavior. The bones are invisible. They are not found in the language people use, and cannot be seen without an Emotional X-Ray image. No other text analysis program (SPSS, SAS, etc) sees these bones; neither does any human observer (moderator, interviewer, analyst, etc.). When these computer programs or human observers process qualitative data they produce a mere summary of the text, no emotional bones. Once we identify the emotional bones, we prepare a report, called Do this do that™, which presents the emotional bones in tables and graphs. This report also includes recommendations for actions that really work. SCI's clients include many Fortune 500 companies, such as Apple Computer, Sears, Allergan Pharmaceuticals, Chrysler, Citibank, IBM, Motorola, Eastman Kodak, Hewlett-Packard, Anheuser-Busch, Gannett Newspapers, Cincinnati Gas & Electric, Xerox, and Frontier Communications. We also serve many smaller companies and individuals.
Mike T. Davis, Ph.D.
SCI - Computer Intuition
2030 Elmwood Avenue,
Rochester, NY 14620