Page 1 of 2
Next Generation Projective Techniques, Written By Tom Anderson and Anna Song, PhD From Anderson Analytics
Combining Psychological Content Analysis and Text Mining in Market Research
This Anderson Analytics white paper explores the history of Thematic Appreciation Tests also known as Picture Story Telling Exercises (TAT/PSE), and explains how the combination of new technology such as text mining and psychological coding software can be used to apply this previously qualitative technique on large numbers of respondents for not only deep and meaningful insights, but statistically valid and projectionable results. The paper concludes with an Anderson Analytics case study: a PSE conducted online for a clothing manufacturer.
Introduction: Content Analysis in Market Research
Market Researchers have experimented with various types of projective techniques in qualitative research. Often though, these experiments were lacking in three important ways. First, they were not carried out with the same methodological rigor as their predecessors in the psychology field. Second, due to their small sample sizes, results could not be statistically validated nor could results be projected to the general population, important prerequisites for buy-in with senior management. Finally, results stood alone without the supporting demographic and behavioral data which is necessary in market analysis.
This paper explores the history of content analysis used in Thematic Appreciation Tests (TAT's), also known as Projective Story Exercises (PSE's), and their recent rebirth and use as a powerful hybrid technique by Anderson Analytics. Combining the advanced coding schemes of psychology with new technologies such as text analysis and text mining, and adding an interface which today's respondents more easily identify with, the future of PSE's as a meaningful research tool looks very promising, and represents one of the better opportunities for consumer insights with deeper meaning.
Content analysis is the examination of verbal text for words or themes associated with psychological constructs. Verbal text includes a wide berth of materials/speeches (Winter, 1987, 1988), letters (Suedfeld, Corteen, & McCormick, 1986), therapy transcripts (Peterson, Luborsky, & Seligman, 1983), plays (Simonton, 1983), poems and sonnets (Simonton, 1990), song lyrics (Zullow, 1991), spontaneous remarks (Hermann, 1980), and corporate annual reports (Zajac & Westphal, 1995). Content analysis has also expanded beyond the bounds of words and into nonverbal text, such as musical pieces (Simonton, 1980) and comic book strips (Sales, 1973).
The genesis of content analysis in personality psychology can be traced to work by Morgan and Murray (1935, 1938) on the Thematic Apperception Test. The scoring for this projective test involves showing participants ambiguous images and asking them to write stories about the picture. The stories are then analyzed for themes and imagery associated with specific motives and needs. McClelland (1953) and his student John Atkinson furthered this work by developing a content-analytical scoring system using experimental manipulation of motives. Unlike the TAT scoring system by Morgan and Murray (1938) that was based on clinical intuition or theoretically based criterion, Atkinson and McClelland (1948) aroused motives in randomly assigned subjects, administered the TAT, then compared the results with a control group. Systematic cross-validation on stories would identify themes and imagery specifically associated with experimentally aroused motive. Winter (1973) continued Atkinson and McClelland's (1948) work by adapting their scoring scheme to verbal text --- the coding scheme yields an imagery score per 1000 words of any oral or written text. This technique allowed researchers to measure motives from virtually any historical document about their subject.
For example, in their analysis of President Nixon, Winter and Carlson (1988) quantitatively measured Nixon on three personality motives, the need for power, achievement, and affiliation.
Winter and Carlson content analyzed Nixon's inaugural addresses using Winter's (1973) at-a-distance scoring scheme for each motive. From his addresses, Winter and Carlson determined a personality profile for Nixon --- he was high in achievement and affiliation, and moderate in the power motive. They also found that his paradoxical behavior in office, such as the Watergate scandal, corresponded to this personality profile, thus providing a psychological explanation for one of the most perplexing presidents in history.
The at-a-distance technique used by Winter and Carlson (1988) is not limited to the measurement of personality motives. Suedfeld, Tetlock, and Streufert (1992) devised an at-a-distance scoring technique for integrative complexity. To assess how political figures might differentiate or integrate complex information, researchers can now quantitatively measure levels of integrative complexity by scoring documents from letters to speeches to interviews. For example, Suedfeld, Corteen, and McCormick (1986) measured General Robert E. Lee's level of integrative complexity during six major battles. They scored dispatches, orders and private letters written by Lee and his opposing generals. They found that when Lee was higher in integrative complexity than his opposing commanders, Lee would win the battles. Only General Ulysses Grant was higher in complexity, and Lee lost both battles to Grant.
Creating a Coding Scheme
Content analysis involves coding categories that signify the presence of some personality dimension. These categories are determined in one of two ways: a priori construction and experimental derivation. A priori construction coding schemes identify categories well before data collection and the construct identifiers are developed in conjunction with theory, research findings, or even a pre-existing psychometric scale. Two examples illustrate this first method of identifying categories. McCrae (1996) investigated behaviors and correlates of the Five-Factor Model dimension of Openness. In this study, McCrae identified several behaviors and thoughts akin to Openness. He also expanded the scope of the construct by linking it with other established personality characteristics, such as authoritarianism, attitude formation, and political affiliation. Using these characteristics of Openness as his construct identifiers, he content analyzed Jean-Jacques Rousseau's biography and used the French philosopher as a an example of an individual high in Openness.
Another example of a priori construction is the adaptation of preexisting standard instruments for content analytical use. For instance, Simonton (1998) used the Holmes-Rahe Social Readjustment Scale (Holmes & Rahe, 1967), an established psychometric tool, and adapted it to measure King George III's stress level at-a-distance. King George was the infamous British monarch who reigned during the American Revolution. His time on the throne was marked with bouts of mental and physical breakdowns that branded him as Mad King George. Numerous academics have posited their theory for his breakdown (see Runyan, 1983 for a review), yet no one ever tested the very simple possibility that King George was stressed. Raters, blind to the subject and hypothesis of the study, read biographical timeline of King George's life and rated his level of stress using the adapted Holmes-Rahe scale. Simonton was able to uncover a relationship between breakdowns and stressful events, a correlation that was overlooked because stressful events preceded breakdowns by an average of six months.