Finding what people agree on.
By John Goslino, Principal Consultant, Audience Dialogue, Australia
Qualitative research produces a wealth of information, not in the form of numbers, but in the form of words. People whose inclinations are verbal rather than mathematical (like most media workers we know) often have trouble interpreting the results of surveys, but they find the results of qualitative research easier to understand and use. However, qualitative research has been regarded as too difficult for untrained people to do successfully. The prerequisite has normally been an advanced degree in psychology.
The most common form of qualitative research is a focus group.
The most common form of qualitative research is the focus group. These are widely used for assessing the viability of proposed new services or products. In each group, about 8 people meet to discuss a particular issue. The group is led by a highly trained moderator, who begins the discussion at a very general level, then gradually focuses in on the specific topic. Respondents are not told about this topic in advance, only the broad area of interest.
For example, if a TV station wants to assess a new type of current affairs program, the people chosen for the group could be those interested in watching information programs on television. The discussion might begin with the types of information program participants like and dislike, and the reasons for those feelings. The discussion might then move onto current affairs programs in general, then some specific current affairs programs, then onto current affairs programs on that channel. At this point the participants might be shown a short pilot of the proposed new program, and asked to discuss it. Such a group typically lasts from 1 to 2 hours.
So the focusing begins with the general, and moves towards the particular. Focusing is also called funnelling - but a focus group is never called a funnel group.
Everything the participants say is recorded, on either audio or video. For the moderator, the hard work now begins. The actual moderating can be learned quickly - it’s mainly a matter of ensuring that everybody has a chance to speak, that some don’t dominate others, and so on. However the analysis is much more difficult and time-consuming. The moderator often watches the video or listens to the tape several times, noting participants’ expressions and gestures as much as the content of their speech. Advanced training in psychology is almost necessary, as is long experience at running focus groups. With untrained moderators, interpretation of focus groups is highly subjective: two different moderators may reach quite different conclusions about the participants’ reaction to whatever was being studied.
When the moderator has studied the tapes or transcripts, he or she writes a report. There is no simple method of converting what participants say and do into conclusions.
Focus groups are usually done in sets of 3 to 6 groups, each group with a different type of person. For example, the assessment of a pilot current affairs TV program might require 4 groups: perhaps men under 35, men over 35, women under 35, and women over 35.
To analyse focus group proceedings thoroughly usually takes a full day’s work for each group, then another day or two to write the report. For this reason, commissioning focus groups from market research companies is expensive. Though few people take part in the groups, far more time is spent on each person than on interviewing respondents in a survey - and a much higher level of skill is needed than for normal interviewing.
Because of the high cost of professionally organized focus groups, some organizations are now running their own focus groups, and even gaining some useful insights. However, their lack of experience often leads them to misleading conclusions. It may seem to them that their customers are highly satisfied, or would watch a proposed new program in large numbers. Later, they often discover that their conclusions were wrong: that the innovation that the participants seemed to welcome is far from popular among their whole audience.
In the mid-1980s we produced the first edition of a book called Radio Survey Cookbook, so that radio station staff could do their own surveys. Some of them managed to do successful surveys, but others found that surveys required too much work, too much organization, and too much skill at manipulating numbers. So we set out to develop a simpler research method.
Bearing in mind the skills of radio (and indeed general media) staff, and the great interest they took in reports from focus group studies, we gradually developed a qualitative research method that could be done easily and accurately by people with little training. This is the consensus group, which is sort of halfway between a focus group and a public meeting. It also includes elements of other research techniques and negotiation techniques.
Principles of consensus groups
In every survey the questionnaire ensures that everybody is asked the same questions. The only variation can be in the number of people giving each answer to each question. So surveys begin with words (questionnaires), but the results are always expressed in numbers.
Consensus groups work in the opposite way: the numbers remain constant (more or less), but the wording of each statement is adjusted until the great majority of participants agree.
It’s important to realize that a consensus group does not try to create a consensus among participants: that’s peace-making. This is research: it simply tries to find and define any consensus that already exists. Unlike a focus group (which narrows in on a single topic) a consensus group normally covers a broad range of topics.
The technique has two main stages: recruiting participants, and holding discussions. Like focus groups, consensus groups are never done singly, because with a small number of participants, any one group may be atypical. Consensus groups are normally done in sets of 3. There can be more than 3, but every extra group adds less and less information.
Before participants are recruited, the planning process is much the same as a survey: the organizers must decide what is to be covered, among what population. When the subject and scope of the study have been decided, nine steps follow:
1. Within the area to be studied, three (or more) sampling points are chosen, contrasting as much as possible.
2. At each of the sampling points, a venue is arranged. All you need is a suitable room.
3. A short screening questionnaire is prepared.
4. At each sampling point, people are interviewed using the screening questionnaire. The purpose of these interviews is to find people who are both eligible and willing to attend a consensus group.
5. The group meets, either immediately, or up to several weeks later. Each meeting lasts for two to three hours. Approximately 12 participants are present, as well as 2 organizers: a moderator and a secretary.
6. The first stage of the meeting is introductory. The participants briefly introduce themselves, giving some relevant background information.
7. In the second stage of the meeting, the topics are discussed by all participants. The moderator manages the discussion, ensuring that everybody speaks freely, while the secretary takes notes.
8. In the final stage of the meeting, consensus is sought. The secretary puts up statements which most participants are expected to agree with. Statements are modified, depending on what participants say. When each statement is ready, participants vote on it. On average, about 20 statements are agreed on. This list of statements is the main outcome of the meeting.
9. When three meetings have been held, the three lists of statements are compared. Any statements shared by at least two groups are the outcome of the study.
To learn about the nine steps in detail please visit our webpage on consensus groups for qualitative research
8 November 2011