Key Executives from Leading Advertisers, Media Companies, Agencies and Market Research Firms Join Together to Address Online Research Accountability
New York, August 9, 2007 – The rapid growth of online research panels, driven by business demand for increasing speed, efficiency and significantly more capabilities, has created a new research services segment whose quality has yet to be broadly scrutinized in an objective manner to determine whether advertisers can rely on the insights generated from this methodology.
For example, reports of the failure of online studies to replicate when repeated are becoming more common. The influence on the results of the “heavy online survey responder” is worrisome, but has not yet been studied in a disciplined and objective manner. Projectability of online samples is often ignored. Critics are describing online research surveys as the “wild west.”
Industry standards are needed. And, once those are published, all suppliers using online panels must commit to applying the standards in a fully transparent, continuous manner.
To that end, The ARF announced today the formation The Online Research Quality Council. Itsmission will be to develop a set of standards for research buyers and sellers to follow. The new Council will enlist support from the global market research industry and will be led by a steering committee of senior research executives:
- Robert Tomei, EVP & Global Director of Access Panels, TNS
- Efrain Ribeiro, COO, Ipsos Interactive Services
- Steve Coffey, Chief Research Officer, The NPD Group
- Josh Chasin, Chief Research Officer, comScore
- Jonathan Jephcott, Executive V.P., ViewsNet, Synovate
- Renee Smith, Vice President, Panel Quality, Harris Interactive
The Council will be managed by a rotating chairperson selected from the steering committee members for a six-month period. As noted above, the initial chairperson will be Robert (Bob)Tomei.
”The Market Research Industry has a significant opportunity to establish and implement standards around online panels that would provide our clients with an objective method to evaluate the quality of the information we are delivering on an ongoing basis. It is a unique and distinctive opportunity to work with my colleagues on the Steering Council and the Council and to have key clients actively participate in the process to establish key quality metrics and practices,” said Tomei.
While many companies have developed their own metrics to assess the quality of their online access panels, as yet there is no standard for the industry. The industry metrics will include appropriate measurement of acceptable panelists’ response rates, contact rates, survey duration, sampling criteria, and any other dimensions that can be shown to impact quality to ensure that a given study’s sample will be representative of the population it is aiming to measure. Proper sample balancing methodologies, appropriate panel management techniques, panelists’ incentives, data collection methods, and any other elements of the research design are also subject to scrutiny by the Council.
Working closely with the steering committee will be an advisory board of advertisers. Stan Sthananathan, Vice President, Knowledge & Insights for the Coca-Cola Company, has committed his backing and expressed the concerns, “The rigor and discipline that characterized sampling plans of the 70s and 80s is missing in the online era. Are the control mechanisms that we have in place adequate to ensure quality? What is the equivalent of traditional back checking in the online research era?”
Robert Barocci, President/CEO of the ARF, stressed the value of the Council, “The ‘good, fast, cheap…pick any two’ syndrome has jeopardized the client credibility of online access panels. This trend, combined with the rise of online communities and blogs, has increased the rate of individuals conducting research online without applying statisticalmethodologies or research techniques essential to quality. To ensure that we approach this issue aggressively, we’ve invited top executives from major market research firms, advertisers and agencies to participate in this council that come from organizations committed to the highest level of quality research.”
The first Council meeting will be held on September 10, 2007 at the ARF offices. A timetable of activities will be agreed which will deliver a draft set of global panel research standards to the industry at the ARF Annual Convention in April, 2008 in New York City.