Whether you call them employees, associates, cast members or wizards, the people employed by your company have priceless intelligence; intelligence needed to manage your company and plan for its future. Their intelligence is hard earned; earned while trying to integrate your policies, procedures, and systems in order to provide products and service to your customers. Employees can provide information that no one else can.
- Why are customers saying they are waiting too long for service? Ask your wizards (As well as your customers).
- How well do your computer systems support selling your products? Ask your wizards.
- What do customers complain most about? Ask your wizards.
- Do your employees’ images of your company, products and services match the ones you are trying to project? Ask your wizards.
- Why is your employee turnover much higher than the industry average? Ask your wizards.
- What benefits are most important to your employees? Why aren’t more employees using your educational benefits? Ask your wizards.
Why Your Wizards Should Be Behind a Screen
You may be thinking that you ask your employees for information all the time – in team meetings, project meetings and in one-on-one conferences with their supervisors. How does employee research differ from the day-to-day exchanges employees have when performing their jobs?
In most cases, the difference is that employee research provides a screen for employees that guarantees them anonymity when they provide requested information.
The screen ensures:
- an employee’s appearance does not affect how the information he/she provides is valued -- the employee’s gender, physical attributes or mode of dress does not modify the way information is received.
- the requestor’s appearance does not keep employees from speaking their minds -- an employee does not feel intimidated and does not modify his/her answers based on the gender, physical attributes, or mode of dress of the person requesting information.
- an employee does not need to factor “chain of command” or his position in the organization into responses when providing his opinions, experience or judgment.
- Trust. An employee can speak freely and not fear that he/she will be treated differently because of what he/she says.
This MRPMI ProjectAdvisory™ is based on 20 years of experience designing, conducting, interpreting, and reporting employee research within Fortune 500 companies. It is written to help companies and organizations initiate and manage employee research projects, and avoid pitfalls. It makes recommendations to ensure that your employee research results in a Win-Win for you and your employees.
This is not a technical manual on how to design, field, analyze, or report the qualitative or quantitative research that can be conducted with employees. These professional skills require semesters of market research education honed with experience.
Definition: Employee Research
For the purposes of this ProjectAdvisory™, employee research is any voluntary* activity conducted to obtain opinions, attitudes, or information from employees that requires one or more of the following:
- Assurance of anonymity so employees do not censor, modify, or withhold information.
- Personal employee information that is not in an employee’s personnel file, but is needed for managing the company.
For example: The number of children employees have might be needed for benefits planning.
- Market research expertise to collect unbiased information from employees with a reliable and consistent qualitative or quantitative methodology.
*Voluntary means that employees are invited and encouraged to participate, but participation is not mandatory.
The Rewards for Your Company
Whether your company has twenty, a thousand, or a hundred thousand employees, day-to-day business challenges often keep you from taking your company’s pulse. Employee research helps you evaluate aspects of your company’s health from the perspectives of those people most able to impact it.
The Rewards for Your Employees
Employees are also caught up in day-to-day business challenges, and often have no time to address issues with a broader, collective focus. Asking for their candid opinions in a manner that focuses on what they have to say is recognition that you value them. It gives employees the opportunity to have their say, and gives them the satisfaction that their opinions and expertise have been considered and factored into management decisions. It is psychic income.
Starting Down the Yellow Brick Road – Initiating Employee Research
Before you start down the yellow brick road to ask your wizards for information, find out whether your need for information is worth their time. Respect your employees’ time and protect it from abuse. Most employees have productivity standards to meet, you know.
Advice: Establish a Wizard Protection Program
* Establish standards for the maximum number of hours employees can be taken away from their jobs each year to participate in research activities – by job function.
* Establish a data base that records the number of hours employees have spent participating in employee research projects held during working hours -- by project and job function (not by employee.)
* Establish a Wizard Protection Committee with membership from all divisions that will review, and approve requests to conduct employee research on a regular basis. Include a market researcher on the committee.
* Require that a Request for Approval be submitted for every employee research project.
* Information that should be provided with the Request for Approval:
- Business need for the information
- The actions, decisions, and strategies will be based on this research
- Job functions of employees to be included
- Type of research to be conducted (e.g.; focus groups, survey, interviews, etc.)
- # of employees required to participate by each job function, at each company location, etc.
- Whether research will be conducted while employees are at work or outside of work
- Whether employees who participate will be anonymous (If not anonymous, include justification for why they must be identified.)
- Name & title of person requesting approval, with contact information
* Ensure that supervisors of employees who are asked to participate in employee research projects during working hours are notified that the research has been authorized. Give them the date the research will be conducted, how much time it will take, and how many of their employees will be asked to participate -- so they have time to adjust work or manpower if necessary.
* Send an invitation to employees requesting their participation. Be sure that employees understand that their participation is voluntary; why you need their help; and they can trust they will have anonymity and confidentiality. If you cannot promise complete anonymity or confidentiality, explain why and explain what will be done to ensure that they can respond candidly.
* If employees are asked to participate in research projects during working hours, ensure that measurements of their productivity are adjusted so they are not penalized for the time taken to participate in the research.
* If employees are asked to participate in research projects outside of working hours, explain how they will be compensated. (Employees should be compensated for research that requires them to travel, readjust their personal schedules, or spend more than half an hour to complete.)
Most of the time anonymity will be a requirement of employee research. To an employee, promises of anonymity or confidentiality mean that no one in your company will be able link the comments or information he/she provides to him/her.
Assuring anonymity is protection for both your employees and your company.
* For employees, it is a message that you really do want to know their attitudes and opinions, and you are promising that they will not be penalized for providing them.
* For the company, it is protection against getting corporate policy or procedures quoted back to you about the way things are supposed to be done – rather than finding out the way things are actually being done. It might also be legal protection. Any information a company collects that can be linked to an employee could be considered as part of an employee’s personnel record, and subject to subpoena.
* Don’t assume that employees will know what you mean if you say that responses will be strictly confidential or that employees will be anonymous. Be specific.
- Tell employees that no one in the company will be able to identify which employee provides comments or responses -- no names or employee identification of any kind will be kept with information an employee provides.
* Verify with researchers, or others conducting the research, that employees will be provided the anonymity and confidentiality you are promising. Ask them:
- How they are going to ensure employee anonymity and confidentiality.
- What the risks are that employee anonymity and confidentiality could be breached.
- If employees cannot be guaranteed anonymity and confidentiality, how they are going to be made aware of who will observe groups, view video-taped results, hear audio-taped results, or read comments linked to them.
* Employees should not be told the information they provide is confidential or their participation is anonymous if there is a risk that someone within the company can associate specific information to the employee who provided it.
If anonymity is important to encouraging employees to speak freely, can employee research ever be conducted without it? The answer is, yes -- with stipulation.
When Employee Anonymity or Confidentiality Cannot Be Assured
There are times when respondents must be identified. This is not a typical circumstance, but it does occur. Consider the following situation:
* New software is being proposed that will affect thousands of employees and cost a great deal of money.
* Management wants to conduct a test with 100 employees over a six month period to determine whether or not to purchase it.
* Researchers are being asked to measure employees’ attitudes toward their jobs and the software
- Before it is installed
- One week after each of 3 training sessions that will occur during the test period
- When the test period is over
* How employees in the test feel about the system will be a major factor in whether or not this software will be adopted.
* It will be important to measure the change in each employee’s attitudes at each of the above intervals in the test.
* In order to measure change in an employee’s attitudes, researchers must be able to track each employee’s attitudes at each interval so they can be compared from one interval to the next. Also, the ability to detect statistically reliable differences across the test group will improve if change can be measured for each employee.
* To accomplish the above goals, employees cannot remain anonymous during the test, and surveys will identify employees who are responding.
* It is important that each employee in the test feels free to say what he/she thinks.
What we have is a legitimate business need to identify employees in the test, and a need for employees to feel free to give their honest opinions, even though we cannot promise anonymity.
Consider this solution:
* Explain the situation to employees.
Explain to employees what measurements will be taken during the test, the importance of their being candid about how they feel, and the need to keep track of opinions for each employee over the test period.
* Explain the steps that will be taken to protect their anonymity even though surveys will identify them.
Explain that to fairly evaluate how employees feel about the software, researchers will need to identify the employees completing the surveys, but only researchers will know who is completing them. When the test is over, researchers will make the comparisons needed, and then drop all employee identification from survey files – so that when research results are provided, no one will be able to link survey responses to individual employees.
- Ideally we would like the researchers to be from an independent firm. But, given that in a test of this type, researchers may have to talk to employees to understand survey responses, the researchers may have be your employees. What is important is that employees have been told that only researchers will know who completed surveys, and that they will ensure employees’ privacy by removing information that could identify employees at the end of the test.
When Employees Conduct Employee Research – Reinforcing Responsibilities
Sometimes it is necessary for employees not trained in research methods to help conduct employee research. In these circumstances it is important to communicate to these employees how they must behave so that employees participating in the research will have anonymity.
Some of the best intentioned, most professional people have said, “No problem, of course I will promise to do that” -- and have failed.
The following scenario can help reinforce what is expected of individuals conducting employee research:
* A company installed a new system to distribute, and manage phone calls. It wanted to know how well it was working so some phone representatives and their managers were invited to attend focus group sessions.
* Because it was important to pursue discussion on technical issues if they arose, two employees who designed and installed the system were taking notes in the room with employee participants. Prior to the session, the two technical employees were trained on how to ask questions if they wanted to pursue a topic without intimidating employees in the group, and on their responsibilities to assure that employees would have anonymity. Without hesitation they gave their assurances.
* During the focus group session, employees were asked how they felt about the new phone system. One manager hotly proclaimed – “I hate it. I am a manager. Every day, I have meetings to attend. At the end of the day when everyone else is leaving, I have 20 to 30 messages from customers that I must return before I can go home. This system was supposed to help manage calls; it’s the worst system imaginable."
* After the group was over, the two technical employees were helping the moderator straighten up the room. One of them said, “Boy, how would you like to work with that manager at the end of the table? She must be really difficult to work with.”
- This technical designer violated the spirit of anonymity promised to employees in the focus groups. He judged the manager at the end of the table and probably would not choose her if he needed a line manager to help him in the future. So although he wouldn’t reveal the manager’s identity to others, the manager’s future in the company could be affected because she spoke her mind.
- If the second technical designer had formed the same opinion, even though he did not voice it; he too would have violated the manager’s trust that she would have anonymity.
- It’s hard not to be judgemental – but when you want employees to speak their minds, you owe them confidentiality and anonymity. This is why hiring external research firms to conduct employee research is the best option. It’s much easier for them to be objective, and to protect a respondent’s anonymity and confidentiality.
Remember to Always Protect Your Wizards
When conducting employee research, your company’s need for information should never pre-empt your employees’ feelings of security in knowing that their opinions are valued and they will not be penalized for being truthful.
Need for Personal Information Not in Personnel Files
Employee research can help employers understand their employees’ personal situations without compromising their privacy. Randomly selecting employees and ensuring their anonymity so they can provide personal information can help management make important decisions.
* Number of children, age of parents, etc. might not be appropriate to keep in a personnel file, but knowing what percent of employees have dependent children and dependent parents can help determine the need for dependent care spending accounts.
* Health issues are very private, but understanding the kinds of health issues employees face, can help determine what kind of health benefits to offer, the type of job flexibility that would most accommodate employees’ needs, etc.
* Understanding employees’ personal beliefs and values can help when training them to communicate company policies.
Using Market Research Expertise to Collect Unbiased Information
It takes specific training to know how to collect information that represents all employees without asking all of them for the information, and to ensure that the information provided accurately represents the views of all employees.
When conducting employee research, it is important to:
* Know the minimum number of employees needed to get reliable results.
* Know the best way to collect the information needed (online, phone, mail, in person, etc.).
* Ask the right questions to get comprehensive and unbiased answers.
Market researchers have this training and are the best professionals to provide it.
Qualitative Research – Talking to Your Wizards
There are times, when the best way to get the information you need is to have an interactive discussion with employees. The interactive discussion between the person asking questions and those answering is called qualitative research. Qualitative research has the following characteristics:
* The person asking questions, and those he/she talks with are able to hold a verbal or written discussion.
* Although the discussion is scripted to focus on selected topics or issues, employees are not limited to what they can say or how they say it.
* The discussion can go in unplanned directions to pursue something unexpected or not clearly understood.
* The purpose is to get a fuller understanding of issues from employees’ perspectives.
* Results are not quantifiable; they will not enable employers to project what percent of employees feel a certain way.
* It helps explore complex issues with employees; issues not easily addressed by asking a fixed set of questions with a fixed set of response choices.
There are several types of qualitative research; each has its own set of management challenges and requires some adjustment when employees are participating. We will pay special attention to what should be done to ensure employee anonymity and confidentiality.
Employee Focus Groups
Employee focus groups are discussions between a person working from a discussion outline (a moderator/facilitator) and a group of two to twelve employees. These groups can be held in person, or on-line.
* In-person Focus Groups
The traditional focus group is conducted in a room where a moderator/facilitator sits at the head of a table and participants are seated around the other sides.
- On the wall behind the moderator is a very large one way mirror. People behind the mirror can observe the focus groups without disrupting the discussion.
- Usually a camera behind the mirror videotapes the discussion so the group discussion can be viewed later.
- A tape recorder usually records the focus group discussion so it can be reviewed later.
The differences between a traditional focus group and an employee focus group are:
- In order to assure that employees can speak freely:
* No people from your company should observe the groups behind the mirror.
* The groups should not be videotaped.
* No people from your company should listen to the audio tapes.
* Employee participants should be instructed not to repeat any conversation that takes place in the group – even if it is a joke they thought was funny.
* Participants should be asked not to refer to co-workers or supervisors by name when they discuss situations and issues; as doing so might be unfair to employees not present.
Confidentiality and anonymity are more at risk when conducting in-person focus groups with employees than when using other research methodologies because employees in the focus group can breach one another’s confidentiality and anonymity by repeating what is said to someone outside the group.
Advice: Have a highly visible person in yourcompany (one whose voice is recognizable) pre-record a message that can be played at the beginning of employee focus groups that:
- Thanks employees for their participation.
- Asks participants to respect the privacy of one-another by not repeating anything said in the session.
- Asks participants to avoid referring to employees not present by name during the session, because doing so could be unfair to them.
* Most of the time, you will not need special facilities to conduct in-person, employee focus groups, so finding a location will be easier.
If you don’t have observers behind a mirror in the wall, or videotape sessions -- you don’t need facilities with a one way mirror or back room. You need a room that will seat the moderator and employee participants around a table, and the ability to tape record the sessions so the moderator can refer to them later.
- Using a room for focus groups that is located within your company, or at your company location has some advantages. People know where it is, transportation will be less of a problem if the group is held immediately before or after working hours (or during working hours), and it will probably be less expensive than renting other faciltities.
- On the other hand, employees might feel less inhibited when talking about company issues if they meet somewhere outside the company. (In most circumstances, location is not an impediment to discussion when employees have been assured anonymity, there are no observers, and sessions are not videotaped.)
* Deciding Whether Sessions Will Be Held During or Outside Working Hours
The advantages of holding in-person employee focus group sessions during working hours are that employees are available, and you don’t have to pay focus group participant fees. The disadvantages are that you are keeping them from job responsibilities, which sacrifices productivity and can interrupt operations.
An advantage of holding focus group sessions outside of working hours is that you are not interrupting operations in the workplace. Disadvantages are that employees have other commitments outside of work that can keep them from participating, and you will have to reimburse them for their time.
- Reimbursing employees for participating in employee research has financial implications you should be aware of. Anything you pay employees can be subject to benefits your company associates with salary, and subject to taxes, etc. Hiring independent market research suppliers to conduct focus groups, makes the supplier responsible for reimbursement, so payment for participation will not cause payroll issues for your employees or you.
* Hiring An Independent Moderator to Conduct Employee Focus Groups
It is desirable to hire an independent moderator to conduct focus groups. An independent moderator provides objectivity to the discussion, analysis and reporting of results, and is very credible when promising participants anonymity.
But, there are times when technical employees will need to be present to hear employees’ responses. They are able to detect that a response needs to be pursued in more detail, or that additional information is needed to fully understand what a respondent is saying.
Consider this scenario:
Your information technology department has installed a new billing system, and wants to know how line employees feel about it and what problems they are encountering. An independent moderator will not know enough about your company’s billing procedures and computer system to know when to ask for more detail about a problem that is mentioned. A way to handle this is:
* Include one or two technical employees from the information technology department to be in the room to observe the discussion. They can take notes, and determine if additional information is needed to clarify employees’ responses.
* As mentioned earlier, these technical employees will need special training on how to interact with employees, and how to ensure participant anonymity and confidentiality.
* Employees should be told that the observers in the room:
- Are employees in your company’s information technology department.
- Will be taking notes.
- Will ask questions if they don’t understand something.
- Will appreciate their candid observations and opinions.
- Will ensure that information discussed in the session is not attributed to any individual participant.
Virtual Focus Groups
Virtual focus groups are focus groups that are held online, over the phone or via some combination of phone and web-based communications. They are similar to in-person focus groups in that there is an objective moderator who leads a discussion among participants. In most virtual focus groups, participants and moderator cannot see one another.
Advantages of virtual focus groups are:
- There is no cost or time spent traveling to a facility.
- Employees in different geographical locations can participate in the same group.
- There is more emphasis on the content of the discussion when employees have no visual contact.
Disadvantages of virtual focus groups are:
- There is a loss of information when facial expressions and body language cannot be observed.
- Employee interaction is less fluid.
Start virtual focus groups off with a pre-recorded message from a highly visible person in your company who:
* Thanks employees for participating.
* Informs employees that no-one but the moderator and employees participating in the groups will be listening or observing groups.
* Assures them that their candid opinions are sought and that no one in the company will be provided any information from the focus groups that is linked to an individual in the group.
* Reminds them that each of them has a responsibility not to repeat anything mentioned during the session.
* In order to protect the rights of all employees in the company, asks them not to refer to co-workers or supervisors by name during the session.
One–on-One Employee Interviews
Advantages of one-on-one interviews are:
- They enable the interviewer to go into greater depth on a subject with a respondent.
- A respondent is not influenced by others’ opinions.
- It is easier to accommodate one person’s schedule when setting up the interview.
- It takes more time to interview one person at a time than holding a group discussion.
- Usually fewer employees participate, making it even more important to provide anonymity. (See above advice for playing a pre-recorded message at the beginning of the interview.)
Hands-on testing is a one-on-one interview that a facilitator conducts with an employee while he/she is using a system, device, or procedure under development. It is usually done in a controlled environment rather than in production. The facilitator asks the employee questions to understand how the employee perceives the system, device, or procedure, and why he/she responds as he/she does. The purpose of hands-on testing is to determine what changes are needed before putting a system, device or procedure into production.
Advantages of hands-on testing are:
- It helps detect and correct problems before the system, device or procedure is released for use by a large number of employees.
- It can support the development of prototypes to test employee receptivity to a system, device, or process.
- It can save a great deal of time, effort and money.
- A facilitator who is not a technical person can be more objective than professionals who developed the system, device or procedure when interpreting an employee’s reaction and use.
- A lot of effort is required by market researchers working with technical developers to create test scenarios to conduct hands-on testing.
- If employees experience significant problems with the system, device or procedure, retesting may be necessary after problems are fixed before development continues -- because employees who participate in testing strongly influence co-workers and may need to be convinced their needs can be met.
- If a system, device or procedure is very complicated, the interviewer might need a technical person to be present when hands-on testing is done. This will require training the technical person on how to ask for additional information during testing, without biasing participants’ responses, or intimidating them. It will also mean training the technical person on her/his responsibility to ensure that employees are not penalized in any way for the feedback they provide, or the way they provide it.
Quantitative Research – Surveying Your Wizards
Surveying your wizards when you want information can:
- Provide a representative view of how all employees feel, perform tasks, or perceive their workplace.
- Quantify the strength of employees’ feelings, opinions or evaluations.
- Ensure that all employees are asked the same questions, in the same consistent format.
- Enable each employee to speak for herself/himself without factoring in opinions of others.
Measures to Assure Anonymity When Surveying Employees
When surveying employees there are special measures you should take to ensure that respondents cannot be identified:
- Make sure the survey contains no employee identification -- no name; no address; no phone number; no employee identification number; no code put on the survey when it is sent out that can be linked to identify an employee.
- Make sure there are no questions on the survey that when answered can identify an employee. For example: If the survey has questions asking for an employee’s age group, gender and department – he may be the only male over 50 in the department – answering those questions identifies who he is.
- Establish a survey editing rule to ensure that if an employee’s name is included in a negative comment on a survey, that the name of the employee, and any information that could identify the employee, are removed immediately.
- Make sure the process for returning a survey will not enable someone to identify the employee who returned it.
Because online surveying is such a fast and inexpensive way to survey employees, it is very popular. However, TV news programs and detective shows have demonstrated to the public that information sent on a computer, even if erased, can be retrieved, so some employees believe that online surveys can be tracked to respondents. Unless you guarantee that no one in the company will ever see an employee identified with her/his survey responses, some employees will censor their responses.
One solution is to instruct employees to e-mail completed surveys to an independent research firm, and tell them that no one in your company will see individual responses.
Written Surveys returned via Post Office or Inter-Office Mail
Not only should the survey not have any information or questions on it that could identify an employee returning it; neither should the envelope or packet in which it is returned.
If an employee is called on the phone and asked to participate in an employee phone survey, he knows that he can be identified, and that it would be possible to link his survey answers to him.
A solution is to use an independent research firm and tell employees when they are called that an independent research firm is conducting the survey and no individual responses will be reported; only aggregated results will be provided to their company.
Deciding Whether To Survey Employees at Home or at Work
Consider the following when trying to decide if your employees should be surveyed at work or at home:
- Anonymity: Is surveying employees at home necessary to provide anonymity?
- Employee Accessibility: Where is the best place to send the survey to be sure employees will receive it?
- Best Location for Employees to Complete the Survey: To answer survey questions, will employees need to refer to information that is kept at work or at home? Is the employee’s home or work environment more suitable for completing the survey? (For example: If an employee works outdoors, will he/she be able to complete the survey while at work?)
- Cost: What is the cost to survey an employee at home vs. at work – considering productive time lost at work, response rate you could expect if survey is sent home, vs. work, etc.
The Return Trip from OZ -- Reporting Results of Employee Research
When you conduct employee research, you ask employees to be open and candid with you. When you get results, it is your turn to be open and candid with them. How you report research results will impact how your employees respond to research requests in the future, and how they feel about “their company.”
Advice: Thanking Your Wizards for Their Help
* Develop a communications plan for sharing employee research results that:
- Optimizes the opportunity to build company morale.
- Demonstrates the value you place on the information employees provided.
- Shares the actions and decisions that will be made based on results.
* Share results that are positive and negative, above average and below average, expected and unexpected. It indicates your desire to be open and candid with employees, and builds trust.
* If you measured attitudes, opinions or evaluations on a numerical scale, think twice before reporting numerical averages as results. Most often an average value of 6.5 doesn’t communicate anything meaningful – unless your employees are statisticians
– and even then your interpretation of the numbers is much more important than the numbers themselves.
- Without context, numbers mean little – emphasize the context. Any information you can provide with results that can help employees put them in perspective is helpful. Comparing results to industry results or indicating results are average, above average or outstanding puts results in a context that is more meaningful and memorable.
- It can be counter productive to report numbers. Employees will remember the numbers reported. The next time a similar survey is conducted and the corresponding numbers are not exactly the same, employees will spend time looking for explanations for why the numbers are different -- when statistically the results could indicate that no change is detectable.
* In general, results for a unit should be reported only to that unit, or to a group formed to identify and make improvements. When reporting results for more than four or five departments/units/locations to all employees, identifying the unit with the highest score will recognize achievement and can help benchmarking activities. However, reporting units with low or below average scores to all employees can be counter-productive.
May be copied and shared in its entirety. Modifying content or selling this document is a violation of copyright.