Luke Skywalker can utilise his Jedi mind control and Professor Xavier can control and read people's thoughts…
Sadly for us (but gladly for the world at large!), we as researchers don’t quite possess the same abilities. Nevertheless, the influx of learnings from Behavioural Economics is making it increasingly clear that extraneous factors can heavily impact on people's decisions and actions – and therefore their responses when asked a question.
Let me explain this further with an allusion to something that the human race is certainly very good at…
Behavioural economist Dan Ariely has conducted a raft of experiments in an effort to understand more about the hidden forces that shape our decisions. In one such experiment Ariely set about trying to understand the role arousal can play on people's sexual and moral decision making. More specifically, Ariely wanted to study the extent to which people can predict how their attitudes will change when sexually aroused.
To do this, Ariely asked respondents (who it should be noted were all male undergraduate students) to answer sets of questions relating to (i) sexual preferences, (ii) likelihood of engaging in immoral behaviours and (iii) likelihood of engaging in behaviours related to unsafe sex. Questions were displayed on a computer as visual-analogue scales from ‘No’ (0) to ‘Possibly’ (50) to ‘Yes’ (100). The twist – as part of this experiment respondents had to answer these questions when either in a ‘cold’ (sexually un-aroused) state or ‘hot’ (sexually aroused) state.
Interestingly, when the numbers were crunched, the results were startlingly clear: respondents who were in a ‘cold’ state were much more conservative in their preferences and predicted behaviours whilst those in a ‘hot’ state were much more likely to consider a range of activities attractive and engage in immoral behaviours.
Impact for Researchers…
So, from Jedi mind control and mutant mental abilities, to sexually aroused decision making, what does this have to do with survey design?
Well, as Ariely points out, whilst in the main his results apply to sexual arousal and the influence it plays on who we are, “we can also assume that other emotional states (anger, hunger, excitement, jealousy and so on) work in similar ways”. If this position holds, then we need to be re-doubling our efforts when it comes to designing our surveys – as we all know, ‘Anger leads to the dark side.’
A quick five minutes searching Google illustrates this point further and brings up blog post after blog post lamenting the experiences that some respondents have had to go through in the quest to give their opinion.
The now unmasked Angry MR Client, who periodically put the industry to rights over the last couple of years, recently (under the guise of Mr R. E. Spondent – see here) hammered home some of the pitfalls that surveys can fall into and alluded to the emotions that certain poor practices can create.
Additionally, comments from other posts such as “By this point, I am almost in tears” and “I admit that some questionnaires can be a bit sloppy or even annoying” further serve to illustrate the emotions that poorly designed questionnaires can conjure up. It is unfortunate therefore that we don’t always have access to the levels of emotions being felt by individual respondents when they take part in a survey, as the tone from the above examples would suggest that respondents are, at times, less than happy with their experience. If this is the case then the impact of poor survey design (and the anger/frustration that it can create) on the validity of responses, the outcomes of any research and the reputation of our industry in general must be considered.
So what do we need to do?
In short, do what we say, and deliver on our promises. Not every survey can be 10 single-code questions and a maximum of 5 minutes – but when they’re not, be honest with respondents and let them know. Aside from that, let’s keep the language and design as simple and respondent friendly as possible – over-complication and unnecessary additions should, like Jar-Jar Binks, be confined to a Galaxy Far, Far Away. Author: Alex McCluckie - Senior Research Executive at DJS Research :: Find him on Twitter
For more information regarding this article please visit: www.djsresearch.co.uk | @DJSResearch
Follow Market Research World on Twitter or join in the conversation with our LinkedIn Group