Laugh, cringe or cry - we’ve all heard horror stories of respondent recruitment failures. In our recent survey on recruitment, researchers from around the world told us the main challenges they face when recruiting research participants. Whether it’s ‘professional’ repeat respondents, participants not matching the required specifications, or even worse, lying to get through screening.
The participants aren’t representative consumers
'Ensuring that the participants closely match the recruitment specifications’ was the number one concern for researchers in our survey.
The best recruiters and sample providers will be able to match complex criteria or find hard-to-reach groups based on real, not claimed, behaviour. However, we heard of many incidences when researchers found the participants to be unrepresentative and completely unsuitable.
A colleague started a focus group about pet food by confirming that all participants were cat owners. Somebody quickly came from behind the mirror to clarify that the group was actually about DOG food. All the participants were magically transformed into dog owners!
Participants and recruiters are untruthful or inattentive
A professional participant’s desire to take part in regular projects raises other concerns about validity as people have been known to bend the truth to match criteria or say what they think researchers want to hear to ensure they’re included.
In a focus group about pregnant women we discovered a participant wearing a pillow under her gown to simulate pregnancy.
Many researchers told us they’d seen the same participants regularly turn up for different projects claiming different demographics, ownerships and buying intentions. The researchers found these experiences professionally embarrassing, particularly when the truth was revealed in front of clients.
I once had a respondent turn up twice to groups for the same client within a month, once as someone looking for an IFA (Independent Financial Advisor) and once as someone wanting to become an IFA. I was angry and embarrassed in front of the client. I felt undermined as the person was recruited twice by the same recruiter.
One researcher was suspicious when a woman in her group kept shifting her opinion to match that of another participant. They acted as if they didn't know each other so the researcher assumed the woman was "one of those respondents who would say anything in order to be accepted within the group". After the group the researcher checked all the participants on Facebook and found out that the participants were a couple. The researcher was disgusted and contacted the recruiter to find out what had happened. It turned out, they had booked them both intentionally, without considering sharing this with the researcher. Apparently, it was easier to recruit two people with one phone call and they were more certain they would come because it would mean double incentive for the household. The researcher made it clear they were unhappy about the unethical working but was left wondering whether it was still happening.
…hearing the recruiters (based in a viewing facility) briefing respondents for my next project on how to present themselves (i.e. lie).
Professional participants and repeat respondents
Have you spoken to the same people again and again? Professional or repeat respondents was a major concern for the researchers we surveyed.
We all know that some people view responding to market research as a regular source of income. We also know that frequent participation makes an individual’s responses less valid as they become less representative of a typical consumer.
A focus group with eight participants… one participant asked when we will be doing a personification exercise and maybe it should have been already done. Clients behind the one way mirror were really annoyed, I had to cancel the group and replace it at my company's cost.
When asked about bad recruitment experiences one researcher simply stated “Respondents becoming too familiar with subject matter. Makes you wonder how respondents that have never taken part in research would respond to stimuli.”
If you’ve experienced anything like these horror stories you’ll benefit from using Behavioural Recruitment, powered by Facebook, which finds the right people for each project, based on real behaviour, for online or face-to-face research worldwide.
We heard many shocking participant recruitment stories from the researchers who took part in Research on Recruitment 2018 - enter your details below to receive the full report and read more…