There’s a lot of talk in a lot of places about online qualitative research. What size? What platform? Who do you recruit? How do you recruit them? How do you pay them / reward them? Do you pay them?
These are all important questions and tricky questions – and I don’t think we have the perfect answers to them. However, I thought I would share my experience about what makes a good online qualitative experience.
A couple of caveats. My experience of qualitative research has been largely limited to what I would call Online Qualitative Forums. I’ve heard them called bulletin boards, asynchronous discussion forums and many other terms. I’ve been running online qualitative forums for the past 6 years or so, having used a number of different and evolving platforms and having researched a number of different topics.
I’ve not had a great deal of experience with larger MROCs – I find them fascinating, but I’ve never found the project or the client to work with them in Australia. However, I think many of these learnings can be applied (or adapted) to these larger communities.
So, if you’re thinking of taking the leap and running an online qualitative project, here are my top tips…
- Consider length and size: Personally, I like smaller and shorter forums. Honestly, this is largely because I’ve never worked on a large MROC, but I also like the intimacy and intenseness of a smaller group. I aim for around 30-40 participants over about a period of 7 days. I find that this way I’m able to keep on top of the board, what’s happening, who’s saying what and how the participants are finding the experience.
- Structure your incentives: It is hard in an online forum to keep participants committed for the full week. To help with this, I tend to structure the incentives so that they are tied to participation. An amount for signing on, and then incentives for each of the next days work, and then something extra for completing. Bonuses for good work, extra tasks can also help to motivate and engage.
- Be respectful of your participants and their time: Be honest and realistic in your exceptions of your participants. This is something I think we need to look at in all aspects of qualitative research, but particularly in online qual. Why? Because, unlike face-to-face methods, in online your participants aren’t stuck in a room with you – if you ask them to do too much (for the time, or the money you’ve outlined) there is a really big chance that they will walk away. How long will the task realistically take to complete? If you’ve said 20 minutes a day – then make sure the tasks will only take 20 minutes a day!
- Make yourself as real as possible: when I’m running an online forum I try to make myself as real as possible to my participants. In a focus group you are there, meeting and building a relationship with your participants. You build a sense of common purpose and a sense of working towards the same goal. In an online forum you need to work a lot harder to build that same sense of purpose. It is important to make yourself as ‘real’ to your participants – not a faceless ‘bot’ that is asking them questions, but part of the community, working towards that goal. Upload a photo, use a causal tone, provide alternative contact methods for your participants (phone number, email address) – will all help to build this relationship.
- Remember who you are talking to: I think we get really excited with the technology – now, we can ask participants to upload photos, upload videos, upload screen shots – which is all fantastic and rich information. But what we need to remember is not everyone spends their day at a desk, using the Internet. Not everyone is as computer savvy as we are. This goes back to the point before – it might take you 5 minutes to find and upload a photo, but this task is significantly harder for someone with less computer nous, or a dial up connection.
- The fine line between clarity in questions and creating a discussion: The key thing to remember is that you are not there with them to correct misinterpretations, follow up probe, get them back on topic. Participants (bless ‘em) will answer the question that they want to answer, not necessarily the question you’ve asked. So to overcome that, it’s important that you are clear and focussed in your questioning (if there is an option to bold or italicise key words in your questions – this does increase them paying attention to what you’ve actually asked them!). It is best each question only ask one thing (that is the only way to make sure they answer that one thing) – Some platforms allow for closed and open ended questions and multiple questions in one activity – if you have that option, this is a better way to collect a lot of specific information on a wider topic.
- Where do they come from: One of my earliest learnings with online qual was the difference response I would get from participants recruited from online quantitative panels and those recruited through traditional qualitative recruiters (using offline methods). Again, I’ll admit, it has now been a few years since I’ve used an online panel to recruit, but I’ve generally had much more success with participants recruited from Qualitative databases. Why? Because the participants frame of reference is different – quant participants recruited to a forum, see the questions from their quant questionnaire experience. They generally give shorter and less detailed answers. In contrast, participants coming from traditional qualitative channels, see the board more like other qualitative projects they’ve been asked to participate in, and are generally more descriptive and detailed in their answers. Using a qualitative recruiter also means that they a greater sense of accountability and responsibility to participate. It also provides you with another way of following up on them, should their participation levels decrease.
- Structure your tasks – Forming, Norming, Storming and Mourning: If you’re running a board over a shorter period, I find that structuring and pacing the questions along the lines of a traditional focus group discussion guide. Day One: Keep it light, simple questions and tasks so that participants can feel their way through the platform and understand how to respond and what will be expected of them (monitor this first step closely and be highly visible to help with the “forming” process). The next few days allow us get to know each other and the rules – then in the final days we hit full speed and can throw more complex and difficult tasks. At the end, make sure you give participants the opportunity to provide feedback (or even to continue their conversations), to help with that mourning process.
But I’m interested in your thoughts. What has lead to online qual success for you? Would you challenge or agree with my points?