How to Avoid Screener Fatigue

It’s well-known among marketing researchers that respondents prefer shorter surveys. Studies on questionnaire length and fatigue effects concluded that surveys and screeners should be less than 20 minutes to ensure data integrity. But why?

After 20 minutes, fatigue sets in: the respondent starts satisficing, where their attention span decreases and the speed with which they answer questions increases.

As a researcher, how can you design a screener that answers critical questions without disengaging your respondents? In addition to streamlining your screeners, here are 3 ways to get high quality data in a limited number of questions.

Ask Opened-Ended Questions Early
Research shows that the number of characters a respondent types in an open-ended text box decreases as those questions are moved back in the screener.  The more information the respondent is asked for, the less likely they are to give insightful answers. To garner rich responses to open-ended questions, design your screener so that easier, multi-choice questions are at the end.

Place “Skippable” Questions in the Beginning
In one famous SSI study, researchers placed a sliding-scale question, where the gauge was placed in the middle, at different points throughout a screener. The later this question was placed in a survey, the more likely the respondent was to leave the slider at its starting position. To avoid having respondents leave questions unanswered, make sure that each question is blank to start, giving the respondent a clean slate for their answer.

Deliberately Phrase Questions
The way information is presented to respondents greatly changes the way they respond to that information. Instead of asking, “Did you watch TV this week?” ask, “When was the last time you watched TV?” Not only does this tactic collect more specific answers, it also ensures that respondent’s aren’t intentionally answering questions in such a way that lets them skip question blocks.

The bottom line is that when respondents get tired, they do as little work as possible to finish the task at hand. Remember: there is far less satisficing and cheating in short surveys.

Have an expert at Focus Pointe Global evaluate your screener for potential fatigue effects by clicking the button below and scheduling a consultation.

If you’d like future Focus Pointe Global blog posts sent directly to your inbox, simply enter your email address in the field to the right to subscribe.

8 Hot Insider Tips from Years of Moderating

By Rich Thau

I've never blogged before about the art of structuring and moderating a successful focus group, but since the folks at Focus Pointe Global asked me to do this, I'm honored to take a stab at it. So, here goes, in sequential order from recruiting to executing:

1) How do you avoid recruiting highly opinionated people into your group so that they don't try to dominate the conversation? Well, there's no sure-fire way of achieving that, but many years ago I started inserting a question into the screener that seems to have helped mitigate the problem: Ask whether the prospective respondents have any bumper stickers on their cars that convey a political message. If they do, that's telling you something: They want to try to convert others to their point of view. You probably don't want them in your group.

2) If you want typical adults ages 22-64 in your groups, try not to host the groups before 5:30 p.m. Sessions conducted during business hours on a weekday tend to be filled with people who are not ideal respondents. Occasionally lunch-time sessions attract good respondents. Also, I've done successful groups over the weekend—but who wants to make a habit of doing groups on a weekend?

3) It makes sense to meet the recruits before bringing them in for the session. Sometimes you spot someone who shouldn't be there—perhaps they're asleep, or not fully engaged—and taking a quick look can help avoid some uncomfortable moments during a group. It also helps to ask folks stationed at the welcome desk if any respondents look familiar, like “professional respondents.”

4) I almost always re-screen groups by asking a key question differently in the re-screening than in the original screening. It's a great way to find people who should have been screened out originally, but weren't. I also regularly try to understand discrepancies between the original screening and the re-screening by pulling people out of the waiting room and asking them to explain the discrepancy. Sometimes the gap is explainable, and sometimes it's not. And when it's not, I've often paid the respondent and sent him/her home.

5) Nothing sets me off more than respondents who try to game the focus group system—they arrive three or four minutes after the group starts, so they expect to get paid without having to attend. I make it a hard rule with facilities: If the respondent isn't there by the time the group starts, they don't get paid unless I take them while in progress (something I am loath to do). The recruiting screener told them they needed to be there 20 minutes before the session starts, so if they are late, no payment.

6) My firm specializes in moment-to-moment dial testing, where we get continuous feedback on video presentations of various types during the focus group. If you ever conduct a session using the dials for this purpose, you MUST not only give detailed instructions, but also conduct a quick warm-up to make sure the instructions are understood. After moderating hundreds of sessions with the dials, I've noticed that there's at least one person in each group who does not follow the instructions—and as moderator I don't know who it is, what they're doing wrong, or how to set them straight, until I do a warm-up.

7) Never moderate groups back-to-back; always take a half hour break between groups to do a "brain dump" of what you learned, eat something, and consult with the client. We typically run groups from 5:30 p.m. to 7:30 p.m., and then from 8:00 p.m. to 10:00 p.m.

8) The most effective time-saving device I've come across for report writing is to do a digital audio backup of my session with my own recorder, and to sync the time on the recorder to the time on an easy-to-read digital stop watch that I place in front of me. When someone says something I want to make sure not to miss, I write down the name of the person and something they said, along with the time, so my associates and I can find it easily on the audio afterwards.

--

Rich Thau is President and Founder of Manhattan-based Presentation Testing, Inc. (http://www.presentationtesting.com). You can reach him at richthau@presentationtesting.com or 212-760-4358.

If you’d like future Focus Pointe Global blog posts sent directly to your inbox, enter your email address in the field to the right to subscribe.

The opinions expressed in this blog post are Rich Thau's and not necessarily those of Focus Pointe Global. 

How To Avoid Focus Group Bias: Fibs, Half-Truths And Outright Lies

If you’ve been following this week’s investigation into avoiding focus group bias, then you know how leading questions and moderator neutrality both play an important role in generating the most honest, open and accurate focus group results possible.

Of course, even if your focus group questions are expertly formulated and presented in a completely objective manner by your moderator, there are a number of other opportunities for focus group bias that must be carefully monitored and managed. We’ve compiled the top 5 for you here:

The Sponsor Bias -- If your focus group respondents are aware of who is sponsoring your marketing research project, they may be inclined to temper their true opinions in response. If they have negative feelings, those feelings may automatically influence their answers negatively. Similarly, if they have positive feelings for the sponsor, they may be inclined to only provide responses that portray the sponsor in a positive light. Either way, the outcome is less than objective results. For this reason, it’s important to conceal the name of a study’s sponsor whenever possible.

The Consistency Bias -- Without intending to lie or mislead the moderator, many respondents will edit their answers over the course of a focus group session based on previous answers that may or may not have been completely accurate or truthful. Of course, as anyone who has ever tried to tell an extended lie knows: maintaining consistency isn’t easy. If an answer does not seem consistent with previous responses, it’s important to ask the respondent to expand on or clarify their answers.

The Domination Bias -- If left unchecked, dominant respondents intent on monopolizing a focus group conversation with their viewpoints and opinions can quickly sway the answers of others. Be sure to engage every member of your focus group and encourage equal participation from all members in order to avoid allowing one participant’s responses to influence the entire group.

The Bad Mood Bias -- The mood of your respondents will naturally affect the answers they provide. If a focus group participant is angry, he or she may offer consistently negative or hostile responses. Similarly, if a respondent is depressed or distracted, that state of mind may translate into short, terse responses. Try to gauge the mood of your group, make note of its influence and respond by drawing out reticent respondents or defusing group tension with a change in approach.

The Social Acceptance Bias -- Human beings are naturally social and are therefore inclined to seek the acceptance of their peers. When it comes to focus group responses, this tendency can have a significant impact on data results. Focus group participants are naturally inclined to offer socially acceptable responses that may be false or only partially true. This phenomenon is especially prominent when dealing with sensitive subjects that participants would rather not talk about. In order to mitigate the effects of the social acceptance bias, it’s important to build trust among all members of your respondent group and try to overcome individual insecurities. When dealing with socially sensitive topics, try using indirect questions and projective techniques.

For more information on how to avoid and minimize focus group bias, click on the button below to schedule a consultation with one of the experts at Focus Pointe Global.

Visit the Focus Pointe Global blog next week to learn how to reach Hispanic populations with your marketing research. If you’d like this and all future FPG blog posts sent directly to your inbox, simply enter your email address in the field to the right to subscribe.

How To Avoid Focus Group Bias: Leading Questions

The goal of every qualitative marketing research project is to gather the kind of open and honest consumer feedback that generates accurate, viable and valuable data results for your clients. But there is one big obstacle that stands in the way of your ability to achieve this goal: bias.

Bias is inevitable. As human beings, we have a natural desire to please others. When it comes to question-and-answer situations like focus groups, that desire translates into a tendency to edit and adjust our responses. People are wired to project and respond to countless verbal and non-verbal clues, often without even realizing it. As a result, there is simply no such thing as completely unbiased focus group results.

But don’t despair … while bias cannot ever be entirely eliminated, there are a number of things you can do to limit its effects and keep your focus group data results as objective as possible. This week on the Focus Pointe Global blog, we’re taking an in-depth look at ways to combat bias in the focus group environment.

First up, it’s important to avoid asking leading questions. Much like “leading the witness” in a courtroom context, leading questions contain clues that suggest a “correct” or desired answer. Focus group participants tend to pick up on even the subtlest hints and adjust their answers accordingly. Some example of leading questions include:

• “Do you think cola drinks are bad for you?”
• “Is recycling a good idea?”
• “9 out of 10 dentists recommend brushing after every meal … what do you think?”

The best way to avoid asking biased questions that lead respondents in a particular direction is to leave your questions open. Instead of inserting a value (e.g. good or bad in the first two examples above) or beginning your question with an authoritative statement, try reformulating your questions to sound more like the following:

• “What is your opinion of cola drinks?”
• “How do you feel about recycling?”
• “What times of day do you brush your teeth?”

By keeping your questions neutral and actively inviting respondents to share their opinions, viewpoints and behaviors in a non-judgmental way, you drastically reduce bias and increase your chances of gathering the most honest and accurate data possible.

To learn more about how to ask the right focus group questions, click here for a look at last week’s discussion of how to encourage focus group participation.

For more valuable guidance to help you reduce and mitigate focus group bias, check the FPG blog later this week or click on the button below to schedule a consultation with one of the experts at Focus Pointe Global.

If you’d like future Focus Pointe Global blog posts sent directly to your inbox, simply enter your email address in the field to the right to subscribe.