DIAL LOG

Focus Groups… Haters Gonna Hate

focus groups - challenges and improvements
Focus groups have been getting a lot of flack lately. Too expensive. Too time-consuming. What can you really learn from eight people in a room? Well, “expensive” is relative, as is “time-consuming,” and if you’re solely relying on eight people in a room to help you make business decisions then you deserve what you get.

Observation techniques, social listening, big data, etc., are all ways to learn what people think and how they behave. In his Extreme Market Research blog post, marketer and cartoonist (marketoonist) Tom Fishburne highlights how household products maker Seventh Generation uses social listening to, “Listen in on the conversations [millennial] moms are having and to really see their problems.” In contrast, Tom goes on to point out that, “Focus groups… give false reads because respondents aren’t reliable when describing their own behaviors and preferences, particularly in an artificial environment like a focus group.”

Tom is 100% right. People are not very good at describing, in retrospect, their behaviors and preferences. In fact, they are even worse at accurately remembering what they thought and felt in the past. In another of his posts, All-knowing Focus Group, Tom writes about how, “Most consumers don’t really know what they want, and if they do, they have a difficult time articulating it.”

Once again, Tom is spot on. Let’s look at two ways to mitigate flawed recall in your projects.

#1- Ranking is more meaningful than Binary Choices

We designed a test to mitigate the typical binary research paradigm: “Do you like this? Yes/No”.

We equipped a University of Oregon market research class with handheld dials and conducted a beverage taste test. Instead of tasting the beverages and giving us their verbal or written opinions, we changed it up. We instead used the dials to conduct a trade-off exercise that evaluated each beverage on four key attributes: appearance, aroma, overall taste, and aftertaste.

To get a baseline, we had participants rank each beverage attribute for importance to them. This gave us relative metrics for how much more important one attribute is relative to the other three. This is a huge distinction because an attribute being slightly more/less important, vs strongly more/less important, factors into decision-making. A simple rank order won’t give you that critical level of detail.

Then, we had them score each beverage, individually and in-the-moment using the dials, on each attribute after looking at it, smelling it, tasting it, and letting the flavor linger for a moment. The immediate result was a dataset for each beverage based on weighted scores for each attribute, a rank-ordered preference of all the beverages based on their scores, and a final forced-choice favorite picked by each participant.

Participants didn’t have to try to remember what they thought and they didn’t have to articulate it. They just had to drink, think and react. Researchers can then use that real-time, in-the-moment data to drive group discussion and start learning why people reacted the way they did.

#2- Market Research ‘To-Go’ Captures Reality Better

Over the years, we’ve conducted research sessions in countless unconventional environments: restaurants, arenas, bars, nightclubs, and Broadway theaters, to name a few. In one case, we worked with our client to simulate a retail store to test a new clothing line.

Participants completed a number of activities ranging from browsing the selection to actually selecting clothing they would be interested in. Along the way, they answered questions with their dials in order to capture their in-the-moment thoughts and feelings. With this data, Researchers can now prompt participants for deeper feedback based on key “moments of truth”.

Now, Disable Your Eject Button

Instead of giving up on focus groups all together, simply change your approach. Research is always evolving. The most effective research methods and designs are not those that rely simply on one approach. They are those that apply elements of multiple approaches (hybrid and/or mixed-mode) to meet research objectives. While Tom is right about focus groups in a traditional sense.  But the benefits of a traditional focus group are unique and still deliver strong value, especially when combined with other tools and techniques. I see this day-in and day-out with the dial testing work we do for our research partners and clients.

We love this stuff and would love to help you. Let’s connect and chat about it. We promise to leave you with plenty to think about.

We’ve also extensively studied the area of recall and memory related to market research and collected our articles, interviews, and panel discussions HERE. Hope you can hop over and take a look.

No comments yet.

Leave a Comment