Think about the items you put in your cart during your last trip to the grocery store. Recall the new fast-food value meal you saw in a commercial, or the app you use to check your bank account balance, or the packaging of the shampoo you purchase for your dog. It’s extremely likely that sometime before any of these products saw the light of day, a half-dozen or so strangers got together, sat down in a conference room with a one-way mirror, and debated their merits in exchange for a couple hundred dollars each.
We now live firmly in the age of big data. Every link we click is tracked and cataloged. We rate our Uber drivers and the things we buy on Amazon and the cleanliness of airport bathrooms. The movies and TV shows we’re prompted to stream on Netflix are based on algorithms that analyze everything we’ve ever watched before. But when designers and marketers need to come up with new ideas or vet products before trying to sell them, they still turn to the same lo-fi method that’s been in use for decades: putting a bunch of people in a room and having them hash it out as part of a focus group.
Focus groups might seem like a throwback — they came to prominence in the middle of the last century — and their demise has been predicted many times over in recent years. But for as many times as they’ve been declared dead, a victim of ever-improving digital analytics, they haven’t gone anywhere. In 2017, $2.2 billion worldwide was spent on conducting focus groups, according to the trade group ESOMAR, with $809 million of that coming in the US.
In the 1950s, focus groups famously led Mattel to make Barbie one of the first adult-looking toy dolls, and in 2014, they convinced the company to introduce a new, “curvier” Barbie to appeal to a more varied audience. Today, corporations use focus groups to study and sell everything from frozen foods to summer blockbusters. Political parties use them to hone campaign messaging, as do US government agencies concerned about public reaction to policy changes. Law firms even use them when trying to gauge how juries will respond to an argument or line of questioning.
Like most of us, I’ve spent my life consuming products shaped by focus groups. So a few months ago, I decided to become an active participant in them. After entering my name in recruiting databases and filling out countless surveys, I took part in seven different groups, offering my views on unique beverages, novel insurance concepts, and new snack flavors. I’ve been asked if I prefer the taste of formula A or formula B, but I’ve spent much more time answering questions that are far less straightforward: which stock images, for instance, most accurately reflect my relationship with a brand. I’ve heard participants contradict what they said a moment beforehand; I’ve met regulars on the focus group circuit who routinely sell their opinions as a side hustle.
All this has left me with a question I keep asking myself as I fill out survey after survey. Focus groups have a hand in designing so many of the things we own, and the cost of conducting them gets passed on to us in the prices we pay for these products. But are they a unique window into our tastes and innermost desires, or an elaborate waste of time and money?
I started my search with help from a friend, a focus group regular for the better part of 10 years, whom I’ll call Claude (he’s asked me not to use his name so as to preserve his future earnings). He showed me the websites where I could sign up to receive recruiting surveys and suggested what sorts of answers could land me in studies: It’s good to say you’re responsible for the majority of your household’s grocery shopping, and that you’re open to trying any brand of a given product.
After filling out a few surveys, I began to get calls from recruiters, who would painstakingly confirm all my survey answers before inviting me to sessions that typically went for an hour or two and paid $100 or so. Often, they added, if I showed up early, I’d be entered into a raffle to win a cash bonus.
I never did win one of these raffles — one of a few trends I saw over and over again as I became a focus group regular myself. The rooms look like you might imagine: flavorless conference rooms with wall-to-wall mirrors at one end that you can just barely see through if you look at the right angle. The facilitators generally begin by asking everyone to go around and say a fun fact about themselves; they seem trained to reply to every comment, no matter how uninspired, with praise. The atmosphere is usually somewhere between that of a holiday office party and jury duty. People generally do make an effort, but there’s also a clear sense that everyone is mainly looking forward to leaving as soon as the hour is up, their obligation fulfilled, so they can grab their check on the way out the door.
While we might be shown ad mockups for new products or offered flavors to taste, vague, open-ended exercises are the real bread and butter of these sessions. I was often asked to relate personal experiences I’ve had with a product, or describe scenarios that might prompt me to buy it. More than once, I was asked to close my eyes and visualize scenes I associate with a brand. In theory, these kinds of activities are meant to probe for emotions or product associations we might not even realize we hold.
This is a tactic that dates all the way back to the accidental invention of the focus group in 1941. As Liza Featherstone details in her fascinating history Divining Desire: Focus Groups and the Culture of Consultation, the idea came out of Columbia sociologist Robert Merton’s impromptu visit to a test airing of This Is War, a US Office of War Information radio program meant to counter Nazi propaganda. His Columbia colleague Paul Lazarsfeld had given the test listeners a device with buttons to record whenever they liked or disliked the broadcast — itself a major market research innovation — while another researcher followed up by asking the audience why they had pressed the buttons at various moments. “[Merton] began to observe the interview keenly, and to send Lazarsfeld notes,” Featherstone writes. “Lazarsfeld asked Merton, if he thought he could do so much better, why not try it himself?”
Merton did so, and over the course of World War II would go on to develop what he called “The Focused Interview”: a discussion emphasizing the “subjective experiences” that shape a person’s response to a particular item, whether a radio program or box of cereal. Rather than collecting raw numbers on how many people preferred one product or the other, Merton sought to dig into “the relevant personal context, the idiosyncratic associations, beliefs, and ideas” that quietly colored people’s preferences, often without their explicit knowledge. He felt it especially important to include open-ended questions so participants could move the conversation in directions meaningful to them, and to drill down to the underlying, sometimes subconscious why of their opinions — not just the what.
The interviews, borrowing heavily from psychology and anthropology research methods, contrasted sharply with the statistical methods used in market research at the time. Madison Avenue ad firms were soon hiring Merton and others to hold focused interviews — usually with a group of people at once — to gauge what consumers wanted to buy, and how to sell it to them.
This era spawned a flurry of Freud-flavored focus group success stories of varying plausibility that are nonetheless celebrated to this day. Ernest Dichter, the psychologist who’d coin the term “focus group,” used them to determine that people supposedly preferred borrowing money from loan sharks to banks because they felt judged by bankers’ stern authority — and recommended that banks offer checking accounts with overdraft, so people wouldn’t have to come in to ask for a loan.
Focus groups led another psychologist, Herta Herzog (the inspiration for Mad Men’s Greta Guttman), to proclaim that for men, cars represented the ultimate fulfillment of repressed sexual desires, pushing auto companies to put out increasingly sexualized advertisements throughout the ’50s.
Focus groups still seem to be preoccupied with emotion, but the way these findings are incorporated into market research has become a whole lot more rigorous over the years. “We always recommend to all of our clients that they do both qualitative and quantitative research,” says Andrew Tuck of ARC Research, which conducts polling and statistical analysis alongside focus groups. “You can’t talk to 10 people outside St. Paul and make a good decision based on that.”
Like many in the industry, he feels that focus groups can be useful for generating hypotheses at the outset of a project so they can later be tested quantitatively, rather than for ranking option A versus option B right before one has to go to market. As Kristen Miles of Branded Research puts it, “With new product development, companies often don’t even know where to start. We need people to tell us as researchers what we should focus on.”
If a client wants to run focus groups, ARC will carry them out with one of a few national panel companies: firms that host sessions, often in their own facilities, and recruit participants by emailing out surveys. “Recruiting participants is usually the most expensive part,” Tuck says, explaining that finding the right people — often based on whether they already buy the client’s products, or those of a competitor in the same category — can lead to a total cost of around $10,000 per session for the client. After finding the right people who promise to show up at the right time, the panel company will typically hold sessions in a few major US cities. The clients themselves will often gather behind the one-way mirror, bowls of snacks and popcorn at hand, to watch the proceedings.
“It’s like you’re looking at ants from above,” says David Reischer, who used focus groups in deciding on a domain name for his website LegalAdvice.com. He remembers being fascinated watching people pick apart various names his team had come up with, unaware that the creators of the list sat feet away behind a thin layer of glass and aluminum.
Steve Thomas, a former executive at Edible Arrangements, says he liked to see participants’ behavior from the moment they walked in the door. “Tasting the product always generates a lot of robust discussion,” he says. “But as a client, we are really watching for responses much earlier. How did they respond to the idea of the product? What was their visual response to it and its packaging?” For him, this sort of spontaneous behavior is more valuable than people’s answers to scripted questions — it’s a deeper look at the sorts of opinions they might not even think to share on, say, a structured survey.
This fits with what Tuck describes as the main purpose of the focus group: to get people talking. “If you’re doing a good focus group, it’s not a question-and-answer session,” he says. “You don’t really care about the answers. What you care about is observing natural conversation.” Listen to enough ordinary conversation, the theory goes, and companies will hear ideas they wouldn’t have conceived of on their own.
But this is a hotly contested point. “In my 40 years working in design and innovation, alongside some of the most brilliant minds in the business, I have never seen innovation come out of a focus group,” wrote designer Gianfranco Zaccai in an anti-focus group polemic a few years ago. “Let me put it more strongly: Focus groups kill innovation.”
When I spoke to Zaccai, he qualified his views, allowing that listening to a product’s users can be extraordinarily helpful in giving a product designer insight into how their designs are actually used. However, he said he’s seldom gotten truly novel ideas from users, and when he’s come up with a hit idea on his own, focus groups have tended to reject it even when it’s gone on to huge success in the actual market. When he debuted the legendary Reebok Pump to a high school basketball team in the 1980s, for example, the players laughed at the idea, turned off by the shoes’ weird appearance; the sneakers are now acknowledged to be among the most influential in Reebok’s history.
This sort of critique — that innovation is the province of the professional, rather than the laypeople who make up focus groups — has been around almost as long as focus groups themselves. It came to even greater prominence in the 2000s, propelled by people like Malcolm Gladwell (who railed against focus groups in a 2005 speech) and Steve Jobs. “It’s really hard to design products by focus groups,” Jobs famously said. “A lot of times, people don’t know what they want until you show it to them.”
At least in my focus group experience, there’s some truth to this. I’ve been part of several groups that tear apart a proposed product, often for good reason. But afterward, when the facilitator asks for alternative ideas, we’re all silent.
One key benefit of focus groups, I was told several times while reporting this article, is that they expose designers and marketers to the relevant context of real people’s lives. But how good are we really at talking about ourselves honestly in front of groups of strangers? I’ve tried to make a sincere effort during my focus groups, but when asked exactly how often and why I buy things like frozen dinners or junk food, there’s always a strong temptation to lie. I don’t think that’s just me.
For many people, it can also be tough to air a dissenting opinion during a focus group. A particularly strong voice often sways everyone’s opinions, and psychologists consistently find that most people, despite what they might claim, prefer to blend in with the group rather than stand out on an island.
You don’t need to do many focus groups to see groupthink in action. During one focus group I did, the moderator passed out design mockups for a new product and asked each of us to rank them from most to least appealing. We each laid them out in our preferred rankings, but then something interesting happened. To my left, there was a hip-looking guy who happened to work in an industry adjacent to the product. He also considered himself something of an authority on matters of aesthetics and went on for some time about the designs. Then several people around the table reordered their choices and justified why they’d done so.
The moderator tried to emphasize that differences of opinion were expected, and even preferable, but it was striking to see more than one participant change their answer to mirror those of a perfect stranger — and then seemingly tell themselves it was what they’d thought all along.
Some view this as a feature of focus groups, not a bug. Tuck, the market research professional, told me it’s part of what his field refers to as “group effects”: the ways people’s behavior changes when they know they’re being observed and judged by everyone else. “Ultimately, people behave in groups. Consumers are groups of people. Voters are groups of people,” he said. “The lies people tell themselves about themselves, or tell their community — that’s the exact kind of stuff we want to hear.”
The idea is that we’re fake in focus groups the same way we’re fake in real life. Our social behavior is who we really are — at least insofar as it matters to retailers. In a world of abundance, what we ultimately choose to buy often hinges on how a product makes us feel, how it makes us view ourselves, and how we imagine it affects how others see us. This is why focus groups spend so much time digging into the emotional. They want to capture our most authentically fake selves: the selves we declare ourselves to be with the clothes we wear and the foods we eat and the vacation photos we post to Instagram.
Still, focus groups are rife with another type of deception, one that starts before participants even walk in the door. Ever since a 2004 New York magazine article that advised readers exactly how to lie to qualify for as many groups as possible, industry professionals say they’ve been fighting the scourge of what they call “professional” respondents, who’ve made focus groups into a lucrative side gig. By his own accounting, my friend Claude has done 95 focus groups for a total of $10,740 over the past decade. The trend has only become more common in the internet era, with focus group tips from sites like Stay a Stay at Home Mom; more than once, Claude’s told me, he’s struck up conversation with a co-participant in the elevator after a focus group and found that they, too, are a regular.
There’s a whole lot of gray area to work within when filling out the surveys. You can, for example, rate your personality as “extremely outgoing” rather than the more accurate “somewhat outgoing,” and you can indicate that you plan to buy a car sometime in the next year, when in reality it might be more like two or three. Fill out enough surveys and you learn that these are the sorts of little lies that get you into groups.
One might blame respondents for exploiting the system, but I frequently got the sense that the panel companies are in on it too. Claude told me about a screening call he got a few months ago from a recruiter at Schlesinger Group, a market research firm. “They asked if I had been to a financial seminar before and I said yes,” he told me. “She said, ‘Did you mean to say no?’” He changed his answer, was invited to join a group, and made $200.
When I reached out to representatives of the company for comment, they seemed concerned, and described the protocols they have in place to these sorts of events. “For this specific part of the process, we perform an initial screener, re-screener, and, for an in-person project, a facility onsite screener,” said Terri-Lyn Hawley, a VP at Schlesinger. “These are all managed by different teams, as well as randomly monitoring phone calls, to mitigate the risk of something like this happening.”
But I saw this pattern with other panel companies I interacted with, and it’s one Liza Featherstone experienced when participating in focus groups as part of the research for her book Divining Desire. The reason is simple, she explained to me: The longer it takes to fill a group, the higher the cost. As a result, panel companies pressure their recruiters to find participants as quickly as possible — and force them to blur the line between finding members of the right demographic and coaching people into saying the right answers. There are other signs that panel companies are well aware of regulars. Another friend of mine, who started doing focus groups over the past year, was recently given a W-9 form from Schlesinger because she earned more than $600 in annual income from them.
For Featherstone, though, the extent to which all this really matters is debatable. “Focus groups are not a scientific and quantitative method of gathering knowledge anyway,” she says. “So the fact that some people are participating frequently really doesn’t matter.” A focus group, in other words, is a case study, not a scientific survey. It’s a limited interview that nevertheless may illustrate something about a bigger whole. It’s not an ideal way to gather accurate data on, say, what percentage of people prefer a new flavor of latte to the original, but it might be a good way to unearth useful truths about how people decide to try a new flavor of latte, and what role that flavor might play in their lives.
In exchange for these details about our private lives over and over, we focus group regulars get something quite valuable too, apart from the money. During the very last group I did for this project, carried out by a cable company, I had a realization. For more than an hour, I was granted the privilege to complain about one of our most reviled consumer services. Normally, we have to navigate complex automated phone systems just to talk to anyone at a cable company; here, professionals spent time politely listening to us rant about deceptive fees and frustrating customer service so they could write reports to executives about it.
As I sat there airing my trivial grievances to a captive audience, I discovered something that surprised me but probably shouldn’t have. Perhaps the most authentic fact about consumers that focus groups elicit is this: It feels good to talk. It feels good to be heard.