The big flaw in Netflix’s research into when it has you hooked

Netflix knows when it has you hooked! Or so it says.

The streaming service released an infographic Wednesday, September 23, that shows which episode served as a turning point in many of its popular TV shows. If you completed this episode, Netflix says, you were most likely to finish the entire first season. (You can read a full list of shows Netflix surveyed at the Hollywood Reporter.)

Netflix

Here's that same information as a line graph, for some reason:

Netflix

If you can read that fine print, though — and you probably can't — you'll notice that Netflix says the "hooked" episode has no relation to how big its audience is. Instead, it's a wholly percentage-based phenomenon. If 70 percent continue on with the show to the end of season one, then that episode becomes the "hooked" episode.

Thus, there's something sort of sneaky going on here. Netflix has stacked some of the most acclaimed shows of our era — including gold standard shows like Mad Men and Breaking Bad, as well as massive hits like The Walking Dead (listed in the full survey linked to above) — right alongside Netflix shows that likely have nowhere near the size audience or acclaim as those shows (outside of maybe the very well-regarded Orange Is the New Black).

I asked Jason Mittell, a professor of film and media culture and American studies at Middlebury College in Vermont, about this, and he pointed out that this is right alongside Netflix's choice not to release viewership numbers.

"By not actually giving viewer numbers, these charts make it seem like all these shows have similar viewership, which I'm sure they do not," Mittell said.

But that conflation — regardless of audience size — is just the least of the issues here. Netflix is trying to take a typical, predictable TV phenomenon and make it a part of the Netflix experience.

Everybody samples shows and eventually drops them

Netflix isn't really proving anything with this infographic, unless it aims to confirm that human beings continue to follow the predictable patterns they always have when it comes to viewership numbers.

Every time a new show launches, it falls into a predictable pattern well over 99 percent of the time. The pilot has the highest viewership, and then there's a steady tapering off over the next several episodes, at which point it reaches a plateau where it's relatively stable. If that plateau is a high enough number, the show runs for years to come.

There are, of course, anomalies. Empire debuted big in January 2015, and then got even bigger with each new episode. And sometimes shows don't catch on until late in their first or even second seasons, thanks to time slot shifts or viewers simply discovering them. For a good example from before the era of streaming, think of Cheers, which was one of the lowest-rated shows on TV in its first season, then started slowly growing in season two, before exploding in season three when it aired as part of a block featuring The Cosby Show.

But for the most part, shows debut, then find their level within four episodes. And notice how many of the shows on Netflix's infographic reach their "hooked" number by episode four? Eight out of 12.

Even the shows that take longer to hit their hooked number reflect TV viewing patterns. Take, for instance, How I Met Your Mother, which Netflix says takes most viewers eight episodes to get addicted to. Comedies have always taken longer to catch on than dramas, and, indeed, if you run Netflix's math (at least as I understand it) on the first-season numbers for HIMYM, the "hooked" episode was episode 10, not so very long after episode eight. And that core audience stuck around for the show's nine-season run.

On streaming, audiences are self-selecting

Mittell also points me to another thought here: When you watch a show on Netflix, you aren't really sampling it blind. Sure, that's true on TV, too, but on Netflix, you've likely had huge numbers of friends telling you about it, or the company's algorithm has singled it out as a show you might be interested in.

Thus, the audience for these shows is self-selecting. Think, again, about Cheers, which was discovered by a mass audience because of when it aired. Such a thing isn't really possible on Netflix because the relationship of the viewer to the program is so different.

Look, for instance, at Bates Motel, which reportedly hooks viewers in episode two, or Better Call Saul, which grabs them by episode four. Aren't both shows boosted, in some ways, by being based on other properties (Psycho and Breaking Bad, respectively) that deliver built-in fan bases? How much of being hooked is from the genuine quality of the program, and how much is simply being predisposed to like it?

I don't want to take away from Netflix's work here, which is certainly a fun discussion point. It's interesting to compare which episode of a show turned you from a casual viewer into a hardcore fan, and it's natural for Netflix to attempt to mathematically quantify this question.

But, as always with Netflix, there's an attempt to redefine TV's normal as being filtered through the Netflix prism of full-season drops and endless marathon viewing sessions. And while some of that really has changed how we watch TV, much of it hasn't.

Explanatory journalism is a public good

At Vox, we believe that everyone deserves access to information that helps them understand and shape the world they live in. That's why we keep our work free. Support our mission and help keep Vox free for all by making a financial contribution to Vox today.

One-Time Monthly Annual

$95/year

$120/year

$250/year

$350/year

Other

$

Yes, I'll give $120/year

Yes, I'll give $120/year

We accept credit card, Apple Pay, and Google Pay. You can also contribute via