NBC just released Netflix’s top-secret ratings — or so they say

Netflix is notoriously tight-lipped with its viewership numbers, preferring vague proclamations of success to releasing statistics that could otherwise be torn apart.

But NBC claims to have cracked the code.

On Wednesday, during the Television Critics Association winter press tour, NBC hosted a lunch including a "Netflix Reality Check" for reporters.

Alan Wurtzel, NBC's president of research and media development, presented numbers the network gleaned from a tech firm called Symphony, which, according to Variety, "measures TV viewing using audio content recognition technology — software loaded on to [users'] phones that tracks viewership by capturing the soundtrack of the program." (The aforementioned software appears to work through a voluntarily downloaded app called Mi Mobile, which "passively collects granular individual behavioral cross media data in real time." Fun!)

Forget Big Brother; NBC is watching. Or listening, as the case may be.

So what are Netflix's numbers, allegedly?

Among Symphony's data conclusions: Jessica Jones averaged about 4.8 million views an episode, Master of None about 3.9 million, and Narcos about 3.2 million. Orange Is the New Black, meanwhile, supposedly averaged just 644,000 viewers for its third season, though the Symphony measurement occurred many months after that third season launched in June. (Netflix has previously said Orange is its most watched show.)

After feeling the thrill of obtaining Netflix's hidden numbers, NBC also searched for ratings for Amazon's Man in the High Castle, and concluded that it averaged around 2.1 million viewers. Notably, NBC didn't announce viewership data for Hulu — which NBC co-owns with ABC and Fox. (NBC's parent company, NBCUniversal, has invested in Vox Media.)

NBC said Symphony drew from a sample group of 15,000 people, which would put it just above Nielsen's current electronic measurement group of around 14,000. (Nielsen hopes to get up to 25,000 People Meters — or the physical boxes that measure viewing patterns — by this fall. Nielsen also measures TV ratings via diaries that viewers voluntarily fill in, but these are less reliable and more prone to human error.)

There's one important difference between Nielsen and Symphony. Nielsen's sample is a true representative sample, while Symphony's is self-selecting. You're only included if you download the app. Symphony likely tries to control for demographics using that information, but almost certainly lacks Nielsen's cross section of the country in miniature.

NBC wanted to assuage fears about broadcast television dying, but the "Reality Check" didn't make Netflix look so bad

With a title like "Netflix Reality Check," NBC clearly wanted to put Netflix's supposed dominance into perspective — or at least NBC's perspective.

Yes, 3 or 4 million viewers pales in comparison to, say, the viewership of The Voice, which regularly pulls about 12 million. But it's also just as many as, say, Parks and Recreation, whose series finale aired in 2015 with just over 4 million viewers — and that was the show's highest-rated episode in three years.

NBC is right on this: Netflix isn't replacing broadcast TV. But that's because it's not trying to do the same thing as broadcast TV. It never was. Sure, Netflix may want bigger numbers, but if a million people subscribe to Netflix solely to watch Jessica Jones (which they can get only on Netflix for a substantial period of time), that's still worth the investment.

Netflix's numbers are actually more in line with those of premium subscription channels like HBO, and sometimes they're even better. Compare, for example, Master of None's 3.9 million with Veep's routine 1.5 million.

For what it's worth: on Sunday January 17, Ted Sarandos, Netflix's Chief Content Officer, opened Netflix's day at the TCA press tour by slamming NBC's method and motives for trying to get their ratings. "The methodology and the measurement and the data itself doesn’t reflect any sense of reality of anything that we keep track of," Sarandos said. "“I hope no one’s paying for [the data, which was] really remarkably inaccurate."

Sarandos further upheld Netflix's view that "ratings have no specific impact on [its] business." He added that they don't even care about the 18-49 age demographic, the very viewership that NBC claimed to be measuring.

So, no, Netflix isn't replacing broadcast television — but it isn't trying to. As far as "reality checks" go, this one might not have said what NBC wanted it to.

Updated to reflect Netflix's response.

Explanatory journalism is a public good

At Vox, we believe that everyone deserves access to information that helps them understand and shape the world they live in. That's why we keep our work free. Support our mission and help keep Vox free for all by making a financial contribution to Vox today.

One-Time Monthly Annual

$95/year

$120/year

$250/year

$350/year

Other

$

Yes, I'll give $120/year

Yes, I'll give $120/year

We accept credit card, Apple Pay, and Google Pay. You can also contribute via