NBC just released Netflix’s top-secret ratings — or so they say

Netflix is notoriously tight-lipped with its viewership numbers, preferring vague proclamations of success to releasing statistics that could otherwise be torn apart.

But NBC claims to have cracked the code.

On Wednesday, during the Television Critics Association winter press tour, NBC hosted a lunch including a "Netflix Reality Check" for reporters.

Alan Wurtzel, NBC's president of research and media development, presented numbers the network gleaned from a tech firm called Symphony, which, according to Variety, "measures TV viewing using audio content recognition technology — software loaded on to [users'] phones that tracks viewership by capturing the soundtrack of the program." (The aforementioned software appears to work through a voluntarily downloaded app called Mi Mobile, which "passively collects granular individual behavioral cross media data in real time." Fun!)

Forget Big Brother; NBC is watching. Or listening, as the case may be.

So what are Netflix's numbers, allegedly?

Among Symphony's data conclusions: Jessica Jones averaged about 4.8 million views an episode, Master of None about 3.9 million, and Narcos about 3.2 million. Orange Is the New Black, meanwhile, supposedly averaged just 644,000 viewers for its third season, though the Symphony measurement occurred many months after that third season launched in June. (Netflix has previously said Orange is its most watched show.)

After feeling the thrill of obtaining Netflix's hidden numbers, NBC also searched for ratings for Amazon's Man in the High Castle, and concluded that it averaged around 2.1 million viewers. Notably, NBC didn't announce viewership data for Hulu — which NBC co-owns with ABC and Fox. (NBC's parent company, NBCUniversal, has invested in Vox Media.)

NBC said Symphony drew from a sample group of 15,000 people, which would put it just above Nielsen's current electronic measurement group of around 14,000. (Nielsen hopes to get up to 25,000 People Meters — or the physical boxes that measure viewing patterns — by this fall. Nielsen also measures TV ratings via diaries that viewers voluntarily fill in, but these are less reliable and more prone to human error.)

There's one important difference between Nielsen and Symphony. Nielsen's sample is a true representative sample, while Symphony's is self-selecting. You're only included if you download the app. Symphony likely tries to control for demographics using that information, but almost certainly lacks Nielsen's cross section of the country in miniature.

NBC wanted to assuage fears about broadcast television dying, but the "Reality Check" didn't make Netflix look so bad

With a title like "Netflix Reality Check," NBC clearly wanted to put Netflix's supposed dominance into perspective — or at least NBC's perspective.

Yes, 3 or 4 million viewers pales in comparison to, say, the viewership of The Voice, which regularly pulls about 12 million. But it's also just as many as, say, Parks and Recreation, whose series finale aired in 2015 with just over 4 million viewers — and that was the show's highest-rated episode in three years.

NBC is right on this: Netflix isn't replacing broadcast TV. But that's because it's not trying to do the same thing as broadcast TV. It never was. Sure, Netflix may want bigger numbers, but if a million people subscribe to Netflix solely to watch Jessica Jones (which they can get only on Netflix for a substantial period of time), that's still worth the investment.

Netflix's numbers are actually more in line with those of premium subscription channels like HBO, and sometimes they're even better. Compare, for example, Master of None's 3.9 million with Veep's routine 1.5 million.

For what it's worth: on Sunday January 17, Ted Sarandos, Netflix's Chief Content Officer, opened Netflix's day at the TCA press tour by slamming NBC's method and motives for trying to get their ratings. "The methodology and the measurement and the data itself doesn’t reflect any sense of reality of anything that we keep track of," Sarandos said. "“I hope no one’s paying for [the data, which was] really remarkably inaccurate."

Sarandos further upheld Netflix's view that "ratings have no specific impact on [its] business." He added that they don't even care about the 18-49 age demographic, the very viewership that NBC claimed to be measuring.

So, no, Netflix isn't replacing broadcast television — but it isn't trying to. As far as "reality checks" go, this one might not have said what NBC wanted it to.

Updated to reflect Netflix's response.

Will you support Vox’s explanatory journalism?

Most news outlets make their money through advertising or subscriptions. But when it comes to what we’re trying to do at Vox, there are a couple of big issues with relying on ads and subscriptions to keep the lights on:

First, advertising dollars go up and down with the economy. We often only know a few months out what our advertising revenue will be, which makes it hard to plan ahead.

Second, we’re not in the subscriptions business. Vox is here to help everyone understand the complex issues shaping the world — not just the people who can afford to pay for a subscription. We believe that’s an important part of building a more equal society. And we can’t do that if we have a paywall.

So even though advertising is still our biggest source of revenue, we also seek grants and reader support. (And no matter how our work is funded, we have strict guidelines on editorial independence.)

If you also believe that everyone deserves access to trusted high-quality information, will you make a gift to Vox today? Any amount helps.

One-Time Monthly Annual

$5/month

$10/month

$25/month

$50/month

Other

$

Yes, I'll give $5/month

Yes, I'll give $5/month

We accept credit card, Apple Pay, and Google Pay. You can also contribute via