Federal report: violent crime rose in 2016. Other federal report: eh, maybe not.

Did violent crime go up in the US in 2016? Here’s a fun answer: Nearly a full year later, we still don’t really know.

On one hand, you have data from the FBI, released earlier this year. It found that the violent crime rate, including murder, rape, and robbery, did indeed go up by more than 3 percent from 2015 to 2016. This was the second year in a row of an increase above 3 percent, according to the FBI data.

On the other hand, you have data from the Bureau of Justice Statistics, released on Thursday. The survey that the data comes from went through routine methodological changes this year to include a different batch of counties than previous years — to keep up with population shifts in the US. But researchers managed to compare the counties that remained in the survey from 2015 to 2016 — finding no significant increase in the violent victimization rate. That’s shown by the dashed red line in the following chart (while the dotted black line shows the data that includes new counties, which is not comparable to previous years since it’s measuring different places).

Bureau of Justice Statistics

Violent crime has, in general, been on the decline for decades. But President Donald Trump has taken to falsely claiming that the murder rate is at a 45-year high, and his attorney general, Jeff Sessions, has similarly warned of rising violent crime to push for punitive “tough on crime” policies. Yet it’s not clear if there even was a violent crime increase in 2016, much less if the decades-long decline in violent crime is now truly on the reverse.

So what is going on here? Which data can we trust? Well, it’s complicated.

The FBI and Bureau of Justice Statistics have different methodologies

The first thing to know: These reports are measuring two different things. The FBI is reporting crime detected by and reported to local and state police, which is then logged and reported to the FBI. The Bureau of Justice Statistics analysis is reporting the results of an annual survey of tens of thousands of Americans 12 and older, known as the National Crime Victimization Survey (NCVS), in which people report what, if any, crimes they’ve been a victim of.

One notable difference caused by the varying methodologies: The FBI report includes murder, but the NCVS does not, since dead people can’t report victimization.

Obviously, this means the NCVS is missing some crime in America — and it’s a particularly important part of the 2016 story, since the FBI report suggested that the murder rate rose by nearly three times as much as the overall violent crime rate.

But murder remains a small portion of all violent crimes in America, making up about 1 percent of the violent crime rate in 2016.

And the FBI report always misses a lot of crimes too, since it only picks up crimes that are detected by and reported to the police. There are many crimes the cops never hear about. In fact, the NCVS found that fewer than half of violent crimes — just 42 percent — were reported to police in 2016.

Still, the murder data is some of the most reliable that federal government collects, since it’s pretty difficult for the police to miss a dead body. And the FBI has found two years of notable increases in the murder rate — 11 percent in 2015 and 8 percent in 2016. That indicates something is going on.

But which one of these reports is right about overall violent crime? Frankly, we just don’t know. As criminal justice expert John Pfaff explained when there was a similar rift between the FBI and NCVS data for 2015, the NCVS is widely considered more reliable. But both reports have enough gaps to prevent any hard conclusions in either direction. (Pfaff also argued after the release of 2016 data that it’s better to look at what the different sources tell us — by giving insight into slightly different populations — than try to figure out which source of data is right.)

So we don’t really know if violent crime went up in 2016 or 2015. That leaves us with a very unsatisfying conclusion: To see if the decades-long crime drop has started to reverse, we’ll probably have to wait for a few more years of data from both the FBI and NCVS to see the long-term crime trends instead of focusing on just one or two years of data.

Our crime data is just awful

As you might have guessed from this discussion, the US does a pretty pathetic job of collecting crime data. Not only are we just now getting data from 2016, but we don’t even know if that data is reliable.

This has policy implications: If the violent crime rate is going up, it would be helpful for lawmakers and police to know about an increase much sooner — preferably as it’s happening — and be able to judge whether the trends are local, statewide, or national to find the right solutions.

Yet it took until late 2017 to conclude that we still don’t really know what was going on in 2015 or 2016.

We have a request

Vox's journalism is free because we believe that everyone deserves to understand the world that they live in. That kind of knowledge helps create better citizens, neighbors, friends, parents, consumers and stewards of this planet. In short, understanding benefits everyone. You can join in on this mission by making a financial gift to Vox today. Reader support helps keep our work free, for everyone. Will you join us?

One-Time Monthly Annual

$95/year

$120/year

$250/year

$350/year

Other

$

Yes, I'll give $120/year

Yes, I'll give $120/year

We accept credit card, Apple Pay, and Google Pay. You can also contribute via