A new study looks at federal energy-efficiency efforts — and the results are grim

(Also read our follow-up to this post, with some responses to the study)

In climate-policy circles, energy efficiency has long been considered the easiest, cheapest way to reduce emissions. There are lots of opportunities to upgrade our insulation, our furnaces, and our appliances so that we're squandering less energy. Not only would that cut down on pollution, but it should save people money. Everyone wins.

But that doesn't mean every single efficiency policy out there will pan out. Some can be surprisingly ineffective.

The study found that the costs of federal weatherization programs were about twice the actual benefits

That's the issue explored in a new working paper by Meredith Fowlie, Michael Greenstone, and Catherine Wolfram. The economists conducted a large randomized controlled trial of 30,000 homes in Michigan involving the federal Weatherization Assistance Program, which helps low-income families replace their furnaces, upgrade insulation, and seal up leaks along doors and windows. This experimental set-up allowed for a more rigorous evaluation of these weatherization efforts.

The researchers found that the upfront cost of efficiency upgrades in the Michigan program came to about $5,000 per house, on average. But their central estimate of the energy savings only amounted to about $2,400 per household, on average, over the lifetime of the upgrades.**

The federal program did help households save energy: after the upgrades, homes used 10 to 20 percent less energy for electricity and heating. But, importantly, that was only about 39 percent of the savings that modelers had predicted ahead of time. The program simply wasn't as effective at saving energy as everyone thought.

One possibility is that households compensated for their reduced utility bills by increasing their energy consumption after the upgrades. But the economists didn't find evidence of a "rebound effect" here — they went knocking door to door and found little sign that people were, say, cranking up their thermostats in the winter.

"We were very surprised by the result," says Greenstone, an economist and director of the Energy Policy Institute at the University of Chicago. He notes that it's still not entirely clear why Michigan's weatherization program didn't save nearly as much energy as modelers had predicted — a fact he calls "unsettling."

Now, to be clear, this study only examined federal weatherization efforts in a single state, and these results don't necessarily apply to all types of residential efficiency programs. Even federal weatherization programs can vary from region to region — what happens in Michigan may not apply to New Jersey.

What's more, experts note that low-income weatherization programs aren't necessarily designed to be as cost-effective as possible — they also have social goals like clearing out mold or helping poor people live comfortably through the winter. (This Michigan study didn't assess those ancillary benefits.) Indeed, past research from Lawrence Berkeley National Laboratory found that low-income weatherization programs were twice as costly, per unit of electricity saved, as the average utility efficiency program. That suggests the much larger array of utility-run initiatives throughout the country are more likely to be cost-effective.

But the results do suggest the need for closer study and field-testing of various policies to boost efficiency.

Saving energy is great, but how much is really possible?

(John B. Carnett/Popular Science/Getty Images)

Many estimates of the value of energy efficiency come from engineering studies that look at what's possible under ideal conditions. These studies typically suggest that we're wasting a lot of energy in our homes, office buildings, and cars — waste that could be eliminated at negative cost. See this big McKinsey report for a great example.

There are lots of potential places we could cut down on energy waste — but efforts to do so need to be field-tested

But, Greenstone says, these engineering studies may not always capture the messiness of the real world. It's easy to find ways to cut down on waste in laboratory conditions. But outside the lab, homes might be irregularly shaped, insulation might not always be installed by highly skilled workers, and there are all sorts of human behaviors that might reduce the effectiveness of efficiency investments.

That's why field tests are a valuable check — and randomized controlled trials are considered the gold standard here. This particular RCT, the first of its kind, found that the federal Weatherization Assistance Program only seemed to be saving about 39 percent as much energy in Michigan homes as engineering tests had predicted:

(EPIC)

Greenstone cautioned that this study hardly undermines the rationale for every single efficiency policy out there. It's entirely possible there are genuine untapped opportunities to reduce energy use and save money elsewhere — in industrial sectors, in transportation, even in other residential programs. But, he says, "this needs to be verified in the field."

It's an important question for climate policy more generally. Energy efficiency is often considered the great low-hanging fruit — the cheapest and easiest policy to reduce CO2 emissions. Peek under the hood of any grand plan for addressing climate change, and you'll usually find that energy efficiency is playing a central role.

And yet, in this particular study, the economists found that the federal home weatherization program was not a particularly cheap way to reduce CO2 emissions. Although energy use (and hence carbon pollution) from the homes studied did go down, it came at a cost of about $329 per ton of carbon. That's much higher than the $38-per-ton value of the social cost of carbon that the US federal government uses to evaluate the costs and benefits of climate policies.

"This underscores the value of field-testing," says Greenstone. "Particularly in a world where economy-wide carbon pricing does not look feasible, we should be redoubling our efforts to find those CO2 reduction measures that have the biggest bang for the buck."

----

** Note: For those interested, the central estimate of the lifetime benefits for the weatherization program in Michigan was $2,400, assuming a 6 percent discount rate over 16 years. The paper adds: "estimates of the present value of the savings range from approximately $1,450 [10% discount rate over 10 years] to about $3,500 [3% discount rate over 20 years]. These estimates are just 32% to 77% of the upfront cost of the energy efficiency measures."

Further reading:

-- Read our follow-up: Energy efficiency can be incredibly valuable — but we should measure it properly

-- You can read EPIC's summary of the new paper here. And here's a full copy of the paper itself (pdf).

-- You can also find thoughtful critiques of the paper (and the hype around it) from Merrian Borgenson and Rebecca Stanfield of the Natural Resources Defense Council.

-- On Twitter, energy analyst Chris Nelder takes issue with the study's assumptions about future electricity and natural gas prices in America (see his critique here, here, here, here, here, here). If you believe those prices are going to rise significantly in the future, then efficiency starts to look like a better bet.

-- Back in 2013, I took a look at the launch of E2e, a joint project between economists at MIT and the University of California Berkeley that aimed to take a more rigorous scientific approach to the concept of energy efficiency. This latest study comes out of that project.

Explanatory journalism is a public good

At Vox, we believe that everyone deserves access to information that helps them understand and shape the world they live in. That's why we keep our work free. Support our mission and help keep Vox free for all by making a financial contribution to Vox today.

One-Time Monthly Annual

$95/year

$120/year

$250/year

$350/year

Other

$

Yes, I'll give $120/year

Yes, I'll give $120/year

We accept credit card, Apple Pay, and Google Pay. You can also contribute via