Last week I attended the ACEEE National Symposium on Market Transformation in Baltimore. Learning and information gathering from conference sessions are typically down the list of reasons I attend conferences. This conference however turned out to be very beneficial on both of those counts. In particular, the net-to-gross (NTG) football, as described in last August’s Energy Program Evaluation Asylum post, was uncased for another game. This time I learned something[1].
One session featured heavy doses of program attribution, and of course, the NTG football. Speakers included Bob Wirtshafter from Wirtshafter Associates and Mike Messenger from Itron. Both gentlemen demonstrated the folly and delusion of pinpointing net savings, a la, NTG ratios.
Bob started with introducing LeBron James (the persona, not the person) to the audience. I haven’t paid attention to the NBA since Michael Jordan left da Bullsss. That’s when the headbands and other worthless uniform accessories, and tattoos galore appeared. But this I know. LeBron went straight from high school to the NBA, Cleveland Cavaliers. He later went to the Miami Heat. Cleveland hated him. He won at least one most valuable player award and at least one championship with the Heat.
So, Bob asks, how much of the Heat’s championship should be attributed to LeBron? Did you know that LeBron took a pay cut to bring on other expensive players to fit under the salary cap – just so they could win a championship? How much of this credit goes to LeBron, and how much of the credit do the other expensive guys get – who, if it weren’t for LeBron, wouldn’t even be there?
The point being, this is a ridiculous exercise that could be argued for millennia, and so it is with determining precise NTG values when evaluating energy efficiency programs. In all, the LeBron James addition to the Heat, with his “sacrifice”, probably increased the Heat’s odds of winning a championship by ten-fold, all else equal. Or maybe five fold, or three fold. Anyone suggesting he lessened their odds would be a biased LeBron hater.
Mike Messenger explained that the concept of net savings was developed to help ensure that energy efficiency programs are not simply a wealth-redistribution scheme. If I understood correctly, it evolved in California (surprise!). My thoughts were, really? Where is the NTG study for the food stamp program and unemployment programs? Would you have purchased that box of Pop Tarts if you did not have the food stamps? If you did not get your unemployment benefits, would you keep your car? Maybe getting rid of the car would be a good thing for the individual, even if they didn’t lose their job. Who’s to say?
Consider Social Security, which everyone knows is headed for insolvency if no action is taken to jack taxes or cut benefits. This was originally a safety net program. Eligibility started at an age that was well beyond the expected life of Americans. As the expected life of Americans has risen to about 80 years, it has transformed into a wealth-transferring retirement system. The free ridership is probably close to 90%. Heck. Is the average 65 year old incapable of supporting themselves these days? Don’t think so.
On the other end of redistribution, Mr. DeCaprio, would you have bought that $120,000 Tesla S if it weren’t for the $10,000 tax credit? I would say so, yes. Does anyone care? Not much. Really. Does anyone think $10,000 for millionaires to buy electric toys is not upside down welfare?
Why does the scrutiny fall only on energy efficiency programs? I would venture to say the government doesn’t trust the private sector, so government regulates. Unfortunately, I believe the vast majority of the voting populous doesn’t realize or care that no one regulates or evaluates government spending. No one. By comparison, free ridership is off the chart, without a doubt.
Should we just can the NTG, free ridership, drivership, spillover, and market effects? I think we all agree the answer is no. At least in the private sector, we are loath to take money from some people and enterprises and give it to others for no reason. We should use the broad brush effects to improve programs, not delude ourselves into thinking the NTG ratio is 83.1469%. As Mr. Messenger put it, it’s 70%, plus or minus twenty percentage points.
Many things aside from the program and incentive make projects happen. For states new to EE, like Louisiana, all other EE-pioneering states, like California, contributed to those projects. The previous two decades of programs contribute to projects in those applicable states. This would include awareness, accumulated workforce expertise, and so on.
As Bob put it, instead of the NTG delusion, let’s focus on what customers need and don’t need and provide the former, and skip the latter. I would say our industry does a poor job at this, particularly for larger customers. They don’t need cash. They may need expertise; time (manpower); someone to manage projects; someone to solicit bids, and things like that. They will do projects, but the program only provides screwdrivers when they need a socket set. Hello!
_________________________
Definitions per the California Public Utilities Commission [paraphrased]:
- Gross savings – energy and/or demand savings achieved by program participants regardless of why they participated in the program. Note: I would add that, in general, this is estimated savings on the front end, whether it is a custom or prescriptive measure. There are also verified gross savings, which are the savings verified by an independent third party evaluator.
- Net savings – all savings attributable to the program. This includes a portion of the gross savings and other phenomena such as spillover, free riders, and free drivers, codes and standards, and other externalities, such as federal tax incentives.
- Free rider – program participant that would have done the energy-saving project regardless of the program.
- Free driver- a non-participant that implemented a project as a result of the program.
- Spillover – a larger bucket of (1) participants that do more, but outside the program, and (2) measures that get implemented due to market transformation and contractor equipment-stocking habits (only stock efficient stuff).
[1] I don’t want to bore avid readers of the Rant with the definitions of impact evaluation terms discussed in this post, right here. However, for a refresher, or for welcome newcomers, I provide them at the end of this post.
I agree with putting more emphasis on what customers need and what will truly help them achieve energy efficiency. For that reason I dislike putting too many scarce resources into questionable tactics for estimating free ridership.
I was surprised to hear, though, that the main reason for estimating free ridership was to minimize wealth-redistribution. I don’t find that a compelling argument. If a wide array of programs are available, anyone who wants to get some of the wealth can usually participate in a program and get a share.
In the first decade or so of my career (thirty years ago), it was my job to forecast future electric growth to make sure we would have the electric generation we needed when we needed it. Estimating free ridership from programs was important for the forecasting effort. We knew that a certain level of energy efficiency improvements would be naturally occurring. That part was already in our forecast as a reduction to electric growth. Then we would reduce the forecast even more for the extra energy efficiency that would occur because of program efforts. Without knowing free ridership, we could be underestimating our future need for electricity, and that would lead to inadequate supply. This always seemed a compelling argument to me about why understanding free ridership in programs was important.
I am 100% agreed, though, that we will never be able to get real accurate on this front. However, it should be acknowledged as important. I would prefer some simple method, like being able to assume that free ridership equals spillover, with some sort of reasonableness check for programs that may have outlived their usefulness in the market.
Thanks for the comments Mary. Very insightful and useful and they further to help me get my arms around the concept, need, and purpose. Regarding the wealth redistribution, that was my understanding from Mike, going back to the early days of California. I understood it, they wanted to make sure the masses weren’t paying a 1% tax (or whatever it was) and giving it to just a few participants. As with attribution itself, the redistribution issue was likely only one concern.