program evaluation Archives

Jan 06

Soothsayer Says: Eight Predictions for the Energy Industry

By Jeff Ihnen Energy Rant No Comments

If it’s worth doing, it’s worth doing now. That quasi-cliché is why I have never had a New Year’s resolution, and I’m not going to start in 2021, but I can review the past and forecast (guess) the future. Soothsaying is part of my job, and I’m at least as accurate as next week’s weather forecast. At the start of 2020, we had just reorganized, defined who we are, what we do, and why we do it. Sounds simple, right? What is your personal purpose? What do you value? What makes you tick? Keeping it concise is very hard. Our...

Feb 25

Love0

Attribution on the Cheap

By Jeff Ihnen Energy Rant 2 Comments

In the last two Rant posts, we learned that our 40-year-old program evaluation frameworks need to change to capture greater, real impacts. Rather than improving programs and accurately determining impacts, archaic evaluation methodologies are impeding progress toward greater energy savings. It may be like solving the percolating national debt crisis, but I will attack this rubber tree plant anyway. Attribution Determination of program attribution is the common thread that weaves through most of the six common flaws (described here and here) of current program policy. Attribution is the quantity of benefits[1] delivered by the program or intervention. Attributable impacts are...

Feb 18

Love0

Stifling Impacts of Jurassic Evaluation Dogma

By Jeff Ihnen Energy Rant One Comment

If efficiency programs were telephones, the evaluation community would still be using wall-mounted analog dial-ups rather than the iPhone. Yes, I’m going to tell you why programs are designed to be evaluated and not to be effective, part 2, herein. The following is the list of flaws in demand-side management theory, as presented last week. Efficiency must cost more than inefficiency Building energy codes are sacrosanct Efficiency has to be the primary factor in customer decision making Customers must “get their money back” The unfamiliar get fifty cents on the dollar Immortality is fantasy Last week we covered the first...

Aug 27

Love1

Efficiency – With Tectonic Power and Pace

By Jeff Ihnen Energy Rant One Comment

I am no mountaineer, but without looking, I know the Himalayan range is growing taller. How do I remember this? Because the earth’s crust is made up of tectonic plates that are always moving. The edge of tectonic plates forms fault lines for earthquakes. Did you know, that at some point, coastal California will be neighbors with Alaska? It’s true. A hell of a lot of earthquakes will happen in between, giving “bumpy ride” new meaning. In Southern Asia, the plate that India sits on is slamming into Asian landmass, thrusting Everest higher, adding roughly 2.4 inches per year to...

Aug 14

Love0

Tales from the Energy Program Evaluation Asylum

By Jeff Ihnen Energy Rant No Comments

I’ve spent a lot of time reviewing papers written for the International Energy Program Evaluation Conference, or IEPEC, over the years. I dig the conference, which I like to call the asylum, as I did four years ago. People in the evaluation business can get pretty fired up. While I witnessed no verbal punches at this conference, one guy was practically shouting his comment and quaking during his peanut gallery monologue. Is there a question here, somewhere? Here’s the thing: lighten up, y’all. Don’t take things so damn seriously. On that note, there were very constructive conversations around climate change....

Jul 31

Love0

Program Evaluation – Most Excellent Avocado Practices

By Jeff Ihnen Energy Rant No Comments

Everyone has applied for health insurance[1], and many of you have applied for life insurance. Anyone over, oh, 40, 50, or for sure 60, knows health flaws start to accumulate like the dumpster’s worth of unwelcome gifts, used shoes, and outdated clothes for those of you who stay current with the style trends of the day. Come to think of it, all you digital natives, back in the day before weddingregistry.com (where you shop and make other people fill your house with stuff you want), we just took whatever the aunts, uncles, or more especially, great aunts and uncles, decided...

Jul 24

Love0

Drive-By Evaluation – Buffalo Bill at Large

By Jeff Ihnen Energy Rant 3 Comments

Thank you to Mike Frischmann (our Director of Evaluation Engineering) for contributing to this week’s Rant. Eight hundred pound gorilla alert! Energy efficiency program evaluation “best practices” need a big overhaul. I am not talking about best practices for doing impact evaluation like the Uniform Methods Project. It’s disgusting that so much money is spent on standards like that and others, while ulterior motives drive program evaluation in entirely different directions. Purpose of Program Evaluation The State and Local Energy Efficiency Action Network, aka SEE Action’s Energy Efficiency Program Impact Evaluation Guide, states the following are objectives of program evaluation....

Aug 17

Love0

The State of Program Evaluation and Tips for Picking Good Evaluation Practitioners

By Jeff Ihnen Energy Efficiency, Energy Rant One Comment

This post is brought to you by the International Energy Program Evaluation Conference (IEPEC), circa 19, er 2015. I moderated one session featuring four great papers and presentations concerning residential space heating and cooling. I also observed one concurrent session for nearly all the timeslots in the conference. The theme I found, which was very pleasing to me, is that doing useful research and evaluation is challenging and expensive. The reason it pleases me is that, well, getting things right is everything, but it also levels the playing field. I hate losing bids, but it is less painful to lose...

Apr 20

Love0

Duct Leakage; The Results Are In

By Jeff Ihnen Energy Efficiency, Energy Rant 5 Comments

About two years ago, I wrote Duct Leakage Chaff, which explained that residential duct leakage is a mole hill, a red herring, a boogeyman. I recently reviewed a report[1] that nails my assertions from June 2013. The program evaluation (residential HVAC tune-up) was thorough, likely expensive, but worthwhile. Too often, not just in evaluation, but any professional service industry, contracts go to the low-ball bidder. The low-ball bidder either uses the tactic because it’s the only weapon they have, they don’t understand the challenge of doing a decent job (ignorance), or providing actionable value simply isn’t that important to them....

Aug 12

Love0

Impact Evaluation Confidence and Precision – Fluffy Illusions or Autopsies

By Jeff Ihnen Energy Efficiency, Energy Rant No Comments

This week’s rant is brought to you by Ryan Kroll, Michaels’ Program Evaluation Manager. Last week Ryan issued a Program Brief discussing 90/10 confidence and precision sampling that is the industry norm for energy efficiency program impact evaluation. The 90/10 simply means the results of the sampled projects have a 90% probability of being within plus or minus 10% of properly representing the entire population – and NOT necessarily the right answer. Sample representation and the right answer are obviously different things. Here is the perversion in impact evaluation that I’ve never seen written anywhere: The less you learn about...

Soothsayer Says: Eight Predictions for the Energy Industry

Attribution on the Cheap

Stifling Impacts of Jurassic Evaluation Dogma

Efficiency – With Tectonic Power and Pace

Tales from the Energy Program Evaluation Asylum

Program Evaluation – Most Excellent Avocado Practices

Drive-By Evaluation – Buffalo Bill at Large

The State of Program Evaluation and Tips for Picking Good Evaluation Practitioners

Duct Leakage; The Results Are In

Impact Evaluation Confidence and Precision – Fluffy Illusions or Autopsies

Search Our Archive

All things energy. That’s our jam.

All things energy. That’s our jam.

Headquarters

About

Solutions

Resources

Get In Touch