impact evaluation Archives - Michaels Energy

Mar 02

Impact Evaluation, Bull Riding, Cookies

By Jeff Ihnen Energy Rant No Comments

I've been doing a lot of program impact evaluation for the last few months. That comes with pluses and minuses. One of the minuses is that I needed to suspend writing this blog for a while. Pluses include problem-solving, working with staff throughout our company, and forensics engineering. Impact Evaluation But first, what is impact evaluation? I have a broad audience, and I get great feedback, some of which is, "I don't know what you're talking about sometimes." Entities delivering efficiency programs (implementers) are responsible for delivering savings or impacts. For custom efficiency portfolios, which entail a wide variety of...

Feb 03

Love0

Inch or Mile? Indispensable Thumbs Guide Impact Evaluation

By Jeff Ihnen Energy Rant No Comments

Hi. How’re you? Jeff Ihnen here. I’m back and almost live. I’ve been away on several temporary assignments that are starting to wind down. We are thrusting forth into the 11th year of the Energy Rant! Indispensable Thumbs Unless you have broken one, lost one, or had one immobilized, you probably have no idea of the value of your thumb and its equivalent, the big toe. I’m sitting on an airplane as I bang this out, munching a bag of almonds. Thumb and a finger. Thumb and a finger? That’s how you and I and anyone with a fully functioning...

Jan 02

Love0

Lessons from a Smart Thermostat Assessment

By Jeff Ihnen Energy Rant No Comments

Many of us at Michaels are gluttons for data and accompanying analysis. Our homes are test beds. In this post, we look at impacts from smart thermostats, the dangers of wrong assumptions, and extrapolating findings for impact evaluation. Every smart thermostat evaluation I’ve seen uses billing regression models to estimate savings. Unfortunately, this provides an answer and typically not one smart thermostat makers like. The evaluations typically provide no “why” to the results, and therefore, smart thermostat programs go unimproved. Test Bed Description My single family home was built in 1934, has crumby, double-pane, double-hung windows, and unknown amounts of...

Oct 23

Love0

Energy Program Evaluation Tales from the Dark Side

By Jeff Ihnen Energy Rant No Comments

The Energy Rant is not for the weak-kneed who prefer to live in the land of unicorns, fairy dust, and lollipops. This week is no exception. This Rant is brought to you by the recent ACEEE paper, Recent Developments in Energy Efficiency Evaluation, Measurement and Verification. Burned on the Alter of 90/10 A major objective of most evaluation contracts is to punch the 90/10 ticket. To recap, 90/10 simply means there is a 90% chance that the results of the population, if evaluated in full, will be within plus or minus 10% of the results for the sampled, studied projects....

Jul 31

Love0

Program Evaluation – Most Excellent Avocado Practices

By Jeff Ihnen Energy Rant No Comments

Everyone has applied for health insurance[1], and many of you have applied for life insurance. Anyone over, oh, 40, 50, or for sure 60, knows health flaws start to accumulate like the dumpster’s worth of unwelcome gifts, used shoes, and outdated clothes for those of you who stay current with the style trends of the day. Come to think of it, all you digital natives, back in the day before weddingregistry.com (where you shop and make other people fill your house with stuff you want), we just took whatever the aunts, uncles, or more especially, great aunts and uncles, decided...

Jul 24

Love0

Drive-By Evaluation – Buffalo Bill at Large

By Jeff Ihnen Energy Rant 3 Comments

Thank you to Mike Frischmann (our Director of Evaluation Engineering) for contributing to this week’s Rant. Eight hundred pound gorilla alert! Energy efficiency program evaluation “best practices” need a big overhaul. I am not talking about best practices for doing impact evaluation like the Uniform Methods Project. It’s disgusting that so much money is spent on standards like that and others, while ulterior motives drive program evaluation in entirely different directions. Purpose of Program Evaluation The State and Local Energy Efficiency Action Network, aka SEE Action’s Energy Efficiency Program Impact Evaluation Guide, states the following are objectives of program evaluation....

Oct 03

Love0

Evaluation, Measurement and Verification 2.0; A Whale Bus or an Airbus?

By Jeff Ihnen Energy Rant 2 Comments

Evaluation, Measurement and Verification 2.0, or EM&V 2.0, is a nerdy term coined in 2014, according to this blog by Northeast Energy Efficiency Partnerships (NEEP). The hype of EM&V 2.0, which I will explain later, is that it will automate measurement and verification, putting us engineers out of business. This is not going to happen anytime soon. Definition 2.0 The definition of EM&V 2.0 boils down to using utility meter interval data[1], typically hourly or sometimes every 15 minutes, or maybe even 5 minutes, to disaggregate and measure impacts from energy efficiency measures. One could consider that EM&V 2.0 is...

May 16

Love0

Real Time Evaluation for Real – Excuses Stay Home

By Jeff Ihnen Energy Rant One Comment

People and firms talk about integrating evaluation findings into programs and portfolios closer to real time than the conventional methods. But what is really being done about it other than talk? This post features some concrete ideas to put in the bank – money. First, what is the conventional evaluation process? Approaches and plans vary all over the place because different clients see different general approaches. For example, some clients evaluate every program every year. Some evaluate every program only once per EE plan cycle and within this subset, some evaluate them all in one shot over the course of…

Aug 12

Love0

Impact Evaluation Confidence and Precision – Fluffy Illusions or Autopsies

By Jeff Ihnen Energy Efficiency, Energy Rant No Comments

This week’s rant is brought to you by Ryan Kroll, Michaels’ Program Evaluation Manager. Last week Ryan issued a Program Brief discussing 90/10 confidence and precision sampling that is the industry norm for energy efficiency program impact evaluation. The 90/10 simply means the results of the sampled projects have a 90% probability of being within plus or minus 10% of properly representing the entire population – and NOT necessarily the right answer. Sample representation and the right answer are obviously different things. Here is the perversion in impact evaluation that I’ve never seen written anywhere: The less you learn about...

May 20

Love0

Impact Evaluation – Or Your Money Back

By Jeff Ihnen Energy Efficiency, Energy Rant No Comments

Without fail, it seems that every custom efficiency or self-directed impact evaluation we do has a controversial, giant project accounting for 25% (or more) of the program’s savings. It turns out it shouldn’t even have been allowed into the program because it doesn’t qualify – and in many cases, aside from that, the savings calculation is demonstrably and by the laws of thermodynamics and utility meter readings, wrong. An analogy to the message today might be a home energy assessment. Our house was built to our liking thirteen years ago. I laid out the floor plan in about fifteen minutes...

Impact Evaluation, Bull Riding, Cookies

Inch or Mile? Indispensable Thumbs Guide Impact Evaluation

Lessons from a Smart Thermostat Assessment

Energy Program Evaluation Tales from the Dark Side

Program Evaluation – Most Excellent Avocado Practices

Drive-By Evaluation – Buffalo Bill at Large

Evaluation, Measurement and Verification 2.0; A Whale Bus or an Airbus?

Real Time Evaluation for Real – Excuses Stay Home

Impact Evaluation Confidence and Precision – Fluffy Illusions or Autopsies

Impact Evaluation – Or Your Money Back

Search Our Archive

All things energy. That’s our jam.

All things energy. That’s our jam.

Headquarters

About

Solutions

Resources

Get In Touch