Skip to main content
Tag

impact evaluation

Mar 02
Love0

Impact Evaluation, Bull Riding, Cookies

By Energy Rant No Comments
I've been doing a lot of program impact evaluation for the last few months. That comes with pluses and minuses. One of the minuses is that I needed to suspend writing this blog for a while. Pluses include problem-solving, working with staff throughout our company, and forensics engineering. Impact Evaluation But first, what is impact evaluation? I have a broad audience, and I get great feedback, some of which is, "I don't know what you're talking about sometimes." Entities delivering efficiency programs (implementers) are responsible for delivering savings or impacts. For custom efficiency portfolios, which entail a wide variety of…
Read More
Feb 03
Love0
impact

Inch or Mile? Indispensable Thumbs Guide Impact Evaluation

By Energy Rant No Comments
Hi. How’re you? Jeff Ihnen here. I’m back and almost live. I’ve been away on several temporary assignments that are starting to wind down. We are thrusting forth into the 11th year of the Energy Rant! Indispensable Thumbs Unless you have broken one, lost one, or had one immobilized, you probably have no idea of the value of your thumb and its equivalent, the big toe. I’m sitting on an airplane as I bang this out, munching a bag of almonds. Thumb and a finger. Thumb and a finger? That’s how you and I and anyone with a fully functioning…
Read More
Jan 02
Love0
energy history

Lessons from a Smart Thermostat Assessment

By Energy Rant No Comments
Many of us at Michaels are gluttons for data and accompanying analysis. Our homes are test beds. In this post, we look at impacts from smart thermostats, the dangers of wrong assumptions, and extrapolating findings for impact evaluation. Every smart thermostat evaluation I’ve seen uses billing regression models to estimate savings. Unfortunately, this provides an answer and typically not one smart thermostat makers like. The evaluations typically provide no “why” to the results, and therefore, smart thermostat programs go unimproved. Test Bed Description My single family home was built in 1934, has crumby, double-pane, double-hung windows, and unknown amounts of…
Read More
Jul 31
Love0

Program Evaluation – Most Excellent Avocado Practices

By Energy Rant No Comments
Everyone has applied for health insurance, and many of you have applied for life insurance. Anyone over, oh, 40, 50, or for sure 60, knows health flaws start to accumulate like the dumpster’s worth of unwelcome gifts, used shoes, and outdated clothes for those of you who stay current with the style trends of the day.Come to think of it, all you digital natives, back in the day before weddingregistry.com (where you shop and make other people fill your house with stuff you want), we just took whatever the aunts, uncles, or more especially, great aunts and uncles, decided to…
Read More
Jul 24
Love0

Drive-By Evaluation – Buffalo Bill at Large

By Energy Rant 3 Comments
Thank you to Mike Frischmann (our Director of Evaluation Engineering) for contributing to this week’s Rant. Eight hundred pound gorilla alert! Energy efficiency program evaluation “best practices” need a big overhaul. I am not talking about best practices for doing impact evaluation like the Uniform Methods Project. It’s disgusting that so much money is spent on standards like that and others, while ulterior motives drive program evaluation in entirely different directions. Purpose of Program Evaluation The State and Local Energy Efficiency Action Network, aka SEE Action’s Energy Efficiency Program Impact Evaluation Guide, states the following are objectives of program evaluation.…
Read More
Oct 03
Love0

Evaluation, Measurement and Verification 2.0; A Whale Bus or an Airbus?

By Energy Rant 2 Comments
Evaluation, Measurement and Verification 2.0, or EM&V 2.0, is a nerdy term coined in 2014, according to this blog by Northeast Energy Efficiency Partnerships (NEEP). The hype of EM&V 2.0, which I will explain later, is that it will automate measurement and verification, putting us engineers out of business. This is not going to happen anytime soon. Definition 2.0 The definition of EM&V 2.0 boils down to using utility meter interval data, typically hourly or sometimes every 15 minutes, or maybe even 5 minutes, to disaggregate and measure impacts from energy efficiency measures. One could consider that EM&V 2.0 is…
Read More
Aug 12
Love0

Impact Evaluation Confidence and Precision – Fluffy Illusions or Autopsies

By Energy Efficiency, Energy Rant No Comments
This week’s rant is brought to you by Ryan Kroll, Michaels’ Program Evaluation Manager.  Last week Ryan issued a Program Brief discussing 90/10 confidence and precision sampling that is the industry norm for energy efficiency program impact evaluation.  The 90/10 simply means the results of the sampled projects have a 90% probability of being within plus or minus 10% of properly representing the entire population – and NOT necessarily the right answer.  Sample representation and the right answer are obviously different things. Here is the perversion in impact evaluation that I’ve never seen written anywhere: The less you learn about…
Read More
Mar 19
Love0

Saints Are Marching In

By Energy Efficiency, Energy Rant, Utility Stuff No Comments
One of the most rewarding aspects of our business is ensuring people get it right and customers get the legitimate energy and monetary savings they should be receiving.  My answer to “why should we pick you?” is: we are passionate about what we do; we get things right; and we want to make a difference – improve things.  The problem occurs when others don’t give a rip about these things. Michaels’ primary thrust in the evaluation arena is impact evaluation, which is – what difference does the program make?  The questions are (1) what is happening versus what would have…
Read More