Skip to main content
Tag

research

Sep 29
Love0

Converting Energy: Site v. Source Energy and Carbon

By The Big Why of Evaluation No Comments
As our industry’s goals shift from simply using less energy to reducing carbon emissions, the evaluation, measurement, and verification (EM&V) of these savings can become a whole lot more complex. Both sites and utilities need to understand how much carbon emissions they are actually saving, and this gets complicated as more types of fuels and sources are included, not to mention considerations of time and location. Let’s explore some of the issues. Background Simply put, energy efficiency is using less energy to perform the same function. Sometimes this is very simple: you can replace an incandescent light bulb with an…
Read More
Sep 01
Love0

Did the Program Actually Cause That?

By The Big Why of Evaluation No Comments
Welcome to another installment of the Challenges of Estimating NTG, where we explore the challenges of assigning a numeric counterfactual value to a complex decision-making process. This post will explore some ideas beyond the traditional survey self-report approach common to many programs and evaluations. We’ll explore other NTG methods and then structural causal models and causal diagrams. Non-Survey NTG Approaches In addition to survey-based approaches, evaluators have a number of other methods to estimate net savings. The Uniform Methods Project digs into these methods in detail and is worth the read. Two common methods for estimating net savings are randomized control trials…
Read More
Aug 04
Love0

Negative Savings?! That’s Not Supposed to Happen!

By The Big Why of Evaluation No Comments
Sometimes in the course of evaluating energy efficiency projects, we come across projects with negative savings. In other words, participating in an energy efficiency program resulted in an increase in overall energy usage. Given the program theory of most energy efficiency programs, it is safe to say that is not supposed to happen! So what went wrong? Why Is This Happening? First, let’s look into why savings might appear negative. There are many potential reasons: Not properly accounting for a change in facility occupancy or use Not accounting for (or incorrectly accounting for) non-routine events Removing overstated savings from additionally incented measures due…
Read More
Jun 09
Love0

How Long Will the Savings Last? Exploring Persistence and the Rebound Effect

By The Big Why of Evaluation No Comments
Last month’s post focused on how to best treat future energy or carbon savings and how this decision can affect whether short-term or long-term investments are more favorable. But even more crucial to the calculation is the persistence of the savings – that is, what savings can we expect in the future and for how long? What if a measure and its efficiency last well into the future but people begin to use that equipment or resource more, reducing or negating any savings? Persistence Whatever one’s reason for installing a new energy-efficient widget or implementing an energy-efficient behavior, the two most important pieces…
Read More
May 12
Love0

Discount Rates, Carbon, and the Ministry for the Future

By The Big Why of Evaluation No Comments
I recently read The Ministry for the Future by Kim Stanley Robinson. It’s a science fiction novel about climate change in the near future (a.k.a. cli-fi). It is realistic, alarming, and ultimately uplifting and I recommend checking it out. Interestingly (to an evaluator, at least), one of the story’s central concerns was the treatment of discount rates. “Having debunked the tragedy of the commons, they now were trying to direct our attention to what they called the tragedy of the time horizon. Meaning we can’t imagine the suffering of people of the future, so nothing much gets done on their behalf. What we do…
Read More
Apr 14
Love0

Cascading Effects and Market Transformation

By The Big Why of Evaluation No Comments
Like many, I was transfixed in late March by the huge container ship stuck in the Suez Canal. And also, like many, I enjoyed the many versions of the meme of a “little” excavator trying its best to free the enormous ship but making minimal progress. As often happens, my thoughts went to the massive problem of climate change and the seeming futility of our efforts in the energy efficiency industry to make a meaningful dent in the CO2 emissions. As an individual or even someone who may have a minuscule impact on energy efficiency programs, what can I do in the…
Read More
Mar 17
Love0

Why Evaluators Need Whiteboards: Avoiding Double-Counting in Evaluation

By The Big Why of Evaluation No Comments
I love explaining things with diagrams. One thing I have really missed working from home over the past year is not being able to illustrate ideas on a whiteboard in discussions with colleagues. Over the past month, there have been several instances where I needed to think about how to account for double counting in an evaluation, and drawing a picture was the best way to wrap my mind around the problem. This post covers some examples of potential double-counting common to energy efficiency evaluation and how to avoid them. Enjoy the pictures. Multiple Measure Interactions One common source of…
Read More
Feb 17
Love0

Pies, Nice Dinners, and Broken Windows – Attribution, Causality, and NTG

By The Big Why of Evaluation No Comments
Welcome back to the Challenges of Estimating NTG, where we explore the challenges of assigning a numeric counterfactual value to a complex decision-making process. Part 1 discussed the baseline in free-ridership questions to determine what would have happened in the absence of program intervention. As an example, without receiving an incentive for installing a high-efficiency A/C system, what level of efficiency would a participant have installed? As described in Part 1, program administrators try to maximize the savings that can be attributed to the program, and net-to-gross research attempts to quantify this attribution. But is attribution what we really should be studying?…
Read More
Jan 20
Love0

Program Goals and Evaluation

By The Big Why of Evaluation No Comments
It’s January and after a somewhat eventful 2020, flipping the calendar to a new year seems extra meaningful. Because many people set New Year’s resolutions, I thought a good topic for this month’s blog would be the goals of energy efficiency programs and how evaluation can help program administrators meet them. Implications of Different Types of Goals The reasons to implement demand-side management programs can vary and have changed over time. The goals can include reducing the need for new construction of generation and T&D capacity, reducing greenhouse gas (GHG) emissions, developing clean energy jobs, reducing energy burden of low income customers, ensuring high…
Read More
Dec 22
Love0

Brad Stevens, Sampling Error, and Cost Effectiveness

By The Big Why of Evaluation No Comments
I am a big fan of the Boston Celtics and their coach, Brad Stevens. How does this relate to sampling error in cost-effectiveness testing? Read on and let’s see if I can stick the landing. Original image credit: nba.com I have always liked Brad Stevens’ mentality that you shouldn’t get too caught up in the results of a close game if it came down to making or missing a last minute shot. The same issues that got the team into that close situation still need to be addressed whether the game ends with a one point win or a one…
Read More