E2e, and I have no idea what that stands for, is a joint initiative of University of California-Berkeley, The University of Chicago, and Massachusetts Institute of Technology. Their charter is to assess and quantify the energy efficiency gap between estimated, or ex-ante savings, and observed or measured savings; kind of like the Energy Rant attempts to provide.
In Kool-Aid and Happy Face Rugs, I first referenced one of their reports in which they critiqued WAP – the federal government’s weatherization assistance program. That paper indicated WAPs produce only 40% of claimed energy savings. The methodology caught a lot of flak from folks in our industry, but not from me. I found it to be credibly reasonable.
E2e released a fresh paper last month to report on the effectiveness of non-monetary drivers delivered through Oracle’s (Opower) Home Energy Reports, or HERs. The title: Social Comparison Nudges Without Monetary Incentives: Evidence from Home Energy Reports. Non-monetary nudges include competition (gamification), social norms (join the crowd), or moral suasion (bald-faced shaming).
I summarize the results borrowing a phrase from last week: show me the money, Jerry, or I ain’t playin. In the absence of saving money, people do nothing to save energy (although nobody wants to admit it). This is consistent with my truism that people favor reducing greenhouse gases and fixing the climate as long as it is free or someone else is paying the tab.
Spoiler: I believe the results, but I question their applicability to the effectiveness of HERs impacts in the not-fake everyday world.
First, to get a home energy report, you need to pay an energy bill, so I question the usefulness of such an exercise. We are testing an energy-saving instrument on specimens who would never see the instrument while taking away the primary driver of the instrument. Why does this matter? Who. Cares. Why would anyone expect the results to be more than zero measurable impact?
Second, the specimens, in this case, are college students. I’m old, but I remember college like I remember kindergarten and learning to ride bike without training wheels, on gravel. While college kids today may be environmentally oriented, they are not stupid, as we were not stupid. Anything free will be devoured like beer at the Delta house during pledge week. These guys would do well to have a door closer to keep the outdoors and livestock out of the home.
Third, fake HERs were provided to only nudge manually setting back thermostats. Why? Apparently, because thermostats with communication capabilities were already in place. There is no discussion on the thermostat type or communication channel that I can find. The reason for the communicating thermostat only is that in my estimation, it is the only thing available for measurement. Again, the specimens are not paying bills, so there is no gas, electric, thermal or other metering available. Maybe they would deploy other energy-saving activities with lighting and electronics. We don’t know because there is no submetering at the housing unit level even. Let us move on.
Fourth, I’ve lived in apartments, and I’ve lived in student housing, but at a time when the only wireless available was AM, FM, UHF, and VHF. Our student housing units were tiny and therefore doors were open. We were a communicative bunch. We would not be manipulated. If the man wanted us to zig, we would zag. Who is the rat and who is the scientist in this scenario? As an apartment dweller, my experience was the opposite. We would go out of our way to avoid the neighbors. To demonstrate, I recommend watching Small Apartments, a dark comedy of apartment misfits in modern day Southern California. This scenario and demographic matters because apartment dwellers often pay their own utility bills and they aren’t playing games with the scientist – the man. They are their own rats, and I would bet they are more susceptible to manipulation and moral suasion, not to mention monetary forces.
Fifth, and maybe this is one of the biggest flaws, the minimum temperature setting is 68F. How do you get any clearance to see results when the minimum setpoint is so high? That’s like assessing diet effectiveness while capping weight loss at five pounds. You will see a bunch of people do nothing while maybe 25% engage and would do a lot more if allowed.
Interestingly (this leads to the sixth), as shown in the figure below, it appears the specimens ratchet the thermostat over time and never set it lower. Every time they feel it is a bit chilly, they bump it up a couple degrees. This happens over the long term, from the start of the heating season until the Thanksgiving break, and then again after the break. They ratchet faster after the break because it’s cold outside!
Sixth, I question the validity of the data. The treatment group consistently has their thermostats set higher than the control group, including very clearly over the Thanksgiving break. This is not random. I don’t care how small the difference. There is something weird with the sample.
Seventh, you are either the tester or the tested – the evaluator or the implementer. When we are program evaluators, we stay on that side of the fence. The ex-ante is right, wrong, or more often, somewhere in between. If we describe what to do and how to do it, we just left the evaluation team. It seems this E2e study is produced and evaluated by the same team. This is not best practice.
From my experience, E2e does good work, but I think my study critique finds this one was a waste of money.
Join the discussion One Comment