I spent last week at the International Energy Program Evaluation Conference, IEPEC, as in, I-E-P-E-C to hard core evaluators or I-Peck for the rest of us.
Ninety-five percent of the conference including content and networking was great. Of course with this being the Energy Rant, I will beat on the remaining 5%.
Recapping, there are generally two portions of program evaluation: impact and process. Impact evaluation, which is what we at Michaels do, involves the assessment of savings (impacts) programs achieve, including what the measure actually saves (gross savings) and what impact the program had on the savings (net savings). For example, my mother started buying LED Christmas lights and practically replaced her Las Vegas scale lighting system with LEDs in one year. She then showed me forms she could submit to get cash back from the utility. Mom loves the lights. The cash was just a handout. Gross savings may have been decent for a residential end user. Net savings were zero because she already bought them without knowledge of the incentives or program.
Per the California Evaluation Framework, 2004, process evaluation is a systematic assessment of an energy efficiency program for the purposes of (1) documenting program operations at the time of the examination, and (2) identifying and recommending improvements that can be made to the program to increase the program’s efficiency or effectiveness for acquiring energy resources while maintaining high levels of participant satisfaction. The term “program’s efficiency and effectiveness” refers to dollars spent on programs. Are they achieving real impacts or just handing out money?
The conference is dominated by process evaluators and their counterparts on the utility and government (state and federal) side of things. Many (10-30%?) of these people I think have orbited a little too far from earth and spun off to other galaxies. They live in a galaxy far away and they argue about things like Zeno’s Paradox, except more obtuse, nebulous, and alien than our pal, Zeno. From here on I’ll refer to life forms as ETs, as in extraterrestrials.
I’m sending out a call to program implementers – utilities and third party implementers – you need to get involved with these evaluation conferences because I think you might be surprised at what you hear. For example, the keynote speaker Naomi Oreskes, a professor of history and science, talked; well let’s just say the title of her book is “Merchants of Doubt” (about global warming). Her message involved the history of deniers going back to the end of the cold war, later risks of tobacco, and so on up to global warming. It was interesting but as you know, I have doubts about significant human derived global warming and the conference itself perpetuates the basis of my doubts. I believe the reason it isn’t getting traction is people do not perceive any climate changes or adverse effects. Change is normal. On the other hand in the tobacco fight, black rotten lungs and people dying a heinous cancerous death are clearly eye openers.
Like climate science, the conference is supposed to be based on engineering, science, social behavior, decision making processes and the like. But in a large sense, to some it has turned into an advocacy group for policies to mitigate climate change and the philosophy of evaluation is starting to look like the philosophy of climate change as presented by the ETs. Papers for the conference have the same “peer review” process that the climate scientists supposedly have. In fact, as moderator, I peer reviewed three papers for my session and those papers focused on subjects relative to program evaluation, including how do customer perceptions of their lighting hours match reality as determined by logged data. What are the market effects of high bay lighting in California relative to a control group of states with no programs? This was interesting stuff and the papers were informative. I learned a lot and even referenced some of the findings already in proposals. To be sure, other sessions presented much pertinent useful information.
However, some latter sessions were really whacko fringe stuff that had nothing whatsoever to do with program evaluation. I attended one session that I swear to my maker was a doomsday cult session. People were pounding away on their smart phones during the session, possibly to book flights to Guyana?? In fact, I was thinking of shooting off an email to my loved ones saying that it’s been great knowing you and thank you for everything, Mom. The presenter was talking about how megatons of methane locked up in silt from a Siberian river flowing into the arctic ocean was going to be released and temperatures would rise 20 degrees or so in 20 years and a billion people would die from drought and famine. No kidding! Sounds like the makings of a Superman IIX movie. And this guy was absolutely convinced this is about to happen and absolutely sure what the precise effects would be. He spoke factually and in reading his paper he speaks of certain things to come – i.e., he was not saying,” could”, “may”, or “worst case” – it was “will” for all claims.
The only “will” there is about the future is, I will die, and I will pay taxes.
At the same conference there were panels of ETs who would debate things for which I could not understand whatsoever. When a guy (me) who’s been in the business for 16 years doesn’t understand the message, the message is miserably and hopelessly off course. The ring leader would say, “so and so is a contrarian and the other three panelists are believers” [in something?]. I don’t know what. They are so far out in left field you have to remember the seven time zones difference to ask them a question so you wake them up in the middle of the night. Finally, one woman in the session – an implementer – stood up and commented that we need to start talking the language of ISOs (independent system operators that control the grid), power suppliers and utilities or we will lose relevance not only for EE and policy, but for evaluation as well. I was about to give her a standing-O right there for COMMON SENSE. The first words out of one of the panelists mouth was, “I disagree.” Boooooooooo! What is wrong with these ETs?
The day before this there was a panel of mostly ETs to discuss politics of evaluation, which turned into politics of global warming of course. Note that probably most old timers still inhabit our planet and they are clear thinking people with their eye on the ball, which is improving programs. They didn’t all arrive at Boston Harbor in the starship lollipop. Moving on, each of the panelists gave their sermons and then it was turned over to ETs in the audience voice their proposed chapters to the holy book. I actually had to leave before it was over. The typical rant would start with, “I just have a couple quick comments.” Five minutes later they were talking about chicken farming on a nickel a day in Kenya. They would babble incoherently like someone who had been dropped on their head and doped up on maximum doses of morphine prescribed by their doctor. Anyone home? Is there a question in there somewhere? Perhaps a comment regarding our industry? Have you ever been introduced to concise?
Many ETs argue that we cannot quantify the true impacts of EE programs so just fuggedaboutit. We cannot treat EE or demand response as a resource so fuggedaboutit. What the? Then why have these programs? They know for sure that they need plane tickets to Guyana but we can’t even come close on program impacts? See what I mean? Does this make sense? The juxtaposition with program evaluation and climate change seems to be, believe. We can’t really measure it. We know neither the day nor the hour. It is going to kill us and these programs are cost effective. Just keep the money coming.
As it turns out, I was not the only flat earther in attendance. Numerous other heretics including one who had been away from these conferences for 10 years said the same thing, unsolicited. “It has become a religion,” one elder statesperson said. Well, heeeyah!
Let’s get rid of the underlying stealth agenda and crap papers and refill with good stuff and save the ET rants for the lollipop ride back to planet Koozebane.
Join the discussion One Comment