A couple years ago I was attending an ACEEE conference and I was speaking with a gentleman who with his company was a program implementer. I remember him saying that program evaluators should always work for the regulators. If evaluators worked for the program implementer, which in many cases is the utility directly, the results would be biased. I thought, no way. There is no way our profession is to be swayed by the desire to not make waves with the client. As I say on our staff page I will tell people what they need to know, not what they like to hear.
As I think someone once said, if you live long enough, you will see everything. While I/we have not compromised our principles, the guy from the ACEEE conference was right. Evaluators should work for (report to) the regulators. In Don’t Ask, Don’t Look, Don’t Tell, I talk about program implementers (utilities in this case) only providing filtered information they think we need to complete energy analyses, and blocking our access to project information for evaluation so we can get to the bottom of things. Well this has been taken a step further.
On a previous project, we provided too much of what they needed to know and not enough of what they wanted to hear. This time we are not to go on site and investigate and gather actual project information or log key operating parameters to precisely measure gross savings for prescriptive measures. We are to do a file review but not call the customer. Good grief. This is like looking at the silhouette of an automobile from 400 meters by moonlight in the fog and verifying the mileage. We can’t even tell if it’s a 1982 Yugo, 1984 Escort Diesel, or a 1977 Royal Monaco. Why bother. The report can be summarized in the following sentence. “The gross savings of 5,000 GWh achieved by this program do not seem unreasonable as viewed from the perfect pitch black vacuum in which we evaluated the savings.” Congratulations. The lack of information leaves us with no choice to in any way determine gross, let alone net, impacts.
We as an industry sell energy efficiency as a resource, like thermal power plants, wind turbines, photovoltaics, hydro and other sources of electricity, whatever they may be. Stepping a bit out of my area of expertise, utilities regularly provide integrated resource plans, IRPs – 20 year projections of how they plan to meet customer demand. If energy efficiency is a resource per an IRP and the evaluation team is bound, gagged, blindfolded and duct taped to a lightning cut-out switch such that plus-or-minus-50% savings are being achieved, how can the utility use energy efficiency in its plan? It is ignorant denial to believe a program delivers exactly the projected savings. Moreover, it is doing ratepayers a disservice by claiming return on their investment in EE that may be significantly in error – not just for trailing years, which is forgivable, but for future years, the ignored errors for which are not forgivable.
At the same time in other jurisdictions, there are crazy, meaningless instrumentation calibration and accuracy requirements. Consider, the typical evaluation provides 90/10 results. That is, there is 90% confidence that the results of the evaluation are within plus or minus 10% of the actual PROGRAM results. Meanwhile, a kW logger needs to have accuracy to within plus or minus 1%. Does the US Congress write these requirements? In a rather depressing analysis of all statistically possible outcomes using a Monte Carlo model (a hideous mathematical model, not a 6,000 lb tank disguised as a car), a typical measure has a range of possibilities of plus 150% or minus 95% after all the possible “errors” are piled on top of one another. But by God, it could have been plus 152% or minus 96% so it’s a good thing we have accurate data loggers.
On top of that, there is this harsh reality: we can monitor and investigate the bajeebas out of a project and it may all change next week. Who eventually wins the Republican nomination for President or even the next debate for that matter will impact savings more than the accuracy required for the instrumentation will. If a category three hurricane makes landfall on Fort Lauderdale in August, it will change the results of a lighting program in Massachusetts by 1.5%. If it makes landfall in October, the Massachusetts results will be off by 3%. Why? For the same reason a butterfly in New Zealand causes a two foot snowstorm in Washington DC. Seriously, there is no way to measure known and unknown influences or travel through time to estimate/verify savings with anywhere near that accuracy, so why bother with the ridiculous instrumentation requirements?
Recommendation – First, program evaluations should be reported in their entirety to regulators. Reasonable (budget restrictions) and unreasonable (don’t want anyone to know) constraints on accuracy and usefulness of the results shall be described in the evaluation report. Second, leave the assessment of gross and net savings estimates to the expertise of professionals in the industry, not someone who doesn’t understand real world influences on savings.
Tidbits
You may know how much a fan I am of electric cars. Last fall I had responded to an opinion piece in The Wall Street Journal regarding electric cars. The writer claimed the electric car would develop like cellular phones have over the past 30 years. I wrote that this would not happen because of physical constraints in electrical infrastructure. To wit, you would have to change out step down transformers feeding homes or you would have to wait forever to “refuel” your electric car. An un-subscribed do-it-yourselfer electrician wannabee blasted my assertions. Well wouldn’t you know it, a couple fine folks from Itron in a paper listed transformers as number two (2) on their list of the top 10 challenges for electric vehicles. They conclude plug in hybrids will serve as the primary vehicles while EVs will serve as a secondary vehicle. A $40,000 secondary vehicle! Gee, I think this is what I meant when I said EVs will be a frivolous novelty. A much more appealing frivolous novelty would be a slightly used 911 Cabrio for that kind of money.
Outrage of the Week
First, as another backup for my whippings (see Oh Behave), Google has axed its “Powermeter” application which allows homeowners to monitor home energy use on iPhones, iPads, iTables, iEtcs.
At nearly the same time, the geniuses at Google issued their own brilliant report “mapping out a path forward for the country’s energy sector.” They pulled some financial and jobs benefits of such a program out of the air, but it is becoming obvious Google has no clue about our industry. Here is the outrage: “It [the study] also models the potential effects of a series of federal policies, including a clean energy standard that would mandate a certain percentage of the country’s electricity come from low-carbon sources, increased energy efficiency and higher fuel economy standards.” While Messrs. Bryn and Schmidt may know how to skim billions from their entrepreneurial endeavors over the internet, they are clueless as Congress is regarding energy policy. They have resorted to the General Electric tactic of setting up shop on K Street to make their own market by having politicians do so for them.
Speaking of GE, one of their legions of government money grubbers wants Congress to make policy (mandates for GE technologies) so they can get on with making money for their shareholders – tax free of course.
