I’m not your lover. I’m not your friend. I am something you will never comprehend. Beeeeewwww. I would die 4 U.
The wonderful energy efficiency industry in which we all (or most of us) work features program administrators/implementers and evaluators. For a given portfolio, you may either implement programs or support these efforts, or you may evaluate how well the program is working in regard to impacts (savings), accuracy, customer satisfaction, and a slew of other things. In no-man’s land, or the DMZ, there exists the incredibly squishy things like potential studies and market effects studies.
Any of the couple dozen major providers of all these services can provide more than one of any type of these services. For instance, Michaels provides implementation services and programs, as well as evaluation services – as do many other firms: DNVGL, AEG, ERS, and Nexant, to name a few. Other firms provide potential studies and evaluation. Examples include Itron, Navigant, Opinion Dynamics Corporation, and Cadmus. Some provide all three services.
Can there be conflicts of interest here? Sure. A firm implementing and evaluating its own work, directly or indirectly, is a conflict of interest. If nothing else, it is difficult to not have bias, even if the evaluator consciously attempts to avoid bias. This is for the same reason I want an independent cold read of a document by a person who hasn’t been involved at all, before we submit a report or proposal. There is a production team and a review team but just before “it” is ready to go out the door, neither are in good mental condition (I’m tired of looking at this and need to move on) to provide the last review.
One thing that cannot be a conflict of interest is when competing firms in one sport (say, implementation) are on the opposite sides of the fence for a given program or portfolio. For instance, the evaluators of a given portfolio may evaluate the work of firms they compete with on implementation elsewhere or at other times. If this were a conflict of interest, practically the whole industry would have to shut down.
Not an implementer? Many times these firms do potential studies including deemed savings estimates. Should they be eligible to evaluate the program for which they produced deemed savings estimates? I would say no. In this case, where is the independent assessment of the savings estimates? However, in many jurisdictions, technical reference manuals are the rule for savings, and the potential study involves everything but the measure-level savings estimates. That could be ok.
There are firms that only work on the implementation side of the fence. They are pure and have no conflict of interest, and their energy models are perfect[1], but in our experience they can also be the most difficult to work with. This is because evaluators seldom increase savings, although it does occur sometimes. It may also be because they don’t appreciate the job we have to do, which is – get it right!
Furthermore, we have experienced implementation firms that grew up 15 years ago before there were industry standards for determining impacts. Namely, what is the appropriate baseline, status quo, or business as usual alternative to use for comparison? I’ve said it a thousand, or maybe a million times: savings depend on existing or baseline conditions as much or more than the efficiency of the new stuff being installed. So I understand; it’s a jagged bitter pill to swallow when you’ve been using electric resistance heaters and 10x oversized pumps as baselines for the past dozen years, and no one knew, and now that kWh counter you have on your home page needs to have a couple digits removed.
Here are some of the non-conflicts that we have experienced:
- Our implementation work has been evaluated by our evaluation competitor – while in other jurisdictions we are evaluating their implementation work
- Our evaluation client (prime contractor) in one state is evaluating our work in another state
- Our evaluation client (different prime contractor) in one state is being evaluated by us in another state
We don’t bend to pressure in these cases. We call the balls and strikes as we see them, including telling our client in the presence of their client that we think they are wrong, and we back it up with lots of precedents and engineering.
It’s Proprietary
The proprietary stonewall typically includes a graphical interface to run some type of simulation engine. Maybe it runs a series of simulations to compare a bunch of alternatives.
Big deal.
Technology has hit Mach speeds in our industry too, and if we don’t have something similar or better, a government laboratory will be rolling things out for free of charge. By the way, this always ticks me off. The government uses tax payer money, or more like infinite borrowing, to develop things that are made available to everyone – taking down a competitive advantage WE paid for.
About that proprietary thing you have: we don’t want it for our use anyway because we won’t trust it; we wouldn’t like its limitations, assumptions and all sorts of stuff. Trust me, we’ve reviewed gobs of others’ stuff. It may be perfectly sound, but it doesn’t match our needs, ever.
Besides, we have contracts with confidentiality agreements and we honor them.
[1] Sarcasm is cheap wit.
