The first brief of this two-brief series defined what forward capacity markets (FCM) are and provided a quick overview of how they work. This brief dives into more detail about how the decision to bid those demand resources from demand side management (DSM) programs affects program evaluations. Each year or program cycle, independent evaluations are usually required to be completed on all DSM programs. Conveniently enough, bidding demand resources into the FCM also requires evaluation and verification that the demand resources are actually there. However, your run of the mill impact evaluation won’t get the job done.
Rules and Regulations
The impact evaluations that are completed for bidding demand resources must be consistent with the rules outlined by that particular market. A couple of widely known guidelines are ISO – New England’s M-MVDR and PJM’s Manual 18B. These documents provide evaluators with the guidelines they must use when quantifying the impacts of DSM programs. Below are a couple of highlights:
- Sample Design – Each individual sample taken and the overall portfolio result must achieve 10% relative precision with an 80% confidence interval. This means there is at least an 80% chance that the verified savings are within plus or minus 10% of the true savings. Some require 10% relative precision with a 90% confidence interval.
- Baseline Selection – the load (kW) that would have existed during the utility’s peak hours if the project had not been completed.
- Data Collection – Data collection must be completed and must utilize equipment that has accuracy of no less than ±2%. This includes measurement of other variables, such as temperature or amperage, if demand isn’t measured directly. The resulting demand calculation can have error of no more than 2%.
These are just a few of the requirements for metering, procedures, and reporting that evaluators must complete. Any good evaluation contractor will have processes that are similar to those required. However, the metering precision and calculation accuracy requires careful metering and expensive equipment. Additionally, data collection, such as metered data, equipment specifications, or interval data, must be completed for each site that is selected in the sample, regardless of whether or not it’s needed. Finally, in order to have enough confidence in the results, data collection must take place over multiple weeks, not just a couple days.
Impact on Cost and Schedule
These requirements all add cost. Collecting data onsite requires two visits instead of one: one to deploy data logging equipment and a second to pick it up. Extra reporting takes additional labor. High accuracy data logging equipment is expensive to purchase, and therefore, adds to the equipment rental fees consultants charge. Recording data for weeks takes, well, weeks. None of this will come as a surprise to program staff that have been through this process before. However, for those that haven’t, forcing an FCM compliant evaluation into an existing standard evaluation schedule and will result in late delivery and blown budgets. Take the time, plan it out, and work with the evaluation contractors. An experienced evaluator will be able to help with the requirements, planning, and potential pitfalls.
 M-MVDR: Manual for Measurement and Verification of Demand Reduction Value from Demand Resources
 PJM Manual 18B: Energy Efficiency Measurement and Verification
 Sample precision and confidence are statistics terms. Relative precision is the estimated error bands around the answer. Confidence interval is how confident someone is the answer is within that range. “Michaels is 80% confident the savings are within plus or minus 10% of 100 kW.”