Methodology: Analyzing New England energy markets

How we calculated the cost of unused pipeline capacity for electricity customers

Our new analysis indicates that New England electricity customers would have saved upwards of $3.6 billion between 2013 and 2016 if unused pipeline capacity had been available to deliver gas for electricity generation. Instead, it was unavailable due to a repeated pattern of last-minute scheduling changes by gas utilities owned by the same two parent energy companies.

To determine the financial impact of unused pipeline capacity on New England's electricity customers, we first assessed what gas prices would have been in the secondary markets – where many electric generators buy their gas – had that unused capacity been available. We then modeled how the wholesale electricity market operated by the New England Independent System Operator (ISO-NE) would have responded to these lower natural gas prices.

Step 1: Determine the natural gas price with a fully-utilized pipeline

Figure 1 (below) illustrates how the gas price changed with unused capacity. We observed the actual price with unused capacity but wanted to estimate the price if the unused capacity had been available to bring more gas to market. To do so, we needed to determine how the gas price changes with an increase in the amount of available gas.

Gas supply and demand curve

Figure 1. Simplified illustration of gas demand and gas supply in the spot market. The supply curve slopes steeply upward as the pipeline approaches its physical capacity constraint (blue line) or an artificial capacity constraint induced by scheduling patterns that block others from using the space (black line).

To determine by how much additional pipeline capacity would have decreased the gas price, we needed to know the sensitivity of gas demand to the gas price, or the "elasticity of demand" for gas. This elasticity tells us the percentage decrease in gas quantity demanded in response to a one-percent increase in the gas price. Conversely, it can also tell us the decrease in the gas price in response to an increase in available quantities of gas.

Using an econometric analysis technique called Instrumental Variables on data on demanded quantities and prices in the Algonquin Citygate spot market, we estimated a demand elasticity of -0.27.

Then, using that figure, we constructed a counterfactual Algonquin Citygate gas price series by taking the quantity of gas downscheduled late in the day from mid-2013 to mid-2016 as the hypothetical increase in the quantity of gas available to the secondary market.

To avoid overestimating the amount of unused capacity, we subtracted the daily average fraction of downscheduled capacity observed at all other nodes where systematic downscheduling was not apparent.

Adding downscheduled quantities back into the supply lowered the gas spot price as indicated by the price elasticity estimate of -0.27, as illustrated in Figure 1. We utilized this series of counterfactual gas prices as a determinant for the gas-fired generators' bids in our electricity simulation model (explained below).

Using this methodology, we estimate that wholesale gas prices paid by New England generators were $1.68 (38 percent) higher on average during our three-year study period and $3.82 (68 percent) higher on average during the winters (December 1 - March 31, as defined by the Algonquin Pipeline).

Step two: Determine the electricity price with a fully-utilized pipeline

To estimate how wholesale electricity prices were impacted by a reduced supply of pipeline capacity, we reconstructed ISO-NE's wholesale electricity market and used a resampling simulation model with our estimated counterfactual gas price series as an input.

For these simulations, we used a dataset of 8 million observations of the actual bids of generators submitted to the ISO-NE day-ahead electricity market. For each hour of each day in our three-year period, each generator submitted a bid function specifying how much electricity the generator would produce if the price was $P1, and how much more it would produce if the price increased to $P2, etc. For each hour of each day in our three-year study period, we aggregated generators' bids to create a generation supply curve for the market, determined where that supply curve intersected demand, and recovered the equilibrium electricity price as the model's output. (See this blog post for an illustration of the generation supply curve and how the equilibrium price was determined by where demand intersected supply.)

To determine the impact of lower gas prices, we adjusted generator bids prior to aggregating using a resampling-matching algorithm. Broadly, this procedure consists of the following:

  1. Match each day with the 5 percent of days in our three-year study period that are most similar in terms of temperature and electricity demand, which we consider to be the two most important determinants of generator bids into the wholesale electricity market (other than the gas price). For each day, this produces a set of about 60 days that are highly similar to the target day in general electricity market conditions.
  2. For each day, select from its subsample of similar days the 5 percent of days (three days) that experienced a gas price most similar to our estimated counterfactual gas price. This produces three match days for each target day that are highly similar in terms of temperature and electricity demand, but have a gas price close to our estimated counterfactual gas price.
  3. For each day, randomly draw generators' bid functions from the three match days, independently for each generator. Then, for each hour of each day, construct the market supply curve using the randomly drawn generator bids and match supply with demand to clear the market, recovering a counterfactual wholesale electricity market clearing price as the output.
  4. Repeat Number 3 a large number of times (we performed 1,000 iterations) to increase the precision of the estimation procedure and recover a distribution of counterfactual electricity price series from which we can calculate point estimates and confidence intervals.

We began by running this simulation model using the actual gas price instead of the counterfactual gas price to check the general performance of our model, finding that it generated an electricity price series very close to the actual one. We then ran it with our estimated counterfactual gas price series to determine what electricity prices would have been without withholding. We used the difference between these two simulations as our estimated cost to electricity ratepayers in order to correct for any bias that might have been introduced by our simulation methodology.

By multiplying this price difference by demand for each hour of each day and then summing it up, we can calculate the estimated additional cost to electricity ratepayers due to pipeline capacity underutilization. We estimate this cost to be $3.6 billion dollars over the entire three-year study period, with a 95 percent confidence interval between $3.2 billion to $4.1 billion.