The Trading of Electricity

Abstract

After having introduced the theoretical foundations of electricity markets and some of the main design features such as marginal pricing of electricity, spatial distribution of electricity prices and the question of system adequacy, the authors discuss how they function in practice. After a description of the functioning of wholesale electricity markets with a focus on the derivatives and the spot market, the authors analyse their main features such as trading venues, traded products, rules and the processes but also some key trends that can be observed. Going forward, with the energy transition, electricity systems around the world are undergoing nascent but profound changes as market architecture and regulatory framework evolve to meet ambitious climate targets while maintaining efficient investment incentives and security of supply. The authors highlight the need to identify the future requirements of the power system and align them with global policy objectives in order to adapt and enhance the way electricity markets generate social welfare.

1 Introduction

1.1 The Electricity Industry and Its Value Chain

Electricity is defined as a set of physical phenomena corresponding to the presence and flow of electric charge. Since the discovery of electricity in the nineteenth century, electricity has become an essential good to our society that has played an immense role for mankind’s economic development, enabling a widespread and cheap energy production and transportation that is used to power our economy and daily lives.

Electricity is an energy source like no other, in the sense that it is immaterial. Other energy sources like wood, gas or oil are material, can be contained and quantified with a volume or a weight and are storable on a large scale, unlike electricity. Moreover, electricity is not freely available in large amounts naturally and must be generated from a primary energy source. Electricity can be produced on a large scale from:

• Thermal power plants: A fluid (most often water) is heated by the combustion of a fuel (gas, coal, oil) or by a nuclear reaction (nuclear reactor). This energy in the form of heat is then converted to electric power. In most designs, the water is turned into steam and spins a turbine driving an electrical generator;

• Hydro power plants: The water flow coming from a reservoir or in a river spins a turbine driving an electrical generator;

• Wind turbines: The wind flow spins a turbine driving an electrical generator;

• Solar photovoltaic installations: Electricity is generated through the conversion of light into an electric current using the photovoltaic effect.

In this chapter, electricity is referred to as either power—expressed in megawatts (MW)—that corresponds to instantaneous ‘work’ or energy—expressed in megawatt-hours (MWh)—which represent work over a period of time.

Once produced, the electricity is transported to the end-consumers through the power grid, that can be divided in two parts:

• The transmission grid, that serves for the transportation of electricity over long distances, mostly through high-voltage overhead power lines;

• The distribution grid, that represents the last stage of the grid before the consumer. It is a low-voltage grid and consists of cables and lines connected to the consumers.

The value chain of the electricity industry can therefore be summarized into the four elements in Fig. 22.1.

Fig. 22.1

Value chain of the electricity sector. (Source: Authors’ elaboration)

1.2 The Emergence of Electricity Markets

When the electricity sector first developed in the nineteenth and twentieth centuries, electricity was not traded or commercialized through markets. The initially decentralized electricity grids were increasingly interconnected, and publicly owned vertically integrated utility companies were created to manage the whole value chain of the electricity system across large geographical areas: electricity generation, transmission, distribution and sale to end consumers. The price of electricity was regulated and determined by regulatory and government bodies. Such an organization worked well in many countries worldwide and has enabled a wide development of the electricity systems, grids and generation capabilities. It is in the late twentieth century that first concepts for the introduction of electricity markets were formulated and implemented.

• In 1990, the United Kingdom privatized the electricity supply industry, followed by the deregulation in several countries of the Commonwealth, notably giving rise to the National Electricity markets of Australia and New Zealand and the Alberta Electricity market in Canada.

• In 1991, the Norwegian electricity market is deregulated.

• In California, the market is deregulated in 1996, followed by many other states in the USA.

• In Europe, a European Union directive dating back to 1996 creates the framework for a liberalized European electricity market, which prompted the implementation of electricity markets throughout Europe in the early 2000s.

Essentially, the deregulation—or liberalization—of the electricity market corresponds to the introduction of (1) competition to sell electricity production with the creation of a wholesale electricity market, and in most cases (2) competition for electricity sale to consumers with the creation of a retail market. In practical terms, this means that formerly vertically integrated monopolies must be unbundled or disintegrated: electricity production and retail activities are competing in markets, whereas grid operations for the transmission and in most countries the distribution of electricity which are natural monopolies are managed within separate independent and regulated entities. The liberalization therefore sees the emergence of new types of companies (as illustrated in Fig. 22.2):

• Electricity generation companies:

  These are companies owning electricity generation assets that are competing to sell their production on the wholesale market. They make investment decisions to build new power plants in the hopes of making returns from the sale of the electricity production on wholesale markets.

• Transmission grid companies:

  They operate the high-voltage transmission grid and are responsible for grid stability and security of supply. They are regulated monopolies, and offer a neutral non-discriminatory grid access to all potential grid users (generators and consumers).

• Distribution grid companies:

  They operate the distribution grids and offer non-discriminatory access to the distribution grid. They are (semi-)regulated monopolies and, in some countries, are not unbundled from retail or generation activities. In this case, there might be no competition in the retail market, as the only possible retailer in a given area is the local distribution grid company. Most European countries and American states have now implemented retail competition.

• Electricity retail companies:

  They are commercial companies procuring electricity on the wholesale market in large amounts (or producing it with their own generation capabilities) and selling it to end-consumers on the retail market.

Fig. 22.2

Diagram showing the structure of the electricity value chain before and after the deregulation of the market. (Source: Authors’ elaboration)

2 Theoretical Foundation and Design of Wholesale Electricity Markets

The main ground for the introduction of electricity markets was to increase the social welfare over the electricity value chain and enable long-term benefits to consumers compared to the regulated monopoly structure, by means of reduced electricity prices and improved security of supply. Indeed, the competitive organization of the sector would (Joskow 2003):

• Provide incentives to improve capital investments and operating costs of existing and new generation assets

• Encourage technological innovation in electricity generation

• Shift the risk of technology choice, construction cost and operating “mistakes” from consumers (through public monopolies) to suppliers (and their private shareholders)

• Create better incentives for transmission and distribution monopolies, that would reduce associated costs for consumers and enable more efficient wholesale and retail markets

In Europe, there was also a second argument—a political one—for the implementation of electricity markets. Such markets would de facto be integrated into one common European market that would increase the cooperation and political ties between European countries.

The implementation of an electricity market is however no easy task on a technical level. Indeed, the electricity system relies on the electricity grid to function. When an electricity quantity is produced, it is injected at one location—or node—of the grid and withdrawn at the same or another node of the grid, where the consumer is. This physical system is bound by the laws of physics, that make electricity a very special commodity. Here are its main characteristics:

• 1. Grid balance: electricity cannot be stored on a large scale. Therefore, the grid must be balanced at all times between electricity generation and consumption;

• 2. Transmission constraints: each line or cable in the electricity grid has a maximum amount of power that can flow through it at any given time. This figure is called the transmission capacity. If a flow exceeds this limit, it creates a congestion that can lead, in the worst case, to a blackout;

• 3. Grid losses: the transportation of electricity through the power grid induces thermal energy losses, as the electric current heats the lines and this energy—in the form of heat—is dissipated into the atmosphere. On average, between 3% and 5% of the energy injected in the grid is lost through grid losses;

• 4. Electricity flows: electricity flows follow several paths in the grid from injection to withdrawal (as per Kirchhoff’s laws) with complex interactions between flow paths and generation or injection points, sometimes resulting in so-called loop flows which are unintended flows that can cause congestions on certain paths.

2.1 How to Define the Price of Electricity?

2.1.1 Marginal Pricing of Electricity

One of the most fundamental questions in the field of electricity system economics, even before the introduction of electricity markets, is the question of the price of electricity: at which price should electricity be sold in order to maximize economic welfare?¹

Long before the introduction of electricity markets, foundational work by the French engineer, economist Marcel Boiteux, published in 1949 (Boiteux 1949), paved the way to answer this question. His research has shown that electricity should be priced at its marginal cost.

In the case of an electricity system with several generation technologies, all electricity generation plants are sorted in the ascending order of their short-run marginal costs,² forming a step-wise curve called “merit-order.” The cheapest generation plants to meet a given electricity demand volume are dispatched to produce electricity. Finally, a unique electricity price is set for all consumers and all producers, as the (1) the short-run marginal cost (SRMC) of the most expensive power plant dispatched to produce electricity or (2) the demand marginal benefit³ when the demand equals the generation capacity of a given technology. In all cases, it is the price at the equilibrium of supply and demand marginal costs and benefits. As the volumes and costs of supply and demand vary in time, the electricity price varies accordingly, but always remains at the equilibrium, as illustrated in Fig. 22.3.

Fig. 22.3

Graph illustrating the concepts of marginal price, merit-order curve and short-run supply-demand equilibrium (as demand varies in time, several demand marginal benefit curves are shown in the graph). (Source: Authors’ elaboration)

Micro-economics theory shows that this price is generating the most economic welfare for a given demand-supply situation. This result is intuitive: if the prices were arbitrarily set any higher or lower, some value-generating consumption or production would not take place (or take place at a loss).

The energy rent earned by electricity generators is the difference between the electricity price and their short-run marginal cost of production. This rent must reimburse sunk investment costs and fixed maintenance costs for a power plant investment to be profitable. The electricity system reaches its long-term investment equilibrium when the annual energy rent of the system marginal power plant equals its annual fixed costs (capital annuities and maintenance). This concept is illustrated in Fig. 22.4.

Fig. 22.4

Graph illustrating the concept of long-term investment equilibrium. (Source: Authors’ elaboration)

In Fig. 22.4, the annual energy rent of a power plant decreases as the ratio of installed generation capacity versus peak demand in the system increases, because electricity prices decrease as the offer to produce electricity relative to peak demand increases. Let us review two cases to illustrate the investment equilibrium:

• If the energy rent earned by existing generators is consistently higher or expected to be higher in the future than their annual fixed costs—for instance because of a wholesale price increase—this indicates economic viability in investing in new generation capacity;

• On the contrary, if the energy rent is (or is expected to be) consistently lower than the fixed costs, some built power plants are not or will not be making positive returns on investment from their electricity sale, and it is a signal that new power plants would not be economically viable.

The long-term investment equilibrium is therefore reached when the energy rent of the least-earning generation plants equals their annual fixed costs, and it sets the optimal amount of generation capacity in the system relative to a given peak demand.

In the short term, because many consumers do not observe prices and cannot respond to prices in real time (the demand is inelastic), when the system load reaches the maximum capacity in the system prices can spike spectacularly to reflect the need for additional capacity (that will not be built overnight).

2.1.2 Spatial Distribution of Electricity Prices

As explained in the introduction of this theoretical section, there are grid limitations to the amount of electrical power that can be transferred from a grid node to another (the transmission constraints) and costs associated with electricity transmission (the grid losses), making the location an important factor for electricity price determination. In 1988, the spatial distribution of electricity prices was theorized by the American engineer Fred Schweppe and his colleagues (Schweppe et al. 1988) and later complemented by Hogan (1992) in 1992, with the emergence of the concept of Locational Marginal Pricing. The underlying idea is that one optimal electricity price—the Locational Marginal Price (LMP)—is determined at each node of the grid, as the price equilibrium between supply and demand at the node, that is to say the cost of delivering one extra megawatt-hour at the node, taking into account grid losses, loop flows and congestion costs. The marginal transmission cost (cost of transferring power) between two nodes is the difference in the cost of generation at these nodes. With such prices, a generator will produce if its SRMC plus the transmission price is lower than the cost of generation at the destination node. If the transmission capacity was unlimited between all grid nodes and there were no grid losses, the price would be equal for all nodes.

A simple two-node LMP example is described in Fig. 22.5 in two cases, with and without congestion.

Fig. 22.5

Simple two-node example illustrating the concept of LMP. (Source: Authors’ elaboration)

In case 1 without congestion, the generation source at node A is used to meet all the demand at nodes A and B, and their respective LMPs are equal to the cost of meeting an extra megawatt-hour of electricity. At node A, it is at the SRMC of the generation, equal to 30 €/MWh. At node B, it is the SRMC at node A plus the 5% grid loss cost, in total equal to 30*(1/0.95) ≈ 31.6 €/MWh.

In case 2 with congestion, the transmission capacity from node A to node B is fully used and the more expensive generator at node B is dispatched to supply 81 MW of demand at node B (as only 19 MW arrive from node A due to grid losses). The LMPs at nodes A and B—in other words the costs of meeting an extra megawatt-hour of demand at nodes A and B—are respectively equal to 30€/MWh and 40€/MWh.

2.2 The Emergence of Different Wholesale Market Designs

The aforementioned principles for the electricity price determination were formulated before the implementation of electricity markets and were originally meant to optimize electricity systems under the vertically integrated monopoly regime. They were however fundamental economic principles for the electricity price determination, that allowed the emergence of electricity markets. The SRMC of generation units and marginal demand benefit are replaced by market offers from generation and retail companies and the regulated investment in generation capacity by monopolies is replaced with private investment.

Regardless of the market design, the wholesale electricity markets are always divided into several timeframes:

• A long-term market allows for the trading of derivative products indexed on the short-term spot price of electricity. Market participants can manage their long-term price risk based on their future consumption needs or production capabilities;

• In the short term, a spot market allows for the physical dispatch of power plants, starting on the day-ahead of delivery down to the real time. This dispatch exercise is first done on the day-ahead of delivery as many large power plants have long start-up and ramping times. The spot market sets a spot price for electricity used to determine the dispatch of power plants in the short term and as a price reference for longer term derivative products.

• In real time, the electricity system is steered by system operators to ensure security of supply as all system constraints must be respected to ensure security of supply.

2.2.1 Nodal and Zonal Market Designs

Several wholesale electricity market designs have been studied and are currently implemented around the world. They differ with regards to how grid constraints are considered, how prices are calculated and information on production and consumption capabilities is centralized. This has consequences on the determination of the spot price and on the form and type of transactions that take place. The two general market designs that have emerged are the nodal design (currently implemented in several US states) and zonal market design (currently implemented in the European Union).

As its name suggests, the nodal market design corresponds to the determination of an LMP at each node of the electricity transmission grid (see Sect. 2.1), whereas in the zonal market design, zonal Market Clearing Prices (MCPs) are calculated for large geographical area, called Bidding Zones, as illustrated in Fig. 22.6. Biddings zones consist of many transmission nodes between which capacity limits are neglected under the so-called copper-plate assumption. For the price determination, each bidding zone in the zonal model is equivalent to a node in the nodal model as only transmission capacities between bidding zones are considered for the MCP calculation. This leads to different paradigms for congestion management.

Fig. 22.6

Illustration of the notion of bidding zone, node, and capacity between bidding zones. (Source: Authors’ elaboration)

In the nodal model, all potential congestions are considered “by design” for price determination leading to LMP differences between nodes in case of congestion. In the zonal model, only constraints between bidding zones are considered for the MCP calculation. Potential “intra-zonal” congestions are alleviated by Transmission System Operators (TSOs) outside of the wholesale market after the MCP has been determined, in several ways:

• Topological changes: changes to the grid topology to re-route electricity flows and alleviate the congestions.

• Re-dispatching measures: changes to the schedules of specific power plants to change the electricity flows throughout the grid and alleviate congestions.

The costs of these congestion-management measures are not reflected in the zonal MCPs and are borne by all grid users within the bidding zone. In nodal markets, such congestion-management measures can also be taken by Independent System Operators (ISOs) in case congestions appear due to an unforeseen event (power plant or transmission outages, forecast errors, etc.).

2.2.2 Centralized and Decentralized Market Organizations

One important characteristic of a market design is whether the market is centrally or decentrally organized. The real-time system steering is necessarily centralized and managed by system operators regardless of the design. However, the spot markets can either be centrally organized (e.g. US nodal markets) or decentrally organized (EU zonal markets).

In the US nodal markets, the short-term spot market algorithm sets the dispatch of power plants based on their technical constraints and marginal cost, submitted at a unit level to the market algorithm. These schedules are binding for power generators and can only be changed in the real-time market. The market bidding is therefore referred to as “centrally dispatched unit bidding.” Each unit has an individual commitment and a tailor-made contract (Ahlqvist et al. 2019). In some ways such markets reflect some procedures from national monopolies and regional power pools that existed before the deregulation (Wilson 2002). New England, PJM, Midwest, New York and California nodal markets are all in central-dispatch. In Europe, the UK England and Wales pool and the Single Electricity Market (SEM) in Ireland were examples of centralized markets, most other markets being decentrally organized. Britain and Ireland both changed to decentralized markets in 2001 and 2018 respectively.

In EU’s zonal spot market, market participants have the responsibility to optimize their assets themselves. They provide aggregated portfolio bids in accordance with the technical characteristics of their assets or with their consumption needs. Accepted bids create a physical delivery responsibility that can be adjusted in a continuous intraday market running until real time. This way of functioning is called the “self-dispatch with portfolio bidding”. Market players have an implicit responsibility to balance the electricity system; the Balance Responsible Parties (BRPs) are financially responsible for keeping their own position (sum of injections, withdrawals and transactions) balanced over given delivery periods, called the imbalance settlement periods. Depending on the state of the system, an imbalance charge is imposed per imbalance settlement period on the BRPs that are not in balance.

2.2.3 Ancillary Services

Under liberalized—or deregulated—electricity markets, the responsibility of the security of supply and grid stability is taken by system operation companies—the ISO in a nodal model and the TSO in a zonal model. As a complement to the wholesale electricity market, and in order to guarantee the security of supply throughout the interconnected electricity grid, there are ancillary services—or system services—managed by these system operating companies. These ancillary services correspond to a large set of operations going beyond the commercial operation of electricity generation, transmission and consumption activities. Historically, ancillary services were procured by system operators from large power plants, but they are nowadays increasingly more open to consumption and storage capacities as well as smaller scale generation. The main services covered by ancillary services are the following:

• Balancing and frequency control: the balancing of the grid to maintain the physical balance between supply and demand at every instant. The frequency of the grid is a value that can be monitored and reflects the real-time demand-supply balance. System operators typically contract flexible generation or storage that form “reserves” that are able to quickly react to frequency variations to keep it within a given range, around 50 Hz in Europe and 60 Hz in the US.

• Voltage control: the voltage in the electricity grid must be maintained within a given range and this is done by system operators through active and reactive power control on some generation assets.

• Black start: this system service guarantees the ability of the electricity grid to get back in operation after a black-out event. Power plants providing this service must be able to start their operations without relying on the electricity from the grid.

• Congestion management: system operators have the ability to steer some power plants and change their scheduled generation in order to solve expected grid congestions.

Ancillary services, although answering the same general objectives—as the laws of physics are the same everywhere—are organized differently from one market to another and are usually specific to given system operators. In the nodal market designs as implemented in some US states, the procurement of several ancillary services, such as the operating reserves for balancing, is co-optimized within the day-ahead wholesale market optimization, whereas in the zonal market design as applied in the EU, ancillary services are procured by TSOs in separate mechanisms and markets outside the wholesale market framework.

Table 22.1 summarizes the main characteristics of nodal and zonal market designs.

Table 22.1 Summary of the main characteristics of nodal and zonal market models

2.3 The Problem of System Adequacy: Capacity Mechanisms

The adequacy or reliability of the electricity system corresponds to the system’s ability to adequately supply the demand for electricity at any given time, and especially in times of peak demand. It relates to system planning (by TSOs or ISOs) and more specifically to the amount of available generation capacity available in the system with regards to the level of electricity demand. Reliability can be quantified with criteria such as the Loss-Of-Load Expectation (LOLE).⁴

In a liberalized electricity market, the amount of electricity generators present in the system depends on the price of electricity and on the generators’ revenues from wholesale markets and ancillary services. This rent must reimburse sunk investment costs paid to build a power plant (capital cost annuities) and fixed operations and maintenance costs (fixed O&M costs) to make a power plant investment profitable.

In an “energy-only” market,⁵ the installed generation capacity relative to peak demand—and therefore the system adequacy—is set by market forces through investment in generation capacity (see the long-term investment equilibrium described in Sect. 2.1).

In some wholesale electricity market setups, the energy-only remuneration of generators is not enough to guarantee the adequacy of the electricity system. The American economist S. Stoft (2002) is the first to have highlighted this issue in energy-only markets. For him, “The missing money problem is not that the market pays too little, but that it pays too little when we have the required level of reliability.” Such a situation can arise for different reasons:

• The market design and regulation do not allow generators to earn enough money to cover their fixed costs. For instance, electricity prices should be able to reach very high levels in times of supply scarcity, up to the level of Value of Lost Load⁶ (VoLL), which is rarely allowed;

• A reliability criterion arbitrarily set for the electricity system⁷ is conservative and maintains many generators in the system, increasing competition and bringing the market price and generation rents down.

The “missing money” is the difference between the generators’ annual fixed costs and their energy rent from the sale of electricity in the wholesale market and ancillary services in a medium- to long-term perspective (the market could be over-supplied in the short term).

The energy-only markets are therefore sometimes complemented with capacity mechanisms or capacity markets, remunerating installed generation capacity for being available in order to compensate the generators’ missing money and ensure a given level of reliability in the system. In the case of a capacity market, market forces adjust the capacity price to compensate the missing money and ensure needed investments in generation capacity to meet the defined reliability criterion. The graph in Fig. 22.7 illustrates the notion of missing money with a comparison to the long-term “energy-only” investment equilibrium.

Fig. 22.7

Graph illustrating the “Missing” Money problem. (Source: Authors’ elaboration inspired by The Brattle Group)

3 Electricity Trading in Practice

After having introduced the theoretical foundations of electricity markets, and some of the main design features, this section shows how they function in practice. We start by a description of the functioning of wholesale electricity markets where we focus on the derivatives and the spot market to analyse their main features, the traded products, the trading venues, the rules, processes and some of the challenges going forward.

3.1 Wholesale Electricity Trading

Since liberalization started, it is not the vertically integrated monopoly that decides which are the least-cost assets to start and stop in order to meet electricity demand. Nor where, when and what to invest. In a decentralized manner, market participants take these decisions based on the long- or short-term power prices.

3.1.1 Electricity Transactions

For various reasons that we will detail going forward, market participants trade before electricity is delivered to end-consumers (residential, businesses or industry) via the grid. A transaction is a contractual agreement made between a buyer and a seller to exchange a given volume of electricity in megawatt-hour during a given delivery period, at a given location for a given price.

3.1.2 Market Participants

Like in markets for other commodities, there are two general categories of actors active in the wholesale electricity markets: fundamental participants and speculative players:

• Fundamental market participants are active to value and optimize physical assets in the market. They carry their “buyer” or “seller” positions until delivery, based on their specific portfolio of assets, be it consumption, generation or both;

• Speculative market players do not have a fundamental need to buy or sell electricity. They participate in the market in hope of making a profit from buying low and selling high. Their activity has a zero-sum volume effect on the market as they do not carry positions to delivery.

In practice, the fundamental market participants are either electricity generators, who trade and sell the output from their power plants, or electricity retailers who trade and source electricity to sell it to their end-consumers. For companies that own generation assets and sell directly to end-consumers, part of the electricity injected into the network is not directly traded in the markets but delivered directly to end-consumers. A utility that produces more energy than its customers’ needs can sell the excess power on the wholesale market (net seller). Symmetrically, a retailer that doesn’t produce enough energy to cover the needs of its customers can buy it from the wholesale market (net buyer). In addition to these traditional actors, a new type of fundamental participant has been emerging over the last decade: aggregators for Demand Side Management (DSM) or small-scale renewables. They act on behalf of a group of producers and/or consumers, aggregating assets they can steer and market at the wholesale level. In Europe, Transmission System Operators are also active participants on the wholesale market although their activity is regulated. They intervene on the spot markets to compensate the transmission system’s grid losses.⁸ In Germany, they are also in charge of marketing green electricity subsidized under the feed-in tariff regulatory scheme.

Trading companies, hedge funds and banks have also entered wholesale electricity markets since the first days of liberalization. They usually perform speculative trading or trade for the account of customers. These financial participants take positions on either the long-term derivatives market or on the spot market and provide market access services to other counterparties (i.e. hedge funds).

3.1.3 Trading Venues

Electricity can be traded on “organized markets” (managed by power exchanges) or “Over-The-Counter” (OTC) bilaterally or through intermediaries called brokers. Power exchanges run auctions and continuous double-sided markets. Brokers usually offer phone and continuous screen trading coverage to their customers.

OTC transactions are bilateral, non-anonymous transactions between a buyer and a seller with the counterparty risk⁹ managed bilaterally between them, even if a broker is involved. On their end, power exchanges give access to anonymous markets creating a level playing field between all exchange members. This is possible as the counterparty risk is centralized by a clearing house that guarantees the fulfilment of all financial obligations of the trading participants through a daily settlement of profits and losses and a margining and collateralization system. OTC transactions can be recorded for clearing at power exchanges as a way of eliminating counterparty risk.

In both the US and EU, dedicated large commodity exchanges, such as European Energy Exchange (EEX), Intercontinental Exchange (ICE), Chicago Mercantile Exchange (CME) or Nasdaq operate power derivatives exchanges. In Europe, a model of exchange alliance has emerged in recent years. Established stock exchanges acquire majority (together with minority TSO shareholders) and integrate power spot and/or derivatives with their commodity businesses. Examples are Nasdaq OMX Commodities, ICE and Endex, EPEX SPOT and EEX, IDEX and London Stock Exchange (LSE), Nord Pool and Euronext.

3.1.4 Liquidity

Liquidity is a desirable characteristic of a competitive market. It can be defined as the ability to transact quickly with little price impact. Liquidity is materialized by a high level of trading activity and a high number of active market participants. It can be measured by price resiliency for an auction and bid-ask spread and market depth for a continuous market:

• As a measure of overall market activity, the Churn ratio is the ratio between domestic consumption of electricity (considered an indicator of fundamental trading needs) and the volumes traded on the wholesale market. In Germany, the largest EU power market for spot and derivatives, the Churn reached 12 times the total consumption (European Commission 2017).

• For an auction, price resiliency can be defined as the sensitivity of the market clearing price to the submission of a price-independent bid of 500 and 1000 MW for a given delivery hour on either the buy or sell side.

• For a continuous market, the bid-ask spread is the spread between the best buy and best sell prices in the order book. The lower the bid-ask spread, the higher the chances are the prices on which buyers and sellers agree reflect the fair value of the good.

Liquidity has traditionally been greater in less concentrated markets with high number of participants.

A trading member can act as a market maker—or liquidity provider. The aim of market makers’ service is to provide liquidity to a continuous market. Market makers in power exchanges provide liquidity for a given product by standing ready to purchase or sell a given amount of power, for instance by providing a continuous bid-ask spread. The specific price range for market makers’ orders is contractually set in advance with the power exchange or broker. Market makers earn the bid-ask spread (when their buy and sell transactions compensate each other) but they can also benefit from fee rebates when they fulfil their bidding requirements based on the size of the spread and its duration.

3.1.5 The Trading Sequence

As first described in Sect. 2.2, electricity markets cover several timeframes, ranging from years ahead of delivery until real time. Market participants use longer term derivatives markets to hedge sales or purchases and manage their electricity price risk. The short-term spot market lets participants schedule their assets close to real time and manage their volume risk (i.e. forecast errors), as described in Fig. 22.8.

Fig. 22.8

Trading sequence from long term to real time. (Source: Authors’ elaboration)

The following sections explain in more details how the derivatives and spot markets work, how they interact and what the listed products are.

3.2 Power Derivatives

Power derivatives correspond to traded contracts that are indexed on an underlying price of electricity, most often the short-term spot price. Most common exchange-based derivatives are futures and options, competing with OTC-traded forwards, options and swaps.

3.2.1 Hedging, Sourcing and Arbitraging

Fundamental market participants are exposed to price variations in different ways:

• Retailers most often offer fixed-price contracts to their clients and their margins are exposed to electricity price variations;

• Generators’ margins directly depend on the price at which they sell electricity and they are exposed to market price variations.

The basic idea behind a hedge is the limitation of the price risk associated with the electricity price variations. Hedging allows consumers, retailers and generators to take a known fixed price now rather than to accept the risk of this price changing. Long-term derivatives markets allow this risk to be shared among market participants through transactions over derivative contracts (e.g. Futures, Forwards, Options, Swaps).

In long-term derivative markets, market players trade for the future supply or demand of electricity for long delivery periods such as weeks, months, quarters or years, at a price negotiated on the contract date. To make trading easier and reinforce liquidity, these derivative contracts apply to standardized products, for example, the supply of 1 MW of baseload electricity (constant power during all hours of the delivery period) or peak load electricity (between 8 am and 8 pm, Monday to Friday during the delivery period). Financial Futures contracts are cash-settled against the spot price, and therefore represent the average of the expected spot prices over a longer period. They are generally used as a basis for determining the prices paid by end-consumers. When retailers enter into contracts with customers, they generally purchase the derivatives products required to cover most of the electricity they will need to supply. As the delivery time approaches the remaining variations around the forecast and finer granularity variations (hourly and sub-hourly) are handled in the day-ahead and intraday timeframes in Europe and in day-ahead and real-time markets in the US.

The very limited storability of electricity explains the lack of a well-defined relationship between spot and long-term power prices. According to the storage theory (Kaldor 1939), companies trading commodities keep stocks to respond to unanticipated demand variations. This exposes them to storage costs but makes possible the selling of retained stocks later when the commodity is valued more (the convenience yield). The non-storability of electricity limits the standard no-arbitrage approach in modelling electricity futures prices. The relationship between spot and futures electricity prices “only” reflects expectations about future supply and demand characteristics for electricity (that determine the spot price) and risk aversion (Shawky et al. 2003).

Allowing participants the opportunity to hedge against locational price differences is an important aspect of a power market. Long-term transmission rights such as Physical and Financial Transmission Rights (PTRs and FTRs) enable market participants to cover the risks of changing conditions between the contracting and delivery of contracts and to hedge short-term price differentials between two bidding zones (EU) or nodes (US). PTRs entitle their holder to physically transfer a certain volume of electricity in a certain period between two zones in a specific direction. FTRs are a mechanism for market participants to hedge against the volatility of transmission congestion between two points on the network. In the US only FTRs are used for the nodal markets. There are long term, yearly and monthly auctions for FTRs organized by ISOs. In Europe, depending on the borders, both Physical Transmission Rights and Financial Transmission Rights are used.

3.2.2 Power Derivatives in Europe and the US

Electricity is traded in Europe and the US on the “curve” several years ahead of delivery through either OTC (bilaterally or through inter-dealer brokers) or exchange-based¹⁰ until the day-ahead of delivery when the “physical” market starts. Traded futures are financially cash-settled against a reference price of the underlying asset (the daily spot settlement price), but in Europe physical futures can also be traded and give rise to a delivery of power (i.e. schedule to the relevant TSO). In the EU, Futures with maturities of up to 10 years can be found on the most liquid hubs but they are less liquid. However, most of the liquidity is concentrated on the next three years ahead of delivery, next three months, next three quarters.

In Europe, Nord Pool, the Nordic Power exchange was the first power market for spot and derivatives in the Scandinavian countries in 1993, followed by EEX and all major stock exchanges (e.g. ICE, Nasdaq, CME). Two standardized products are traded on Futures and Options: quotation is made with a tick size of 0.01 €/MWh and a minimum size of 1 MW. The nominal of the contract is expressed in megawatt-hours. The futures price is denominated in Eur/MWh, and the contract is financially settled against the average hourly spot price (base and peak load contracts). The daily settlement price is used as a reference for the clearing house¹¹ to value on a daily basis a position and to close a position in case of a defaulting buyer or seller. European power options are financially settled on the futures index with monthly, quarterly and yearly delivery periods.

In the US, Futures (Forwards and Swaps on the OTC) contracts listed at exchanges have also been created to cover specific geographic regions or hubs (electricity products can be traded at several dozen hubs and delivery points in North America). After the COB (California Oregon Border) and PV (Palo Verde, Arizona) contracts introduced in 1996, the NYMEX allowed trading the Cinergy contract (covering the Midwestern region), Entergy contract (Louisiana region) and PJM contract, whose delivery point is the border intersect of Pennsylvania, New Jersey and Maryland (Eydeland and Wolyniec 2003). Major hubs have developed around the Regional Transmission Operator (RTO) markets:¹² ISO New England (ISO-NE), New York ISO (NYISO), PJM Interconnection (PJM), Midwest ISO (MISO), Electric Reliability Council of Texas (ERCOT), two locations in the California ISO (CAISO), Louisiana (into Entergy), Southwest (Palo Verde) and Northwest (Mid-Columbia) (Table 22.2).

Table 22.2 Contract examples for US and EU

With the expansion of the Nodal Pricing implemented in most competitive power markets states, Nodal Futures can be traded allowing to decrease basis risk management with futures contracts traded at the Nodal Exchange. Nodal futures are financially settled using the monthly average of the appropriate hourly Locational Marginal Prices (LMPs) for the location(s) specified in the contract as published by the organized power markets, which are overseen by the Federal Energy Regulatory Commission (FERC) (Fig. 22.9).

Fig. 22.9

Selected price hub for wholesale electricity and natural gas reported by Intercontinental Exchange. (Source: US Energy Information Administration)

Although for derivatives both EU and US share similar arrangements, in the case of spot markets there are major differences.

3.3 Spot Electricity Markets

Market structures in both continents differ by the nature and role of the stakeholders. Day-ahead markets are operated in the US by Independent Systems Operators (e.g. PJM, MISO, ERCOT, etc.) which are non-profit federally regulated organizations, while such markets are organized in the European Union (EU) by power exchanges which are for-profit companies that are designated as Nominated Electricity Market Operators (NEMO) in the European legislation.¹³ In both the US and the EU the main physical market is the auction that takes place the day-ahead of delivery for all the hourly delivery periods of the next day. Such a “physical dispatch” market on the day-ahead of delivery is necessary considering that some power plants have long ramp-rates.

3.3.1 The Day-Ahead Spot Market

Power Exchanges can be either based on the “pool” or “exchange” models. Most European countries have adopted an exchange model with bilateral contracts and a voluntary electricity trading (self-scheduling model) and the pools running centralized dispatch with often some mandatory features. A power pool is often the result of a public initiative, that is, a government wants to implement competition at the wholesale level, and participation is mandatory, that is, no trade is allowed outside the pool.¹⁴ Currently in Europe a semi-mandatory solution has been applied in Iberian OMIE and Italian GME where bilateral deals are possible but need to be registered through the pool. In the US, for the states that have moved to competitive power markets, spot market operation activities are performed by non-profit Independent System Operators through semi-pool type arrangements operating a central dispatch. Power plants may have obligations to bid in the pool or all trades need to register though the pool. The most important characteristic of power pools is that they consider many technical characteristics, like the availability of plants and unit commitment parameters.¹⁵

In Europe, the day-ahead market is a single auction for all countries¹⁶ and all 24 hours of the next day’s delivery. The auction is run at noon 7 days a week, year-round. The auction is a double-sided sealed-bid uniform-price auction where all buyers and sellers make offers that are not visible to the other market participants and pay/receive the same Market Clearing Price (MCP) respectively. All cross-border interconnectors are considered in the market clearing algorithm through a process called “market coupling” that implicitly allocates the interconnection capacity between bidding zones together with the energy and optimizes its usage. If enough transmission capacity is available, then a common market clearing price is determined. If transmission capacity is saturated, separate clearing prices are determined across the border. Market participants send their orders to their respective power exchange. All orders are collected and submitted to the market coupling algorithm that decides which orders are to be executed and which orders are to be rejected such that the social welfare¹⁷ generated is maximal and the power does not exceed the capacity of the relevant network elements.

In the US, ISOs run a nodal day-ahead auction taking into account a full topology of the transmission grid. A second auction is performed for the real-time market with the same grid topology but updated bids.¹⁸ In the Day-ahead Market hourly clearing prices are calculated for each hour of the next operating day based on generation offers, demand bids at the node level. Moreover, there is a simultaneous clearing of energy and reserves (co-optimization). Market participants bid technical/cost data by unit (unit-bidding) and the ISO solves a co-optimization based on market participants bids and bilateral transaction schedules submitted in the Day-ahead Market.

In both the US and EU, price caps have been set on the energy market prices for technical reasons but also to avoid extreme prices that would result from the abuse of market power. Price-limits are set quite arbitrarily today in the 000 EUR/MWh or 000 $/MWh level, due to the difficulty to define a market-wide Value of Loss of Load and the difficulty of consumers to express their real willingness to pay. Across Europe there are single day-ahead harmonized price caps at (−500; +3000 €/MWh) (EPEX Spot 2020) with an obligation set by the authorities to increase the price-cap every time it is reached (ACER 2017). Figure 22.10 shows an example of supply and demand aggregated curves.

Fig. 22.10

Aggregated curves, Austria 2/06/2020, hour 19–20. (Source: Authors’ elaboration on EPEX Spot data)

As an example of a US-based nodal design, the PJM market offers are capped at 2000 $/MWh and need to justify cost-basis but during scarcity conditions the price can rise to 3700 $/MWh (PJM 2018). Usually, offer caps on units are imposed when the local market structure is non-competitive. Offer capping is a means of addressing local market power. The market rules require that offers in the energy market be competitive, where competitive is defined to be the short-run marginal cost of the units. The short-run marginal cost can and should also reflect opportunity costs.

Because generators face non-convex cost functions due to technical constraints such as startup costs, minimum up and down times, ramp rates (depicted in Fig. 22.11), in Europe, the market coupling algorithm allows for “block orders” of a given amount of electric energy in multiple consecutive hours, as an addition to simpler hourly orders.

Fig. 22.11

Technical constraints of a thermal power plant. (Source: Authors’ elaboration)

Block orders “link” several hours and allow a better modelling and optimization of power plants in the day-ahead auction. The uniform price auction rule means that the same price applies to all and there are no side-payments (make-whole payments) linked with non-uniform pricing rules.

In the US, producers typically use three-part bids specifying start-up costs, no-load costs and marginal costs (Sioshansi et al. 2009). Centralized markets have a non-linear pricing scheme with make-whole or uplift payments to ensure that a unit does not make a loss. Uplift payments are made to market participants for operating a unit under specific conditions as directed by ISOs to ensure that they recover their total offered costs when market revenues are insufficient or when their dispatch instructions diverge from their dispatch schedule.

Spot power markets are very computation-intensive and hard to scale up, especially if they include complex network topologies or complex bid structures (Ahlqvist et al. 2019). This is true for the US with nodal pricing and in the case of Europe for the Day-ahead algorithm complexity with the integration of all borders, national requirements (i.e. Italian single national price Prezzo Unico Nazionale (PUN)) and block order complexity in the market coupling algorithm. This is a potential problem as the global trend is to increase the geographical size of electricity markets, to introduce finer granularity products and integrate millions of assets, including storage which creates dynamic time dependencies and high algorithmic complexity.

3.3.2 The Intraday and Real-Time Markets

As day-ahead auctions are based on a prediction of the next day’s required load or generation, the actual demand or supply for power is not known when the auctions are run. Intraday markets are the last opportunity for market participants to adapt their offers and assets before real time. These variations can occur for several reasons, but traditionally the intraday and real-time markets have been used to balance volume risk as a result of:

• Forced outages of generation units;

• Forecast errors of demand. A drop in the temperature or a rise in cloud coverage might require additional generation resources to meet load in real time;

• Forecast errors of intermittent Renewable Energy Sources (RES) such as wind and solar.

In the US a real-time market is used to correct deviations very close to real time. In the Real-Time Market the product is procured for immediate delivery. The locational marginal prices are calculated for every five-minute step on the actual system operations security-constrained economic dispatch. The real-time market acts as a balancing market where day-ahead commitments are balanced against actual demand and system constraints. The generation offers are updated and used to make real-time dispatching decisions. A higher amount of price volatility can occur in the real-time market as dispatching is adjusted to the real-time system load and outages. When the two-auction settlement system is performing well and the day-ahead forecasts were accurate, the real-time price will clear similar to the day-ahead. Virtual bids can be placed in both markets (in opposite directions) to arbitrage the price differences between the day-ahead and real-time markets (Jha and Wolak 2016).

In the EU, the aforementioned forecast errors can be rebalanced on the cross-border continuous intraday 24 hours a day, 7 days a week, year-round. From 3 pm on the day ahead of delivery until 5 minutes before delivery with a gradual opening of 15, 30 and 60-minute granularity products. In 2015, an additional uniform-price auction for 15-minute time slots was introduced in Germany at the beginning of the intraday trading session at 3 pm to help the market participants market their solar ramps (Fig. 22.12).

Fig. 22.12

The “Spot” trading process. (Source: Authors’ elaboration)

The continuous trading implements a pay-as-bid continuous matching algorithm which implies that market participants must anticipate the market price. Figure 22.13 shows an example of the evolution of the bid-ask spread and market depth during a trading session.

Fig. 22.13

Bid-ask spread and market depth of the continuous intraday market. (Source: Authors’ elaboration on EPEX Spot data)

Since their introduction in 2007, the intraday market volumes have increased a lot reflecting the higher needs to re-balance supply and demand between day-ahead and real-time as a result of ever-growing renewable capacities of wind and solar. Figure 22.14 shows the evolution of intraday volumes in Germany for the period 2010–2018 during which they have been multiplied by ten.

Fig. 22.14

Continuous intraday volumes in Germany 2010 to 2018. (Source: Authors’ elaboration on EPEX spot data)

Other trends that have been observed following the integration of massive amounts of renewables are trading in more granular products and closer to real time:

• The finer granularity products allow market participants to match generation and demand for each 15-minute time step to satisfy their balancing obligations. They represent roughly 20% of the total traded volumes of the intraday continuous market;

• The trading activity in the last 30 minutes before real-time has increased over the last years as participants benefit from trading opportunities until the last minutes. On the German Intraday 15% of intraday continuous volumes are traded in the last 30 minutes before real time (Fig. 22.15).

• Automated trading applications are developed either in-house or by Independent Software Vendors (ISVs) and automate power trading on the basis of algorithms. The applications are connected 24/7 through APIs (Automated Programming Interfaces). This enables market participants to react quickly to fluctuations in power production and demand.

Fig. 22.15

Lead time to delivery in minutes in the intraday market (% of yearly volumes). (Source: Authors’ elaboration on EPEX Spot data)

4 Looking Ahead, New Challenges for the Power Markets

Electricity systems around the world have been undergoing nascent but profound changes in recent years, that are expected to further progress in the years to come. These intertwined trends are sometimes referred to as the 3 Ds: Decarbonation, Decentralization and Digitalization.

Global awareness around climate change makes the decarbonation of the electricity sector one of the important stakes to curb global warming. Along emerging carbon pricing initiatives creating an economic signal for CO₂ emissions (by “internalizing” their negative externalities), many governments and policymakers have implemented renewable energy sources (RES) support schemes and subsidies to promote clean energy sources. As a result, there has been a strong development of solar and wind RES worldwide. In Germany alone, a pioneering country in this field, there is more than 110 GW of wind and solar capacity installed with more than 36% of domestic electricity consumption covered by RES in 2016 (BMWi 2017) (from only 6% in 2000). In California, the famous “duck” curve illustrates the gradual penetration of solar PV in the market (CALISO 2012) (Fig. 22.16).

Fig. 22.16

Duck-shaped curve of load at the California ISO. (Source: CAL ISO)

This RES development trend is expected to continue, as illustrated by ambitious policy objectives. The EU aims to be climate-neutral by 2050 (EU Commission 2019). Such a shift in the electricity generation mix, towards more renewables, mostly intermittent electricity production, induces new challenges for cost-efficiency, resource adequacy and security of supply:

• Increased missing money: RES having an SRMC close to 0 €/MWh (as there are no fuel and emission costs, and little O&M variable costs), they have a bearish effect on wholesale spot electricity prices¹⁹ and tend to increase the “missing money” problem (first introduced in Sect. 2.3). Indeed, they cannot fully replace dispatchable generation for system adequacy, as the need for available dispatchable generation remains high to cover peak demand events with no wind nor sun;

• Need for flexibility: furthermore, with rising RES penetration, resource adequacy and system reliability do not only depend on peak demand anymore. Production flexibility is also increasingly needed to compensate for large and short-term RES-induced production variations. Capacity mechanisms can contribute to solving the intermittency backup problem although their primary purpose is not to increase flexibility. Efficient measures, in market design and regulatory fields, will be needed to further enhance flexibility incentives in the market. Paradoxically, to further develop RES going forward, there is a need for flexibility that can currently mainly come from fossil fuels (e.g. flexible gas power plants), as demand-response and batteries remain respectively not fully exploited or too expensive on a large scale, but could emerge as a result of decentralization and digitalization trends.

Decentralization corresponds to the growing development of smaller scale assets (RES, storage, demand-response) in the distribution grids, slowly shifting the traditional paradigm of the electricity sector from a centralized electricity supply from large power plants to a more distributed supply. In this context, digitalization acts as a catalyst with the deployment of smart metering, energy management systems and new communication technologies, paving the way for a more precise, data-intensive and coordinated power system management and enabling the development of smaller scale flexibility. New opportunities can emerge for consumers, suppliers and aggregators to adapt their load or production profiles, provided that the right price signals are in place to foster their development. It will be needed to combine the largest number of players in the market with a better ability to react to prices to manage the electrical system at a lower cost.

Going forward, it seems essential to identify the future needs of the power system and align them with global policy objectives in order to adapt and enhance the way electricity markets generate social welfare. It is a continuous process that is becoming increasingly complex: how should the market architecture and regulatory framework evolve to meet ambitious climate targets while maintaining efficient investment incentives and security of supply? The answer probably lies in more temporal and spatial market granularity, the emergence of the right price signals and incentives along with the proper integration of new opportunities stemming from technological breakthroughs.