2012 23rd International Workshop on Database and Expert Sytems Applications

Negotiating Bilateral Contracts

in a Multi-Agent Electricity Market: A Case Study

Fernando Lopes Tiago Rodrigues, Jorge Sousa, Member, IEEE

LNEG−National Research Institute Cie3, ISEL: Lisbon Engineering Superior Institute
Estrada do Paço do Lumiar 22 Rua Conselheiro Emı́dio Navarro 1
1649-038 Lisbon, Portugal 1950-062 Lisbon, Portugal
fernando.lopes@lneg.pt jsousa@deea.isel.ipl.pt

Abstract—Electricity markets are systems for effecting the Opening up electrical energy production to competition is
purchase and sale of electricity using supply and demand an important tool to improve the efficiency of the electricity
to set energy prices. Three major market models are often industry and therefore to benefit all customers. However,
distinguished: pools, bilateral contracts, and hybrid models.
This article addresses the challenge of using software agents
the analysis of important EMs yields the main observation
with negotiation competence to help manage the complexity that they are still far from liberalized. In particular, tariffs
of electricity markets, particularly the issues associated with do not reflect the pressure of competition. Clearly, there is
the negotiation of bilateral contracts. Specifically, it presents a pressing need for both modeling approaches simulating
a multi-agent energy market and describes a case study on how market participants react to economic and regulatory
forward bilateral contracts.
changes, and energy management tools supporting key mar-
Keywords-electricity markets; bilateral contracts; intelligent ket activities. Against this background, an ongoing study is
software agents; automated negotiation; looking at using software agents with negotiation compe-
I. I NTRODUCTION tence to help manage the complexity of EMs. It intends to
go a step forward in the development of EM simulators, by
Electricity markets (EMs) are systems for effecting the
developing a tool able to simulate the negotiation of bilateral
purchase and sale of electricity using supply and demand to
contracts in a more realistic way.
set energy prices. EMs differ from their more traditional
counterparts because energy is difficult to store or keep In particular, this paper presents a multi-agent energy
in stock—unlike other products, it is not efficient to keep market composed of a collection of autonomous agents,
energy in stock. Market participants must generate energy each responsible for one or more market functions, and
on-demand and, as a result, work with consumption prog- each interacting with other agents in the execution of their
noses. This limitation creates a number of risks, notably responsibilities. It also describes a case study on forward
price risk (related to a high price volatility at times of peak bilateral contracts—a retailer agent (a seller) and an in-
demand and supply shortages) and volume risk (associated dustrial customer agent (a buyer) negotiate a three-rate
with uncertainty in consumption or production). DSM-tariff, i.e., negotiation involves three major issues:
Three major market models are often distinguished [12]: price#1 (for peak-load period), price#2 (for medium-load
pools, bilateral contracts, and hybrid models. A pool is a period), and price#3 (for off-peak period). The two agents
market place where electricity-generating companies sub- have a choice between three different negotiation strategies
mit production bids and corresponding market-prices, and involving a relatively consistent, coherent effort to move
consumer companies submit consumption bids. A market toward agreement. Accordingly, we will take up all the
operator uses a market-clearing tool, typically a standard possible pairs of strategies, one at a time, and examine their
uniform auction, to set market prices. Bilateral contracts impact on the negotiation outcome.
are negotiable agreements on delivery and receipt of power
The remainder of the paper is structured as follows.
between two traders. These contracts are frequently used
Section II describes a multi-agent electricity market, placing
to hedge against pool price volatility. The hybrid model
emphasis on both the individual and the social behavior
combines several features of pools and bilateral contracts.
of autonomous software agents. Section III presents a case
This work was performed under the project MAN-REM: Multi-agent study and illustrates how agents equipped with a negotiation
Negotiation and Risk Management in Electricity Markets (FCOMP-01- framework operate in a simplified retail market. Finally,
0124-FEDER-020397), and supported by both FEDER and National funds
through the program “COMPETE−Programa Operacional Temático Fac- related work and concluding remarks are presented in sec-
tores de Competividade”. tions IV and V respectively.

1529-4188/12 $26.00 © 2012 IEEE 326

DOI 10.1109/DEXA.2012.77

Authorized licensed use limited to: INTERNATIONAL ISLAMIC UNIVERSITY. Downloaded on July 30,2020 at 09:20:20 UTC from IEEE Xplore. Restrictions apply.
 Figure 1. Multi-agent electricity market involving a wholesale market and a retail market.

II. M ULTI -AGENT E LECTRICITY M ARKET In the near future, we intend to work on the following
types of agents:
The electrical power industry, which has long been dom-
inated by vertically integrated utilities, has evolved into a • system operator: maintains the system security, admin-

distributed and competitive industry in which market forces isters transmission tariffs, and coordinates maintenance
drive electricity prices and reduce the net costs through scheduling; it analyzes the technical feasibility of all
increased competition. Liberalization has led to the estab- negotiated contracts;
lishment of a wholesale market for electricity generation and • market operator: regulates pool negotiations, and thus,

a retail market for electricity retailing. Practically speaking, is present only in a pool or hybrid market; it uses
the role of the wholesale market is to allow trading between a market-clearing tool, typically a standard uniform
generators and retailers—competing generators can offer auction, to set market prices.
their electricity output to retailers. The role of the retail • virtual power players (VPPs): responsible for managing

market is to allow trading between energy consumers and coalitions of producers, including the role of negotiat-
retailers—end-use customers can choose their supplier from ing on behalf of such coalitions;
competing electricity retailers. • traders: promote liberalization and competition, and

Multi-agent systems (MAS) are essentially loosely cou- simplify dealings either between sellers/buyers and the
pled networks of software agents that interact to solve market operator or between sellers and buyers.
problems that are beyond the individual capabilities of each The agents are computer systems capable of flexible,
agent. MAS represent a relatively new and rapidly expand- autonomous action and able to communicate, when appropri-
ing area of research and development. Agent technology ate, with other agents to meet their design objectives. They
has been used to solve real-world problems in a range are currently being developed using the JAVA programming
of industrial and commercial applications (see, e.g., [9]). language and the JADE platform [1]. We chose this platform
Conceptually, a multi-agent approach in which autonomous for two main reasons:
agents are capable of flexible action in order to meet their 1) it is an agent-oriented platform offering a framework
design objectives is an ideal fit to the naturally distributed for multi-agent system development; this framework
domain of a deregulated energy market. can support different agent models;
This work focuses on bilateral contracts, either physical 2) it is built on top of and fully integrated with the Java
or financial, between a buyer and a seller pair. Accordingly, programming language; it includes all components of
we are currently working on the following types of agents: Java and it offers specific extensions to implement
• generator or producer agents: represent utility compa- agents’ behaviours.
nies and operate in a wholesale market; The agents are implemented as Jade agents, i.e. they inherit
• retailer or supplier agents: represent electricity retailers from the basic “Agent” class. Agent communication is done
and operate in both a wholesale and a retail market; by sending and receiving messages. Figure 1 shows the
• customer or consumer agents: represent end-use cus- different agents operating in a deregulated electricity market
tomers and operate in a retail market. and the main interactions between them.

327

Authorized licensed use limited to: INTERNATIONAL ISLAMIC UNIVERSITY. Downloaded on July 30,2020 at 09:20:20 UTC from IEEE Xplore. Restrictions apply.
 A. Autonomous Software Agents 1) Pre-Negotiation: Effective preparation and planning
The agents are equipped with a generic model of individ- involves mainly the creation of a well-laid plan specifying
ual behavior [4]. Specifically, each agent has the following the activities that negotiators should attend to before actually
key features: starting to negotiate. These activities include:
A-1 A set of beliefs representing information about the • prioritizing the issues;

agent itself and the market. Beliefs are formulae • defining the limits and targets;

of some logical language (the precise nature of the • selecting an appropriate protocol;

language is not relevant to our model). • specifying the preferences.

A-2 A set of goals representing world states to be achieved. Let A = {a1 , a2 } be the set of autonomous agents (ne-
Goals are also formulae of some logical language. gotiating parties). Let I = {x1 , . . . , xn } be the negotiating
A-3 A library of plan templates representing simple agenda—the set of issues to be deliberated during negotia-
procedures for achieving goals. A plan template pt tion. Let D = {D1 , . . . , Dn } be the set of issue domains. For
has an header and a body. The header defines a name each issue xk , the range of acceptable values is represented
for pt. The body specifies either the decomposition of by the interval Dk = [mink , maxk ].
a goal into more detailed subgoals or some numerical Prioritization involves deciding which issues are most
computation. important and which are least important. Target setting in-
A-4 A set of plans for execution, either immediately or volves defining two key points for each negotiation issue: the
in the near future. A plan is a collection of plan resistance point or limit—the point where every negotiator
templates structured into a hierarchical and temporally decides to stop the negotiation rather than to continue,
constrained And-tree. The nodes of the tree are instan- because any settlement beyond this point is not minimally
tiated plan templates retrieved from the library. The acceptable, and the target point or level of aspiration—the
header of each instantiated plan template is referred point where every negotiator realistically expects to achieve
to as intention. a settlement.
The generation of a plan p from the simpler plan tem- The negotiation protocol is an alternating offers protocol
plates stored in the library is performed through an iterative [8]. Two agents bargain over the division of the surplus of
procedure involving four main tasks: (i) plan retrieval, n ≥ 2 issues (goods or “pies”) by alternately proposing offers
(ii) plan selection, (iii) plan addition, and (iv) plan inter- at times in T = {1, 2, . . .}. This means that one offer is made
pretation. In brief, plan retrieval consists of searching the per time period t ∈ T , with an agent offering in odd periods
library for any plan template whose header unifies with the and the other agent offering in even periods. A proposal is
description of a goal. Plan selection consists of selecting a vector (v1 , . . . , vn ) specifying a division of all the goods.
the preferred plan template (from the set of retrieved plan Once an agreement is reached, the agreed-upon allocations
templates). Plan addition consists of adding the preferred of the goods are implemented.
plan template to p. Plan interpretation consists of selecting The agents have continuous utility functions. Specifically,
a composite plan template from p, establishing a temporal we consider that each agent ai ∈ A has a utility function
order for the elements of its body, and picking the first Ui : {D1×. . .×Dn } ∪ {Opt, Disagreement} → . R The
ordered element (which is interpreted as a new goal). outcome Opt is interpreted as one of the agents opting
out of the negotiation in a given period of time. Perpetual
B. Computational Negotiation disagreement is denoted by Disagreement. The players’
The agents are equipped with a negotiation framework preferences are modelled by the well-known additive
that handles two-party and multi-issue negotiation (see also model—the parties determine weights for the issues at
[4], [5] and [6]). The framework enables them to: stake, assign scores to the different levels on each issue,
1) enter into hedge contracts to protect themselves from and take a weighted sum of them to get an entire offer
financial risks, notably high price volatility; evaluation [11].
2) negotiate various issues at the table, particularly dif- 2) Actual Negotiation: The heart of negotiation is the ex-
ferent rates for different periods of a day (e.g., a three- change of offers and counter-offers—the nature, timing, and
rate tariff or even an hour-wise tariff); pattern of offers, and the concessions they elicit, constitute
3) exhibit strategic behaviour, with the objective of dis- the very essence of negotiation. Negotiation is iterative—
tributing demand over time. negotiators’ offers usually contain changes in position and
This section presents the key features of the framework, suggest alterations to the opponent’s position. There is
focusing on the operational and strategic process of prepar- an unstated assumption that the parties will show their
ing and planning for negotiation (usually referred to as commitment to the process of finding a solution by making
pre-negotiation), and the central process of moving toward concessions, and not simply by rejecting the offers of the
agreement (usually referred to as actual negotiation). others out of hand.

328

Authorized licensed use limited to: INTERNATIONAL ISLAMIC UNIVERSITY. Downloaded on July 30,2020 at 09:20:20 UTC from IEEE Xplore. Restrictions apply.
 Table I
I NITIAL OFFERS AND PRICE LIMITS FOR THE NEGOTIATING PARTIES

Agent Time Period Price (e/MWh) Limit (e/MWh) Energy (MWh)

Customer 1 51.30 55.62 12.32

2 44.27 48.00 27.00
3 25.08 27.19 03.90
Retailer 1 56.70 51.84 —–
2 48.93 44.74 —–
3 27.72 25.34 —–

Concession making involves reducing negotiators’ de- The 24 hours of every day are divided into three periods
mands or aspirations to (partially) accommodate the oppo- as follows:
nent. A reduction in demands usually involves a reduction • period 1 (peak-load): 10:30–13:00h and 19:30–21:00h;
in the level of benefit underlying these demands, which is • period 2 (medium-load): 8:00–10:30h, 13:00–19:30h
called a concession. Specifically, a concession is a change of and 21:00–22:00h;
offer in the supposed direction of the other party’s interests • period 3 (off-peak): 22:00–8:00h.
that reduces the level of benefit sought [10]. Concessions
David Colburn sets the hourly rates in accordance with
ordinarily result from the belief that they will hasten agree-
the global demand, i.e., more expensive in periods where
ment, prevent the other party from leaving the negotiation,
the demand is higher, cheaper when it is lower. Accord-
or encourage the other to make reciprocal concessions.
ingly, this agent proposes the following rating scheme:
The negotiation protocol marks branching points at which
period 1 (56.70e/MWh), period 2 (48.93e/MWh), and off-
agents have to make decisions according to their strategies.
peak period (27.72e/MWh). This scheme can be seen as a
Clearly, negotiation strategies can reflect a variety of be-
communication tool between Colburn and Tom Britton. On
haviours and lead to strikingly different outcomes. In this
the one hand, Colburn “advises” Britton when to place con-
work, we consider the following three groups of concessions
sumption. From the point of view of Britton, the increment
strategies:
of the price at certain hours constitutes an incentive to move
1) Starting reasonable and conceding moderately consumption into cheaper hours.
(SRCM): negotiators adopt a realistic opening Table I shows the initial offers and the price limits for
position and make substantial concessions during the the two agents. Negotiation involves an iterative exchange
course of negotiation; of offers and counter-offers. We consider the following:
2) Low-priority concession making (LPCM): involves
• the first agent to submit a proposal is the retailer agent;
changes of proposals in which larger concessions are
• the agents are allowed to propose only strictly
made on low-priority than on high-priority issues;
monotonically—the customer’s offers increase mono-
3) Energy dependent concession making (EDCM): nego-
tonically and the retailer’s offers decrease monotoni-
tiators concede strategically throughout negotiation, by
cally;
considering the amount or quantity of energy traded
• the acceptability of a proposal is determined by a ne-
in each period of the day.
gotiation threshold—an agent ai ∈ A accepts a proposal
The formal definitions of these and other relevant concession pt−1
j→i , submitted by aj ∈ A at period t−1, when the
strategies appear elsewhere [7]. difference between the benefit provided by the proposal
pti→j that ai is ready to send in the next time period t
III. A C ASE S TUDY ON B ILATERAL C ONTRACTS is lower than or equal to the negotiation threshold;
Our case study illustrates how software agents negotiate • the agents are allowed to exchange only a maximum
forward bilateral contracts in the multi-agent electricity mar- number of proposals, denoted by maxp —failure to
ket. The two negotiating parties are David Colburn, CEO of reach agreement after maxp proposals results in a
N2K Power—a retailer agent—and Tom Britton, executive at deadlock.
SCO Corporation—a customer agent. The agents negotiate a The negotiation strategies are the three concession strategies
three-rate DSM-tariff, i.e., negotiation involves three major presented earlier: SRCM, LPCM and EDCM. These strate-
issues: price#1 (for peak-load period), price#2 (for medium- gies involve a relatively consistent, coherent effort to move
load period), and price#3 (for off-peak period). toward agreement and the agents can pursue any of them.

329

Authorized licensed use limited to: INTERNATIONAL ISLAMIC UNIVERSITY. Downloaded on July 30,2020 at 09:20:20 UTC from IEEE Xplore. Restrictions apply.
 Table II
B ENEFIT OF AGENTS IN THE FINAL AGREEMENT: (C USTOMER ; R ETAILER ) PAIRS

SRCM LPCM EDCM

SRCM (39.14; 47.28) (38.99; 47.25) (38.61; —)

LPCM (39.36; 39.26) (39.29; 47.25) (38.68; —)

EDCM (39.64; 47.51) (39.64; 47.51) (39.33; —)

Now, Colburn operates in a fine zone between profit and V. C ONCLUSION

loss. In particular, if the price to Britton is too high then This article has presented a multi-agent electricity market
this agent may summarily reject the offer—starting high and described a case study on bilateral contracts. Results
frequently communicates an attitude of toughness that can from a simplified retail market, involving interaction be-
be reciprocated by the opponent, thus making negotiation tween a retailer and a customer, shown that a computational
difficult to resolve. Also, if the price from producers is negotiation approach can help protect market participants
higher than the price in the contract with Britton, then from price volatility. As this research progresses, we aim to
Colburn will lose money. Therefore, it is essential that tackle increasingly more complex and realistic scenarios.
Colburn selects the “right” strategy to negotiate with Britton,
while entering into favorable contracts with producers. The R EFERENCES
main point here is that strategic choice is a critical activity [1] F. Bellifemine, G. Caire and D. Greenwood, Developing Multi-
in negotiation. Agent Systems with JADE, John Wiley & Sons, Chicester, 2007.
[2] V. Koritarov, Real-World Market Representation with Agents:
We have taken up the possible pairs of strategies, one at a
Modeling the Electricity Market as a Complex Adaptive Sys-
time, and examined their impact on the negotiation outcome. tem with an Agent-Based Approach, IEEE Power & Energy
The main response measure was the benefit of each agent magazine, pp 39–46, 2004.
in the final agreement. The experimental results are shown [3] H. Li and L. Tesfatsion, Development of Open Source Software
in table II. The results indicate that the following strategies for Power Market Research: The AMES Test Bed, Journal of
Energy Markets, 2:111–128, 2009.
yielded superior outcomes:
[4] F. Lopes, N. Mamede, A. Q. Novais and H. Coelho, A Nego-
• Customer agent: energy dependent concession making tiation Model for Autonomous Computational Agents: Formal
(EDCM); Description and Empirical Evaluation, Journal of Intelligent &
• Retailer agent: low-priority concession making (LPCM) Fuzzy Systems, 12, pp. 195–212, 2002.
[5] F. Lopes, N. Mamede, A. Q. Novais and H. Coelho, Nego-
and starting reasonable and conceding moderately tiation Strategies for Autonomous Computational Agents, In:
(SRCM). 16th European Conference on Artificial Intelligence (ECAI-
It is important to note, however, that these are initial (and 04), IOS Press, pp 38–42, 2004.
partial) results in a simple bargaining scenario. There is a [6] F. Lopes, M. Wooldridge and A. Q. Novais, Negotiation
Among Autonomous Computational Agents: Principles, Anal-
need to conduct extensive experiments to evaluate the effect ysis and Challenges, Artificial Intelligence Review, 29, pp. 1–
of the aforementioned strategies on both the outcome of 44, 2008.
negotiation and the convergence of the negotiation process. [7] F. Lopes and H. Coelho, Concession Behaviour in Automated
Negotiation, In: E-Commerce and Web Technologies, Springer
IV. R ELATED W ORK Verlag, Heidelberg, pp. 184-194, 2010.
[8] M. Osborne and A. Rubinstein, Bargaining and Markets,
Multi-agent energy markets have received some attention Academic Press, San Diego, 1990.
lately and a number of prominent tools have been proposed [9] M. Pechoucek and V. Marik, Industrial Deployment of Multi-
in the literature, including EMCAS [2], AMES [3], and agent Technologies: Review and Selected Case Studies, Au-
MASCEM [13]. Despite their power and elegance, they tonomous Agents and Multi-Agent Systems, vol. 17, pp. 397–
431, 2008.
often lack generality and flexibility, mainly because they are [10] D. Pruitt, Negotiation Behavior, Academic Press, New York,
limited to particular features of market players. 1981.
Lopes et al. [6] present an extensive review and highlight [11] H. Raiffa, The Art and Science of Negotiation, Harvard
some of the major challenges for future automated negoti- University Press, Cambridge, 1982.
ation research. The authors are aware of no similar efforts [12] M. Shahidehpour, H. Yamin and Z. Li, Market Operations in
Electric Power Systems, John Wiley & Sons, Chicester, 2002.
to formalize important negotiation strategies motivated by [13] Z. Vale, T. Pinto, I. Praça and H. Morais MASCEM -
rules-of-thumb distilled from good behavioral practice in Electricity markets simulation with strategically acting players.
real-life negotiations. IEEE Intelligent Systems, 26(2), 2011.

330

Authorized licensed use limited to: INTERNATIONAL ISLAMIC UNIVERSITY. Downloaded on July 30,2020 at 09:20:20 UTC from IEEE Xplore. Restrictions apply.