1330 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 20, NO.

3, AUGUST 2005

A Wholesale Power Trading Simulator

With Learning Capabilities
Toshiyuki Sueyoshi and Gopalakrishna Reddy Tadiparthi, Student Member, IEEE

Abstract—The U.S. wholesale power market comprises a large A large demand occurs on electricity in summer and winter, de-
commodity market. The growth in power trading is due to the pending upon the place where the demand occurs.
ongoing deregulation policy of the electric power industry. Most In addition to the above unique features regarding electricity,
deregulation scenarios indicate a further separation of power pro-
duction from transmission and retailing. The power production a price change in the wholesale power market occurs due to
is opened to more competition. Unfortunately, the power trading other socio-economic and engineering factors. Such factors in-
mechanism is not clearly investigated in the level that we can pre- clude: 1) an imperfect market structure; 2) a possible existence
dict a price change in the U.S. wholesale power market. Such a of a market power; 3) an occurrence of congestion in transmis-
price change in the U.S. wholesale power market is explored from
a simulation system with learning capabilities. Using the new in-
sion; 4) different speculation views among traders; 5) imperfect
telligence system, we investigate the bidding strategies of traders information on electricity transmission, (f) different bidding ap-
in the wholesale power market and examine how the price change proaches among traders; 6) a system failure and a maintenance
occurs under different economic and engineering environments. problem; and 7) a price change of fuel (e.g., coal, oil, and nat-
Index Terms—Electricity competition, machine learning, market ural gas). Thus, the wholesale power market can be considered
model, strategic pricing, wholesale power trading. as a complex system.
Many individuals, who are involved in the power industry,
are looking for a policy guidance regarding how to determine
an appropriate pricing system of electricity. To investigate the
I. INTRODUCTION business/policy issue, this study examines the wholesale power
market by a simulation study, because the numerical approach is
D EREGULATION of the electric power industry is a gen-
eral business trend occurring in many industrial nations.
Regulated or state-owned monopoly markets have been dereg-
more effective in dealing with the complex system than the ana-
lytical (mathematical) approach. Many different types of traders
ulated and competition has been introduced into the electric participate in a dynamic pricing mechanism of the power market
power industry. The business trend first occurred in the United on simulation. Thus, the purpose of this research is to develop a
Kingdom [10] and was followed by other nations such as trading simulator that predicts a price fluctuation of the whole-
New Zealand, Sweden [2], Norway [1], and Australia [6]. Such sale power market.
deregulation could be also found in American jurisdictions such The remaining structure of this article is organized as follows:
as New England, New York, California, and Pennsylvania-New The next section briefly reviews previous research efforts on
Jersey-Maryland (PJM). Recently, Japan has been planning to power trading. Section III describes the industrial structure of
deregulate her wholesale power market. Twenty-seven electric the current U.S. wholesale power market. Section IV describes
power firms have opened a new wholesale exchange power the computational structure of the proposed simulator. Section V
market in April 2005. documents simulation results. Section VI concludes this study,
Electricity has several unique features that are different from along with future research extensions.
other commodities. First, it cannot be stored as found in other
commodities. Second, a constant monitoring system is required II. PREVIOUS RESEARCH
to stabilize a balance between supply and demand; both are
This literature review indicates the position of the proposed
often expressed by a nonlinear relationship. Third, most resi-
approach by comparing itself with other previous research ef-
dential consumers are, on a short time, unaware of or indifferent
forts on the power trade issue. The previous research is method-
to a price change. In other words, price elasticity is considerably
ologically classified into the following three groups.
small, compared with other commodities. Finally, the demand
is influenced by a change in temperature and a seasonal change.
A. Behavioral Analysis
The first group discussed their bidding strategies of traders
Manuscript received June 28, 2004; revised December 27, 2004. Paper no. in the deregulated power market [15], [19], [7]. The behavioral
TPWRS-00348–2004.
T. Sueyoshi is with the Department of Management, New Mexico Institute of aspect of trading was examined from a perspective of bilateral
Mining and Technology, Socorro, NM 87801 USA (e-mail: toshi@nmt.edu). electricity contracts that need a risk hedge in the power market
G. R. Tadiparthi is with the Department of Computer Science, New Mexico [9], [21]. Game theory was used to investigate their negotiation
Institute of Mining and Technology, Socorro, NM 87801 USA (e-mail: gtadi-
parthi@ieee.org). behaviors for bilateral and multilateral contacts [32], [11], [25],
Digital Object Identifier 10.1109/TPWRS.2005.851948 [30]. A contribution of this research group is that these research
0885-8950/$20.00 © 2005 IEEE
SUEYOSHI AND TADIPARTHI: A WHOLESALE POWER TRADING SIMULATOR WITH LEARNING CAPABILITIES 1331

efforts provide the analytical features of electricity trading be- to understand both how traders accumulate their knowledge
haviors. For example, it provides analytical rationale and expla- and how to use their experiences for power trading. Thus, this
nation regarding how a market power(s) makes an impact on a research belongs to the above third group. However, this type
wholesale price. Meanwhile, the behavioral research excludes of research has been not sufficiently explored in the previous
important uncontrollable factors such as different speculations research efforts.
and bidding preferences among traders. Consequently, this type Finally, this research is concerned with numerical analysis on
of analytical approach is methodologically limited in price pre- power trading. Therefore, we do not discuss its public policy
dictability of the power market. issues. Readers who are interested in the policy perspectives
on power trading can find many previous research efforts such
B. Economic Analysis as Joskow and Schmalensee [14], Stoft [28], Schweppe et al.
The second group of research has developed mathematical [24], and Wilson [31]. See also [23] for a detailed description
models (e.g.,integer programming, stochastic programming and on electric power from a general engineering perspective.
forecasting models) to find an optimal bidding strategy. The
power allocation, derived from the optimal bidding strategy, III. INDUSTRIAL STRUCTURE OF U.S. WHOLESALE MARKET
needs to satisfy transmission requirements from a demand side
of electricity [3], [8], [16], [18], [26]. An important feature of The U.S. wholesale market is functionally separated into: 1)
the second group is that the research efforts do not consider the a transmission market and 2) a power exchange market. This
behavioral aspect of traders, as identified in the first group. A study focuses upon only the power exchange market. (Research
main objective of these approaches is to achieve the economic on bidding behaviors on transmission is an important future ex-
efficiency of the power market. tension of this study.)
The U.S. wholesale power exchange market is further func-
C. Numerical Analysis tionally broken down into: 1) a real-time market (transaction on
a five-minute interval); 2) an hour-ahead market; 3) a day-ahead
The last group of previous research efforts fully utilizes com-
market; and 4) a long-term market (transaction from one week
puter science methods (e.g., neural networks (NNs), a genetic
to five or more years). Each market has unique features in terms
algorithm, and a multiadaptive learning model) to predict a
of an auction/exchange process and transmission agreement.
change of power price and a dynamic fluctuation of other factors
This research focuses upon trading strategies for both a Day-
(e.g., weather) that influence the amount of electricity demand.
Ahead (DA) market and a Real Time (RT) market, because their
The NN technique, incorporating a learning process, was ap-
bidding behaviors of traders in both DA and RT markets have
plied for one-hour ahead forecasting of a power demand based
a close bidding linkage between them. Moreover, the two mar-
upon a given temperature [22], [29]. The volatility of power
kets are important in the investigation of a price fluctuation in
price was also measured by NNs [33]. The NN technique is a
the wholesale power market. In this study, the RT market implies
widely used numerical approach that heuristically approximates
not only the real time market but also the hour-ahead market, be-
the dynamic trend of a power price by duplicating the learning
cause the two are functionally similar and decided on the same
process of a human brain. A methodological strength of NNs is
day. Many researchers (e.g., Stoft, [28, p. 204]) consider that the
that the approach is so flexible that the estimation reliability of
DA market is a “financial and forward” market because all the
NNs is considerably high. However, the NN estimation lacks a
transactions in the DA market stop one day prior to RT and the
mathematical rationale. Moreover, the convergence rate of NNs
bidding decisions in the DA market are determined by the spec-
is very slow in producing a final estimation result(s), because
ulation of traders. Meanwhile, the RT market can be considered
many weights are associated with the NN computation. In
as a “physical and spot” market, because the delivery of power
the same vain, a genetic algorithm was applied to understand
in the RT market is not optional like that of the DA market. All
bidding strategies in the power market [20]. The approach is
traders enter the market to correspond to actual power flows.
useful when the power market is not volatile. In addition to the
Hence, the aspect of financial speculation is very limited in the
NN technique and the genetic algorithm, Jacobs [13], Bagnall
RT market. Thus, the RT can be considered as a physical market.
[5], and Morikiyo and Goto [17] have considered the power
In the RT market, traders need to make their decisions within a
market from a multiagent adaptive system [27] and applied a
limited time (e.g., one hour or five minutes). So, it can be con-
simulation-based approach to understand the dynamic bidding
sidered as a spot market.
process of the power market. These research efforts comprise
Finally, the risk hedge behavior of a trader(s) in the long-term
the third subgroup of the numerical analysis. A methodological
market is different from their trading approaches in the other
benefit of the approach is that many traders are considered as
power markets. The power trading for the long-term contract is
agents with different bidding preferences. (See Axelrod [4]
bilaterally determined between a generator and a wholesaler/re-
for a detailed discussion on historical and current issues on
tailer. Hence, the long-term market is excluded from the inves-
agent-based modeling.)
tigation of this study.
Admitting the previous contributions on power trading, this
study explores the wholesale power market from a simulation
study, because the market can be considered as a complex A. Auction Theory
system, as mentioned previously. Furthermore, a machine According to [12], the auctions are classified into the fol-
learning process is incorporated into the proposed approach lowing four types: 1) English: buyers start bidding at a low
1332 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 20, NO. 3, AUGUST 2005

price—the highest bidder wins and pays the last price bid; 2)
Dutch: the auctioneer starts very high and calls out progres-
sively lower prices. The first buyer, who accepts the price, wins
and pays the price; 3) Vickrey (second-price): buyers submit
sealed bids. The winner pays the price of the highest losing bid;
and 4) Sealed-bid (first-price): buyers submit sealed bids. The
winner pays the price that is the highest among bids.
When examining the four types of auction from the perspec-
tive of wholesale power trading, it is important to note that all
the auctions, listed above, belong to a single-settlement system
where traders bid their prices once to determine a winning price. Fig. 1. Bidding structure for wholesale market.
However, the wholesale power trading examined in this study
belongs to a two-settlement system (TSS), where traders bid trade agents whose bidding strategies are examined in our sim-
their prices in the DA market and then make their bids again in ulation study. Each trader independently behaves for the en-
the RT market (Stoft [28, p. 210]). Furthermore, multiple bid- hancement of his/her own interest or benefit. Based upon the
dings are accepted in the DA and RT markets. previous bidding results (success/ failure), each trader accumu-
lates knowledge for his/her future decision-making.
B. Design of Power Market (Bilateral vs Central as Well as We acknowledge that the proposed simulator cannot perfectly
Exchange Versus Pool) duplicate real trading behaviors in the power trade markets.
Hence, this study needs to consider first how to structure DA
As mentioned previously, this study does not consider bilateral
and RT markets from their bidding strategies and economic
trading (e.g., a long-term market). The bilateral trading usually
rationales of traders. Then, different parameters regarding
responds slowly because it lacks a transparent market price.
bidding strategies and different initial starting points are used
Meanwhile, the power allocation for DA and RT is traded in
to express their various trading behaviors in the two markets.
a centralized market [often found in an Independent System
Furthermore, it is widely accepted that a reward obtained from
Operator (ISO)] that provides traders with a transparent price.
his/her previous trading often becomes a major economic
Consequently, traders in the central market find an efficient
incentive for many traders. The learning process incorporated
set of trades much faster and easier than those in the bilateral
in the simulator can be considered as a computational approach
market.
in which a trader tries to maximize a total amount of reward
The central market is functionally separated into an exchange
obtained from power trading.
market and a pool market. The power exchange market is defined
The bidding processes of traders in both DA and RT markets
as a centralized market that does not use side payments (Stoft
can be visually summarized in Fig. 1. The assumption and bid-
[28, p. 223]). In the exchange market, wholesalers pay the same
ding strategies incorporated in Fig. 1 are summarized as follows.
price to generators at any given time and location. A trader
Assumption: generators ( ) and wholesalers
simply trades with exchange at the trader location. California’s
( ) participate in the DA and RT markets at the th
Power Exchange is an example of the standard power exchange
period ( ).
in operation. In the market, traders use multiple rounds of
Supply-side Strategy in DA Market: is the maximum
bidding (24 hourly bids) to determine a market price and can
power generation capacity of the th generator at the th period.
implement full nodal pricing, where the nodal price implies
The generator bids as the amount of power generation for
a price at every node or bus of a power network. Meanwhile,
the DA market ( ). Here, the superscript “1” indicates
the pool market is adopted by PJM and NYISO (New York
the DA market and the subscript “ ” indicates a power delivery
ISO). In the market, generators report a detail of their costs
time. The bidding amount is expressed by , where
and limitations to a market operator. The market operator then
( ) is a parameter to express the ratio of the bidding
computes which generators need to be started ahead of time.
amount to the maximum capacity.
It finds an optimal allocation under an assumption that all
Let be the marginal cost of the th generator at the th
bids and demand forecasts are correct. The market operator
period. Each generator determines a bidding price ( ) for the
may find a market price whose level is lower than the price
DA market by . Here, ( ) is
that generators can start up. In this case, these generators can
a mark-up rate of the generator for the DA market. The mark-up
receive side payments.
rate expresses numerically how much the bidding price is in-
flated from the marginal cost. The mark-up rate reflects a price
strategy toward the DA trading. Considering different magni-
IV. SIMULATION STRUCTURE FOR POWER TRADING tudes of , the simulator examines various price strategies in
the DA market.
A. A Market Design for Wholesale Power Trading
After and are submitted by all generators into ISO, the
In the proposed simulator, the wholesale power market is organization determines the real allocation ( ) for each gener-
structured by the DA and RT markets. In each market, gener- ator in the DA market. The real allocation ( ), determined by
ators (suppliers) and wholesalers/retailers (buyers) consist of the DA market, is different from (a bidding amount).
SUEYOSHI AND TADIPARTHI: A WHOLESALE POWER TRADING SIMULATOR WITH LEARNING CAPABILITIES 1333

Supply-side Strategy in RT Market: In the wholesale market,

each generator is expected to allocate the whole generation ca-
pacity to the DA and/or RT market. Hence, the th generator
bids in the RT market, where the superscript “0”
indicates the RT market. The pricing strategy of the generator
is expressed by , where is a mark-up
rate ( ) and is the bidding price of the generator.
In the RT market, ISO obtains their bids on and from all
generators to determine (a real allocation for the generator)
and (a market price) through the RT market.
Demand-side Strategy in DA Market: A wholesaler predicts
an expected amount of electricity demanded on a delivery day,
using a forecasting method. (The forecasting technique will be
Fig. 2. Equilibrium point in DA market.
discussed in the proceeding subsection.) Let be the demand
estimated by the th wholesaler or ISO. The wholesaler pre-
dicts a bidding price ( ) by using an inverse function (IF) of
demand. That is, . In the proposed simulator, the
wholesaler makes a demand bid ( ) whose amount is less than
or equal to . That is, , where ( ) is a
parameter to express how each bid is strategically reduced from
the demand estimate. Similarly, the bidding price for demand is
determined by . Here, ( ) is a param-
eter for price adjustment from the estimated price. Both and
are submitted to ISO and then the organization determines
(a real power allocation to the wholesaler) and (a market
price) in the DA market.
Demand-side Strategy in RT Market: In the RT market, all
the wholesalers specify their required quantities on electricity. It Fig. 3. Equilibrium point in RT market.
is assumed that they must purchase all the necessary electricity
from DA and/or RT markets. Let be a real demand for the market price? Such a required market coordination mechanism
th wholesaler on the delivery day. The wholesaler specifies the can be found in ISO.
purchasing amount, ( ), in the RT market. ISO Fig. 2 visually describes the market coordination mechanism
adjusts all the requests from market participants to determine of ISO in the DA market. In the proposed simulator, ISO re-
(a real allocation) and (a market price) through the RT orders their biddings of generators and wholesalers. That is, the
market. supply side combinations ( and ) are reordered according
Finally, the following three additional comments are impor- to the ascending order of these bidding prices ( ). Thus, the
tant for understanding Fig. 1. bidding process can be considered as a sealed English auction
1) The design of the DA and RT markets has a computational with acceptance of multiple bids. Meanwhile, the demand side
benefit that the bidding prices and power allocations are combinations ( and ) are reordered according to the de-
numerically expressed by the parameters on the range be- scending order of these bidding prices ( ). The bidding process
tween 0 and 1. As a result of such numeration, the trading can be considered as a sealed Dutch auction with acceptance of
structure for the wholesale power market can be artifi- multiple bids. In Fig. 2, ISO allocates the generation amount
cially duplicated by the simulator. ( ) of the first generator to satisfy the demand ( ) of the
2) Information on , , , and needs to be first wholesaler. Such a power allocation is continued until an
given to the simulator. The remaining other variables are equilibrium point is found in the DA market. In Fig. 2, the equi-
determined by each trader. librium point is identified as EP, where the four generators are
3) The simulation needs to satisfy the following condition: used to satisfy the demand required by the three wholesalers.
Consequently, (the bidding price of the fourth generator)
becomes the market price ( ) for all participating traders in the
DA market.
Fig. 3 depicts the market coordination mechanism for the RT
market, where generators bid both and . The combination
of quantities and bidding prices are reordered according to
B. Function of ISO in DA and RT Markets the ascending order of these bidding prices ( ). Meanwhile,
In the DA market, generators bid their and . Simi- wholesalers submit their demands ( ), but not the bidding
larly, wholesalers bid their and in the DA market. How prices, because the RT market is a physical market where the
can we coordinate these bidding prices and amounts to obtain a demand of end users must be always satisfied. In Fig. 3, ISO
1334 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 20, NO. 3, AUGUST 2005

accumulates the generation amounts until the total demand is

satisfied. In the figure, is such a point, where

In Fig. 3, an equilibrium point is identified as EP, where

five generators are used to satisfy the total demand required by
wholesalers. Consequently, (the bidding price of the fifth
generator) becomes the market price ( ) for all participating
traders in the RT market.

C. Demand Forecasting
Each wholesaler (or ISO) needs to forecast the demand of
electricity. In this study, average, moving-average, exponential Fig. 4. Inverse function of demand.
smoothing, and random forecasting methods are used to esti-
mate the amount of demand. A choice from the four forecasting
A reward for the th generator may be specified in the fol-
techniques depends upon each wholesaler. Since a description
lowing manner. If , then the generator cannot have any
on the average and random forecasting are trivial to us, this study
chance to produce power, so resulting in no profit in the DA
omits the description on those methods except noting the fol-
market. Meanwhile, if , then the generator receives a
lowing.
total profit . Similarly, if , the generator
a) Moving Average: This method averages a data set for only loses a chance to generate electricity, so resulting in no profit in
the last periods as the forecast for the next period. That the RT market. Meanwhile, if , the generator can pro-
is, , where is the observed real vide electricity so that it produces a profit .
demand of the th wholesaler at the th period. Consequently, the reward for the th generator can be summa-
b) Exponential Smoothing: This method uses the following rized as follows:
formula: , where ( )
is referred to as a smoothing constant.
It is true that different agents have different preferences on if p
^ p and p
^ p , then (^
p 0 MC )^
s + (^
p 0 MC )^
s
their forecasting methods. However, it is also true that we cannot if p
^ <p and p
^ p , then (^
p 0 MC )^
s ,
perfectly duplicate their minds related to future demand predic- if p
^ p and p
^ <p , then (^
p 0 MC )^
s , and
tion. Hence, this study provides a choice in which all the traders if p
^ <p and p
^ <p , then the reward becomes zero.
can select one of the four methods for their forecasting.
The inverse function (IF) of demand used in this study may
be depicted in Fig. 4. As visually described in Fig. 4, the de- Next, we return to the reward to the th wholesaler. If
mand is structurally separated into a residential use (the left , then the wholesaler cannot access a power supply
hand side from the vertex) and an industry/business use (the through the DA market. Meanwhile, if , then the
right hand side from the vertex). The former has low price elas- wholesaler can obtain the power from the DA market. Sim-
ticity, while the latter has relatively high price elasticity. Hence, ilarly, if , then the wholesaler can access the power
the slope ( ) of the residential use is higher than the slope ( ) supply in the RT market. An opposite case can be found if
of the industrial/commercial use. The intercept of the two lines . The wholesaler usually provides electricity whose
are expressed by (for the residential use) and (the indus- price is ruled by a regulatory agency(s). Hence, let be the
trial/business use). The four coefficients need to be specified by retail price. Then, the reward for the th wholesaler can be
each wholesaler (or ISO). In Fig. 4, indicates the maximum specified as follows:
amount of power estimation of the th wholesaler. In the DA
market, each wholesaler forecasts the demand estimate ( ) and
if p
^ p ^
and d > 0, ^ +d
then p (d ^ ) 0 p^ d^ 0 p^ d^
its related price ( ) on the IF of demand. As mentioned previ-
if p
^ >p ^
and d > 0, ^ )
then p (d 0 p ^
^ d
ously; since the market is DA, the wholesaler determines his/her
if p
^ p ^
and d ^ )
= 0, then p (d 0 p^ d^
, and
bidding amount ( ) in a manner that the quantity is less than or
if p
^ >p ^ = 0, then the reward becomes zero.
and d
equal to . Moreover, the bidding price ( ) is set to be smaller
than the price estimate ( ).
E. Machine Learning
D. A Reward to Traders In the proposed simulator, traders accumulate knowledge
In the proposed simulator, traders are looking for the best from their bidding results to adjust their bidding strategies. The
combination of bidding prices and amounts in order to maxi- learning process incorporated in the simulator is separated into:
mize their individual rewards. 1) a knowledge accumulation (KA) process ( ) and 2) an
SUEYOSHI AND TADIPARTHI: A WHOLESALE POWER TRADING SIMULATOR WITH LEARNING CAPABILITIES 1335

The symbol ( ) indicates a parameter estimate obtained by

OLS. The above equations suggest that the winning probability
can be determined immediately from parameter estimates of the
sigmoid model. (The sigmoid model is often incorporated into a
neural network computation. The proposed learning process can
be, therefore, considered as a simple application of the neural
network. The sigmoid model is often called “a logit model” by
economists.)
The winning probability of the th wholesaler at the th pe-
riod can be obtained, via replacing (1), by the following linear
probability model:

(2)

The winning probability can be specified as

Learning Algorithm: In the proposed learning process, each

trader constantly looks for an increase in the estimated win-
Fig. 5. Adaptive learning process.
ning probability. In other words, the trader looks for a combi-
nation of parameters that produces a high winning probability.
own-bidding (OB) process ( ). Fig. 5 depicts such a To describe the learning process more clearly, we consider the
learning process. own bidding process of the th wholesaler at the th period.
The KA process provides each trader with a win-loss expe- For a visual description, this study considers the learning algo-
rience as a result of their biddings. In the process, all parame- rithm from the perspective of the wholesaler, because the bid-
ters are obtained from a random number generation. After the ding process can be expressed as a two-dimensional figure. The
learning process is over, traders start their own bidding decisions algorithm discussed here can be easily extended to that of a gen-
based upon their previous trading experiences. Of course, they erator. Such a description on the generator needs a three-dimen-
update and accumulate their knowledge at each trading. The bid- sional figure.
ding/learning process is considered as the OB process in this The knowledge accumulation process of the wholesaler pro-
study. The proposed simulator incorporates an adaptive sigmoid vides three parameter estimates of the sigmoid function. Two
decision rule. The decision rule can be analytically specified as parameter estimates ( and ) are important in determining
follows. the bidding strategies of wholesalers and both are associated
Adaptive Sigmoid Decision Rule: The win or lose of a with two decision variables ( and ), respectively. If a
trader is considered as a binary response. To express an oc- parameter estimate is positive, the wholesaler should increase
currence of the binary response, a sigmoid model is widely its corresponding decision variable to enhance his/her winning
used to predict its probability. Mathematically, the probability probability in the wholesale market. (Note that the winning
cumulative function of the sigmoid model is expressed by probability, obtained from the sigmoid function, does not
. The win/loss immediately imply that the trader can win in the wholesale
status of the th generator is predicted by the following linear market with the estimated probability. That is a theoretical
probability model: guess. The win or lose is determined through the DA and RT
market mechanism.)
(1)
Based upon the signs of parameter estimates obtained from
Here, is a reward given to the th generator at the th period. the knowledge accumulation process, the wholesaler has nine
Parameters are denoted by , , , and . An observational different bidding strategies (with as the start).
error is listed as . Those parameters are unknown and hence, Step 1: Set the initial bidding variables as ( , ). Also,
need to be estimated by ordinary least squares (OLS) regression. set the upper ( and ) and lower ( and )
The winning probability (Prob) of the th generator at the th bounds on the bidding variables based upon the bid-
period can be specified as follows: ding strategy of the th wholesaler. Both
and are required conditions
on the two parameters. In the case where no infor-
mation is available, ( , ) are determined by the
moving average or exponential smoothing score of
the previous biddings in the KA process. Further-
more, and are used as
these decision variables.
1336 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 20, NO. 3, AUGUST 2005

Step 2: Use OLS to obtain parameter estimates of the sig- TABLE I

UPDATES OF BIDDING VARIABLES
moid function from the KA process.
Step 3: Based upon the signs of the parameter estimates,
the decision variables on bidding are changed as
follows:
a) If & , then
.
b) If & , then
.
c) If & , then
.
d) If & , then
.
e) If & , then .
f) If & , then
.
g) If & , then
.
h) If & , then
.
i) If & , then
.
Note that
.
Step 4:) Compute and using ( , ) and submit the
bids to the DA market. If , then stop. Other-
wise, go to Step 5.
Step 5:) Set . If the trader loses (his/her reward is
zero), then go to Step 1. If the trader wins (his/her
reward is positive), then go to Step 6.
Step 6:) Reset the bidding variables as the current ones.
Reset the upper and lower bounds according to Fig. 6. Adaptive learning process.
Table I and go to Step 1.
Note that the above table implies, for example, that if TABLE II
and , then the upper and lower bounds of the next ( ) MARKET AND PRICING STRATEGIES OF GENERATORS
step is set by , , and
. See the cell at the second column and the second row of the
table. The remaining eight cases can be explained by a similar
manner.
Fig. 6 visually describes the above learning process for
strategic bidding. In the figure, the initial bidding variables of
a trader (for wholesale) are expressed on C. There are four
different directions toward NE (North-East), SE (South-East),
NW (North-West) and SW (South-West). As specified in Step
3, the new bidding amounts can be identified on A, E, B, upper and lower bounds, as depicted in Fig. 6. Comparing
D, respectively. If a parameter estimate contains zero, these it with other numbers (e.g., 1/4 and 3/4), the convergence
bidding amounts are found on i, f, g, h and C. (Note that if speed under 1/2 is slight better than the others in this study.
both parameter estimates are zero, the trader keeps the current Furthermore, we cannot find any difficulty on convergence (e.g,
strategy at the next bidding, so being on C in Fig. 6. Even if no convergence) in this study. However, there is no theoretical
the trader keeps the same strategy, the market result may be justification on the convergence issue. That is an important
different from the previous one, because the market determines future research task.
the price and amount of power allocation.) Thus, the parameter estimates of the sigmoid function ob-
Step 3 uses a value of 1/2 in the nice bidding strategies to find tained in the KA/OB process provide each trader with informa-
these locations (i.e., A, E, .. ,g and h). There is no theoretical tion regarding which is the best bidding choice among these nine
rationale on the selection of 1/2. However, the value selection alternatives. As long as the trader can win, the search area is re-
indicates the center of each moving area that is shaped by the duced to a smaller bidding range.
SUEYOSHI AND TADIPARTHI: A WHOLESALE POWER TRADING SIMULATOR WITH LEARNING CAPABILITIES 1337

TABLE III
EIGHT DIFFERENT STRATEGIES AND AVERAGE REWARDS

As depicted in Fig. 1, electricity is procured from either the

DA or RT market. Hence, if a generator prefers to sell electricity
mainly in the DA market, is set to be between 0.5 and 1.
Meanwhile, if the generator prefers to sell electricity mainly in
the RT market, the parameter is set to be between 0 and 0.5.
The pricing strategy of the generator is further separated into
two cases (risk taker or risk avoider). Here, the risk taker looks
for a high reward by offering a high bidding price. This type
of bidding strategy leads to a high risk, because a probability,
that the bidding price is larger than an equilibrium market price,
becomes larger and consequently, a chance to sell his/her elec-
tricity becomes lower than before. Conversely, an opposite case
(low risk and low reward) can be found in the risk avoider.
Parameters listed in Table II are used to express the bidding
strategy of the generators. As structured in Table II, eight
Fig. 7. Computer monitor of software. [ - - ] dif-
ferent combinations of parameters are examined to investigate
the market and pricing strategies.
F. A Visual Description on Computer Software
Under each strategy, these parameters are randomly gener-
Fig. 7 depicts a monitor of our software that shows the DA ated on the specified data ranges. Those ranges are selected ar-
and RT markets. The programming language of software is C# bitrarily in a way that the selection reflects the required types of
(Microsoft.NETFramework1.1).1 We can observe the dynamics market preference and pricing strategy.
of equilibrium points in the DA and RT markets, as found in In this study, ten generators belong to each group and
the top of the computer monitor. Those changes correspond to their traders bid on the parameter ranges, as specified in
Figs. 2 and 3. The graph in the middle of Fig. 7 depicts the trend Table II. The remaining parameters are randomly gener-
of market prices (in DA and RT markets). The last one at the ated between 0 and 1. The total number of generators is 80
bottom of Fig. 7 indicates the trend of power amounts traded in ( ). Meanwhile, 900 wholesalers
the two markets. are incorporated into the simulation study. In this situation,
three different types of group mix are considered in terms of
V. A SIMULATION STUDY learning (L) and nonlearning (NL). In each case, the average
The following assertions are examined by our simulation reward of the generators equipped with the proposed learning
study: 1) there are two wholesale power markets (DA and RT). process is compared with the average rewards of the remaining
Which market is important for generators? 2) What type of others without the learning process. Consequently, we can in-
risk taking is important for generators? Table II sets parameter vestigate strategic effectiveness related to the eight alternatives
ranges to examine the two assertions. The table is structured by on market and pricing strategies.
2 2 factors, where the first 2 is related to DA or RT and the Such strategies of generators and simulation results are doc-
other 2 is related to pricing or quantity strategy. This simulation umented in Table III (unit: $1000). In the table, 100 replications
study excludes the strategies of wholesalers, because their are generated for each combination. Consequently, the total av-
decisions are often influenced by external factors such as a erage listed at the last column of Table III indicates the average
temperature change and a seasonal effect. reward of 3000 [
] bids of each generator under eight different
1Source: http://msdn.microsoft.com/netframework. bidding strategies [i.e., Agent Types from a to h at the left hand
1338 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 20, NO. 3, AUGUST 2005

side of Table III]. Findings from Table III can be summarized Using the proposed simulator, we can predict various nodal
as follows. (local) wholesale power prices, each of which depends upon a
grid system through which electricity is provided. Such a differ-
A. Finding 1 ence among nodal prices is generated in our simulator by using
The combination between DA (in quantity allocation) and a different marginal costs, different electricity loads and other
risk-avoider (bidding price) produces the best reward ($1855.86) economic and engineering concerns. Furthermore, the simulator
among the eight different strategies. The next ($1810.00) is can change the number of participating traders, depending upon
the combination between RT and a risk-avoider. The worst the size of each grid system. Thus, the proposed simulator can be
performer ($111.95) is found on the combination between used for not only a single market price for an entire grid system,
RT and risk-taker. All the traders, who use the strategies (a), but also location-specific nodal prices for many different grid
(c), (e), and (g), result in low rewards ($175.32, $122.70, subsystems.
$144.75, and $111.95), because their bidding strategies in RT Since this study is an initial step, the proposed approach may
include a risk-taking decision on bidding prices. These results be not perfect from a practical perspective. The following future
indicate that generators need to set their bidding prices in the research tasks are envisioned to enhance the practicality of the
manner that each price is not far above its marginal price. proposed simulator:
If they become greedy, they will not obtain their expected a) Demand Side: The wholesale power market is considered
rewards. as a complex dynamic system whose many components
are complicatedly connected to each other with imperfect
B. Finding 2 information [24], [31]. As a result of the complexity, we
The comparison between the first four strategy combinations cannot mathematically prove the methodological validity
and the remaining four indicates that the DA market is slightly of our simulation results. Thus, a bidding strategy sug-
more important than the RT market in terms of reward enhance- gested by the proposed simulator does not always guar-
ment, because the former has more bidding freedom than the antee that it produces a positive reward. Such may be
latter. indeed a shortcoming of this type of simulation-based
approach. To overcome the methodological problem, we
C. Finding 3 need to obtain real data on wholesale prices from PJM
In the two wholesale markets, traders equipped with the and other power markets. Using the real data, we examine
learning capability outperform the other traders without the how the proposed simulator can accurately predict a real
learning capability. price fluctuation. Furthermore, a major source of the price
Here, it is important to note the following comments. fluctuation (and complexity) is often due to a temperature
(weather) change and its related demand change. Hence,
a) We have slightly changed the upper and lower bounds of
we need to incorporate data mining techniques into the
these parameters (e.g., ) to confirm
proposed simulator to enhance our forecasting capability
the robustness of the above findings. The sensitivity anal-
of the demand change.
ysis has produced an almost similar result documented in
b) Market Function: After the demand forecasting capability
Table III.
is added to our simulator, we need to incorporate a com-
b) This simulation study does not consider a special bidding
puter networking capability into the simulator. The com-
case, like a “must-run” operation, in which a generator,
puter network artificially opens a wholesale market where
like a nuclear power plant, provides a wholesale power
traders communicate with each other. In this research ex-
market with free electricity in order to avoid a temporal
tension, game theory will serve as a conceptual basis for
cease of its generation facility.
exploring how traders make a coalition to win their trades.
Furthermore, we can investigate various types of negotia-
VI. A CONCLUSION AND FUTURE EXTENSIONS tion and cooperation among traders. The behavioral study
To predict a price change of the U.S. wholesale power market, based upon the game theory will enhance the learning ca-
a computer simulator with learning capabilities has been devel- pabilities of traders to the level that the proposed simulator
oped in this study. Using the software, we can investigate both becomes a multiagent adaptive system.
how traders determine the market price in the DA and RT mar- c) Supply Side: After the network function is added to the
kets and how the price change occurs under different trading simulator; each computer monitor, linked to the power
environments. trading network, is designed to show different types of
The proposed simulator has two important features. One of cost on a computer monitor. Each trader can select which
the two is that the two settlement system (DA and RT) is incor- type of cost is used for his/her trading decisions. In this
porated into the artificial wholesale power market of the simu- study, the marginal cost is used as a basis for determining
lator. The other important feature may be found in the learning a bidding price and a reward in the current simulator. The
capability of traders who accumulate their bidding experience validity can be confirmed on the Web site of PJM where
and make a new bidding strategy to increase their winning prob- the marginal cost is listed for real power trading markets.
abilities. A simulation study is conducted and simulation results This indicates that the marginal cost is used for the deci-
are summarized as the three findings in this study. sions of traders. However, it is true that other information
SUEYOSHI AND TADIPARTHI: A WHOLESALE POWER TRADING SIMULATOR WITH LEARNING CAPABILITIES 1339

on cost is also useful for traders if that is available for them [3] J. M. Arroyo and A. J. Conejo, “Multiperiod auction for a pool-based
on a computer monitor. For example, as discussed in [23, electricity market,” IEEE Trans. Power Syst., vol. 17, no. 4, pp.
1225–1231, Nov. 2002.
Ch. 4], the generators (or ISO) are expected to optimize [4] R. Axelrod, The Complexity of Cooperation. Princeton, NJ: Princeton
their generation resources to satisfy a demand load at a Univ. Press, 1997.
minimized operation cost. The type of operation cost is re- [5] A. J. Bagnall, “A multi-adaptive agent model of generator bidding in the
ferred to as “cost-based unit commitment (UC).” The cost UK market in electricity,” in Proc. Genetic and Evolutionary Computa-
tion Conf., Las Vegas, NV, 2000, pp. 605–612.
concept is further extended by considering a transmission [6] D. Brennan and J. Melanie, “Market powering the Australian power
function that needs a careful monitoring process between market,” Energy Econom., vol. 20, pp. 121–133, 1998.
supply and demand. The extended cost concept is referred [7] J. W. M. Cheng, F. D. Galiana, and D. T. McGillis, “Studies of bilateral
contracts with respect to steady-state security in a deregulated environ-
to as “security-constrained unit commitment (SCUC).”
ment,” IEEE Trans. Power Syst., vol. 13, no. 3, pp. 1020–1025, Aug.
The SCUC implies a total operation cost for generation 1998.
scheduling, supplying load, maximizing power security, [8] J. Contreras, O. Candiles, J. I. Fuente, and T. Gomez, “A cobweb bidding
and managing other transmission constraints. Thus, the model for competitive electricity markets,” IEEE Trans. Power Syst., vol.
17, no. 1, pp. 148–153, Feb. 2002.
cost measures such as UC and SCUS need to be incor- [9] T. W. Gedra, “Optional forward contracts for electric power markets,”
porated into the proposed simulator in addition to the IEEE Trans. Power Syst., vol. 9, no. 4, pp. 1766–1773, Nov. 1994.
marginal cost. The two cost concepts are closely related [10] R. Green and D. Newbery, “Competition in the British electricity spot
to a power transmission period. Therefore, both UC and market,” J. Polit. Econ., vol. 44, pp. 205–201, 1992.
[11] X. Guan, Y. C. Ho, and D. L. Pepyne, “Gaming and price spikes in
SCUS need to be examined in a generation scheme with electric power markets,” IEEE Trans. Power Syst., vol. 16, no. 3, pp.
a specific time horizon. 402–408, Aug. 2001.
d) Transmission: The simulator does not incorporate any [12] V. Krishna, Auction Theory. San Diego, CA: Elsevier Science, 2002.
[13] J. M. Jacobs, “Artificial power markets and unintended consequences,”
aspect on the power transmission. As a result, for ex-
IEEE Trans. Power Syst., vol. 12, no. 2, pp. 968–972, May 1997.
ample, a price increase due to congestion has not been [14] P. Joskow and R. Schmalensee, Markets for Power. Cambridge, MA:
considered in this study, because the incorporation of MIT Press, 1983.
transmission into the proposed simulator needs a large [15] J. W. Lamont and S. Rajan, “Strategic bidding in an energy brokerage,”
IEEE Trans. Power Syst., vol. 12, no. 4, pp. 1729–1733, Nov. 1997.
mathematical modeling capability on maximum network [16] Y. Liu and X. Guan, “Purchase allocation and demand bidding in electric
flow and its related algorithmic development. Further- power markets,” IEEE Trans. Power Syst., vol. 18, no. 1, pp. 106–112,
more, such congestion depends upon each grid system. Feb. 2003.
Simply, different grid systems have different network [17] T. Morikiyo and M. Goto, “Artificial trading for US wholesale electric
power market,” Asia-Pacific Manag. Rev., vol. 9, pp. 751–782, 2004.
structures. The system development, including the trans- [18] F. J. Nogales, J. Contreras, A. J. Conejo, and R. Espinola, “Forecasting
mission constraints on an entire power system, needs next-day electricity prices by time series models,” IEEE Trans. Power
to consider various regional and geographical concerns. Syst., vol. 17, no. 2, pp. 342–348, May 2002.
Thus, the simulator excludes the transmission issue in [19] C. W. Richter Jr. and G. B. Sheble, “Genetic algorithm evolution of
utility bidding strategies for the competitive marketplace,” IEEE Trans.
our price setting mechanism. Of course, we will develop Power Syst., vol. 13, no. 1, pp. 256–261, Feb. 1998.
a simulation capability on transmission line constraints [20] C. W. Richter Jr., G. B. Sheble, and D. Ashlok, “Comprehensive bidding
and add it to the proposed power exchange simulator in strategies with genetic programming/finite state automata,” IEEE Trans.
Power Syst., vol. 14, no. 4, pp. 1207–1212, Nov. 1999.
our future work. [21] G. Rosenwald and C. C. Liu, “Consistency evaluation in an operational
The completion of the above research plan will enhance the environment involving many transactions,” IEEE Trans. Power Syst.,
practicality of the proposed simulator. It is believed that the net- vol. 11, no. 4, pp. 1757–1762, Nov. 1996.
work-based multiagent simulator can replace the software that [22] T. Senjyu, H. Takara, K. Uezato, and T. Funabashi, “One-hour-ahead
load forecasting using neural network,” IEEE Trans. Power Syst., vol.
is currently used for market operation in many wholesale power 17, no. 1, pp. 113–118, Feb. 2002.
markets like PJM and California ISO. [23] M. Shahidehpour, H. Yamin, and Z. Li, Market Operations in Electric
Finally, it is hoped that this study makes a small contribution Power System. New York: Wiley, 2002.
for research on the electric power industry. We look forward to [24] F. C. Schweppe, M. C. Caramanis, R. D. Tabors, and R. E. Bohn, Spot
Pricing of Electricity. Boston, MA: Kluwer, 1988.
seeing future research extensions, as indicated in this study. [25] H. Song, C. C. Liu, and J. Lawarree, “Nash equilibrium bidding strate-
gies in a bilateral electricity market,” IEEE Trans. Power Syst., vol. 17,
no. 1, pp. 73–79, Feb. 2002.
ACKNOWLEDGMENT
[26] H. Song, C. C. Liu, J. Lawarree, and R. W. Dahlgren, “Optimal elec-
The authors would like to express their deep appreciation to tricity supply bidding by Markov decision process,” IEEE Trans. Power
Syst., vol. 15, no. 2, pp. 618–624, May 2000.
Dr. Bhavaraju (PJM) and three anonymous referees for their
[27] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduc-
constructive comments that improve the quality of this article. tion. Cambridge, MA: MIT Press, 1998.
[28] S. Stoft, Power System Economics. Piscataway, NJ: IEEE Press, 2002.
[29] J. W. Taylor and R. Buizza, “Neural network load forecasting with
REFERENCES
weather ensemble predictions,” IEEE Trans. Power Syst., vol. 17, no.
[1] E. S. Amundsen, J. Bjordalen, and H. Rasmussen, “Export of Norwegian 3, pp. 626–632, Aug. 2002.
hydropower under a common European regime of environmental taxes,” [30] J. Villar and H. Rudnick, “Hydrothermal market simulation using game
Energy Econom., vol. 16, pp. 271–280, 1994. theory: Assessment of market power,” IEEE Trans. Power Syst., vol. 18,
[2] B. Andersson and L. Bregman, “Market structure and the price of no. 1, pp. 91–98, Feb. 2003.
electricity: An ex ante analysis of the deregulated Swedish electricity [31] R. Wilson, “Architecture of power markets,” Econometrica, vol. 70, pp.
market,” Energy J., vol. 16, pp. 97–109, 1995. 1299–1340, 2002.
1340 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 20, NO. 3, AUGUST 2005

[32] C. S. K. Yeung, A. S. Y. Poon, and F. F. Wu, “Game theoretical multi- Gopalakrishna Reddy Tadiparthi (S’03) received
agent modeling of coalition formation for multilateral trades,” IEEE the B.E. degree in computer science from the Uni-
Trans. Power Syst., vol. 14, no. 3, pp. 929–934, Aug. 1999. versity of Madras, Chennai, India, in 1999. He re-
[33] L. Zhang, P. B. Luh, and K. Kasiviswanathan, “Energy clearing price ceived the M.S. degree in computer science from the
prediction and confidence interval estimation with cascaded neural net- New Mexico Institute of Mining and Technology, So-
works,” IEEE Trans. Power Syst., vol. 18, no. 1, pp. 99–105, Feb. 2003. corro, in 2003, where he is currently working toward
the Ph.D. degree in computer science.
He worked as a Network Engineer at Satyam
Infoway Limited (SIFY), Chennai, India, from 1999
Toshiyuki Sueyoshi is a Full Professor in the to 2002. His research interests include application
Department of Management, New Mexico Institute of artificial intelligence techniques to business,
of Mining and Technology, Socorro. He is currently multiagent simulation systems, power markets, and intelligent agents.
interested in power trading on internet auctions.
He has published more than 100 articles, including
in the following publications: American Economic
Review, European Journal of Operational Research,
Management Science, IIE Transactions, Decision
Sciences, and Operations Letters.