BIDDING IN INTERRELATED DAY-AHEAD ELECTRICITY MARKETS:

INSIGHTS FROM AN AGENT-BASED SIMULATION MODEL

Anke Weidlich and Daniel Veit
Institute for Information Systems and Management, University of Karlsruhe (TH)
Englerstr. 14, 76131 Karlsruhe, Germany
{anke.weidlich,daniel.veit}@iw.uni-karlsruhe.de

Abstract
In this paper we present results from an agent-based simulation model of two sequentially cleared
electricity markets. Agents can bid on both a day-ahead market for physical delivery contracts and a
day-ahead balancing power market and learn from their achieved results. Different scenarios of the
order of market clearing and pricing rules are tested and their results are compared. We show that
prices are lower in both markets when the day-ahead market is cleared first. We also show that pay-
as-bid leads to lower resulting prices than a uniform price mechanism.
Key words: agent-based simulation, agent-based computational economics (ACE), electricity market modelling,
sequential market clearing

1. Introduction
During the last decades, electricity markets have been restructured in many regions worldwide. One
effort in this ongoing change in the regulatory framework is to separate transmission system
operations from other parts of the electricity system, such as generation and retail services. When
transmission and generation are separated, the transmission system operator (TSO) needs to procure
balancing power from the market. This leads to the emergence of markets for primary and
secondary reserve, as well as for minute reserve. In this environment, a generator that disposes of
fast controllable generation units faces the problem of deciding whether to bid in a day-ahead
market or to commit his units for balancing purposes. This increasing complexity of electricity
trading and bidding decisions raises the necessity for methods and tools that allow modelling a
variety of aspects.
One important issue in electricity market modelling is strategic behaviour. In competitive
equilibrium models of electricity markets it is assumed that no agent tries to game the market and
all market participants are price takers. In equilibrium, all agents bid their marginal generating costs
and the (uniform) market clearing price is equal to the marginal unit bid. In real electricity markets,
however, the market structure is oligopolistic and each agent can potentially influence market
prices. As one example, Cramton [2004] shows how agents rationally bid in oligopolistic electricity
markets and argues that bidding above marginal cost is a rational behaviour and must not be
considered as anti-competitive. Realistic electricity market modelling must therefore account for
profit-seeking agent behaviour that involves bidding above marginal costs.
A second important aspect that a realistic electricity market model should account for is the factor
of daily repetition in trading. Rothkopf [1999] argues that the repetitive nature of electricity trading
is a crucial factor in market design. He argues that the choice of an efficient market mechanism may
differ when moving from one-shot auctions to repeated auctions.
A third aspect that should be addressed by electricity models is market interrelationships. Electricity
is traded on different time scales, e.g. day-ahead, real-time, and also in form of different products,
e.g. physical delivery, reserved (balance) capacity. The outcome of one market may influence an
agents bidding strategy on another market. Realistic electricity market modelling should thus
account for interrelations between different markets.
Agent-based simulation has the potential to meet the aforementioned requirements and is a very
promising approach for realistic electricity market modelling. It allows representing bidders as
profit-maximising adaptive agents that can learn from their trading results in daily repeated
auctions. It is also possible to simulate learning agents trading on different markets. The markets are
then linked together through the agents trading decisions, which allows for studying market
interrelationship aspects.
Several studies have demonstrated the usefulness of agent-based simulation for the analysis of
energy systems, e.g. [Koesrindartoto et al. 2005], [Bunn, Oliveira 2003], or [Bagnall 2004]. Agent-
based models considering market interrelationships include [Ruprez Micola, Estaol, Bunn 2006],
who studies vertical integration in a multi-tier energy market including gas shippers, electricity
generators and retailers; [Veit et al. 2006] study the interplay between a forward market and a spot
market for electricity contracts and its effect on price stability.
In the following, we report results from an agent-based simulation that comprises a day-ahead
market for physical delivery contracts and a day-ahead balancing power market. Agents are
endowed with learning capabilities and develop strategies that maximise their profits. The
remainder of this paper is structured as follows: section two introduces the simulation model and
the agents trading problem. Section three summarises the characteristics of the simulation
scenarios and section four reports and discusses the results from these simulations. Section five
finally concludes and gives an outlook on future research.
2. The model
In our model we simulate sequenced trading in both a day-ahead market for physical delivery with
uniform price clearing, or pay system marginal price, and a balancing power market where agents
bid their capacity to a transmission system operator as minute reserve. On the latter market, both
pay-as-bid and uniform pricing schemes have been implemented for different scenarios. The
simulation model has been implemented with Java using Repast
1
, a toolkit for agent-based
modelling.
2.1. The day-ahead market
A bid on the day-ahead market consists of different price volume pairs. A supply bid by generator i
takes the following form:
{ } , , ,1 , ,1 , , , , , , , ,
, ,..., , ,..., ,
dayAhead dayAhead dayAhead dayAhead
i h i h i h i h s i h s i h S i h S
b p q p q p q =
where p is the bid price and q is the quantity that the agent is willing to sell/buy at this bid price.
The index h denotes the hour for which the bid is valid and s is an index for the different price
volume pairs
2
. In our simulations we assume that an agent submits the same bid for every hour, so
the index h can be omitted in the following presentation. In further extensions of the model,
however, it is envisaged to let agents optimise their bids individually for each hour.
The day-ahead market operator collects all supply and demand bids and orders them into 24 hourly
bid collections. Market clearing is done once per trading day and separately for each hour; physical
delivery of the sold electricity is effected on the following day. The day-ahead market is designed
as a sealed-bid double auction. However, in the current model active demand bidding doesnt occur.
Instead, generators submit supply bids for satisfying a fixed inelastic demand.

1
Information about this tool can be found on http://repast.sourceforge.net.
2
The bid format is designed in the style of the spot market concept used at the [European Energy Exchange 2005].
In order to determine the market outcome, the market operator sums up all supply and demand bids.
Supply bids are ordered from the lowest to the highest bid price and from highest to lowest bid
volume in the case of equal bid prices. The market clearing price and volume are determined by the
intersection of the supply and demand curves. If necessary, the marginal bid is only partially
fulfilled. If many market equilibria exist, i.e. when the intersection between the supply and demand
curves is not a single point but a vertical line segment, the midpoint between the lower and the
upper limit of the intersecting line segment is taken as the market clearing price. Figure 1 shows an
example of the summed supply and demand bids in a day-ahead electricity auction.

Figure 1: Summed supply and demand bids in a day-ahead market
At the end of each trading round, every agent receives information about the resulting market
clearing price and the individual volume that it has been able to sell or buy in the auction. Agents
use this information in order to calculate their profits and reinforce their chosen strategies.
2.2. The balancing power market
In the balancing power market an agent bids a capacity price (cap) at which it is willing to provide
its capacity as positive minute reserve to a transmission system operator. If the TSO actually needs
electric work for stabilising the transmission system, he bases the decision on which capacity to
deploy on the basis of the bid work price, among all accepted bids. A bid on the balancing power
market thus consists of a capacity price and a work price, together with the bid quantity for each
bidding bloc b:
{ }
, , , ,
, ,
balance cap work
i b i b i b i b
b p p q =
The market clearing of the procurement auction for minute reserve is effected in two stages. In the
first stage, which takes place one day ahead of physical delivery, the TSO selects the power plants
that are held in reserve. For simplicity, we assume that there is only one bidding bloc covering all
24 hours of one day. Hence, the index b can be omitted in the following. Among several different
options for choosing the best bids, the algorithm applied in our simulations is solely based on the
capacity bid prices. This reflects the fact that in real-world balancing power markets for minute
reserve, the actual minute reserve deployment is only in few cases greater than zero
3
and thus work
prices play a minor role in the optimal bid allocation. Bids are ordered from the lowest to the
highest capacity bid price and from highest to lowest bid volume in the case of equal bid prices.
Bids are accepted up to the demand volume as put to tender by the TSO.

3
For data on minute reserve deployment and resulting capacity and work prices in the German balancing power
markets, the interested reader is referred to the websites of the four German TSOs: http://www.eon-netz.com,
http://www.rwe.com, http://transmission.vattenfall.de, and http://www.enbw.com.
In the second stage, all accepted bids are sorted according to their bid work prices (merit order). If
on the following day the TSO is in need of electric work from minute reserve, he determines the
capacity to be started-up on the basis of the work price merit order. In the case of uniform price
clearing, all successful generators are paid the uniform capacity price, which corresponds to the
capacity bid price of the last unit needed to satisfy the demand. Those generation units that actually
delivered electric work will be remunerated the uniform work price, which is again determined by
the last successful bid. In the pay-as-bid case, all successful generators receive their individual bid
prices for their committed capacity and, if applicable, their bid work prices for delivered electric
work.
2.3. Agent learning
Agents choose their bids according to a probability distribution developed within a reinforcement
learning algorithm. The learning representation applied in our simulation takes the form of a
Modified Erev-Roth reinforcement learning algorithm as presented in [Nicolaisen, Petrov,
Tesfatsion 2001]. It is a three parameter algorithm developed on the basis of the findings in [Erev
and Roth 1998] and is described in the following.
For each situation in which learning is applied, an agent can choose from a set of possible actions
M. If generator i chooses his kth action at time t and receives a reinforcement of R(x), it updates its
individual propensities to choose action j = (1,,M) according to the following function:

( ) (1 ) =
( 1) = (1 ) ( )
( )
( 1)
ij ij
ij
R x j k
s t s t
s t otherwise
M
e
f e
-

+ - +

Here, s
ij
is the propensity to choose a specific action in the next round, f is the recency parameter
and e the experimentation parameter. The recency parameter reflects the agents tendency to
forget past experience over time, and the experimentation parameter defines the extent to which
agents engage in information exploration through trying strategies that dont have the highest
propensity. It has the effect that agents do not lock into one choice at a too early stage. The third
parameter specifying the presented reinforcement learning algorithm is the scaling parameter s(0). It
defines the initial propensities for all actions. Agents choose an action k according to the following
probability:

=1
( 1)
( 1) =
( 1)
ik
ik
M
ij
j
s t
p t
s t
+
+
+

They thus reinforce actions which have resulted in a high payoff and choose these again with a
higher probability in the future. Accordingly, less successful actions are weakened and chosen again
with a lower probability.
As probabilities should not be negative, it must be ensured that the propensities always have a
positive value. If agents bid a price below their marginal generation costs, they face the risk to be
called into operation at a price at which they make losses. For ensuring positive propensities we
follow the proposition of Erev and Roth [1998], who subtract the smallest possible payoff x
min
from
all payoffs:
min
( ) R x x x =
In order for the algorithm to represent agent learning in a realistic manner, the parameters
specifying the algorithm have to be set carefully. Erev and Roth [1998] state that a combination of
f = 0.1 and e = 0.2 leads to the best prediction of the empirical outcome for their studied matrix
games. The order of magnitude of the scaling parameter s(0) depends on the magnitude of the
possible reinforcements. In an effort standardise this parameter we restrict the reinforcement to be
in the interval [0,1] by dividing each reinforcement by the highest possible reinforcement. In other
simulations that we have carried out, we find that the value for the scaling factor doesnt influence
the results significantly
4
. We also found that the parameter combination stated by Erev and Roth as
the best for predicting the results of their studied human experiments actually leads to a low
variability of simulation outcomes over a series of 100 simulations, which means that the choice of
a random number seed influences the results less than most other tested combinations
5
. These
findings led us to the following choice of a parameter combination for every Modified Erev-Roth
reinforcement learning algorithm: f = 0.1; e = 0.2; s(0) = 1.0.
2.4. The agents action domains
On the day-ahead market, we model an agent as trying to optimise both its bid prices and quantities
in order to maximise its individual profit. Thus, an agent can engage in withholding strategies if it
finds these profitable. The possible bid prices for an agent on the day-ahead market range from the
minimum to the maximum admissible price, where
min
dayAhead
p = 0 and
max
dayAhead
p = 100. A generator
chooses the bid quantity as a fraction of his available capacity. The intervals [a,b] of possible prices
and capacity fractions are stratified into 21 discrete values for the bidding price and six values for
the bidding quantity. The action domain for an agent bidding on the day-ahead market thus
comprises M = 126 possible actions and summarises to the following form:
{ } { } { } { } { } { } , 0, 0 , 0, 0.2 ,..., 0,1.0 , 5, 0 , 5, 0.2 ,..., 100,1.0
balance dayAhead dayAhead
M p q = =


On the balancing power market, an agent always chooses its bid quantity as equal to the net
installed capacity, reduced, if applicable, by the amount of capacity it has already committed on the
day-ahead market. Thus, an agent only learns to choose its bid prices, and employs a fixed
strategy for the bid volume. As a bid on the balancing power market comprises a capacity price and
a work price, this again leads to a two-dimensional action domain. Admissible prices range from
,
min
balance cap
p = 0 to
,
max
balance cap
p = 500 and
,
min
balance work
p = 0 to
,
max
balance work
p = 100, stratified into 21 possible
capacity prices and five work prices, which results into the following action domain:
{ } { } { } { } { } { }
, ,
, 0, 0 , 0, 25 ,..., 0,100 , 25, 0 , 25, 25 ,..., 500,100
balance balance cap balance work
M p p = =


2.5. The agents strategy
The problem facing the agent can be divided into the strategy to choose for the day-ahead market
and the strategy for the balancing power market. The learning task for each agent is consequently
separated into two individual learning problems. On the level of implementation, this results in each
agent employing two instances of the learning algorithm, with the same parameter values but
different action domains for both instances.
In our simulation scenario, each agent i owns one generation unit u which is characterised by a
linear variable generation cost function, i.e. constant marginal generation costs MC
i,u
. Each power
plant has a net installed capacity
,
inst
i u
Q and no-load costs NLC
i,u
. Ramping costs or other commitment
constraints are abandoned in this simulation for simplification reasons.
As some capacity
, ,
comm
i u t
q of a generation unit may have been committed in one electricity market for a
period of time t, the available capacity for this unit
, ,
avail
i u t
q is defined as

4
The tested scaling parameters range from s(0) = 0.5 to s(0) = 1.0.
5
The tested parameter values range from f = 0.0 to f = 0.5 and e = 0.1 to e = 0.5
, , , , ,
avail inst comm
i u t i u i u t
q Q q =
for this specific period. Other reasons for reduced power plant availability, such as maintenance
periods or unplanned outages, are not included in the current simulation implementation. We
assume that an agent bids one price volume pair for every generation unit it owns; agents cannot bid
more than their available capacity into the market:
, , , ,
dayAhead avail
i u h i u h
q q ,
, , , ,
balance avail
i u b i u b
q q
An agent tries to maximise profits on both markets. It does so by favouring actions that have
yielded higher profits in the past trading round through reinforcement learning. The reinforcement
of each chosen action comprises the profit achieved on the market, and also includes opportunity
costs to a certain extent. On the day-ahead market, the agents profit is defined as follows:
( )
, , , ,
( )
dayAhead dayAhead dayAhead dayAhead
i u i u i u i u
q p MC q =
Here, p
i,u
is the profit that agent i achieves for its generating unit u, p
dayAhead
is the resulting uniform
price on the day-ahead market and
,
dayAhead
i u
q is the quantity of unit u that agent i was able to sell on
the day-ahead market. The opportunity cost oc
dayAhead
for agent i on the day-ahead market is the
profit it could have achieved if it had bid its capacity on the balancing power market. It is defined
as:
( )
, , ,
dayAhead dayAhead balance
i u i u i u
oc q p NLC =
The profit an agent achieves on the balancing power market is defined as follows:
( ) ( )
, , , , ,
, , ( , ) , , ( , ) ,
( )
balance balance cap balance cap balance work balance work balance work
i u i u i u i u i u i u i u
q p NLC q p MC q = +
Here, the subscript (i,u) indicates that the capacity and work price can either be a single uniform
price or an individual bid price for each agent/unit, depending on the pricing rule employed. As the
produced electric work is remunerated separately, the cost that an agent i faces for providing minute
reserve capacity is only limited to the no-load costs of its unit. However, an agent faces high
opportunity costs, because it could also use its unit for producing electricity that it can sell on the
day-ahead market. This opportunity cost is calculated as:
( )
, ,
, , ,
( )
balance balance cap dayAhead balance cap
i u i u i u
oc q p MC q =
In both markets, the opportunity costs are subtracted from the profit that an agent receives. In order
to prevent unintended effects, we restrict the opportunity costs to be positive through applying a
min() function. In addition, we ensure that the inclusion of opportunity costs doesnt lead to
negative reinforcements.
3. Simulation scenarios
Because of the probabilistic nature of the applied reinforcement learning algorithm, the outcome
from our agent-based simulations partly depends on the random number seed with which learning
instances are initialised. We derive individual random number seeds for each instance of the
learning module, so as to avoid unintended similarity among agents which would occur if all
learners had the same seed. Every simulation setting is run 100 times with different random number
seeds at each run. All seeds employed in one set of 100 runs are stored and used again for different
simulation scenarios. By doing so, we can best exploit the advantages of agent-based simulation:
different settings, e.g. different market mechanisms can be tested under exactly the same conditions.
By ensuring that all agents in principal learn exactly in the same way in one setting as in another,
we can derive qualitative conclusions about market efficiency through comparing the resulting
outcome of both settings.
In this paper, we explore four different scenarios which differ in the order of market execution and
the pricing mechanism on the balancing power market:
DayAheadBalance_uniform
DayAheadBalance_payAsBid
BalanceDayAhead_uniform
BalanceDayAhead_payAsBid
DayAheadBalance refers to a scenario where the day-ahead market is cleared first, and then results
from this market are published before the balancing power market clears. BalanceDayAhead
switches the order of market execution. The endings _uniform and _payAsBid correspond to a
uniform price and a pay-as-bid pricing rule on the balancing power market. On the day-ahead
market, uniform pricing is employed in all cases.
The participating agents and their generation unit characteristics are summarised in Table 1.
Agent name
Capacity of
generating
unit
Marginal
cost
No-load
cost
[MW] [EUR/MWh] [EUR/h]
Generator1 300 5.00 75.00
Generator2 300 10.00 75.00
Generator3 300 15.00 75.00
Generator4 300 20.00 75.00
Generator5 300 30.00 75.00
Generator6 300 5.00 150.00
Generator7 300 10.00 150.00
Generator8 300 15.00 150.00
Generator9 300 20.00 150.00
Generator10 300 30.00 150.00
Table 1: Characteristics of the agents generating units
The static demand throughout the simulation is D
dayAhead
= 1500 MW and D
balance,cap
= 800 MW,
both constantly for every hour. The demand for electric work from minute reserve is D
balance,work
=
160 MW, which occurs in one hour per day.
4. Simulation outcome
Each run within one simulation scenario has been simulated over at most 2,000 trading days. In
many runs, however, all agents locked into some preferred action earlier, because the propensities
of all other actions were so low that they were hardly chosen. A simulation run was stopped if the
resulting market price did not change over a period of 200 trading days. In all cases, the mean
bid/market prices and bid/resulting volumes over the last 200 trading days was recorded as a result
for each run. In the following, the simulation results for each scenario are reported. They comprise
the average, minimum and maximum resulting values over 100 runs with different random number
seeds, as well as the standard deviations (SD).
Resulting prices on the day-ahead market are depicted in Table 2. Prices attain a higher level if the
day-ahead market is cleared after the balancing power market. The intuition behind this result is that
competition is lower on the supply side in these cases, as some generators have already committed a
part of their capacity on the balancing power market. When fewer agents compete on the day-ahead
market, they can more successfully bid above marginal cost and thus achieve higher prices.
Scenario
Average
p
dayAhead
Minimum
p
dayAhead

Maximum
p
dayAhead

SD
6

p
dayAhead

DayAheadBalance_uniform 40.65 30.00 60.00 6.45
DayAheadBalance_payAsBid 37.01 20.00 65.00 7.32
BalanceDayAhead_uniform 66.14 35.00 100.00 15.56
BalanceDayAhead_payAsBid 64.03 35.00 100.00 16.28
Table 2: Simulated market clearing prices on the day-ahead market
Scenario
Average
p
balance,cap
/
p
balance,work
Minimum
p
balance,cap
/
p
balance,work

Maximum
p
balance,cap
/
p
balance,work

SD
p
balance,cap
/
p
balance,work

DayAheadBalance_uniform 358.82 19.10 158.00 0.00 500.00 75.00 65.70 21.52
DayAheadBalance_payAsBid 273.80 17.54 60.23 0.00 431.25 75.00 76.42 20.26
BalanceDayAhead_uniform 393.79 17.51 225.00 0.00 475.00 99.75 45.83 20.53
BalanceDayAhead_payAsBid 354.29 17.21 206.25 0.00 457.88 75.00 55.04 21.24
Table 3: Simulated capacity/work prices on the balancing power market
Table 3 represents the resulting capacity and work prices on the balancing power market. Here, we
observe the opposite structure as on the day-ahead market, i.e. prices tend to be lower when the
market is cleared second. This outcome cannot be explained by the supply concentration; another
aspect influences the outcome in a stronger manner in this case. Whereas price information on the
second market plays only a minor role for bidding strategies on the day-ahead market, on the
balancing power market agents mainly base their bid decision on their opportunity costs. High
prices on the day-ahead market mean high foregone profits for an agent that commits his capacity
for minute reserve purposes. So, when prices are high on the day-ahead market, agents tend to bid
higher on the balancing power market as well.
Agent
Average
,
dayAhead
i u
p
/
,
dayAhead
i u
q

Minimum
,
dayAhead
i u
p
/
,
dayAhead
i u
q

Maximum
,
dayAhead
i u
p
/
,
dayAhead
i u
q

SD
,
dayAhead
i u
p
/
,
dayAhead
i u
q

Generator1 22.26 262.25 0.00 0.00 60.00 300.00 14.56 51.35
Generator2 20.53 244.34 0.00 0.00 80.00 300.00 16.40 62.95
Generator3 28.64 237.32 0.00 0.00 80.00 300.00 19.88 75.16
Generator4 40.66 214.74 0.00 0.00 90.00 300.00 24.59 81.40
Generator5 49.06 187.08 0.00 0.00 100.00 300.00 27.25 98.87
Generator6 22.67 252.60 0.00 0.00 65.00 300.00 14.97 64.23
Generator7 23.78 247.79 0.00 0.00 84.48 300.00 17.59 60.26
Generator8 26.92 234.00 0.00 0.00 100.00 300.00 20.27 74.70
Generator9 39.49 209.56 0.00 0.00 100.00 300.00 25.32 81.95
Generator10 46.33 177.95 0.00 0.00 100.00 300.00 69.61 93.08
Table 4: Individual agents bid prices and volumes on the day-ahead market (simulation scenario:
DayAheadBalance_uniform)
The single agents bidding decisions on the day-ahead market are summarised in Table 4. It can be
shown that agents with low marginal costs realise their strategic advantage through bidding at lower
prices on average. This ensures that their bids are accepted with a higher probability, so they are
able to sell more electricity. It can also be shown that agents with high marginal costs tend to bid
less capacity into the market. This can be interpreted as a withholding strategy, which is applied in
order to raise the market price. According to the observed bidding strategies, it can be stated that
no-load costs dont play a significant role in the bidding decision on the day-ahead market; agents

6
SD = standard deviation
whose generation unit characteristics only differ in no-load costs tend to apply very similar bidding
strategies.
Table 5 shows the individual bidding decisions on the balancing power market. Only capacity bid
prices are displayed. It can be seen that on average, most agents bid higher prices in a market with
pay-as-bid pricing as compared to a uniform-price market. However, this rise in bid prices does not
lead to higher market clearing prices in a pay-as-bid setting (see results for the balancing power
market in Table 3). The average gain agents attain from receiving their higher bid prices under
pay-as-bid does not outweigh the gain that infra-marginal bidders have from receiving the uniform
price, which corresponds to the highest accepted bid. It is also interesting to note that generators
with lower marginal cost tend to bid higher prices in the balancing power market; this, too, is due to
the strong influence of opportunity costs on the bidding decision.
Agent
,
,
balance cap
i u
p
, BalancingDayAhead_uniform
,
,
balance cap
i u
p
, BalancingDayAhead_payAsBid
Average Min Max SD Average Min Max SD
Generator1 432.51 275.00 500.00 49.75 431.67 250.00 500.00 54.55
Generator2 434.21 275.00 500.00 49.33 432.65 275.00 500.00 50.28
Generator3 431.21 275.00 500.00 56.14 433.75 300.00 500.00 54.81
Generator4 424.47 300.00 500.00 50.71 427.13 275.00 500.00 51.53
Generator5 408.51 200.00 500.00 73.87 419.79 200.00 500.00 66.82
Generator6 402.97 75.00 500.00 87.31 407.71 150.00 500.00 77.24
Generator7 414.54 100.00 500.00 72.26 417.59 200.00 500.00 63.20
Generator8 412.22 25.00 500.00 90.86 416.26 100.00 500.00 79.56
Generator9 397.46 100.00 500.00 85.92 400.72 50.00 500.00 79.33
Generator10 388.59 75.00 500.00 99.77 397.47 100.00 500.00 86.54
Table 5: Individual agents bid prices and volumes on the balancing power market (simulation scenario:
BalanceDayAhead_uniform and BalanceDayAhead_payAsBid)
The question whether pay-as-bid or uniform price leads to lower market prices is controversially
discussed in the literature (e.g. [Kahn et al. 2001], [Rassenti, Smith, Wilson 2003], [Bower, Bunn
2001], [Xiong, Okuma, Fujita 2004]). Some researchers argue that pay-as-bid leads agents to bid at
higher prices, resulting in higher average market prices. Others, in contrast, argue that it leads to
lower overall prices, because infra-marginal agents reach lower prices as they would under uniform-
pricing. Our simulation results suggest that pay-as-bid does in fact result in higher bid prices, but
the bid price increase is not high enough to result in higher overall prices.
5. Conclusions and outlook
In this paper we modelled different scenarios with sequentially cleared electricity markets and two
different pricing mechanisms. We apply agent-based simulation for evaluating market outcomes for
the different tested scenario and for analysing market interrelationships. Generators are modelled as
adaptive agents that apply a reinforcement learning algorithm which allows them to iteratively
approximate their profit maximizing strategy. Within the tested scenarios, we differentiate between
bidding strategies in a day-ahead market with physical settlement, and strategies in a day-ahead
balancing power market. We shift the order of market execution and vary the pricing mechanism
from pay-as-bid to uniform price. Simulations for each setting are repeated over 100 runs in order to
level out the influence of the random number seeds taken for each learning instance.
We find that prices on the day-ahead market are higher if this market is cleared after the balancing
power market. We argue that this is due to the fact that competition is weaker in this case, as some
agents have already committed (part of) their capacity on the balancing power market. The reduced
supplier concentration enables agents to successfully bid higher mark-ups to their marginal costs.
Results on the balancing power market give a different picture: prices are lower if this market is
cleared second. Here, the effect of agents integrating their opportunity costs into their result
evaluation leads to higher prices when day-ahead prices are high, and lower prices when day-ahead
prices are low. As for the pricing rule on the balancing power market, we find the following result:
average prices are higher under uniform price than under pay-as-bid although agents bid at higher
prices under pay-as-bid. The increase in bid prices is outweighed by the effect of all infra-marginal
bidders receiving the marginal, i.e. highest accepted bid under uniform pricing.
The findings presented in this paper leave us confident that agent-based simulation can reproduce
realistic market outcomes. It is therefore a suitable tool for market design questions, because
different market mechanisms can be tested under exactly the same condition of agent learning. In
order to make agent-based simulation a useful market design tool, we will include more realistic
data characterising the electricity system into our model. Our simulation model is implemented in a
flexible way, so that is also easily extensible to other forms of generation or demand representation.
For example, marginal costs can also take a linear form instead of being constant, or agents can own
several generation units and engage in combined trading strategies for all their capacity. Hourly
contracts and smaller bid blocs for the minute reserve can also be modelled with the current
implementation. Using these features for analysing more settings is subject to our future research.
References
Bagnall, A. (2004) A Multi-Agent Model of the UK Market in Electricity Generation, In Bull, L. (Ed.): Applications
of Learning Classifier Systems Series: Studies in Fuzziness and Soft Computing, Vol. 150, pp. 167-181, 2004.
Bower, J.; D. Bunn (2001) Experimental analysis of the efficiency of uniform-Price versus discriminatory auctions in
the England and Wales electricity market, Journal of Economic Dynamics and Control, Vol. 25, pp. 561-592,
March 2001.
Bunn, D.; F. Oliveira (2003) Evaluating individual market power in electricity markets via agent-based simulation,
Annals of Operations Research 2003 Vol. 121 p 57-78, 2003.
Cramton, P. (2004) Competitive Bidding Behaviour in Uniform-Price Auction Markets, Proceedings of the 37
th

Hawaii International Conference on System Sciences 2004.
European Energy Exchange (2005) EEX Spot Market Concept, http://www.eex.de.
Erev, I.; A. Roth (1998) Predicting How People Play Games: Reinforcement Learning in Experimental Games with
Unique, Mixed Strategy Equilibria, American Economic Review, 88(4):84881, September 1998.
Kahn A.; P. Cramton, R. Porter, R. Tabors (2001) Uniform Pricing or Pay-as-Bid Pricing: A Dilemma for California
and Beyond, Electricity Journal, 70-79, July 2001.
Koesrindartoto, D.; J. Sun, L. Tesfatsion (2005) An agent-based computational laboratory for testing the economic
reliability of wholesale power market designs, IEEE Power Engineering Society General Meeting, pp. 931-936,
2005.
Nicolaisen, J.; V. Petrov, L. Tesfatsion (2001) Market Power and Efficiency in a Computational Electricity Market
With Discriminatory Double-Auction Pricing, ISU Economic Report No. 52, revised August 2001.
Rassenti, S.; V. Smith, B. Wilson (2003) Discriminatory Price Auctions in Electricity Markets: Low Volatility at the
Expense of high Price Levels, Journal of Regulatory Economics, vol. 23, no. 2, pp. 109-123, March 2003.
Rothkopf, M. (1999) Daily Repetition: A Neglected Factor in the Analysis of Electricity Auctions, The Electricity
Journal 11, pp 60-70, April 1999.
Ruprez-Micola, A., A. Estaol, D. Bunn (2006) Incentives and Coordination in Vertically Related Energy Markets,
CIC Working Papers, No. SP II 2006-02.
Veit, D.; A. Weidlich, J. Yao, S. Oren (2006) Simulating the Dynamics in Two-Settlement Electricity Markets via an
Agent-Based Approach, Working Paper (http://www.ieor.berkeley.edu/~jyao/pubs/yao-MS06.pdf).
Xiong, G.; S. Okuma, H. Fujita (2004) Multi-agent Based Experiments on Uniform Price and Pay-as-Bid Electricity
Auction Markets, IEEE International Conference on Electric Utility Deregulation, Restructuring and Power
Technologies (DRFT2004), Hong Kong, April 2004.