Calibration of Microscopic Traffic

Simulation Models
Methods and Application

Ramachandran Balakrishna, Constantinos Antoniou, Moshe Ben-Akiva,

Haris N. Koutsopoulos, and Yang Wen

A mathematical framework and a solution approach are presented for ment site, is thus a vital step in the application of complex traffic
the simultaneous calibration of the demand and supply parameters and simulation tools.
inputs to microscopic traffic simulation models as well as a large-scale Several papers present frameworks for the calibration of micro-
application emphasizing practical issues. Microscopic traffic simulation scopic traffic simulation models. Ben-Akiva et al. (1), for example,
models provide detailed estimates of evolving network conditions by divide the process into two steps. First, individual model components
modeling time-varying demand patterns and individual drivers’ detailed are calibrated with disaggregate data such as vehicle trajectories and
behavioral decisions. Such models are composed of elements that simu- surveys. Such detailed data are well suited for the estimation of
late different demand and supply processes and their complex interactions. driver and travel behavior phenomena such as car following, lane
Several model inputs (such as origin–destination flows) and parameters changing, and route choice. This step is followed by the fine-tuning
(car-following and lane-changing coefficients) must be specified before of all parameters so that the simulator’s output matches aggregate
these simulation tools can be applied, and their values must be deter- data (such as time headways, speeds, and flows).
mined so that the simulation output accurately replicates the reality Disaggregate data, although ideal for microscopic model calibra-
reflected in traffic measurements. A methodology is presented here for tion, are extremely costly and difficult to collect. In most practical
simultaneously estimating all microscopic simulation model parameters applications, only aggregate traffic measurements will be available.
by using general traffic measurements. A large-scale case study for the Model calibration in such cases must be performed by using aggre-
calibration of the MITSimLab microscopic traffic simulation model by gate data alone, so as to minimize the deviation between observed
using the network of Lower Westchester County, New York, is employed and simulated measurements. However, because of the large set of
to demonstrate the feasibility, application, and benefits of the proposed unknown parameters and the significant computational burden asso-
methodology. ciated with large-scale traffic simulation runs, such calibration has
largely been restricted to trial-and-error approaches (2–4) and simpler
subproblems that estimate subsets of parameters while fixing the
Microscopic traffic simulation tools find wide-ranging applications
remaining variables at some predetermined values. Often, the focus
in network design and evaluation, planning and analysis, and the
is on only the set of origin–destination (O-D) flows to be assigned
evaluation of intelligent transportation systems. Such tools possess
onto the network or just a few key behavioral parameters selected
the flexibility to represent rich spatial and temporal demand pat-
through past experience or a sensitivity analysis. For example, the
terns, model complex driving behavior phenomena and the interactions
O-D flows (the primary inputs to a simulator’s demand models) may
between individual vehicles, mimic drivers’ travel behavior decisions
such as route choice and response to information, and simulate the be estimated while the driving and travel behavior model parameters
operations of traffic management strategies, technologies, and infra- are fixed (5–7). Alternatively, a few coefficients in the car-following
structure. However, owing to the complex nature of their constituent and lane-changing models (affecting supply-side processes and
model components with their dependence on a large number of inputs traffic dynamics) may be estimated while fixing the O-D flows at
and parameters, their use requires that the various inputs be determined their best-known values (5, 8–13).
to best represent observed local traffic data. Model calibration, which Aggregate calibration is based on the output of the simulation
matches simulator output to real measurements from the deploy- model, which is a result of the interaction among all the components
of the simulation. Therefore, it is generally not possible to isolate the
effects of individual models on various traffic measurements. For
R. Balakrishna, Caliper Corporation, 1172 Beacon Street, Newton, MA 02461. example, common O-D estimation methods require an assignment
C. Antoniou, Department of Transportation Planning and Engineering, National matrix or its equivalent as an input. The time-dependent O-D matrix
Technical University of Athens, 5 Iroon Polytechniou St., Zografou 15773, Athens,
xh defines the number of trips between each O-D pair departing the
Greece. M. Ben-Akiva and Y. Wen, Massachusetts Institute of Technology,
77 Massachusetts Avenue, Room 1-181, Cambridge, MA 02139. H. N. Koutsopou- respective origin during time interval h. The assignment matrix maps
los, Northeastern University, 437 Snell Engineering Center, Boston, MA 02115. these O-D flows to link flows (in both current and future intervals).
Corresponding author: R. Balakrishna, rama@caliper.com. Such a mapping is a complex function of drivers’ route choice and
driving behavior decisions, as well as the travel times on the network,
Transportation Research Record: Journal of the Transportation Research Board,
No. 1999, Transportation Research Board of the National Academies, Washington,
and has to be generated by the simulation itself. Similarly, the most
D.C., 2007, pp. 198–207. critical element in route choice models are the link travel times,
DOI: 10.3141/1999-21 which would be affected by the simulated flows, which in turn are a

198
Balakrishna, Antoniou, Ben-Akiva, Koutsopoulos, and Wen 199

function of the O-D flow matrices. Effectively, the values of each of subject to the following constraints:
these parameters influence the values of the other parameters that
are being calibrated. The calibration approach therefore needs to take Mh = f ( x1 , . . ., x h , 1 , . . ., h , G1 , . . ., Gh ) ⎫
these interactions into consideration. Often, this step is done by iter- ⎪⎪
atively calibrating the different components—that is, the behavior lhx ≤ x h ≤ uhx ⎬ ∀h ∈ {1, 2, . . ., H } (2)
models, route choice model, and O-D matrices—while keeping the β β
⎪
l ≤ h ≤ u
h h ⎪⎭
other parameters in the simulation model fixed at each step.
Ben-Akiva et al. (1) provide an iterative approach that reestimates where
each parameter subset in turn until convergence. However, the
computational effort involved is usually found to be too great to be Mh, Mh = observed and fitted sensor measurements for interval h,
practical. Further, an approach that simultaneously solves for all x ah, ah = a priori values corresponding to xh and h, and
variables at once is expected to yield the most accurate estimates and z1, z2, z3 = goodness-of-fit functions.
is thus desirable. The simulation model, f(), is a function of the O-D flows, the net-
Previous work (14–17 ) has clearly demonstrated the benefits of work Gh, and model parameters h up to interval h. The terms lhx, l βh,
calibrating O-D flows and all other simulation parameters together uhx, and uβh represent lower and upper bounds on the O-D flows and
in the context of dynamic traffic assignment (DTA) systems. Here the model parameters.
methodology is adapted for the simultaneous calibration of all demand A priori parameter values, when available, can be used to ensure
and supply models within a microscopic traffic simulation model reasonable calibrated estimates. Such values can be based on the
using aggregate, time-varying traffic measurements. The method modeler’s experience and judgment, may be obtained from past
accurately captures the complex relationships between various model studies, or can be transferred appropriately from similar studies at
parameters and data without using any linear approximations such other locations. Individual parameters may also be subject to lower
as the assignment matrix. The approach also provides the flexibility to and upper bounds. For example, O-D flow variables must be non-
use any general traffic data to estimate all parameters and facilitates negative, and the travel time coefficient in a route choice model can
the simultaneous estimation of O-D flows from multiple time inter- be constrained to be negative.
vals. This latter capability is a requirement on large and congested The direct use of the simulation model’s output accurately captures
networks (when a majority of the counts measured in a particular the complex nonlinear dependencies between the variables (xh and h)
interval are caused by O-D flows departing in previous intervals),
and the data (Mh). This approach is superior to existing methods
yet computational limitations restrict a similar approach based on
that approximate the complex relationships between the calibration
assignment matrices.
variables and traffic measurements. In traditional O-D estimation
The detailed nature of microscopic traffic simulation models
approaches, for example, the mapping between the unknown O-D
generally results in higher running times and greater stochasticity
flows and the observed link counts is replaced by a linear measurement
than in more aggregate (mesoscopic or macroscopic) models. The
equation:
calibration of such models has therefore typically been performed
by focusing on a small subset of parameters and a data set suited to h
that subset. For example, car-following and lane-changing model yh = ∑ a hp x p + v h (3)
parameters are generally estimated with detailed trajectory data p= h − p '

available for just a few locations on the network, whereas O-D flows
are estimated with count data. An efficient methodology and solution where
approach to simultaneously estimate all parameters of interest using yh = vector of link sensor counts for interval h,
readily available sensor data, and in a reasonable time, will there- a hp = assignment matrix,
fore be invaluable to both practitioners and researchers. The main vh = error vector, and
contribution of this study is a demonstration of the feasibility and
p′ = number of intervals spanning longest trip.
benefits of such an approach when applied to a large-scale traffic
microsimulation model. The resulting problem is typically solved with a generalized least-
squares approach (18, 19).
Assignment matrices may be computed within the traffic simulation
METHODOLOGY
model by tracking vehicle trajectories or by measuring the simulated
A systematic calibration methodology [reported elsewhere (14–17)] travel times from each demand origin node to every sensor location
is presented that possesses several advantages over existing methods. (20). However, since the simulated assignment matrices depend on
The time period of interest is divided into intervals h = 1, 2, . . . , H, xh the starting demand profile, this O-D estimation formulation requires
denotes the vector of O-D flows departing their respective origins the solution of a fixed-point problem [see paper by Cascetta and
during time interval h, and h is the vector of simulation model Postorino (21), for example]. In the approach proposed here, the
parameters that must be calibrated together with the O-D flows. The assignment mappings are captured directly by treating the simulation
calibration problem may then be formulated mathematically in the model as a black box.
following optimization framework: The proposed approach introduces the flexibility to easily incor-
porate any general traffic measurements, beyond the standard link
Minimize counts. For example, speeds and densities, also recorded by loop
detectors, may be added to the data set. More sophisticated data
H
z ( x1 , ..., x H , 1 , ..., H ) = ∑ ⎡⎣z1 ( M h , Mh ) + z 2 ( x h , x ah ) + z3 (h , ah )⎤⎦ such as point-to-point counts and travel times from automatic vehi-
h =1 cle identification (AVI) systems and probe vehicles may also be
(1) incorporated. Further, all model inputs and parameters of interest
200 Transportation Research Record 1999

may be calibrated simultaneously, fully using all information con- putational cost is high. Studies have shown that SPSA and FDSA
tained in the available measurements and without the need to iterate require a comparable number of iterations to reach the global optimum.
between various parameter subsets. The computational savings of SPSA per iteration thus result in a sig-
nificantly more efficient algorithm for large-scale applications. Spall
shows, through several standard stochastic problems, that SPSA
SOLUTION APPROACHES outperforms FDSA in terms of overall convergence speed. The per-
formance of SPSA for the calibration of large-scale traffic simulation
Equations 1 and 2 together represent a complex, nonlinear, non- models has also been demonstrated in studies by Balakrishna (14)
analytical optimization problem, owing to the use of a sophisticated and by Balakrishna et al. (15, 16); more details about the algorithm
simulator to obtain the fitted measurements. The high degree of non- are given therein.
linearity introduces an objective function with potentially many local A case study comparing genetic algorithms (GAs) and SPSA
optima. The local minimum closest to the starting solution may thus for calibrating microscopic models is presented by Ma et al. (13).
be far from a global optimum. The nonanalytical nature is attributed However, their application is limited to a 2-mi freeway corridor over
to the lack of an explicit form for f( ) as a function of the calibration a 1-h study period. A small set (up to 15) of supply-side parameters is
variables. Consequently, classical algorithms that rely on the knowl- calibrated assuming that O-D demand and other model parameters are
edge of exact analytical gradients are not suitable for calibration. calibrated elsewhere. The authors report very long running times for
Methods that work directly with function values are more appropriate the GA despite the small problem instance, which can affect its scal-
in the context of the current application. When the simulator is stochas- ability to large networks. Further, the simulation output generated
tic (as is often the case), the optimization problem must also account with SPSA-calibrated parameters shows flow–occupancy values lower
for the inherent noise in model outputs. Further, the problem is large than the observed values.
in scale, with the number of O-D pairs and time intervals increasing
rapidly with the size of the network and the desired temporal model-
ing resolution. Consequently, appropriate solution algorithms must CASE STUDY
be able to
The presented methodology is applied to the calibration of the
• Perform a global search by overcoming local hills and valleys, MITSimLab microscopic traffic simulator (26, 27). The case study
• Work without analytical derivatives (which would generally be is based on the freeway and parkway network of Lower Westchester
unavailable), and County (LWC), New York. Drivers there experience heavy traffic
• Converge in a reasonable time frame that does not grow rapidly conditions, especially during commute periods. The main arteries
with problem size. in the network include the New York State Thruway (I-87), New
England Thruway (I-95), Cross Westchester Expressway (I-287),
The proposed calibration framework can be solved with several Cross County Parkway, Hutchinson River Parkway, Sprain Brook
alternative approaches. Population-based simulation optimization Parkway, Saw Mill River Parkway, Bronx River Parkway, and Taconic
methods such as Box–complex (22) and stable noisy optimization by State Parkway. Four adjoining arterials (Tuckahoe Road, Ardsley
branch and fit (SNOBFIT) (23) are ideally suited for such applications, Road, Hartsdale Road, and Weaver Street) and Routes 9, 22, 100,
since they work directly with function evaluations and do not require and 119 provide alternate routes (Figure 1).
the calculation of derivatives. Such methods maintain a set of points Truck traffic is prohibited from parkways. Given the significant
(and their function values) and systematically traverse the feasible truck percentage in the network traffic, it was decided that passenger
search space to potentially identify global optima. Although their cars and trucks should be treated independently; that is, multiclass
capabilities have been demonstrated in other areas, the application of calibration would be needed.
these algorithms to complex and large-scale transportation problems The network representation of the study area comprises 1,767
is limited. directed links, further subdivided into 2,564 segments that capture
The calibration problem has been established as a large-scale changing link characteristics. The data that were available for the
iterative problem optimizing an output from a stochastic simulator. calibration process included counts from 33 sensors and an all-day
Therefore stochastic approximation techniques can also be used in static, planning-level, O-D matrix, provided by the New York State
the calibration context. The finite difference stochastic approximation Department of Transportation. Dynamic O-D profiles, derived from
(FDSA) method computes a search direction by comparing the objec- this static matrix in a previous study (28), were used as starting values.
tive function after perturbing each variable individually. Although Disaggregate data from toll plazas were also available, consisting of
FDSA is an option with proven performance, it is associated with individual vehicle observations recorded at the toll plazas. These data
considerable computational complexity: each gradient calculation therefore contained records of each vehicle’s type, which were used
involves at least n + 1 function evaluations, where n is the number for the estimation of the vehicle mix (and subsequently employed in
of parameters to be calibrated. the development of multiclass O-D matrices). There are 482 O-D
An optimization algorithm developed by Spall (24; 25, pp. 529–542) pairs in the network. The sensor locations are shown in Figure 1.
specifically addresses stochastic problems. Simultaneous perturbation
stochastic approximation (SPSA) provides significant computational
improvements over FDSA by perturbing all variables at once. SPSA Data Processing and Analysis
requires only two computations of the objective function at a given
iteration for estimating the gradient vector, irrespective of the The available surveillance data are limited and include counts at sev-
number n of parameters to be calibrated. eral locations in the network. Because of the different technologies,
Although FDSA is expected to give a more accurate and reliable sensor setup, and data-collection infrastructure, data are available at
estimate of the gradient vector at each iteration, the associated com- different levels of aggregation. Data from most sensors are available
Balakrishna, Antoniou, Ben-Akiva, Koutsopoulos, and Wen 201

2-way Toll Plaza

1-way Toll Plaza

Sensors:

Northbound

Southbound

Eastbound
Westbound

FIGURE 1 Sensor locations used for MITSimLab calibration.

in hourly intervals, whereas some sensors’ data are available in 15-min continuity. Because of different traffic patterns, sensors were grouped
intervals. Because of these limitations, extensive data processing geographically, and smoothing was performed for each group. For
and analysis was undertaken, consisting of several steps. Figure 2 each 15-min interval, the Euclidean norm of the vector of the counts
summarizes the data preparation approach. Hourly data were first in each sensor group was calculated. These norms were then fed into
disaggregated and then smoothed. The data from individual days a seven-point Savitzky–Golay filter with degree 2, which generated
were analyzed and weekdays were pooled together and separated smoothed norms (29). This approach preserves the features of the dis-
from weekend data, since a clear distinction was observed. Vehicle tribution such as relative maxima, minima, and width. The smoothed
mix information was incorporated in order to separate the data into norm of each interval was then divided by the original norm to obtain
two classes: passenger cars and heavy vehicles. the corresponding smoothing coefficient. Smoothed counts were
The disaggregation process involved breaking down the hourly obtained by multiplying the sensor counts in the same interval with
counts into 15-min counts and smoothing the results in order to ensure this coefficient. Figure 3 (top) shows the sensor count data for one
202 Transportation Research Record 1999

Original hourly data Disaggregation Disaggregate data

Pooling Smoothed data Smoothing

Vehicle mix Multi-class data

Weekday data classification

FIGURE 2 Data processing methodology.

of the sensors in the original disaggregated form (3a), as well as the (trucks and commercial vehicles). Vehicle classification counts (the
result after smoothing (3b). It was verified that the total flow per hour proportion of each vehicle type) are important inputs to O-D esti-
was preserved after smoothing. Data from the toll plazas were of a mation, since the route choice process for various vehicle types could
much finer interval and were aggregated into 15-min counts. differ. For example, heavy vehicles are not permitted to use the
It was important to segregate the sensor counts on weekdays from parkways. The availability and implementation of this information
those on weekends or other holidays, since the prevalent demand are critical for O-D estimation, or the model would otherwise not
patterns over these two kinds of days can be very different. Figure 3c reflect actual field situations: if all vehicle types on the network
and d demonstrates that the weekdays show similar traffic patterns. followed a similar route choice process, it would become theoreti-
Hence, all weekday data were combined for calibrating a typical cally impossible to match the observed sensor counts. Hence, the
weekday. ratio of commercial traffic to passenger-car traffic must be specified
The LWC network consists of predominantly freeway and park- for each O-D pair and can be used to form two separate O-D matrices
way traffic that contains automobile commuters and heavy vehicles (one for each of the two vehicle types). These O-D matrices can

700 700
600 600
Smoothed Counts

500 500
Counts

400 400
300 300
200 200
100 100
0 0
0 5 10 15 20 25 30 0 5 10 15 20 25 30
Interval ID Interval ID
(a) (b)
200 400 600 800 1000
1500

Sensor Counts (veh/15 min)

Sensor Counts (veh/15 min)
0 500 1000

4:00 6:00 8:00 10:00 12:00 4:00 6:00 8:00 10:00 12:00
Time (a.m.) Time (a.m.)
(c) (d)

FIGURE 3 Sensor count smoothing sample for (a) disaggregated form and (b) after smoothing; indicative
weekday sensor count distribution for (c) sensor E0115N and (d) sensor M0036S.
Balakrishna, Antoniou, Ben-Akiva, Koutsopoulos, and Wen 203

900
No. of Cars
800
No. of Trucks
700

600

Counts
500

400

300

200

100

000
04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00
Time Interval (a.m.)

FIGURE 4 Vehicle mix: number of trucks versus number of cars by time interval.

then be loaded onto the network, with different individual route where
choice models.
N = number of observations,
Data from toll plazas were used to assign vehicles to two classes
Y no = observation, and
(Figure 4). The toll plazas, though few in number compared with the
Yns = simulated value at time n.
total number of sensors in the network, were the only source of data
on vehicle classification counts. There were no other plausible ways RMSPE is calculated as follows:
of obtaining reliable classification counts by O-D pair. The average
heavy-vehicle proportion among all toll sensors for each time inter- 2
1 N
⎡ Yns − Yno ⎤
val was therefore used as an approximation of the vehicle mix ratio
for every O-D pair for that corresponding time interval.
RMSPE =
N
∑ ⎢
n =1 ⎣ Yno ⎦
⎥ (5)

The MPE statistic indicates the existence of systematic under-

Measures of Goodness of Fit or overprediction in the simulated measurements and is calculated
as follows:
The following statistics were used (30–33):
1 N
⎡ Yns − Yno ⎤
• Normalized root-mean-square error (RMSN) (32, 33); MPE =
N
∑ ⎢⎣ Yno ⎦
⎥ (6)
• Root-mean-square percent error (RMSPE) (34); n =1

• Mean percent error (MPE) (34);

• Theil’s U-coefficient, with its bias, variance, and covariance Percent error measures are often preferred to their absolute error
components (35); and counterparts because they provide information on the magnitude of
• The GEH statistic, used in traffic engineering, traffic forecasting, the errors relative to the average measurement. Another measure
and traffic modeling to compare sets of traffic volumes (36). that provides information on the relative error is Theil’s inequality
coefficient:
Multiple statistics are used to capture different aspects of the
obtained results. For example, two sets of values are considered: N

∑ (Y − Yno )
1 s 2
5 and 8, and 50 and 72. A statistic measuring the percent change in n
N
these two sets of values would suggest that a bigger change has been U= n =1
(7)
N N

∑ (Y ) ∑ (Y )
obtained in the first case (60% versus 44%). However, a statistic focus- 1 s 2 1 o 2
n + n
ing on the actual numerical increase would suggest that a bigger N n =1 N n =1
change is obtained in the second case (22 versus 3).
The RMSN and RMSPE quantify the overall error of the simulator. U is bounded between zero and 1 (U = 0 implies perfect fit
These measures penalize large errors at a higher rate than small errors. between observed and simulated measurements). Theil’s inequality
RMSN is calculated as follows: coefficient may be decomposed into three proportions of inequality:
the bias (UM), the variance (US), and the covariance (UC ) proportions:
N
N ∑ (Y ns − Y no )
2

(Y −Y o)
s 2

RMSN = n =1
N
(4) UM = N
(8)
∑Y o
n
1
N
∑ (Y n
s
−Y n
o 2
)
n =1 n =1
204 Transportation Research Record 1999

(s − so ) (without performing any calibration iterations) and the smoothed

s 2

US = N
(9) observed counts are reported as the before case in this table. The after
∑ (Y − Yno )
1 s 2
n
case corresponds to the difference between the counts obtained by
N n =1 loading the final calibrated O-D matrices to MITSimLab from the
smoothed observed counts. The reported percent change between the
2 (1 − ρ) s s s o fit of the simulated counts by using the seed matrix and the simulated
UC = N
(10)
∑ (Y − Yno )
1 s 2 counts by using the calibrated matrix is meant to demonstrate that
n
N n =1
the calibration process performs as expected and has the capability
to steer the seed O-D matrix toward the values that will produce
where ρ is the correlation between the two sets of measurements, simulated counts that will best match the observed measurements.
and ss and so are the standard deviations of the average simulated and RMSN and RMSPE capture the deviation of the simulated counts
observed measurements, respectively. By definition, the three pro- from their observed counterparts in a manner that penalizes higher
portions sum to 1. The bias proportion reflects the systematic error. errors more than smaller errors. The percent change in terms of RMSN
The variance proportion indicates how well the simulation model is ranges between 25% and 71% for the various 15-min time intervals,
able to replicate the variability in the observed data. These two pro- with overall RMSN improvement equal to 51%. Similarly, the per-
portions should be kept as close to zero as possible. The covariance cent change in terms of RMSPE ranges between 22% and 65%, with
proportion measures the remaining error and therefore should be close an overall improvement of 47%.
to 1. Since the various measurements are taken from nonstationary Besides capturing the deviation around the observed values, MPE
processes, the proportions can only be viewed as rough indicators to provides a measure of the direction of these errors, namely, whether
the sources of error. there is a distinct bias toward under- or overestimation. A clear
The statistics listed, while widely used, are sometimes perceived negative bias was evident in the results obtained by the starting values,
to be ineffective if the network consists of several roadway func- which has been effectively eliminated as a result of the calibration.
tional classes. The GEH statistic overcomes this potential problem In absolute terms, the MPE value has been reduced between 55%
by computing percent errors with respect to the mean value of and 99% for the 15-min intervals, whereas for the entire calibration
the observed and the simulated counts. The formula for the GEH period the improvement is on the order of 82%.
statistic is (36) Theil’s U should be as close to zero as possible, and indeed the
calibration procedure decreases it by 57% overall for the entire
calibration period. Improvement in the 15-min intervals ranges
2 (Yns − Yno )
2
between 29% and 75%. The first two components of the coefficient
GEH = (11)
(Y n
s
+ Yno ) are presented; the third one can be obtained as UC = 1 − UM − US.
The overall improvement of the bias component (UM) is 75% for
where Y sn is the simulated traffic volume and Y no is the observed the entire time period, and its improvement in the 15-min periods
traffic count. GEH values below 5 are considered a good match ranges between 36% and 100%. The overall improvement of
between estimated and observed counts. According to FHWA guide- the variance component (US) is 27%. The improvement in all but
lines (36), at least 85% of the observed links in a traffic model should two 15-min periods ranges between 35% and 100%. However, it
have a GEH less than 5.0. If the GEH is greater than 10.0, there is should be noted that for two time periods (6:15 to 6:30 and 7:15 to
a probability of error or errors with either the travel demand model 7:30) the value of the variance component decreases considerably
or the data. (highlighted values).
Traffic problems involving general networks present an additional
difference that often challenges the appropriateness of the foregoing
CALIBRATION RESULTS statistics. For example, traffic flows may have different magnitudes,
with freeway flows in the thousands and arterial flows in the low
Owing to the limited amount of data available, the methodology hundreds or less. The GEH statistic provides an attractive and prac-
presented earlier was applied for the calibration of time-dependent tical “self-scaling” alternative. Table 1 presents the GEH statistic both
O-D matrices. Flows between 482 O-D pairs were estimated for each before and after calibration. The overall improvement is consistent
15-min departure time interval. The SPSA algorithm was selected with the improvement of the previous statistics; it ranges between
for its proven performance and computational properties in large- 30% and 74%, with an overall improvement of 56%. The overall
scale problems (14–16). For the driving-behavior model parameters, GEH value for the analysis period is less than the threshold of 5.0
values obtained in earlier studies (1) were used. Sensitivity analyses recommended by FHWA guidelines (36), although individual GEH
of these parameters confirmed that the selected values were appro- values are also close to this value.
priate for this network. A dynamic O-D matrix calibrated in another Figure 5 presents scatterplots of an indicative subset of the same
study (28) in the same network was used as a seed. The O-D calibra- results. Each point in these plots represents the counts for one sensor,
tion estimated a single flow per time interval for each O-D pair, which with the observed value plotted on the x-axis and the corresponding
was further decomposed into two components (passenger cars and simulated value plotted on the y-axis. A diagonal (45-degree) line
trucks) before being loaded onto MITSimLab. Time-dependent truck indicates the location of the points in the case of perfect fit. Ideally,
percentages from the observed toll-plaza data were used for this all points would lie on that line. The goal of successful calibration
purpose. is to bring these points as close as possible to this line. Further-
The results for each 15-min interval in the 6:00 to 9:00 a.m. analy- more, the deviations of the points should be as balanced as possible
sis period are summarized in Table 1. The difference between the above and below the line, to indicate that no bias is evident in the
counts obtained by loading the seed O-D matrices to MITSimLab calibration.
Balakrishna, Antoniou, Ben-Akiva, Koutsopoulos, and Wen 205

TABLE 1 Calibration Results Statistics

RMSN RMSPE MPE

Time
Period Before After % Change Before After % change Before After % Changea

6:00–6:15 0.537 0.179 67 0.535 0.216 60 −0.474 0.004 99

6:15–6:30 0.219 0.158 28 0.224 0.172 23 −0.128 −0.006 95
6:30–6:45 0.208 0.155 26 0.190 0.148 22 −0.137 −0.044 68
6:45–7:00 0.263 0.198 25 0.253 0.181 29 −0.171 −0.052 70
7:00–7:15 0.405 0.248 39 0.381 0.225 41 −0.271 −0.121 55
7:15–7:30 0.528 0.245 54 0.443 0.237 47 −0.335 −0.118 65
7:30–7:45 0.553 0.236 57 0.454 0.217 52 −0.353 −0.120 66
7:45–8:00 0.546 0.160 71 0.453 0.157 65 −0.337 −0.062 82
8:00–8:15 0.656 0.236 64 0.638 0.230 64 −0.614 −0.070 89
8:15–8:30 0.435 0.249 43 0.390 0.242 38 −0.262 −0.065 75
8:30–8:45 0.427 0.285 33 0.409 0.294 28 −0.199 −0.011 94
8:45–9:00 0.420 0.268 36 0.403 0.281 30 −0.192 0.030 84
6:00–9:00 0.476 0.231 51 0.416 0.221 47 −0.289 −0.053 82

U UM US
Time
Period Before After % Change Before After % Change Before After % Change

6:00–6:15 0.316 0.084 73 0.769 0.003 100 0.008 0.001 91

6:15–6:30 0.111 0.076 32 0.462 0.064 86 0.083 0.140 −68
6:30–6:45 0.106 0.075 29 0.463 0.127 72 0.045 0.000 100
6:45–7:00 0.136 0.097 29 0.451 0.115 75 0.004 0.000 99
7:00–7:15 0.225 0.127 44 0.526 0.315 40 0.025 0.016 35
7:15–7:30 0.304 0.126 59 0.488 0.312 36 0.007 0.024 −240
7:30–7:45 0.325 0.119 63 0.509 0.301 41 0.016 0.000 100
7:45–8:00 0.317 0.079 75 0.480 0.195 59 0.011 0.001 91
8:00–8:15 0.436 0.118 73 0.862 0.161 81 0.029 0.015 50
8:15–8:30 0.245 0.124 49 0.511 0.143 72 0.087 0.023 74
8:30–8:45 0.233 0.139 40 0.373 0.037 90 0.038 0.007 82
8:45–9:00 0.227 0.127 44 0.355 0.000 100 0.016 0.005 72
6:00–9:00 0.264 0.113 57 0.461 0.116 75 0.020 0.015 27

GEH
Time
Period Before After % Improved

6:00–6:15 13.27 3.51 74

6:15–6:30 4.71 3.19 32
6:30–6:45 4.47 3.15 30
6:45–7:00 6.24 3.92 37
7:00–7:15 10.57 5.82 45
7:15–7:30 13.43 5.98 55
7:30–7:45 13.89 5.39 61
7:45–8:00 13.89 3.67 74
8:00–8:15 21.88 5.67 74
8:15–8:30 11.47 6.08 47
8:30–8:45 11.28 6.67 41
8:45–9:00 10.68 6.24 42
6:00–9:00 11.31 4.94 56

aIn absolute values

CONCLUSION map O-D flows and other model parameters to traffic measurements
are instead captured implicitly through the black-box simulation-
Microscopic traffic simulation models are a mature and proven based assignment model itself. This approach allows the estimation
technology with applications in research and practice. Calibration procedure to utilize richer data other than link counts, thus improv-
of such models is a key aspect of any application. The detailed ing the accuracy and efficiency of the resulting flow and parameter
models in microscopic traffic simulators make calibration difficult, estimates. The formulation is flexible to accommodate the simul-
especially in the absence of adequate and reliable surveillance data. taneous estimation of O-D flows and model parameters that affect
The application of a systematic traffic simulation model calibration O-D estimation, such as route choice and supply parameters. Algo-
methodology is presented here that does not rely on the traditional rithms to solve the highly nonlinear and large-scale problem are
assignment matrix approximation. The complex transformations that outlined, and their feasibility is demonstrated through a real case
206 Transportation Research Record 1999

2000 2000

Simulated Counts (veh/15 min)

Simulated Counts (veh/15 min)
1500 1500

1000 1000

500 500

0 0
0 500 1000 1500 2000 0 500 1000 1500 2000
Observed Counts (veh/15 min) Observed Counts (veh/15 min)

2000 2000

Simulated Counts (veh/15 min)

Simulated Counts (veh/15 min)

1500 1500

1000 1000

500 500

0 0
0 500 1000 1500 2000 0 500 1000 1500 2000
Observed Counts (veh/15 min) Observed Counts (veh/15 min)

FIGURE 5 Indicative calibration results.

study involving a detailed microscopic traffic simulation model REFERENCES

and a large, complex freeway network.
The calibration methodology is applied in a large-scale network 1. Toledo, T., H. N. Koutsopoulos, A. Davol, M. E. Ben-Akiva, W. Burgh-
in Lower Westchester County, New York, and practical aspects and out, I. Andreasson, T. Johansson, and C. Lundin. Calibration and Vali-
dation of Microscopic Traffic Simulation Tools: Stockholm Case Study.
difficulties of model calibration are identified. A multistep data In Transportation Research Record: Journal of the Transportation Re-
processing and analysis approach is presented that overcomes the search Board, No. 1831, Transportation Research Board of the National
limitations associated with the available data. An interesting aspect Academies, Washington, D.C., 2003, pp. 65–75.
of this application is the use of a multiclass O-D matrix to model 2. Ozbay, K., B. O. Bartin, and S. Mudigonda. Microscopic Simulation and
Calibration of an Integrated Freeway and Toll Plaza Model. Presented at
parkway access restrictions for heavy vehicles. The calibration results 85th Annual Meeting of the Transportation Research Board, Washington,
show that this process is successful in matching prevailing traffic D.C., 2006.
conditions. 3. Liu, H. X., P. Chootinan, A. Chen, J. X. Ban, and L. Ding. Streamlined
The available sensor data for Westchester County are sparse Network Calibration Procedure for California SR-41 Corridor Traffic
Simulation Study. Presented at 85th Annual Meeting of the Transportation
relative to the number of links in the network. However, such sensor
Research Board, Washington, D.C., 2006.
coverage levels are not uncommon in real-world situations, under- 4. Smith, M. C., and A. W. Sadek. Challenges Calibrating a Large-Scale
lining the need for a calibration methodology that can extract max- Microscopic Simulation Model of a Diverse Urban, Suburban, and Rural
imum information from limited data. In this context, the proposed Network: A Practical Guide. Presented at 85th Annual Meeting of the
methodology efficiently estimates all model parameters of interest Transportation Research Board, Washington, D.C., 2006.
5. Darda, D. Joint Calibration of a Microscopic Traffic Simulator and
simultaneously. More data will be available for calibration as sensor Estimation of Origin-Destination Flows. Master’s thesis. Massachusetts
coverage improves and is expected to result in increased accuracy. Institute of Technology, 2002.
An important advantage of the proposed method is its ability to 6. Mahanti, B. Aggregate Calibration of Microscopic Traffic Simula-
handle any type and amount of sensor data. tion Models. Master’s thesis. Massachusetts Institute of Technology,
Cambridge, 2004.
Future research should consider the impact of additional and more 7. Zhang, M. H., J. Ma, and H. Dong. Calibration of Departure Time and
reliable surveillance data and test the sensitivity of the calibration Route Choice Parameters in Microsimulation with Macro Measure-
process to such information. Such an analysis for DTA models is ments and Genetic Algorithm. Presented at 85th Annual Meeting of the
available elsewhere (14). In particular, the impact of AVI and probe- Transportation Research Board, Washington, D.C., 2006.
8. Abdulhai, B., J. B. Sheu, and W. Recker. Simulation of ITS on the Irvine
vehicle information should be considered. AVI and probe-vehicle data FOT Area Using the ‘PARAMICS 1.5’ Scalable Microscopic Traffic
have already been successfully used for O-D estimation (37, 38) and Simulator. Phase I: Model Calibration and Validation. PATH Research
DTA models (39). Report UCB-ITS-PRR-99-12. University of California, Berkeley, 1999.
Balakrishna, Antoniou, Ben-Akiva, Koutsopoulos, and Wen 207

9. Kurian, M. Calibration of a Microscopic Traffic Simulator. Master’s 25. Spall, J. C. Stochastic Optimization, Stochastic Approximation and Sim-
thesis. Massachusetts Institute of Technology, Cambridge, 2000. ulated Annealing. In Wiley Encyclopedia of Electrical and Electronics
10. Lee, D.-H., X. Yang, and P. Chandrasekr. Parameter Calibration for Engineering (J. G. Webster, ed.), Wiley-Interscience, New York, 1999.
PARAMICS Using Genetic Algorithm. Presented at 80th Annual Meeting 26. Yang, Q., and H. N. Koutsopoulos. A Microscopic Traffic Simulator for
of the Transportation Research Board, Washington, D.C., 2001. Evaluation of Dynamic Traffic Management Systems. Transportation
11. Kim, K.-O., and L. R. Rilett. Genetic-Algorithm-Based Approach to Research, Vol. 4C, 1996, pp. 113–129.
Traffic Microsimulation Calibration Using ITS Data. Presented at 83rd 27. Yang, Q., H. N. Koutsopoulos, and M. E. Ben-Akiva. Simulation Labora-
Annual Meeting of the Transportation Research Board, Washington, tory for Evaluating Dynamic Traffic Management Systems. In Trans-
D.C., 2004. portation Research Record: Journal of the Transportation Research
12. Brockfeld, E., R. D. Kuhne, and P. Wagner. Calibration and Validation Board, No. 1710, TRB, National Research Council, Washington, D.C.,
of Microscopic Traffic Flow Models. In Transportation Research Record: 2000, pp. 122–130.
Journal of the Transportation Research Board, No. 1876, Transportation 28. Akhil, C. Development and Evaluation of Diversion Strategies Under
Research Board of the National Academies, Washington, D.C., 2005, Incident Response Using Dynamic Traffic Assignment System. Master’s
pp. 62–70. thesis. Massachusetts Institute of Technology, Cambridge, 2003.
13. Ma, J., H. Dong and H. M. Zhang. Calibration of Microsimulation with 29. Savitzky, A., and M. J. E. Golay. Smoothing and Differentiation of Data
Heuristic Optimization Methods. In Transportation Research Record: by Simplified Least Squares Procedures. Analytical Chemistry, Vol. 36,
Journal of the Transportation Research Board, No. 1999, Transporta- 1964, pp. 1627–1639.
tion Research Board of the National Academies, Washington, D.C., 30. Toledo, T. Integrated Driving Behavior Modeling. PhD thesis. Massa-
2007, pp. 208–217. chusetts Institute of Technology, Cambridge, 2003.
14. Balakrishna, R. Off-line Calibration of Dynamic Traffic Assignment 31. Toledo, T., and H. N. Koutsopoulos. Statistical Validation of Traffic
Models. PhD thesis. Massachusetts Institute of Technology, 2006. Simulation Models. In Transportation Research Record: Journal of the
15. Balakrishna, R., M. Ben-Akiva, and H. N. Koutsopoulos. Time-Dependent Transportation Research Board, No. 1876, Transportation Research Board
Origin-Destination Estimation Without Assignment Matrices. Presented of the National Academies, Washington, D.C., 2004, pp. 142–150.
at Second International Symposium on Transport Simulation, Lausanne, 32. Ashok, K., and M. Ben-Akiva. Alternative Approaches for Real-Time
Switzerland, 2006. Estimation and Prediction of Time-Dependent Origin–Destination Flows.
16. Balakrishna, R., M. Ben-Akiva, and H. N. Koutsopoulos. Offline Cali- Transportation Science, Vol. 34, No. 1, 2000, pp. 21–36.
bration of Dynamic Traffic Assignment: Simultaneous Demand-and- 33. Ashok, K., and M. Ben-Akiva. Estimation and Prediction of Time-
Supply Estimation. In Transportation Research Record: Journal of the Dependent Origin-Destination Flows with a Stochastic Mapping to Path
Transportation Research Board, No. 2003, Transportation Research Flows and Link Flows. Transportation Science, No. 36, No. 2, 2002,
Board of the National Academies, Washington, D.C., 2007, pp. 50–58. pp. 184–198.
17. Balakrishna, R., H. N. Koutsopoulos, and M. Ben-Akiva. Simultaneous 34. Pindyck, R. S., and D. L. Rubinfeld. Econometric Models and Economic
Offline Demand and Supply Calibration of Dynamic Traffic Assignment Forecasts, 4th ed. McGraw-Hill, New York, 1997.
Systems. Presented at 85th Annual Meeting of the Transportation Research 35. Theil, H. Economic Forecasts and Policy. North-Holland, Amsterdam,
Board, Washington, D.C., 2006. Netherlands, 1961.
18. Cascetta, E., and S. Nguyen. A Unified Framework for Estimating or 36. Traffic Analysis Toolbox, Volume III: Guidelines for Applying Traffic
Updating Origin/Destination Matrices from Traffic Counts. Transportation Microsimulation Modeling Software. Publication FHWA-HRT-04-040.
Research, Vol. 22B, No. 6, 1988, pp. 437–455. FHWA, U.S. Department of Transportation, 2004. http://ops.fhwa.dot.
19. Cascetta, E., D. Inaudi, and G. Marquis. Dynamic Estimators of Origin– gov/trafficanalysistools/tat_vol3/index.htm. Accessed Nov. 15, 2006.
Destination Matrices Using Traffic Counts. Transportation Science, 37. Antoniou, C., M. Ben-Akiva, and H. N. Koutsopoulos. Incorporating
Vol. 27, No. 4, 1993, pp. 363–373. Automated Vehicle Identification Data into Origin-Destination Estimation.
20. Ashok, K. Estimation and Prediction of Time-Dependent Origin- In Transportation Research Record: Journal of the Transportation
Destination Flows. PhD thesis. Massachusetts Institute of Technology, Research Board, No. 1882, Transportation Research Board of the National
Cambridge, 1996. Academies, Washington, D.C., 2004, pp. 37–44.
21. Cascetta, E., and M. N. Postorino. Fixed Point Approaches to the Esti- 38. Antoniou, C., M. Ben-Akiva, and H. N. Koutsopoulos. Dynamic Traffic
mation of O/D Matrices from Traffic Counts on Congested Networks. Demand Prediction Using Conventional and Emerging Data Sources.
Transportation Science, Vol. 35, No. 2, 2001, pp. 134–147. IEE Proceedings Intelligent Transport Systems, Vol. 153, No. 1, 2006,
22. Box, M. J. A New Method of Constrained Optimization and a Comparison pp. 97–104.
with Other Methods. The Computer Journal, Vol. 8, No. 1, 1965, pp. 42–52. 39. Antoniou, C., B. Nowotny, A. Rechbauer, and M. Linauer. Calibration of
23. Huyer, W., and A. Neumaier. SNOBFIT—Stable Noisy Optimization by DTA Models Using Floating Car Data: An Application of DynaMIT in
Branch and Fit. ACM Transactions on Mathematical Software (submitted Vienna. Presented at Second International Symposium in Transportation
2004). Simulation, Lausanne, Switzerland, 2006.
24. Spall, J. C. Implementation of the Simultaneous Perturbation Algorithm
for Stochastic Approximation. IEEE Transactions on Aerospace and The Traffic Flow Theory and Characteristics Committee sponsored publication of
Electronic Systems, Vol. 34, No. 3, 1998. this paper.