Funke 1

SAFETY IMPACT METHODOLOGY (SIM): APPLICATION AND RESULTS OF THE

ADVANCED CRASH AVOIDANCE TECHNOLOGIES (ACAT) PROGRAM

James Funke
National Highway Traffic Safety Administration
Gowrishankar Srinivasan
Raja Ranganathan
August Burgett (Retired)
Bowhead Systems Management
United States of America
Paper Number 11-0367


ABSTRACT

This paper provides a summary of four cooperative
research projects conducted under the National
Highway Traffic Safety Administrations (NHTSA)
Advanced Crash Avoidance Technologies (ACAT)
program. The ACAT program sought to determine
the safety impact of new and emerging crash
avoidance technologies that are intended to help
drivers avoid crashes, reduce crash severity, and
prevent injuries and fatalities. This research
developed and applied a Safety Impact Methodology
(SIM) framework to estimate safety benefits for the
proposed pre-production crash avoidance systems.

This paper presents the application and results of the
Safety Impact Methodology for four different crash
avoidance technologies including: Advanced
Collision Mitigation Braking System by Honda, Lane
Departure Warning by Volvo-Ford, Pre-collision
Safety System by Toyota, and Backing Crash
Countermeasures by General Motors.

INTRODUCTION

Advanced crash avoidance technologies help drivers
to avoid crashes or, if the crash is unavoidable, to
reduce the harm of the crash. Crash avoidance
systems are able to warn the driver of dangerous
situations and pro-actively deploy countermeasures
before a crash occurs. These countermeasures may
include warnings (by means of haptic, auditory, or
visual alarms) and/or actively controlling the vehicle
(by braking or steering) for a limited time.

Oftentimes, in cases of driver inattention (e.g. driver
distraction), the countermeasure would occur prior to
the driver sensing a critical situation, giving the
driver additional time to react. It is this additional
time margin which enables the driver, in coordination
with the crash avoidance system, to avoid or mitigate
the crash. This is important as it was observed in the


100-Car Naturalistic Driving Study that nearly 80%
of crashes and 65% of near-crashes involved some
form of driver inattention within three seconds before
the event [1].

Developing approaches for crash avoidance safety
technologies is challenging in that, prior to
significant market penetration, it is difficult to
determine real world effectiveness and safety benefits
of new technologies. While there are numerous
challenges, the agency believes that it has a role in
encouraging the development and deployment of all
beneficial safety technologies especially, crash
avoidance technologies [2].

Crash avoidance technologies are moving from the
development phase to the deployment phase at an
accelerated pace. Although the potential of these
advanced technologies to reduce crashes, fatalities,
and injuries is great, their effectiveness is largely
unknown. In order to better understand the potential
safety impact of crash avoidance systems, NHTSA
started the Advanced Crash Avoidance Technologies
(ACAT) program in September 2006.

The ACAT program was established to identify new
or emerging advanced technologies and to estimate
the safety impact of these technologies. In support of
this goal, this research program had two main
objectives. The first objective was to develop and
utilize a Safety Impact Methodology (SIM) to
evaluate the ability of advanced technology
applications to solve specific motor vehicle safety
problems. The second objective was to demonstrate
how the results of objective tests can be used by the
SIM to assess the safety impact of a real system.

NHTSA entered into a cooperative research
agreement with four partners in the automotive
industry. The cooperative agreement partners joined
with other subcontractors in industry and academia to
form the following four teams:
Funke 2
- Team 1: Honda R&D Co., Ltd.-Dynamic
Research, Inc. (Honda-DRI) Team for the
Advanced Collision Mitigation Braking System
(A-CMBS)

- Team 2: Volvo Car Corporation-Ford Motor
Company-University of Michigan Transportation
Research Institute (Volvo-Ford-UMTRI) Team
for Lane Departure Warning (LDW)

- Team 3: Toyota Motor Corporation (Toyota)
Team for the Pre-Collision Safety System (PCS)

- Team 4: General Motors Corporation-Virginia
Tech Transportation Institute (GM-VTTI) Team
for Backing Crash Countermeasures

[The] Advanced Crash Avoidance Technologies
(ACAT) program, in which NHTSA partners with
automobile manufacturers to improve information on
safety impacts of crash avoidance technologies, is
intended to encourage manufacturer efforts to
develop such technologies [3]. This research was
completed in the fall of 2009 and final reports from
each team have been submitted. This paper
summarizes the findings of the four ACAT teams.

Each team carried out the following tasks:

- Identification of the safety problem area
- Description of the advanced technology
- Development of SIM tool
- Development of objective tests
- Conduct and analysis of objective tests
- Estimation of safety benefits using SIM tool

Each team was able to evaluate the estimated
effectiveness of their specific pre-production crash
avoidance system using their SIM, using the
framework described below. Each of the approaches
taken by the by the various teams represents a novel
method to predict the estimated safety impact of pre-
production crash avoidance technologies.

NHTSA SIM FRAMEWORK

Using the SIM framework shown in Figure 1, each
team developed their own computational SIM tool
based on this methodology. The framework identifies
the principle components of SIM and interactions
between these components including: Data Usage,
Case Scenarios, Objective Testing, Model Creation,
Data Generation, Countermeasure Performance
Analysis, and Safety Benefits. A short description of
each component and its function is given:
Data Usage identifies the sources and frequencies
of the data used to define the traffic conflict(s) of
interest.

Case Scenarios describe the specific problem area
and corresponding conditions selected for
countermeasure application.

Objective Testing collects empirical data from
test tracks, simulators and other sources to provide
distributions of values for model parameters (driver,
vehicle, and countermeasure).

Model Creation develops a set of computational
equations and executable tools to describe driver-
vehicle operation in normal and conflict related
driving and their respective response both with and
without the countermeasure.

Data Generation applies the model to the
selected scenarios to document the performance of
countermeasure operation versus performance with
no countermeasure.

Countermeasure Performance Analysis
compares the number of events (e.g., crashes)
associated with presence and absence of the
countermeasure to assess system effectiveness.

Safety Benefits reports the estimated benefits in
terms of quantity of events avoided or mitigated.

The framework does not dictate a specific approach
or method. The framework communicates NHTSAs
operational vision of a SIM and the activities
NHTSA identified as critical to developing a sound
methodology. This framework can be adjusted to
accommodate and communicate various approaches
to estimate safety benefits. A more detailed
description of the SIM framework is given in [4].

BASIC SAFETY BENEFIT EQUATIONS

The methodologies used to estimate safety benefits
are based on the benefits equation [5]:

W WO
N N B = (1)

where,
B = benefits, (which can be the number of crashes,
number of fatalities, harm, or other such
measures)
N
wo
= value of this measure, (for example, number
of crashes) that occurs without the system
N
w
= value of the measure with the system fully
deployed
Funke 3
The benefits equation can be rewritten as:

SE N B
WO
= (2)
where,
Nwo = size of the problem addressed, and
SE = effectiveness of the system.

An extension of this idea is that the overall benefits
consist of the sum of benefits across a number of
specific scenarios:

=
i
i WO
E N B
i
(3)
where,
i is an index referring to individual scenarios;
N
woi
= the number of crashes that occur in scenario
i when the ACAT countermeasure is not
available
E
i
= the effectiveness of the ACAT countermeasure
in preventing crashes in scenario i.

ACAT PROGRAM

The ACAT program was a proof-of-concept effort
that sought to determine the feasibility of developing
estimates of effectiveness for specific safety
technologies in the absence of data from real world
crashes or field operational tests. Although these
estimates are provided, the focus of this project was
on the development of the SIM and linking it to the
results of the objective tests. The SIM used the data
available at the time of the study to estimate safety
benefits, the calculation of which involved various
assumptions and limitations.

Note that each team estimated safety benefits using
their own set of equations derived from the basic
benefits equations shown. The target populations and
addressable crashes for each ACAT were distinctly
different as each ACAT team was trying to solve a
different safety problem. Team 1 estimated the
overall effectiveness and safety benefits using the
entire US motor vehicle fleet as the baseline
population. Team 2, Team 3, and Team 4 estimated
the system effectiveness and safety benefits using
specific target populations as the baseline population,
which were different for each technology. Therefore,
each ACAT project should be viewed as an
independent, stand-alone effort.

All four teams implemented a SIM which can be
expressed within the framework stipulated by
NHTSA. A summary table comparing the approaches
of the four teams and the details of their approaches
within the respective components of the SIM
framework are shown in Table 1. This is followed by
a summary of the implementation of the SIM for each
of the four teams, respectively. For more information
please refer to the Final Reports of each project.


Figure 1. NHTSA SIM Framework
Funke 4
Table 1.
Comparison of SIM for the four ACAT teams

SIM
Blocks
Components of
SIM
Framework
Team 1: Advanced
Collision
Mitigation Braking
System (A-CMBS)
Team 2: Lane
Departure
Warning (LDW)
Team 3: Pre-
Collision Safety
System (PCS)
Team 4: Backing
Crash
Countermeasures
D
a
t
a

U
s
a
g
e

Archival Data,
Real world data,
Corporate body
of knowledge
and Technology
characteristics
NASS/CDS, PCDS,
FARS, and GES
data
GES, CDS,
Highway
Performance
Monitoring System
(HPMS), and
RDCW-FOT data
GES, CDS,
FARS and Event
data recorder
(EDR)
FARS, GES, SCI,
Public domain
research, GM
research archives,
and VTTI data
C
a
s
e

S
c
e
n
a
r
i
o
s
Breakdown of
scenarios,
Crash
Characteristics
and Technology
relevant
scenarios
- Vehicle-vehicle,
intersecting paths
- Vehicle-vehicle,
rear-end/forward
impact
- Vehicle-vehicle,
head-on
- Single vehicle,
pedestrian
Inadvertent lane or
road departure
SAP-98
Rear-end
collision
- Lead vehicle
stopped
- Lead vehicle
decelerating
Head-on collision
Collision to
object
10 scenarios (6
pedestrian crashes,
3 vehicle-to-
vehicle crashes,
and 1 vehicle-to-
fixed-object
crashes).
O
b
j
e
c
t
i
v
e

T
e
s
t
i
n
g

Driving
simulator,
test track and
Lab/HMI test
Driving simulator
and lab tests
involved. Tests
include Guided Soft
Target - vehicle
conflict tests using
nave and trained
driver.
Driving simulator
tests with nave
subjects to develop
the driver model.
Trained driver tests
for system
validation.
Driving simulator
involved.
(LVS, LVD).
Vehicle tests with
-fixed obstacles
for system
performance
Track and public
road tests involved.
All 10 scenarios
tested. Pedestrian
tests conducted
using mannequins.
M
o
d
e
l

C
r
e
a
t
i
o
n

Model
definition,
validation and
calibration
Indigenous
simulation model.
Cases validated
against automated
reconstruction and
simulation
Using distribution
of parameters.
Model generated
with Matlab/
Simulink/CarSIM
Model validated
from test track
and EDR data.
Matlab/Simulink
model. Validated
based on previous
corporate
sponsored research
D
a
t
a

G
e
n
e
r
a
t
i
o
n

Digital
computer
simulation and
simulator
testing
Reconstructed
crashes simulated
with and without the
ACAT with a
sample of typical
drivers.
Monte Carlo
simulation run with
and without ACAT
Simulator Testing
results
Monte Carlo
simulation run
with and without
ACAT
C
o
u
n
t
e
r
m
e
a
s
u
r
e

P
e
r
f
o
r
m
a
n
c
e

A
n
a
l
y
s
i
s

Without
countermeasure,
With
countermeasure,
System
effectiveness.
DeltaV, Crash/No
crash, Exposure
ratio, Prevention
ratio
Crash/No, Crash,
Exposure ratio,
Prevention ratio
Crash/ No crash,
Speed reduction
Crashes avoided,
Fatalities/Injuries
reduced
Crash/No crash,
Prevention ratio
S
a
f
e
t
y

B
e
n
e
f
i
t
s

Safety benefits
Crashes, fatalities,
injuries (Fatality
Equivalents)
Crashes reduced/
mitigated
Crash reduction,
fatalities and
injury reduction.
Crashes
reduced/ mitigated
Funke 5
TEAM 1: ADVANCED COLLISION
MITIGATION BRAKING SYSTEM (A-CMBS)

For the ACAT program, DRI, with support from
Honda, developed a tool to evaluate Hondas
prototype Advanced Collision Mitigation Brake
System (A-CMBS). The A-CMBS addresses four
primary collision types including: intersecting paths,
rear-end/forward impact, head-on, and pedestrian
crashes. The SIM tool provides an estimate of safety
benefits in terms of reduction in crashes, vehicles
involved, and fatalities [6].

Data Usage

The Honda-DRI approach begins with the
construction of a crash scenario database from
archival national accident databases in the United
States such as the National Automotive Sampling
System/Crashworthiness Data System (NASS/CDS)
and Pedestrian Crashworthiness Data System
(NASS/PCDS). The CDS database provides detailed
descriptions of tow-away crashes involving one or
more light passenger vehicles based on in-depth at-
scene crash investigations. The PCDS database
provides detailed descriptions of vehicle-pedestrian
crashes also based on in-depth at-scene crash
investigations. These data are also supplemented by
information from the Fatality Analysis Reporting
System (FARS) for fatal crashes. This crash scenario
database contains in-depth information and time-
space reconstructions of real-world accidents based
on their time-domain relationship. This data was used
to classify the crash scenarios in terms of technology
relevance and to create sub-samples of cases in each
Technology Relevant Crash Type (TRCT).

Case Scenarios

The characterization of the crash scenarios begins
with identifying the combination of driver, vehicle,
and environment dynamics presented during the
crash. Unlike traditional segmentation processes,
Team 1 reconstructs actual scenarios from the
NASS/CDS and PCDS databases using the
Automated Accident Reconstruction Tool (AART)
and segments them into their respective TRCTs. The
AART estimates the time-space relationships of the
Subject Vehicle and Collision Partner trajectories
based on digitized scene diagrams, coded data, and
Newtonian physics combined with a number of
assumptions. The process adopts a stratified sampling
process to select a subset of TRCTs to facilitate
simulation. The primary TRCTs from the AART
reconstructions used for the safety benefit estimation
were:
- Vehicle-vehicle, intersecting paths
- Vehicle-vehicle, rear-end/forward impact
- Vehicle-vehicle, head-on
- Single vehicle, pedestrian

Objective Testing

The purpose of objective testing in the Honda-DRI
ACAT project was to observe and measure the
response of an expert driver to the countermeasure
intervention; and to observe and measure the
response of the vehicle to automatic interventions. A
small sample of reconstructed cases was used for this
purpose. These response measurements were used
for parameterizing and calibrating the driving
simulator test conditions and the models in the
CSSM. The objective tests included Laboratory
Tests, Track Tests, and Driving Simulator Tests.

Laboratory Tests were conducted to measure the
characteristics of the countermeasure warnings as
experienced by a driver during a potential conflict
event. The results from these tests were used to create
and calibrate the Driving Simulator (DS) as well as to
provide parameter values for the CSSM model. Lab
tests involved testing the vehicle fitted with ACAT
for human factor attributes like warning location,
magnitude and spectra as well as vehicle components
like vehicle weight, dimensions, etc. that serve as
input to the simulations.

Track Tests involved driver-in-the-loop tests for
expert driver response, delays and magnitudes to
warnings and driver-out-of-the-loop tests for vehicle
response to the ACAT system. The results were used
to calibrate the DS and CSSM models of the ACAT
system. In order to run the track tests two targets
were developed: the Car Guided Soft Target (GST)
and the Pedestrian Guided Soft Target (PGST). The
GST consists of a self-propelled, self-steering and
braking, GPS-guided, low-profile, hardened Dynamic
Motion Element chassis, to which soft, 3D targets of
a light passenger vehicle (constructed of separable
foam panels) are attached as shown in Figure 2.



Figure 2. GST on Dynamic Motion Element base.
Funke 6
The Pedestrian GST consists of a cable-driven, low-
profile, hardened turtle trolley, the longitudinal
position of which is GPS-guided, and to which an
inflatable pedestrian form is attached. Both
prototypes were instrumental in obtaining driver
performance measures for the objective track tests.

Driving Simulator Tests involved driver-in-the-
loop tests and were used to measure the response of
subject drivers to the intervention by the
countermeasure system. Both an expert driver and 12
typical drivers were tested. The measurements
included the type of driver response (braking,
steering or a combination) and the delay and
magnitude of each response. 12 cases were selected
for the simulator testing, which included three cases
for each of the four TRCTs described earlier. An
example case used in the driving simulator for each
of the four TRCTs is shown in Figure 3.


Figure 3. Illustration of the four primary
Technology Relevant Crash Types

To ensure test reproducibility and repeatability,
timing, and consistency of Subject Vehicle (SV)
speed, cruise control was used for the SV. A visual
distraction task was used in which a light was turned
on at 2.0 sec prior to the start of the first expected A-
CMBS warning and was turned off at 0.82 sec prior
to either the pre-calculated start of A-CMBS braking,
or the reconstructed time of impact to the Collision
Partner (CP) if there was no A-CMBS braking.

A suite of models form the core of the simulation tool
and lie within the framework of development of the
Crash Sequence Simulation Module (CSSM). The
core function of the CSSM is a time domain
simulation of the Driver model, which is based on the
NASA Architecture for Procedure Execution (Apex)
human operator programming language, the Vehicle
model (with and without ACAT) and Environment
model in Matlab /Simulink. The Apex and Simulink
models are linked together providing visual object
information to the driver model; and driver control to
the vehicle model. The virtual reality display used to
view the runs is also driven by the Simulink model.

The CSSM has a graphical user interface that enables
the user to select the desired crash scenarios and
driver behaviors for simulation. The CSSM then
initializes and runs the time domain simulations for
all desired combinations of crash scenarios and driver
behaviors specified by the user. Simulation post
processing was accomplished by creating a graphical
summary of the driver behavior and other time
domain outputs. The CSSM driver model comprises
Long term memory, Sensing/Perception, Working
memory, and Motor response as illustrated in Figure
4. Long term memory comprises declarative
knowledge and procedural knowledge, such as
vehicle steering and speed control procedures. The
sensing and perception passes visual, tactile, and
auditory information to the long term memory.
Working memory is implemented in the NASA Apex
Action Selection Architecture. The Motor response
function outputs the commanded steering wheel
angle, forward acceleration in gs, and brake
deceleration in gs to the Simulink vehicle-ACAT-
environment model.



Figure 4. Apex driver model.
Funke 7
The vehicle dynamics model in the CSSM was
constructed as a Matlab/Simulink model. The model
uses three inputs: primary control inputs, vehicle
states, and a number of vehicle-specific parameters to
calculate the vehicle state for each time step. This
information is used to calculate values for parameters
including: yaw rate, heading angle, and lateral
velocity, which is fed back to the model. The vehicle
model also includes a distance and azimuth sensor
model, control logic, and a brake actuator model,
apart from the conventional vehicle dynamics model.
The pre-crash vehicle motions are described using
quasi-steady state equations-of-motion in terms of
state variables.

The countermeasure model was also constructed as a
Matlab/Simulink model that allows an interface with
the vehicle model and the Apex driver model. The
model uses information from the subject vehicle and
data from the sensors to determine the state of its
outputs. There are many intermediate outputs of the
A-CMBS model that are processed further in order to
get the final output signals. The resultant final
outputs of the A-CMBS model include: braking level,
warning, and seat belt tensioner mode.

Data Generation

A time domain simulation of the driver, vehicle, and
environment is conducted by the CSSM. The output
from each CSSM simulation includes a yes/no data
element that indicates whether or not a crash
occurred during the simulation. If a crash occurred
then the change in vehicle velocities (i.e. Vs) for the
crash were computed based on the impact geometry
and speeds. The V values were in turn used to
estimate the probability of driver fatality (POF) and
injury Fatality Equivalents (FE). If a crash did not
occur then the POF and FE are zero. Results for
simulations with and without the countermeasure
were combined to determine the reduction in the
probability of crash, POF and injury FEs.

The advantages of using a sample of reconstructed
crash cases are that: they include co-variations that
have been observed to occur in all the case variables
(i.e., not just those judged to be key variables); they
are more likely to be realistic; currently, they are
more recent (e.g., in the case of NASS/CDS) from
most regularly updated databases; they have
established weighting factors that relate them to
national level crash data; and in general, they appear
to be nearly the best available, most representative
and most complete detailed level data for crashes in
the United States.

Countermeasure Performance Analysis and
Safety Benefits

The core of the performance analysis lies in the
application of the Overall Safety Effects Estimator
(OSEE) which estimates the overall safety benefits in
terms of the reduction in the numbers of collisions
and fatalities at the US level using the fleet systems
model. This is based on data for technology
effectiveness functions, crash scenarios, retrospective
as well as forecasted data. The technology
effectiveness functions describe the Exposure,
Prevention and Fatality Ratios (ER, PR, FR) for each
technology relevant crash type and are based on
results from the CSSM simulations.

The estimated safety benefits were computed based
on extensions to the baseline benefits equations
described in [5] starting with Eq. (3). Depending on
the type of benefits (the number of conflicts, crashes,
or fatalities) the effectiveness term (E
i
) is:

=
fatalities for 1
crashes for 1
conflicts for 1
,i p i i
i i
i
i
FR PR ER
PR ER
ER
E


where,
ER
i
and PR
i
, are the estimated Exposure Ratio and
Prevention Ratio, respectively for scenario i and
FR
p,i
is the Fatality Ratio for person p in scenario
i.

The overall estimated safety benefits are the sum of
the benefits for each crash type. The benefits for each
crash type are equal to the estimated effectiveness
(E
i
) times the size of the problem for each crash type
(N
woi
). The overall benefits estimates of the Honda A-
CMBS, if it had been installed in the entire US Light
Passenger Vehicle Fleet in the 2005 calendar year are
shown in Table 2. The baseline population in this
table comes from Traffic Safety Facts 2005.

Table 2.
Safety benefit estimates for the Advanced
Collision Mitigation Braking System (A-CMBS)


Crash
Problem
Size for the
Entire
US Fleet
Estimated
Overall
Effectiveness
for the Entire
US Fleet
Estimated
Safety
Benefits
Crashes 6,146,907 8% 511,000
Vehicles 10,838,878 9% 1,013,000
Fatalities 43,510 4% 1,623
Funke 8
TEAM 2: LANE DEPARTURE WARNING
(LDW)

As part of the ACAT program, Volvo, Ford and the
University of Michigan Transportation Research
Institute (UMTRI) developed a SIM that addresses
lane departure crashes. The Volvo-Ford-UMTRI
(VFU) team used interactions between driver,
vehicle, environment and technology elements in a
Monte Carlo simulation model to estimate safety
benefits in terms of crashes avoided [7].

Data Usage

The VFU SIM was tailored to lane departure crash
types. These include road departure and lane
departure crashes. The VFU SIM utilizes GES, CDS,
Road Departure Crash Warning (RDCW) Field
Operational Test data, Michigan State data and a
Swedish in-depth crash database (Factors Influencing
the Causation of incidences and Accidents, FICA) to
establish the typical characteristics of the LDW
relevant crash population.

Case Scenarios

A key component for effective development of a
computational model is defining the driving and
crash characteristics of the typical scenarios where
LDW might be of assistance. These characteristics
were captured through a set of Driving Scenarios
(DS). Each driving scenario represents a typical
combination of driver, vehicle, and environment
states that precedes lane/road departures. However, it
should be noted that the DS are not pre-crash
scenarios as such, since driving under the DS
conditions does not automatically result in a crash.
Rather they represent combinations of conditions that
have the potential to produce lane departures and
resultant crashes.

For the purpose of scenario development, baseline
population and pre-crash scenario factors were
obtained from NASS/GES and NASS/CDS and the
Swedish in-depth database. Roadway geometry data
were obtained from Michigan State data and Federal
Highway Administration (FHWA) Highway
Performance Monitoring System (HPMS) data.
Vehicle kinematics data were obtained from
naturalistic RDCW data. All these sources were used
in developing a combination of fixed as well as
variant parameters of the DS, that feed into to the
SIM model. An example of such a driving scenario
would be a vehicle traveling on a dry but curved
roadway with two or more divided lanes in daylight
with no adverse conditions, with a driver who is not
distracted or fatigued. Overall, 25 such high priority
technology relevant scenarios, capturing the typical
conditions of slightly more than 90% of the relevant
crashes were developed, which form the basis for
input to the computational model.

Objective Testing

Objective tests were performed in the ACAT project
in the form of track tests for system performance
verification and parameter estimation in the
computational model, road tests to establish system
availability under different DSs, and two driving
simulator studies were conducted to analyze
distracted and drowsy driver reactions to various
HMI warnings. It should be noted that the outputs of
objective tests were not used directly in the
computational model, but rather were used to
generate parameter values for running simulations as
well as to validate and calibrate the computational
model.

Model Creation

VFUs approach focuses on developing a
computational model that ties driver, vehicle,
environment, and technology elements together to
generate realistic interactions between them in a
dynamic environment, in order to produce reliable
performance outputs. This was accomplished by
developing models for the vehicle, technology, and
driver, respectively. The Vehicle Model was
implemented using CarSIM and was embedded as a
subsystem in Matlabs Simulink tool. Output from
the driving simulator studies was used to calibrate
and validate the Vehicle Model. An illustration of the
CarSIM model is shown in Figure 5.


Figure 5. Architecture of the Vehicle Model.

For the countermeasure model, a generic model of
the Volvo Lane Departure Warning (LDW) system
was developed for implementation in the SIM. The
warning algorithm is a function of the lane position
and the vehicles lateral velocity with respect to the
lane markings. When the distance between inside of
the lane marker and the outside of the nearest front
tire is less than the set threshold distance, a lane
excursion is flagged, as depicted in Figure 6. While
the system was fitted with two levels of sensitivity,
the LDW model was implemented with high
sensitivity.
Funke 9


Figure 6. Vehicle location when the lane
departure is detected.

A unique feature of the VFU model is the approach
to modeling driver performance. The underlying
principle is that drivers leave the lane due to
inattention. This principle is coupled with the idea
that when drivers become inattentive they switch
from a lane-keeping mode, that keeps the vehicle
within the lane, to a mode of no action. A return to
alertness is modeled by the driver returning to the
lane-keeping mode. A warning is one event that will
cause a driver to return to alertness from an
inattentive state of mind. The modeling utilizes a
partitioning of parameters into those that are derived
from crash data and those that are derived from
quantitative analysis of naturalistic driving data.

Data Generation

The basic process used for data generation is a Monte
Carlo simulation program, which is implemented
without and with support from the Lane Departure
Warning system. A novel feature of this process is
the use of randomly selected cases from the Michigan
State crash data files as the means of obtaining
variations in key variables such as lane width and
detailed road geometry at sampled crash sites. This
approach reduces the need for simplified distributions
of these key variables which would be used in a more
traditional Monte Carlo process.

The core of the Monte Carlo process lies in defining
and simulating the virtual driving event, which is a
driving scenario (a combination of the Driver,
Vehicle, Environment and Technology (DVET)
components) presented as a combination of initial
conditions, model parameters, road conditions, and
environmental conditions. Each of these parameters
is selected randomly for a single run in the
simulation. Each simulation is designed to represent a
single potential lane or road departure event
without the technology. Thousands of such runs are
executed to form the baseline Virtual Crash
Population. Repeat simulations with the technology
enabled generates the data required to assess safety
benefits estimates.

Some elements that are unique to the VFU ACAT
project in the data generation process are as follows:

Inverse Time to Lane Change (ITTLC) ITTLC
is the reciprocal of the estimated time to lane crossing
given the instantaneous position and lateral velocity
of the subject vehicle. The ITTLC serves as the
primary control variable while sampling initial
parameters, which include vehicle kinematic
variables obtained from naturalistic driving and
parameters obtained by sampling from random
distributions.

Transition Probabilities In the context of a
driving scenario, transition probability is defined as
the expected probability of a vehicle transitioning
from a normal driving scenario to a crash scenario.
This process applies a sampling approach from the
ITTLC bins and obtains expectations based on the
relative frequencies in those bins. Transition
probabilities are an essential component of the VFU
SIM methodology, providing an efficient method to
amplify crash risk in simulations without introducing
systematic bias.

Crash Metric As an alternative to generating
actual crashes and representing the detailed locations
of potential collision objects, a distancebased
measure of crash probability was developed. This
basic crash risk model associates a crash metric with
the lateral or longitudinal distance traversed at
various locations outside the desired lane, as shown
in Figure 7.


Figure 7. Crash risk related to trajectory output
from simulation.

The logic behind it is that the lane deviations are
unplanned and hold a uniform risk of colliding with a
fixed or moving object that is proportional to
exposure, i.e. the size and duration of the lane
excursion. For fixed objects and neighboring lane
excursions, this is a distance-based metric based on
an arbitrary boundary layer, while for road
excursions, the Maximum Road Excursion (MRE)
metric is used, which increases linearly within the
clear zone (for road departure) to a maximum value
at the edge of the clear zone.
Funke 10
Countermeasure Performance Analysis and
Safety Benefits

The basic calculation in this activity is a comparison
of crashes that occur without the system to crashes
that occur with the system. The VFU team
emphasized that this process is different for crash
avoidance systems than it is for crashworthiness
systems. The difference is that the sequence of
relevant events for crash prevention starts with an
aberration from normal driving. This starting point
may lead to a variety of types of crashes or, as occurs
most of the time, a corrective action that avoids a
crash altogether. The VFU team implemented this
process by defining a transition matrix that contained
the probability of each type of crash, or no crash, for
each type of starting condition, scenario, or event.

The basic benefits equation extends from the original
equations such that,

(4)


where ,
N and N = the number of crashes with and
without the system
E = the Overall system effectiveness

is the Exposure ratio


Are the transition
probabilities and scenario
weights with the system.

Are the transition
probabilities and scenario
weights without the
system.

This equation is rewritten into a more general form,

(5)

where,


and

Many simplifications and assumptions were
necessary to be able to complete the overall project
within the time and resources available. These
included use of a limited number of driving simulator
runs; use of data from one state (Michigan) instead of
national data for road environment parameters; no
consideration of fatigue, distraction or non-driving
workload in the driver model; and use of a single
model of passenger car to represent the entire
passenger car fleet. Within the available resources,
the team developed models that had adequate fidelity
in terms of processes and mechanisms, but otherwise
were as simple as possible.

For effectiveness, an initial raw estimte of 47% for
the 181,000 crashes was calculated. This
effectiveness estimate was then refined based on
estimates of other influential factors like system
availability, driver responsiveness, and driver
compliance, which effect the outcome of the benefits
estimation process. The resulting range of the final
estimate is given below in Table 3.

Table 3.
Safety benefit estimates for Lane Departure
Warning (LDW)


Target
Population
Estimated
System
Effectiveness
Estimated
Safety
Benefits
Crashes 181,000 13% - 32%
24,000 -
57,000


TEAM 3: PRE-COLLISION SAFETY SYSTEM
(PCS)

For the ACAT program, Toyota developed a Toyota
SIM (T-SIM) that estimates safety benefits for
advanced driver assistance systems such as the Pre-
collision Safety System (PCS) that reduces the
vehicle impact speed in a crash. The PCS addresses
rear-end crashes, head-on crashes, and collision-in-
to-objects. The T-SIM generates estimated safety
benefits including the number of crashes avoided,
fatalities reduced, and casualties reduced. A graphical
view of T-SIM is shown in Figure 8 [8].



Figure 8. Structure of the T-SIM for the PCS.
Data Usage.
Funke 11
GES, CDS, FARS, and Event Data Recorder (EDR)
data were the primary databases used in the
development and application of the T-SIM. The SIM
process is initiated by classifying the vehicles in GES
and FARS crash database as culpable and counter
party. Using this classification and crash variables
such as accident type, location of crash, and traffic
control, the crashes in GES and FARS are grouped
into 486 Standard Accident Patterns (SAPs). This set
of 486 accident patterns is reduced to 98 SAP by
eliminating the minor (representing less than 0.025%
of all fatalities in FARS) and unclear cases. The
remaining 98 SAPs represent approximately 85% of
all crash cases in the accident database. EDR data is
used to retrieve information concerning driver
performance in baseline crashes.

Case Scenarios

The relevant crashes are separated into three major
subdivisions: rear-end (Lead Vehicle Stopped (LVS)
and Lead Vehicle Decelerating (LVD)), head-on, and
collisions with other objects. Each subdivision is then
further subdivided by travel speed ranges (5 mph
bins) into the final set of scenarios. Of the 98 SAPs,
15 are considered to be relevant to the PCS. The
number of crashes without the countermeasure active
(N
woi
) for each one of the relevant scenarios/SAP is
calculated from FARS and GES data.

Objective Testing

The speed reduction attained by Pre-collision Brake
Assist (PBA) and Pre-collision Brake (PB) is
modeled using the deceleration profiles generated
from test track tests, where a PCS equipped vehicle is
driven, into a polyurethane-foam pole with a radar
reflector, by an expert driver using several different
braking levels (e.g. 0.2 g, 0.4 g), including no
braking. The driver reaction and brake application
profile is modeled using data from the driving
simulator study, where the distracted driver reacts to
a PCS warning. An example from the Driving
Simulator is shown in Figure 9 where a Lead Vehicle
Stopped (LVS) scenario is shown.

Model Creation

The model of driver performance consists of a delay
after the occurrence of an alert, a warning or other
causes that may bring the driver back to alertness, a
level of braking and a gradual onset between no
braking and the selected level of braking. A key
assumption is that drivers react similarly to a


Figure 9. Driving Simulator LVS scenario
showing cut-out revealing stopped vehicle.

warning, in terms of braking magnitude and
application rate, as they would in normal driving
when they become aware of an impending crash. The
model for the combination of vehicle and
countermeasure system has two parts. The Pre-
collision Brake Assist (PBA) acts as an amplifier of
the driver level of braking and the Pre-collision
Brake (PB) produces a constant high level of
deceleration, once the respective specified criteria
have been reached. The effect of PBA and PB is
numerically overlaid on the driver reaction data
generated from the Driving Simulator (DS) as shown
in Figure 10. The difference in reduction of impact
speed between with and without PCS is used to
estimate safety benefits.


Figure 10. Effects of PBA and PB overlaid on the
deceleration profile measured in the DS.

A key element of the T-SIM is the use of EDR data
to estimate pre-crash speed reductions when drivers
do not have the benefit of the countermeasure.
Funke 12
EDR data were analyzed to estimate speed reduction
before crash without a PCS to know the distribution
when the drivers stepped on the brake before crash.
The result implies that drivers braking behavior
obtained by the DS was similar to the actual
situations by EDR. It also implies how much drivers
can brake during the crash imminent situation may
not be different regardless of the warning. Therefore,
the DS data was used for the simulation by weighting
the data to have a closer distribution to the EDR data.

Data Generation

The probability of a fatality or casualty without the
countermeasure for each scenario is calculated
directly from FARS and GES data. The reduction of
travel speed with the countermeasure is determined
from the DS tests that determine driver response to a
warning and from test track experiments to determine
the supplementary impact of the PBA and PB
subsystems. The probability of a fatality or casualty
with the countermeasure for each scenario is
determined by subtracting the reduction in travel
speed from the original travel speed (taking into
account any pre-crash braking) for each scenario and
applying the probability from the original data for the
reduced speed. The difference in speed reduction is
used to estimate the fatality reduction for accident
patterns.

Countermeasure Performance Analysis and
Safety Benefits

The effectiveness of a system can be calculated by
multiplying the fatality reduction and the number of
fatalities in the applicable accident patterns. The
effectiveness in preventing fatalities or casualties for
each of 15 scenarios is equal to:

E
]
= 1 -
P(]utuIty)
WO i
P(]utuIty)
W i
(6)

where,
P(otolity)
w0
is number of fatalities for scenario
i without the countermeasure active.
P(otolity)
w
is number of fatalities for scenario
i with the countermeasure active.

and similarly for casualties:

E
c
= 1 -
P(CusuuIty)
WO i
P(CusuuIty)
W i
(7)

System Effectiveness is the weighted sum of the
effectiveness for the individual scenarios.

While safety benefit estimates were developed by
Team 3, the final report was still being reviewed at
the time of this printing. Therefore these results are
not published here but will be published in the final
report for this ACAT project.


TEAM 4: BACKING CRASH
COUNTERMEASURE SYSTEM

As part of the ACAT program, General Motors
Corporation (GM) with support from Virginia Tech
Transportation Institute (VTTI) developed a basic
methodological framework and simulation model to
estimate the effectiveness and safety benefits of a
prototype backing crash countermeasure system. The
SIM tool provides an estimate of safety benefits in
terms of reduction in crashes and fatalities [9].

Data Usage

The data sources that were used were primarily
national databases like GES, FARS, NHTSA Special
Crash Investigations (SCI) Database; state databases,
specifically, Nebraska, Kentucky, and North
Carolina; supplemental uata souices incluue uata
fiom National Electronic Injury Surveillance System
(NEISS), Police Accident Reports (PARs), and death
certificates and other non traditional sources
including other naturalistic data sources from
archives and literature studies. Parametric data
sources including: brake reaction time, braking
performance, vehicle kinematics, glance
distributions, and driver trust, were also used from
research archives.

Case scenarios

The scenario development and crash characterization
process was undertaken to aid objective testing of
specific countermeasure systems. Since the required
data was not captured in the national databases
directly, the activity involved a multi-step reasoned
process that broke down the target population into the
following 10 scenarios shown in Table 4. These
scenarios were considered to be reasonably
representative, but not exhaustive, of the types of
backing crash scenarios with emphasis on pedestrian
backing crash situations.

Objective Testing

The purpose of objective testing is to produce
parameter estimates that can populate the SIM model
to produce estimated safety benefits applicable to the
overall crash problem size.
Funke 13
Table 4.
Objective test scenarios for Backing Crash
Countermeasures

Test
scene
#
Roadway
Type
Maneuver
Pedestrian
Posture/Motion
Offset/Direction
of Encroachment
Distance
from
Bumper at
Initiation
of Backing
P1 Parking Lot Backing out
Standing
on center line
Near (e.g.,
5)
P2 Street
Parallel
Parking
Sitting on curb,
Right (e.g., 2)
Farther
(e.g., 30)
P3 Driveway Backing out
Prone,
Left (e.g., 2)
Moderate
(e.g., 15)
P4 Driveway Backing out
Incurring,
from right
Moderate
(e.g., 15)
P5
Parking
Space
Backing out
Incurring
from left
Near (e.g.,
5)
P6
Long
Driveway
Driving in
Reverse
Incurring
from left
Farther
(e.g., 30)
V1
Inter
section
Backing
Stopped behind
on center line
Near
(e.g., 5)
V2
Driveway /
Street Junction
Backing out
Approaching
from the left
Moderate
(e.g.,15)
V3 Parking Lot Backing out
Parked
Behind
Farther
(e.g.,30)
FO1
Driveway/
Roadside
Junction
Backing out
Utility Pole,
encroach
to the Right
Moderate
(e.g., 15)

An example of such a scenario would the incurring
pedestrian scenario as shown in Figure 11. Here the
pedestrian incurs from the left on a long driveway
with a distance of 30 feet from bumper at the time of
initiation.

Figure 11. Illustration of the 5 year old incurring
pedestrian scenario.

A highlight of the objective test development process
was the development of pedestrian test devices. This
involved development and modification of off-the-
shelf dummies to develop child pedestrian test
devices that have realistic radar cross-sections at the
24 GHz frequency that is used in some rear object
detection systems. Test Objects used in Grid tests and
Camera Field of View evaluations are depicted in
Figure 12 below.



Figure 12. Test Objects used during testing (from
left to right: Gen II 5 year old, Gen II 2 year old,
Cardboard Cylinder, Gen II sitting child, PVC
Pole, Gen 1 Prone 5 year old).

Model Creation

A unique feature of the GM-VTTI process was the
development of a driver model that has three distinct
submodels: the Visibility Model, the Glance
Behavior Model, and the Driver Response Model.
Visibility Model In the Visibility Model, the
parameters define the outside visibility by first
determining which displays are available to the
driver from look-up tables which are used to obtain a
probability of visibility. That probability is compared
to a pre-determined threshold to ascertain visibility,
which includes left mirror, right mirror, rear - view
mirror and over the shoulders. In occluding
situations, the visibility matrices are zeroed out. The
data for the model is obtained from objective testing
performed as part of research from other projects as
part of GMs corporate body of knowledge.
Glance Behavior Model The Glance Behavior
Model not only provides distributions of driver
glance behaviors in the presence and absence of
counter measure systems but also probabilities of
subsequent glance locations based on current glance
locations and length of glance. The Glance Behavior
Module accomplishes these goals by generating
glances and keeping track of when new glances
should be generated.
Funke 14
Driver Response Model The outputs of the
Glance Behavior Model are fed to the Driver
Response Model, which determines the presence,
type, and level of driver response. The model first
determines if the driver has detected the obstacle and
countermeasure. If no detection is determined, the
simulation proceeds to the next step. If there is
evidence of detection, then the model generates a
reaction time and braking effort appropriate for the
situation and applies these parameters to determine if
a crash or no crash occurred. The same process
applies for automatic braking as well. A no crash
situation is detected by the end of simulation time or
if the vehicle has stopped before impacting the
obstacle. A crash situation is identified if the
obstacle is within the vehicles width, the distance to
the object is less than or equal to 0, and the vehicle
speed at impact is > 0.

Data Generation

The core of the SIM model is the Monte Carlo
simulation process exercised in a Matlab/Simulink
environment that will extract data from a given set of
distributions. The process involves picking values
from a given distribution for a given iteration, which
are obtained from objective tests and other sources of
data. Each iteration is run several times for a new set
of parameter values with and without the
countermeasure active to account for the variability
in outcomes. A comprehensive set of data is
produced for all situations which are used in the
estimation of safety benefits.

The initial modules of the SIM model define
parameters that will remain fixed throughout the
simulation (Non-Variant Parameters). Subsequent
modules define parameters that change as the
simulation progresses (Variant Parameters), which
are defined inside the Simulation Control loops.

Once all the parameters are defined, the SIM starts
the Monte Carlo Simulation. The simulation model
should be seen as a representation of the overall
backing maneuver, from the point where the vehicle
is shifted to reverse all the way through to when a
crash with the obstacle is recorded or an avoidance of
the crash is achieved. The simulation model consists
of numerous modules and sub-modules in Matlab that
are initialized and called upon repeatedly and are
exercised at various points of the iterations. Once all
simulation control loops are completed, estimation of
safety benefits is performed. A summary flowchart
of the SIM flow is shown in Figure 13.


Figure 13. Flowchart of the Monte Carlo process.

Countermeasure Performance Analysis and
Safety Benefits

The main outcome of the countermeasure
performance analysis and safety benefits estimation
process is the estimated number of crashes avoided
annually following the deployment of a particular
crash countermeasure. The equations that are used to
determine this are given as follows:
C
A
= C
wo

C
SE (8)
where,
C
A
= annual number of the type of crashes of
interest
C
wo
= annual number of the type of crashes of
interest prior to a countermeasures deployment
D
C
= potential countermeasure deployment rate
SE = System Effectiveness

Another potential safety benefit is the reduction in
fatalities, which is given by:
E
R
= E
wo

C
SR (9)
where:
H
R
= predicted annual reduction in fatalities
H
wo
= annual total fatalities for the type of crashes
of interest prior to a countermeasures deployment
D
C
= potential countermeasure deployment rate in
the vehicle fleet.
SR = System Harm-Reduction Effectiveness

Table 5 below summarizes the target crash
population, estimated system effectiveness, and
estimated safety benefits.
Funke 15
Table 5.
Safety benefit estimates for Backing Crash
Countermeasures


Target
Population
Estimated
System
Effectiveness
Estimated
Safety
Benefits
Crashes 202,000 32% 65,000
Fatalities 182 62% 113


CONCLUSION AND SUMMARY

This paper describes a Safety Impact Methodology
framework which was used by each of the ACAT
teams to estimate safety benefits for pre-production
crash avoidance systems. The SIM framework
includes the following components: Data Usage,
Case Scenarios, Objective Testing, Model Creation,
Data Generation, Countermeasure Performance
Analysis, and Safety Benefits. The specific
extensions to the SIM framework developed in each
of the ACAT projects were unique and can be used to
estimate safety benefits for various types of crash
avoidance systems.

PROGRAM INFORMATION

Detailed Final Reports [6][7][8][9] from the teams
describing their ACAT projects have been submitted
and are in the process of being published. These
reports will be available on the NHTSA website at:
http://www.nhtsa.gov/Research/Crash+Avoidance/Of
fice+of+Crash+Avoidance+Research+Technical+Pub
lications

The ACAT program continues with a second series
of research projects known as ACAT-II. This
research program involving two teams is currently
underway and is set to finish in June 2011. Technical
questions on the ACAT program should be referred
to James Funke of NHTSAs Office of Vehicle
Safety Research at (202) 366-5213 or via e-mail at
james.funke@dot.gov.

REFERENCES

[1] Dingus, T., Klauer, S., Neale, V. L., Petersen, A.,
Lee, S. E., Sudweeks, J., et al. The 100-Car
Naturalistic Driving Study, Phase II - Results of the
100-Car Field Experiment, DOT HS 810 593,
National Highway Traffic Safety Administration,
Washington DC, April 2006.

[2] NHTSA. The New Car Assessment Program
Suggested Approaches for Future Program
Enhancements, DOT HS 810 698, National Highway
Traffic Safety Administration, Washington DC,
January 2007.

[3] GAO Report to the Chairman, Committee on
Commerce, Science, and Transportation, U.S. Senate
GAO. HIGHWAY SAFETY: Foresight Issues
Challenge DOTs Efforts to Assess and Respond to
New Technology-Based Trends, GAO-09-56, United
States Government Accountability Office,
Washington DC, 2008.

[4] Carter, A.A., Burgett, A., Srinivasan, G.,
Ranganathan, R., Safety Impact Methodology
(SIM): Evaluation of Pre-Production Systems,
Proceeding of the 21st International Technical
Conference on the Enhanced Safety of Vehicles,
Paper Number 09-0259, Stuttgart, June 2009.

[5] Burgett, A., Srinivasan, G., and Ranganathan, R.,
A Methodology for Estimating Potential Safety
Benefits for Pre-Production Driver Assistance
Systems, DOT HS 810945, National Highway Traffic
Safety Administration, Washington DC, May 2008.

[6] Van Auken, R.M, Zellner, J.W., Chiang, D.P.,
Kelly, J., Silberling, J.Y., Dai, R., et al. Advanced
Crash Avoidance Technologies (ACAT) Program
Final Report of the Honda-DRI Team, Volume I:
Executive Summary and Technical Report, DOT HS
811 454, National Highway Traffic Safety
Administration, Washington DC, Forthcoming.

[7] Gordon, T., Sardar, H., Blower, D., Ljung Aust,
M., Bareket, Z., Barnes, M., et al. Advanced Crash
Avoidance Technologies (ACAT) Program Final
Report of the VolvoFordUMTRI Project: Safety
Impact Methodology for Lane Departure Warning
Method Development and Estimation of Benefits,
DOT HS 811 405, National Highway Traffic Safety
Administration, Washington DC, October 2010.

[8] Toyota Motor Corporation. Advanced Crash
Avoidance Technologies (ACAT) Program Final
Report, National Highway Traffic Safety
Administration, Washington DC, Forthcoming.

[9] Perez, M., Angell, L.S., Hankey, J., Deering,
R.K., Llaneras, R.E., Green, C.A., et al. Advanced
Crash Avoidance Technologies (ACAT) Program
Final Report of the GMVTTI BackingCrash
Countermeasures Project, DOT HS 811 452, National
Highway Traffic Safety Administration, Washington
DC, Forthcoming.