Design of Experiments (DOE)

Source: "What is DOE?"; E-Chip, Inc.,www.echip.com 1

Experimentation and the
Engineer
Not going to work in a lab?
Not interested in research?
Plan to work in design and/or
production?
Why should you be concerned with
experimentation?

2
 Experimental Analysis in
Product Realization
DEVELOPMENT

Research Design
(properties, processes) (concepts, performance)

PRODUCTION

Validation QA
(processes, performance) (monitoring)

3
 Experimentation
The Blind Man and the Elephant

What we learn from an experiment may depend on

where we look, how we look, and the scope of our view!
4
Experimentation
Experimenters first goal: Understand
the process!
Experiments - used to study effects of
parameters as they are set at various
levels Noise factors

Signal Measured
System
factors response

Control factors
5
Cost of Experimentation
Resources (people, equipment, etc.)
Time
Material (unprocessed or unusable
product)
Usable product that is not being
produced

6
Approaches to
Experimentation
Build-test-fix
One-factor-at-a-time (the classical
approach)
Designed experiments (DOE)

7
Approaches to Experimentation:
Build-Test-Fix
Build-test-fix
the tinkerers approach
pound it to fit, paint it to match
impossible to know if true optimum
achieved
you quit when it works!
consistently slow
requires intuition, luck, rework
reoptimization and continual fire-fighting

8
Approaches to Experimentation: One-
Factor-at-a-Time
One-factor-at-a-time
procedure (2 level example)
run all factors at one condition
repeat, changing condition of one factor
continuing to hold that factor at that condition, rerun
with another factor at its second condition
repeat until all factors at their optimum conditions
slow, expensive: many tests
can miss interactions!

9
 One-Factor-At-A-Time
Process: Yield = f(temperature, pressure)
50% yield
30%
yield

Max yield: 50% at 78C, 130 psi?

10
One-Factor-At-A-Time
A better view of the maximum yield!
Optimized yield is over 85%

Process: Yield = f(temperature, pressure)

11
Approaches to Experimentation: DOE
Design of Experiments (DOE)
A statistics-based approach to designed
experiments
A methodology to achieve a predictive
knowledge of a complex, multi-variable
process with the fewest trials possible
An optimization of the experimental
process itself

12
Major Approaches to DOE
Factorial Design
Taguchi Method
Response Surface Design

13
DOE - Factorial Designs
Full factorial
simplest design to create, but extremely inefficient
each factor tested at each condition of the factor
number of tests, N: N = yx
where y = number of conditions, x = number
of factors
example: 8 factors, 2 conditions each,
N = 28 = 256 tests
results analyzed with ANOVA
cost: resources, time, materials,

14
DOE - Factorial Designs - 23
Trial A B C
1 Lo Lo Lo
2 Lo Lo Hi
3 Lo Hi Lo
4 Lo Hi Hi
5 Hi Lo Lo
6 Hi Lo Hi
7 Hi Hi Lo
8 Hi Hi Hi 15
DOE - Factorial Designs
Fractional factorial
less than full
condition combinations are chosen to
provide sufficient information to determine
the factor effect
more efficient, but risk missing interactions

16
DOE Factorial Designs
(Fractional: 7 factor, 2 level; 128 8)
Trial A B C D E F G
1 Lo Lo Lo Lo Lo Lo Lo
2 Lo Lo Lo Hi Hi Hi Hi
3 Lo Hi Hi Lo Lo Hi Hi
4 Lo Hi Hi Hi Hi Lo Lo
5 Hi Lo Hi Lo Hi Lo Hi
6 Hi Lo Hi Hi Lo Hi Lo
7 Hi Hi Lo Lo Hi Hi Lo
8 Hi Hi Lo Hi Lo Lo Hi
17
DOE - Taguchi Method
Taguchi designs created before desktop
computers were common
pre-created, cataloged designs intended to quickly
find a set of conditions that meet the criteria of
success
previous slide an example of an L8 template
Designs cannot support response surface
models and are limited to only predicting at
the points where data was taken

18
DOE - Response Surface: RSM
Goal: develop a model that describes a
continuous curve, or surface, that
connects the measured data taken at
strategically important places in the
experimental window

19
DOE - Response Surface: RSM
RSM uses a least-squares curve-fit
(regression analysis) to:
calculate a system model (what is the process?)
test its validity (does it fit?)
analyze the model (how does it behave?)

Bond = f(temperature, pressure, duration)

Y = a0 + a1 T + a 2 P + a3 D
+ a11T2 + a22P2 + a33D2
+ a12TP + a13TD + a23PD

20
Experimental Design Process
1. Determine the goals
2. Define the measures of success
3. Verify feasibility (rough estimate)
4. Design the experiment (precise estimate)
5. Run the experiment
6. Collect and analyze the data
7. Determine and verify the response
8. Act on the results
21
Experimental Design Process
1. Determine the goals
doing so often leads to:
goals are too many to cover in a single study
goals that seemed concrete are actually very
negotiable
once consensus achieved, a valid
experimentation strategy can be developed
plan the action to be taken if the experiment is a
success or a failure

22
Experimental Design Process
2. Define the measures of success
once the goals are set, how do we know when
we are meeting them?
measures must be metric and refer to an intrinsic
feature of the process or product
qualitative good/bad cannot be modeled
including a large number of responses just to see
how they change often diverts focus from the
responses that are critical to meeting the goals

23
Experimental Design Process
3. Verify feasibility (rough estimate)
use a power calculation to determine
whether any information can be found
with a reasonable number of trials
a function of the amount of noise associated
with a response
the more noise in the process, the more trials
required to see a change in the desired
parameter

24
Experimental Design Process
3. Verify feasibility (rough estimate)
example: how many runs needed to observe
changes of 5,000 psi in the tensile strength of a
plastic extruded part?
Resolution (psi) Number of runs
10000 5
5000 22
2500 90
1250 362

25
Experimental Design Process
4. Design the experiment (precise
estimate)
Identify the controls to be varied
Make the design
Determine whether the number of
experiments is too large
If necessary, use a screening design to sift
through to find the critical few

26
Experimental Design Process
5. Run the experiment
A task in resource management
Complete the work as efficiently and as
effectively as possible

27
Experimental Design Process
6. Collect and analyze the data
Best to examine the data as a whole
Analysis of a set of data has significant
advantage over contrasting the results
between two data points
Ability to find suspect data is greatly
enhanced
If there is a choice as to order, you may
wish to obtain the most critical data first

28
Experimental Design Process
7. Determine and verify the response
A Response Surface gives you the ability
to predict, with statistical limits, the
behavior of the process at any point
within the design window
Combining predictions from several
responses allows you to simultaneously
optimize for several key specifications

29
Experimental Design Process
8. Act on the results
Goals set earlier identified what was to
be done if success obtained do it!
If no action is taken, why was the experiment
done?
Complete the documentation of the
experiment

30
Summary

Experimenters first goal: Understand

the process!

31
Summary (cont.)
The cost of experimentation
Resources (people, equipment, etc.)
Time
Material (unprocessed or unusable
product)
Usable product that is not being produced

32
Summary (cont.)
Approaches to experimentation
Build-test-fix
One-factor-at-a-time (the classical approach)
Designed experiments (DOE)
Major approaches to DOE
Factorial Design (full, fractional)
Taguchi Method
Response Surface Design

33
Summary (cont.)
Full factorial
simplest design to create, but extremely inefficient
each factor tested at each condition of the factor
results analyzed with ANOVA
cost: resources, time, materials,
Taguchi Method
Taguchi designs created before desktop
computers were common
Designs cannot support response surface models
and are limited to only predicting at the points
where data was taken
34
Summary (cont.)
Response Surface Modeling
Goal: develop a continuous curve or
surface that models the effects of
parameters at different levels
Speed
Dye = 250.0
Reaction_time = 150.0
5.6

5.5

5.4

5.3

5.2
EC
50 H IP
50
S e 60 60
ns 7
itiz 0 70 1
er_ er_
2 80 80 sitiz
90 90 Sen

35
Summary (cont.)
Experimental Design process
Determine the goals
Define the measures of success
Verify feasibility (rough estimate)
Design the experiment (precise estimate)
Run the experiment
Collect and analyze the data
Determine and verify the response
Act on the results

36
 END

Source: "What is DOE?"; E-Chip, Inc.,www.echip.com 37