MSA FOR CONTINUOUS DATA

Basic Procedure for Continuous Data

1. Select 30 units that are representative of the total process variation and give each unit a unique
identifying mark or tag.
It is essential that the sample units span the full range of variation in The Y…
2. Randomly measure each unit twice, using the same gauge & appraiser and record the results. Be
careful that the appraiser does not know or remember the first result when performing the second
measurement.
3. Plot the results on a Youden plot (Square scatter diagram with X and Y axes same size)
The ideal sample size is 30 units. The 30 units are being used to estimate a standard deviation (the
average measurement error, e). A sample size less than 30 will result in less accuracy of the estimate
Measurement Variation Formulas:
Measurement Error, e:
The Range of measurements of each unit, Ri: R i = XA − XB

The Average Range of the set of units, R : R=

R i

R
e =
The repeatability or measurement error, e: d2

(d2 = 1.128)

Total Measured Variation, T:

X Ai + X Bi
The average value of each unit, X i : Xi =
2

 (X − X )
2

The product variation, P:

i
P =
n −1

1
The total variation, T:  T =  P2 +  e2
2
Since the estimate of e2 is based on a very small sample size (n=2) the estimate of the measurement
error is modified by the small sample size correction factor, (n-1)/n:
n −1 2 1 2
e = e
n 2

1 of 8
MSA FOR CONTINUOUS DATA
To determine how much of the observed measurement variation is due to measurement error, the
intraclass correlation coefficient,  is calculated12:
The percentage of the total variation that is due to product variation is  x 100.
 e2
ρ ICC = 1 -
 T2

%e =  x 100
The percentage of the total variation that is due to measurement error is (1-) x 100
%e = (1- ) x 100
Measurement Stability
A further analysis is made to determine if the repeated measurements were statistically stable. Unstable
results are an indication that one or more test pairs may have an invalid reading causing a large
difference between the pairs. These values should be investigated for possible exclusion and
investigation of the measurement system.
Potential assignable causes:
• Typo,
• Mistake in the measurement process
• Change in the measurement gauge
• Change or degradation in the product being measured.

Measurement stability is assessed with a basic control chart for the ranges.

The upper control limit for ranges is calculated using:

UCLR = D4 R

where D4= 3.267 for n = 2

Any values outside the upper control limit would indicate a statistically significant difference in
measurement error for that pair.

2 of 8
MSA FOR CONTINUOUS DATA

Do NOT Perform Standard Regression or Correlation Tests:

Regression assumes that Y is dependent on X, this is not true in a repeated measurement test.
Measurement error is the scatter in the direction that is perpendicular to a 45 degree line (If there were
no measurement error, there would be a 1:1 relationship between the first reading and the second and the
‘best fit line’ would be a 45 degree line originating from zero.)
Error in a regression (where the X axis is a suspected causal factor to the Y axis factor) is in the vertical
direction: it is the variation in Y due to all other factors besides the X axis factor…
If regression is used to display measurement error, the regression line is the best fit line of the data
points so that bias will not be visually detected. With measurement error the “best fit line” is a 45° line.

Measurement Discrimination Plot

Best Fit
Regression
1.4 Line

1.4

1.3

1.3

1.2

1.2

1.1

1.1 45 degree line

1.0
1.0 1.1 1.2 1.3 1.4

3 of 8
MSA FOR CONTINUOUS DATA

METHOD COMPARISON
Comparison of measurement methods is a subset of Measurement Systems Analyses. One technique
that may be encountered in published studies is regression analysis. This is NOT the correct choice for
method analysis. Regression analysis relies on the calculation of the line of best fit and the data’s
adherence to this line in the Y axis direction. The best fit line and its accompanying r value only
consider variation in the Y, not the X vector. In the case of method comparison, both measurement
systems have variation that needs to be considered in analysis of method agreement. Also, just because
the data fits perfectly to an arbitrary line does not mean that there is a causal relationship between the
two methods.

The Youden plot and associated statistics is the correct method for plotting and analyzing data for
measurement systems analysis. The plot provides immediate visual indication as to whether any
systemic bias exists between measurements. The associated statistics directly calculate the variation due
to measurement error and the magnitude of this error with respect to the range of variation in the rest of
the system.

Another method that can be used in conjunction with the Youden is the Bland-Altman approach.3,4 The
Bland-Altman method uses the graphical approach of plotting the differences between the measurements
over the full range of values measured. In this approach, the quantitative range of bias can clearly be
seen over the sample range.

4 of 8
MSA FOR CONTINUOUS DATA

Mechanics Using the Youden Plot

The Youden plot for two separate measurement methods of SNAP sample spot OD is provided below.
Two measurements for each of 30 devices were taken for each method. The product variation was
minimal in this experiment; a single panel was run on all 30 devices. The repeatability and
discrimination of these two measurement systems were determined independently and each system is
considered to be adequate.

Method Comparison of SNAP OD Readings

Sample Spot Density for Single Panel Member
0.8

Systemic Bias
Measurement 2

0.6

Method 1

Method 2

45-degree Line
0.4
0.4 0.6 0.8

Measurem ent 1

The Youden plot above shows that while each measurement system is indeed repeatable within itself,
there is a systemic bias between the two methods.

5 of 8
MSA FOR CONTINUOUS DATA

An alternative way to show this on the Youden is to take the first measurement from each measurement
system and to plot those responses against each other. The Youden below shows the bias very clearly.

Comparison of Measurement Methods for SNAP

OD Reading
Sample Spot Density for Single Panel Member
0.8
Method 2

0.6

0.4
0.4 0.6 0.8

Method 1

While it is very clear that there is bias between the two methods, the Bland-Altman method is better able
to show the magnitude of the bias and provide statistical limits for the differences between the methods.
The basic method for performing the Bland-Altman analysis is as follows:
1. The difference between replicates (di) and the mean value ( X i ) for the two replicates are
calculated
2. The mean difference ( d ) is calculated, and the standard deviation (σd) of the differences is
calculated.
3. The individual differences are plotted against their respective mean value
4. d is used as a centerline and d + 2 limits are plotted

6 of 8
MSA FOR CONTINUOUS DATA

Method Comparison Formula

Difference between replicates, di: d i = bi − ai

(where a and b are the two respective readings of a single sample)

(ai + bi )
Mean of replicates, X i : Xi =
2

Centerline, d : d=
d i

Standard deviation of the differences, σd: d =  (d − d ) i

2

n −1

Upper and Lower Limits: d  2 d

7 of 8
MSA FOR CONTINUOUS DATA

Bland-Altman Plot
The Bland-Altman plot for comparison of the two measurement systems is provided below. It can be
seen from the calculation of the mean difference between methods that there is a systemic 0.04 bias
between the two methods, ranging from -0.11 to 0.03. No trend or “CV” effect can be observed over
this range of sample results. The differences between measurement methods appears to be random, and
so the calculated 2 σd limits can be considered a good estimation for future results.

Method Comparison: Bland-Altman Plot for Measurement Methods

0.15
Difference (Measurement B -

0.10
Measurement A)

0.05

0.00

-0.05

-0.10

-0.15
0.54 0.55 0.56 0.57 0.58 0.59 0.60 0.61
Mean Value

The range of differences between methods can be compared to the range seen within each measurement
system. As seen in the initial Youden plots, the first measurement system had much smaller variation
between measurements, with values ranging from -0.05 to 0.05.
The test of statistical significance for method comparison is the paired t test…

1
Donald J Wheeler, Richard W Lyday., Evaluating The Measurement Process, Second Edition, SPC Press, 1988
2
Donald S. Ermer and Robin Yang E-Hok, “Reliable data is an Important Commodity”, The Standard, ASQ Measurement
Society Newsletter, Winter 1997, pp. 15-30.
3
Bland, Martin, J., Altman, Douglas, G., “Statistical Methods For Assessing Agreement Between Two Methods Of Clinical
Measurement”, The Lancet, February 8, 1986
4
Dietmar Stockl, Diego Rodrıguez Cabaleiro, Katleen Van Uytfanghe, Linda M. Thienpont “Interpreting Method
Comparison Studies by Use of the Bland–Altman Plot: Reflecting the Importance of Sample Size by Incorporating
Confidence Limits and Predefined Error Limits in the Graphic”, Letter to the Editor of Clinical Chemistry, 50, No. 11, 2004

8 of 8