Nbsspecialpublication700 2

U.S.
DEPARTMENT OF COMMERCE
National Bureau of Standards
r
Ication 700-2
mm t Series
Measurement
Evaluation
J. Mandel and L. F. Nanni
NBS NBS NBS NBS NBS NBS NBS NBS NBS NBS /
iS NBS NBS NBS NBS NBS NBS NBS NBS NBS NBl
iS NBS NBS NBS NBS NBS NBS NBS NBS NBS NBi
tS NBS NBS NBS NBS NBS NBS NBS NBS NBS NBi
NBS NBS NBS NBS NBS NBS NBS NBS NBS NBS il
tS NBS NBS NBS NBS NBS NBS NBS NBS NBS NB^
NBS NBS NBS NBS NBS NBS NBS NBS NBS NBS A
iS NBS NBS NBS NBS NBS NBS NBS NBS NBS NB\-
NBS NBS NBS National Bureau of Standards NBS NBS /
NBS NBS NBS NBS NBS NBS NBS NBS NBI
m^^BS NBS NBS NBS NBS NBS NBS NBS NBS /
Wmm NBS NBS NBS NBS NBS NBS NBS NBS NB\
W^^BS NBS NBS NBS NBS NBS NBS NBS NBS A
Q^P NBS NBS NBS NBS NBS NBS NBS NBS NB,
.
TM
# he National Bureau of Standards' was established by an act of Congress on March 3, 1901. The
Bureau's overall goal is to strengthen and advance the nation's science and technology and facilitate
their effective application for public benefit. To this end, the Bureau conducts research and provides: (1) a
basis for the nation's physical measurement system, (2) scientific and technological services for industry and
government, (3) a technical basis for equity in trade, and (4) technical services to promote public safety.
The Bureau's technical work is performed by the National Measurement Laboratory, the National
Engineering Laboratory, the Institute for Computer Sciences and Technology, and the Institute for Materials
Science and Engineering
The National Measurement Laboratory

Provides the national system of physical and chemical measurement; • Basic Standards^
coordinates the system with measurement systems of other nations and • Radiation Research
furnishes essential services leading to accurate and uniform physical and • Chemical Physics
chemical measurement throughout the Nation's scientific community, in- • Analytical Chemistry
dustry, and commerce; provides advisory and research services to other

Government agencies; conducts physical and chemical research; develops,
produces, and distributes Standard Reference Materials; and provides
calibration services. The Laboratory consists of the following centers:
The National Engineering Laboratory

Provides technology and technical services to the public and private sectors to Applied Mathematics
address national needs and to solve national problems; conducts research in Electronics and Electrical
engineering and applied science in support of these efforts; builds and main- Engineering'
tains competence in the necessary disciplines required to carry out this Manufacturing Engineering
research and technical service; develops engineering data and measurement Building Technology
measurement traceability services; develops
capabilities; provides engineering Fire Research
testmethods and proposes engineering standards and code changes; develops Chemical Engineering^
and proposes new engineering practices; and develops and improves
mechanisms to transfer results of its research to the ultimate user. TTie
Laboratory consists of the following centers:
The Institute for Computer Sciences and Technology

Conducts research and provides scientific and technical services to aid Programming Science and
Federal agencies in the selection, acquisition, application, and use of com- Technology
puter technology to improve effectiveness and economy in Government Computer Systems
operations in accordance with Public Law 89-306 (40 U.S.C. 759), relevant Engineering
Executive Orders, and other directives; carries out this mission by managing
the Federal Information Processing Standards Program, developing Federal
ADP standards guidelines, and managing Federal participation in ADP
voluntary standardization activities; provides scientific and technological ad-
visory services and assistance to Federal agencies; and provides the technical
foundation for computer-related policies of the Federal Government. The In-
stitute consists of the following centers:
77?^ Institute for Materials Science and Engineering

Conducts research and provides measurements, data, standards, reference Ceramics
^
materials, quantitative understanding and other technical information funda- Fracture and Deformation
mental to the processing, structure, properties and performance of materials; Polymers
addresses the scientific basis for new advanced materials technologies; plans Metallurgy
research around cross-country scientific themes such as nondestructive Reactor Radiation
evaluation and phase diagram development; oversees Bureau-wide technical
programs in nuclear reactor radiation research and nondestructive evalua-
tion; and broadly disseminates generic technical information resulting from
its programs. The Institute consists of the following Divisions:
'Headquarters and Laboratories at Gaithersburg, MD, unless otherwise noted; mailing address
Gaithersburg, MD
20899.
'Some di' isions within the center are located at Boulder, CO 80303.
'Located at Boulder, CO, with some elements at Gaithersburg, MD.
NBS Special Publication 700-2
Industrial Measurement Series
Measurement
Evaluation
J. Mandel
National Measurement Laboratory
Gaithersburg, Maryland 20899
and
L. F. Nanni
Federal University
Porto Alegre, Brazil
March 1986
U.S. Department of Commerce

Malcolm Baldrige, Secretary

Ernest Ambler, Director
Library of Congress U.S. Government Printing Office For sale by the Superintendent
Catalog Card Number: 86-60051 0 Washington: 1986 of Documents
National Bureau of Standards U.S. Government Printing Office,
Special Publication 700-2 Washington, DC 20402
Natl. Bur. Stand. (U.S.),
Spec. Publ. 700-2,
70 pages (March 1986)
CODEN: XNBSAV
FOREWORD •s.
When the National Bureau of Standards was established more than 80

years ago, it was given the specific mission of aiding manufacturing
and commerce. Today, NBS remains the only Federal laboratory with this
explicit goal of serving U.S. industry and science. Our mission takes
on special significance now as the country is responding to serious
—
challenges to its industry and manufacturing challenges which call for
government to pool its scientific and technical resources with industry
and universities.
The links between NBS staff members and our industrial colleagues
have always been strong. Publication of this new Industrial
Measurement Series, aimed at those responsible for measurement in
industry, represents a strengthening of these ties.
The concept for the series stems from the joint efforts of the
National Conference of Standards Laboratories and NBS. Each volume
will be prepared jointly by a practical specialist and a member of the
NBS staff. Each volume will be written within a framework of
industrial relevance and need.
This publication is an addition to what we anticipate will be a

long series of collaborative ventures that will aid both industry and
NBS.
Ernest Ambler, Director
i i i
INTRODUCTION
This paper was published originally as a chapter in the book

entitled "Quality Assurance Practices for Health Laboratories".* It is
for that reason that the examples used as illustrations are taken from
health-related fields of research. However, the statistical concepts
and methods presented here are entirely general and therefore also
applicable to measurements originating in physics, chemistry,
engineering, and other technical disciplines. The reader should have
no difficulty in applying the material of this paper to the systems of
measurement in his particular field of activity.
J. Mandel
January, 1986
* J. Mandel and L.F. Nanni, Measurement Evaluation Quality Assurance

Practices for Health Laboratories. Washington: American Public
Health Association; 1978: 209-272.124^1 p.
iv
ABOUT THE AUTHORS
John Mandel
John Mandel holds an M.S. in chemistry from the University of

Brussels. He studied mathematical statistics at Columbia University
and obtained a Ph.D in statistics from the University of Eindhoven.
Dr. Mandel has been a consultant on statistical design and data

analysis at the National Bureau of Standards since 1947. He is the
author of a book, "^he Statistical Analysis of Experimental Data", and
has contributed chapters on statistics to several others. He has
written numerous papers on mathematical and applied statistics, dealing
more particularly with the application of statistical methodology to
the physical sciences.
Mandel has served as a Visiting Professor at Rutgers University

and at the Israel Institute of Technology in Haifa. He has contributed
to the educational program of the Chemical Division of the American
Society for Quality Control through lectures and courses.
A fellow of the American Statistical Association, the American

Society for Testing and Materials, the American Society for Quality
Control, and the Royal Statistical Society of Great Britain, Mandel, is
the recipient of a number of awards, including the U.S. Department of
Commerce Silver Medal and Gold Medal, the Shewhart Medal, the Dr. W.
Edwards Deming Medal, the Frank Wilcoxon Prize and the Brumbaugh Award
of the American Society for Quality Control.
He was Chairman of one of the Gordon Research Conferences on

Statistics in Chemistry and Chemical Engineering and has served on
several ASTM committees and is, in particular, an active member of
Committee E-1 on Statistical Methods.
1
Luis F. Nanni
Luis F. Nanni holds a Civil Engineering degree from the National

University of Tucuman, Argentina and the M.A. from Princeton
University. He was a member of the faculty of Rutgers University
School of Engineering for many years and served there as Professor of
Industrial Engineering. Professor Nanni also has extensive experience
as an industrial consultant on statistics in the chemical sciences, the
physical sciences and the health sciences. He is a member of several
professional societies including the American Statistical Association,
the Institute of Mathematical Statistics, the Operations Research
Society of America, the American Institute of Industrial Engineers the
American Society for Engineering Education.
Professor Nanni 's fields of specialization are statistical

analysis and operations research; his scholarly contributions include
statistical methods, random processes and simulation, computer
programming and engineering analysis. At the present time he is
Professor of Civil Engineering at the Federal University in Porto
Alegre, Brazil.
V
CONTENTS
Page
Foreword iii
Introduction iv
About the authors v
1 . Basic Statistical Concepts 1
Random variables 1
Frequency distribution and histograms 1
Population Parameters and Sample Estimates 2

Random Samples 2
Population Parameters-General Considerations 4
Sample Estimates 4
Population Parameters As Limting Values of Sample Estimates 4
Sums of Squares, Degrees of Freedom, and Mean Squares ... 5
Grouped Data 6
Standard Error of the Mean 7
Improving Precision Through Replication 8
Systematic errors .... 8
The normal distribution 8
Symmetry and Skewness 8
The central limit theorm 9
The Reduced Form of a Distribution 9
Some numerical Facts About the Normal Distribution .... 10
The Concept of Coverage 10
Confidence Intervals 10
Confidence Intervals for the Mean 11
Confidence Intervals for the Standard Deviation 13
Tolerance Intervals 14
Tolerance Intervals for Average Coverages 15
Non- parametric Tolerance Intervals-Order Statistics.... 16
Tolerance Intervals Involving Confidence Coefficients ... 17
Non-normal Distributions and Testsof Normality 17
Tests of normality 17
The binomial Distribution ...... 18
The Binomial Parameter and its Estimation 19
The Normal Approximation for the Binomial Distribution . . 20
Precision and Accuracy 21
The Concept of Control 21
Within-and Between-Laboratory Variability 21
Accuracy-Comparison With Reference Values 23
Straight Line Fitting 24
A General Model 25
Formulas for Linear Regression 26
Examination of Residuals-Weighting 26
vi
Propagation of Errors 27
An example 27
The General Case 28
Sample Sizes and Compliance with Standards 30
An Example 30
General Procedure-Acceptance, Rejection, Risks 31
Inclusion of Between-Laboratory Variability 32
Transformation of Scale 33
Some Common Transformations 33
Robustness 33
Transformations of Error Structure 34
Presentation of Data and Significant Figures 35
An Example 35
General Recommendations 37
Tests of Significance 37
General Considerations 37
Alternative Hypotheses and Sample Size-The Concept of Power 38
An Example 39
Evaluation of Diagnostic Tests 40
Sensitivity and Specificity 41
Predictive Values-The Concept of Prevalance 41
Interpretation of Multiple Tests 42
A General Formula for Multiple Independent Tests 43
2. Quality Control 44
3. The Control Chart 44

Statistical Basis for the Control Chart 45
General Considerations 45
Control Limits 45
Variability Between and Within Subgroups 47
Types of Control Charts 48
Preparing a Control Chart 48
Objective and Choice of Variable . 48
Selecting a Rational Subgroup 49
Size and Frequency of Control Sample Analyses 49
Maintaining Uniform Conditions in Laboratory Practice ... 49
Initiating a Control Chart 49
Determining Trial Control Limits 50
Computing Control Limits 50
Calculating the Standard Deviation 51
Control Limits for the Chart of Averages 52
Control Limits for the Chart of Ranges 52
Initial Data 53
Computing Trial Control Limits 54
Analysis of Data 54
Additional Data 55
Future Control Limits 56
Control Chart for Individual Determinations 58
vii
other Types of Control Charts 59
Control Chart for Attributes-The P-Chart
Control Limits and Warning Limits 59
Control Charts for Number of Defects Per Unit-The C-Chart . 60
The Poisson Distribution 61
Detecting Lack of Randomness 61
Rules Based on the Theory of Runs 61
Distribution of Points Around the Central Line 62
Interpreting Patterns of Variation in a Control Chart 62
Indication of Lack of Control 62
Patterns of Variation 62
The Control Chart as a Management Tool 63
References 64
viii
Measurement
Evaluation
J. Mandel (principal author), and L. F. Nanni.
Basic Statistical Concepts
Random variables
This chapter concerned with the evaluation of measurements by

is
means of methods. This qualification is important, for the total

statistical
evaluation of measurements involves many different points of view. What
differentiates the statistical viewpoint from all others is that each measure-
ment is considered as only one realization of a hypothetical infinite popu-
lation of similar measurements. Although, in general, all members of this
population refer to the measurements of the same property on the same
sample (e.g., the glucose content of a given sample of serum), they are not
expected to be identical. The differences among them are attributable to
chance effects, due to unavoidable fluctuations in many of the conditions sur-
rounding the measuring process. Alternatively, the members of the popu-
lation of measurements may refer to different samples, or different individ-
uals. Thus, one may consider the glucose content of serum of all healthy indi-
viduals in a certain age range. In such cases, the observed differences among
the measured values include what is referred to as sampling error, meaning
the differences in the measured property among the members of the popu-
lation of samples or individuals. A variable whose value is associated with a
statistical population is called a random variable or variate.
Frequency distribution and iiistograms

A mathematical representation can be made of a statistical population,
such as the hypothetical infinite population of measurements just mentioned.
To obtain this representation, called ^frequency distribution one divides all ,
the measurements in the population into group intervals and counts the num-
ber of measurements in each interval. Each interval is defined in terms of its
lower and upper limit, in the scale in which the measurement is expressed.
Since in practice one is always limited to a statistical sample, i.e., a finite
number of measurements, one can at best only approximate the frequency
distribution. Such an approximation is called a histogram. Figure 4.1 contains
a histogram of glucose values in serum measurements on a sample of 2,197
individuals. It is worth noting that the frequency tends to be greatest in the
vicinity of the mean and diminishes gradually as the distance from the mean
1
HISTOGRAM FOR GLUCOSE IN SERUM
400
300
>-
o
UJ
Z) 200
o
UJ
-
100
50 75 100 125 150 175

GLUCOSE (mg/dl)
Fig. 4.1. Histogram of glucose serum values on a sample of 2,197 individuals, with a
range of 47.5-157.5 mg/dl and a mean of 100.4 mg/dl.
increases. The grouped data on which the histogram is based are given in
Table 4.1.
Population parameters and sample estimates
Random samples
The sample of individuals underlying the histogram in Table 4. 1 is rather
large. A large size, in itself, does not necessarily ensure that the histogram's
characteristics will faithfully represent those of the entire population. An ad-
ditional requirement is sample be obtained by a random selection
that the
from the entire population. A random selection is designed to ensure that
each element of the population has an equal chance of being included in the
sample. A sample obtained from a random selection is called a random
sample, although, strictly speaking, it is not the sample but the method of
obtaining it that is random. Using the concept of a random sample, it is pos-
sible to envisage the population as the limit of a random sample of ever-in-
creasing size. When the sample size N
becomes larger and larger, the charac-
teristics of the sample approach those of the entire population. If the random
sample is as large as the sample used in this illustration, we may feel con-
fident that its characteristics are quite similar to those of the population.
2
T
Table 4.1. Grouped Data for Glucose in Serum

Glucose Number of Glucose Number of
(mg/dl) individuals (mg/dl) individuals
47 S 1
1c\n c
lU/.J 313
L 111 c
112.5 220
S7 S L 111
11/.^ 132
-I
Di. J J 122.5 50
D / . J 1
12 /.5 26
/Z. J 20 132.5 8
77.5 52 137.5 6
82.5 118 142.5 4
87.5 204 147.5 1
92.5 281 152.5 0

97.5 351 157.5 1
102.5 390
Total number of individuals: 2,197
Thus, upon inspection of Table 4.1, we may feel confident that the mean se-
rum glucose for the entire population is not far from 100.4 mg/dl. We also
may feel confident in stating that relatively very few individuals, say about 1
percent of the entire population, will have serum glucose values of less than
70 mg/dl. Our confidence in such conclusions (which, incidentally, can be
made more quantitative), however, would have been much less had all of the
available data consisted of a small sample, say on the order of five to 50 indi-
viduals. Two such sets of data are shown in Table 4.2. Each represents the
serum glucose of ten individuals from the population represented in Table
4.1. The mean glucose contents of these samples are 107.57 and 96.37 mg/dl,
respectively. If either one of these samples was all the information available
Table 4.2. Two Small Samples of Glucose in Serum
Sample I Sample II
Individual Glucose (mg/dl) Individual Glucose (mg/dl)
1 134.2 1 88.2
2 119.6 2 82.0
3 91.9 3 96.0
4 96.6 4 94.1
5 118.8 5 96.3
6 105.2 6 108.8
7 103.4 7 106.3
8 112.1 8 101.1
9 97.0 9 89.4
10 96.9 10 101.7
Average 107.57 Average 96.37

Variance 179.44 Variance 70.48
Standard deviation 13.40 Standard deviation 8.40
3
to us,what could we have concluded about the mean serum glucose of the
entire population?And, in that case, what could we have stated concerning
the percentage of the population having a serum glucose of less than 70
mg/dl?
Population parameters —general considerations

The answer to these and similar questions requires that we first define
some basic characteristics of a statistical sample and relate them to the char-
acteristics of the population. Fortunately,most populations can be charac-
terized in terms of very few quantities, called parameters In many cases, .
only two parameters are required, in the sense that these two parameters
contain practically all the pertinent information that is required for answer-
ing all useful questions about the population. In cases where more than two
parameters are needed, it is often possible to perform a mathematical opera-
tion, called a transformation of scale, on the measured values, which will
reduce the required number of parameters to two. The two parameters in
question are the mean and the standard deviation, measuring, respectively,
the location of the center of the population and its spread.
Sample estimates
Let JCi, JC2,
. . Xs represent a sample of
.
,
measurements belonging to
a single population. The sample mean is generally denoted by x and defined
by
^ Xi + X2 + . . . + Xn _ XXj
N ~ N ^^-^^
The sample variance is denoted by si and defined by
(4.2)
N - 1
The sample standard deviation is denoted by Sj. and defined by
Sx^\/sf (4.3)
Table 4.2 contains, for each of the samples, the numerical values of x, si,
and Sj..
Population parameters as limiting values of sample estimates

The quantities defined by Equations 4.1, 4.2, and 4.3 are not the popu-
lation parameters themselves but rather are sample estimates of these pa-
rameters. This distinction becomes apparent by the fact that they differ from
sample to sample, as seen in Table 4.2. However, it is plausible to assume
that as the sample size N becomes very large, the sample estimates become
more and more stable and eventually approach the coiTesponding population
parameters. We thus define three new quantities: the population mean, de-
noted by the symbol /x; the population variance, denoted by the symbol a'i
or by the symbol Var(.r); and the population standard deviation, denoted by
(Tr. Thus:
o-^ = Vo-l = VVarU) (4.4)
It is customary to denote population parameters by Greek letters (e.g., /x, a)

and sample estimates by Latin letters (e.g., s). Another often used conven-
tion is to represent sample estimates by Greek letters topped by a caret C);
thus s and & both denote a sample estimate of a. It is apparent from the
above definitions that the variance and the standard deviation are not two
independent parameters, the former being the square of the latter. In prac-
tice, thestandard deviation is the more useful quantity, since it is expressed
in the same units as the measured quantities themselves (mg/dl in our ex-
ample). The variance, on the other hand, has certain characteristics that
make it theoretically desirable as a measure of spread. Thus, the two basic
parameters of a population used in laboratory measurement are: (a) its
mean, and (b) either its variance or its standard deviation.
Sums of squares, degrees of freedom, and mean squares

Equation 4.2 presents the sample variance as a ratio of the quantities
X{Xi - and (N - 1). More generally, we have the relation:
MS =^ (4.5)
where MS stands for mean square, SS for sum of squares, and DF for de-
grees offreedom. The term "sum of squares" is short for "sum of squares of
deviations from the mean," which is, of course, a literal description of the
expression X(Xi - x)^, but it is also used to describe a more general concept,
which will not be discussed at this point. Thus, Equation 4.2 is a special case
of the more general Equation 4.5.
The reason for making the divisor N - rather than the
1 more obvious
N can be understood by noting that the N quantities
Xi X, X2 X, . . .
, Xf^ X
are not completely independent of each other. Indeed, by summing them we

obtain:
^ (Xi - i) = Xxj - Sjc = Ixi - Nx (4.6)
Substituting for x the value given by its definition (Equation 4. 1), we obtain:
X {x< -x) = lx^-N ^ =0 (4.7)
This relation implies that if any {N - 1) of the quantities {Xi - x) are giv- N
en, the remaining one can be calculated without ambiguity. It follows that
while there areN independent measurements, there are only - 1 indepen- N
dent deviations from the mean. We express this fact by stating that the
sample variance is based on - 1 degrees offreedom. This explanation pro-
vides at least an intuitive justification for using - 1 as a divisor for the N
calculation of s^. When N
is very large, the distinction between and - 1 N N
becomes unimportant, but for reasons of consistency, we always define the
5
sample variance and the sample standard deviation by Equations 4.2 and
4.3.
Grouped data
When the data in a sample are given in grouped form, such as in Table
4. 1 ,
and 4.2 cannot be used for the calculation of the mean and
Equations 4. 1
the variance. Instead, one must use diflFerent formulas that involve the mid-
points of the intervals (first column of Table 4.1) and the corresponding fre-
quencies (second column of Table 4.1).
Formulas for grouped data are given below.
To differentiate the regular average (Equation 4.1) of a set of jc; values
from their "weighted average'' (Equation 4.8), we use the symbol x (x tilde)
for the latter.
5. ^v^r (4.10)
where / (the "frequency") represents the number of individuals in the zth

interval,and x, is the interval midpoint. The calculation of a sum of squares
can be simplified by "coding" the data prior to calculations. The coding con-
sists of two operations:
1) Find an approximate central value Xo (e.g., 102.5 for our illustration) and
subtract it from each Xj.
2) Divide each difference Xj - x^by a. convenient value r, which is generally
the width of the intervals (in our case, c = 5.0).
Let the mean
— c
(4.11)
The weighted average u is equal to (i - Xo)/c. Operation (1) alters neither the
variance nor the standard deviation. Operation (2) divides the variance by
and the standard deviation by c. Thus, "uncoding" is accomplished by multi-
plying the variance of // by and the standard deviation of // by c. The for-
mulas in Equations 4.8, 4.9, and 4. 10 are illustrated in Table 4.3 with the data
from Table 4.1.
We now can better appreciate the difference between population param-
eters and sample estimates. Table 4.4 contains a summary of the values of
the mean, the variance, and the standard deviation for the population (in this
case, the very large sample TV = 2,197 is assumed to be identical with the
population) and for the two samples of size 10.
6
1
Table 4.3. Calculations for Grouped Data

X u f X 1
Li f
47.5 -11 1 107.5 1 313

52.5 -10 2 112.5 2 220
57.5 -9 2 117.5 3 132
62.5 -8 3 122.5 4 50
67.5 -7 12 127.5 5 26
72.5 1 n
liZ.j? 6 8
77.5 -5 52 137.5 7 6
82.5 -4 118 142.5 8 4
87.5 --3 204 147.5 9 1
92.5 -2 281 152.5 10 0

97.5 -1 351 157.5 U 1
102.5 0 390
u — -0.4156 X = 102.5 + 5ii = 100.42

si - 5.9078 si = 254 = 147.70
Su = 2.4306 12.15
We first deal with the question: "How rehable is a sample mean as an

estimate of the population mean?" The answer requires the introduction of
—
two important concepts the standard error of the mean and the method of
confidence intervals. Before introducing the latter, however, it is necessary
to discuss normal distribution.
Standard error of the mean
The widely held, intuitive notion that the average of several measure-
ments is "better" than a single measurement can be given a precise meaning
by elementary statistical theory.
Let Xv represent a sample of size
Xi, a:2, . . . taken from a population
, N
of mean /X and standard deviation cr.
Let jCi represent the average of the measurements. We can visualize a N
repetition of the entire process of obtaining the results, yielding a new av- N
erage X2. Continued repetition would thus yield a series of averages jCj,
. . (Two such averages are given by the sets shown in Table 4.2). These
. .
averages generate, in turn, a nev/ population. It is intuitively clear, and can

readily be proved, that the mean of the population of averages is the same as
that of the population of single measurements, i.e., fx. On the other hand, the
Table 4.4. Population Parameter and Sample Estimates (Data of Tables 4. and 1 4.2)
Source Mean (mg/dl) Variance (mg/dl)^ Standard Deviation (mg/dl)
Population^ 100.42 147.70 12.15

Sample I 107.57 179.55 13.40
Sample II %.37 70.56 8.40
^We consider the sample of Table 4.1 as identical to the population.
7
variance of the population of averages can be shown to be smaller than that
of the population of single values, and, in fact, it can be proved mathemati-
cally that the following relation holds:
VarW
VarU) =--j^ — (4.12)
From Equation 4.12 it follows that
cr.
(4.13)
This relation is known as the law of the standard error of the mean, an ex-
pression simply denoting the quantity cr^. The term standard error refers to
the variability of derived quantities (in contrast to original measurements).
Examples are: the mean of individual measurements and the intercept or
the slope of a fitted line (see section on straight line fitting). In each case, the
derived quantity is considered a random variable with a definite distribution
function. The standard error is simply the standard deviation of this distribu-
tion.
Improving precision tlirough replication

Equation 4. 13 justifies the above-mentioned, intuitive concept that aver-
ages are "better" than single values. More rigorously, the equation shows
that the precision of experimental results can be improved, in the sense that
the spread of values is reduced, by taking the average of a number of repli-
cate measurements. It should be noted that the improvement of precision
through averaging is a rather inefficient process; thus, the reduction in the
standard deviation obtained by averaging ten measurements is only VTo, or
about 3, and it takes 16 measurements to obtain a reduction in the standard
deviation to one-fourth of its value for single measurements.
Systematic errors
Asecond observation concerns the important assumption of random-
ness required for the validity of the law of the standard error of the mean.
The N values must represent sl random sample from the original population.
If, for example, systematic errors arise when going from one set of N meas-
urements to the next, these errors are not reduced by the averaging process.
An important example of this is found in the evaluation of results from difi'er-
ent laboratories. If each laboratory makes N
measurements, and if the within-
laboratory replication error has a standard deviation of a, the standard
deviation between the averages of the various laboratories will generally be
larger than cr/VN, because additional variability is generally found between
laboratories.
The normal distribution
Symmetry and skewness

The mean and standard deviation of a population provide, in general, a
great deal of information about the population, by giving its central location
8
and its spread. They fail to inform us, however, as to the exact way in which
the values are distributed around the mean. In particular, they do not tell us
whether the frequency or occurrence of values smaller than the mean is the
same as that of values larger than the mean, which would be the case for a
symmetrical distribution. A nonsymmetrical distribution is said to be skew,
and it is possible to define a parameter of skewness for any population. As in
the case of the mean and the variance, we can calculate a sample estimate of
the population parameter of skewness. We will not discuss this matter fur-
ther at this point, except to state that even the set of three parameters, mean,
variance, and skewness, is not always sufficient to completely describe a
population of measurements.
The central limit theorem
Among the infinite variety of frequency distributions, there is one class

of distributions that is of particular importance, particularly for measure-
ment data. This is the class of normal, also known as Gaussian, distribu-
tions. Allnormal distributions are symmetrical, and furthermore they can be

reduced by means of a simple algebraic transformation to a single distribu-
tion, known as the reduced normal distribution The practical importance of
.
the class of normal distributions is related to two circumstances: (a) many

sets of data conform fairly closely to the normal distribution; and (b) there
exists a mathematical theorem, known as the central limit theorem, which
asserts that under certain very general conditions the process of averaging
data leads to normal distributions (or very closely so), regardless of the
shape of the original distribution, provided that the values that are averaged
are independent random drawings from the same population.
The reduced form of a distribution
Any normal distributioncompletely specified by two parameters, its

is
mean and its variance (or, alternatively, its mean and its standard deviation).
Let a: be the result of some measuring process. Unlimited repetition of
the process would generate a population of values Xi, X21 -^31 If the fre-
• • • •
quency distribution of this population of values has a mean fi and a standard

deviation of cr, then the change of scale efiTected by the formula
z =^^^^ '
(4.14)
cr
will result in a new frequency distribution of a mean value of zero and a

standard deviation of unity. The z distribution is called the reduced form of
the original x distribution.
If, in particular, x is normal, then z will be normal too, and is referred to
as thereduced normal distribution.

To understandthe meaning of Equation 4.14, suppose that a particular
measurement x lies at a point situated at k standard deviations above the
mean. Thus:
x = fJL + ka
9
Then, the corresponding z value will be given by
cr
Thus the z value simply expresses the distance from the mean, in units of
standard deviations.
Some numerical facts about the normal distribution

The following facts about normal distributions are noteworthy and
should be memorized for easy appraisal of numerical data:
1) In any normal distribution, the fraction of values whose distance from the
mean (in either direction) is more than one standard deviation is approxi-
mately one-third (one in three).
mean is more than two standard deviations, is approximately 5 percent
(one in twenty).
mean is more than three standard deviations is approximately 0.3 percent
(three in one thousand).
These facts can be expressed more concisely by using the reduced form
of the normal distribution:
1) Probability that |z| > 1 is approximately equal to 0.33.
2) Probability that |z| > 2 is approximately equal to 0.05.
3) Probability that |z| > 3 is equal to 0.003.
The concept of coverage

If we define the coverage of an interval from A to 5 to be the fraction of
values of the population falling inside this interval, the three facts (1), (2),
and (3) can be expressed as follows (where "sigma" denotes standard devia-
tion):
1) A plus-minus one-sigma interval around the mean has a coverage of about

2/3 (67 percent).
2) A plus-minus rwo-sigma interval around the mean has a coverage of about
95 percent.
3) A plus-minus three-sigma. interval around the mean has a coverage of 99.7
percent.
The coverage corresponding to a ±z-sigma interval around the mean has
been tabulated for the normal distribution for values of z extending from 0 to
4 in steps of 0.01, and higher in larger steps. Tabulations of the reduced nor-
mal distribution, also known as the "normal curve," or "error curve," can
be found in most handbooks of physics and chemistry,^ and in most text-
books of statistics. ^"^ Since the coverage corresponding to z = 3.88 is 99.99
percent, it is hardly ever necessary to consider values ofz larger than four.
Confidence intervals
A confidence interval aims at bracketing the true value of a population

parameter, such as its mean or its standard deviation, by taking into account
the uncertainty of the sample estimate of the parameter.
10
Let
Xi, X2, . . represent a sample of size N from a population of
.
,
mean and standard deviation a. In general fx and a are unknown, but can
/jl
be estimated from the sample in terms of i and s, respectively.
Confidence intervals for ttie mean

Aconfidence interval for the mean /x is an interval, AB, such that we
can state, with a prechosen degree of confidence, that the interval AB brack-
ets the population mean (jl.
For example, we see in Table 4.3 that the mean of either of the two sam-
ples of size 10 is appreciably different from the (true) population mean
(100.42 mg/dl). But suppose that the first of the two small samples is all
the information we possess. We then would wish to find two values, A and
B, derived completely from the sample, such that the interval is likely to
include the true value (100.42). By making this interval long enough we can
always easily fulfill this requirement, depending on what we mean by 'iike-
ly." Therefore, we first express this qualification in a quantitative way by
stipulating the value of a confidence coefficient Thus we may require that
.
the interval shall bracket the population mean "with 95 percent con-
fidence." Such an interval is then called a "95 percent confidence interval."
—
The case of known cr. We proceed as follows, assuming for the mo-
ment that although is unknown, the population standard deviation cr is
known. We will subsequently drop this restriction.

We have already seen that the population of averages, x, has mean ^t
and standard deviation a/\/N. The reduced variate corresponding to x is
therefore:
z= ^~ ^ (4.15)
a/ \/ N
By virtue of the central limit theorem, the variable x generally may be
considered to be normally distributed. The variable z then obeys the reduced
normal distribution. We can therefore assert, for example, that the probabili-
ty that
-1.96 < z < 1.96 (4.16)
is 95 percent. Equation 4.16 can be written
~
-1.96 <-, ^ ^ < 1.96
a/ VN
or
i-1.96-^ <M<i+1.96 (4.17)
probability that this double inequality will be fulfilled is 95 percent.

The
Consequently, Equation 4.17 provides a confidence interval for the mean.
The lower limit A of the confidence interval is i - .96 a/\^\ its upper lim-
1
it B isx + 1.96 a/\/N. Because of the

particular choice of the quantity 1.96,
the probability associated with this confidence interval is, in this case, 95
11
percent. Such a confidence interval is said to be a "95 percent confidence
interval," or to have a confidence coefficient of 0.95. By changing 1.96 to
3.00 in Equation 4.17, we would obtain a 99.7 percent confidence interval.
General formula for the case of known a. More generally, from the —
table of the reduced normal distribution, we can obtain the proper critical
value Z(. (to replace 1.96 in Equation 4.17) for any desired confidence
coefficient. The general formula becomes
X - — < IX < X + z,.

—~ (4.18)
\/n Vn
Values of z^. for a number of confidence coefficients are listed in tables of
the normal distribution.
The length L of the confidence interval given by Equation 4.18 is
L = ix + z,- ]-{x-Zc- )-2z,- (4.19)
The z^. and also L. It

larger the confidence coefficient, the larger will be
is also apparent that L
increases with cr, but decreases as becomes larger. N
This decrease, however, is slow, as it is proportional to only the square root
of N. By far the best way to obtain short confidence intervals for an un-
known parameter choose a measuring process for which the dispersion
is to
cr is small —
in other words, to choose a measuring process of high precision.
The case of unknown cr. Student's t distribution. A basic difficulty as- —
sociated with the use of Equation 4. 18 is that cr is generally unknown. How-
ever, the sample of N
values provides us with an estimate s of a. This esti-
mate has N - 1 degrees of freedom. Substitution of s for a in Equation 4. 18
is not permissible, since the use of the reduced normal variate z in Equation
4.15 is predicated on a knowledge of cr.

It has been shown, however, that if x and s are the sample estimates
obtained from a sample of size N, from a normal population of mean and /jl
standard deviation cr, the quantity, analogous to Equation 4.15, given by
/ - (4.20)
s/ VN
has a well-defined distribution, depending only on the degrees of freedom,
A/ - 1, with which s has been estimated. This distribution is known as Stu-
dent's / distribution with N
- 1 degrees of freedom.
For a unknown, it is still possible, therefore, to calculate confidence in-
tervals for the mean by substituting in Equation 4.18 5 for cr, and tr for z^.
The confidence interval is now given by
X - t,
^- < IX < X + tc - — (4.21)
Vn Vn
The critical value tc, for any desired confidence coefficient, is obtained
from a tabulation of Student's / distribution. Tables of Student's t values can
12
.
be found in several references. xhe length of the confidence interval

based on Student's t distribution is
L = It, —^ (4.22)
Vn
For any given confidence coefficient, t, will be larger than z^, so that the
length of the interval given by Equation 4.22 is larger than that given by
Equation 4. 19. This difference is to be expected, since the interval now must
take into account the uncertainty of the estimate s in addition to that of x.
Applying Equation 4.21 to the two samples shown in Table 4.2, and
choosing a 95 percent confidence coefficient (which, for 9 degrees of free-
dom, gives tc = 2.26), we obtain:
1) For the first sample:
107.57 - 2.26 < 107.57 + 2.26

VlO VlO
or
98.0 < /A < 117.2
The length of this interval is
117.2 - 98.0 = 19.2
2) For the second sample:
^'^^
96.37 - 2.26 -^4?: < /a < 96.37 + 2.26
VTo VTo
or
90.4 < /Lt < 102.4
The length of this interval is
102.4 - 90.4 = 12.0
Remembering that the population mean

see that the confidence
is 100.4, we
intervals, though very different in length from each other, both bracket the
population mean. We also may conclude that the lengths of the intervals,
which depend on the sample size, show that a sample of size 10 is quite un-
satisfactory when the purpose is to obtain a good estimate of the population
mean, unless the measurement process is one of high precision.
Confidence intervals for tlie standard deviation

—
The chi-square distribution. In many statistical investigations, the
standard deviation of a population is of as much interest, if not more, than
the mean. It is important, therefore, to possess a formula that provides a con-
fidence interval for the unknown population standard deviation cr, given a
sample estimate s
13
If the number of degrees of freedom with which 5 is estimated is denoted
by V, a confidence interval for a is given by the formula:
<fji<s /-^ (4.23)
In this formula, the quantities X f and Xl are the appropriate upper and low-
er percentage points of a statistical distribution known as chi-square, for the
chosen confidence coefficient. These percentage points are found in several
'^"^
references.
This formula can be illustrated by means of the two samples in Table
4.2. To calculate 95 percent confidence intervals for a (the population stand-
ard deviation), we locate the limits at points corresponding to the upper and
lower 2.5 percentage points (or the 97.5 percentile and the 2.5 percentile) of
chi-square. From the chi-square table we see that, for 9 degrees of freedom,
the 97.5 percentile is 19.02, and the 2.5 percentile is 2.70. The 95 percent
confidence interval in question is therefore:
1) For the first sample:
13.40 < a< 13.40

2.70
or
9.2 < o- < 24.5
2) For the second sample:
8.40
V
/ ^
19.02
< o- < 8.40
V
/
2.70
or
5.8 < o- < 15.3
Here again, both intervals bracket the population standard deviation 12.15,
but again the lengths of the intervals reflect the inadequacy of samples of
size 10 for a satisfactory estimation of the population standard deviation.
Tolerance intervals
we observed that it was possible to

In introducing the data of Table 4.1,
infer that percent of the population has serum glucose values of less
about 1
than 70 mg/dl. This inference was reliable because of the large size of our
sample (N = 2,197). Can similar inferences be made from small samples,
such as those shown in Table 4.2? Before answering this question, let us first
see how the inference from a very large sample (such as that of Table 4.1)
can be made quantitatively precise.
The reduced variate for our data is
14
.
^ X - _ X - 100.42
' '
fx <
^ ~ a ~ 12.15
Making x = 70 mg/dl, we obtain for the corresponding reduced variate:
70 - 100.42
12.15
If we now assume that the serum glucose data are normally distributed (i.e.,
follow a Gaussian distribution), we read from the table of the normal distribu-
tion that the fraction of the population for which z is less than -2.50 is
0.0062, or 0.62 percent. This is a more precise value than the 1 percent esti-
mate we obtained from a superficial examination of the data.
It is clear that if we attempted to use the same technique for the samples
of size 10 shown in Table 4.2, by substituting x for /x and s for cr, we may
obtain highly unreliable values. Thus, the first sample gives a z value equal
to (70 - 107.57)/13.40 or -2.80, which corresponds to a fraction of the popu-
lation equal to 0.25 percent, and the second sample gives z = (70 - 96.37)/
8.40 = -3.14, which corresponds to a fraction of the population equal to
0.08 percent. It is obvious that this approach cannot be used for small sam-
ples. It is possible, however, to solve related problems, even for small sam-
ples. The statistical procedure used for solving these problems is called the
method of tolerance intervals
Tolerance intervals for average coverages

Generally speaking, the method of tolerance intervals is concerned with
the estimation of coverages or, conversely, with the determination of inter-
vals that will yield a certain coverage. Let us consider an interval extending
from X - ks to x + ks, where k is any given value. The coverage correspond-
ing to this interval will be a random variable, since the end points of the inter-
val are themselves random variables. However, we can find a A: value such
that, on the average, the coverage for the interval will be equal to any pre-
assigned value, such as, for example, 0.98. These k values, for normal distri-
butions, have been tabulated for various sample sizes and desired average
coverages. As an illustration, we consider the first sample of size 10 given
in Table 4.2, where
X - 107.57, s = 13.40
For a coverage of 98 percent and 9 degrees of freedom, the tabulated value is
k = 3.053
Hence the tolerance interval that, on the average, will include 98 percent of
the population is
107.57 - (3.053)(13.40) to 107.57 + (3.053)(13.40)
or
66.7 to 148.5
15
.
We can compare this interval to the one derived from the population itself
(for all practical purposes, the large sample of 2,197 individuals may be con-
sidered as the population). Using the normal table, we obtain for a 98 per-
cent coverage
100.42 - (2.326)(12.15) to 100.42 + (2.326)(12. 15)
or
72.2 to 128.7
The fact that the small sample gives an appreciably wider interval is due to
the uncertainties associated with the estimates i and s.
For a more detailed discussion of tolerance intervals, see Proschan.^ Ta-
bles of coefficients for the calculation of tolerance intervals can be found in
Snedecor and Cochran^ and Proschan.^
Non-parametric tolerance intervals—order statistics

The tabulations of the coefficients needed for the computation of toler-
ance intervals are based on the assumption that the measurements from
which the tolerance intervals are calculated follow a normal distribution; the
table is inapplicable if this condition is grossly violated. Fortunately, one can
solve a number of problems related to tolerance intervals for data from any
distribution, by using a technique known as non-parametric or distribution-
free. The method always involves an ordering of the data. First one rewrites
the observational, jcg, jcv in increasing order of magnitude. We will de-
• • • ,
note the values thus obtained by
For example. Sample I in Table 4,2 is rewritten as:

~ 91.9
~ 105.2
-^(1) -^(6)
•^(2)
~ 96.6 ^(7)
~ 112.1
•^(3)
~ 96.9 -^(8)
~ 118.8
-^(4)
~ 97.0 ^(9)
~ 119.6
~ 103.4
~ 134.2
-^(5) -^(10)
The values x^), x,2), x^!^^ are denoted as the first, second,
. . A'^th order . . . ,
statistic.The order statistics can now be used in a number of ways, depend-

ing on the problem of interest. Of particular usefulness is the following gener-
altheorem.
A genera! theorem about order statistics. On the average, the fraction —
of the population contained between any two successive order statistics
from a sample of size N is equal to
^ ^ ^
. The theorem applies to any con-
tinuous distribution (not only the Gaussian distribution) and to any sample
size N
Tolerance intervals based on order statistics. It follows immediately —
from the above theorem that, on the average, the fraction of the population
contained between the first and the last order statistics (the smallest and the
largest values in the sample) is
^ ^
. For example, on the average, the frac-
|
16
tion of the population contained between the smallest and the largest value
of a sample of size 10 is ^ = ~~ The meaning
. of the qualification "on
J
the average" should be properly understood. For any particular sample of

size 10, the actual fraction of the population contained in the interval
N —
~ -^(1) Will generally not be equal to
1
But if the average of those

-^(iV)
^ ^ ^
.
fractions is taken for many samples of size N, it will be close to + \
Tolerance intervals involving confidence coefficients

One can formulate more by in-
specific questions related to coverages
troducing, in addition to the coverage, the confidence of the statement about
the coverage. For example, one can propose to find two order statistics such
that the confidence 90 percent that the fraction of the population
is at least
contained between them (the coverage) is 95 percent. For a sample of size
200, these turn out to be the third order statistic from the bottom and the
third order statistic from the top (see Table A30 in Natrella^). For further
discussion of this topic, several references are recommended. ^-^'^
Non-normal distributions and tests of normality
Reasons for the central role of the normal distribution in statistical theo-
ry and practice have been given in the section on the normal distribution.
Many situations are encountered in data analysis for which the normal distri-
bution does not apply. Sometimes non-normality is evident from the nature
of the problem. Thus, in situations in which it is desired to determine wheth-
er a product conforms to a given standard, one often deals with a simple di-
chotomy: the fraction of the lot that meets the requirements of the standard,
and the fraction of the lot that does not meet these requirements. The statisti-
cal distribution pertinent to such a problem is the binomial (see section on
the binomial distribution).
In other situations, there is no a priori reason for non-normality, but the
data themselves give indications of a non-normal underlying distribution.
Thus, a problem of some importance is to "test for normality."
Tests of normality
Tests of normality should never be performed on small samples, be-

cause small samples are inherendy incapable of revealing the nature of the
underlying distribution. In some situations, a suflficient amount of evidence
is gradually built up to detect non-normality and to reveal the general nature
of the distribution. In other cases, it is sometimes possible to obtain a truly

large sample (such as that shown in Table 4.1) for which normality can be
tested by "fitting a normal distribution" to the data and then testing the
"goodness of the fit."^
Probability plots.— A graphical procedure for testing for normality can
be performed using the order statistics of the sample. This test is facilitated
through the use of "normal probability paper," a type of graph paper on
which the vertical scale is an ordinary arithmetic scale and the horizontal
17
scale is labeled in terms of coverages (from 0 to 100 percent), but graduated
in terms of the reduced z-values corresponding to these coverages (see sec-

tion on the normal distribution). More specifically, suppose we divide the
abscissa of a plot of the normal curve into N + 1 segments such that the area
under the curve between any two successive division points is ^^77. The
division points will bezj, 7.2, • •
, Zn, the values of which can be determined
from the normal curve. Table 4.5 lists the values 777^, 2 ' • •
'
^^ J
, in percent, in column 1, and the corresponding normal z values in
column 2, for N = 10. According to the general theorem about order statis-
tics, the order statistics of a sample of size N= \0"attempt" to accomplish
just such a division of the area into + 1 equal parts. Consequently, the N
order statistics tend to be linearly related to the z values. The order statistics
for the first sample of Table 4.2 are listed in column 3 of Table 4.5. A plot of
column 3 versus column 2 will constitute a "test for normality": if the data
are normally distributed, the plot will approximate a straight line. Further-
more, the intercept of this line (see the section on straight line fitting) will be
an estimate of the mean, and the slope of the line will be an estimate of the
standard deviation.^ For non-normal data, systematic departures from a
straight line should be noted. The use of normal probability paper obviates
the calculations involved in obtaining column 2 of Table 4.5, since the hori-
zontal axis is graduated according to z but labeled according to the values
7777, expressed as percent. Thus, in using the probability paper, the ten
order statistics are plotted versus the numbers
'«»
Ti •
'«« ^ '«»
11
or 9.09, 18.18, ,90.91 percent. It is only for illustrative purposes that we

. . .
have presented the procedure by means of a sample of size 10. One would
generally not attempt to use this method for samples of less than 30. Even
then, subjective judgment is required to determine whether the points fall
along a straight line.
In a subsequent section, we will discuss transformations of scale as a

means of achieving normality.
The binomial distribution
Referring to Table 4.1, we may be interested in the fraction of the popu-

lation for which the serum glucose is greater than, say, 1 10 mg/dl. A problem
of this type involves partitioning the range of values of a continuous variable
(serum glucose in our illustration) into two groups, namely: (a) the group of
individuals having serum glucose less than 110 mg/dl; and (b) the group of
individuals having serum glucose greater than 1 10 mg/dl. (Those having se-
rum glucose exactly equal to 110 mg/dl can be attached to one or the other
group, or their number divided equally among them.)
18
Table 4.5. Test of Normality Using Order Statistics^
Expected cumulative Reduced normal Order statistics

drcaS in pcrccni variate of sample
9.09 -1.335 91.9

18.18 -0.908 96.6
27.27 - - -0.604 96.9
36.36 -0.348 97.0
45.45 -0.114 103.4
54.54 0.114 105.2
63.64 0.348 112.1
72.73 0.604 118.8
81.82 0.908 119.6
90.91 1.335 134.2
Straight Line Fit of column 3 versus column 2:
Intercept = 107.6 = ft
Slope = 5.5 = &
^The example is merely illustrative of the method. In practice one would never test normality on a sample
of size 10.
"values of 100 .,
N +
, ,
1
, where N = 10.
Suppose now that we have a random sample of only 100 individuals

from the entire population. What fraction of the 100 individuals will be found
in either group? It is seen that the binomial distribution has shifted the em-
phasis from the continuous variable (serum glucose) to the number of individ-
uals (or the corresponding /rafr/<9«, or percentage) in each of the two
groups. There are cases in which no continuous variable was ever involved:
for example, in determining the number of times a six appears in throwing a
die. However, the theory of the binomial applies equally to both types of
situations.
The binomial parameter and its estimation
Let P (i.e., a number between zero and one) of

represent the fraction
individuals in one of the two groups (e.g., serum glucose greater than 110
mg/dl) in the population. It is customary to represent the fraction for the oth-
er group by Q. Then it is obvious that Q ^ - Z'. (If the fractions are ex- 1
pressed as percentages, we have percent Q ^ 100 - percent P.) For the

data in Table 4.1 and the dividing value 110 mg/dl, we can calculate P by
using the normal distribution:
The reduced value corresponding to 110 mg/dl is
110 - 100.42
^^^^
12.15 .
From the table of the normal distribution, we then obtain for P:
P = 0.215
Hence Q = \ - 0.215 = 0.785
19
Let p represent the fraction of individuals having the stated character-
istic (serum glucose greater than 1 10 mg/dl) in the sample of size N\ and let
^ = - /?. It is clear that for a relatively small, or even a moderately large
1
TV, p will generally differ from P In fact, p is a random variable with a well-
.
defined distribution function, namely the binomial.

The mean of the binomial (with parameter P) can be shown to be equal
to P. Thus
E{p) = P (4.24)
where the symbol E{p) represents the "expected value" of p, another name
mean. Thus the population mean of the distribution of p is
for the population
equal to the parameter f*. Ifp is taken as an estimate forP, i\\\s estimate will
therefore be unbiased.
Furthermore:
Var(p) = (4.25)
Hence
1^^ (4.26)
The normal approximation for the binomial distribution
It is a remarkable fact that for a large A^, the distribution of p can be
approximated by the normal distribution of the same mean and standard de-
viation. This enables us to easily solve practical problems that arise in con-
nection with the binomial. For example, returning to our sample of 100 indi-
viduals from the population given in Table 4.1, we have:
E(p) = 0.215
= (0.215)(0.785) =
cr„ 0 0411
100
these values, one may infer that in a sample of

From = 100 from theN
population in question, the chance of obtaining p values of less than 0.13
(two standard deviations below the mean) or of more than 0.30 (two standard
deviations above the mean) is about 5 percent. In other words, the chances
are approximately 95 percent that in a sample of 100 from the population in
question the number of individuals found to have serum glucose of more
than 110 mg/dl will be more than 13 and less than 30.
Since, in practice, the value of P is generally unknown, all inferences
must then be drawn from the sample itself. Thus, if in a sample of 100 one
finds a p value of, say, 0.18 (i.e., 18 individuals with glucose serum greater
than 110 mg/dl), one will consider this value as an estimate forP, and con-
sequently one will take the value
20
/ (0.18)(1 - 0.18) ^ 0 038
V 100
as an estimate for cr^. This would lead to the following approximate 95 per-
cent confidence interval forP:
0.18 - (1.96)(.038) < P < 0.18 + (1.96)(.038)
or
0.10 < P < 0.25
The above discussion gives a general idea about the uses and usefulness
of the binomial distribution. More detailed discussions will be found in two
general references.
Precision and accuracy
The concept of control

In some ways, a measuring process is analogous to a manufacturing
process. The analogue to the raw product entering the manufacturing proc-
ess is the system or sample to be measured. The outgoing final product of the
manufacturing process corresponds to the numerical result produced by the
measuring process. The concept of control also applies to both types of proc-
esses. In the manufacturing process, control must be exercised to reduce to
the minimum any random fluctuations in the conditions of the manufacturing
equipment. Similarly, in a measuring process, one aims at reducing to a mini-
mum any random fluctuations in the measuring apparatus and in the environ-
mental conditions. In a manufacturing process, control leads to greater uni-
formity of outgoing product. In a measuring process, control results in high-
er precision, i.e., in less random scatter in repeated measurements of the
same quantity.
Mass production of manufactured goods has led to the necessity of inter-
changeability of manufactured parts, even when they originate from diflFer-
ent plants. Similarly, the need to obtain the same numerical result for a par-
ticular measurement, regardless of where and when the measurement was
made, implies that /oca/ control of a measuring process is not enough. Users
also require interlaboratory control, aimed at assuring a high degree of "in-
terchangeability" of results, even when results are obtained at different
times or in different laboratories.
Methods of monitoring a measuring process for the purpose of achiev-
ing "local" within-laboratory) control will be discussed in the section
(i.e.,
on quality control of this chapter. In the following sections, we will be con-
cerned with a different problem: estimating the precision and accuracy of a
method of measurement.
Within- and between-laboratory variability
Consider the data in Table 4.6, taken from a study of the hexokinase
method for determining serum glucose. For simplicity of exposition, Table
21
Table 4.6. Determination of Serum Glucose
Serum sample
Laboratory A B C D
J
I 76 0 i _7 / . O ?06 ^
42.3 78.6 137.4 208.5

42.3 77.5 138.5 204.9
40.5 77.8 138.5 210.3
2 43.4 78.6 135.2 211.6

43.8 76.0 131.3 201.2
43.1 76.8 146.7 201.2
42.3 75.7 133.4 208.7
3 41.3 75.0 134.5 205.1

40.2 76.1 134.8 200.3
40.6 76.4 131.5 206.9
42.0 76.4 133.4 199.9
"All results are expressed in mg glucose/dl.
4.6 contains only a portion of the entire set of data obtained in this study.
Each of three laboratories made
four replicate determinations on each of
four serum, samples. It can be observed that, for each sample, the results
obtained by different laboratories tend to show greater differences than re-
sults obtained through replication in the same laboratory. This observation
can be made quantitative by calculating, for each sample, two standard de-
viations: the standard deviation ''within" laboratories and the standard de-
viation "between" laboratories. Within-laboratory precision is often re-
ferred to as repeatability and between-laboratory precision as reproduc-
,
ibility.' We will illustrate the method for serum sample A.

The data for serum A can first be summarized as follows:
Laboratory Average Standard Deviation
1 41.50 0.938
2 43.15 0.635
3 41.02 0.793
The three standard deviations could be averaged to obtain an "average
within-laboratory"standard deviation. However, if one can assume that
these three standard deviations are estimates of one and the same population
standard deviation, a better way is to "pool" the variances,^ and take the
square root of the pooled variance. Using this procedure, we obtain for the
best estimate of the within-laboratory standard deviation 5 „,:
= (0-938 )2 + (0 635)2 + (0.793) 2 ^ ^^^^

s,.
^
Let us now calculate the standard deviation among the three average values
41.50, 43. 15, and 41.02. Denoting this standard deviation by Sj., we obtain:
22
5j. = 1.117
If no systematic differences, this standard devia-

the laboratories displayed
tion, being calculated from averages of four individual results, should be
equal to 5u/\ 4 = 0.798/v^= 0.399. The fact that the calculated value,
1.117, is appreciably larger than 0.399 can be explained only through the
presence of an additional, between-laboratory component of variability.
This component, expressed as a standard deviation and denoted by Si^
(where L stands for "laboratories"), is calculated by subtracting the ''antici-
pated" variance, (0.399)^ from the "observed" variance, (1.117)-, and tak-
ing the square root:
St = V (1.117)2 - (0.399)2 = 1.04
The calculations for all four serum samples are summarized in Table 4.7. in
which standard deviations are rounded to two decimal places.
It may be inferred from Table 4.7 that Sy. tends to increase with the glu-
cose content of the sample. The between-laboratory component. Si. shows

no such trend. However, the data are insufiicient to establish these facts
with reasonable confidence. Since our purpose is to discuss general prin-
ciples, and the use of these data is only illustrative, we will ignore these
shortcomings in the discussion that follows.
Accuracy —comparison with reference values

The two components,
Su- and s^, define the precision of the method. To
estimate accuracy, one requires reference values for all samples. Let us
its
assume that such values have been established and are as follows:
Serum Sample Reference Value
A 40.8
B 76.0
C- 133.4
D 204.1
The values given here as "reference values" are actually only tentative. We
willassume, however, in our present discussion, that they can be considered
to be free of systematic errors. Our task is to decide whether the values ob-
tained in our study are, within random experimental error, equal to these
reference values. The grand average value for sample A, 41.89 mg/dl, which
Table 4.7. Summary of Analysis for Serum Glucose Data

Standard deviation
Serum sample Average (mg/dl) 5^. (mg/dl) 5;, (mg/dl)
A 41.89 0.80 1.04

B 76.74 1.05 0.54
C 136.08 4.08 1.07
D 205.41 3.91 1.08
23
we denote by the symbol .v, involves 12 individual determinations and four
laboratories. Its variance, therefore, can be estimated by the formula:
. / (0.W (1.04)2 ^ ^
\ 12 4
Now, X differs from the reference value by the amount:
41.89 - 40.8 = 1.09
Corresponding values for all four samples are shown in Table 4.8.
It can be seen that, on the one hand, all four grand averages are larger
than the corresponding reference values but, on the other hand, the diflFer-
ences D are of the order of only one or two standard errors s^. One would
tentatively conclude that the method shows a positive systematic error (bias)
but, as has been pointed out above, the data are insufficient to arrive at defi-
nite conclusions.
Straight line fitting
The fitting of straight lines to experimental data is a subject of great im-

portance, particularly in analytical laboratories. Many analytical and clinical
methods make extensive use of linear calibration curves for the purpose of
converting a measured quantity, such as an optical absorbance or a ratio of
peaks-heights on a mass-spectrometer scan, into a concentration value for
an unknown constituent. Calibration curves are established by subjecting
samples of known concentrations to the measuring process and fitting lines
to the resulting data. Let.v be the known concentration, and>' the measure-
ment (e.g., optical absorbance). The data will consist of a set of paired val-
ues, as shown for an illustrative example in the columns labeled x and >' in
Table 4.9.
Inspection of the table shows that there is a "blank": for zero concen-
tration, one finds a nonzero absorbance value. If one "corrected" the sub-
sequent two values for the blank, one would obtain 0.189 - 0.050 = 0.139,
and 0.326 - 0.050 = 0.276. If the "corrected" absorbance were proportion-
al to concentration (as required by Beer's law), these two corrected absor-
bances should be proportional to 50 and 100, i.e., in a ratio of to 2. Ac- 1
tually, 0.139 is slightly larger than (0.276/2). We will assume that this is due
Table 4.8. Study of Accuracy of Glucose Determination
Serum sample Reference value Grand average D Si

(R) (.f) (.? - R)
A 40.8 41.89 1.09 0.57

B 76.0 76.74 0.74 0.41
C 133.4 136.08 2.68 1.29
D 204.1 205.41 1.31 1.25
2A
r
Table 4.9. Calibration Curve for Glucose in Serum

X y y d
0 0.050 0.0516 -0.0016

50 0.189 0.1895 -0.0005
'J . JZ / J —0 no M
150 0.467 0.4652 0.0015
200 0.605 0.6030 0.0020
400 1.156 1.1545 0.0015
600 1.704 1.7059 -0.0019
214.29 0.6425 0.6425 0
y = 0.0516 + 0. 0027571.
s, = 0.0019
X - concentration of glucose, in mg/dl

y = absorbance
y = "fitted value"
d = residual
solely to experimental error in the measured absorbance values, thus assum-

ing that any errors in the concentration values are negligibly small.
A general model
If a represents the true value of the "blank" and ft the absorbance per
unit concentration, we have, according to Beer's law:
E{y) - I3x (4.27)
where Eiy) the expected value for absorbance, i.e., the absorbance value
is
freed of experimental error. Now

the actual absorbance, y, is affected by an
experimental error, which we will denote by e. Hence:
y = Eiy) + e (4.28)
Combining Equations 4.27 and 4.28 we obtain the "model" equation

y = a + + e (4.29)
This equation should hold for all x-values, i.e., Xi,X2, . . . ,XAr, with the same
values of a and (3. Hence
yi
= a + /3jc, + (4.30)
where i — 1 to N.
The errors should, on the average, be zero, but each one departs from zero
by a random amount. We will assume that these random departures from
zero do not increase with the absorbance (in some cases, this assumption is
not valid) and that their distribution is Gaussian with standard deviation o-p.
The object of the analysis is to estimate: (a) a and /3, as well as the uncer-
tainties (standard errors) of these estimates; and (b) the standard deviation
of e; i.e., o-g.
25
Formulas for linear regression
The fitting process is known in the statistical literature as the "linear
regression of>' onx." We will denote the estimates of a, /3, and o-^ by a, ^,
and Se, respectively. The formulas involve the following three quantities:
U= SU, - xf (4.31)
W = X{y^ - yf (4.32)
P = liXi - x){yi - y) (4.33)
In terms of these three quantities, we have the formulas:
a = y- ^x (4.34)
,4.35,
x' (4.36)
VU V TV
For the data of Table 4.9, the calculations result in the following values:
a = 0.0516, 55 = 0.0010, /8 = 0.0027571, 5^ = 0.0000036, 5^ = 0.0019.
Since a and ^ are now available, we can calculate, for each x, a "calcu-
lated" (or "fitted") value, y, given by the equation y = a + ^x. This is, of
course, simply the ordinate of the point on the fitted line for the chosen value
of X.
The diflferences between the observed value v and the calculated value v
is called a "residual." Table 4.9 also contains the values of _v and the resid-
uals,denoted by the symbol "t/."
important to observe that the quantity
It is (W - P^/U), occurring in
Equation 4.35, is simply equal to Xdî. Thus:
Id' (4.37)
N
This formula, though mathematically equivalent to Equation 4.35, should be
used in preference to Equation 4.35, unless all calculations are carried out
with many significant figures. The reason for this is that the quantities di are
less affected by rounding errors than the quantity (W - P'^/U).
Examination of residuals —weighting

The residuals should behave like a set of random observations with a
mean of zero and a standard deviation o-p. It follows that the algebraic signs
should exhibit a random pattern similar to the occurrence of heads and tails
in the flipping of a coin. In our example, the succession of signs raises some
suspicion of nonrandomness, but the series is too short to decide on this mat-
ter one way or the other. In any case, the errors are quite small, and the
calibration curve is quite satisfactory for the intended purpose.
26
J
The assumptions underlying this procedure of fitting a straight Hne are

not always fulfilled. The assumption of homoscedasticity (i.e., alle, have the
same standard deviation), in particular, is often violated. If the standard de-
viation of the error nonconstant and depends on x,, the fitting of the
is
straight line requires the application of "weighted regression analysis."

Briefly, assuming a diflferent value of o-p for each /, one defines a "weight"
Wi equal to the reciprocal of the square of o-p.. Thbs:
Wi = l/o-l. (4.38)
The weights w, are then used in the regression calculations, leading to for-
mulas that are somewhat dififerent from those given in this section. For fur-
ther details, two references can be consulted. ^-^
Propagation of errors
often necessary to evaluate the uncertainty of a quantity that is not

It is
directly is derived, by means of a mathematical formula, from

measured but
other quantities that are directly measured.
An example
As an example, consider the determination of glucose in serum, using
an enzymatic reaction sequence. The sequence generates a product, the opti-
cal absorbance of which is measured on a spectrophotometer. The proce-
dure consists of three steps: (a) apply the enzyme reaction sequence to a set
of glucose solutions of known concentrations, and establish in this way a cal-
ibration curve of "absorbance" versus "glucose concentration," (b) by use
of the same reaction sequences, measure the absorbance for the "un-
known," and (c) using the calibration curve, convert the absorbance for the
unknown into a glucose concentration.
It turns out that the calibration curve, for this sequence of reactions, is
linear. Thus, if>' represents absorbance, and concentration, the calibration

curve is represented by the equation:
y = a + I3x (4.39)
The calibration curve is established by measuring j for a set of known x val-
ues. We will again use the data of Table 4.9 for illustration. Fitting a straight
line to these data, we obtain:
y = 0.0516 + 0.0027571a: (4.40)
Let us now suppose that an unknown sample of serum is analyzed m times

(for example, m = 4), and that the average absorbance found is yu = 0.3672
(where y^ stands for absorbance for the unknown). Using the calibration
line, we convert the value yu into a concentration value, Xu, by solving the
calibration equation for jc:
yu- 0.3672 - 0.0516 ^ .
How reliable is this estimate?
27
Let us assume, uncertainty of the calibration Hne
at this point, that the
is negligible. Then the only quantity affected by error isy,,, and it is readily
seen from Equation 4.41 that the error ofi,, is equal to that ofyu, divided by
/3. If we assume that the standard deviation of a single measured y-value is
0.0019 absorbance units, then the standard error of the average of four
determinations, is
0.0019/ \/4 = 0.00095
Hence the standard deviation of x,, is
0.00095/^8 = 0.00095/0.0027571 = 0.34 mg/dl
A more rigorous treatment would also take account of the uncertainty of the
calibration line.
The general case

More generally, a calculated quantity z can be a function of several
measured quantities Xi, X2, .V3, each of which is affected by experimen-
. . . ,
tal error. The problem to be solved is the calculation of the standard devia-
tion of the error of z as a function of the standard deviations of the errors of

Xi, X2, X3, ....
We only deal with the case of independent errors in the quantities
will
A i, .^2, JC3, . i.e., we assume that the error of any one of the jc's is totally
. .
;
unaffected by the errors in the otherx's. For independent errors in the meas-
ured values Xi, X2, JC3, some simple rules can be applied. They are all
. . . ,
derived from the application of a general formula known as "the law of prop-
agation of errors," which is valid under very general conditions. The reader
is referred to MandeF for a general discussion of this formula.
Linear relations. — For

= fliXi + 02X2 + a^Xs + . . . (4.42)
the law states:
VarCy) = a? VarUi) + a| VarUs) + a| VarUs) + . . . (4.43)
As an example, suppose sample for chemical analysis

that the weight of a
has been obtained as the difference between two weights: the weight of an
empty crucible, Wj, and the weight of the crucible containing the sample,
W2. Thus the sample weight S is equal to
S = W2 - (4.44)
This is in accordance with Equation 4.42 by writing:
S = {\)W, + (-1)^2
Hence, according to Equation 4.43,
Var(5) = (l)2Var(Wi) + (- l)2Var(V^2)
Var(5) = Var(Wi) + Var(\y2)
28
Hence
0-5= Vo-^H', + o-V. (4.45)
Note that in spite of the negative sign occurring in Equation 4.44, the vari-
ances of Wi and W2 in Equation 4.45 are added (not subtracted from each
other).
It is also of great importance to emphasize that Equation 4.43 is vahd
only if the errors in the measurements x^, .Vg, X3, . . . , are independent of
each other. Thus, a particular element in chemical analysis was deter-
if
mined as the diflference between 100 percent and the sum of the concentra-
tions found for all other elements, the error in the concentrations for that
element would not be independent of the errors of the other elements, and
Equation 4.43 could not be used for any linear combination of the type of
Equation 4.42 involving the element in question and the other elements. But
in that case. Equations 4.42 and 4.43 could be used to evaluate the error vari-
ance for the element in question by considering it as the dependent variable
y. Thus, in the case of three other elements Xi, X2, and x^, we would have:
y = 100 - {x, + + xs)
where the errors of Xi, JC2, and x^ are independent. Hence:

VarCv) = Var(jci) + Var(jC2) + Var(jC3)
since the constant, 100, has zero- variance.

Products and ratios. —For products and ratios, the law of propagation
of errors states that the squares of the coefficients of variation are additive.
Here again, independence of the errors is a necessary requirement for the
validity of this statement. Thus, for
y = Xi X2 (4.46)
with independent errors for Xi and X2, we have:
100
-^J
= |l00
J
+ |lOO
^ J
(4.47)
We can, of course, divide both sides of Equation 4.47 by 100^, obtaining:
+ •
(4.48)
y J \ I \ X2
Equation 4.48 states that for products of independent errors, the squares of
the relative errors are additive.
The same law applies for ratios of quantities with independent errors.
Thus, when Xi and X2 have independent errors, and
(4.49)
X2
we have
+ ^
X2
(4.50)
29
As an illustration, suppose that in a gravimetric analysis, the sample weight
is 5, the weight of the precipitate is W, and the "conversion factor" is F.
Then:
y = mF^W
The constants 100 and F are known without error. Hence, for this example,
> 2 N 2 \ 2
+
w
If, for example, the coefficient of variation for 5 is 0.1 percent, and that for
W is 0.5 percent, we have:
cr
^ = V (0.005)2 + (0.001)2 = 0.0051
It is seen that in this case, the error of the sample weight S has a negligible
effecton the error of the "unknown" y.
—
Logarithmic functions. When the calculated quantity y is the natural
logarithm of the measured quantity x (we assumed that x > 0):
y = Inx (4.51)
the law of propagation of error states
^y=~ (4.52)
For logarithms to the base 10, a multiplier must be used: for
y = logio X (4.53)
the law of propagation of error states:
Sample sizes and compliance with standards
Once the repeatability and reproducibility of a method of measurement

are known, it is a relatively simple matter to estimate the size of a statistical
sample that will be required to detect a desired effect, or to determine wheth-
er a given specification has been met.
An example
As an illustration, suppose that a standard requires that the mercury
content of natural water should not exceed 2/xg/l. Suppose, furthermore,
that the standard deviation of reproducibility of the test method (see section
on precision and accuracy, and MandeF), at the level of 2^tg/l, is 0.88/xg/l. If
subsamples of the water sample are sent to a number of laboratories and
30
each laboratory performs a single determination, we may wish to determine
the number of laboratories that should perform this test to ensure that we
can detect noncompliance with the standard. Formulated in this way, the
problem has no definite solution. In the first place, it is impossible to guaran-
tee unqualifiedly the detection of any noncompliance. After all, the decision
will be made on the basis of measurements, and measurements are subject to
experimental error. Even assuming, as we do, that the method is unbiased,
we still have to contend with random errors. Second, we have, so far, failed
to give precise meanings to the terms "compliance" and "noncompliance";
while the measurement in one laboratory might give a value less than 2/u,g/l
of mercury, a second laboratory might report a value greater than 2;ag/l.
General procedure —acceptance, rejection, risks

To remove ambiguities regarding sample size, we might proceed in
all
the following manner. We consider two situations, one definitely acceptable

and the other definitely unacceptable. For example, the "acceptable" situa-
tion might correspond to a true mercury content of 1.5/u,g/l, and the "unac-
ceptable" situation to a mercury content of 2.5fxg/\ (see Fig. 4.2).
Because of experimental errors, we must consider two risks: that of re-
jecting (as noncomplying) a "good" sample (1.5 /u,g/l); and that of accepting
(as complying) a "bad" sample (2.5/u-g/l). Suppose that both risks are set at 5
percent.
Let us now denote by N
the number of laboratories required for the test.
The average of the N
measurements, which we denote by x, will follow a
normal distribution whose mean will be the true value of the mercury^ con-
tent of the sample and whose standard deviation will be cr/V/V, or
0.88/\/n. For the "acceptable" situation the mean is 1.5/u,g/l, and for the
"unacceptable" situation it is 2.5/xg/l. We now stipulate that we wiW accept
CALCULATION OF SAMPLE SIZE

FOR PREDETERMINED RISKS
ACCEPTABLE UNACCEPTABLE
1.0 1.5 2.0 2.5 3.0
CONCENTRATION OF MERCURY
(yug/l)
Fig. 4.2. Distribution of measurements of mercury in subsamples of a water sample
sent to N laboratories.
31
the sample, as complying, whenever x is less than 2.0,and reject it, as non-
complying, whenever x is greater than 2.0. As a result of setting our risks at
5 percent, this implies that the areas A and J5 are each equal to 5 percent (see
Fig. 4.2). From the table of the normal distribution, we read that for a 5 per-
cent one-tailed area, the value of the reduced variate is 1.64. Hence:
2.0^
0.88/Va^
(We could also state the requirement that (2.0 - 2.5)/(0.88/V/V) = -1.64,
which is algebraically equivalent to the one above.) Solving forN, we find:
We conclude that nine laboratories are required to satisfy our requirements.

The general formula, for equal risks of accepting a noncomplying sample and
rejecting a complying one, is:
N = i^^\ (4.56)
where a is the appropriate standard deviation, z^. is the value of the reduced
normal variate corresponding to the risk probability (5 percent in the above
example), and d is the departure (from the specified value) to which the cho-
sen risk probability applies.
Inclusion of between-laboratory variability
If the decision as to whether the sample size meets the requirements of a

standard must be made in a single laboratory, we must make our calculations
in terms of a different standard deviation. The proper standard deviation, for
an average of Ndeterminations in a single laboratory, would then be given
by:
cr (4.57)
The term al must be included, since the laboratory mean may differ
from the true value by a quantity whose standard deviation is a^. Since the
between-laboratory component al is not divided by N, a cannot be less
than (Ti no matter how many determinations are made in the single laborato-
ry. Therefore, the risks of false acceptance or false rejection of the sample
cannot be chosen at will. If in our case, for example, we had a^, = 0.75/u-g/l
and o-/^ = 0.46/Ltg/l, the total cr cannot be less than 0.46. Considering the fa-
vorable case, (x = 1.5/u,g/l, the reduced variate (see Fig. 4.2) is:
0.46
This corresponds to a risk of 13.8 percent of rejecting (as noncomplying) a

sample that is actually complying. This is also the risk probability of accept-
32
ing (as complying) a sample that is actually noncomplying. The conclusion to
be drawn from the above argument is that, in some cases, testing error will
make it impossible to keep the double risk of accepting a noncomplying prod-
uct and rejecting a complying product below a certain probability value. If,
as in our illustration, the purpose of the standard is to protect health, the
proper course of action is to set the specified value at such a level that, even
allowing for the between-laboratory component of test error, the risk of de-
claring a product as complying, when it is actually noncomplying, is low. If,
in our illustration, a level of 2.5/xg/l is such that the risk of false acceptance
of it (as complying) should be kept to 5 percent (and cr^ = 0.46)Ltg/l), then the
specification limit should be set at a value x such that:
= 1.64
0.46
which, solved forx, yields 1.75 /xg/1
Transformation of scale
Some common transformations

Non-normal populations are often skew (nonsymmetrical), in the sense
that one tail of the distribution is longer than the other, Skewness can often
be eliminated by a transformation of scale. Consider, for example, the three
numbers 1, 10, and 100. The distance between the second and the third is
appreciably larger than that between the first and the second, causing a se-
vere asymmetry. If, however, we convert these numbers to logarithms (base
10), we obtain 0, 1, and 2, which constitute a symmetrical set. Thus, if a dis-
tribution is positively skewed (long-tail on the right), a logarithmic transfor-
mation reduce the skewness. (The simple logarithmic transformation is
will
possible only when all measured values are positive). A transformation of
the logarithmic type is not confined to the function y = log x. More gener-
ally, one can consider a transformation of the type:
y = K\og{A + Bx) (4.58)
or even
y = C + K\og{A + Bx) (4.59)
where C, K, A, and B are properly chosen constants. It is necessary to

choose A and B such that A + Bx positive for all x values. Other common
'is
types of transformations are:
y = V7 (4.60)
and
y = arcsin Vx (4.61)
Robustness
The reason given above for making a transformation of scale is the pres-
ence of skewness. Another reason is that certain statistical procedures are
33
valid only when the data are at least approximately normal. The procedures
may become grossly invalid when the data have a severely non-normal distri-
bution.
A statistical procedure that is relatively insensitive to non-normality in
the original data (or, more generally, to any set of specific assumptions) is
called "robust." Confidence intervals for the mean, for example, are quite
robust because, as a result of the central limit theorem, the distribution of
the sample mean x will generally be close to normality. On the other hand,
tolerance intervals are likely to be seriously affected by non-normality. We
have seen that nonparametric techniques are available to circumvent this dif-
ficulty.
Suppose that, for a particular type of measurement, tests of normality
on many always show evidence of non-normality. Since many
sets of data
techniques are based on the assumption of normality, it would be
statistical
advantageous to transform these data into new sets that are more nearly nor-
mal.
Fortunately, the transformations that reduce skewness also tend, in gen-
eral, to achieve closer compliance with the requirement of normality. There-
fore, transformations of the logarithmic type, as well as the square root and
arcsine transformations, are especially useful whenever a nonrobust analy-
sis is to be performed on a set of data that is known to be seriously non-
normal. The reader is referred to MandeP for further details regarding trans-
formations of scale.
Transformations and error structure

important to realize that any nonlinear transformation changes the
It is
error structure of the data, and transformations are, in fact, often used for
the purpose of making the experimental error more uniform over the entire
range of the measurements. Transformations used for this purpose are called
"variance-stabilizing" transformations. To understand the principle in-
volved, consider the data in Table 4.10, consisting of five replicate absor-
bance values at two different concentrations, obtained in the calibration of
Table 4. 10. Error Structure in a Logarithmic Transformation of Scale
Original data Transformed data

(Absorbance) (log,o Absorbance)
Set A^ SetB" Set A SetB
0.2071 1.6162 -0.6838 0.2085

0.2079 1.5973 -0.6821 0.2034
0.1978 1.6091 -0.7038 0.2066
0.1771 1.7818 -0.7518 0.2509
0.2036 1.6131 -0.6912 0.2077
Average 0.1987 1.6435 -0.7025 0.2154

Standard deviation 0.0127 0.0776 0.0288 0.0199
Âbsorbance values for a solution of concentration of 50 mg/dl of glucose.

•"Absorbance values for a solution of concentration of 600 mg/dl of glucose.
34
spectrophotometers for the determination of serum glucose. At the higher
concentration level, the absorbance values are of course higher, but so is the
standard deviation of the replicate absorbance values. The ratio of the aver-
age absorbance values is 1.6435/0. 1987 = 8.27. The ratio of the standard de-
viations is 0.0776/0.0127 - 6. 1 1. Thus the standard deviation between repli-
cates tends to increase roughly in proportion to the level of the measure-
ment. We have here an example of "heterogeneity of variance." Let us now
examine the two sets of values listed in Table 4. 10 under the heading "trans-
formed data." These are simply the logarithms to the base 10 of the original
absorbance values. This time, the standard deviations for the two levels are in
the proportion 0.0199/0.0288 = 0.69. Thus, the logarithmic transformation
essentially has eliminated the heterogeneity of variance. It has, in fact, "sta-
bilized" the variance. The usefulness of variance stabilizing transformations
is twofold: (a) a single number will express the standard deviation of error,
regardless of the "level" of the measurement; and (b) statistical manipula-

tions whose validity is contingent upon a uniform error variance (homo-
scedasticity) and which are therefore inapplicable to the original data, can be
applied validly to the transformed data.
Presentation of data and significant figures
The law of propagation of errors (see that section) enables one to calcu-
late thenumber of significant figures in a calculated value. A useful rule of
thumb is to report any standard deviation or standard error with two signifi-
cant figures, and to report a calculated value with as many significant figures
as are required to reach the decimal position of the second significant digit of
its standard error.
An example
Consider the volumetric determination of manganese in manganous cy-
clohexanebutyrate by means of a standard solution of sodium arsenite. The
formula leading to the desired value of percent Mn is
w(mg)
where w the weight of the sample, v the volume of reagent, and / the titer
is
of the reagent, and the factor 200/15 is derived from taking an aliquot of 15
ml from a total volume of 200 ml.
For a particular titration, the values and their standard errors are found
to be:
V - 23.67 0.0040
/ = 0.41122 0.000015
200 0.0040
15 cr 0.0040
w = 939.77 w 0.0060
35
The values are reported as they are read on the balance or on the burettes
and pipettes; their standard errors are estimated on the basis of previous ex-
perience. The calculation gives:
Percent Mn = 13.809872
The law of propagation of errors gives:
8099 ^^"^^ f + M*^^^ 0^0040^\2 0i)040^\2 0.0060

f +
/ /
13 // / /
^
23.67 0.41122 [ 200 [' 15 j
^ 939.77
j I ] j I j
= 0.0044
On the basis of this standard deviation, we would report this result as:
Percent Mn = 13.8099; o-^n = 0.0044
It should be well understood that this calculation is based merely on weigh-

ing errors, volume reading errors, and the error of the titer of the reagent. In
repeating the determination in different laboratories or even in the same labo-
ratory, uncertainties may arise from sources other than just these errors.
They would be reflected in the standard deviation calculated from such re-
peated measurements. In general, this standard deviation will be larger, and
often considerably larger, than that calculated from the propagation of
weighing and volume reading errors. If such a standard deviation from re-
peated measurements has been calculated, it may serve as a basis to redeter-
mine the precision with which the reported value should be recorded.
In the example of the manganese determination above, the value given
is just the first of a series of repeated determinations. The complete set of
data is given in Table 4.1 1. The average of 20 determinations is 13.8380. The
Table 4.11. Manganese Content of Manganous Cyclohexanebutyrate

Determination Result Determination Result
number (Percent Mn) number (Percent Mn)
1 13.81 11 13.92
2 13.76 12 13.83
3 13.80 13 13.73
4 13.79 14 13.99
5 13.94 15 13.89
6 13.76 16 13.76
7 13.88 17 13.88
8 13.81 18 13.82
9 13.84 19 13.87
10 13.79 20 13.89
Average = v = 13.838
= 0.068
Si = 0.068/ V20 - 0.015
36
standard deviation of the replicate values is 0.068; therefore, the standard
error of the mean is 0.068/V20 = 0.015. The final value reported for this
analysis would therefore be:
Percent Mn ^ x = 13.838; = 0.015
This example provides a good illustration of the danger of basing an esti-

mate of the precision of a value solely on the reading errors of the quantities
from which it is calculated. These errors generally represent only a small por-
tion of the total error. In this example, the average of 20 values has a true
standard error that is still more than three times larger than the reading error
of a single determination
General recommendations
good practice to retain, for individual measurements, more signifi-
It is
cant figures than would result from calculations based on error propagation,
and to use this law only for reporting the final value. This practice enables
any interested person to perform whatever statistical calculations he desires
on the individually reported measurements. Indeed, the results of statistical
manipulations of data, when properly interpreted, are never affected by un-
necessary significant figures in the data, but they may be seriously impaired
by too much rounding.
The practice of reporting a measured value with a ± symbol followed
by itsstandard error should be avoided at all costs, unless the meaning of
the ± symbol is specifically and precisely stated. Some use the ± symbol to
indicate a standard error of the value preceding the symbol, others to in-
dicate a 95 percent confidence interval for the mean, others for the standard
deviation of a single measurement, and still others use it for an uncertainty
interval including an estimate of bias added to the 95 percent confidence in-
terval. These alternatives are by no means exhaustive, and so far no stand-
ard practice has been adopted. It is of the utmost importance, therefore, to
define the symbol whenever and wherever it is used.
It should also be borne in mind that the same measurement can have,
and generally does have, more than one precision index, depending on the
framework (statistical population) to which it is referred. For certain pur-
poses, this population is the totality of (hypothetical) measurements that
would be generated by repeating the measuring process over and over again
on the same sample in the same laboratory. For other purposes, it would be
the totality of results obtained by having the sample analyzed in a large num-
ber of laboratories. The reader is referred to the discussion in the section on
precision and accuracy.
Tests of significance
General considerations
Aconsiderable part of the published statistical literature deals with sig-
nificance testing. Actually, the usefulness of the body of techniques classi-
fied under this title is far smaller than would be inferred from its prominence
37
in the literature. Moreover, there are numerous instances, both pubHshed
and unpubhshed, of serious misinterpretations of these techniques. In many
applications of significance testing, a "null-hypothesis" is formulated that
consists of a statement that the observed experimental result —for example,
the improvement resulting from the use of a drug compared toa placebo— is
not "real," but simply the effect of chance. This null-hypothesis is then sub-
jected to a statistical test and, if rejected, leads to the conclusion that the
beneficial effect of the drug is "real," i.e., not due to chance. A closer exami-
nation of the nature of the null-hypothesis, however, raises some serious
questions about the validity of the logical argument. In the drug-placebo
comparison, the null-hypothesis is a statement of equality of the means of
two populations one referring to results obtained with the drug and the oth-
,
er with the placebo. All one infers from the significance test is a probability
statement regarding the observed (sample) difference, on the hypothesis that
the true difference between the population means is zero. The real question,
of course, is related not to the means of hypothetical populations but rather
to the benefit that any particular subject, selected at random from the rele-
vant population of patients, may be expected to derive from the drug.
Viewed from this angle, the usefulness of the significance test is heavily de-
pendent on the size of the sample, i.e. on the number of subjects included in
,
the experiment. This size will determine how large the difference between
the two populations must be, as compared to the spread of both popu-
lations, before the statistical procedure will pick it up with a reasonable prob-
ability. Such calculations are known as the determination of the "power" of
the statistical test of significance. Without indication of power, a test of sig-
nificance may be very misleading.
Alternative hypotheses and sample size —the concept of power

An example of the use of "power" in statistical thinking is provided by
our discussion in the section on sample sizes. Upon rereading this section,
the reader will note that two situations were considered and that a probabili-
ty value was associated with each of the two situations, namely, the probabil-
ity of accepting or rejecting the lot. In order to satisfy these probability re-
quirements, it was necessary to stipulate a value of the sample size.
Smaller values of N
would not have achieved the objectives expressed by
the stipulated probabilities.
In testing a drug versus a placebo, one can similarly define two situa-
which the drug is hardly superior to the placebo; and
tions: (a) a situation in
(b) a situation inwhich the drug is definitely superior to the placebo. More
specifically, consider a very large, hypothetical experiment in which sub-
jects are paired at random, one subject of each pair receiving the placebo
and the other the drug. Situation (a) might then be defined as that in which
only 55 percent of all pairs shows better results with the drug than with the
placebo; situation (b) might be defined as that in which 90 percent of the
pairs shows greater effectiveness of the drug.
If we now perform an actual experiment, similar in nature but of moder-
ate size, we must allow for random fluctuations in the percentage of pairs
38
that show
better results with the drug as compared to the placebo. There-
fore, our acceptance of the greater effectiveness of the drug on the basis of
the data will involve risks of error. If the true situation is (a), we may wish to
have only a small probability of declaring the drug superior, say, a probabili-
ty of 10 percent. On the other hand, if the true situation is (b), we would
want this probability to be perhaps as high as 90 percent. These two probabil-
ities then allow us to calculate the required sample size for our experiment.
Using this sample size, we will have assurance that the power of our experi-
ment is sufficient to realize the stipulated probability requirements.
An example
An problems is shown in Table 4. 12. The data
illustration of this class of
result from the comparison of two drugs, S (standard) and E (experimental),
for the treatment of a severe pulmonary disease. The data represent the re-
duction in blood pressure in the heart after administration of the drug. The
test most commonly used for such a comparison is Student's t test.^"^ In the
present case, the value found for t is 3.78, for DF = 142 (DF = number of
degrees of freedom). The probability of obtaining a value of 3.78 or larger by
pure chance (i.e. for equal efficacy of the two drugs) is less than 0.0002. The
,
smallness of this probability is of course a strong indication that the hypothe-

sis of equal efficacy of the two drugs is unacceptable. It is then generally
concluded that the experiment has demonstrated the superior efficacy of E
as compared to S. For example, the conclusion might take the form that "the
odds favoring the effectiveness of E over S are better than to 1 " where M M
is a large number (greater than 100 in the present case). However, both the
test and the conclusion are of little value for the solution of the real problem
underlying this situation, as the following treatment shows. If we assume, as
a first approximation, that the standard deviation 3.85 is the "population pa-
rameter" cr, and that the means, 0.10 for S and 2.53 for E, are also popu-
lation parameters, then the probability of a single patient being better off
Table 4. 12. Treatment of Pulmonary Embolism — Comparison of Two Drugs

Decrease in Right Ventricular Diastolic Blood Pressure (mm Hg)
Standard treatment (S) Experimental treatment (E)
Number of patients 68 76
Average 0.10 2.53
Standard deviation 3.15 4.28
Standard error of average 0.38 0.50
True mean
t test for Ho: /x, = /X2, (Ho = null hypothesis)

Spooled = 3.85 DF = 67 + 75 = 142, (DF = degrees of freedom)
.
^ 1, 5 3 - 0.10 = 3.78 (P < 0.0002)
39
)
with E than with S is a function of the quantity Q defined by

Q= {1^2 ~ At,i)/cr. In the present case:
2.53 - 0.10
Q = 3.85
= 0.63
This can be readily understood by looking at Figure 4.3, in which the means
of two populations, S and E, are less than one standard deviation apart, so
that the curves show a great deal of overlap. There is no question that the
two populations are distinct, and this is really all the t test shows. But due to
the overlap, the probability is far from overwhelming that treatment E will
be superior to treatment S for a randomly selected pair of individuals. It can
be shown that this probability is that of a random normal deviate exceeding
the value! — or, in our case (- - -0.45. This probability is
0.67, or about 2/3. Thus, in a large population of patients, two-thirds would

derive more benefit from S than from E. Viewed from this perspective, the
significance test, with its low "P value" (of 0.0002 in our case) is seen to be
thoroughly misleading.
The proper treatment of a problem of this type is to raise the question of
interest within a logical framework, derived from the nature of the problem,
rather than perform standard tests of significance, which often merely pro-
vide correct answers to trivial questions.
Evaluation of diagnostic tests
The concepts ofprecision and accuracy are appropriate in the evalua-

measure, such as the glucose level of
tion of tests that result in a quantitative
serum or the fluoride content of water. For medical purposes, different types
of tests denoted as "diagnostic tests" are also of great importance. They dif-
COMPARISON OF TWO DRUGS
0 5 10
Decrease in Ventricular Diastolic Blood Pressure

(mnn Hg
Fig. 4.3. Comparison of two drugs for the treatment of pulmonary disease, as meas-
ured by the reduction in right ventricular diastolic blood pressure (mm Hg).
40
ferfrom quantitative types of tests in that their outcome is characterized by
a simple dichotomy into positive or negative cases.
As an example, consider Table 4.13, representing data on the alpha-feto-
protein (AFP) test for the diagnosis of hepatocellular carcinoma.^ What do
these data tell us about the value of the AFP test for the diagnosis of this
disease?
Sensitivity and specificity

The statistical aspects of this type of problem are best understood by
introducing a number of concepts that have been specifically developed for
these problems.^
Sensitivity is the proportion of positive results among the subjects af-
fected by the disease. Table 4.13 provides as an estimate of sensitivity:
90
Sensitivity ='j^= 0.8411 = 84.11%
Specificity is the proportion of negative results among the subjects who

are free of the disease. From Table 4.13:
2079
Specificity =^~~ = 0.9816 = 98.16%
2 1 lo
The concepts of sensitivity and specificity are useful descriptions of the

nature of a diagnostic test, but they are not, in themselves, sufficient for pro-
viding the physician with the information required for a rational medical deci-
sion.
For example, suppose that a particular subject has a positive AFP test.
What is the probability that this subject has hepatocarcinoma? From Table
4.13 we infer that among all subjects for whom
the test is positive a propor-
tion of 90/129, or 69.77 percent, are aflfected by the disease. This proportion
is called the predictive value of a positive test, or PV+.
Predictive values —the concept of prevalence

Predictive value of a positive test. —
(PV + ) is defined as the proportion
of subjects affected by the disease among those showing a positive test. The
(PV + ) value cannot be derived merely from the sensitivity and the specifici-
ty of the test. To demonstrate this, consider Table 4.14, which is fictitious
and was derived from Table 4.13 by multiplying the values in the "Present"
Table 4. 13. Results of Alpha-Fetoprotein Tests for Diagnosis of Hepatocellular

Carcinoma
Hepatocarcinoma
Test result Present -

Absent Total
+ 90 39 129
17 2,079 2,096
Total 107 2,118 2,225
Al
Table 4.14. Values for Alpha-Fetoprotein Tests Derived from Table 4.13
Hepatocarcinoma
Test Result Present Absent Total
+ 900 39 939
170 2,079 2,249
Total 1,070 2,118 3,118
column by 10, and by leaving the values in the "Absent" column un-
changed. Table 4.14 leads to the same sensitivity and specificity values as
Table 4.13. However, the (PV + ) value is now 900/939 = 95.85 percent.
It is seen that the (PV + ) value depends not only on the sensitivity and
the specificity but also on the prevalence of the disease in the total popu-
lation. In Table 4.13, this prevalence is 107/2225 = 4.809 percent, whereas
in Table 4.14 it is 1070/3118 = 34.32 percent.
A logical counterpart of the (PV + ) value is the predictive value of a neg-
ative test, or PV-.
Predictive value of a negative test. (PV — ) is defined as the proportion—
of subjects free of the disease among those showing a negative test. For the
data of Table 4.13, the (PV-) value is 2079/2096 = 99.19 percent, whereas
for Table 4.14, (PV-) = 2079/2249 = 92.44 percent. As is the case for
(PV + ), the (PV-) value depends on the prevalence of the disease.
The following formulas relate (PV + ) and (PV-) to sensitivity, specifici-
ty, and prevalence of the disease. We denote sensitivity by the symbol SE,
specificity by SP, and prevalence by P; then:
(PV + ) = ^
(4.62)
^
{I - SP) {I - P)
1 +
SE P
•
(PV-) = ^
„ (4.63)
(1- SE) P
1 +
SP{\ - P)
As an illustration, the data in Table 4.13 yield:
= '''''
(1 - 0.9816) (1 - 0.04809) = = ''
1 +
(0.8411) (0.04809)
(PV-) = ^ = 0.9919 = 99.19%

^ '
(1 - 0.8411) (0.04809)
1 +
(0.9816) (1 - 0.04809)
Apart from rounding errors, these values agree with those found by direct
inspection of the table.
Interpretation of multiple tests

The practical usefulness of (PV + ) and (PV-) is now readily apparent.
Suppose that a patient's result by the AFP test is positive and the prevalence
42
of the disease is Then the probabiHty that the patient suffers
4.809 percent.
from hepatocarcinoma about 70 percent. On the basis of this result, the
is
patient now belongs to a. subgroup of the total population in which the preva-
lence of the disease is 70 percent rather than the 4.8 percent applying to the
total population. Let us assume that a second test is available for the diag-
nosis of hepatocarcinoma, and that this second test is independent of the
AFP test. The concept of independence of two diagnostic tests is crucial for
the correct statistical treatment of this type of problem, but it seems to have
received little attention in the literature. Essentially, it means that in the
class of patients affected by the disease, the proportion of patients showing a
positive result for test B is the same, whether test A was positive or nega-
tive. A similar situation must hold for the class of patients free of the dis-
ease.
In making inferences from this second test for the patient in question,
we can with a value of prevalence of the disease (P) of 70 percent, rath-
start
er than 4.8 percent, since we know from the result of the AFP test that the
patient belongs to the subgroup with this higher prevalence rate. As an illus-
tration, let us assume that the second test has a sensitivity of 65 percent and
a specificity of 90 percent and that the second test also is positive for this
patient. Then the new (PV+) value is equal to
^ (0.65) (0.70)
If, on the other hand, the second turned out to be negative, then the
test
probability that the patient is free of disease would be:
^ (0.90) (1 - 0.70)
In that case, the two tests essentially would have contradicted each other,
and no firm diagnosis could be made without further investigations.
A general formula for multiple Independent tests
can easily be shown that the order in which the independent tests are
It
carried out has no effect on the final (PV + ) or (PV-) value. In fact, the fol-
lowing general formula can be derived that covers any number of indepen-
dent tests and their possible outcomes.
Denote by {SE)i and {SP)i the sensitivity and the specificity of the
ith = test, where / = 1, 2, 3, N. Furthermore, define the symbols
. . . ,
and Bi as follows:
_
= \ {SE)i when the result of test / is +
Ai -
[ 1 when the result of test /
{SE)i is
D ^ [ 1
- {SP)i when the result of test / is +
'
{SP)i when the result of test is - /
43
If P
the prevalence rate of the disease before administration of any of
is
the tests, and P' is the probability that the subject has the disease after ad-
ministration of the tests, then:
{A, A^. . . A,)P
It is important to keep in mind that Equation 4.64 is valid only if all tests
are mutually independent in the sense defined above.
Quality Control
The remainder of this chapter deals with the fundamental principles of a

quality control and quality assurance program for monitoring and assessing
the precision and accuracy of the data being processed within a laboratory.
The definitions of Quality, Quality Assurance, and Quality Control by
the American Society for Quality Control (ASQC)'' apply to either a product
or a service, and they are quoted here in their entirety.
1) Quality. — ''The of features and characteristics of a product or
totality
service that bear on its ability to satisfy a given need."
2) Quality assurance. —
"A system of activities whose purpose is to provide
assurance that the overall quality-control job is in fact being done effec-
tively. The system involves a continuing evaluation of the adequacy and
effectiveness of the overall quality-control program with a view of having
corrective measures initiated where necessary. For a specific product or
service, this involves verifications, audits, and the evaluation of the quali-
ty factors that affect the specification, production, inspection, and use of
the product or service."
3) Quality control. —
"The overall system of activities whose purpose is to
provide a quality of product or service that meets the needs of users; al-
so, the use of such a system.
"The aim of quality control is to provide quality that is satisfactory,
adequate, dependable, and economic. The overall system involves inte-
grating the quality aspects of several related steps, including the proper
specification of what is wanted; production to meet the full intent of the
specification; inspection to determine whether the resulting product or
service is in accordance with the specification; and review of usage to
provide for revision of specification.
"The term quality control is often applied to specific phases in the
overall system of activities, as, for example, pwcv^^ quality control.''
The Control Chart
According to the ASQC,^ the control chart is "a graphical chart with
control limits and plotted values of some statistical measure for a series of
samples or subgroups. A central line is commonly shown."
44
The results of a laboratory test are plotted on the vertical axis, in units
of the test results, versus time, in hours, days, etc., plotted on the horizontal
axis. Since each laboratory test should be checked at least once a day, the
horizontal scale should be wide enough to cover a minimum of one month of
data. The control chart should be considered as a tool to provide a "real-
time" analysis and feedback for appropriate action. Thus, it should cover a
sufficient period of time to provide sufficient data to study trends, "runs"
above and below the central line, and any other manifestation of lack of ran-
domness (see section on detection of lack of randomness).
Statistical basis for the control chart
General considerations
W. A. Shewhart, in his pioneering work developed the prin-
in 1939,^^
ciples of the control chart. They can be summarized, as was done by E. I.
Grant, as follows: "The measured quantity of a manufactured product is
always subject to a certain amount of variation as a result of chance. Some
stable 'System of Chance Causes' is inherent in any particular scheme of pro-
duction and inspection. Variation within this stable pattern is inevitable. The
reasons for variation outside this stable pattern may be discovered and cor-
rected." If the words "manufactured product" are changed to "laboratory
test," the above statement is directly applicable to the content of this
section.
We can think of the "measured quantity" as the concentration of a par-
ticular constituent in a patient'ssample (for example, the glucose content of
a patient's serum). Under the "system of chance causes," this concentra-
tion, when measured many times under the same conditions, will fluctuate in
such a way as to generate a statistical distribution that can be represented by
a mathematical expression. This expression could be the normal distribu-
tion, for those continuous variables that are symmetrically distributed about
the mean value, or it could be some other suitable mathematical function ap-
plicable to asymmetrically or discretely distributed variables (see section on
non-normal distributions). Then, applying the known principles of probabili-
ty, one can find lower and upper limits, known as control limits, that will
define the limits of variation within "this stable pattern" for a given accept-
able tolerance probability. Values outside these control limits will be consid-
ered "unusual," and an investigation may be initiated to ascertain the rea-
sons for this occurrence.
Control limits
According to the ASQC,^ the control limits are the "limits on a control
chart that are used as criteria for action or forjudging whether a set of data
does or does not indicate lack of control."
Probability limits. — If the distribution of the measured quantity is
known, then lower and upper limits can be found so that, on the average, a
predetermined percentage of the values (e.g., 95 percent, 99 percent) will fall
within these limits if the process is under control. The limits will depend on
the nature of the probability distribution. They will diflfer, depending on
45
whether the distribution of the measured quantity is symmetric, asymmetric
to the left or to the right, unimodal or bimodal, discrete or continuous, etc.
The obvious difficulty of finding the correct distribution function for
each measured quantity, and of determining the control limits for this distri-
bution, necessitates the use of procedures that are not overly sensitive to the
nature of the distribution function.
—
Three-sigma limits. The three-sigma limits, most commonly used in in-
dustrial practice, are based on the following expression:
Control limits = Average of the measured quantity ± three standard
deviations of the measured quantity
The ''measured quantity" could be the mean of two or three replicate
determinations for a particular chemical test, the range of a set of replicate
tests, a proportion defective, a radioactive count, etc.
The range of three standard deviations around the mean, that is, a width
of six standard deviations, usually covers a large percentage of the distribu-
tion. For normally distributed variables, this range covers 99.7 percent of
the distribution (see section on the normal distribution). For non-normally
distributed variables, an indication of the percentage coverage can be ob-
tained by the use of two well-known inequalities:
1) Tchehychejf s Inequality. For any distribution, (discrete or continuous,
symmetric or asymmetric, unimodal or bimodal, etc.) with a finite stand-
ard deviation, the interval mean ± Ka covers a proportion of the popu-
lation of at least
1 8
' " ^ . Thus for = 3, the coverage will be at least
- =
1
^ ^, or roughly 90 percent of the distribution.
2) Camp-Meidel Inequality. If the distribution is unimodal, the interval

mean ± Ka will cover a proportion of at least i
'
— of the popula-
tion. Thus, for =
the coverage will be at least
3, i - or roughly
95 percent of the population.
From the above discussion,
follows that the three-sigma limits cover a
it
proportion of the population that is at least equal to 90 percent for non-nor-

mal distributions and is equal to exactly 99.7 percent when the distribution is
normal.
Most control charts are based on the mean of several determinations of
the same measured equality. By the Central Limit Theorem, (see section on
the normal distribution), the larger the sample size, the closer to normality
will be the mean of this measured quantity. However, since most clinical
tests are based on single or, at best, duplicate determinations, caution
should be used in interpreting the amount of coverage given by the control
limits for those distributions that are suspected to be skewed, bimodal, etc.
—
Warning limits. The warning limits commonly used in practice are de-
fined as:
Warning limits = Average of the measured quantity ± two standard
deviations of the measured quantity.
For interpretation of points falling outside the warning and control lim-
its, see the section on the control chart as a management tool.
46
Variability between and within subgroups
The hypothesis = 0
In control charts for variables, the variability is partitioned into two
components: within and between subgroups. To this eflfect, the sequence of
measurements is divided into subgroups of n consecutive values each. The
variability within subgroups is estimated by first computing the average of
the ranges of all subgroups and dividing this average by a factor that depends
on n, which can be found in standard statistical tables. As an example, con-
sider the sequence: 10.2, 10.4, 10.1, 10.7, 10.3, 10.3, 10.5, 10.4, 10.0, 9.8,
10.4, 10.9. When divided into subgroups of four, we obtain the arrangement:
Subgroup Average Range

10.2,10.4,10.1,10.7 10.350 0.6
10.3, 10.3, 10.5, 10.4 10.375 0.2
10.0,9.8,10.4,10.9 10.275 1.1
Average 10.333 0.63

In this case n =
4, and the average range is ^ =
0.63.
Generally, n is a small number, often between 2 and 5. Its choice is
sometimes arbitrary, dictated only by statistical convenience. More often,
and preferably, the choice of n is dictated by the way in which the data were
obtained. In the example above, the data may actually consist of three sam-
ples, each measured four times. In this case, "within groups" means "with-
in samples," and "between groups" means "between samples."
Another possibility is that there were actually 12 samples, but that the
measuring technique requires that they be tested in groups of four. If that is
the situation, the relation of between-group to within-group variability de-
pends not only on the sample-to-sample variability but also on the stability
of the measuring instrument or technique from one group of four to another
group of four. The location of the control limit and the interpretation of the
control chart will depend on the nature and the choice of the subgroup.
If the standard deviation within subgroups is denoted by aw, and the
standard deviation between subgroups by o-g, a control chart is sometimes,
but by no means always, a test as to whether ctb exists (is different from ze-
ro). If o-fi = 0, then the variation between Xht averages of subgroups can be
predicted from ctw (or, approximately, from R). The hypothesis 0-5 = 0 can
be tested by observing whether the subgroup averages stay within the con-
trol limits calculated on the basis of within-subgroup variability. Failure of
this event to occur indicates the presence of causes of variability between
subgroups. The nature of these causes depends on the criteria used in the
selection of the subgroups.
The case o-g 9^ 0. Baseline data
In applications, the hypothesis 0-^ = 0 is not justified by the physi-

many
may, for example, already be known that
cal reality underlying the data. It
the subgroups vary from each other by more than can be accounted for by
within-subgroup variability. Thus, each subgroup may represent a different
47
day of and there may be more variability between days than within
testing,
days. The initial set of data (baseline data) is then used primarily to estimate
both the within- and the between-components of variability, and control lim-
its are calculated on the basis of both these components (see section on com-
putation of control limits). Data that are obtained subsequent to the baseline
period are then evaluated in terms of these control lines. From time to time,
the control lines are recalculated using all the data obtained up to that time,
eliminating, however, those data for which abnormal causes of variability
were found.
Types of control charts
Depending on the characteristics of the measured quantity, control

charts can be classified into three main groups:
1) Control charts for variables (the X, R Chart). These are used for variables
such as clinical chemical determinations, some hematological parame-
ters, etc.
2) Control charts for attributes (the P-Chart). These are used for proportion
defective, proportion of occurrence of given disease, etc.
3) Control charts for number of defects per unit (the C-Chart). These may be
used for counts, such as the number of cells observed in a given area,
radioactive counts, etc.
Preparing a control chart
Objective and choice of variable
The general objectives of a control chart are: (a) to obtain initial esti-
mates for the key parameters, particularly means and standard deviations.
These are used to compute the central lines and the control lines for the con-
trol charts; (b) to ascertain when these parameters have undergone a radical
change, either for worse or for better. In the former case, modifications in
the control process are indicated; and (c) to determine when to look for as-
signable causes of unusual variations so as to take the necessary steps to
correct them or, alternatively, to establish when the process should be left
alone.
A daily review of the control chart should indicatewhether the result-
ing product or service accordance with specifications. For example, in
is in
clinical chemistry, if a control chart based on standard samples shows statis-

tical control for the measurement of a given constituent, then one can pro-
ceed with confidence with the determination of this constituent in patient
samples. If the chart shows lack of control, an investigation should be start-
ed immediately to ascertain the reasons for this irregularity.
No general recommendations can be made here about the types of vari-
ables to use for quality control purposes, since they will obviously vary ac-
cording to the various disciplines of the laboratory. Considerations of this
type will be found in the respective specialty chapters of this book. The
same statements apply to the types of stable pools or reagents that should be
used, and to the methods of handling these materials in normal laboratory
practice.
Selecting a rational subgroup
The generally recommended approach for the selection of a subgroup of

data for control purposes (using a single pool of homogeneous material) is
that conditions within subgroups should be as uniform as possible (same in-
strument, same reagents, etc.), so if some assignable causes of error are pres-
ent, they will show up between subgroups (see Duncan, p. 347, and
Grant, Ch. 6, for further discussions).
When tests on patient samples are performed at regular intervals using
standard laboratory equipment, the subgroup becomes automatically de-
fined, since control samples are, or should be, included in each run. Other-
wise, tests on control samples should be run at regular intervals during the
day in order to detect possible changes in environmental conditions, re-
agents, calibrations, technicians, etc.
Size and frequency of control sample analyses

A minimumof two replicates should be obtained in each run of the con-
trol sample. To account for the possible effects of carryover from other sam-
ples, and to have a better indication of the capability of an instrument to re-
produce itself under normal conditions within a run, the replicate samples
should not be tested back-to-back, but should be separated by patient sam-
ples.
As
indicated before, the frequency of the runs on control materials is
generally tied to the frequency of the tests on patient samples. One general
rule is to test the control samples as frequently as possible at the beginning
of a control procedure, and to reduce this frequency to a minimum of two or
three per day when the results of the control chart show a satisfactory state
of control.
Maintaining uniform conditions in laboratory practice
A properly prepared control chart will tend to reflect any change in the
precision and accuracy of the results obtained. To avoid wasting time in
hunting for unnecessary sources of trouble, care should be taken to maintain
laboratory conditions and practices as uniform as possible. These include
sampling procedures, dilution techniques, aliquoting methods, storage meth-
ods, instrumental techniques, calculating procedures, etc.
Initiating a control chart

When meaningful historical data are not available (as is often the case
when a quality control procedure is to be initiated), a plan should be set up to
collect a minimum amount of data for each variable to be controlled during
an initial baseline period.
For a control chart for variables, with a minimum of two replicates for
each run, data should be collected for a baseline period of at least one month
in order to allow sufficient time for the estimation of day-to-day variability.
49
Means and ranges should be computed for each run and plotted on separate
charts. Records should be accurately kept, using standard quality control
(QC) forms that are readily available. Any value that appears to be the result
of a blunder should be eliminated, and the source of the blunder carefully
noted. It is recommended that the number of runs or subgroups be at least 25
for the baseline period.
The same considerations apply to control charts of proportions and
counts, except that the number of observations for each subgroup is gener-
ally larger than the corresponding number used in a control chart of vari-
ables. Statistical procedures for determining the sample size, n, for the P-
chart or the C-chart can be found in the literature (see Duncan, pp. 345 and
361). In general, n should be large enough to provide a good chance of find-
ing one or more defectives in the sample.
Determining trial control limits
Based on the initial set of data collected during the baseline period, trial
control limits can be determined using the procedure outlined in the section
on random samples. After plotting these limits in the initial control chart (see
section on the case otb 0), points that are outside or very near the limits
should be carefully examined, and if some valid reasons are found for their
erratic behavior, they should be eliminated and new control limits should be
computed. In general, it is better to start with control limits that are relative-
ly narrow in order to better detect future trends, shifts in mean values, and
some other types of irregularities. A common experience is that some initial
subgroups of data will not be under control but, in general, after some knowl-
edge is gained in the use of the control chart, the process will tend to reach a
state of statistical equilibrium. After this time period, one generally has an
adequate amount of data to produce realistic estimates of the mean and
standard deviations.
Computing control limits
Two variable control charts should be kept, one for the average value,
and the other for the range of individual determinations in each subgroup. In
all cases in which a non-zero component for between-subgroups is known to
exist, the control limits for the chart of averages will be based on the "total"
standard deviation for subgroup averages.
If the subgroups are of size n, and if 6- pi and 6-| represent the estimated
components of variance within subgroups and between subgroups, respec-
tively, then the "total standard deviation" for the averages of subgroups is
(4.65)
This quantity can also be obtained by directly calculating the standard devia-
tion of the subgroup averages in the baseline period.
The control chart of the ranges will be used to ascertain whether the
variability among individual readings within subgroups is consistent from
50
subgroup to subgroup. The limits for this chart will be based on the within-
subgroup standard deviation.
Calculating the standard deviation
Using the available data for k subgroups, each of size n we will have the ,
layout shown in Table 4. 15. The standard deviation within subgroups can be
estimated from
Sw^-^ (4.66)
where
(4.67)
and the value of d2 can be obtained from standard control chart tables (see
Duncan/2 p. 927). Values of d2 for typical sample sizes are given in the fol-
lowing table:
n ^2
2 1.128
3 1.693
4 2.059
5 2.326
The value of can be accurately determined by pooling the variances
from each subgroup (see section on precision and accuracy). However, the
above estimate, based on the average range, is sufficiently accurate if the
number of subgroups is large enough (say, 25 or more).
The standard deviation of the k sample averages is:
S,= MrzM.
-
(4.68)
k 1
The between-subgroups standard deviation is given by
(4.69)
Table 4. 15. Layout ¥OkX, R Control Charts

Subgroup Determinations Mean Range
I . ,X,n
2 A'2i,A'22, • • • yîn ^2 R2
3 X3\,X32t • i^3n
• • • •
• • • •
k • fXicn Rk
X R
51
and the total standard deviation for individual determinations is:
St^ = Vsf+Sf (4.70)
The total standard deviation for averages of n daily determinations is:
St=Is-^+^'^ (4.71)
Note that St^'is identically equal to Sj-.
Control limits for the chart of averages
The control limits for the chart of averages are given by:
UCLj, =1 + 3Sj. (4.72)
and
LCLj. =X - 3Sj. (4.73)
where UCL =
upper control limit; LCL = lower control limit.
The warning limits are:
UWLj,= X + 2Sj. (4.74)
and
LWLj,= X - 2Sjr (4.75)
where UWL = upper warning limit; LWL = lower warning limit.
Control limits for the chart of ranges
Based on the three-sigma concept (see section on control limits),

limits
the control limits for the chart of ranges are given by ^ ± 3aii. Using stand-
ard control chart notation, these limits are:
fyCLft= D,R (4.76)
and
LCLn = D^R {4.77)
where
D,= I + 3^ (4.78)
and
D,= \- 3'^' (4.79)

d
and the values of dz, d-^, Dg, and are given in Natrella"' and Duncan.''^ For
n = 2, those values are = 3.267, and = 0.
52 r
The warning limits for « =2 are:
UWLfi^ 2.5\2R
LWLji= 0
The numerical value 2.512 is obtained as follows:
2.512=1 + 24
d2
=1 + 2^^
1.128
Examples of average and range (X and R) charts
Initial data
The data in Table 4.16 represent 25 daily, duplicate determinations of a
cholesterol control, run on a single-channel Autoanalyzer I, 40 per hour. It
may appear strange that all 50 values are even. This is due to a stipulation in
the protocol that the measured values be read to the nearest even number.
The data cover a period of two months, with the analyzer run at a frequency
Table 4.16. Example of X, R Chart: Cholesterol Control Run

Dim
Kun 1
1
Dun 0 jvicdn Range
Day Xn X i2 Xi Ri
1
1 iy\) iyZ 1Q1 Z
2 392 388 390 4
3 392 388 390 4
4 388 388 388 0
5 378 396 387 18
6 392 392 392 0
7 392 390 391 2
8 398 402 400 4
9 404 406 405 2
10 400 400 400 0
11 402 402 402 0
12 392 406 399 14
13 398 396 397 2
14 380 400 390 20
15 398 402 400 4
16 388 386 387 2
17 402 392 397 10
18 386 390 388 4
19 386 382 384 4
20 390 386 388 4
21 396 390 393 6
22 396 394 395 2
23 384 388 386 4
24 388 382 385 6
25 386 384 385 2
S = 9,810 120
53
of three days per week. The two daily determinations were randomly located
within patient samples. The control consisted of 0.5 ml of sample extracted
with 9.5 ml of 99 percent reagent-grade isopropyl alcohol.
Computing trial control limits
From the data in Table 4.16:
X = 9810/25 = 392.4
R - 120/25 = 4.8
5| = - {XXi)Vn]/{n - 1) = [3850320 - (981 0)^/25 ]/24 = 36.5
= VieJ = 6.04
The control limits forX can be computed:

UCLjr = 392.4 + 3(6.04) = 410.5
LCLjr = 392.4 - 3(6.04) = 374.3
The warning limits for A" are:
UWLj, = 392.4 + 2(6.04) = 404.5
LWLjr = 392.4 - 2(6.04) = 380.3
The control limits for/? are:
UCL,i = (3.367) (4.8) = 15.7
LCLr = 0
The warning limits of R are:
UWLf, = (2.512) (4.8) = 12.1
Analysis of data
In Figures 4.4 and 4.5, a graphical representation is shown of the control
charts for the mean and range of the daily runs, together with their appropri-
ate control limits.
The means of the daily runs appear to be under control. Only one point,
day above the warning limit, and all points appear to be randomly lo-
9, is
cated around the central line.
The control chart of the range shows two points out of control, days 5
and 14, and one point, day 12, on the upper warning limit.
Let us assume, for the purpose of illustration, that a satisfactory reason
was found for those two points to be out of control in the range chart, and
that it was decided to recompute new limits for both the X and the R charts
based on only 23 days of data.
The new values are: A' = 392.7,^ = 3.57, 5' = 6. 17, and a? = 23. j.
UCLj, = 392.7 + 3(6.17) = 4\\.2; UWL-^ = 405.0
5A
CHOLESTEROL
CONTROL CHART FOR THE MEAN
(Two determinations per day)
2 4 6 8 10 12 14 16 18 20 22 24 26
DAYS
Fig. 4.4. Control chart for the mean, based on 25 daily duplicate determinations of a
cholesterol control.
LCLi = 392.7 - 3(6.17) - 374.2; LWLjr = 380.4

The new limits for the X chart are practically the same as the previous
limits.
UCLn = (3.267)(3.57) = \\.1;UWLr = 9.0
LCL« = 0 LWLn =0
These values establish the final limits, based on the baseline period.
CHOLESTEROL
CONTROL CHART FOR THE RANGE
20
16
12
8 10 12 14 16 18 20 22 24 26
DAYS
Fig. 4.5. Control chart for the range, based on 25 daily duplicate determinations of a
cholesterol control.
55
Additional data
Nineteen additional points were obtained for days 26 to 44, running

through a period of about one-and-a-half months. The values are shown in
Table 4.17.
Figures 4.6 and 4.7 show the results of the 19 additional data points
plotted against the (corrected) control limits based on the baseline period.
The A'-chart shows two points, days 38 and 39, out of control, about 40
percent of the points near the warning limits, and a definite trend toward
large values of X after day 30. There is a run of seven points above the cen-
tral line after day 37 and, in fact, if one considers day 37 to be "above" the
central line (the mean of day 37 is 392), the run of points above the central
line is of length 12. As indicated in the section on control limits, these consid-
erations are indications of a process out of control.
The /?-chart shows one point out of control and two points above the
upper warning limit; although the value of R based on the 19 additional val-
ues, 4.32, is larger than the previous value, R = 3.57, the difference is not
significant.
The new set of points taken by produced the following values:
itself
X = 396.5, R = 4.32, and 5j. = 12.17, where n = 19.
Future control limits
generally desirable to have a well-established baseline set so future

It is
points can be evaluated with confidence in terms of the baseline central line
Table 4. 17. Additional Values for Cholesterol Control Run
Run 1 Run 2 Mean Range

Day Xn Xi2 Xi Ri
26 392 392 395 6

27 376 376 376 0
28 390 386 388 4
29 394 384 389 10
30 382 378 380 4
31 384 382 381 2
32 384 388 386 4
33 402 392 397 10
34 390 398 394 8
35 402 402 402 0
36 398 394 396 4
37 390 394 392 4
38 426 428 427 2
39 414 428 421 14
40 402 398 400 4
41 402 400 401 2
42 402 404 403 2
43 400 402 401 2
44 404 404 404 0
S = 7,533 82
56
CHOLESTEROL
CONTROL CHART FOR THE MEAN,
USING CORRECTED LIMITS
(Additional Data)
- UCL= 411.2
••
UWL= 405.0
- X = 392.7
•
LWL= 380.4
-• LCL= 374.2
DAYS
Fig. 4.6. Control chart for the mean, based on 19 additional data points, plotted
against the corrected control limits.
and control limits. If, in the example under discussion, the additional set
(days 26 to 44) was found to be satisfactorily consistent with the baseline
data, then it would be proper to extend the baseline period by this set, i.e., a
total of 25 + 19 = 44 points. However, we have already observed a number
of shortcomings in the additional set, and the proper action is to search for
the causes of these disturbances, i.e., "to bring the process under control."
This is of course not a statistical problem.
For the purpose of our discussion, we will assume that an examination
of the testing process has revealed faulty procedure starting with day 37.
Therefore, we will consider a shortened additional set, of days 26 through
36. The following table gives a comparison of the baseline set (corrected to
23 points as discussed previously) and the shortened additional set (11
points).
Baseline Set Additional Set
Number of points, TV 23 11
Average, Z 392.7 389.5
Average Range, 7? 3.57 4.73
Standard deviation, 5 J - 6.17 8.15
By using the F test, ^"^ it is easily verified that the diflference between
the two standard deviations is well within the sampling variability that may
be expected from estimates derived from samples of 23 and 1 1 points, respec-
57
CHOLESTEROL
CONTROL CHART FOR THE RANGE
USING CORRECTED LIMITS
R (Additional Data)
34 36
DAYS
Fig. 4.7. Control chart for the range, based on 19 additional data points, plotted
against the corrected control limits.
tively. The between the averages, X, is 392.7 - 389.5 = 3.2. A

difiference
rough test can be made to see whether this difference indicates a real shift
between the two sets. The standard error of the difiference is approximately
[(6.17)723 + (8.15)711]^ = 2.77. Thus the difference, 3.2, is equal to -j~
= and this is well within sampling errors.
1.15 standard errors,
therefore not unreasonable in this case to combine the 34 points of
It is
both sets to construct a new baseline. This results in the following parame-
ters: N = 34, 1 = 391.7, R = 3.95, and Sj^ = 6.93.
The new control limits are:
ForX: UCL 412.5 UWL 405.6
LCL 370.9 LWL 377.8
For/?: UCL 12.9 UWL 9.9
LCL 0 LWL 0
Using these new parameters, it can be noted that the points correspond-
ing to days 37 through 44 may indicate a potential source of trouble in the
measuring process.
Control chart for individual determinations
It is possible, although not recommended, to construct charts for indi-

vidual readings. Extreme caution should be used in the interpretation of
points out of control for this type of chart, since individual variations may
not follow a normal distribution. When a distribution is fairly skewed, then a
transformation (see section on transformation of scale) would be applied be-
fore the chart is constructed.
58
The steps to follow are:
1) Use a moving range of two successive determinations;
2) Compute R =
3) Determine the controllimits for
^±34 "2
For /7 = 2, ^2 = 1.128, and hence the control limits are:
X ± 2.66 R
4) The upper control limit fori? is D4R = 3.267 R. The lower control limit is
equal to zero.
Other types of control charts
Control charts can also be constructed based on the average standard

deviation, a, of several subgroups of sample data, or on "standard" values
of cr, called a' in the quality control literature (See Duncan, Chap. 20).
Control chart for attributes —the P-chart

The generally used for quality characteristics
fraction defective chart is
that are considered attributes and are not necessarily quantitative in nature.
To use this chart, it is only necessary to count the number of entities that
have a well-defined property, such as being defective, have a certain type of
disease, or have a glucose content greater than a given value, and translate
this number into a proportion. The data used in this chart are easy to handle,
and the cost of collection is normally not very high. In some instances, the
P-chart can do the job of several average and range charts, since the classifi-
cation of a "defective" element may depend on several quantitative charac-
teristics, each of which would require an individual set of average and range
charts for analysis.
The sample size for each subgroup will depend on the value of the pro-
portion Pbeing estimated. A small value of P will require a fairly large
sample size in order to have a reasonable probability of finding one or more
"defectives" in the sample (See Duncan^^). In general, a value of n between
25 and 30 is considered adequate for the calculation of a sample proportion.
Control limits and warning limits

Since the standard deviation of a proportion is directly related to the
value of the proportion, an estimate;? ofP is all that is needed for the calcula-
tion of the central line and of the control limits.
The central line is located at the value p. The three-sigma control limits
are:
UCL^ = p + 3 (4.80)
59
LCLp = p - 3 (4.81)
where q ^ \ - p. The estimate p is obtained as follows:

Let the data be represented by the table:
Number of Elements
Sample Having a Certain
2
In
Number Size
n
Characteristic
X,
X2
Proportion
p,
P2
3 n Z3
n Xn Pk
Total XXi Xpi
where pt
n
Average proportion:
(4.82)
k
(4.83)
LWL, = p-2 (4-84)
When the sample size does not remain constant from subgroup to sub-
group, the recommended procedure is to compute control limits using the
average sample size. However, when a point falls near the control limits thus
calculated, then the actual limits for this point, using its own sample size,
should be estimated before a conclusion is reached about its state of control.
Control charts for number of defects per unit — tfie C-cfiart
In some instances, it is more convenient to maintain control charts for

thenumber of defects per unit, where the unit may be a single article or a
subgroup of a given size. The "number of defects" may be, for instance, the
number of tumor cells in an area of a specified size, the number of radio-
active counts in a specified period of time, etc. In all these instances, the
probability of occurrence of a single event (e.g., an individual defect) is very
60
small, but the unit is large enough to make the average number of occur-
rences (number of defects) a measurable number.
The Poisson distribution

It can be shown that, when the probability P of an event is very small
but the sample size n is large, then the distribution of the number of occur-
rences c of this event tends to follow a Poisson distribution with parameter
nP = c' The mean and standard deviation of c are:
.
E{c) = c' (4.85)
o-c =y/c^ (4.86)
The random variable c represents the number of defects per unit, the
number of radioactive counts in a given period of time, the number of bacteria
in a specified volume of liquid, etc.
—
Control limits. The upper and lower limits are given by:
UCLc = c + 3 (4.87)
LCL^ = c - 3 Vc (4.88)
Here c is the average number of defects, or counts, obtained using a suffi-

ciently large number, k, of units, c is a sample estimate of the unknown, or
theoretical valuer'.
UWL, = c + 2 \/~d~ (4.89)
LWL, = c - 2 VV (4.90)
Detecting lack of randomness
a process is in a state of statistical control, the observations plotted in

If
the control chart should randomly fall above and below the central line, with
most of them falling within the control limits. However, even if all the points
fall within the upper and lower control limits, there might still exist patterns
of nonrandomness that require action, lest they lead eventually to points
outside the control limits. Procedures for detecting such patterns will be dis-
cussed.
Ruies based on the theory of runs

The most frequent test used to detect a lack of randomness is based on
the theory of runs. A
run may be defined as a succession of observations of
the same type. The length of a run is the number of observations in a given
run. For example, if the observations are classified as a or b, depending on
whether they fall above or below the mean, then one set of observations may
look like:
aaababbbaab
Here we have six runs, of length 3, 1, 1, 3, 2, 1, respectively.
61
Another would be the property of
criterion for the definition of a run
increase or decrease of successive observations. Such runs are called ''runs
up and down/' For example, the sequence 2, 1.7, 2.2, 2.5, 2.8, 2.0, 1.8, 2.6,
2.5, has three runs down and two runs up. In order of occurrence, the
lengths of the runs are 1, 3, 2, 1, 1.
Returning to runs above and below the central value, it is possible
through use of the theory of probability, and assuming that the probability is
one-half that an observation will fall above the central line (and, con-
sequently, one-half that it will fall below the central line), to determine the
probability distribution of the lengths of runs. Tables are available for sever-
al of these distributions (See Duncan, Chap. 6), Some rules of thumb based
on the theory of runs that are very useful in pointing out some lack of ran-
domness are:
1) A run of length 7 or more. This run may be up

or down, above or below
the central line in the control chart. (For runs above or below the median,
the probability of a run of length 7 is 0.015.)
2) A run of two or three points outside the warning limits.
3) Ten out of successive points on the same side of the central line.
1 1
Distribution of points around the central line

When a sufficient number of observations is available, the pattern of
distribution of points around the central line should be carefully examined.
In particular, if the points tend to cluster near the warning or control limits,
or if they show characteristics of bimodality, or if they show a pronounced
skewness either assumption of normality
to the left or the right, then the will
not be satisfied and some transformation of scale may be necessary.
Interpreting patterns of variation in a control chart
Indication of lack of control

A process is out of control when one or more points falls outside the
control limits of either the x or the R-chart, for control of variables, or out-
side the limits of the P-chart, for control of attributes.
Points outside the control limits of the R-chart tend to indicate an in-
crease in magnitude of the within-group standard deviation. An increase in
variability may be an indication of a faulty instrument which eventually may
cause a point to be out of control in the x-chart.
When two or more points are in the vicinity of the warning limits, more
tests should be performed on the control samples to detect any possible rea-
sons for out-of-control conditions.
Various rules are available in the literature about the procedures to fol-
low when control values are outside the limits (see, for example, Haven^^).
Patterns of variation
the x and R-charts over a sufficient period of time, it may
By examining
be possible to characterize some patterns that will be worth investigating in
order to eliminate sources of future troubles.
Some of these patterns are shown in Figure 4.8.
62
Cyclic Variations Stiift in the Average
UCL X
UCL
LCL LCL
High Proportion of Observations

Trend Near the Control Limits
Fig. 4.8. Four patterns of variation in an A'-chart.
The control chart as a management tool

As indicated in the ASQC definition of quality assurance, The sys- . .
tem involves a continuing evaluation of the adequacy and efifectiveness of

the overall quality-control program with a view of having corrective meas-
ures initiated where necessary ."'^
. .
The key words, "continuing evaluation" and ''having corrective meas-

ure initiated," indicate the essence of a quality control program. It is impor-
tant that the results of the control chart be subjected to a daily analysis in
order to detect not only the out-of-control points but also any other manifes-
tation of lack of randomness as shown by a time sequence of daily observa-
tions. It is always better and more economical to prevent a disaster than to
take drastic measures to cure one. Since each test method should be sub-
jected to quality control, the control charts should be prominently displayed
at the location where the test is performed, not only to facilitate the logging
of results as soon as they are obtained but also to give the technician respon-
sible for the test an easy graphical representation of the time sequence of
events. In addition, preprinted forms containing the relevant classification
should be available for easy recording of information such as names, dates,
time of day, reagent lot number, etc.
When all the pertinent data provided by the control charts are available,
the supervisor, or section manager, should have all the meaningful informa-
tion required to take corrective measures as soon as a source of trouble has
been detected. Monthly or periodic review of the results, as performed by a
central organization with the aid of existing computer programs, is impor-
tant to provide the laboratory director with an important management tool,
since the output of these programs may include such items as costs, inter-
and intra-laboratory averages, historical trends, etc. However, as pointed
out by Walter Shewhart^*' and other practitioners of quality control, the most
important use of the control chart occurs where the worker is, and it should
be continuously evaluated at that location as soon as a new point is dis-
played on the chart.
63
References
1. Chemical Rubber Publishing Company. 1974. Handbook of chemistry and physics.

55th ed. Cleveland, Ohio.
2. Mandel, J. 1964. The statistical analysis of experimental data. Interscience-Wiley, New
York.
3. Natrella, M. G. 1963. Experimental statistics. Natl. Bur. Stand. Handb. 91, Washington,
D.C.
4. Davies, O. L., and P. Goldsmith, eds. 1972. Statistical methods in research and produc-
tion. Oliver & Boyd, Hafner, New York.
5. Snedecor, G. W., and W. G. Cochran. 1972. Statistical methods. Iowa State Univ.
Press,Ames.
6. Proschan, F. 1969. Confidence and tolerance intervals for the normal distribution. In
H. H. Ku, ed.. Precision measurement and calibration, statistical concepts and procedures.
Natl. Bur. Stand. Tech. Publ. 300, vol. 1. Washington, D.C.
7. Mandel, J. 1971. Repeatability and reproducibility. Mater. Res. Stand. 11(8): 8-16.
8. Galen, R. S., and S. R. Gambino. 1975. Beyond normality, the predictive value and effi-
ciency of medical diagnoses. Wiley, New York.

9. American Society for Quality Control, Statistical Technical Committee. 1973.
Glossary and tables for statistical quality control. Milwaukee, WI.
10. Shewhart, W. a. 1931. Economic control of manufactured product. Van Nostrand, New
York.
11. Grant, E. L., and R. S. Leavenworth. 1972. Statistical quality control. McGraw-Hill,
New York.
12. Duncan, A. J. 1974. Quality control and industrial statistics. Richard D. Irwin, Home-
wood, IL.
13. Haven, G. T. 1974. Outline for quality control decisions. The Pathologist 28: 373-378.
64 U.S. GPO: 1986-l|91-070/t00i|7

Technical Publications
Periodical
Journal of Research —
The Journal of Research of the National Bureau of Standards reports NBS research
and development in those disciplines of the physical and engineering sciences in which the Bureau is active.
These include physics, chemistry, engineering, mathematics, and computer sciences. Papers cover a broad
range of subjeas, with major emphasis on measurement methodology and the basic technology underlying
standardization. Also included from time to time are survey articles on topics closely related to the Bureau's
technical and scientific programs. Issued six times a year.
Nonperiodicals
Monographs — Major contributions to the technical literature on various subjects related to the Bureau's scien-
tific and technical activities.
—
Handbooks Recommended codes of engineering and industrial practice (including safety codes) developed in
cooperation with interested industries, professional organizations, and regulatory bodies.
Special Publications — Include proceedings of conferences sponsored by NBS, NBS annual reports, and other
special publications appropriate to this grouping such as wall charts, pocket cards, and bibliographies.
Applied Mathematics Series — Mathematical tables,manuals, and studies of special Interest to physicists,
engineers, chemists, biologists, mathematicians, computer programmers, and others engaged in scientific and
technical work.
—
National Standard Reference Data Series Provides quantitative data on the physical and chemical properties
of materials, compiled from the world's literature and critically evaluated. Developed under a worldwide pro-
gram coordinated by NBS under the authority of the National Standard Data Act (Public Law 90-396).
NOTE: The Journal of Physical and Chemical Reference Data (JPCRD) is published quarterly for NBS by
the American Chemical Society (ACS) and the American Institute of Physics (AIP). Subscriptions, reprints,
and supplements are available from ACS, 1155 Sixteenth St., NW, Washington, DC 20056.
—
Building Science Series Disseminates technical information developed at the Bureau on building materials,
components, systems, and whole structures. The series presents research results, test methods, and perfor-
mance criteria related to the structural and environmental functions and the durability and safety
characteristics of building elements and systems.
Technical Notes — Studies or reports which are complete in themselves but restrictive in their treatment of a
subject. Analogous to monographs but not so comprehensive in scope or definitive in treatment of the subject
area. Often serve as a vehicle for final reports of work performed at NBS under the sponsorship of other
government agencies.
—
Voluntary Product Standards Developed under procedures published by the Department of Commerce in
Part 10, Title 15, of the Code of Federal Regulations. The standards establish nationaDy recognized re-
quirements for products, and provide all concerned interests with a basis for common understanding of the
characteristics of the products. NBS administers this program as a supplement to the aaivities of the private
sector standardizing organizations.
Consumer Information Series —

Practical infonnation, based on NBS research and experience, covering areas
of consumer. Easily understandable language and illustrations provide useful background
interest to the
knowledge for shopping in today's technological marketplace.
Order the above NBS publications from: Superintendent of Documents, Government Printing Office,
Washington, DC
20402.
Order the following NBS publications— FIPS and NBSIR's—from the National Technical Information Ser-
vice, Springfield, VA 22161.
Federal Information Processing Standards Publications (FIPS PUB)— Publications in this series collectively
constitute the Federal Information Processing Standards Register. The Register serves as the official source of
information in the Federal Government regarding standards issued by NBS pursuant to the Federal Property
and Administrative Services Act of 1949 as amended, Public Law 89-306 (79 Stat. 1127), and as implemented
by Executive Order 11717 (38 FR 12315, dated May 11, 1973) and Part 6 of Title 15 CFR (Code of Federal
Regulations).
NBS Interagency Reports special series of interim or final reports on work performed by NBS
(NBSIR)— A
for outside sponsors (both government and non-government). In general, initial distribution is handled by the
sponsor; public distribution is by the National Technical Information Service, Springfield, VA 22161, in paper
copy or microfiche form.

Nbsspecialpublication700 2

Uploaded by

Copyright:

Available Formats

Nbsspecialpublication700 2

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Nbsspecialpublication700 2

Uploaded by

Copyright:

Available Formats

U.S.

The National Measurement Laboratory

dustry, and commerce; provides advisory and research services to other

The National Engineering Laboratory

The Institute for Computer Sciences and Technology

77?^ Institute for Materials Science and Engineering

U.S. Department of Commerce

National Bureau of Standards

When the National Bureau of Standards was established more than 80

This publication is an addition to what we anticipate will be a

Ernest Ambler, Director

This paper was published originally as a chapter in the book

* J. Mandel and L.F. Nanni, Measurement Evaluation Quality Assurance

John Mandel holds an M.S. in chemistry from the University of

Dr. Mandel has been a consultant on statistical design and data

Mandel has served as a Visiting Professor at Rutgers University

A fellow of the American Statistical Association, the American

He was Chairman of one of the Gordon Research Conferences on

Luis F. Nanni holds a Civil Engineering degree from the National

Professor Nanni 's fields of specialization are statistical

About the authors v

1 . Basic Statistical Concepts 1

Frequency distribution and histograms 1

Population Parameters and Sample Estimates 2

3. The Control Chart 44

Basic Statistical Concepts

This chapter concerned with the evaluation of measurements by

means of methods. This qualification is important, for the total

Frequency distribution and iiistograms

50 75 100 125 150 175

Population parameters and sample estimates

Table 4.1. Grouped Data for Glucose in Serum

92.5 281 152.5 0

Table 4.2. Two Small Samples of Glucose in Serum

Individual Glucose (mg/dl) Individual Glucose (mg/dl)

Average 107.57 Average 96.37

Population parameters —general considerations

The sample variance is denoted by si and defined by

The sample standard deviation is denoted by Sj. and defined by

Population parameters as limiting values of sample estimates

It is customary to denote population parameters by Greek letters (e.g., /x, a)

Sums of squares, degrees of freedom, and mean squares

are not completely independent of each other. Indeed, by summing them we

^ (Xi - i) = Xxj - Sjc = Ixi - Nx (4.6)

X {x< -x) = lx^-N ^ =0 (4.7)

where / (the "frequency") represents the number of individuals in the zth

Table 4.3. Calculations for Grouped Data

47.5 -11 1 107.5 1 313

92.5 -2 281 152.5 10 0

u — -0.4156 X = 102.5 + 5ii = 100.42

We first deal with the question: "How rehable is a sample mean as an

Standard error of the mean

averages generate, in turn, a nev/ population. It is intuitively clear, and can

Source Mean (mg/dl) Variance (mg/dl)^ Standard Deviation (mg/dl)

Population^ 100.42 147.70 12.15

^We consider the sample of Table 4.1 as identical to the population.

From Equation 4.12 it follows that

Improving precision tlirough replication

The normal distribution