See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/226549131

ISO 5725 and GUM: comparison and comments

Article in Accreditation and Quality Assurance · March 2009

DOI: 10.1007/s00769-008-0478-3

CITATIONS READS

31 2,573

2 authors, including:

Diego Zappa
Università Cattolica del Sacro Cuore
50 PUBLICATIONS 163 CITATIONS

SEE PROFILE

All content following this page was uploaded by Diego Zappa on 03 January 2022.

The user has requested enhancement of the downloaded file.

Accred Qual Assur (2009) 14:159–166
DOI 10.1007/s00769-008-0478-3

DISCUSSION FORUM

ISO 5725 and GUM: comparison and comments

Laura Deldossi Æ Diego Zappa

Received: 23 July 2008 / Accepted: 25 November 2008 / Published online: 13 December 2008

Springer-Verlag 2008

Abstract We compare the approach to measure uncer- measurement procedures must be deeply tested to assess
tainties proposed in ISO 5725 and GUM from a statistician their adequateness (see, for a review [1]). The most
point of view. In particular we give some warnings to the important international standards on this topic are the ISO
application of the expanded uncertainty introduced in 5725:1994 (Part 1–6) [2] and the ISO ‘‘Guide to the
GUM when the input variables are few and we report some expression of Uncertainty in Measurement’’ (GUM) [3]
considerations on the relevant role of the interactions published in 1995, both proposing a method to calculate
among the input variables in the measurement equation as the uncertainty of a measurement.
well as the role of statistical design of experiments to In this paper we comment these two approaches mainly
measure uncertainties. from a statistician point of view and at the light of a
research in collaboration with a microelectronic company.
Keywords Measurement equation
In this field, it is easy to imagine that accuracy of mea-
Confidence intervals
Design of experiments surement systems is essential due to the very small
dimension of their products. Almost every day the capa-
bility of very expensive measurement instruments, used
Introduction both in the production and in the development phase of a
product, is tested, using, where appropriate, either the
In the last 15 years the International Organization of approach based on the gauge repeatability and reproduc-
Standardization (ISO) has produced several documents on ibility studies (GRR) (that is the principles in the ISO
uncertainty measurement. The interest of ISO in this topic 5725) or the GUM procedure. The same issue may be
relies on the consideration that quantitative evaluation of of interest for the testing and calibration laboratories, as
the uncertainty of a measurement is necessary to say ISO/IEC/EN 17025 [4] standard demands that ‘‘Testing
whether the differences among the results of an experiment laboratories shall have and shall apply procedures for
or the variability of a process are due to either an out of estimating uncertainty of measurement’’ and suggests to
control process or to a not capable measurement system. use ISO 5725 and GUM to compute measurement
This is relevant because process variation is becoming uncertainties.
small due to technological improvements and therefore To stress furthermore the relevance of this topic, a round
table was recently organized on the evolution of these two
standards, within the Advanced Mathematical and Com-
Papers published in this section do not necessarily reflect the opinion
of the Editors, the Editorial Board and the Publisher. putational Tools in Metrology and Testing Conference,
held in Paris in June 2008.
Presented at AMCTM conference, June 2008, Paris, France. We are aware that this is an area traditionally involving
scientists from chemistry, physics, biochemistry and other
L. Deldossi (&)
D. Zappa
related sciences, but recently it has been broadening,
Department of Statistical Sciences, Catholic University,
Largo Gemelli 1, Milan 20123, Italy including statisticians too. In fact many arguments reported
e-mail: laura.deldossi@unicatt.it in the standards concern concepts and methods typical of

123
160 Accred Qual Assur (2009) 14:159–166

this area, mostly due to the randomness that affects the time intervals. Reproducibility conditions exist when test
measurement. results are obtained with the same method on identical
In particular in ‘‘ISO 5725 and GUM approaches to test items in different laboratories with different operators
measure uncertainty’’ analogies and differences between using different equipment. Thus repeatability and repro-
ISO 5725 and GUM are illustrated. In ‘‘Some consideration ducibility are the two extremes of precision, the former
about GUM’’ some warnings to the ISO users are given in describing the minimum and the latter the maximum
order to avoid wrong applications of the standards and, variability of the results.
consequently, wrong evaluation to measure uncertainty. In Then ISO 5725 concerns exclusively with measurement
‘‘How to collect data to define the uncertainty of a mea- methods which yield measurements y typically on a con-
surement?’’ we report some considerations on statistical tinuous scale with a single value for each test result. The y
experimental design in the computation of the uncertainty values describe, in quantitative terms, the ability of a
of a measurement. method to give a correct result (trueness) or to replicate a
given result (precision) and they are obtained by a col-
laborative study, that is an interlaboratory experiment run
ISO 5725 and GUM approaches to measure uncertainty under reproducibility condition.
The model explaining the measurand Y depends on the
ISO 5725 and GUM approaches to the validation of the experimental design used to obtain the measurement. It is
measurement procedure are quite different even if they generally a random effect model such as
were published almost in the same year. The main differ- Y ¼lþBþE ð1Þ
ences are about: (a) the terms used (e.g. accuracy in ISO
5725, uncertainty in GUM); (b) how to gather data in the where l is the overall mean, B is the laboratory compo-
measurement procedure; (c) the model and the techniques nent of bias under repeatability conditions and it is
to analyze data; (d) how to present the results. We will assumed to be normally distributed as N(0, r2L), E is the
analyze these differences in ‘‘Comparison between ISO random error under repeatability conditions assumed to be
5725 and GUM’’, after having introduced the main char- N(0, r2E), B and E are supposed to be uncorrelated.1
acteristics of both approaches in ‘‘ISO 5725: measurement Observe that r2L includes the between-operator and the
methods and results’’ and ‘‘The guide to the expression of between-equipment variability, while r2E is the repeat-
uncertainty in measurement (GUM)’’. ability variance. The reproducibility standard deviation
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
rR ¼ r2L þ r2E is estimated using the analysis of vari-
ISO 5725: measurement methods and results ance (ANOVA) method.
In ISO 5725 no confidence interval for l is present. Only
ISO 5725 deals with accuracy of measurement methods repeatability (r) and reproducibility (R) limits are defined.
and results. It uses two terms—trueness and precision— These limits are the values less than or equal to which the
to describe the accuracy of a measurement method. absolute difference between two test results, obtained
Trueness refers to the closeness of agreement between under either repeatability or reproducibility conditions,
the arithmetic mean of a large number of test results and may be expected to be with (1 - a) probability level.
the accepted reference value, while precision refers to the Given model in Eq. 1, r and R corresponds to
closeness of agreement between test results. Two mea- pffiffiffi pffiffiffi
r ¼ tdf ;a=2
sr
2; R ffi tdf ;a=2
sR
2
sure of trueness may be of interest: bias of the
measurement method (the difference between the where tdf,a/2 is the upper a/2 percentile of a t Student
expectations of test results obtained from all laboratories distribution, df are appropriate degrees of freedom and sr,
using that method and the accepted reference value) and sR are the square roots of the estimate of the variance under
the laboratory bias (the difference between the expecta- repeatability and reproducibility conditions, respectively.
tion of the test results for a particular laboratory and the Observe that R is defined on the basis of an approximation
accepted reference value). In both cases a reference value because s2R corresponds to a linear combination (with
is required. On the other hand, precision does not require positive signs) of chi-squared random variables for which
a reference value, as it involves only comparisons an exact distribution does not exist.
between replicate measurements of a quantity under
specified conditions. It can be measured under repeat-
ability or reproducibility conditions. The former exists
when independent test results are obtained with the same 1
The assumption of uncorrelation between B and E is not present in
method on identical test items in the same laboratory by ISO 5725, but it is correctly reported in the Appendix A.2
the same operator using the same equipment within short ‘‘Collaborative study approach’’ of the ISO 21748:2004 [5].

123
Accred Qual Assur (2009) 14:159–166 161

Table 1 Comparison between

Scope ISO 5725 GUM
ISO 5725 and GUM
Accuracy: trueness and precision Measurement uncertainties

Measurand Y is measurable Y may be not directly measured

Data obtained by Statistical experimental design NO statistical experimental design
Model Y=l?B?E Y = f(X1, X2,…,Xm)
Variability of the measurand Repeatibility and reproducibility Combined standard uncertainties
variance u2c (y)
Confidence interval NO: only the definition of NO: only the definition of an
repeatibility and reproducibility interval, y ± Up, that could
limit reasonably be attributed to the
measurand.

The guide to the expression of uncertainty Around the measurement y the expanded uncertainty,
in measurement (GUM) Up, defines an interval [y ± Up] that should include the
values that could reasonably be attributed to the measu-
On the contrary to the ISO 5725, in the GUM framework, Y rand. The coverage probability of the interval [y ± k uc(y)]
may not be directly measured and then no experimental is generally not directly measurable since a state-of-
design is generally run to obtain data. The basic idea is that knowledge probability distribution for Y cannot be exactly
a measurement equation exists. It may be a known (or even determined in general. Relying on the central limit theo-
an unknown) function generically written as follows: rem, the probability density function (PDF) of Y is in many
Y ¼ f ðX; hÞ X 2 B  Rm ð2Þ cases supposed to be Gaussian. k = 2, or an appropriate
percentile of the t Student, is generally used to guarantee
where X = (X1,…,Xm) is a vector of m random variables that the confidence interval has an expected confidence
and h = (h0, h1,…,hm) is a vector of m ? 1 coefficients. probability level of 0.95.2
Given a vector of values, x = (x1,…,xm), the estimate y is From the beginning GUM was a widely adopted stan-
obtained from Eq. 2 as dard approach. It gave the possibility to take into
y ¼ f ðx; hÞ ð3Þ consideration the uncertainty of Type B, that is uncer-
tainties specified by information not obtained through
where (x1,…,xm) may be composed of either/both the mean collaborative trials, such as calibration. In the practice the
of n independent repeated measurements of X (called Type experimenter has to prepare ‘‘only’’ the ‘‘uncertainty bud-
A method) or/and the expected mean of the probability get’’, that is the list of all input variables Xi in Eq. 2 with
density function (PDF) assumed for X (called Type B their corresponding u(xi) and ci.
method). The latter may be based also on non statistical
evaluations including expert judgment.
To avoid the non linearities in Eq. 2 the Taylor series Comparison between ISO 5725 and GUM
expansion of f(X; h) is calculated in the neighbourhood of
X = x. Truncating the expansion at the first order, we may As it is clear from the previous paragraphs there are many
approximate Eq. 2 as follows differences between the two ISO guidances. A comparison
X is reported in Table 1.
Y  c0 þ c i Xi ð4Þ We observe that:
where ci ¼ ½of =oXi Xi ¼xi (i = 1,…,m) are the ‘‘sensitivity (a) ISO 5725 and GUM use different terms but they share
coefficients’’. Putting x in Eq. 4 we obtain an the same goal, as uncertainty of a measurement is the
approximation of the unknown value of y in Eq. 3. As it opposite of its accuracy. In fact VIM defines accuracy
is often necessary to find the lower and the upper bound for of the measurand as ‘‘the closeness of agreement
y, GUM introduced the concept of expanded uncertainty, between a quantity value obtained by measurement
Up. It is defined as Up = k
uc(y), where and an accepted reference value’’, while uncertainty is
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
X ffi defined in GUM (clause 2.2.3) as ‘‘a parameter
uc ðyÞ ¼ c2i u2 ðxi Þ ð5Þ associated with the result of a measurement that
is called combined uncertainty of y. It depends on the characterizes the dispersion of the values that could
uncertainties u(xi) of each input variables and k is a con-
stant called coverage factor. Formula 5 is based on the 2
Note that k should be 1.96 to guarantee exactly 0.95 if Y is supposed
assumption that the input variables are not correlated. to be normally distributed.

123
162 Accred Qual Assur (2009) 14:159–166

reasonably be attributed to the measurand’’. It follows variability, to measurement configuration (differences due
that the greater is the dispersion of the values the to different instruments, operators, geometries) or to
greater is the uncertainty of the measurement and the material inhomogeneity, while it uses the GUM approach
lower is its accuracy. to compute the expanded uncertainty.
(b) In ISO 5725 the measurand is directly measurable on ISO 21748 deals with the evaluation of measurement
the contrary to GUM, where the measurand may be uncertainties using data obtained from studies run in
not directly measured, for example when the measu- accordance with ISO 5725-2, that is relative to the inter-
rand is an area or a volume. laboratory experiments. These studies yield performance
(c) Data are obtained by a statistical experimental design indicators (sr and sR) estimated using Eq. 1. In the practice
in ISO 5725 and not so in GUM: the consequences of sR does not necessarily include variation of all the effects
this approach will be discussed in ‘‘How to collect that influence a measurement result. This is the reason for
data to define the uncertainty of a measurement?’’. which this guidance modifies Eq. 1 introducing the fol-
(d) Eq. 1, explaining the measurand in the ISO 5725, is a lowing model
statistical model, while f(X; h), in the GUM frame- Xm
work, is a mathematical function. Y ¼ l þ d þ B þ ci Xi þ E ð6Þ
(e) The combined standard uncertainties uc(y) is com- i¼1

puted in GUM through the law of propagation of where in particular d represents the intrinsic bias due to the
uncertainties, while sr and sR are computed in ISO measurement method, X*i are input variables assumed to be
5725 by ANOVA method. N(0, u2(xi)) (in the GUM approach of type A only) and the
(f) GUM defines an interval around the measurement y summation is over all those effects subject to deviations
while in ISO 5725 only the repeatibility and repro- other than those incorporated in B or E. Then Eq. 6 is a
ducibility limits (r, R) are defined. combination of Eq. 1 used by ISO 5725 and Eq. 4 used by
However, analogously to GUM, in the ISO framework the GUM approach.
an interval around y may be found. In fact, under repro-
ducibility conditions, we may assume that
 pffiffiffi Some consideration about GUM
y  tdf ;a=2
sR = n
corresponds to an approximate (1 - a) confidence interval The attractive features of the GUM framework contrast with
for l = E[Y]. The interval is an approximation because, as the many assumptions that are usually necessary to find Up:
already mentioned in ‘‘ISO 5725: measurement methods the application of the law of propagating errors, the distri-
and results’’, s2R is approximately v2 distributed with degree butional assumption especially for Type B components,
of freedom, df, computed using the Welch–Satterwhaite how to choose k (we remind the many criticism appeared in
formula. the metrological literature, see, e.g. the special issue of
It is clear that the GUM approach, unlike ISO 5725, Metrologia in 2006), the assumption of independence (or,
allows to consider also the calibration uncertainties, so less strongly, of uncorrelation) among (X1,…,Xm) and so on.
guaranteeing the traceability of the measurement instru- In particular the first order Taylor expansion in Eq. 4 may
ments to primary standards [6]. be a poor approximation if significant non linearities exist.
For some particular circumstances (linearity of the We agree with [8] when it is said that every method of
function f(X;h) in Eq. 2, normality distribution and inde- evaluation of expanded uncertainty is an approximate
pendence of the input variables) the estimates of the method, and the degree of approximation cannot be evalu-
variance of the measurand obtained by the two approaches ated because there is often no defined reference method that
may be comparable. This is probably the reason these two is considered accurate. Therefore, if we wish to discuss the
guidances are sometimes proposed to be alternative (as in accuracy of approximate methods of evaluation of expan-
[4]) or even to be used jointly. ded uncertainty, we must define an accurate method against
A combination of the ISO 5725 and GUM’s approaches which the approximate methods may be compared.
may be found in two ISO technical specifications (ISO/TS In this direction [9] constitutes a very significant
21748:2004 [5], ISO/TS 21749:2005 [7]). improvement. Conditions for valid application of GUM are
ISO 21749 follows the GUM approach to compute the there reported in detail, as well as how to apply Monte Carlo
expanded uncertainties but it proposes to use the analysis Methods (MCM) (see also, e.g. [10]) to compute (generate)
of variance to estimate uncertainties of components clas- the distribution of Y in Eq. 2 is broadly commented.
sified as Type A. This guidance considers nested designs to Maybe the major property of MCM methods is that an
estimate the uncertainties due to time-dependent sources of interval for y, e.g. [ylow, yhigh], such that it almost exactly

123
Accred Qual Assur (2009) 14:159–166 163

Z Z
encompasses a desired (1 - a) fraction of the simulated
distribution for Y instead of the approximate interval gY ðy; k; hÞ ¼ ... jJj gX ðy1 ; x2 ; . . .; xm ; kÞdx2


dxm
[y ± Up] may be found. In particular, MCM works well ð7Þ
even when the distribution of Y is asymmetric. Analo-
gously to the GUM approach, MCM cannot manage where |J| = qf-1(y, x2,…,xm; h)/qy.
dependencies among X if no prior information are formu- Supposing, as it is often assumed by GUM and in the
lated or if no appropriate design of experiment is run to practice, that the variables in X are independent and letting,
measure the correlations. May be a suggestion is to put a without loss of generality and for a matter of simplicity,
prior also on the correlation matrix among X or to adopt the c0 = 0 and ci = 1 Vi in Eq. 4, Y is a convolution (less
opinion of an expert or even to exploit the power of the so- properly it may be thought of as a sum) of random vari-
called copulas. But all these solutions are far to be practical ables. ThenZEq. 7Zbecomes
and certainly they contrast with the main property of GUM: gY ðy; kÞ ¼ ... gX1 ðy1  x2 


 xm ; kX1 Þ
the one to be a quite easy technique to manage Eq. 2. Some
hints are available in [9] on how to deal with these not gX2 ðx2 ; kX2 Þ


gXm ðxm ; kXm Þdx2


dxm ð8Þ
irrelevant details. Many other detailed technical notes are which may be solved iteratively, considering first the
in the Supplement 1 to the GUM and interesting is, among convolution of X1 þ X2 ¼ Y12 
; then the convolution of
the many issues, the proposal to use the principle of  
Y12 þ X3 ¼ Y123 and so forth.
maximum entropy to choose an appropriate distribution for To exemplify, Eq. 8 has been implemented in Excel for
X when no experimental data are available. a case that is very common in the practice. We have created
Notwithstanding the simplicity of the MCM approach we in a single cell of the spreadsheet a formula that computes
believe that if Eq. 2 is not too complex or the number of the probability density function of Eq. 8 supposing
input variables is small and for a matter of simplicity X1 * N(0, r2) and Xi * U(-ai, ?ai) for i = 2,…,5. This
(Eq. 4) is thought of simply as a sum (or a weighted sum) of case corresponds to the hypothesis that X1 is a Type A
variables, it is better to apply the method of transformation variable and the others are Type B uniform random vari-
of random variables because, given the experimenter’s ables (with different domain). Of course the same example
assumptions, we may find mathematically the interval that may be modified substituting the gaussian distribution with
encompasses the desired (1 - a) fraction of the exact dis- another one. We have chosen the gaussian distribution just
tribution for Y. Such a little bit more complex approach may for a matter of simplicity well knowing that it is often an
solve both the computational time requested by the MCM idealization of reality. It may be thought of as the appli-
approach and the validation of the procedure. In addition it cation of the central limit theorem when many Type A
is possible to show that the convolution of (few) random input variables are considered. Additionally, we make
variables may be numerically done even with very easy notice that also in [9] with the exception of only two cases,
tools such as a spreadsheet. The major drawback is the most of the suggested distributions for X are symmetric,
possibility to appreciate how much great the approxima- and then the convergence of Eq. 4 to normality should be
tions are due to the adoption of the expanded uncertainty, sufficiently fast.
Up, with respect to the exact solution, even for a very wide Suppose that [y ± kuc(y)] is found using k = U-1
range of alternatives for the domain of X. (0.975) = 1.96, where U(
) is the cumulative density
To show this, we may say that given Eq. 2, supposing function (CDF) of a N(0,1) random variable.
that the joint PDF of X is known and that the inverse f-1 qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
VarðX2 þX3 þX4 þX5 Þ
exists at least on a partition of the domain, then we may Figure 1 is the scatter plot of v ¼ Var ðX1 Þ for
find the distribution for Y as follows: suppose gX(x; k) r2 = 0.2(0.2)2 and ai = 0.2(0.2)2 versus the ratio r ¼
is the (multivariate) PDF of X and let arbitrarily Pr 1 ðY 0:975Þ
where Pr 1 ðY 0:975Þ ¼ y0:975 is the per-
Y1 : Y = f(X;h). Using f-1, let U1 ð0:975ÞrU
centile of order 0.975 of the convolution Y and
1 qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
ffi
f ðY1 ; X2 ; . . .; Xm ; hÞ ¼ X1 : a2 a2 a2 a2
rU ¼ 32 þ 33 þ 34 þ 35 þ r2 :
To find the PDF of Y the method of transformations of From Fig. 1 it is clear that as the contribution to the
random variables may be used. We must introduce (m - 1) overall variability due to the uniform distributions increa-
1:1 transformations such that Yi : Xi for i = 2,…,m. ses, that is as v increases, r decreases, showing that having
Call J the Jacobian matrix with elements (i,j) equal to fixed to 0.95 the (1 - a) probability level of the interval
qXj/qYi for i ,j = 1,2,…,m. Then the PDF of gY(y; k, h) will [y ± k uc(y)], the practitioners choice of k = U-1(0.975)
be given by overestimates the exact value found using the convolution.

123
164 Accred Qual Assur (2009) 14:159–166

1 request of the presence, in the panel of experts, of at least a

0.99
person with experience in statistical design and analysis of
0.98
experiment and familiar with the measurement method and
0.97
0.96
its application (e.g. see clause 6.1.1 of ISO 5725: Part 1).
r 0.95
Using a different approach, in GUM n independent
0.94 repeated measurements of input quantities are requested to
0.93 estimate the uncertainties of type A while only expert
0.92 judgement and/or probability distributional assumptions
0.91 are necessary to estimate the uncertainties of type B. Some
0.9
0 1 2 3 4 5 6 interesting considerations on repeated measurements in the
v GUM context in contrast with replicated measurements are
reported in [16].
Fig. 1 Scatterplot of the approximations due to the adoption
Even if, generally, in ISO 5725 simple design (one
of the expanded uncertainty, Up, with respect to the exact
solution found using the convolution of the random variables factor or nested design) are considered, the opportunity to
for the example in ‘‘Some consideration about GUM’’
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
profitably extend the use of DOE to collect data in order to
1
þX3 þX4 þX5 Þ
½X axis : v ¼ VarðX2VarðX 1Þ
; Y axis : r ¼ PrU1ðY 0:975Þ
ð0:975Þr
 measure uncertainty is underlined, e.g. in [11], to include
U
design with several factors (factorial, nested or split-plot)
and mixed design (with fixed and/or random factors).
The major drawback is that the interval [y ± Up] is wider In [17] a Central Composite Design (CCD) and a corre-
than necessary so that the probability to consider a mea- spondent response surface model for the experimental
surement within the uncertainty limits when it is not, estimation of measurement uncertainty in chemical
increases. The minimum of r has been observed at measurement is proposed.
r2 = 0.2, a2 = 2, a3 = a4 = a5 = 0.2, confirming that if a Depending on the kind of experiment and factors, i.e.
dominant contribution to the combined uncertainty from an depending on how data are collected, an appropriate model
input quantity exists, the choice of k = U-1(0.975) is far to explain the measurand can be identified and statistical
from the true value. methods are available to estimate the parameters of interest
Certainly using the GUM approach, how to include the and variance components using ANOVA. Additionally the
assumption of dependence among (X1,…,Xm) continue to main by-products of obtaining data by a statistical experi-
be a problem. The reason may be addressed to the fact that ment are:
using the GUM, the experimenter samples from Xi
1. the randomization of the trials in the design guarantees
marginally with respect to all the other variables and this
the independence of the random effects and the error
does not allow to evaluate the dependence among the
component;
variables. The consequence is that uc(y) may be over or
2. the factorial experimental design gives the possibility
under estimated depending on the sign of the covariances if
to estimate, if they exist, interactions among factors
they are not zero.
(the latter is in particular not possible in the GUM
May be the previous issue is not a problem at all for the
procedure if not explicitly requested);
practitioners. Those who apply the GUM procedure are
3. numerous statistical results are available to obtain
interested in completing the uncertainty budget [11] with-
approximated confidence intervals of variance compo-
out any validation of the assumption of independence
nents as modified large sample (MLS), generalized
among input quantities (see, e.g. [12]). Analogously no
confidence intervals (GCI) and simulated confidence
explicit function (Eq. 2) may be formulated resulting the
intervals (SCI). These methods generally performs
combined uncertainty to be equal to Eq. 5 (see, e.g. [13]).
better than the Welch–Satterwhaite approximation [18,
pp. 146–147] as it allows to find more precise
confidence intervals and they could easily be used in
How to collect data to define the uncertainty
metrological context to assess whether each variance
of a measurement?
component is relevant before pooling them to obtain
the estimate of the variance of Y.
How to gather data to define the uncertainty (or accuracy)
of a measurement is an important task. ISO 5725 and GUM Can experimental design be useful to collect data also in
seems to propose a quite different answer to this question. GUM approach?
The use of experimental designs (DOE) (see, e.g. [14, As it has been suggested by a referee—and we agree
15]) to obtain data is a natural procedure in ISO 5725 with him—when the measurement is measurable, DOE
where the relevance of DOE is also underlined by the should be profitably adopted to collect data, both to

123
Accred Qual Assur (2009) 14:159–166 165

establish which input quantities in (2) are relevant to and an empirical method (based on an experimental design
explain the uncertainties of the measurand and to estimate as ISO 5725). In modelling, the estimation of uncertainty
the uncertainties components of the input quantities of type involves the identification, quantification and summation
A. In fact DOE allows to collect data and analyze them by (as variances) of each potential source of uncertainty. This
statistical methods using the smallest number of trials and approach becomes increasingly problematic in identifying
then at the minimum cost (see, e.g. [19]), and, most all of such sources when it is applied, e.g. to primary
important, to isolate the effect of each factor to determine if sampling. On the other hand the empirical approach to
their interactions are relevant. In many parts of GUM and uncertainty estimation involves replicated measurements
in [9], it is written that possible dependence among input from either inter-laboratory trials and/or internal method
quantities have to be taken into account. But it is not validation and quality control. In [22] a strategy for rec-
absolutely explained to the users how to check such a onciling the two approaches is proposed. In [23] the
dependence (see, e.g. [20]). DOE can resolve this question strengths and weakness of the two methods of uncertainty
as it allows to put in evidence the effective relevance of estimation are discussed and it has been proved, by a case
each input quantities on Y and their possible correlation study, that the estimate of sampling uncertainty made using
both in the case of random and fixed factors. Sometimes the modelling approach resulted to be six times larger than
the dependences among input quantities are due to the that found using the empirical approach. The difficulty in
effect of a dominant variable and, if data are collected establishing reliable estimates for the input variable for the
marginally, these effects are masked. In [9] it is said that modelling approach is thought to be the main cause of the
‘‘the evaluation of uncertainty is neither a routine task nor a discrepancy and the empirical approach to uncertainty
purely mathematical one; it depends on detailed knowledge estimation was recognized to be generally the one pro-
of the nature of the measurand and of the measurement viding the more reliable estimates. If the response variable
method and procedure used. The quality and utility of the Y is measurable and the function that links it to the inde-
uncertainty quoted for the result of a measurement there- pendent variables is not known, the design of a suitable
fore it ultimately depends on the understanding, critical statistical experiment and the use of the correspondent
analysis, and integrity of those who contribute to the ANOVA table provide the best approach to estimate the
assignment of its value’’. In this sense DOE can represent a different sources of variation derived from a measurement
substantial support to know the nature of the measurand system.
and to understand what are the sources of uncertainties to But when Y is not measurable or there are input vari-
be taken into account to estimate the global uncertainties ables of type B, the GUM approach to obtain the expanded
(see, e.g. [21]). uncertainty should to be used choosing with care the value
Also in the case in which the measurement is not of k and being aware the assumption of independence
measurable but the function f in Eq. 2 and the input among factors may be relevant. GUM on one hand is
quantities (of type A) are known, DOE can help to collect apparently easier to be implemented and with the intro-
data in a way suitable to show the possible relations among duction of the type B variables it gives the users great
dependent variables. Consider the simple case: Y is the area flexibility to the choice of the kind of input variables, but,
of a rectangular (Y = L 9 W) and follow the GUM on the other hand, it gives them also the responsibility of
approach to estimate its standard deviation. We compute the exactness of the results. In our experience, even when Y
separately the standard uncertainties associated with L, W is measurable we have noticed a liberal approach towards
and combine the two standard uncertainties into one, using the implementation of the GUM especially in the definition
the approximation for the product of two variables. If we of Eq. 2 (often considered as a sum of input variables) and
adopt an appropriate DOE to collect data on L and W of the no preliminary check for the presence of correlation among
rectangular we may take into account also the influence variables is usually made. In this context DOE could help
that, for example, the environmental condition (tempera- practitioners in collecting data to measure uncertainties
ture, humidity,…) can simultaneously have on L and W and even using GUM procedure.
in this way we can better estimate the uncertainties of Y. Finally, even if in ISO standards the uncertainty due to
the manufacturing process is never taken into account in
the model as ‘‘measurement of check standards are used’’,
Conclusion in the practice not always the measurements are referred to
a check standard. In these cases it is necessary to introduce
Measurement uncertainty is a vital issue within analytical the factor ‘‘part’’ in the model and to consider also its
science. The question is: what is the most appropriate interaction with the other factors. The use of mixed effect
method to estimate the measurement uncertainty? Gener- model is then an appropriate solution that may help to
ally speaking we may compare a modelling method (GUM) consider jointly fixed and random factors in a single

123
166 Accred Qual Assur (2009) 14:159–166

factorial (crossed or nested) experiment to estimate all 10. Bich W, Cox MG, Harris PM (2006) Metrologia 43:161–166. doi:
uncertainties components (systematic and random). 10.1088/0026-1394/43/4/S01
11. Coleman S, Greenfield T, Stewardson D, Montgomery DC (2008)
Statistical practice in business and industry. Wiley, New York
Acknowledgment The authors wish to thank the reviewers for 12. Jurado JM, Alcàzar A (2005) Accred Qual Assur 10:373–381.
helpful comments. doi:10.1007/s00769-005-0005-8
13. Tosello G, Gava A, Hansen HN, Lucchetta G, Guarise M (2007)
In: Proceedings of ENBIS-DEINDE 2007 conference. Torino,
References Italy, 11–13 April 2007, pp 313–323
14. Kennet RS, Kennet DA (2008) Accred Qual Assur (to appear)
1. Burdick RK, Borror CM, Montgomery DC (2003) J Qual Technol 15. Montgomery DC (2001) Design and analysis of experiments.
35:342–354 Wiley, New York
2. ISO 5725 (1994) International Organization for Standardization 16. Pavese F (2007) Accred Qual Assur 12:525–534. doi:10.1007/
(ISO), Geneva, Switzerland s00769-007-0303-4
3. International Organization for Standardization (ISO) (1995) ISO 17. Ellison SLR, Holcombe DG, Burns M (2001) Analyst (Lond)
Guide to the Expression of Uncertainty in Measurement, Geneva, 126:199–210. doi:10.1039/b008099n
Switzerland 18. Burdick RK, Borror CM, Montgomery DC (2005) Design and
4. ISO/IEC 17025 (2005) International Organization for Standardi- analysis of gauge R&R studies. ASA-SIAM, Philadelphia
zation (ISO), Geneva, Switzerland 19. Thompson M (2008) Accred Qual Assur 13:479–482. doi:
5. ISO/TS 21748 (2004) International Organization for Standardi- 10.1007/s00769-008-0417-3
zation (ISO), Geneva, Switzerland 20. van der Veen AMH, Alink A (1998) Accred Qual Assur 3:20–26.
6. Arri E, Cabiati F, D’Emilio S, Gonella L (1995) Measurement doi:10.1007/s007690050177
16:51–57. doi:10.1016/0263-2241(95)00017-F 21. Feinberg M, Laurentie M (2006) Accred Qual Assur 11:3–9. doi:
7. ISO/TS 21749 (2005) International Organization for Standardi- 10.1007/s00769-005-0081-9
zation (ISO), Geneva, Switzerland 22. Ellison SLR, Barwick VJ (1998) Analyst (Lond) 123:1387–1392.
8. Turzeniecka D (1999) Metrologia 36:113–116. doi:10.1088/ doi:10.1039/a706946d
0026-1394/36/2/6 23. Lyn JA, Ramsey MH, Damant AP, Wood R (2007) Analyst
9. JCGM 101 (2008) Evaluation of measurement data—supplement (Lond) 132:1231–1237. doi:10.1039/b709539m
1 to the guide to the expression of uncertainty in measurement—
propagation of distributions using a Monte Carlo method

123

View publication stats