Validation of Test methods

General principles and concepts

(In cooperation with EAL, European
cooperation for
Accreditation of Laboratories)

EL1545/96

VALIDATION OF TEST METHODS

GENERAL PRINCIPLES AND CONCEPTS

December 1996

FOREWORD
EAL and Eurolab have a Permanent Liaison Group (PLG), which is a
forum where EAL and Eurolab are discussing matters of mutual
interest. The PLG consists of five members from each organization.
This document has been prepared in the PLG and endorsed by both
organizations.
The document is intended to give general views on certain issues
related to the validation of test methods and should be seen as a
common understanding and position of EAL and Eurolab. In order to
define and describe the activities behind the concept Validation of test
methods more detailed guidance documents are needed. This
document should be seen as a basis for such guidance documents.

VALIDATION OF TEST METHODS

1. Introduction
The definition used for "validation" in the ISO standard 8402 is
"confirmation by examination and provision of objective evidence that
the particular requirements for a specific intended use are fulfilled".
This definition gives the impression of confined and well-defined
(exact) operations. Test methods are normally developed for an
intended application range. The reference to particular requirements
must in many cases be interpreted in a flexible way as the
requirements can be of general nature.
Both standardized and non-standardized test methods are covered.
They can be exact or associated with large uncertainties. Even novel
test methods will be considered. The validation of a test method
becomes in this context a way of demonstrating that the method is fit
for its intended purpose. The fitness for purpose includes an
assessment and a balancing of technological possibilities, risks and
costs.
There are very few papers in the open literature dealing with the
general principles of test method validation. On the other hand, a lot of
detailed descriptions of the validation of specific test methods are
available. A brief overview of the concepts, aims and procedures in
validation is given in this document.
2. General principles to be used in validation
In the validation process an estimate is made of the
representativeness, repeatability and
reproducibility of the test method. The
definitions are given in annex 1.
In the validation process the ultimate aim is to secure that the test
methods are good enough with respect to representativeness,
reproducibility and repeatability. How much effort should be spent on
validation must be decided on a case by case basis. If large economic
values as well as considerable health, safety and environmental issues
are involved, much more emphasis must be paid to the validation of
the test methods. The frequency of use of the test method should also
be considered when determining the extent of validation. The total
consequences of wrong results are of course larger for methods in
extensive use than for test methods used occasionally.
The validation of test methods covers to a large extent the uncertainty,
repeatability and reproducibility of the test method. As the factors
affecting the results and contributing most to the uncertainty change

5
from one technical sector to another or even from one test method to
another, a universal solution cannot be given. Guidance on the
expression of uncertainties can be found for example in the
international Guide to the expression of uncertainty in measurement
and EAL guidance document Expression of uncertainty in quantitative
testing.
Standardized test methods should be considered validated for their
intended application range and thus good enough for that purpose
although their repeatability and reproducibility are not known in detail.
The testing laboratory must, however, check that they apply the
method correctly. For non-standardized test methods it is up to the
testing laboratories to determine how far they go in defining the level
of repeatability and reproducibility.
To develop a representative test method, adequate knowledge is
required of the practical use of the test results and of the real service
conditions of the object of the test. Based on such knowledge, the
"representative" properties to be determined by the test may be
identified.
The factors affecting the test results and their uncertainty may be
grouped into three main categories:
Instrumental and technical factors
- sampling
- homogeneity
- test method
- equipment
Human factors
Environmental factors
- testing environment
Instrumental and technical factors are related to the
constructional and functional characteristics of the test and
measurement equipment, as well as to other technical operations
involved in the test (e.g. sampling, preparation of samples, test object
homogeneity). Their effect may be minimized and kept under control
by the following provisions:
- define the equipment as precisely as necessary
- provide a clear description of the test procedure as well as the
equipment operation
- establish procedures for operational control and calibration
- ensure where applicable traceability of measurements to the SI
units.

6
Whenever practical, the above provisions should be included in the
description of the test
method. References to internal procedures or
applicable standards should be included.
Human factors are related to the competence of the staff and may
be controlled through:
- education/basic knowledge
- on job training/practical experience.
The qualification required for the personnel employed for a given test
may be specified in
the test method or reference can be made to the
applicable internal procedures.
Environmental factors are associated to the environment where the
test is performed.
Among others the effect of the following parameters
must be assessed and properly
controlled:
- atmospheric conditions (temperature, pressure, humidity)
- pollution/contamination
- other environmental characteristics (e.g. EMC).
The effect of the above parameters should be described in the test
method or reference to other applicable documents should be made.
However, for new test methods this information is often not available.
In some cases the data base for method validation is so large that
statistical methods should be applied.
The validation process must consider the expected or required
uncertainty of the test results and their intended use.
Critical threshold values (e.g. in health and environment) cannot
generally be technically justified with a small uncertainty. However, if a
legal limit is set, there must be test methods suited for the purpose.
Reference is made to a recent ILAC Guide.
The required depth of the validation process depends also on the
maturity of the test method and the prevalence of its use. One can
distinguish between the following categories:
-

novel methods
methods used by several laboratories
modification of established methods
standardized methods.

The ways, in which the validation is performed in the different cases,

need not be clearly differentiated. If the fitness for purpose concept is
maintained, it is often possible to validate at reasonable cost but with a
higher degree of uncertainty.
The novel methods are first developed in one single laboratory, often
on the basis of a special request from a customer or on ideas created in

7
the laboratory. That customer cannot pay for a wide range validation
nor can the laboratory itself. The aim of the validation of test methods
must always be to demonstrate that the method is fit for the intended
purpose and that the results have an acceptable uncertainty. It is
important that the rules of validation of test methods do not prevent
the natural technological development from taking place. The
laboratory does not expect (although it does want) outside financial
help for validation of novel methods and in many cases tries to protect
its new development from going to its competitors or from becoming
generally available to all.
When a certain number of laboratories work in the same area,
cooperation and inter-laboratory comparisons can be arranged. The
coordination of such activities is an extra economic burden. In order to
speed up the process, external financing is needed.
The testing laboratories need to update their existing test methods.
The flexible scope of accreditation as agreed between EAL and Eurolab
was also intended to allow modifications to be made to accepted
(accreditation covered) test methods. This requires validation
procedures applicable to method modifications. It is up to the
laboratories to describe their procedures for validating modified test
methods.
The most thorough validation procedure is required for test method
standardization purposes. The work needed is considerable and covers
proficiency testing, the determination of factors affecting the
uncertainty, measuring range, etc. The financial burden cannot be laid
on the laboratories but on the standardization organizations.
Standardized test methods must be considered sufficiently validated
for their intended application ranges. If they are not, they should be
withdrawn.
The validation of test methods consists of two interrelated steps:
(i) suitability of the test to solve the problem (customer
needs)
(ii) demonstration of the technical capability of the test
method within the specified test range
i.e. measuring the right properties with a sufficiently reliable method.
The suitability or representativeness of a test method is in many cases
an attribute which is difficult to define especially for tests related to
product acceptance. The test methods must be such that the results
obtained correlate with the performance characteristics and
operational experience of the product.
3. Validation procedure

8
Both testing laboratories and accreditation bodies are looking for
procedures and guidelines for planning and controlling the test method
validation process.
However, the discussion above has clearly indicated that one single
procedure cannot be developed. Consequently, a palette of different
choices of validation techniques has to be developed. How detailed the
validation will be, depends on the circumstances (needs, costs,
possibilities, risks, etc.).
The validation of the test methods is, of course, of interest also to the
accreditation bodies. The principle to be applied should be that the
laboratory describes the way it is validating the test methods and the
accreditation body should make the judgement if the procedure used is
acceptable in that case. The different validation possibilities are built
up around
- utilization of calibration
- intercomparisons including the use of reference materials and
reference methods
- well qualified staff and their professional judgement
- simulation and modelling
- other approaches.
Method validation is often based on the combined use of validation
procedures. The validation used can be "direct" or comparative. The
selection of the validation procedures should also be justified on a
cost-benefit basis as long as the fitness-for-purpose is maintained.
Focusing the effort on the most critical factors affecting the test
method will lead to a different solution for the validation of "exact"
physical and chemical test methods as compared to that for product
or subjective testing. For example, in the validation of ergonomics and
sensory test methods not all possibilities are applicable.
As said above different validation procedures may be followed, their
effectiveness and applicability depending on the type of test
considered. They can be characterized as "scientific" or
"comparative":

"Scientific approach"
In the scientific approach the assessment of the representativeness,
repeatability and
reproducibility of the method is performed with reference
to the different constitutive
elements and features. Evidence should
describe the representativeness of the selected
properties and the
associated uncertainty. This can be based on information published in
the
scientific and technical literature or on ad hoc investigations performed by the
laboratory developing the method. The laboratory shall
demonstrate that relevant
influencing factors (instrumental and
technical, human, environmental) have been
analyzed and that they
are under control within the uncertainty associated with the
method.
"Comparative approach"
The test method is assessed by comparing its results to those
obtained by means of
another already validated test method, which has
been developed for the same purposes.
If this is not possible, the
performance characteristics of the method may be assessed
through
interlaboratory comparisons. The method is "valid" if the results obtained by the
different laboratories fall within the expected uncertainty limit.
Deviations beyond such
limits may indicate e.g. a lack of control of the
influencing parameters. The causes of this
behaviour should be clarified
and the method is to be redefined accordingly. The
interlaboratory
comparison does not always provide a comprehensive validation of the
representativeness of the method, which may be accurate and stable,
though physically
"wrong".
The acceptance procedure for new or modified test methods is either
(i) determined internally in the laboratory (ii) agreed upon between the
customer and the laboratory or (iii) accepted by the authorities and/or
accreditation bodies. A higher degree of reliance is needed when
safety, health and large economic values are involved. Calibration has
been emphasized as an important element in the method validation
process, but it is not necessarily the most dominating factor. The
understanding of the testing method with its systematic and random
errors is crucial. A scientific approach to analyze sources of error as
well as the competence of the personnel doing that job is of great
importance.
The laboratory should always describe the way the validation of test
methods is done and this description should be a part of the quality
system/manual when appropriate.
As simplified validation procedures ("fast" validation methods) must be
used in many cases, the capability to use professional judgement in
assessing whether the validation is comprehensive enough becomes
pronounced. However, even when talking about simplified or fast
validation procedures, the validation must be done with such a depth

10
that the method is fit for the intended use and acceptable to the
customer and/or authorities. It is clear that the definition of the use
and scope of the method and assumption of uncertainty should not be
misleading and too optimistic.
When the use of new test methods becomes more extensive, work
describing the effect of changes in test parameters can be initiated in
order to show the robustness of the method. Prenormative research
should also be initiated.
The need for new or improved test methods arises when we lack
methods or the existing ones are not complete, good or efficient
enough. There is no need for the laboratory community to develop new
methods if existing ones can be considered adequate.

11
ANNEX 1
DEFINITIONS
Repeatability (of results of measurement)
Closeness of the agreement between the results of successive
measurements of the same measurand carried out under the same
conditions of measurement. (VIM)
Notes
1.

These conditions are called repeatability conditions

2.

Repeatability conditions include:

- the same measurement procedure
- the same observer
- the same measuring instrument, used under the same
conditions
- the same location
- repetition over a short period of time

3.

Repeatability may be expressed quantitatively in terms of

the dispersion characteristics of the results.

Reproducibility (of results of measurements)

Closeness of the agreement between the results of measurements of
the same measurand carried out under changed conditions of
measurement. (VIM)
Notes
1.

A valid statement of reproducibility requires specification of

the conditions changes.

2.

The changes conditions may include

- principle of measurement
- method of measurement
- observer
- measuring instrument
- reference standard
- location
- conditions of use
- time.

3.

Reproducibility may be expressed quantitatively in terms of

the dispersion characteristics of the results.

4.

Results are here usually understood to be corrected results.

12

Uncertainty (of measurement)

Parameter, associated with the result of a measurement, that
characterizes the dispersion of the values that could reasonably be
attributed to the measurand. (BIPM/IEC/IFCC/ISO/IUPAC/IUPAP/OIML).
Notes
1.

The parameter may be, for example, a standard deviation

(or a given multiple of it), or the width of a confidence
interval.

2.

Uncertainty of measurement comprises, in general, many

components. Some of these components may be evaluated
from the statistical
distribution of the results of series of measurements and can
be
characterized by experimental standard deviations. The
other components, which can also be characterized by
standard deviations, are evaluated from assumed probability
distributions based on experience or other information.

3.

It is understood that the result of the measurement is the

best estimate of the value of the measurand, and that all
components of uncertainty, including those arising from
systematic effects, such as components associated with
corrections and reference standards, contribute to the
dispersion.

Validation
Confirmation by examination and provision of objective evidence that
the particular requirements for a specific intended use are fulfilled (ISO
8402).
Notes
1.

In design and development, validation concerns the process

of examining a product to determine conformity with user
needs.

2.

Validation is normally performed on the final product under

defined operating conditions. It may be necessary in earlier
stages.

3.

The term "validated" is used to designate the corresponding

status.

4.

Multiple validations may be carried out if there are different

intended uses.

13
Verification
Confirmation by examination and provision of objective evidence that
specified requirements have been fulfilled (ISO 8402).

Notes
1.

In design and development, verification concerns the

process of examining the result of a given activity to
determine conformity with the stated requirements for that
activity.

2.

The term "verified" is used to designate the corresponding

status.

14
ANNEX 2
REFERENCES AND BACKGROUND MATERIAL
ISO 5725 (1986): Precision of test methods - Determination of
repeatability and reproducibility for a standard test method by interlaboratory tests.
ISO 8402 (1994): Quality management and quality assurance vocabulary.
BIPM/IEC/IFCC/ISO/IUPAC/IUPAP/OIML (1995): Guide to the expression
of uncertainty in measurement.
EAL-G 23 (1996): Expression on uncertainty in quantitative testing.
EAL-G 12 (1995): Traceability of measuring and test equipment to
national standards.
Forstn Jarl (1991): A view on the assessment of the technical
competence of testing laboratories. Nordtest Technical Report 149,
46p.
ILAC (1996): Guide an assessment and reporting of compliance with
specification, based on measurements and tests in a laboratory.
VIM (1993): International vocabulary of basic and general terms in
metrology.

15

H:\tekstit1\validat\3.12.1996