EURAMET procedures

and review criteria for CMCs

EURAMET Guide no. 8

Issued: 02-06-2008
Version: 1.0
EURAMET Guide n° 8
Issued: 02-06-2008
Version 1.0

EURAMET procedures and review criteria

for CMCs

This EURAMET Guidance Document specifies requirements and the procedures for reviewing
calibration and measurement capabilities (CMCs) declared by NMIs 1 being Member or Associate
of EURAMET under CIPM’s Mutual Recognition Arrangement of national measurement standards
and of calibration and measurement certificates issued by national metrology institutes (CIPM
MRA 2 ).

The measurement capabilities of each NMI are published after final approval in Appendix C of the
BIPM key comparison database (KCDB) 3 , maintained by the BIPM and publicly available on the
Web. In order for CMCs to be approved for publication in Appendix C, they must first be reviewed
and approved by the appropriate Technical Committee of EURAMET. Once this approval is ob-
tained, CMCs undergo an inter regional review, where TC/WGs 4 from other RMOs verify that the
JCRB Criteria for acceptance of data for Appendix C 5 have been followed, thus providing the tech-
nical confidence required for publication.

TC/WGs play a key role in this process as they conduct the intra regional and inter regional techni-
cal reviews and eventual approval of each CMC submission. This document specifies the require-
ments and explains the steps EURAMET TC Chairpersons need to follow in order to carry out this
process.

This edition is based on EUROMET Guide 8 version 2.2 (2002) and is a complete revision, taking
into account the latest JCRB documents for drawing up and reviewing CMCs and the new organ-
isational structure of EURAMET.

1
In this document the acronym NMI is used both in reference to National Metrology Institutes and/or further Designated
Institutes (see document CIPM2005-07 NMIs and other Designated Institutes for more details)
2
http://www.bipm.org/en/convention/mra/
3
http://www.bipm.org/kcdb
4
Groups of technical experts in each area are called "Technical Committees" in APMP, COOMET and EURAMET, and
"Working Groups" in SADCMET and SIM
5
http://www.bipm.org/utils/common/documents/jcrb/AppC_criteria.pdf

EURAMET Phone: +49 531 592 3001 The copyright of this guide is held by
Secretariat Fax: +49 531 592 3002 © EURAMET e.V. 2008
Bundesallee 100 secretariat@euramet.org
38116 Braunschweig, Germany www.euramet.org
1 General policy
Calibration and measurement capabilities to be recognised under the CIPM MRA consist of the
best measurement capabilities ordinarily available to the customer through the NMI’s services. In
addition, each one of these CMCs must be supported in range and uncertainty of measurement by
a fully implemented Quality Management System satisfying the MRA requirements and being ap-
proved and regularly reviewed by EURAMET TC-Quality as described in document TCQ_03_01
QMS Review Procedure 6 .
The CMCs are to be reviewed by the appropriate EURAMET Technical Committee under the respon-
sibility of the Chairperson, based on this guide and according to the procedure described in chapter 3.
Depending on the subject field, additional field specific requirements may apply.

2 Drawing up CMC Excel files

CMCs should normally be submitted by country, by metrology area, and where necessary by cate-
gory (i.e. TC-QM). NMIs which submit their first set of CMCs or a larger number of new CMCs in a
metrology area or a branch shall fill in a questionnaire (Appendix 1) which gives additional informa-
tion to the reviewer(s). Modifications of CMCs must only be made on the Excel files available from
the link "Get published CMCs", located in the summary box of the JCRB Website 7 . These files
have been produced by the BIPM KCDB office from the files posted for final approval and it has
been verified that they do not contain any formatting errors. Submitting NMIs can view this page
using the following information:
Username: guest
Password: guest2001
It is required that all CMC information should be submitted in a consistent format. Detailed instruc-
tions on how to draw up these files can be found in the JCRB web page 8 . A zip file with documen-
tation related to this topic can be downloaded from the JCRB web page for "Open access JCRB
documents" in the section "Related articles" by clicking on the link labelled "Instructions for drawing
up CMC Excel files". Additional advice on the preparation of Excel files can be given by the coordi-
nator of the KCDB.
NMIs that do not hold primary standards or primary measurement capabilities are required
to have traceability to the SI (or if not yet feasible to another internationally agreed reference) of
the national standards or measurement capabilities established through the BIPM or through
adequate calibration services of another NMI or other designated institute published in Ap-
pendix C of the CIPM MRA.

6
http://www.euramet.org/index.php?id=tc-quality-open
7
http://www.bipm.org/JCRBCMCs/
8
http://www.bipm.org/en/committees/jc/jcrb/cmc_excel_files.html

EURAMET Guide N° 8 -2-

Issued: 02-06-2008
Version: 1.0
3 CMC review criteria
The JCRB requires that the range and uncertainty of the CMCs submitted is consistent with infor-
mation from some or all of the following sources 9 :
1. Results of key and supplementary comparisons
2. Documented results of past CC, RMO or other comparisons (including bilateral)
3. Knowledge of technical activities by other NMIs, including publications
4. On-site peer assessment reports
5. Active participation in RMO projects
6. Other available knowledge and experience
Technical support for CMCs is achieved mainly through the results of comparisons. Key compari-
sons are designed to provide evidence on the proficiency of NMIs in the principal techniques in
each field. Supplementary comparisons are undertaken by RMOs when CMCs require additional
support not provided by key comparisons. While it is important that each CMC submission is sup-
ported by some comparison evidence and EURAMET NMIs are encouraged to participate in com-
parisons whenever they have the possibility to do so, it is not the intention of the MRA to have a
one-to-one correspondence between CMCs and comparisons.
While the results of comparisons are the ideal supporting evidence, all other sources may be con-
sidered to underpin CMCs not directly related to the available comparison results and those for
which comparison results are not yet available. The NMIs that issue the CMCs are primarily re-
sponsible for providing the information that they believe is necessary to support their claims. This is
preferably done by means of a questionnaire (Appendix 1) which serves as a tool to gain more
information about the submitted CMCs. The experts involved in the intra or inter regional review
may request additional information, if needed. Knowledge of the NMI's work and activities may
require on-site visits by peers 10 .

4 CMC review process

In order for CMCs to be approved for publication in Appendix C, they must first be reviewed and
approved by the appropriate Technical Committee within EURAMET (Intra regional review). Once
this approval is obtained, CMCs undergo an inter regional review, where the TC/WGs from other
RMOs verify that the review criteria have been followed, thus providing the technical confidence
required for publication.

9
Document JCRB-14/06(2a)_final
10
Document CIPM/07-25 ACCEPTED

EURAMET Guide N° 8 -3-

Issued: 02-06-2008
Version: 1.0
Intra regional reviews
NMIs through their contact persons forward CMCs to the relevant EURAMET TC Chair. These
CMCs must have been approved by the NMI's management and fully covered by its quality man-
agement system. Some TCs have installed a Working Group for CMC Review and set a fixed date
for submission (i.e., the annual TC meeting) and a fixed period for intra regional review. In this
case, the TC Chair collects the CMC sets and send them to the members of the working group for
review. Otherwise, the Contact Persons of the TC are directly involved in this process, and the
results of the review will be discussed at the coming TC meeting. The CMC review is performed by
direct contact between the reviewers and the persons responsible from the submitting NMIs until a
consensus can be reached. In case that not all concerns can be resolved, the TC Chair contacts
the reviewers and the submitting NMIs for resolution, the final decision being with the EURAMET
Chairperson. The results of the review together with possible comments of the reviewers and the
submitting NMIs are listed in the CMC tables. The intra regional review process is presented in
flowchart form in Appendix 2. The intra regional review must cover all submitted CMCs, sample
review will only be accepted for the inter regional review.
For a CMC entry, the statement of an uncertainty must be based on a careful consideration of all
uncertainty contributions of Type A and B following the “Guide to the expression of uncertainty in
measurement” (GUM). Results of key, supplementary and bilateral comparisons are important in-
formation providing evidence for validity of CMC claims. But uncertainties of CMC claims must be
equal or larger than the corresponding results of a key comparison, and reference to key, supple-
mentary or bilateral comparisons can only be used to support uncertainties that are larger or equal
than those stated for the corresponding measurement in the key comparison. A laboratory may,
however, wish to re-evaluate the uncertainties after a successful key comparison or because of
changes and improvements in the technical equipment, methods and procedures, staff education,
or other conditions. In this case a new uncertainty budget must be drawn up.

Inter regional reviews

The inter regional review procedure is explained in a JCRB- document 11 , which is also presented
in flowchart form 12 . The CMC review process is carried out through a password-protected web
page which can be entered via the JCRB section of the BIPM site 13 by clicking on the link labelled
"JCRB CMC site". TC Chairs can obtain a username and a password from the JCRB Executive
Secretary. The CMC files to be reviewed can be downloaded from this page. The JCRB requires
that CMCs submitted for publication in Appendix C are accompanied by an RMO report indicating
that the local TC/WG has approved the range and uncertainty of said CMCs and that each of them
is supported by a fully implemented Quality Management System reviewed and approved by the
local RMO. Once the files are posted for review, the corresponding technical contacts in each area
get in touch directly with each other to exchange comments or concerns.

11
JCRB-14/06(1) JCRB "Rules of Procedure for CMC entry into Appendix C"
12
JCRB-12/06a_rev "JCRB CMC review process"
13
http://www.bipm.org/JCRBCMCs/

EURAMET Guide N° 8 -4-

Issued: 02-06-2008
Version: 1.0
Fast track approval of revised CMCs
An NMI may request through its TC/WG Chair that a small group of already-published CMCs be
reviewed by the other TC/WG in the same area without the formality of posting the file in the web
page. This may occur, for example, when an upgrade of the laboratory facilities justifies an im-
provement of the declared scope of the published CMCs. In this case, all communications will be
done directly among the TC/WG Chairs. After they reach a consensus on the new scope of the
revised CMCs, these will be entered into Appendix C.

Modifying CMCs already published in Appendix C

Modifications of a published CMC usually arise for reasons falling into one of three categories:
a) material or editorial errors and improvements to the explanatory text
b) increase of the uncertainty or reduction in scope
c) change of the method of measurement or reduction of the uncertainty or increase in scope.
For modifications of categories a) and b), the intra- and inter-RMO reviews are unnecessary. The
NMI will send its proposal for change to the EURAMET TC Chair, who will contact the coordinator
of the BIPM database. If a NMI decides to withdraw CMCs from the BIPM database, the same pro-
cedure will be applied. Modifications under category c) should follow the full procedure of intra- and
inter-RMO review, as if they were new CMCs, or the fast track approval.

5 Impact of comparison results on CMCs

It must be ensured that CMC claims made by an NMI are consistent with the results obtained in
key and supplementary comparisons. A measurement of a key comparison is considered to be
discrepant when the degree of equivalence represented by the difference di = xi - xKCRV and the
expanded uncertainty U(di) = k·u(di) do not fulfil the condition |di| < U(di) 14 . The NMI making the
CMC claim has primary and principal responsibility, i.e., it must check the comparison results
against its CMC claims and state whether or not these claims are supported by the comparison
results. If not, it must take appropriate measures to remove this inconsistency.
Through its TCs, EURAMET must monitor the impact of key and supplementary comparison re-
sults on CMC claims for its member NMIs. This can best be done by means of a comparison ex-
ecutive report in which the participants of a comparison declare in writing, that they have checked
their CMC claims in the light of the comparison results. This comparison executive report is confi-
dential to the comparison participants, the TC Chair and the corresponding RMO working group
and is not part of the publicly available comparison report. Reasons for withdrawals or changes in
results must also be given in this report.

14
M.G. Cox, "The evaluation of key comparison data", Metrologia, vol. 39 (2002), pp. 589-595

EURAMET Guide N° 8 -5-

Issued: 02-06-2008
Version: 1.0
Appendix 1: Questionnaire for the Review of EURAMET CMCs*)

NMI/DI:

Person responsible:

Metrology area
Branch
Service
Sub-service:
(Consistent with items in KCDB) (Please specify area, branch and service to which the information reported below applies)

Participation in comparisons

CC or EUROMET KCs ?
(Please name comparison identifier)

Supplementary Comp. ?
(Please name comparison identifier)

Past comparisons ?
(CIPM, EUROMET/EURAMET or others, please specify)

Bilateral comparisons ?
(Please specify)

Technical activities

Measurement methods
(Brief description of the methods used)

Traceability of standards
(Name NMIs/DIs which provide traceability)

Written measurement in-

structions (Written procedures available? Language?)

Uncertainty budgets
(Are they already available? If yes, are they calculated following the ISO Guide to the
Expression of Uncertainty in Measurement?)

Key publications
(Please specify)

EURAMET Guide N° 8 -6-

Issued: 02-06-2008
Version: 1.0
Quality management system

Status of implementation of
QMS ? (ISO/IEC 17025 and/or ISO Guide 34 fully implemented?)

Self-declared or
accredited QM system ? (If accredited, please name accreditation body.)

Which CMCs are covered

by the accreditation ? (Are all CMCs covered by the accreditation? If not, please specify the corresponding
areas of measurement)

Which CMCs are covered

by self-declaration ?
(List here all areas of measurement which are not covered by accreditation)

Engagement in TC-Quality
(Participation in meetings and other activities)

QM system reviewed by
TC-Quality ? (Meeting when the QMS was presented, status of review)

Additional information

Participation in EURAMET
technical activities ? (Projects, meetings etc; please specify)

Visits of technical
experts ? (Please specify name and date of visit)

On-site visits by peers ?

(Please specify names and date)

Any other information ?

(Please specify, if relevant)

*)
This questionnaire is thought to be filled in as additional information for the reviewer, when a NMI submits
it's first CMC set or a larger number of new CMCs in a metrology area, branch or service.

EURAMET Guide N° 8 -7-

Issued: 02-06-2008
Version: 1.0
Appendix 2: Intra-regional review process

NMI

Draws up CMCs Establishes and

for its services maintains a QS

Forwards new QS QS
or revised CMCs documentation approval

TC Chair TC Quality

Informs through
TC-Q annual
Forwards report
CMC sets

Questions,
comments CMC Working Group
concerns
Submitting or
NMIs TC Contact Persons
TC meeting
Answers,
modified
CMCs
Reviewed
CMC sets

TC Chair

Questions, Submission
comments to Inter-RMO
concerns review

Submitting RMO Inter-RMO

NMIs CMC WG review

Answers,
modified
CMCs

Final
approval

Publication
in App. C

EURAMET Guide N° 8 -8-

Issued: 02-06-2008
Version: 1.0