Recommendations For Digital Transformation

REPRODUCIBILITY IN RESEARCH: THE ROLE OF MEASUREMENT
SCIENCE WORKSHOP
RECOMMENDATIONS FOR THE INFRASTRUCTURE
UNDERPINNING DIGITAL TRANSFORMATION
SUNDEEP BHANDARI
DANNI CROUCHER
APRIL 2020
 NPL Management Limited, 2020
National Physical Laboratory

Hampton Road, Teddington, Middlesex, TW11 0LW
Extracts from this report may be reproduced provided the source is acknowledged
and the extract is not taken out of context.
Approved on behalf of NPLML by

Neil Stansfield, Head of Digital Sector
Reproducibility in Research: The Role of Measurement Science Workshop
Recommendations for the infrastructure underpinning digital transformation
CONTENTS
1. EXECUTIVE SUMMARY ............................................................................................................3

2. BACKGROUND .............................................................................................................................5
2.1. OUR CONSULTATION ...............................................................................................................6
2.2. THE MEASUREMENT CHALLENGE........................................................................................6
3. RECOMMENDATIONS ...............................................................................................................8
3.1. A WORLD-CLASS DIGITAL INFRASTRUCTURE FOR THE UK ..........................................8
3.2. WHO SHOULD UNDERTAKE THIS ACTIVITY? ....................................................................9
4. BENEFITS ....................................................................................................................................10
5. WHY NPL .....................................................................................................................................12
5.1. WHAT IS THE NATIONAL PHYSICAL LABORATORY? .....................................................12
5.2. NPL’S DATA CAPABILITIES ...................................................................................................13
MORE INFORMATION .....................................................................................................................14
LIST OF ATTENDEES .......................................................................................................................14
CONTACT US ....................................................................................................................................14
2
1. EXECUTIVE SUMMARY
The Government has committed to building a digital infrastructure which will support and underpin
digitalisation as well as encompass digital standards and digital infrastructure technologies.
Measurement science needs to be at the heart of these standards and infrastructure, to ensure we
can have confidence in the system.

The UK needs dynamic data quality frameworks and indicators which can support future digital
traceability chains, including digital (machine readable) calibration certification and uncertainty
quantification.

This should be developed alongside the creation of an internationally accepted and standardised
infrastructure for provenance of data, digital calibration certificates and accepted ontologies for
machine readable methods.

Between 2018 and 2019, the National Physical Laboratory (NPL) undertook a consultation process to
discuss the infrastructure requirements to support the digital transformation of society. In 2018, NPL
hosted a conference with its international counterparts – other National Metrology Institutes
investigating the role of measurement science supporting reproducibility in research. The findings
from this conference were built upon through a further workshop focusing on confidence in the
intelligent and effective use of data, as well as and wider stakeholder engagement activity over the
course of 2019.

Outcomes of the first conference produced two recommendations:

There is a need for an internationally accepted and standardised infrastructure for the
1 provenance of data, digital calibration certificates and accepted ontologies for machine
readable methods. These should be embedded within international standards and
created through cross‐disciplinary and cross‐sectoral collaboration.
As digital manufacturing, such as Industry 4.0, and artificial intelligence develop rapidly,
2 these require machine readable methods and protocols as well as the transfer of
digital calibration certificates.

The following workshop and stakeholder engagement provided a further four recommendations to
help deliver data quality frameworks and digital infrastructure:

Promote the use of FAIR data principles: (findable, accessible, interoperable and re‐
usable)
3 Enable all stakeholders in a digital ecosystem to maximise the value extracted from
research data through the provision of a set of guidance to manage, curate and
steward data (Wilkson, 2016)
Develop publicly disseminatable implementations and standards for Data Quality
(and data model) Frameworks:
4 Data Quality Frameworks will ensure that decisions are driven by data and digital
process’ that are fit‐for‐purpose, supporting the digitalisation of infrastructure
technologies and digital traceability chains.

3
In order to deliver the UK’s infrastructure to underpin digital transformation it has been proposed
that:

5 NPL take the lead on the development of data quality frameworks alongside
colleagues from the UK’s Quality Infrastructure.
The role of NPL as the UK’s National Metrology Institute alongside Designated
Institutes that deliver the National Measurement System should be expanded to
6 support the development of the digital infrastructure which will enable the UK to
benefit from digital technology, data‐driven decision making and innovation, to provide
confidence in the intelligent, safe and effective use of data and artificial intelligence
(AI).

Through the broadening of its’ activities in digital and data science, NPL can establish the foundations
upon which the UK can capitalise upon ‘confidence in the intelligent and effective use of data’, a
cornerstone of global growth for the UK.

4
2. BACKGROUND
The UK’s technology environment is developing rapidly, stimulating and catalysing wide ranging
technical and societal digital transformation. This transformation is pervasive and impactful,
resulting in business and society becoming increasingly dependent on digital enablement to drive
efficiencies, innovate, and increase the quality of life for our citizens.

Digital developments have the potential to create significant benefits for society, through underlying
process and product innovation, as well as through cultural advancement. These benefits are well
acknowledged; the Department for Digital, Culture, Media and Sport’s 2017 UK Digital Strategy1
encourages the harnessing and exploitation of digital and data infrastructure and technologies,
operating across different platforms, to enable successful digital transformation. The Digital Strategy
identifies seven strands which are designed to “create an economy which is resilient to change and fit
for the future”.

The priorities centralise around the development and security of the digital infrastructure; and
ensuring that digitalisation is exploited to its full potential. The strands include:

Building a world‐class digital infrastructure for the UK
Maintaining the UK government as a world leader in serving its citizens online
Unlocking the power of data in the UK economy and improving public
confidence in its use

Helping every British business become digital

Making the UK the safest place in the world to live and work online
The UK government understands the benefits that digitalisation can create for the country’s
economy and society. It has further committed to achieving a national investment of 2.4% of GDP in
research and development (R&D) by 2027, a pledge which has the potential to bolster the process of
technological innovation and push digital transformation forward faster.

As digital transformation progresses, business and society become increasingly dependent on digital
enablement. Consequently, it is vital that a reliable quality infrastructure is in place to support
digitalisation and the generation of robust, accurate and reliable data.
1 https://www.gov.uk/government/publications/uk‐digital‐strategy
5
2.1. OUR CONSULTATION
In May 2018, NPL hosted an international conference, Reproducibility in Research: The Role of
Measurement Science, in association with the International Bureau of Weights and Measures2 (BIPM)
and partners from National Metrology Institutes (NMIs) around the world. This was designed to
enable the leading metrology institutes to collectively identify how measurement can support the
process of digital transformation and ensure future technologies conform to international standards.

The event cumulated in two co‐created recommendations for the improvement of reproducibility of
research and to improve confidence in the research data which underpins the development of digital
transformation, enabling this to develop efficiently and safely:

There is a need for an internationally accepted and standardised infrastructure for the
1 provenance of data, digital calibration certificates and accepted ontologies for machine
readable methods. These should be embedded within international standards and
created through cross‐disciplinary and cross‐sectoral collaboration.
As digital manufacturing, such as Industry 4.0, and artificial intelligence develop rapidly,

2 these require machine readable methods and protocols as well as the transfer of digital
calibration certificates.
To build on these recommendations and further develop them for the UK, NPL hosted a data
capability workshop on the 3rd October 2019: Confidence in the intelligent and effective use of data –
a new infrastructure technology for the UK. This event consisted of 30 experts from industry,
academia and the government, who gathered to identify and discuss what stakeholders expect and
want NPL, as the UK’s National Measurement Institute, to be doing to support UK industry in this
space.

This report details the recommendations which were formed at the international conference and
further refined at the subsequent data capability workshop. It details the importance of NPL in its
role as an NMI to enable and support digital transformation in the UK.

2.2. THE MEASUREMENT CHALLENGE
Digitalisation, or the digital revolution, is progressing rapidly in the UK, creating large scale
dependence on digital systems. However, the necessary underpinning quality infrastructure for these
developing systems is not in place. Alike to the way in which physical infrastructures support physical
assets, digital infrastructures are needed to provide support and protection to our data, digital assets
and systems.

It is widely agreed that this quality infrastructure needs to be developed and implemented to
accommodate this rapidly evolving paradigm and prevent digital technology being negatively
perceived. However, at present there are no standards for data collection, transmission, processing,
storage, curation, analysis and ultimately confident, high value, decision making.

Whilst the government’s Digital Transformation strands and 2.4% R&D investment targets have the
potential to ensure that effective and secure digitalisation occurs throughout the UK at a fast pace, it
is recognised that these targets require data certainty to enable capabilities and infrastructures to
work effectively and safely.
2 https://www.bipm.org/en/about‐us/
6

The workshops hosted at NPL, as well as further discussions with stakeholders and across sectors,
have highlighted that there is a lack of understanding of the importance of the Data Life Cycle.

Figure 1 - The Data Lifecyle

Data Quality Frameworks are key to the whole Data Life Cycle. Data capture is an early step in the life
cycle that can introduce uncertainties which then propagate through the whole cycle, thereby
reducing the maximum potential value of the data being extracted to create information. This is
further compounded by the emergence of Artificial Intelligence where traceability and uncertainties
are not understood and therefore cannot be used with confidence for high value or business/safety
critical decision making in autonomous systems and networks.

Measuring the accuracy and precision of data, the provenance of data, and the propagation of
uncertainty through data processing and curation process’ is ultimately required to provide the
infrastructure technology that will underpin acceleration of digital transformation across society and
industry for the greater good of our citizens and economy.
The technology behind innovative developments and digitalisation must be rooted in reliable and
repeatable evidence‐based science to ensure confidence. This requires world‐leading measurement
science, or metrology, which enables the creation, capture and use of valid data, based on high‐
precision measurements which are reliable and repeatable. Ensuring measurement certainty will lead
to an efficient infrastructure, which promotes economic and societal stability and prosperity.
7
3. RECOMMENDATIONS
The Government has committed to building a digital infrastructure which will support and underpin
digitalisation, this will encompass digital standards and digital infrastructure technologies. The
workshops hosted at NPL have highlighted the need of measurement science at the heart of these
standards and infrastructure, to ensure we can have confidence in the system.

The UK needs dynamic data quality frameworks and indicators that can support future digital
traceability chains, including digital (machine readable) calibration certification and uncertainty
quantification.

The workshops culminated in the recommendation that this should be developed alongside the
creation of an internationally accepted and standardised infrastructure for provenance of data,
digital calibration certificates and accepted ontologies for machine readable methods. These
standards will largely support the development of AI, as they will both ensure the data used by, and
to train, AI systems, is fit for purpose.

3.1. A WORLD-CLASS DIGITAL INFRASTRUCTURE FOR THE UK
The following details the recommendations from the Data Capability Workshop: Confidence in the
intelligent and effective use of data – a new infrastructure technology for the UK, made by
independent experts across industry on how the UK can begin to develop the digital infrastructure
and data quality frameworks:

3 Promote the use of FAIR data principles:
Findable, Accessible, Interoperable and Re‐usable Data
Develop publicly disseminatable implementations and standards for Data Quality
4 (and data model) Frameworks:
Data Quality Frameworks will ensure that decisions are driven by data and digital
process’ that are fit‐for‐purpose.

Recommendation 3: Promote the use of FAIR data principles

FAIR data principles (findable, accessible, interoperable and re‐usable) enable all stakeholders in a
digital ecosystem (from producers to consumers of data) to maximise the value extracted from
research data through the provision of a set of guidance to manage, curate and steward data
(Wilkinson 2016).
Findable With the use of meta‐data
Accessible With the use of machine‐readable data
Interoperable With the use of graph databases and standardise ontologies
Re‐usable With the use of machine learning and AI

As the UK’s National Metrology Institute (NMI), NPL has already undertaken significant work in this
area. The Data Science team are already working to implement data management for the lab’s own
work. It is proposed that NPL become a role model for the FAIR principles, providing public access to
data and methodologies to record data provenance.
8
Recommendation 4: Develop publicly disseminatable implementations and standards for Data
Quality (and data model) Frameworks

Developing publicly disseminatable implementations and standards for Data Quality (and data
model) Frameworks, will enable the movement towards full digital data life cycles. Data Quality
Frameworks will ensure that decisions are driven by data and digital process’ that are fit‐for‐purpose,
supporting the digitalisation of infrastructure technologies and digital traceability chains through:

 Traceability/Provenance
 Well‐defined uncertainty within budgets
 Dynamic calibration
 Dynamic uncertainty calculations
 Persistent indicators
 Trusted algorithms & AI/ML
This will ensure the internationally accepted and standardised infrastructure for provenance of data,
digital calibration certificates and accepted ontologies for machine readable methods are available. It
will result in a defined framework for uncertainty, reliability, and provenance for AI and machine
learning, and establish measurement standards and best practices for research areas that must deal
with large numbers of hidden variables, sparsely sampled data. Lastly, it will define a framework for
uncertainty, reliability, and provenance for digital twins.

3.2. WHO SHOULD UNDERTAKE THIS ACTIVITY?
Through the workshops and stakeholder consultation, the community proposed that there was a
wider role for NPL as the UK’s National Metrology Institute to take. The recommendations were
that:

NPL take the lead on the development of data quality frameworks alongside
5 colleagues from the UK’s Quality Infrastructure (British Standards Institute, UK
Accreditation Service and Office for Product Safety and Standards), as well as world
leading UK academics and industry, to develop these frameworks.
The role of NPL as the UK’s National Metrology Institute alongside Designated
Institutes that deliver the National Measurement System should be expanded to
6 support the development of the digital infrastructure which will enable the UK to
benefit from digital technology, data‐driven decision making and innovation, to provide
confidence in the intelligent, safe and effective use of data and artificial intelligence
(AI).

Together this will in turn both unlock of the power of data in the UK economy and improve public
confidence in its use, by providing a quality‐assured digital infrastructure, creating significant
benefits which align to the digital strategy.
9
4. BENEFITS
By enabling the NMI and DI’s to support the development of this infrastructure, a number of benefits
will occur.

The infra‐technologies which support the data economy are numerous and complex. They are
underpinned and reconciled by data science. If the role of NPL and the DIs was expanded to enable
the establishment of a measurement science infra‐technology, the presence of measurement
certainty and support will boost digitalisation by accelerating development, deployment and
adoption of all things digital.
Figure 2 - Flow of data infra-technologies

The benefits of successful strategic execution could unlock significant addressable markets. For
example:
 By 2025, the global market for telecoms software development is estimated to be in the region
of £290 billion.
 IHS forecasts global Internet of Things (IoT) market growth from 15.4 bn devices to 30.7 bn
(2020) and 75.4 bn (2025).
 The SMMT predicts that the advent of connected and autonomous vehicles (CAV) could
provide the UK with a £62bn (US$81bn) boost by 2030, with over 420,000 new jobs created
and more than 47,000 serious accidents prevented.
This new area of work is applicable across all areas covered within the Industrial Strategy, such as
Healthcare & Lifesciences, Future Communications Networks, Industry 4.0, and Intelligent mobility.
It will establish foundations for best practice and support policy and regulation decisions across Data
science‐based applications.

Lastly, effective data management, and the ability to understand and measure the level of
confidence in data, will act as a catalyst to enable the successful deployment of technologies such as
automation, Artificial Intelligence and the Internet of Things. By enabling this development, the UK
can lead digital development on the international stage, supporting the Digital Strategy’s aim of
capturing and cementing international leadership.

Successful implementation will confer international scientific and technological leadership in R&D
and associated competitive advantage upon us; creating the infrastructure to leverage existing, and
develop new, world leading skills; as well as stimulating a rise in new industries and employment
opportunities.

10
Case study: Autonomous systems
The use of autonomous systems is rapidly growing: artificial intelligence (AI)
and machine learning (ML) systems are increasingly being used in industries
that demand greater and greater levels of reliability.
The problem These autonomous systems require validation that they are safe. Without
this, their development and production will either create a growing risk to

human life, or will slow down the deployment of such systems and their
accompanying benefits.

NPL has been developing a strategy to address the challenges of reliability and
validation.
Before even considering the explainability of an (or a set of) AI algorithm(s),
The proposal the AI systems require access to trusted and reliable data for both training
in action purposes and during operation. Such data needs to meet acceptable quality
and integrity standards, be traceable (such as having a known provenance),

and have well characterised uncertainty, to ensure they are safe and reliable
for operation.
11
5. WHY NPL
5.1. WHAT IS THE NATIONAL PHYSICAL LABORATORY?
NPL is a government owned and funded national laboratory that specialises in measurement science,
and is the UK’s National Measurement Institute (NMI). As the UK’s NMI, NPL is part of the UK’s
National Measurement System (NMS). The NMS consists of a core infrastructure of measurement
laboratories and a wider community of service providers that ensure you can have confidence in the
measurements you make or are made on your behalf.

At its core, NPL maintains the UK’s units of measurement. It works with the NMS labs to ensure that
measurement in the UK is consistent with the global common system of measurement units, The
International System of Units, or the SI (Système International d'Unités). Each country has one
National Measurement Institute (NMI), whose role it is to take the lead in international
representation and to underpin delivery of a measurement infrastructure consistent with the SI
system. NPL delivers this for the UK, and works in partnership with a number of other designated
institutes to maintain and strengthen the national measurement infrastructure.

Alongside this core role, NPL conducts research into fundamental metrology and develops primary
standards, as well as new instrumentation. It further works in partnership with a number of grant
providers, such as Innovate UK, to offer grant funded collaborative R&D projects, to provide
measurement support to a mixture of firms and research organisations. Lastly, NPL undertakes a
number of commercial activities, such as supplying commercial calibration, testing, and training
services to firms, hospitals, and universities.

NPL at a NPL has 820 scientific and technical staff and 200 PhD
glance researchers.
It has a turnover of around £100m with £56.8m in annual
NMS funding.
It generates £13m of revenue from sales of measurement
services.
NPL publishes around 350 articles in peer reviewed
journals each year
It partners with or otherwise provides services to around
500 UK-based firms each year.

12
5.2. NPL’S DATA CAPABILITIES
The support that is necessary for digitalisation is rooted in measurement science. Measurement
values, data, algorithms, mathematical and statistical procedures, as well as communication and
security architectures are key enablers of true digital transformation. These are core capabilities of
NPL, acting as the nation’s NMI. NPL is able to use these in developing and validating measurement
procedures that ensures the highest precision, integrity and explainability of algorithms and data
analysis methods, ultimately traceable back to the International System of Units (SI).

NPL undertakes and leads cutting‐edge work in data science and the laboratory provides support
across sectors. For example, it supports the health sector and life sciences by undertaking data
curation; allowing datasets to be linked, improving the measurement capabilities of medical imaging,
and supporting the sector to become data driven. It supports advanced manufacturing by developing
ML in the agri‐tech sector, digitally enabling supply chains and developing predictive maintenance.

As the UK’s National Metrology Institute, it is incumbent upon NPL to rise to the emerging challenges
21st century society and digital transformation brings, and transform itself in order to fulfil its role
within government.
To support the development of data quality frameworks and digital infrastructure NPL is
committed to undertake the following:

Support activities around standards…
Develop a comprehensive directory of relevant standards with their
purpose/scope and ensure they incorporate proper metrology.
Develop best practice guidelines in trustworthy AI and Data Quality
Frameworks.
Identify gaps in standards, and where possible consider extending or adapting
existing standards.
Encourage the adoption of standards, where appropriate.

Encourage broader use of reference materials and reference data, to improve
research reproducibility and confidence in measurement.

Develop and transfer skills and knowledge…
Lead by example, demonstrating best measurement practices internally and
sharing these with the broader research and metrology community.
Support universities to embed training in the principles of data metrology into
undergraduate and postgraduate courses.
Lead specific inter‐comparisons studies and proficiency exercises aimed at the
same measurand; and to encourage participation from industry and
academia.

Aim to have data from all measurements to be openly available within digital
calibration certificates.
Support and help adoption of AI into traditional industries, i.e. manufacturing.
Develop skills across NPL to deliver the Digital SI.
Investigate increasing automation of data acquisition to minimise potential
for human error.
Aim for automatic capture of the research process (workflow), through to
publication of machine‐actionable research articles.

13
MORE INFORMATION
LIST OF ATTENDEES
May 2018
NPL, NIST, PTB, LGC KRISS, NIBSC and the BIPM brought together experts from the measurement and
wider research communities (physical scientists, data and life scientists, engineers and geologists) to
understand the issues and to explore how good measurement practice and principles can foster
confidence in research findings.
The workshop involved 63 participants; 38 of which were from metrology laboratories, 16 from
academia, 5 from industry, 2 from funding agencies and two publishers. The participants came from
UK, US, Korea, France, Germany, Australia, Bosnia and Herzegovina, Canada, Turkey, and Singapore.
Read the conference findings here.

Confidence in the intelligent and effective use of data: A new infrastructure technology for the UK.
October 2019
This event consisted of 30 experts from industry, academia and the government, who gathered to
identify and discuss what stakeholders expect and want NPL, as the UK’s National Measurement
Institute, to be doing to support UK industry in this space.
Attendees include representatives from:
Department of Business,
Energy and Industrial IBM University of Surrey
Strategy (BEIS)
The National Physical
BT Group University of Southampton
Laboratory (NPL)
Defence, Science and
GKN Digital Institute, Newcastle University
Technology Laboratory (dstl)
Met Office Rolls‐Royce University of Edinburgh
British Computer Science
Logica CGI University College London (UCL)
Institute (BCS)
Heathrow Airport GlaxoSmithKline (GSK) University of Strathclyde
CONTACT US
For more information, contact us:
Head of Digital Sector: Neil Stansfield
Strategy Manager, Digital Sector: Sundeep Bhandari, sundeep.bhandari@npl.co.uk
Policy and analysis: Danni croucher, danni.croucher@npl.co.uk

National Physical Laboratory
Hampton Road, Teddington, Middlesex, TW11 0LW
Web: https://www.npl.co.uk/digital
Tel: 020 8977 3222
14

Recommendations For Digital Transformation

Uploaded by

Copyright:

Available Formats

Recommendations For Digital Transformation

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Recommendations For Digital Transformation

Uploaded by

Copyright:

Available Formats

REPRODUCIBILITY IN RESEARCH: THE ROLE OF MEASUREMENT

National Physical Laboratory

Approved on behalf of NPLML by

1. EXECUTIVE SUMMARY ............................................................................................................3

2.1. OUR CONSULTATION

Figure 2 - Flow of data infra-technologies

5.1. WHAT IS THE NATIONAL PHYSICAL LABORATORY?

5.2. NPL’S DATA CAPABILITIES

You might also like