A SPECIAL ADVERTISING SECTION

A sponsored
whitepaper.

Data Integrity in
Pharmaceutical Quality
Control Laboratories:
What You Need to Know
By: Loren Smith, Agilent Technologies

Introduction

Loren Smith
Agilent Technologies

Data integrity problems in pharmaceutical quality control laboratories are driving more regulatory
action than ever before. It is obvious that something has changed to drive all this activity. While
there is plenty of information available, much of it seems to confuse or frustrate rather than clarify
or help. This article will provide clarity to (hopefully) dispel common myths by looking at facts
based on a study of available resources and direct interactions with US Food and Drug Administration (FDA) staff and their consultants. It will discuss how:
to put the current enforcement environment into historical context
to apply critical thinking skills to various myths regarding data integrity
to evaluate current laboratory software and associated
processes against these new expectations
vendors are redesigning laboratory software to help respond to these new realities

Definition and History of Data Integrity and FDA Compliance

In a recent article in Scientific Computing (September 2013), Bob McDowall defined data integrity as:
generating, transforming, maintaining, assuring the accuracy, completeness, and consistency of data
over its entire life cycle in compliance with the applicable regulations. While it is a common myth that
the applicable regulations are new, 21 CFR Part 11 Electronic Records; Electronic Signatures was
first released in 1997. In 2003, after the pharmaceutical industry spent a few years struggling with
the regulation, and after discussions between industry and the FDA, the FDA released their Scope
and Application guidance document, clarifying some of the requirements in Part 11. This guidance
also included a specific discussion regarding the FDAs selective enforcement strategy based on
what they were finding during their inspections. In 2010, the FDA announced their data integrity
inspection focus. At that time, there were very few people within the Agency that were qualified to
understand data integrity aspects of computerized systems. The result was a real conundrum; it
was difficult to have a focus on data integrity inspections without enough skilled people to do the
inspections. Consequently, the Agency started a significant training exercise in data integrity, data
integrity inspections, and regulations for personnel that included chemistry experts, manufacturing
experts, and people with other GxP expertise. They also did some very strategic hiring of experts in
fraud investigation, including people from the US Federal Bureau of Investigation. Thus, beginning
in 2013, data integrity has been a primary inspection point, and there has been a visible increase in
data integrity enforcement activity, not only in India or China, but across all geographies. In addition,
starting in 2014, as a result of these inspections, the FDA often includes names of hardware and
software products and systems in their warning letters and related public information documents
in a less than subtle message to the hardware and software manufacturers that the Agency has
begun expecting them (the hardware and software manufacturers) to provide their customers help
and assistance with data integrity and compliance concerns.

SPONSORED BY

A SPECIAL ADVERTISING SECTION

DATA INTEGRITY IN PHARMACEUTICAL QUALITY CONTROL LABORATORIES

Mythbusting
Another myth commonly heard is that if a pharmaceutical company is using a particular software system, the FDA will shut
them down. The question takes the form of,
Systems throughout manufacturing organizations and laboratories may have potential weaknesses that could be considered a data integrity risk. If that weakness has not been
exploited, does the FDA have grounds for delivering an observation in a 483 warning letter?
The very clear answer is no. Capability does not equal violation (this is a contentious point worth further discussion). The
analogy is that, while the speed limit throughout much of the
USA is 65 miles an hour, the car I drive has the ability to exceed
130 miles an hour. However, officers must observe a speeding
infraction, via direct witness or electronic means, in order for
them to issue a citation for violating the speed limit. My cars
ability to exceed 65 miles per hour does not justify a citation.
Similarly, in a pharmaceutical company, if potential data integrity
weaknesses have not been exploited for data manipulation or
deletion of data (or other fraudulent activities), then there is no
basis for the agency to issue a citation, 483, or warning letter.
One should expect a detailed conversation with the inspector,
as the focus will turn to procedural controls to ensure that the
known weaknesses are not exploited.

Audit trails within computerized systems are an example of

technical controls. The software must have the ability to generate audit trails that contain all the components the regulations
require, and then those controls need to be enabled. Audit trail
records automatically (21 CFR Part 11.10(e)) show that that system
is working and its doing its job properly, and can be consulted in
an audit or an inspection without any extra work by the humans.

Certificates of Software Validation or Capability:

Do They Provide Value?

Many software vendors provide a Certificate of Software Validation, or they may issue a certificate that is titled along the lines
of 21 CFR Part 11 Readiness Claims. The reality is that such a
certificate that you might receive from your software vendor is of
extremely limited value, given the fact that the FDA expects the
software to be validated for its intended use by the users in the
environment where it will be used. While vendors should engage
customers in order to build and design systems according to
customers needs, and usually spend considerable time testing
that software before they deliver it, the development and testing
work does not (and cannot) substitute for the customer declaring
their intended use and then validating their system according
to that intended use.
Another related, interesting point is that the US FDA does not
have legal jurisdiction over a vendors informatics software organization. Therefore, (unless the software is registered as a Medical
Procedural versus Technical Controls
Device
with the FDA) any documentation that they might provide
Lets consider the statement: This software is Part 11 compliant. There are a few problems with this statement. First of all it is cannot be recognized by the FDA because of that lack of ena logical impossibility. There are components of Part 11 that are forcement authority. In addition, vendors may be able to provide
not meant to be satisfied by technical controls within a comput- detailed information about their products abilities relative to Part
erized system. For example, CFR Part 11.10(j) requires policies 11, however, that information and the software solution that they
for the use of electronic signatures. This is a requirement that a provide is based on the vendors interpretation of the regulations.
chromatography data system is not going to satisfy. It is an ele- This interpretation may or may not agree with various pharmaceument of the regulation, but it is not something that is expected tical manufacturers, or the FDA itself. The vendors interpretation
to be a technical control. The software, in fact, does not comply cannot substitute for an audit to determine the softwares funcwith regulations. The software itself is inert; software contains tional ability to satisfy regulatory concerns.
the technical controls to support compliance with the applicable
regulations. In addition to technical controls, procedural controls Your Responsibilities: Audit, Assess, and Validate
must also be in place. A discussion about procedural controls Data integrity compliance responsibilities include vendor audits,
versus technical controls is seen quite often in FDA Warning a computer system assessment, and software validation.
Letters, particularly when there are exploitations of gaps in a
When auditing vendors, there is a fourfold dilemma, or a set
systems ability to support technical controls required by vari- of problems that are actually focused on the auditors themselves
ous regulations.
(See Mourrain, J., Therapeutic Innovation and Regulatory SciA standard operating procedure (SOP), used as a procedural ence, Volume 40 (#2), pp. 177-183.).
control, can substitute for a technical control as long as:
The first dilemma is disparity. Regulations are few. The Part
the procedural control/SOP exists,
11 regulation, for example, is a grand total of three pages, not
people are trained on that SOP,
counting the information in the Preamble section of the 1997
the SOP is followed, and
Federal Register entry. Guidelines are many, and interpretations
adherence to the SOP is confirmed by quality
of the guidelines are even more plentiful. The disparity of interoversight and/or compliance auditing activities.
pretation is the problem in many auditing situations. For example,
Very often, however, even if SOPs exist, they are not followed, a customer during an audit might see a regulation a certain way,
and adherence isnt properly verified. Consequently, the FDA will whereas the vendor may look at the regulation differently.
demand system remediation in order to prevent a recurrence
Partiality is also an issue with auditors. Audit reports are parof the behavior.
tial, meaning incomplete. There are certain auditors that feel they

A SPECIAL ADVERTISING SECTION

DATA INTEGRITY IN PHARMACEUTICAL QUALITY CONTROL LABORATORIES

A SPECIAL ADVERTISING SECTION

need to have a large number of audit findings. As a result, they will parative evaluations between multiple software or service venhave separate findings for every problem in every SOP, and they dors. The point of all of this activity is that the vendor audit can
will list all of them separately, creating the potential misconcep- contribute to a risk-based validation strategy (a la FDAs General
tion that the audit report is good simply based on volume. Other Principles of Software Validation, Section 6.3): the better job a
auditors may take examples and put them into a very few large vendor is doing, (theoretically) the less work required during
observations in order to support their case for a major finding in software validation.
an audit. It is possible that neither of these types of reports will tell
the whole story of what was learned during the audit.
Assessing the Softwares (and the Vendors)
Another auditor issue is variability. Auditors, to no ones Compliance Support
surprise, are human. Some auditors will obsess over certain Assessing a softwares compliance support ability requires
subjects such as disaster recovery, while others are consumed attention to all of the regional regulations where a regulated
with Part 11.
company does business or may intend to do business. Some
Legibility (not in the handwriting sense) is next (but perhaps regulated companies, if they are solely doing business within the
not last). The auditor may be able to communicate the issues US, or solely within Europe, may choose to pay attention only
that they found during an audit, however, they are often chal- to Part 11, or only Annex 11 requirements. Part 11 and Annex
lenged to communicate the so what?; the computerized sys- 11 share commonalities. However, any software evaluation for
tem impact on patients, to the product, and to data integrity.
compliance should be based on evidence rather than hearsay
What is the solution to these audit dilemmas? The solution is a (that shiny Part 11 Certificate).
model, rather than a checklist, the components of which include:
Agilent, like many other vendors, receives numerous cus procedures,
tomer checklists, and provides a straightforward response with
training of personnel,
product answers. However, it is important that the evaluation of
software development activities,
the responses be based on evidence, rather than being strictly
testing activities,
limited to what the vendor has said the system will do. Areas
quality management systems, and
for review should include data integrity issues, access controls,
infrastructure.
audit trails, device checks, etc., as per applicable regulations.
The latter is often irrelevant unless the vendor takes custody This review is valuable during a software assessment process
of the GxP records in, for example, a cloud-based application. because observed gaps indicate where procedural controls or
Using a model approach, scoring can make what is a fairly customization may be required. Feedback to the vendor regardsubjective process into an objective measurement system that ing any gaps observed is therefore important so that the procesupports a defensible individual vendor audit, as well as com- dural controls or custom solutions do not become permanent.
Figure 1: How Agilent is responding: system design

Prioritize technical controls over procedural controls

Prioritize prevention controls over detection controls

Prioritize on-line records over hard-copy printouts

A SPECIAL ADVERTISING SECTION

A SPECIAL ADVERTISING SECTION

DATA INTEGRITY IN PHARMACEUTICAL QUALITY CONTROL LABORATORIES

Another common software compliance support myth is, I

bought the vendors IQ/OQ package implying that, therefore
my system is validated. However, buying the vendors validation
package (traditionally limited to a basic software IQ and a core,
generic OQ) is generally insufficient to establish validation of the
system for its intended use. Validation responsibility cannot be
abdicated to vendors, but they are (or should be) a reliable source
of information and assistance for your validation effort.
A corresponding validation myth that is, Any system change
requires full revalidation. The reality is that whenever software
is changed, the software change needs to be evaluated to
determine the extent and impact on the change on the entire
software system, and the risk of that change to patient safety,
(drug) product quality, or data integrity. Very often, companies
will limit their change validation to the change only, or they will
go to the other extreme and they will needlessly repeat the entire
validation. The right answer is somewhere in between.

In Conclusion: Agilents Response

Agilents goal is to avoid shipping software that introduces or
maintains regulatory exposure for customers, by incorporating
FDA priorities into our system designs. The US FDA is clearly

pushing for more technical controls, prioritizing technical controls over procedural controls, and prioritizing prevention controls over detection controls. 21 CFR Part 11.10(a) talks about
systems needing to have the ability to detect invalid or altered
records. That is the only part of the regulation that actually
talks about detecting records that may have been manipulated
through nefarious means. The remainder of the controls are
preventive. So yes, detection controls are important, but prevention controls are better. Prioritizing online records over hard copy
printouts has also been emphasized. The FDA cannot always
trust the paper thats being handed to them.
Another example of system design is the use of an online
audit trail review, as illustrated in Figure 1. There are a lot of
systems that may allow the generation of audit trail reports
in printed form, but Agilents new CDS version has a built in
tool that allows a user to electronically review electronic audit
trails entries. These audit trail entries are organized by type,
an online review can be performed, and electronic signatures
incorporated.
These examples serve to demonstrate how Agilent is applying critical thinking to redesigning laboratory software to help
respond to the new regulatory compliance realities.

A Sponsored Q&A
LC/GC: Do pharma companies typically do onsite audits at Agilent to review SOPs, et cetera
or are these audits typically paper audits?

LC/GC: What about systems with no audit

trail or an incomplete audit trail? Whats
the work around for such systems?

Loren Smith: Approximately 90 percent of the informatics soft- Loren Smith: There are a lot of old systems that still get used in
ware product audit requests that Agilent receives are paper regulated labs and the systems work perfectly fine to do what they
audits, and the remaining 10 percent will actually come and do do. But the work around for this is a heavy dose of procedural
on-site audits. However a paper audit is not evidence based. controls that would include handwritten records. Depending on
It is based on declarative statements, or whatever Agilent the criticality and the risk of the particular process thats involved,
chooses to say, consequently there is nothing in the way of or that the system is addressing, the procedural requirements
verification of anything that is said in a paper audit. Paper au- may need to go as far as a second person verification of informadits are typically performed by box checkers; people doing tion or actual concurrent witnessing of the work being performed.
the paper audit so that they can say that they did it. Some The purpose of the audit trail is to be able to tell the story or reconcustomers will use a paper audit as a preamble to an on-site struct the history of an electronic record, not just the creation of
audit to get a rough idea of Agilents position before they that record but also the changes or the deletion of records. If the
come on-site.
system has limited or incomplete audit trail ability then the record
must be supplemented in some other way, often taking the form
of
paper records, for example, a use notebook next to an instruLC/GC: Some say that [purchasing] the IQ-OQ
package [as a complete validation solution] is a myth. ment. Technical controls make everybodys life much simpler.
Loren Smith: The IQ-OQ package itself is real, not a myth. However,
the IQ-OQ package is necessary but insufficient because it does LC/GC: Will new data systems eliminate
not satisfy all of the FDAs requirements for validating a computer- all need for procedural controls?
ized system. For example, the IQ-OQ package does not include Loren Smith: No; procedural controls will always be required. For
validation planning documentation or validation summary reporting. example, the procedures for granting users access to a data
The IQ-OQ package also does not include requirement specifica- system or the procedures for changing, modifying, or removing
tion, user requirements, a functional specification, or traceability a persons access to a data system would be an example of a
from the IQ and OQ test themselves to the users requirement either procedure required by the regulation that the data system is not
user requirements or functional specification.
necessarily going to be able to address. Currently, the data system can create those accounts and change user privileges. But

A SPECIAL ADVERTISING SECTION

DATA INTEGRITY IN PHARMACEUTICAL QUALITY CONTROL LABORATORIES

the procedure for how that is done is normally going to require

some type of record keeping that includes management reviews
and approvals of system access or changes to system access.
So, procedural controls are not going to go away completely.

LC/GC: Is it an essential element to execute a

disaster recovery process on a manufacturing
kit during qualification and do you see any
risk with respect to software removal?
Loren Smith: The subject of disaster recovery testing is something
that is expected by the regulations; European regulations are a
little bit more explicit on the subject. A disaster recovery test can
often fail due to incorrect assumptions. Disaster recovery testing
is particularly useful for mission critical systems.

LC/GC: Are there examples of audit trail

review that are less cumbersome than what
Agilent is building into its software?
Loren Smith: Some systems that have electronic audit trails dont
have the means to indicate that those audit trails have been reviewed, resulting in having to divide the audit trail review activity
up into smaller chunks and primarily having those audit trails
be very, very relevant to the data that is being reviewed. For
example, for pharmaceutical batch release, a QA review must
be performed for all of the relevant records that go into manufacturing that product and all of the systems that support its
manufacturing. If the QA review looks at audit trail entries that
are relevant to those records, even if it happens to be on paper
the audit trail review activity is more relevant to the record review
itself. So the review is less cumbersome simply as a result of
dividing the problem into smaller chunks that are more relevant
to the actual record.

LC/GC: Can you talk about addressing the challenges

of peak integrations in auto mode instead of manual?
Loren Smith: Automatic peak integrations are normally addressed
in a procedure that determines the parameters for the allowed
automatic peak integrations and potential re-integrations. Generally, the flexibility of the integration or re-integration activity is addressed in an SOP and part of the system or process validation
for that particular analytical method.

LC/GC: At some point, there must be an administrator

for every system who will have access to modify
or delete records, how do you recommend dealing
with this potential data integrity problem?
Loren Smith: The best practice for selecting a system administrator is making sure that the administrator on a particular system
does not have a vested interest in the data on that system. What
this means in larger organization is that very often someone
in the IT department is in charge of administering the system.

A SPECIAL ADVERTISING SECTION

Separation of duties is a subject that IBM has discussed in the

past and it is also significant in the financial world. If a person
has no motivation to do anything with the data on that system,
then its very unlikely that there will be a problem. The FDA has
said that if there is any evidence of intentional fraud that they
would pursue it as a criminal activity and that is something that
they do look for.

LC/GC: When samples are processed, is

an audit trail comment enough or is an
approval process also necessary?
Loren Smith: It depends on what a companys policies and procedures expect. If there are sufficient controls in the system and
people are sufficiently trained, so that when sample reprocessing occurs, its occurring under the proper conditions with the
proper controls and procedures in place, then the audit trail
comment may be enough. However if the procedure doesnt address the reprocessing of samples, or if its considered anomalous or unusual, then it may be important to have an explicit
documented approval process.

LC/GC: Is it ever acceptable to ever

delete data in the eyes of the FDA?
Loren Smith: The FDA has answered this question explicitly and
implicitly in two different areas. First, they talk about data retention requirements for pharmaceutical products, and the requirement is to maintain pharmaceutical manufacturing records
for seven years after the expiration of the last manufactured lot
of a particular pharmaceutical product. So, if after seven years
production of a drug product is halted, and the product has an
expiry thats 12 or 18 months out, then seven years after that
expiration date it is acceptable to delete data. Secondly, CFR
Part 11 specifies that audit trails need to deal with three things:
they need to deal with creation of regulated records, modification
of regulated records, and the deletion of records. However, prePart 11 in the mid-90s, the FDA found that many companies, in
their analytical laboratories, were deciding that the final analytical
report was the regulated record, and companies were deleting
the raw data.

LC/GC: Is a vendor postal audit enough

to satisfy GMP requirements?
Loren Smith: Time and resources will play a role in deciding how
to approach audits. One approach is to organize software suppliers based on a risk assessment; risks to patient safety, product quality, and data integrity for a particular system, based on
the results of prior audits. If vendors have gone through audits
and produced positive results in the past, that can be factored
into the risk assessment for that particular vendor. In that way,
a determination based on risk can define the audit cycle for
many suppliers.

A SPECIAL ADVERTISING SECTION