GXP

Approaching Microbiological
Method Validation | IVT
By 
Tim Sandle, Ph.D.
 
 Dec 4, 2015 7:21 am PST

Peer Reviewed: Microbiology

Introduction

Microbiology, through regulatory requirements and advances in technology, has become a

more exacting science. This is despite the relatively high variation with many methods,
especially the culture based ones, when compared with analytical methods designed to
recover chemical substances. 

While it is unlikely that microbiological methods will meet the more exacting demands of
the analytical ones. Nevertheless, the advances with rapid microbiological methods and
alternative techniques require microbiologists to become more familiar with assay
requirements and parameters. Many novel methods are automated and use forms of data
capture where the data is subject to statistical analysis.

These developments have impacted upon the approaches taken to the validation (or
‘qualification’) of microbiological methods. Validation, in this context, can be defined as
the process by which it is established, by laboratory studies, that the performance
characteristics of a method meet the requirements for the intended application.   In terms
of microbiological tests, one of the most important aims is in determining whether the
sample to be examined has any inherent anti-microbial properties and whether the
incubation and growth conditions can recover microorganism that may be present to an
acceptable level. Appendix I provides a generalized flow chart to assist with some of these
considerations. 

Aspects that may need to be considered in drawing up assessment criteria for a

microbiological test include the limit of detection (for example ‘what is the lowest level of
microorganisms that can be detected?’); specificity (for example ‘what range of different
microorganisms can be detected?’); and quantification (for example, the counting
accuracy). These types of questions should form the basis of a microbiological methods
validation strategy (1). 

This paper presents an overview of the validation of microbiological methods, considering

some of the limitations and outlining the key criteria that may be applicable for
assessment. Such a review is additionally useful given the revision process in relation to
the United States and European Pharmacopeia in relation to alternative methods, and
associated regulatory interest (2). The extent to which any of the described criteria are
suitable will depend upon the sophistication of the method and the limit quantification or
detection required. The idea here is not to explain the validation of any one method, but
to provide a set of criteria, together with appropriate explanation, for the microbiologist
to consider. Once relevant criteria are selected they can be built into a validation protocol.
Appropriate selection will enable a successful validation study to be run.

Types of microbiological methods

Microbiological methods fall into one of three major categories (although this
categorization is not always exclusive) (3): 

a) Qualitative tests for the presence or absence of microorganisms, e.g. the

pharmacopoeia sterility test. Commonly, qualitative tests are assessed through the use of
turbidity or other growth related changes in a culture medium, as evidence of the
presence of viable microorganisms in a test sample.
b) Quantitative tests for the enumeration of microorganisms, e.g. a total viable count test
or flow cytometry.
c) Identification tests, e.g. the speciation of bacteria using a biochemical test. This
includes morphological and biochemical characterization such as biochemical reactions,
carbon substrate utilization, characterization of fatty acid composition, restriction
endonuclease banding patterns and use of 16S or DNA sequence analysis (4).  

Variability with microbiological methods

Methods that fall within the three described categories have a degree of variability, and
with these variations microbiological methods are inherently different from analytical
ones (5). Not least in the lack of agreed demonstrative criteria. The wide variability is
acknowledged in Ph. Eur. 5.1.6, the chapter relating to alternative microbiological
methods (6). This is particularly with regards to relative broad ranges.   

The reason for the variation is because microbiology is a logarithmic science.

Microbiological methods are capable of distinguishing between 100 and 1000 cells (1 log),
but not smaller differences, such as 0.3 or 0.5 of a log. This means that 18 CFU is
technically no different than 10 CFU; and 1 CFU is different than 10 CFU but not 5 CFU.
Therefore cultural based methods in particular provide estimates rather than exact cell
counts.
 Variability may result in difficulties in comparing two methods.  In general, only a 50%
comparison is achievable for culture methods.  With rapid and alternative microbiological
methods greater comparability can be achieved, however levels of precision remain in the
order of 15 to 35% relative standard deviation (obtained by multiplying the standard
deviation by 100 and dividing this product by the average).

Method recovery

Culture based microbial validation is limited by the ability of microorganisms to

reproduce under a set of conditions in relation to sample preparation, cultivation and
incubation.  Any method is, therefore, a general indicator only. With cultural based
methods, as described in the pharmacopeia, it is recognized that the limit of detection has
never been established quantitatively.  

Many variables can affect recovery. These include:

 Growth media,
 The colony forming unit or “CFU” is not a true cell count (individual cells are rare
in nature, leading to the CFU being an underestimation of the number of microorganisms
present),
 Incubation conditions (temperature and time),
 Nutritional requirement of the organism,
 Physical condition of the organism (stressed or sublethally damaged due to
temperature, humidity, high ionic strength, pH extremes, osmotic shock (relating to
liquid), residues of antimicrobial chemicals),
 Dilution errors (7),
 Environmental organisms are unlikely to be recovered when they are in the
exponential growth phase (exponential growth leads to better recovery),
 Characteristic of the item under test,
 The types of neutralizers used (8).

Some of these considerations are contained within Appendix I, which provides a flow
chart for a generalized approach relating to experimental design.

With non-culture based methods, these are unlikely to be affected by the above list.

Method comparison

The introduction of a new method into a laboratory, where one method is intended to
replace another, invariably requires a level of comparison.  When comparing an older
method with a new method, new methods will most probably recover a greater number of
microorganisms. Given that a ‘positive’ or higher result could be obtained with an
alternative method and a ‘negative’ or lower result could be obtained with an established
method, this does not necessarily suggest that the ‘positive’ result is a false positive. 

However, because greater numbers of organisms may be recovered with an alternative

method, this does not mean that the risk is now greater. This is normally because of
 process controls in place. Whether limits should be revised in light of greater microbial
recoveries, however, remains a contested point.

In evaluating an alternative method a close match is not necessarily required, in terms of

the result obtained, between the alternative method and the current method. What is
important that the alternative method is capable of allowing the microbiologist to make
an equivalent decision in relation to sample or product quality in a consistent fashion.

Devising microbiological studies

Before commencing experimental work, the aim, hypothesis (if applicable) and title
should be developed.  All subsequent experimental planning must develop from the aim
and hypothesis.  It is also important at this stage to consider how any generated data will
be examined.   

Thus, at an early stage the following should be established:

 Aim - What the objective of the experiment is.

 Hypothesis - This is a statement concerning an expected outcome of the
experiment which the experimental result will prove or disprove.

The objectives for the experiment should be ‘SMART’. 

 Specific,
 Measurable,
 Achievable,
 Realistic,
 Time based.

In defining the objectives, the following should be considered: 

 Validation requirements,
 Test controls,
 Microorganisms,
 Type of data/statistical analysis (this is an area requiring care, especially when
comparing compendial methods with alternative ones) (9),
 How many times does the experiment need to be run?
 How many samples are required?

Reference sources may need to be examined before undertaking an experiment in order to

determine if a similar study has been performed.  Part of this approach is set-out in
Appendix I. 

As part of a study, preliminary work (‘method development’) may be undertaken when

drawing up of a protocol. This gives some idea about the capability and limitations of the
method, and helps with the development of appropriate assessment criteria. Doing so also
helps to determine if there are aspects within the protocol that require clarification; and
 to help determine the logistics of the study (such as culture media requirements, number
of technicians required and so on).

Factors to consider for experimental design

This section lists several aspects of experiment design that need to be considered and
incorporated into the design stage.

Samples 

The size of the test sample must be considered. Importantly, the number of samples must
be representative and of a sufficient number. A statistical technique may be used to set the
number of samples required. Care must be taken here since the statistical technique
selected may influence the sample size.  The appropriate volume of sample may be a
factor, particularly with bioburden testing and ensuring that the sample tested is
representative of the final homogenous bulk.

For most validation exercises the number of batches tested is three (or more).  With areas
like environmental monitoring, consideration should be given to the location of samples,
such as the number of locations within a cleanroom.

Consideration should also be given to:

 Testing samples at the end of the shelf-life or expiry time (this may include
assessment at interim time points);
 Degrading samples stored in containers;
 Holding samples under ‘worst case’ conditions (such as upper or lower
temperatures);
 Testing samples at the end of any required process hold times. 

The above points are applicable to many bioburden and bacterial endotoxin tests. 

Any pre-requisite treatment of the sample, e.g. neutralization, or microorganisms

(promotion of reproduction) should be considered.

Test controls 

Experiments should normally have duplicate positive controls, negative controls and,
where recovery needs to be demonstrated, positive product controls (direct product
challenges or ‘spikes’).  The level of recovery must be defined at the outset; and the level of
recovery must be justifiable in terms of experimental aims and the test method employed.

Microorganisms 

An experiment normally requires a range of microorganisms.  For all experiments,

microorganisms from an approved culture collection should be used to ensure uniformity
and traceability. In some cases compendia will indicate the types (or event specific
strains) of the microorganisms required (for example, the sterility test chapter within the
main pharmacopeia). In other cases, the microorganisms will need to be selected based on
professional judgment. Experiments may be supplemented by environmental isolates, or
'wildtypes', as appropriate (this is normally appropriate when a culture medium is used to
monitor a production process). 

In selecting microorganisms it is often a good idea to draw these across a range of

different morphological types. Suitable categories include:

   i) Gram positive rod and/or a Gram positive spore bearing rod, e.g. Bacillus sp.

  ii) Gram positive cocci e.g. Staphylococcus aureus

  iii) Gram negative rod e.g. Pseudomonas aeruginosa

  iv) Fungi, (yeast), e.g. Candida albicans

  v) Fungi (filamentous), e.g. Aspergillus brasiliensis

In all instances either the specific culture collection reference must be quoted or the
source of the isolate identified. For certain Gram-positive bacteria, the protocol should
specify if the organisms should be in the endospore state. 

The type of experiment must be considered when allocating microorganisms.  For

example, a project involving the examination of water would most likely require a Gram
negative rod (such as Pseudomonas sp.) and a coliform (such as Escherichia coli). In
contrast, an experiment conducted at 55°C (as with a test for thermophilic
microorganisms) would involve a thermophile such as Geobacillus stearothermophilus. 
Consideration should also be given to the storage of cultures. 

Sometimes microorganisms are not recovered as expected. This is a case for carrying
method development work in advance. An example of poor recovery can occur with
Gram-negative rods. With these organisms, desiccation can occur during aerolisation.
This may occur when using an active air sampler or following loss of moisture content in
an exposed settle plate surface.  This effect tends to damage Gram-negative bacteria more
greatly.  This is because soluble cell contents tend to leak and mechanisms to control the
transfer of molecules and ions in and out of Gram-negative cells in particular are
considerably impaired.  Damage to the mucopeptide lipopolysaccharide center cell
structure also causes cell damage and loss of viability.  Such damage occurs very shortly
after weight loss to a medium or after aerolisation.  Such cell damage is typically
irreversible (10).  

Microorganisms used should normally be prepared from cultures which are no more than
24 hours only and no more than five passages from the seed lot. However, some
microorganisms require longer cultivation than 24 hours.  In these instances the culture
age must be documented in advance to ensure that the culture used is as young as it can
be. With passages (or subcultures), this is to prevent phenotypic variations from occurring
 which might influence the way the microorganism behaves in the presence of the sample
(11).

It is good practice that the purity of cultures is confirmed in advance. This can either be by
acceptance of a certificate of analysis from the supplier of the cultures, or by confirmatory
identification conducted by the recipient laboratory. In some circumstances, such as when
conducting a challenge test, a post identification confirmation may also be deemed
necessary. 

With some experiments, an attempt may be made to induce a stressed state to the
microbial population. This will need to be decided at the time of writing the test protocol.
The reason for attempting this is based on the “unstressed” batch culture grown
organisms being are an artificial creation that rarely exists outside the laboratory (12).
Stress factors faced by microorganisms in the environment include:

 Desiccation, 
 Nutrient deprivation, 
 Nutrient limitation, 
 Cold shock, 
 Heat shock, 
 Exposure to ultra-violet light, 
 Other types of radiation leading to sublethal damage,
 Responses to disinfectant or detergent residues, 
 Responses to preservative residues,
 Osmolality.

Creating a stressed state is difficult in itself and difficult to verify, given the unknown
effects of causing damage to cells or with suppressing cellular growth (13). In addition, the
age of a culture will affect its recovery and will be important for certain identification
methods.

Temperature

Consideration needs to be given over the temperature ranges the experiment needs to be
performed at. Here there is little value in restricting validation to one temperature range
for testing that takes place over multiple ranges. An example here would be testing culture
media at 20-25 C and then using it across the range 20-40 C, and expecting its growth
o o

promoting properties to be consistent.  Commonly used ranges include: 

 2°C - 8°C, 
 20°C - 25°C, 
 30°C - 35°C, 
 36°C - 38°C
 55°C - 60°C
In addition to the above, dual incubations across two or more ranges may be required
(such as for the enumeration of bacteria and fungi with a single-use culture medium for
environmental monitoring) (14).

Time

As with temperature, consideration needs to be given to the time period over which the
study is to be conducted. Getting this right is important since the selected time becomes
the maximum run-time for the test. It is important to ask if incubation time is set long
enough to show the limit of detection? The time of the validation read must never exceed
the time used by the testing laboratory for the reading of samples. 

Moreover, in specifying incubation times the minimum time must be clearly stated. For
example, “incubate plate at 20-25 C and read at 24 and 48 hours” rather than “incubate
o

plate at 20-25 C and read after 48 hours.” Consideration of acceptable tolerance should be
o

stated. Is “read at 48 hours”, for instance, reading within ±1 hour of 48 hours or will a
wider tolerance be used, and with what justification? Depending on the test method, it
may be appropriate to time samples going into and out of incubation.

Incubation times should be realistic in terms of loss of viability or loss of growth

supporting properties of media (e.g. the time set should not be so extensive that plate
media desiccation would occur).

Growth Promotion 

A key question here is: “What type of growth promoting conditions are required for the
cultivation of microorganisms?” This would include different types of agars, broths,
dilution reagents and so on.  These may require a separate (or first phase) validation.

Atmosphere 

Whether the microorganism(s) to recover are obligate aerobes, facultative

aerobes/anaerobes, microareophilic, capnophiles, or obligate anaerobes, should be
considered. A microaerophile, for example, is a microorganism that requires oxygen to
survive, but requires environments containing lower levels of oxygen than are present in
the atmosphere (that is <21% O ; many require 2-10% O ). Once established it will be
2 2

known whether a specific combination of gases is required, for example, CO N.2

Antimicrobial Activity/Interference

Note should be taken of whether any of the experimental conditions produce an

antimicrobial effect or interfere with the test in way of enhancement or inhibition. 
Attention should be paid to the best method for neutralizing any anti-microbial
properties.  Common methods for neutralization include dilution, rinsing, filtration or the
use of general or specific neutralizers.

pH
 Note should be taken of pH conditions. pH is important to microbial growth and certain
ranges will be unsuitable for some microorganisms.

Growth Phase 

The growth phase of a microbial population can impact upon the accuracy of a method,
for example, a turbidity method.  Ordinarily, in culture, the following growth dynamics
are observed (15):

 Cells initially adjust to the new medium (lag phase). Cells maybe growing by mass,
but not in number. Lag phase is influenced by size of the inoculum; time to recover from
physical damage; time required for synthesis of essential coenzymes; and time required
for synthesis of new enzymes necessary to metabolize the substrates present in the
medium.
 Cells then start dividing regularly by the process of binary fission (exponential
phase).  Here cells divide at a constant rate, expressed as generational or doubling time.
Ideal generation times wider according to different microbial species. Escherichia coli, for
example, will double every 17-20 minutes, whereas Mycobacterium tuberculosis doubles
every 790-930 minutes.
 When their growth becomes limited, the cells stop dividing (stationary phase).
 Eventually cells show loss of viability (death phase).  

With other methods growth is not a requirement and instead cell count is of importance.
Such methods may or may not be able to distinguish between viable and non-viable
microorganisms. A method like flow cytometry, for instance, will count all cells
irrespective of whether they are viable.

Number of Microorganisms

Some methods will require a minimum number of microorganisms in order to detect

them.  For example, a method linked to microbial catabolism; adenosine triphosphate
(ATP) growth; bioluminescence etc.  A pre-growth preparatory step may be required in
order to maximize recovery. 

Single Cultures

Most methods require a single (pure) culture in order to obtain a valid result.  Mixed
cultures can be employed in certain circumstances, for example, container closure
integrity.  Here, care must be taken that one microorganism does not significantly out-
grow the other. In general, mixed cultures should be avoided. 

Enzymes

When detecting presence of specific microorganisms using a selective medium, the

specificity and selectivity of the microorganism and media must be demonstrated using
selective and non-selective microorganisms.

 
 Validation Parameters

In studying a microbiological method, different validation parameters require

assessment. Not every parameter will be appropriate, either due to the nature of the
method (qualitative or quantitative) (16) or due to its capabilities (a modern, rapid
microbiological method, for example, will have been designed to meet more parameters
than an older cultural method). Several of the categories have been revisited by
microbiologists in light of the emergence of rapid and alternative microbiological methods
(17).

Specificity 

Specificity is the capability of the method to resolve or measure a range of

microorganisms in the presence of other compounds or microorganisms.  This may be a
single microorganism (such as a test for coliforms) or a range (such as a general
bioburden test).  Freedom from interference from excipients or active pharmaceutical
ingredients, degradation products or impurities must be noted as part of a recovery
(accuracy) study.  Where selecting an appropriate culture medium is part of the study, the
properties of the medium against selective, non-selective and mixed cultures must also to
be considered.

The microbial challenge should be set above the limit of detection or quantification, while
also being at level that provides a measure of the efficacy of the method. Here:

 For a growth based method, a low number of <100 CFU is appropriate. All
challenge microorganisms should be recovered. Where atypical colony morphology is
observed, supporting identification should be considered. 
 For a non-growth based method, suitable positive and negative controls should be
used to show that any extraneous matter does not interfere with the detection of the
microorganisms.

Appropriateness

A new method or test must demonstrate that it is appropriate for its intended use.  If a
method or test is intended to replace an established method, parallel testing of both
methods must take place and the collected data compared if possible, ideally by statistical
tests of significance. In some cases, a direct comparison is not possible (for example, two
different models of particle counter cannot be directly compared because they will not be
sampling the same volume of air). 

Accuracy

Accuracy is the closeness of agreement between the measured value and the ‘true’ or
expected measure or reaction across the range of the test.  This can be assessed by
determining the recovery of known quantities of a microorganism that has been added to
a sample.   

For quantitative tests, this is predicted from:

 The dilution of a microbial suspension; or,
 By examining for presence/absence; or,
 From a taxonomic identification; or,
 By comparing the new method to an established test method (here the new method
must give equivalent or better results to the established method).  

For enumeration methods, the level of recovery should reflect the test method.  This is
normally by the percentage of microorganisms recovered by the method.  

 a) If ‘good’ recovery is considered to be achievable, then a recovery level of 50% should be
the minimum (this can also be expressed as a productivity ratio).  Where an upper level of
recovery is required, this is normally set at 200% (with a range of 50-200% quoted).
b) When comparing between two methods, 70% is sometimes set although other
acceptance criteria may be appropriate.  

Comparison of accuracy can be further examined by significance testing, such as Student’s

t-test or an alternative method. For example, accuracy can be expressed as:

Accuracy % = (Number of Correct Results in Agreement/Total Number of Results) x 100

For qualitative methods it is recognized that many hundreds of comparisons may be

required if a negative result is the expected outcome, such as with the sterility test.  A
limitation must be established in relation to the number of samples; this is because
testing samples until a positive result is obtained will be impractical.  When comparing
two methods, the relative rates of positive and negative results should be compared.

Precision

Precision is the closeness of agreement between a series of test results or the variation in a
series of test results, when a method is applied repeatedly to multiple samples. 

Precision can be subdivided into:

Repeatability (within test variation).

This is the variation in results obtained on the same sample when assayed repeatedly with
one test or within a short period of time, by the same technician using the same reagents
and equipment.  

The key to acceptability is the amount of variation.  It may be expressed as:

a) Standard deviation, 
b) Coefficient of variation (relative standard deviation),
c) Confidence interval of the mean. 
d) With specific tests, such as microbial identification systems, other criteria will be used
to determine the similarity of the recovered organisms.
e) Other statistical techniques like Chi-squared, maybe more appropriate (18). 
Normally at least 3 replicates are required.  Depending on the types of sample, more than
one determination may be required (for example different dilutions or a range of
microorganisms). Because the testing technician and consumables are the same, this
approach shows variation in sample as assessed against the method. 

Intermediate Precision

This is the variation in results obtained on the sample when assayed on several separate
occasions by different technicians using different reagents and equipment etc.  This shows
reproducibility. This may be expressed as: 

a) Standard deviation, 
b) Coefficient of variation (relative standard deviation),
c) Confidence interval of the mean. 
d) With specific tests, such as microbial identification systems, other criteria will be used
to determine the similarity of the recovered organisms.
e) Other statistical techniques like Chi-squared, maybe more appropriate. 

A minimum of three determinations should be carried out.  Appropriate acceptance

criteria (such as ≥95%) should be set for repeatability. This approach shows variation
across people and reagents, and variability within the method.

Range

Range is the interval between the upper and lower levels of microbial count, for which the
procedure has been established as suitable with accuracy, linearity (if appropriate, where
there is requirement to construct a curve), and precision.  

For example, the commonly used range for microorganism recovery is less than 100 CFU
for total count techniques. Where a range is required, in order to assess the range of the
test, this is covered by diluting a microbial population; for example, 10  to 10  cells.   With
0 6

some methods, a regression analysis can be considered to compare two methods. 

Robustness and ruggedness

Robustness is the reliability of a method or test to withstand small (but deliberate)

variations due to external influence.  For example, different technicians, instruments,
incubation time, ambient temperature, and reagents.  With rapid methods, robustness
can be undertaken by the method supplier.  However, once in the laboratory, long term
performance must be considered and, if possible, checked by internal control samples. 
For example, the distribution of microorganisms on a membrane can affect robustness.  

Ruggedness is the degree of reproducibility by testing samples using different testers and
equipment; this is assessed by coefficient of variation. 

Limit of Detection 
This is the lowest number of microorganisms which can be detected, but not necessarily
quantified (such as a low level challenge) under the stated experimental conditions.  This
test is generally used for rapid and alternative methods. 

Often the amount of sample tested the initial dilution of the sample and any subsequent
dilution of the sample may determine the limit of detection.  The challenge
microorganisms selected should be of an appropriate range as indicated above). The
challenge can consist of taking each microorganism and making a serial dilution range.

The outcome can be expressed as presence/absence or enumeration.  Presence/absence is

normally qualitative.  An attempt can be made for a semi-quantitative analysis by varying
the microbial challenge (to develop a limit test, i.e. to detect <100 CFU, <10 CFU etc.) 

It is recognized that the act of dilution may result in a greater loss due to lack of
homogeneity and the typical Poisson distribution of microorganisms in liquid; such as in
the evaluation of a raw material by using the pour plate method and where the limit of
detection is <10cfu/g.  This is further complicated by the impossibility of obtaining
reliable samples containing a single microorganism.  Therefore, microbiologically limits of
detection must sometimes be considered as theoretical rather than practically
demonstrable. For this reason, many pharmacopeial` tests require the use of a low level
challenge (<100 CFU) and this is normally considered sufficient. For comparing methods
Chi-squared is often the statistic of choice.

Limit of Determination

This is the lowest level of the sample where the microbial content can be quantitatively
determined with defined precision and accuracy.  Again, further complication arises by
the impossibility of obtaining reliable samples containing a set number of
microorganisms.   

Therefore, microbiologically limits of detection must sometimes be considered as

theoretical rather than practically demonstrable. Many pharmacopeial tests require the
use of a low level challenge (<100 CFU) and this is normally considered sufficient. Limits
of quantification are normally determined by 3 or more replicates across the range.

Linearity

This is the ability to elicit results which are proportional to the concentration of
microorganisms within a given range.  This is measured by correlation coefficient or a
goodness of fit test (such as Chi-Square).  This will only be applicable to enumeration
methods using an analytical system.

When comparing two methods, non-linearity will occur if one method is superior to the
other (in terms of microbial recovery). Here Spearman’s rank may be appropriate as a
statistical tool to use (a non-parametric measure of statistical dependence between two
variables). 

Predictive Value
 For qualitative tests such as the sterility test or growth of selective media, the use of
positive or negative predictive values may be appropriate.  This can be expressed as a
percentage of the observed test results against the total or expected test results.

Rapid and alternative microbiological methods

Rapid and alternative microbiological methods embrace those methods that are distinct
from compendial methods. Rapid indicates that the method gives a faster time-to-result
and alternative indicates that the method differs from one presented in a recognized
pharmacopeia (19). In many cases such methods are more accurate (although this is not
always the case) and they are invariably automated (20). Given the pace of technological
development and commercial gains, such methods are becoming more commonplace.

When considering the introduction of an alternative method, the following should be

considered (21):

 Prepare a User Requirement Specification. 

 Consider instrument qualification; validation requirements of the alternative
technology and aim (e.g. to be equivalent to current methods or to improve upon a
current method); method suitability (e.g. sample volume requirements, critical
parameters of compendial test, suitability for a range of different products to be tested)
(22).
 Sample size and number of samples:
o With a new method, it may be appropriates to assess all parameters on one
type of product and then to run method suitability tests only on other products against the
same method.
o In terms of results interpretation, non-growth based methods (such as those
looking at metabolic activity or ATP) will provide more accurate cell counts and these are
not comparable with growth-based estimates of colony forming units.
o Results from alternative methods, such as cell count, cannot be compared
statistically to CFU results from culture based methods.
o Data handling.
o Test controls e.g. use of control microorganisms.
o Consideration of test limits for samples tested by alternative methods.

With alternative methods the aim is to verify the detection capability of the alternative
method.

 In terms of parameters to review, the following acts as guidance (based on USP chapter
<1223>) (23):
            
Validation parameter Qualitative test Quantitative test
 

Accuracy (a method No Yes  

suitability test)
 

Precision (a method No Yes  

suitability test)
 

Specificity (a method Yes Yes  

suitability test)
 

Limit of detection Yes Yes  

 
Limit of quantification No Yes
 

Linearity No Yes  

 
Operational (dynamic) No Yes
 
range
 
Robustness Yes Yes With the
above, a
Repeatability Yes Yes degree of
interpretation
is required. It
Ruggedness Yes Yes follows:

i) Method
Equivalency Yes Yes suitability:
accuracy,
precision and
specificity
(recovery of challenge organisms) for quantitative methods; for qualitative methods, only
specificity would apply.
ii) Robustness is assessed as above. It is not directly compared with a current method.
Manufacturer’s data may be considered.
iii) Ruggedness is assessed as above. It is not directly compared with a current method.
Manufacturer’s data may be considered.
iv) Limit of detection: this is assessed by using a range of microorganisms. 
a. Approach #1: Each organism is prepared as a serial dilution, with the inoculum
adjusted to a target of 50% that shows growth in an existing method. Both methods
should be run over several replicates. An appropriate statistical comparison method is
Chi- squared.
b. Approach #2: A Most Probable Number (MPN) method is used. For a ten-fold series a
range of 10  to 10  microorganisms is used; or for a two-fold series, using the range 5 to 10
-1 -2 -

 microorganisms. The alternative and established method should be run five times using
1

3 dilutions (the dilutions should provide at least one positive and one negative dilution).
Chi-squared is an appropriate statistical tool for method comparison.

Where the aim is to show equivalence between two methods, there are four possible ways
to assess this (again as referenced in USP <1223>). The appropriate category should be
selected in advance of conducting the validation and referenced in the validation protocol.
The options are: 

The
Option Demonstratio Comparison Based on Number of
n to existing numerical characteristics
method results or
conclusions

1.     Acceptable No Results Multiple

Acceptable procedure

2.     Equivalent Yes Results Multiple

Performance
equivalence

3.     Equivalent Yes Results Singular

Results equivalence

4.     Equivalent Yes Conclusions Singular

Decision equivalence

above options can be interpreted as:

1. Acceptable procedure: this is not strictly an equivalence option, it is about

the new method meeting a minimum performance or acceptance requirement. The
qualification may involve a standard inoculum with a range of microorganisms to
assess recovery, or a level of ATP.
2. Performance equivalence: better or equivalent results are required from the
new method compared with the existing method. This is with regard to: accuracy,
precision, specificity, limit of detection, limit of quantification, robustness and ruggedness
(only those factors that are relevant require inclusion).
3. Results equivalence: this is similar to performance equivalence expect with
the added requirement that the new method must give equivalent or better numerical
results. Because the same sample cannot be tested against two methods, a tolerance value
is required. 
4. Decision equivalence: this is similar to results equivalence with the
exception that ‘pass’ or ‘fail’ results are compared (as with a ‘growth’ / ‘no growth’ test
result). It is also possible to consider a Most Probable Number method for qualitative
tests. When evaluating instruments there should be a one-sided inferiority hypothesis. 

Discussion

This paper has outlined the key criteria to be considered when undertaking
microbiological method validation, either in relation to the qualification of the method
itself or in relation to testing samples against a specific method. The approach outlined is
partly applicable to established method and partly applicable to rapid and alternative
methods. Here not all of the criteria outlined are applicable to all methods.

In drawing up the list of possible criteria, examples have been provided in order to help
the reader consider the applicability of each item. With this it is important to plan and to
devise a suitable validation protocol, outlining acceptance criteria and the way that the
validation will be executed.

References

1. Newby, P. (2005) Implementation, validation and registration of rapid

microbiological methods. European Pharmaceutical Review. 13(3): 67-73
2. Hussong, D.; Mello, R. (2006) Alternative Microbiology Methods and
Pharmaceutical Quality Control.American Pharmaceutical Review. 9(1): 62-69
3. Sandle, T. (2014) Approaching the Selection of Rapid Microbiological
Methods, Journal of Validation Technology, 20 (2): 1-10)
4. Sandle, T. (2013). Automated Microbial Identifications: A comparison of USP and
EP approaches, American Pharmaceutical Review, 16 (4): 56-61
5. Sandle, T. (2015) FDA Signals a New Approach for Analytical Method
Validation, Journal of Validation Technology, 21 (2): 1-5
6. Pharmeuropa (2015) Chapter 5.1.6 Alternative Methods for Control of
Microbiological Quality, European Directorate for the Quality of Medicines &
HealthCare. 27.1:8
7. Jennison, M.W. and Wadsworth, G.P. (1940) Evaluation of the Errors Involved In
Estimating Bacterial Numbers by the Plating Method. J Bacteriol. 39:389-397
8. Sutton, S. V. W. (1992) The evaluation of biocide neutralization in the
determination of microbicidal efficacy of contact lens solutions. Int’l. Contact Lens
Clinics, 19, 167-173
9. Friedman, E.M.; Warner, M.; Shum, S.C.; Adair, F. (2014) In-Process Microbial
Testing: Statistical Properties of a Rapid Alternative to Compendial Enumeration
Methods. PDA Journal of Pharmaceutical Science and Technology. 69(2): 264-269
10. Anderson, J D (1966):  Biochemical Studies of Lethal Processes in Aerosols
of E.coli, Journal of General Microbiology, 45: 303 – 313
11. Snell, J.J.S. (1995) Preservation of Control Strains. In J.J.S. Snell; D.F.B. Brown;
and C. Roberts. (Editors).Quality Assurance: Principles and Practice in the Microbiology
Laboratory; London: Public Health Laboratory Service, pp 69-76
12. Gray, J.C.; Staerk, A.; Berchtold, M.; Hecker, W.; Neuhaus, G.; and Wirth, A.
(2010) Growth promoting properties of different solid nutrient media evaluated with
stressed and unstressed micro-organism: Prestudy for the evaluation of a rapid sterility
test. PDA J. Pharm. Sci. Technol. 64 249-263
13. Dupont, C.; Augustin, J. (2009) Influence of stress on single-cell lag time and
growth probability for Listeria monocytogenes in half fraser broth App. Environ
Microbiol. 75: 3069-3076
14. Sandle, T. (2014) Examination of the Order of Incubation for the Recovery of
Bacteria and Fungi from Pharmaceutical Cleanrooms, International Journal of
Pharmaceutical Compounding, 18 (3): 242 – 247
15. Sandle, T. (2016) Pharmaceutical Microbiology: Essentials for Quality Assurance
and Quality Control, Woodhead Publishing, Cambridge, UK, pp15-23
16. Sutton, S. (2005) Validation of Alternative Microbiology Methods for Product
Testing: Quantitative and Qualitative Assays. Pharmaceutical Technology. 29(4): 118-122
17. Miller, M.J. (2015) Rapid Methods Update: Revisions to a United States
Pharmacopeia Chapter. European Pharmaceutical Review. 20(4): 38-43
18. Eisenhart, C. and P. Wilson. (1943) Statistical Methods and Control in
Bacteriology. Bact Rev. 7:57-137
19. Miller, M.J. A (2015) Fresh Look at USP <1223> Validation of Alternative
Microbiological Methods and How the Revised Chapter Compares with PDA TR33 and
the Proposed Revision to Ph. Eur. 5.1.6. American Pharmaceutical Review. 18(5): 22-35
20. Sandle, T. (2015) Embracing Rapid Microbiology, The Medicine Maker, Issue 12,
pp18-19
21. Sutton, S.V.K. (2005) Validation of Alternative Microbiology Methods for Product
Testing: Quantitative and Qualitative Assays. Pharm Technol. 29(4):118-122
22. Sandle, T., Leavy, C., Jindal, H. and Rhodes, R. (2014) Application of rapid
microbiological methods for the risk assessment of controlled biopharmaceutical
environments, Journal of Applied Microbiology, 116 (6): 1495-1505
23. USP (2015) <1223> Validation of Alternative Microbiological Methods. United
States Pharmacopeial Convention. USP 38/NF33:1439