Kimori et al.

BMC Bioinformatics 2010, 11:373

http://www.biomedcentral.com/1471-2105/11/373

METHODOLOGY ARTICLE Open Access

Extended morphological processing: a practical

method for automatic spot detection of
biological markers from microscopic images
Yoshitaka Kimori1,2,3, Norio Baba4, Nobuhiro Morone2,5*

Abstract
Background: A reliable extraction technique for resolving multiple spots in light or electron microscopic images is
essential in investigations of the spatial distribution and dynamics of specific proteins inside cells and tissues.
Currently, automatic spot extraction and characterization in complex microscopic images poses many challenges to
conventional image processing methods.
Results: A new method to extract closely located, small target spots from biological images is proposed. This
method starts with a simple but practical operation based on the extended morphological top-hat transformation
to subtract an uneven background. The core of our novel approach is the following: first, the original image is
rotated in an arbitrary direction and each rotated image is opened with a single straight line-segment structuring
element. Second, the opened images are unified and then subtracted from the original image. To evaluate these
procedures, model images of simulated spots with closely located targets were created and the efficacy of our
method was compared to that of conventional morphological filtering methods. The results showed the better
performance of our method. The spots of real microscope images can be quantified to confirm that the method is
applicable in a given practice.
Conclusions: Our method achieved effective spot extraction under various image conditions, including aggregated
target spots, poor signal-to-noise ratio, and large variations in the background intensity. Furthermore, it has no
restrictions with respect to the shape of the extracted spots. The features of our method allow its broad
application in biological and biomedical image information analysis.

Background microscopic images have a low signal-to-noise ratio

Biological imaging such as confocal fluorescence micro- (SNR) and the differences in intensity between signal
scopy and electron microscopy require the use of pro- spot and background are not always clear. Moreover,
tein-labeling techniques to localize individual proteins the texture of those backgrounds is complicated. For
within cells. Biological markers such as green fluores- these reasons, microscopy images are often difficult to
cence protein [1] and a variety of fluorescent dyes [2,3] manage computationally. Currently, there are several
for fluorescence microscopy, and colloidal gold [4,5] for automatic processing and recognition systems for biolo-
electron microscopy are widely used. Molecules labeled gical images and they have been applied in the quantita-
with biological markers are generally observed as small tive analysis of biological objects ranging from
specific spots against a background of high brightness. molecules to cells to whole organisms [6-10].
Quantitative comprehension of the localization and sta- The purpose of this study was to extract and charac-
tistical distribution of the spots are essential for deci- terize biological spots of intricate morphology and low
phering biological information. In general, cellular contrast in an automatic manner. Current standard
techniques for spot extraction consist of edge enhance-
* Correspondence: morone@icems.kyoto-u.ac.jp ment for image morphology, including discrete convolu-
2
Department of Ultrastructural Research, National Institute of Neuroscience, tion by a high-pass mask and the use of first- or
National Center of Neurology and Psychiatry, Ogawahigashi-cho 4-1-1,
Kodaira, Tokyo, 187-8502, Japan
second-order differential operators, based on the

© 2010 Kimori et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons
Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in
any medium, provided the original work is properly cited.
Kimori et al. BMC Bioinformatics 2010, 11:373 Page 2 of 13
http://www.biomedcentral.com/1471-2105/11/373

magnitude of the spatial differences of the spots [11]. extraction, conventional morphological processing is not
One major problem with this approach, however, results effective.
from the blurring and degradation of the image contrast Advanced morphological processing with multiple SEs
during image acquisition. For some spots with weak has been reported [35,36]. In this approach, multiple sets
contrast, edge extraction is not sufficient. In real-world of line-segment SEs generated by rotation of the single
applications, most biological images contain object line-segment SE in different directions are applied. How-
boundaries, artifacts, and noise. Therefore, edge ever, in the discrete space of the images, it is difficult to
enhancement filters may cause difficulties in distinguish- generate a straight line-segment as the SE that can be
ing the exact edge of the object’s structure from artifacts rotated in an arbitrary direction. This restriction in the
such as trivial geometric features. Additionally, these rotational direction of the SE prevents adequate spot
techniques can amplify background noise in the image detection in complicated biomedical images.
while enhancing the object edge [12,13]. In this study, we solved these problems by introducing
In other methods based on conventional frequency- a simple and practical approach, an extended mathema-
selective filters [14-18], the precise localization of low- tical morphology, into the automatic detection of spots
contrast spots may not be possible. High-density areas in biological images. This technique is based on top-hat
resulting from the integration of many spots may not transformation with a single SE as the straight line-seg-
allow the isolation of individual spots through fre- ment. In our algorithm, an original image can be rotated
quency-selective filters. In addition, the parameter set- in arbitrary directions with respect to the single SE. This
tings are often so complex as to require their novel method, which we named rotational morphologi-
modification whenever the target spot images are chan- cal processing (RMP), can homogeneously treat with
ged [19,20]. Furthermore, these methods cannot deal geometrical features in an image under various orienta-
with the varied morphology of the spots. tions. Top-hat transformation based on RMP has been
Spot extraction methods based on conventional math- applied to spot extraction, in the absence of any hypoth-
ematical morphology [21] effectively capture the spots’ eses to fit the spots by 2-D Gaussian shape or minimal
location and their shape information [22-26]. These intensity. Finally, by isotropic processing with the line-
methods employ a morphological algorithm for back- segment SE, contiguous spots can be segmented into
ground subtraction known as the top-hat transformation individual parts. Our novel method was developed in
[27] or rolling-ball transformation [28]. It is well recog- order to automatically extract spots, such as biological
nized that the principle of these methods is very effec- markers consisting of antibodies conjugated with fluor-
tive for extracting a target object from a wide variety of escent molecules, from a biological image of intricate
image types [29-34]. morphology and low contrast.
Morphological operations use small synthetic images Smal et al. evaluated the spot detection methods most
called structuring elements (SEs), which are a funda- frequently used in fluorescence microscopy [37], includ-
mental tool in mathematical morphology. The SE used ing wavelet-based multiscale detecting [16,18], morpho-
as a probe moves along each pixel of the image. To logical based methods [23,24,38], and the machine
apply morphological filtering for spot extraction from learning method [39]. In this study, we compared the
various types of biological images, the procedure to performance of our proposed method with other mor-
determine the shape and size of the SE is very impor- phological based methods, such as conventional top-hat
tant. A commonly used SE shape is the square or disk. transformation and h-dome transformation, by using
In the rolling-ball transformation, a ball-shaped SE synthetic-noise images.
(such as a disk SE with weights arranged in order to This report is organized as follows. A brief introduc-
describe a hemisphere in gray scales) is used. In the tion describing the basics of conventional mathematical
above-described methods for spot extraction, these SEs morphology is followed, in the Methods section, by a
were also used. However, most small contiguous spots detailed presentation of our spot extraction technique.
cannot be individually distinguished, such that several In the Results section, the application of the detection
spots are extracted as one connected region because the method to synthetic images as well as to real image data
size (width) of the SEs is wider than the minimum dis- from electron and fluorescence micrographs is dis-
tance between the peaks of adjacent spots. A suitable SE cussed. In the final section, the effectiveness of our
shape for spot extraction includes a straight-line seg- novel method is summarized and evaluated.
ment (a fuller description of which is given in the Meth-
ods and Results sections); however, since processing by Conventional mathematical morphology
common morphological operations with a single line- Mathematical morphology is based on set-theory con-
segment SE is not isotropic, it cannot consider the geo- cepts of the shape of an objective image [21]. An image
metrical details of an intricate image. Thus, for spot can be represented by a set of pixels. Morphological
Kimori et al. BMC Bioinformatics 2010, 11:373 Page 3 of 13
http://www.biomedcentral.com/1471-2105/11/373

operations always deal with a set of two images: an where Dh(f) is the h-dome image of a gray-scale image
objective image and a SE. Each SE has shape and size f, (f-h) represents the result of subtracting a constant
characteristics as parameters of the operation. Let f value h from the gray-scale image, and rf(f-h) the mor-
denote a gray-scale image function from Z2 into [0, I-1], phological reconstruction of the gray-scale image from
where I is a positive integer. Let B denote a binary SE. f-h. The gray-level reconstruction is obtained by iterative
The fundamental operators of mathematical morphology geodesic dilation of f-h under f until stability is reached
are dilation and erosion. [40].
dilation:
Method
 B( f )( x ,y ) = max{ f ( x − s, y − t ) + b( s, t ) | Spot extraction filter: top-hat transformation by RMP
(1)
( x − s, y − t ) ∈ D f ;( s, t ) ∈ D b}, The essential elements required for the spot extraction
filter are processing of the biomedical image isotropi-
erosion: cally and isolation of the adjacent spots from the image
background. In order to fulfill these requirements, our
 B( f )( x ,y ) = min{ f ( x + s, y + t ) − b( s, t ) | method proposes that the objective image can be
(2) rotated in arbitrary directions with respect to a single
( x + s, y + t ) ∈ D f ;( s, t ) ∈ D b}.
straight line-segment SE whose width has only 1 pixel.
where D f and D b are the domains of the functions f The width of this SE ensures determination of the mini-
and B, respectively. The opening and closing operations mum distance between two different spots for isolation.
are delivered from dilation and erosion. By using this SE, spots separated by distances of only 1
opening: pixel can be distinguished individually. The length of
the SE should be adjusted so that it is longer than the
 B( f )( x ,y ) =  B( B( f )( x ,y ) ), (3) size of the target spot. A spot that is smaller than the
length of this SE is extracted by the top-hat
closing: transformation.
This top-hat transformation by RMP with the straight
 B( f )( x ,y ) =  B( B( f )( x ,y ) ). (4) line-segment SE consists of the following steps:
Algorithm 1 (Top-hat transformation by RMP with
The top-hat transformation is one of the commonly line-segment SE)
used morphological operations for extracting local 1. Original image rotation. The original image f (Figure
bright objects from a low contrast image in gray-scale 1a) is rotated in a clockwise direction with respect to
[27]. It is obtained by subtracting from the original the center of the image frame. Assume that dividing a
image f the opening image gB using the SE B. half of the circle (π [rad]) into N equiangles gives us
Top-hat: each direction at an angle of π/N [rad] (Figure 1c),
which is an increment angle. Namely, fi (Figure 1d, top
 B ( f )( x , y ) = f ( x , y ) −  B ( f ) ( x , y ) . (5) row) denotes the rotated image of f with the angle of π
i/N [rad], where i = 0, 1,..., N-1.
It yields an image in which all the residual features 2. Opening. All rotated images are subjected to an
(peaks and ridges) are subtracted by the opening opera- opening operation with the straight line-segment SE B
tion. Adding these residual features to the original (Figure 1b). The opening operations of the rotated image
images has the effect of accentuating objective struc- fi are represented as gB(fi) (Figure 1d, middle row).
tures with high intensity [27]. If the difference in inten- 3. Opened image rotation. The opened images (gB(fi))
sity between the target objects and the background of are rotated π·i/N [rad] in an anticlockwise direction.
the image is markedly small, it is difficult to detect The rotation at i times of the opened image is denoted
these differences with the human eye. However, these by hi (Figure 1d, bottom row).
low-contrast objects can be extracted and enhanced by 4. Union of the rotated and opened images (opening by
the top-hat transformation. RMP). The processed images (hi) are unified. In union
Another method to extract the local bright object in processing, the maximum intensity value, which corre-
biological images, based on mathematical morphology, sponds to the same pixel coordinate among all opened
is the h-dome transformation [38]. images, is taken to generate the whole image.
h-dome: 5. Top-hat transformation by RMP. The unified
opened image (g’ B (f)) is subtracted from the original
D h ( f )( x ,y ) = f( x ,y ) −  f ( f( x ,y ) − h). (6) image (f).
Kimori et al. BMC Bioinformatics 2010, 11:373 Page 4 of 13
http://www.biomedcentral.com/1471-2105/11/373

Figure 1 Schematic procedure of rotational mathematical morphology. (a) Original image. (b) Straight line-segment SE B. (c) Setting of the
rotation angles. (d) Process of RMP opening. Original image is rotated and each rotated image is processed by opening with the straight line-
segment SE B. Orange arrows in each rotated image denote the scan lines by this SE. This rotational processing permits the isotropic processing
with a single straight line-segment SE. All processed images (hi) are unified as a result of the RMP opening.

Figures 1a and 1b show the original image f and the The entire practical process consists of the following
straight line-segment SE B, respectively. Figure 1c illus- steps:
trates a setting of the rotation angles. The process in Algorithm 2 (Spot extraction for practical biological images)
the RMP opening with the straight line-segment SE is 1. Noise reduction: Noise, which is less than the resolu-
represented in Figure 1d. RMP opening is defined as fol- tion limit of the micrograph or target spot, is removed
lows: via opening by RMP (equation (7)) with the straight
line-segment SE. The length of the SE is set to be smal-
 B’ ( f )( x ,y ) = max {h i ( x , y)}. (7) ler than the diameter of the target spots.
i∈(0,1,..., N −1)
2. Spot extraction: The spots are extracted by the top-
The image g’B( f )(x, y) is defined as the maximum value hat transformation by RMP (equation (8)) with the
of h0 (x, y), h1 (x, y),..., hN-1(x, y). The top-hat transforma- straight line-segment SE. The length of SE is set to be
tion by RMP is also given by the following equation: larger than the diameter of the target spots.
3. Binarization: The extracted spots are binarized by
 ′B( f )( x ,y ) = f( x ,y ) −  ′B( f )( x ,y ). (8) equation (9) for recognition and measurement of the
spots computationally.
This operation is used as the spot extraction filter in Binarization is performed according to the threshold-
our proposed method. ing approach:
In this study, we utilized the top-hat transform by
RMP for spot extraction from biological images ⎪⎧ 255,  ′B( f )( x , y ) > 0
g ( x , y) = ⎨ . (9)
obtained with electron and fluorescence microscopy. ⎩⎪ 0, Otherwise
Kimori et al. BMC Bioinformatics 2010, 11:373 Page 5 of 13
http://www.biomedcentral.com/1471-2105/11/373

Namely, the pixels of residual regions by the top-hat opening by RMP with BL (i.e., top-hat transformation by
transformation by RMP are assigned an intensity of 255 RMP) is shown in Figure 2b. The result of conventional
in an 8-bit gray-scale value. top-hat transformation with BD of f is shown in Figure
In several cases, the subtraction process and the binar- 2c. In Figure 2d, the black line is the profile of the spots
ization process leave the small isolated pixels on the image f (the position of the profile line is marked by the
image such that they represent residual background arrowhead in Figure 2a), which denotes the surface of
noise. The conventional opening operation (equation the spots. The red line is the result of the opening by
(3)) can also be applied to remove the remaining noise. RMP with B L of f (where the value of the rotational
This post-processing should be adapted to the particular direction N is 36). The green line is the result of the
application. conventional opening with BD. Figure 2b shows that the
two overlapped spots were segmented clearly by our
Results method.
Isolation of overlapped spots The opening operation can be geometrically processed
The performance of our proposed method in the isola- by pressing the SE up against the surface of the original
tion of overlapped spots was compared with that of con- image f and sliding it underneath the entire surface. The
ventional top-hat transformation. Figure 2a shows the surface of the opened image is constructed from the
model image of adjacent spots ( f ). On the left of Figure highest point in the region reached by the SE. Since the
2a is the original gray-scale image f and on the right is line-segment SE BL has a width of 1 pixel, in the process
the 3-D topographic map of f. The vertical height of the of RMP opening the SE fits the narrow intervening
map represents the intensity. In top-hat transformation, space between the spots. Thus, line-segment SE reaches
the original image f is first opened and then this opened the baseline of the individual spots (i.e., the upper level
surface is subtracted from the original surface. In this of the overlapped region). In contrast, the disk SE BD
example, the straight line-segment SE BL (11 × 1 pixels) has a width of 11 pixels as diameter. Since it is larger
was used in our method, and the flat disk SE BD (the than the intervening space of the spots, the SE cannot
diameter was 11 pixels) was used in the conventional fit the space. Thus, the disk SE cannot reach the level at
top-hat transformation. The difference between f and its which the two spots are distinguished. Accordingly, the

Figure 2 Isolation of overlapped spots. (a) Original image of two overlapped spots. Left: original gray-scale image, right: the 3-D map. (b) The
result of the top-hat transformation by RMP with the straight line-segment SE BL (11 × 1 pixels). (c) The result of the conventional top-hat
transformation with the flat disk SE BD (diameter: 11 pixels). The actual length of 11 pixels is shown in (a). The gray-scale images in (b) and (c)
were enhanced by linear contrast stretching. (d) The intensity profiles of the original spots (black line), the result of RMP opening with the
straight line-segment SE BL (red line) and of conventional opening with the flat disk SE BD (green line). The position of the profile line is marked
by an arrowhead in (a). Residues after subtraction of f-g ’BL( f ) and of f-gBD ( f ) correspond to (b) and (c), respectively. The horizontal black dotted
line in (d) denotes the baseline, which is the limit to distinguish between these two spots. The gray region below the baseline is the overlapped
region of these spots. To isolate these two spots, the top-surface of the opened image should surpass the baseline.
Kimori et al. BMC Bioinformatics 2010, 11:373 Page 6 of 13
http://www.biomedcentral.com/1471-2105/11/373

spots were not isolated by the conventional top-hat PSNR with N, which ranged from 0 to 90 directions.
transformation. The range of small values of N (from 0 to 8) is
enlarged in the insertion. The black line denotes the
Verification of optimal number of rotational direction result of this operation. The highest value of PSNR
The optimal number of the image rotational directions was obtained at N = 36 and it subsequently remained
(N) was determined. For this experiment, we used a at almost the same level. In terms of the performance
model image containing several artifacts. The image was of reconstruction and the computational cost of pro-
reconstructed by RMP opening with the straight line- cessing, it is reasonable to assume that the optimal
segment SE set at various values for the rotational number of image rotational directions is 36.
direction. Peak signal-to-noise ratio (PSNR) was used to In addition to this experiment, performance was also
provide a quantitative evaluation of this performance. tested using the multiple SEs method, shown as a red
Assuming pixel values in the range of [0: 255], the line in Figure 3e. The straight line-segment SE was used
PSNR is calculated using the following formula: for these tests. In the case of N = 1, the single SE
(orientation θ = 0 [rad], horizontal direction) was
255 2 applied; at N = 2, two SEs (θ = 0 and π/2 [rad]) were
PSNR = 10 log 10( ).
1 (10)
m1 m 2
∑i =1 ∑ j =1[F(i, j)− F ′(i, j)]
2 used, and at N = 4, four SEs (θ = 0, π/2, π/4 and 3π/4
m1⋅m 2
[rad]) were used. These results showed that reconstruc-
where F(i, j) denotes the original image, F ’(i, j) the fil- tion by the multiple SEs was insufficient.
tered image by RMP opening, and m 1 × m 2 the total
number of pixels. Noise reduction
Figures 3a and 3b show the original 8-bit gray-scale The noise reduction technique in our method (in algo-
image (256 × 256 pixels) and the model image over- rithm 2), i.e., RMP opening with line-segment SE, was
lapped by artifacts, respectively. Some of the round used. Figure 4a shows the spot model with background
objects (diameter: 9 pixels) are regarded as artifacts to noises. These noises were removed by RMP opening
be removed. The RMP opening with the straight line- (Figure 4b). The length of the line-segment SE was
segment SE (31 × 1 pixels) of N = 8 allowed removal determined from the spatial resolution of the micro-
of the artifacts but also of some of the elongated target scopic image, with objects having a shape size smaller
objects (Figure 3c). At N = 36, artifacts were removed than the resolution limit regarded as noise. If the spot
while the other objects were well preserved (Figure size is known in advance, the SE length should be set to
3d). The graph in Figure 3e shows the variation of remove objects smaller than the spot size.

Figure 3 Verification of the optimal number of rotational directions. (a) Original image (256 × 256 pixels). (b) Artifact-contaminated image.
Some round objects (diameter: 9 pixels) are denoted as artifacts. The image was restored by RMP opening with the straight line-segment SE (31
× 1 pixels) (c) Restored image when N (rotational direction number) is 8. (d) Restored image when N is 36. (e) PSNR [dB] results of the restored
image of (b) by different N. The red and black lines show the result of the conventional multiple SEs and our proposed methods, respectively.
Kimori et al. BMC Bioinformatics 2010, 11:373 Page 7 of 13
http://www.biomedcentral.com/1471-2105/11/373

recall and precision rate and takes into account both

measures:

2⋅recall ⋅precision
F − measure = . (11)
recall + precision

In each image type, the synthetic images had a total of

150 spots; thus, TP + FN = 150. The results detected by
our proposed method were compared with those of the
conventional top-hat and h-dome transformations.
Figure 4 Noise reduction through RMP opening. (a) Spot model
image with background noises. (b) Noises removed from the image
In this experiment, noise-added images were first
by RMP opening with the straight line-segment SE. smoothed with the Gaussian filter (3 × 3 kernel).
Processing by our proposed method followed algo-
rithm 2. Since the width of the spot domain was 17 pix-
Comparison with conventional morphological methods els (Figure 5b), any structure smaller than this width
by using synthetic-noise images was regarded as noise or artifact. In step 1, the straight
The ability of the proposed method was compared with line-segment SE of 13 × 1 pixels was used, and in step
that of conventional morphological methods, such as 2, the straight line-segment SE of 21 × 1 pixels. In both
the conventional top-hat and h-dome transformations. steps, the rotational direction number (N) of RMP pro-
Synthetic-noise images were generated to evaluate the cessing was 36. The subtracted image was binarized by
quantitative performance of spot detection. Ten source the method in step 3.
images of size 512 × 512 pixels were generated, with For the method based on the conventional top-hat
one such example shown in Figure 5a. Each image con- transformation, the processing in step 2 differs from
tained 15 spots, which were modeled as a 2-D Gaussian that of the proposed method. Top-hat transformation
distribution in random locations. This distribution is a was applied with disk SE (diameter: 21 pixels) and the
good approximation of the theoretical shape of the spot subtracted image was binarized by the method in step 3.
[41]. Two types of spot models with different intensity For the method based on h-dome transformation, this
levels (255 or 127 in an 8-bit gray-scale) were generated transformation was applied to the smoothed image
(Figure 5b). First, a 2-D Gaussian distribution with ker- obtained in step1 of algorithm 2. We set the value of
nel width = 17 (pixels) and standard deviation s = 2 parameter h to 50, and the subtracted image (h-dome)
(pixels) was generated. This distribution was then was binarized by the method in step 3.
expanded to the normalized gray-scale range of 0-255. Finally, binary images obtained from these methods
For the second spot model, the distribution was were cleaned by the conventional opening with disk SE
expanded to the range of 0-127. (diameter: 13 pixels) as post-processing.
The Poisson-distributed noise was added to each The results of the spot extraction are seen in Figure
source image with a uniform (type-A) and a gradient 5c. The type-A and type-B images are shown in the top
(type-B) background (Figure 5c). This is one of the main and bottom row, respectively. All spots were correctly
sources of noise in fluorescence microscopy imaging detected by our proposed method but not by the other
[42]. methods. Figure 5d shows the actual values of recall,
The PSNR between each source image and the corre- precision, and F-measure for each type image set. The
sponding noise-added image was calculated. The average performance in terms of the F-measure for the proposed
value of the PSNR of the type-A image set was 13.848 ± method was consistently 100% in all type images. Thus,
0.242 (mean ± SD) dB, while for the type-B image set it the proposed method is more tolerant of Poisson noise
was 11.245 ± 0.169 dB. images than other methods.
Since the spots in the synthetic image are prospec- In addition, the performance of our method was tested
tively known, the recall, precision, and F-measure rates using synthetic images under various noise levels. For the
can be calculated. Let TP denote the number of spots source image (Figure 5a), nine degrees of Poisson-distrib-
detected correctly, FN the number of spots missing uted noise images with uniform background were gener-
detection, and FP the number of spots false alarmed. If ated, with the PSNR decreasing from 17.732 to 7.797 dB.
adjacent spots are not separated, those spots are The upper part of Figure 5e shows the synthetic images,
regarded as FN. The recall and precision rates are captured only in the rectangular region in Figure 5a. The
defined by the equations TP/(TP+FN) and TP/(TP+FP), proposed method was applied to these synthetic images
respectively. The F-measure rate is the harmonic mean with the same procedure and SEs as in the previous
Kimori et al. BMC Bioinformatics 2010, 11:373 Page 8 of 13
http://www.biomedcentral.com/1471-2105/11/373

Figure 5 Experimental results using synthetic images. (a) Example of source image with an 8-bit gray-scale. (b) 2-D Gaussian distributions as
spot models. Two spot models with different intensity levels were used in this experiment. The actual image of the spot models (cropped from
the white dotted rectangular region of (a)) are shown on the left, and the intensity profiles on the right. (c) Example results of spot detection
using synthetic noise images with a uniform (type-A) and a gradient (type-B) background. The performance of the proposed method was
compared with that of conventional top-hat (TH) and h-dome (HD) transformation. (d) The recall, precision, and F-measure rates of this
experiment. (e) The performance of our method when synthetic images with various noise levels are used.

experiment. The bottom of Figure 5e shows that the extraction. Electron micrographs containing colloidal
scores for the recall, precision, and F-measure rates were gold particles were used as test images, with two differ-
consistently 100% in the PSNR range of 17.732 to 10.057 ent sizes of particles (10 nm and 1.8 nm in diameter).
dB. Subsequently, the recall rate decreased with decreas- Figure 6a (left) shows a part of the original micrograph
ing PSNR although the precision rate remained at 100%. with 10-nm-diameter gold particles (British BioCell).
This indicates that FP was zero and ensures the accuracy The spatial resolution was 0.90 nm/pixel. Figure 6b
of our method for spot detection. (left) shows a part of the original micrograph with 1.8-
nm-diameter gold particles (Nickel (II)-Nitrilotriacetica-
Spot extraction of colloidal gold particles cid-Nanogold, Nano Probes). The spatial resolution of
The proposed method was applied to electron micro- this image was 0.32 nm/pixel. These images were on an
scopic images to evaluate its performance in spot 8-bit gray-scale, the intensity values of the routine
Kimori et al. BMC Bioinformatics 2010, 11:373 Page 9 of 13
http://www.biomedcentral.com/1471-2105/11/373

Figure 6 Spot extraction of colloidal gold particles from electron microscopy images. (a) Original micrograph of 10-nm-diameter gold
particles (left). Spot-extracted image (right). The extracted spots are denoted in red regions. Bar: 200 nm. (b) Original micrograph of 1.8-nm-
diameter gold particles (left). Spot-extracted image (right). Scale bar: 10 nm.

image quality were inverted. The purpose of this test diameter of the extracted spots was consistent with the
was to verify whether the diameter of the extracted nominal diameter.
gold-particle spot as determined by our method was
consistent with the nominal diameter. Spot extraction of fluorescent antibodies
Processing of the spot extraction was carried out with Our proposed spot extraction method was applied to
algorithm 2. In the case of the micrograph containing fluorescence microscopy images in which caveolin-1
10-nm gold particles, line-segment SE with a size of 5 × molecules in fibroblasts were stained with secondary
1 pixels (4.5 × 0.9 nm) and 13 × 1 pixels (11.7 × 0.9 fluorescent antibodies (Figure 7a). This image had an 8-
nm) was used in noise reduction and spot extraction, bit gray-scale and a spatial resolution of 60 nm/pixel.
respectively. The extracted spots were binarized and Line-segment SE with a size of 3 × 1 pixels (180 × 60
overlaid as seen in the red region on the original image. nm) and 7 × 1 pixels (420 × 60 nm) was used for noise
The result is shown on the right side of Figure 6a. For reduction and spot extraction, respectively.
the electron micrograph containing the 1.8-nm gold par- By visual observation, the diameter of each spot was
ticles, line-segment SE with a size of 3 × 1 pixels (0.96 × found to be about 5 pixels (ca. 300 nm). Therefore, a SE
0.32 nm) and 7 × 1 pixels (2.24 × 0.32 nm) was used for length longer than the diameter of the target spots was
noise reduction and spot extraction, respectively. The selected for the spot extraction process. Binarization was
result is shown on the right side of Figure 6b. For all carried out using equation (9). The regions of the
extracted particles, the averaged Feret’s diameter was extracted spots were superimposed on the original
calculated. The mean ± SD of the diameter of the image as red-colored regions (Figure 7b). From this
extracted spots was 10.16 ± 0.77 nm (164 spots) for the micrograph, the 627 spots were extracted. The quantita-
10-nm gold particles, and 1.81 ± 0.15 nm (813 spots) tive estimation of the bright spots corresponding to the
for the 1.8-nm gold particles. This result shows that the caveolae is provided in Figure 7c. The sum of the
Kimori et al. BMC Bioinformatics 2010, 11:373 Page 10 of 13
http://www.biomedcentral.com/1471-2105/11/373

Figure 7 Quantitative spot extraction of fluorescent antibodies from a fluorescence microscopy image. (a) Original image. Fluorescence
micrograph of caveolin-1 molecules stained with the primary antibody and a secondary fluorescent antibody in normal fibroblasts. Scale bar: 10
μm. (b) The results of spot extraction using the proposed method. These spots were superimposed on the original image as red regions. (c) The
histogram of the intensity as estimated from the extracted spots.

intensity of the pixel values in each extracted spot Discussion and conclusions
region was calculated and the distribution depicted in a Our novel method to extract the spots in electron and
histogram. The total number of extracted spots was fluorescence microscopic images uses the extended mor-
6701 from five micrographs (which included approxi- phological filter through the top-hat transformation by
mately 5 cell regions). The median of the histogram was RMP. We have successfully shown that the method is
1319. The largest cluster can be seen centered at the useful for extracting spots in biomedical images in
histogram’s median value. which the conventional method is inadequate. The key
As shown in Figure 8, the proposed method was concepts of our spot extraction method are the use of a
applied to the various shapes of the fluorescent spots straight line-segment SE and the rotation of the original
(top row, original images): (a) Gaussian-like shape (low image. By changing the length of SE, target spots of var-
peak height), (b) Gaussian-like shape (high peak ious sizes can be extracted. The method avoids the tech-
height), (c) irregular shape, and (d) volcano shape. The nical difficulties of traditional morphological processing
3-D maps of the original images are shown in the mid- and its performance is robust in the processing of bio-
dle row, which illustrates the relief of the pixel surface. medical images. The main advantages of our method
The 3-D maps of the extracted spots, as determined by are that it is computationally simple and easily modified
our method, are shown in the bottom row. All spots for the extraction of target spots of different sizes and
were well extracted even though they had intricate shapes, and that it can handle images in various condi-
shapes. tions, e.g., aggregated target spots, poor SNR, and a
Kimori et al. BMC Bioinformatics 2010, 11:373 Page 11 of 13
http://www.biomedcentral.com/1471-2105/11/373

Figure 8 Precise extraction of various shaped spots. Examples of the four shapes of fluorescence spots from micrograph are shown in the
top row. (a) Gaussian-like shape (low peak height), (b) Gaussian-like shape (high peak height), (c) irregular shape, and (d) volcano shape. The 3-D
maps of the original images and of the extracted spots using our proposed method are shown in the middle and bottom rows, respectively.

background with large variations in intensity. The a better trade-off because the value of PSNR was low
method yields directional information regarding the spa- for N < 36 while for N > 36 the processing time became
tial distribution of spots within the cell as well as the longer. In our method, a large computational cost,
frequency distribution of the size and intensity of the which is proportional to the size of the input images, is
spots. inevitable.
Our method is based on a line-segment SE with a 1- We compared our spot extraction method with the
pixel width as the minimum separation distance and conventional top-hat and h-dome transformations. As
therefore allowed two or more target spots located close seen in Figure 5, our method outperformed the others
to each other to be clearly distinguished (Figure 2). with respect to the three criteria (Figure 5d). For the
With conventional morphological top-hat transforma- proposed method, the performance in terms of F-mea-
tion using the common SE shape (such as a disk or sure rate was maintained at 100% among all background
square), it is difficult to separate such spots. A similar typed images. The precision rate of the conventional
difficulty arises when the “ball” SE is used. Since it has a top-hat transformation was much lower due to its
radius that is larger than the inter-space distance higher FP value (124 in type-A and 228 in type-B,
between adjacent spots, it cannot fit within the space. respectively) in detection of the noise. Furthermore,
The top surface obtained during opening with the roll- conventional top-hat transformation could not separate
ing ball cannot reach the baseline allowing for separa- adjacent spots, as it used the disk SE. Meanwhile, the
tion of the spots. recall rate of the h-dome transformation was much
To verify the optimality of the number of rotational lower due to its higher FN value (113 in type-A and 28
directions (N) shown in Figure 3, we investigated how in type-B, respectively). Thus, the number of undetected
an artifacts-contaminated image (Figure 3b) could be true spots was large.
restored by the RMP opening with increasing N. The We further investigated the change in the three mea-
experimental result (Figure 3e) showed that N = 36 was surements as a function of decreasing PSNR from
Kimori et al. BMC Bioinformatics 2010, 11:373 Page 12 of 13
http://www.biomedcentral.com/1471-2105/11/373

17.732 to 7.797 dB (Figure 5e). The F-measure rate was appreciation to Prof. E. Katayama (University of Tokyo) and Dr. K. Aoyama
(FEI Japan). This work was supported in part by a Health Labor Science
maintained at 100% until PSNR decreased to about 10 Research Grant (Nano-001) and Grants-in-Aid for Scientific Research from
dB. Subsequently, when PSNR decreased further, the F- the Ministry of Education Culture, Sports, Science and Technology to N.
measure rate decreased as well due to a decreasing Morone.
recall rate (thus, increasing the FN value); however, the Author details
precision rate was constantly 100%. These results 1
Japan Association for the Advancement of Medical Equipment, Hongo 3-
showed that our method is accurate in spot detection. 42-6, Bunkyo-ku, Tokyo, 113-0033, Japan. 2Department of Ultrastructural
Research, National Institute of Neuroscience, National Center of Neurology
In the measurement of gold particles in the electron and Psychiatry, Ogawahigashi-cho 4-1-1, Kodaira, Tokyo, 187-8502, Japan.
micrograph (Figure 6), the value of the averaged Feret’s 3
Center for Novel Science Initiatives, National Institutes of Natural Sciences,
diameter of the extracted spots and the value of the Toranomon 4-3-13, Minato-ku, Tokyo, 105-0001, Japan. 4Faculty of
informatics, Kogakuin University, Nishi-shinjuku 1-24-2, Shinjuku-ku, Tokyo,
nominal diameter of the gold particles were in close 163-8677, Japan. 5Institute for Integrated Cell-Material Sciences (iCeMS),
agreement. Thus, our method effectively extracted spots Kyoto University, Yoshidahonmachi Sakyo-ku, Kyoto, 606-8085, Japan.
of the specified size with high accuracy.
Authors’ contributions
Figure 7 shows the location of the small spots in the YK conceived the study, developed the algorithms, carried out the testing
cell and the estimation of the spots intensities. Pre- and fine-tuning of the algorithms using the programming language C/C + +,
viously, Orlichenko reported that stimulation of epithe- and wrote the first draft of the paper. NB provided useful comments on
methodology and helped to revise the manuscript. NM carried out all of the
lial cells with epithelial growth factor (EGF) resulted in LM, TEM, and molecular biology experiments, supervised the work, and edited
a profound increase in the number of caveolar struc- and revised the manuscript. All authors read and approved the final
tures at the plasma membrane [43]. Our method was manuscript.
able to carry out precise quantitative measurements of Received: 6 November 2009 Accepted: 8 July 2010
the spatial and intensity distributions of the membrane Published: 8 July 2010
domain with respect to caveolae.
Furthermore, our method allows effective extraction of References
1. Shimomura O, Johnson FH, Saiga Y: Extraction, purification and properties
various shaped spots. Since it is based on the top-hat of aequorin, a bioluminescent protein from the luminous
transformation, the spots are extracted independently in hydromedusan, Aequorea. J Cell Comp Physiol 1962, 59:223-239.
terms of the shape of the surface relief, which is based 2. Paddock S: Over the rainbow: 25 years of confocal imaging. Biotechniques
2008, 44(5):643-644.
on variations in the intensity value within a spot region 3. Giepmans BNG, Adams SR, Ellisman MH, Tsien RY: The fluorescent toolbox
(Figure 8). In the conventional spot detection methods for assessing protein location and function. Science 2006, 312:217-224.
that rely on matched filtering, a 2-D Gaussian distribu- 4. Handley DA: The development and application of colloidal gold as a
microscopic probe. Colloidal Gold: Principles, Methods, and Applications San
tion is commonly used for the matched filter, assuming Diego: Academic PressHayat MA 1989, 1:1-32.
that a point-spread function of a signal spot has a 2-D 5. Hainfeld JF, Powell RD: New frontiers in gold labeling. J Histochem
Gaussian distribution. However, because most spots Cytochem 2000, 48:471-480.
6. Peng H: Bioimage informatics: a new area of engineering biology.
have an irregular topology, as in the example in Figure Bioinformatics 2008, 24:1827-1836.
8, accurate spot extraction is difficult using the matched 7. Bozinov D, Rahnenfuhrer J: Unsupervised technique for robust target
filtering method. separation and analysis of DNA microarray spots through adaptive pixel
clustering. Bioinformatics 2002, 18:747-756.
Signal spots extracted by our method can be trans- 8. Rahnenfuhrer J, Bozinov D: Hybrid clustering for microarray image
formed into a 2-D Gaussian distribution as a normaliza- analysis combining intensity and shape features. BMC Bioinformatics 2004,
tion of spot shape. This allows the application of our 5:47.
9. Mete M, Hennings L, Spencer HJ, Topaloglu U: Automatic identification of
method to the conventional automatic tracking system angiogenesis in double stained images of liver tissue. BMC Bioinformatics
of individual fluorescent particles [44]. 2009, 10(Suppl 11):S13.
The RMP-based method enables a shape and intensity 10. Berth M, Moser FM, Kolbe M, Bernhardt J: The state of the art in the
analysis of two-dimensional gel electrophoresis images. Appl Microbiol
analysis for various types of biomedical images (2-D gel Biotechnol 2007, 76:1223-1243.
electrophoresis image, DNA microarray image, electron 11. Gonzalez RC, Woods RE: Digital Image Processing. New Jersey: Prentice
micrograph, X-ray mammographic image, etc.). It can Hall, 3 2008.
12. Suri JS, Wilson DL, Laxminarayan S, Eds: Handbook of Biomedical Image
be applied not only to spot extraction but also to a wide Analysis: Volume 2: Segmentation Models Part B. New York: Springer-
variety of important image processing techniques, such Verlag 2005.
as segmentation, smoothing, and pattern extraction [45]. 13. Bankman IN, Ed: Handbook of Medical Imaging: Processing and Analysis.
San Diego: Academic Press 2000.
Overall, it provides a wide-ranging analytical approach 14. van der Heijden F, Apperloo W, Spreeuwers LJ: Numerical optimisation in
to biological and biomedical informatics. spot detector design. Pattern recognition letters 1997, 18:1091-1097.
15. Blanford RP, Tanimoto SL: Bright-spot detection in pyramids. Comput Vis
Graph Image Process 1988, 43:133-149.
Acknowledgements 16. Olivo-Marin J-C: Extraction of spots in biological images using multiscale
We thank Dr. S. Yuasa (National Center of Neurology and Psychiatry) for products. Pattern Recognition 2002, 35:1989-1996.
helpful discussions and encouragement, Prof. T. Kodama (Osaka University) 17. Genovesio A, Liedl T, Emiliani V, Parak WJ, Coppey-Moisan M, Olivo-Marin J-
for continuing guidance and encouragement. We also express our C: Multiple particle tracking in 3-D + t microscopy: Method and
Kimori et al. BMC Bioinformatics 2010, 11:373 Page 13 of 13
http://www.biomedcentral.com/1471-2105/11/373

application to the tracking of endocytosed quantum dots. IEEE Trans 43. Orlichenko L, Huang B, Krueger E, McNiven MA: Epithelial growth factor-
Image Process 2006, 15:1062-1070. induced phosphorylation of caveolin 1 at tyrosine 14 stimulates
18. Zhang B, Fadili J, Starck J-L, Olivo-Marin J-C: Multiscale variance-stabilizing caveolae formation in epithelial cells. J Biol Chem 2006, 281:4570-4579.
transform for mixed-Poisson-Gaussian processes and its applications in 44. Cheezum MK, Walker WF, Guilford WH: Quantitative comparison of
bioimaging. Proceedings of the IEEE International Conference on Image algorithms for tracking single fluorescent particles. Biophys J 2001,
Processsing 2007, 6:VI-233-VI-236. 81:2378-2388.
19. Zeng L, Wu W: Motion Objects Detection Based on Wavelet Clustering. 45. Kimori Y, Oguchi Y, Ichise N, Baba N, Katayama E: A procedure to analyze
Proceedings of 2nd IEEE International Conference on Computer Science and surface profiles of the protein molecules visualized by quick-freeze
Information Technology 2009, 562-566. deep-etch replica electron microscopy. Ultramicroscopy 2007, 107:25-39.
20. Bao W, Zhou R, Yang J, Yu D, Li N: Anti-aliasing lifting scheme for
mechanical vibration fault feature extraction. Mech Syst Signal Process doi:10.1186/1471-2105-11-373
2009, 23:1458-1473. Cite this article as: Kimori et al.: Extended morphological processing: a
21. Serra J: Image analysis and mathematical morphology. London: Academic practical method for automatic spot detection of biological markers
Press 1982. from microscopic images. BMC Bioinformatics 2010 11:373.
22. Meyer F: Iterative image transformations for an automatic screening of
cervical smears. J Histochem Cytochem 1979, 27:128-135.
23. Bright DS, Steel EB: Two-dimensional top hat filter for extracting spots
and spheres from digital images. J Microsc 1987, 146:191-200.
24. Breen EJ, Joss GH, Williams KL: Locating objects of interest within
biological images: The top hat box filter. J Comput-Assist Microsc 1991,
3:97-102.
25. Angulo J: Mathematical morphology operators for reading radioactivity
DNA array images. Proceedings of the IASTED International Conference on
Visualization, Imaging, and Image Processing 2004, 802-807.
26. Janson ME, Setty TG, Paoletti A, Tran PT: Efficient formation of bipolar
microtubule bundles requires microtubule-bound g-tubulin complexes. J
Cell Biol 2005, 169:297-308.
27. Meyer F: Contrast features extraction. Quantitative analysis of
microstructures in material sciences, biology and medicine Stuttgart: Rieder
VerlagChermant JL 1977, 374-380.
28. Sternberg SR: Grayscale morphology. Comput Vis Graph Image Process
1986, 35:333-355.
29. Manders EM, Hoebe R, Strackee J, Vossepoel AM, Aten JA: Largest contour
segmentation: a tool for the localization of spots in confocal images.
Cytometry 1996, 23:15-21.
30. Giakoumis I, Nikolaidis N, Pitas I: Digital image processing techniques for
the detection and removal of cracks in digitized paintings. IEEE Trans
Image Process 2006, 15:178-188.
31. Yang Y, Huang S, Rao N: An automatic hybrid method for retinal blood
vessel extraction. Int J Appl Math Comput Sci 2008, 18:399-407.
32. Kapsalas P, Maravelaki-Kalaitzaki P, Zervakis M, Delegou ET, Moropoulou A:
A morphological fusion algorithm for optical detection and
quantification of decay patterns on stone surfaces. Constr Building Mater
2008, 22:228-238.
33. Kim H, Morgan DE, Zeng H, Grizzle WE, Warram JM, Stockard CR, Wang D,
Zinn KR: Breast tumor xenografts: diffusion-weighted MR imaging to
assess early therapy with novel apoptosis-inducing anti-DR5 antibody.
Radiology 2008, 248:844-851.
34. Ji Q, Engel J, Craine E: Texture analysis for classification of cervix lesions.
IEEE Trans Med Imaging 2000, 19:1144-1149.
35. Stevenson RL, Arce GR: Morphological Filters: Statistical and Further
Syntactic Properties. IEEE Trans Circuits and Systems 1987, 34:1292-1305.
36. Song J, Delp EJ: The analysis of morphological filters with multiple
structuring elements. Comput Vis Graph Image Process 1990, 50:308-328.
37. Smal I, Loog M, Niessen W, Meijering E: Quantitative comparison of spot
detection methods in fluorescence microscopy. IEEE Trans Med Imaging
2010, 29:282-301.
38. Vincent L: Morphological grayscale reconstruction in image analysis:
Applications and efficient algorithms. IEEE Trans Image Process 1993, Submit your next manuscript to BioMed Central
2:176-201. and take full advantage of:
39. Jiang S, Zhou X, Kirchhausen T, Wong STC: Detection of molecular
particles in live cells via machine learning. Cytometry A 2007, 71:563-575.
• Convenient online submission
40. Soille P: Morphological Image Analysis: Principles and Applications.
Berlin: Springer-Verlag 1999. • Thorough peer review
41. Lacoste TD, Michalet X, Pinaud F, Chemla DS, Alivisatos AP, Weiss S: • No space constraints or color figure charges
Ultrahigh-resolution multicolor colocalization of single fluorescent
probes. Proc Natl Acad Sci USA 2000, 97:9461-9466. • Immediate publication on acceptance
42. Smal I, Draegestein K, Galjart N, Niessen W, Meijering E: Particle filtering for • Inclusion in PubMed, CAS, Scopus and Google Scholar
multiple object tracking in dynamic fluorescence microscopy images:
• Research which is freely available for redistribution
application to microtubule growth analysis. IEEE Trans Med Imaging 2008,
27:789-804.
Submit your manuscript at
www.biomedcentral.com/submit