Dip Notes@Azdocuments - in
Dip Notes@Azdocuments - in
Dip Notes@Azdocuments - in
IMAGE RESTORATION
Syllabus
Restoration: Noise models, Restoration in the Presence of Noise Only using Spatial
Filtering and Frequency Domain Filtering, Linear, Position-Invariant Degradations,
Estimating the Degradation Function, Inverse Filtering, Minimum Mean Square Error
(Wiener) Filtering, Constrained Least Squares Filtering.
[Text: Chapter 5: Sections 5.2, to 5.9] L1, L2, L3
Introduction
The principal goal of restoration techniques is to improve an image in some predefined sense.
Restoration is an objective process and it attempts to recover an image that has been degraded
by using a priori knowledge of the degradation phenomenon. Thus, restoration techniques are
oriented toward the modeling the degradation and applying the inverse process in order to
recover the original image.
Image restoration vs. image enhancement:
• Enhancement:
• largely a subjective process.
• Priori knowledge about the degradation is not a must (sometimes no
degradation is involved).
• Procedures are heuristic and take advantage of the psychophysical
aspects of human visual system.
• Restoration:
• more an objective process.
• Images are degraded.
• Tries to recover the images by using the knowledge about the
degradation.
• Image restoration: recover an image that has been degraded by using a prior knowledge
of the degradation phenomenon.
• Model the degradation and applying the inverse process in order to recover the original
image.
• Image enhancement: “improve ” an image subjectively.
• Image restoration: remove distortion from image, to go back to the “original”--objective
process.
Why Image Restoration?
Image restoration is to recover the original image by removing noise and blur from image.
Image blur is difficult to avoid in many situations like photography, to remove motion blur
caused by camera shake, radar imaging to remove the effect of image system response, etc.
Image noise is unwanted signal which comes in image from sensor such as thermal or electrical
signal and Environmental condition such as rain, snow etc.
If H is a linear, position- invariant process, then the degraded image is given in the spatial
domain by
g(x, y) = h(x, y)*f(x, y) + η(x, y)
Where h(x, y) is the spatial representation of the degradation function and symbol * indicates
convolution. Convolution in the spatial domain is analogous to multiplication in the frequency
domain.
The model of the degraded image is given in the frequency domain by
G(u, v) = H(u,v)F(u, v) + N(u, v)
where the terms in capital letters are the Fourier transforms of the Corresponding terms of
previous spatial domain expression.
Noise Models
Noise Sources :
The principal sources of noise in digital images arise during image acquisition and/or
transmission.
Image acquisition:
e.g., light levels, sensor temperature, etc.
Sensor performance is affected by environmental conditions during image acquisition, and by
the quality of sensing elements.
Transmission:
Interference in the channel used for transmission.
e.g., lightning or other atmospheric disturbance in wireless network.
Spatial and frequency properties of Noise:
When the Fourier spectrum of noise is constant, the noise is usually called as white noise.
Dr. Suresh Delampady, Professor, RNS Institute of Technology
With the exception of spatially periodic noise, we assume that the Noise is independent of
spatial coordinates and that it is uncorrelated with respect to the image itself
Statistical behavior of the intensity values in the noise component is a matter of concern in the
analysis of image restoration model. These may be considered random variables, characterized
by a probability density function(PDF). The following are the most common PDFs found in
image processing applications.
Gaussian Noise:
Gaussian noise is characterized by two parameters, (mean) and σ2 (variance), by
1
p( z ) = e −( z − ) / 2
2 2
2
p( z ) = b
0 for z a
The mean and variance of this density are given by
b( 4 − )
= a + b / 4 and 2 =
4
a and b can be obtained through mean and variance
Exponential noise:
The PDF of Exponential noise is given by
ae− az for z 0
p( z ) =
0 for z 0
The mean and variance of this density are given by
1
= 1 / a and 2 = 2
a
Special case of Erlang PDF with b=1.
Uniform noise:
The PDF of Uniform noise is given by
1
if a z b
p( z ) = b − a
0 otherwise
The mean and variance of this density are given by
(b − a) 2
= (a + b) / 2 and 2 =
12
Impulse (Salt-and-Pepper) Noise:
The PDF of (bipolar) impulse noise is given by
Pa for z = a
p( z ) = Pb for z = b
0
otherwise
if b a, gray-level b will appear as a light dot,
while level a will appear like a dark dot.
If either Pa or Pb is zero, the impulse noise is called
unipolar
Above PDFs provide useful tools for modeling a broad range of noise corruption situations
given below.
Gaussian noise arises in an image due to factors such as Electronic circuit noise, sensor noise
due to poor illumination and/or high temperature
Mean Filters
In this this topic noise reduction capabilities of the various spatial filters are discussed.
Arithmetic mean filter:
The simplest mean filter is arithmetic mean filter.
Let S xy represent the set of coordinates in a rectangle
subimage window of size m n, centered at ( x, y ).
Generally, a geometric mean filter achieves smoothing comparable to the arithmetic mean
filter, but it tends to lose less image detail in the process.
Harmonic mean filter:
The Harmonic mean filter operation is given by the expression
mn
fˆ ( x, y ) =
1
( s ,t )S x , y g ( s, t )
It works well for salt noise, but fails for pepper noise. It does well also with other types of
noise like Gaussian noise.
Contraharmonic mean filter:
This filter yields a restored image based on the expression
g ( s, t )
( s ,t )S x , y
Q +1
fˆ ( x, y ) =
g ( s, t )
( s ,t )S x , y
Q
fˆ ( x, y) = median {g ( s, t )}
(s,t)S x , y
It has excellent noise reduction capabilities for certain types of random noise.
Less blurring than linear smoothing filters of similar size.
It is effective in the presence of both bipolar and unipolar noise.
Max and min filter:
Max filter uses the 100th percentile of a ranked set of numbers
Maxfilter
fˆ ( x, y) = max {g ( s, t )}
(s,t)S x , y
This filter combines order statistics and averaging filter. It works best for randomly distributed
noise, like Gaussian or uniform noise.
Alpha-Trimmed Mean Filter:
Where d is ranging from 0 to mn-1. when d = 0, it becomes arithmetic mean filter. When d =
mn – 1, this filter becomes a median filter.
gr(s,t) represent the remaining mn-d pixels
It is useful in situations involving multiple types of noise like a combination of salt-and-pepper
and Gaussian.
Adaptive filters
The behavior of adaptive filter changes based on statistical characteristics of the image inside
the filter region defined by the mхn rectangular window.
The performance is superior to that of the mean filters or order statistics filter.
Filter complexity is increased when the filter is designed for improved filtering power.
Adaptive, Local Noise Reduction Filters:
The simplest statistical measures of random are its mean and variance.
The mean gives a measure of average intensity in the region over which the mean is computed.
The variance gives a measure of contrast in that region.
Filter is to operate on local region Sxy
S xy : local region
The response of the filter at the center point (x,y) of S xy
is based on four quantities:
(a) g ( x, y ), the value of the noisy image at (x, y );
(b) 2 , the variance of the noise corrupting f ( x, y )
to form g ( x, y );
(c) mL , the local mean of the pixels in S xy ;
(d) L2 , the local variance of the pixels in S xy .
If the condition B1 > 0 AND B2 < 0 is false, then either zxy = zmin or zxy = zmax. In either case
the value of the pixel is an extreme value and the algorithm outputs the median value zmed
which is not a noise impulse.
And transfer function of Gaussian band reject filter of the order n is the given by the
expression
1 D 2 ( u ,v ) − D02
−
2 D ( u ,v )W
H (u, v) =1 − e
Band-pass Filters
Band-pass filter performs the opposite of a band-pass filter.
The transfer function of Band-pass filter is obtained from corresponding transfer function of
Band – reject filter.
H bp (u, v) = 1 − H br (u, v)
Generally It removes too much image details. It is quite useful in isolating the effects on an
image caused by selected frequency bands.
Figure 3.12 shows the 3 D plots of ideal, Butterworth and Gaussian notch (reject) filters.
The notch filter reduces the noise in the image with out blurring the given image.
fˆ ( x, y ) = g ( x, y ) − w( x, y )ˆ ( x, y )...............(1)
Here the modulation or weighting function w(x, y) is a constant within a neighborhood of size
(2a+1) by (2b+1) about a point (x,y). Eqn. (1) is an estimate of f(x, y).
We optimize its performance by minimizing the local variance of the restored image at the
position (x,y).
a b 2
1 fˆ ( x + s, y + t ) − fˆ ( x, y ) ..(2)
2 ( x, y ) =
(2a + 1)( 2b + 1) s = − at = −b
AverageValue..of ..eqn.(1)
a b
1
fˆ ( x, y ) =
(2a + 1)( 2b + 1) s = − at = − b
fˆ ( x + s, y + t )
(2a + 1)( 2b + 1) s = − a t = − b
− w( x, y )ˆ ( x + s, y + t )]
− [ g ( x, y ) − w( x, y )ˆ ( x, y )}2 .....(6)
The input – output relationship in the above figure before the Restoration stage is expressed as
g ( x, y) = H f ( x, y) + ( x, y)
H is linear
H af1 ( x, y ) + bf 2 ( x, y ) = aH f1 ( x, y ) + bH f 2 ( x, y )
f1 and f 2 are any two input images.
= H f ( , ) ( x − , y − ) d d
− −
= H f ( , ) ( x − , y − ) d d
− −
= f ( , ) H ( x − , y − ) d d
− −
g ( x, y ) =
− −
f ( , ) h( x − , y − ) d d + ( x, y )
= h ( x, y ) f ( x, y ) + ( x, y )
G (u , v ) = H (u , v ) F (u , v ) + N (u , v)
Gs (u, v)
H s (u, v) =
Fˆs (u, v)
Now, we can construct a function H(u, v) on a large scale, but having the same shape.
Estimation by Experimentation:
If equipment similar to the equipment used to acquire the degraded image is available, it is
possible in principle to obtain an accurate estimate of the degradation.
Images similar to the degraded image can be acquired with various system settings until they
are degraded as closely as possible to the image we wish to restore.
An impulse is simulated by a bright dot of light, as bright as possible to reduce the effect of
noise. Then the estimated function is
G (u, v)
H (u, v) =
A
where G(u,v) is the Fourier Transform of the observed image, and A is the constant describing
the strength of the impulse.
Mathematical Modeling:
Environmental conditions cause degradation. A degradation model is based on physical
characteristic of atmospheric turbulence.
H (u , v) = e − k ( u + v 2 )5/6
2
T
f x − x0 (t ), y − y0 (t ) e − j 2 ( ux + vy ) dxdy dt
=
0 − −
T − j 2 ux0 ( t ) + vy0 ( t )
=
0
F (u , v )e dt
T − j 2 ux0 ( t ) + vy0 ( t )
= F (u , v) e dt
0
T − j 2 ux0 ( t ) + vy0 ( t )
H (u , v ) = 0
e dt
Suppose that the image undergoes uniform linear motion
in the x -direction only, at a rate given by x0 (t ) = at / T .
T
H (u , v ) = 0
e − j 2 ux0 ( t ) dt
T
= e − j 2 uat /T dt
0
T
= sin( ua )e − j ua
ua
Suppose that the image undergoes uniform linear motion
in the x -direction and y -direction, at a rate given by
x0 (t ) = at / T and y0 (t ) = bt / T
T − j 2 ux0 ( t ) + vy0 ( t )
H (u , v ) = 0
e dt
T
= e − j 2 [ ua + vb ]t /T dt
0
T
= sin (ua + vb ) e − j ( ua + vb )
(ua + vb)
Inverse Filtering
The simplest approach to restoration is direct inverse filtering. Estimation of the transformation
of the original image is obtained by dividing the transform of the degraded image, G(u, v), by
the degradation function as given below
G(u, v)
Fˆ (u, v) =
H (u, v)
N (u, v)
Fˆ (u, v) = F (u, v) +
H (u, v)
where E {.} is the expected value of the argument. It is assumed that the noise and the image
are uncorrelated.
The minimum of the error function is given in the frequency domain by the Expression.
H * (u, v) S f (u, v)
Fˆ (u, v) = G (u, v)
S f (u, v) H (u, v) + S (u, v)
2
H * (u, v)
= G (u, v)
H (u, v) + S (u, v) / S f (u, v)
2
1 H (u, v)
2
= G (u, v)
H (u, v) H (u, v) + S (u, v) / S f (u, v)
2
H (u , v ) : degradation function
H * (u , v): complex conjugate of H (u , v )
| H (u , v) |2 = H * (u , v) H (u , v)
S (u , v) =| N (u , v) |2 = power spectrum of the noise
S f (u , v) =| F (u , v) |2 = power spectrum of the undegraded image
G(u, v) is the transform of the degraded image. The restored image in the Spatial domain is
obtained by the inverse Fourier transform of the frequency domain estimate.
If the noise is zero, noise power spectrum vanishes and the Wiener filter reduces to inverse
filter.
A number of useful measures are based on the power spectra of noise and the un-degraded
image. These are as follows.
Singal-to-Noise Ratio (SNR)
M −1 N −1
| F (u, v) | 2
SNR = u =0 v =0
M −1 N −1
| N (u, v) |
u =0 v =0
2
When the power spectrum of undegraded image can not be estimated, then the above equation
is approximated by the following expression.
1 H (u, v)
2
ˆ
F (u, v) = G(u, v)
H (u, v) H (u, v) + K
2
From the definition of convolution, and from the explanation of vector and matrix
operations, We can express following equation.
g ( x, y ) = f ( x, y ) h ( x, y ) + ( x, y )
This can be written in matrix form as
g = Hf + η
Here H is sensitive to noise, and this can be minimized by second derivative of an image
called Laplacian.
It is desired to find the minimum of a criterion function, defined as
M −1 N −1
C = 2 f ( x, y)
2
x =0 y =0
w =w w
2 T
Where is the Euclidean vector norm, and f̂ estimate of the
undegraded image.
The frequency domain solution to this optimization is given by the expression
2. Compute r
3. Stop if (1) is satisfied
η +a
2 2
r
or decreasing if
Use the new value of to recompute
H * (u, v)
Fˆ (u, v) = G(u, v)
H (u, v) + P(u, v)
2 2
2 2
In order to use this algorithm, we need the quantities r and η
Module-4
Wavelets:
Background, Multiresolution Expansions.
[Text: Chapter 7: Sections 7.1 and 7.2, Chapter 9: Sections 9.1 to 9.5]
L1, L2, L3
( A) A A B
Where B is suitable structuring element.
Following Figure 9.13 illustrates the mechanics of boundary extraction. It shows a simple
binary object, a structuring element B, and the result of using the above equation. Structuring
element shown in fig.b is most commonly used. Using 5x5 structuring element of 1s would
result in a boundary between 2 and 3 pixels thick.
until X k X k -1
4. Convex Hull
A set A is said to be convex if the straight line segment joining any two points in A lies
entirely within A.
The convex hull H or of an arbitrary set S is the smallest convex set containing S.
Figure 9.19(a) shows the structuring elements used to extract the convex hull. The origin of
each element is at its center. The X entries indicate “don’t care” conditions. This means that a
structuring element is said to have found a match in A if the 3x3 region of A under the
structuring element mask at that location matches the pattern of the mask.
A B A ( A * B)
A ( A * B)c
A more useful expression for thinning A symmetrically is based on a sequence of structuring
elements:
The structuring elements used for thickening have the same form as in thinning, but with
all 1’s and 0’s interchanged.
A separate algorithm for thickening is often used in practice, Instead the usual procedure
is to thin the background of the set in question and then complement the result.
In other words, to thicken a set A, we form C=Ac , thin C and than form Cc. Figure 9.22
illustrates this procedure.
Depending on the nature of A, this procedure may result in some disconnected points.
Therefore thickening by this procedure usually require a simple post-processing step to
remove disconnected points.
7. Skeletons:
As Fig.9.23 shows, the notion of a skeleton S(A) of a set A is intuitively defined, we deduce
from this figure that:
a. If z is a point of S(A) and (D)z is the largest disk centered in z and contained in A
(one cannot find a larger disk that fulfils this terms) – this disk is called “maximum
disk”.
b. The disk (D)z touches the boundary of A at two or more different places.
With
Example:
8. Pruning:
a. Thinning and skeletonizing tend to leave parasitic components
b. Pruning methods are essential complement to thinning and
skeletonizing procedures
X 1 A {B}
Where {B} denotes the structuring element sequence shown in Figs. 9.25(b) and (c).
Where Structuring Element sequence is as given below
{B} = { B1, B2, B3, ……………. Bn }
The x in Fig. 9.25(b) signifies don’t care conditions, in the sense that it does not matter whether
the pixel in that location has a value of 0 or 1.
Applying above equation X1 to A three times yields the set X1 in Fig.9.25 (d). The next step is
to store character to its original form, but with the parasitic branches removed. This require
forming a set X2 containing all end points in X1.
X 2 X1 * B k
8
k 1
X3 X2 H A
H : 3 3 structuring element
The union of X3 and X1 yields the desired result,
X 4 X1 X 3
is shown in Fig. 9.25 (g).
9. Morphological Reconstruction:
It involves two images and a structuring element
a. One image contains the starting points for the
transformation (The image is called marker)
b. Another image (mask) constrains the transformation
c. The structuring element is used to define connectivity
Morphological Reconstruction: Geodesic Dilation:
Let F denote the marker image and G the mask image,
F G. The geodesic dilation of size 1 of the marker image
with respect to the mask, denoted by DG(1) ( F ), is defined as
DG(1) ( F ) F B G
The geodesic dilation of size n of the marker image F
with respect to G , denoted by DG( n ) ( F ), is defined as
DG( n ) ( F ) DG(1) ( F ) ( n 1)
DG (F )
with DG(0) ( F ) F .
RGD ( F ) DG( k ) ( F )
RGE ( F ) EG( k ) ( F )
F nB
OR( n ) ( F ) RFD
1 I ( x, y ) if ( x, y ) is on the border of I
F ( x, y )
0 otherwise
then
c
H
RI c ( F )
D
is a binary image equal to I with all holes filled.
Border Clearing:
An algorithm for removing objects that touch ( i.e., are connected to )the border is a useful tool
because
Fig. 9.33 summarizes the basic types of structuring elements used in the various morphological
processed discussed.
******************