International Journal on Advanced Electrical and Computer Engineering (IJAECE)

______________________________________________________________________________________________

Image Fusion based on Joint Sparse Method with DWT for Medical
Applications

1
Mamatha M V, 2Chandrakala
1,2
Dept.of Electronics and Communication ,The Oxford College Of Engineering, Vtu Banglore
Email: 1mamatha1291@gmail.com, 2cssangu@yahoo.co.in

Abstract: — In this paper, a novel joint image fusion 2. Orthogonal Matching Pursuit (OMP)
algorithm which is the hybrid of Orthogonal Matching
Pursuit (OMP) and Principal Component Analysis (PCA) 3. Stage wise Orthogonal Matching Pursuit (St OMP).
is proposed to properly utilize the advantages and to
These techniques are used to represent signals with the
overcome the disadvantages of both OMP and PCA
methods. Firstly, common and innovative images are
fewest number of non-zero coefficients. Principal
extracted from the source images. Secondly, sparse PCA Component Analysis (PCA) is one of the powerful state-
method is employed to fuse the information of innovative of-the-art image fusion approaches in terms of visual
features. Then weighted average fusion is used to fuse the inspection and quantitative evaluation metrics.
sparse PCA result with the common feature thereby
preserving the edge information and high spatial resolution This fusion is carried out by integrating the principal
by using Discrete Wavelet Transform(DWT). components of images to be fused. Both PCA and
Sparse fusion have specific advantages and
Keywords— Image fusion, orthogonal matching pursuit, disadvantages. PCA fusion will enhance the spatial
Principal component analysis, sparse representation, DWT quality but have dense nonzero entries that might
I. INTRODUCTION represent uninformative features. Sparse fusion
preserves important information but high spatial
Multiple images of the same scene can be captured resolution is lacking.
simultaneously using different sensors. Perceiving the
complete picture of the scene from the captured images This paper proposes a new algorithm inspired by, which
of multiple sensors is not possible. employs different fusion rules for common and
innovative sparse components of the source images The
Image fusion algorithms allow the combination of proposed algorithm utilizes the advantages of both PCA
multiple captured images to generate a more informative and Sparse representation for fusing common and
composite image integrating the complementary innovative features of the captured images. The
information from multiple sensors, even when they are proposed algorithm utilizes the advantages of both PCA
out of focus and of differing resolution. Medical and Sparse representation for fusing common and
Imaging has revolutionized the medical diagnosis. innovative features of the captured images.
The arrival of imaging modalities such as Magnetic This algorithm also overcomes the disadvantages of both
resonance imaging (MRI) and computed tomography PCA and Sparse representation. In this paper, we
(CT) tend to give different perspectives of the same demonstrate the effectiveness of our proposed method
scene which can hinder clinical decision making and the by comparing its results with PCA and Sparse Fusion.
diagnostic process. For example MRI provides the best
view of soft tissues, while CT is better for its assessment The Output can be obtained by following equations:
of bone structures. Y= (Yc + Yi)/2 (1)
This motivates the need for image fusion for precise Yc=Yi= U * D + n (2)
diagnosis by merging the complementary information.
Three challenges to be addressed while fusing the Where Yc-Common Features
images [1] are: 1) The fused image should preserve all Yi- innovative Components
the important information needed for further processing.
2) Artifacts should not be introduced in fused image. 3) III. METHOD
Noise and unimportant information should be
The main objective is to design a novel joint image
suppressed.
fusion algorithm which is the hybrid of Orthogonal
II. PROBLEM STATEMENT Matching Pursuit (OMP) and Principal Component
Analysis (PCA) is proposed to properly utilize the
Sparse representation of signals is now possible utilizing advantages and to overcome the disadvantages of both
many different Greedy approaches, including: OMP and PCA methods. Firstly, common and
1. Matching Pursuit (MP) innovative images are extracted from the source images.

_______________________________________________________________________________________________
ISSN(Online): 2349-9338, ISSN(Print): 2349-932X Volume -2, Issue -2, 2015
7
 International Journal on Advanced Electrical and Computer Engineering (IJAECE)
______________________________________________________________________________________________

Secondly, sparse PCA method is employed to fuse the IV. SIMULATION RESULTS
information of innovative features.
In order to test the performance of the proposed joint
Then weighted average fusion is used to fuse the sparse fusion algorithm, we compared the quantitative and
PCA result with the common feature thereby preserving qualitative results with two state the of art methods.
the edge information and high spatial resolution. We Qualitative measurement is done through visual
demonstrate this methodology on medical images from inspection that considers sharpness and noise
different sources and the experimental results proves the suppression. Since the proposed joint fusion uses both
robustness of the proposed method. PCA and sparse domain, use PCA, Sparse OMP fusion
A block diagram of our proposed Flow of Sparse PCA methods for comparison . For the evaluation, we
joint fusion algorithm is shown in Fig. 1. adopted proposed joint strategy for Multi resolution,
Multi focus and Multimode images and compared the
The algorithm is summarized as follows: results with existing algorithms.
1) Registered multiple images in an ensemble have one The experiment is carried out with the assumption that
common component and multiple innovative the source images are registered already. Based on
components. Innovative components contain the visual inspection, The Joint PCA Sparse algorithm
complementary information from different images of performs the best since the resultant image contain more
same scene geometric structures while sparse fusion comes the
2) Innovative components {I}1=i of different images i second. Result of proposed method seems to contain
are decomposed into sparse vectors s1,s2,s3,…si via sharp edges.
orthogonal matching pursuit method described in
Figure.1
3) Combine the sparse coefficients of innovative
components using PCA fusion rule, for which the
covariance matrix s C of innovative images is calculated
as follows, 𝐶𝑠 = cov(𝐼𝑠) = cov([𝑠1(: ), 𝑠2(: )])= 1 𝑖−1 𝐼𝑠
∙ 𝐼𝑠
Where s1, s2 are the sparse vectors of the innovative (a) (b)
components. Find the Eigen sparse and normalized
Eigen sparse vector of maximum Eigen value. Eigen
vector will be used as weightings for innovative sparse
vectors to be fused.
4) Fused PCA result Ip is reshaped into a block of 8×8
and each pixel position is the sum of several block
values. Reconstructed image 1 I is obtained by dividing
each pixel by number of addition operations performed
at each pixel.
(c) (d) (e)
5) For fusing common component and fused innovative
component, the fuse rule of weighted average scheme Fig.2 Comparison of performance of different fusion
proposed by Burt et al. [2] is adopted. algorithms for case of Multimodal medical images
(512×512) (a) CT Image ((b) MRI Image (c) Proposed
Joint fusion (d) PCA fusion (e) Sparse Fusion
Fig.2 illustrates the results of applying two multi modal
medical images to three image fusion algorithms. The
medical images are MRI and CT image of same scene
which have been registered already.
CT image provides the information on bone structures
and MRI image contains tissue information. Medical
image fusion needs great accuracy as it’s used for
diagnosis. Hence, multimodal image fusion would give
sufficient details necessary for diagnosis. Based on the
visual inspection, the Joint Sparse results contain more
detail information.
Results of PCA seem to have high spatial resolution but
Fig1:Flow of Sparse PCA Joint Algorithms with DWT
they are disappointing in terms of detailed information.

_______________________________________________________________________________________________
ISSN(Online): 2349-9338, ISSN(Print): 2349-932X Volume -2, Issue -2, 2015
8
 International Journal on Advanced Electrical and Computer Engineering (IJAECE)
______________________________________________________________________________________________

Bone details are not visible in PCA resultant image. Experiment results are shown in the above Table.
Sparse result is better but some artifacts are easily Tabulated result demonstrates the effectiveness of the
observed Reconstructing fused image through joint proposed algorithm over existing methods in terms of
fusion algorithms seems to be more precise Qualitative and Quantitative methodologies.
comparatively.
We can observe that the results of multimodal image
In order to analyze the quality of the algorithms fusion and multi focus image fusion utilizing our
quantitatively, we consider 5 metrics: Mutual proposed fusion strategy outperforms PCA and Sparse
Information (MI), PSNR, Correlation, Entropy and fusion. The PCA by itself performs poorer results for all
Structural Similarity (SSIM) index. Mutual Information cases.
shows how much information has been transferred from
Using DWT we can retrieve more information with
source images to the resultant images.
composite values because DWT will operate in
Entropy shows the amount of important details available frequency and spatial domain.
in the image. PSNR is Peak Signal to Noise Ration
Discrete wavelet transform has been applied to the
which is used to measure the reconstruction quality of
average values of common and innovative features.
fused image.
Performance of Fusion Method by the quality Evaluation Metrics:-
Image Type Fusion Strategy PSNR Mutual Entropy Correlation SSIM
Information
Multi Focus Joint PCA Sparse 34.18 2.15 7.36 0.999 1.00
PCA 31.63 2.01 7.26 0.9981 0.9999
OMP Fusion 32.33 2 7.3 0.9981 1.00
Multi Dose Joint PCA Sparse 30.86 1.9 7.25 0.9997 1.00
PCA 22.69 0.75 4.79 0.9991 1.00
OMP Fusion 24.46 0.76 4.79 0.9995 0.9998
Multimodal(MRI & CT) Joint PCA Sparse 26.41 0.96 6.74 0.9043 0.9977
PCA 20.83 0.8 6.55 0.869 0.9919
OMP Fusion 24.8 0.94 6.73 0.8985 0.9975

Experiment results are shown in the above Table. VI. REFERENCES

Tabulated result demonstrates the effectiveness of the
proposed algorithm over existing methods in terms of [1] H. Yin, S. Li,”Multimodal image fusion with
Qualitative and Quantitative methodologies. joint sparsity model”, Opt Eng., 50(6), (2011).
[2] O.Rockinger. “Image sequence fusions using a
V. CONCLUSION
shift-invariant wavelet transform.” In image
In this paper, we presented a fusion rule for JSR-based processing , 1997 International Conference on,
image fusion. The fusion rule combined the “weighted vol. 3, pp. 288-291. IEEE1997
average” with the “choose-max.” Experimental results
[3] He, D-C., Li Wang, and Massalabi Amani. "A
show that the fusion rule combined the “weighted
new technique for multi-resolution image
average” with the “choose-max” is superior to previous
fusion." In Geoscience and Remote Sensing
fusion rule for JSR based fusion.
Symposium, 2004. IGARSS'04. Proceedings.
Then, we applied discrete wavelet transform to the 2004 IEEE International, vol. 7, pp. 4901-4904.
average value.” This has lower complexity compared IEEE, 2004.
with the K-SVD algorithm. Furthermore, we proposed
[4] Li, Hui, B. S. Manjunath, and Sanjit K. Mitra.
the generalized JSR which the signal’s ensemble
"Multisensor image fusion using the wavelet
depends on two dictionaries. JSR has better fusion
transforms." Graphical models and image
performance than some of the state-of the- art
processing , vol. 3,pp. 235-245. IEEE 1997
algorithms.
[5] Z. Sadeghipour, M. Babaie-Zadeh, and C. Jutten,
ACKNOWLEDGMENTS "An adaptive thresholding approach for image
The authors would like to thank the anonymous denoising using redundant representations", IEEE
reviewers and editor for their insightful comments and international workshop on Machine Learning for
suggestions. This work is supported by Chandrakala, Signal Processing, 2009, pp. 1-6.
Assi. Professor [6] Zhou Wang, Alan C. Bovik, Hamid R. Sheikh
and Eero P. Simoncelli, “Image Quality
Assessment: From Error Visibility to Structural

_______________________________________________________________________________________________
ISSN(Online): 2349-9338, ISSN(Print): 2349-932X Volume -2, Issue -2, 2015
9
 International Journal on Advanced Electrical and Computer Engineering (IJAECE)
______________________________________________________________________________________________

Similarity”, IEEE transactions on Image Engineering (MECBME) February 17-20, 2014,

Processing, vol. 13, no. 4, April 2004. Hilton Hotel, Doha, Qatar
[7] Anuyogam, Venkataraman , Javad Alirezaie, [8] T. Guha and R. K. Ward, “Learning sparse
Paul Babyn and Alireza Ahmadian “Medical representations for human action recognition,”
Image Fusion Based on Joint Sparse Method”- IEEE Trans. Pattern Anal. Mach. Intell
2014 Middle East Conference on Biomedical 34(8),1576–1588 (2012).


_______________________________________________________________________________________________
ISSN(Online): 2349-9338, ISSN(Print): 2349-932X Volume -2, Issue -2, 2015
10