Improvement of Clustering Methods for Modelling Abrupt Land Surface Changes in Satellite Image Fusions
Abstract
:1. Introduction
2. Methodology
2.1. Overview of PSRFM
2.2. Modelling Abrupt Changes by Residual Adjustment
2.3. Modelling Abrupt Changes by Optimized Clusters
- (1)
- CLUSTER-1—Use the fine-resolution image at t0 or t2 as the input data for the classification of forward and backward predictions, respectively;
- (2)
- CLUSTER-2—Use the fine-resolution image at t0 or t2 and the MODIS image at t1 (resampled to the fine-resolution) as the input data for the classifications of forward and backward predictions, respectively;
- (3)
- CLUSTER-3—Use the fine-resolution image at t0 and t2, as well as the resampled MODIS image at t1 all together as the input data for the classifications for both forward and backward predictions.
- (4)
- CLUSTER-4—Use the reflectance change ratios calculated from coarse-resolution images between t0 and t1 or between t2 and t1 as the input data for the classification of forward and backward temporal predictions, respectively; the classification method is to divide the reflectance change ratios into optimized k clusters based on the values of each spectral band.
2.4. Implementation
- (1)
- Preprocessing of required input images (e.g., Landsat or Sentinel-2 and MODIS images). This includes data download, cloud mask generation, remapping all input images to a common projection if they are different, resampling coarse-resolution image to fine resolution by bilinear interpolation, co-registering one image to another by maximizing correlation between the coarse and fine images, and then cropping them to cover the same area [11,17]. For co-registering, the input MODIS images should cover a little bigger area than the Landsat or Sentinel-2 images. According to our tests, an extension of 2 km at each side of the fine-resolution images is reasonable.
- (2)
- Performing forward predictions by clustering the input data, estimating reflectance change rates of all clusters for predetermined cluster number range (e.g., from 5 to 20) and determine the final optimized forward prediction [10].
- (3)
- Performing the same processing as step 2 for backward prediction.
- (4)
- Combining the optimized forward and backward predictions as the final prediction of the fine-resolution-like image with weights based on either temporal intervals of the input and the prediction images or uncertainties of the predicted images.
3. Validation Test Cases
4. Results
- (1)
- PSRFM-11—CLUSTER-1 without RA
- (2)
- PSRFM-12—CLUSTER-1 with RA
- (3)
- PSRFM-21—CLUSTER-2 without RA
- (4)
- PSRFM-22—CLUSTER-2 with RA
- (5)
- PSRFM-31—CLUSTER-3 without RA
- (6)
- PSRFM-32—CLUSTER-3 with RA
- (7)
- PSRFM-41—CLUSTER-4 without RA
- (8)
- PSRFM-42—CLUSTER-4 with RA
- (9)
- STARFM—Spatial and Temporal Adaptive Reflectance Fusion Model
- (10)
- ESTARFM—the Enhanced version of STARFM
4.1. Test Case with Flood
- (1)
- All of the methods PSRFM-ij (i = 1, 2, 3, 4; j = 1, 2), STARFM and ESTARFM except the PSRFM-11 can largely maintain the spatial patterns and generate the similar spectral information of the observed Landsat-5 TM image, but some deviations from the reference image are visible.
- (2)
- Without the RA for the abrupt surface changes, an unmixing-based image fusion with the clusters determined only from the fine-resolution (Landsat-5 TM) image on the start date 26 November 2004 or the end date 28 December 2004 such as the PSRFM-11 is unable to predict the abrupt land surface changes (Figure 7a). Its performance is worse than STARFM and ESTARFM. This can be verified from the visual comparison of Figure 7a PSRFM-11 vs. Figure 7e STARFM and ESTARFM and the comparison of the quality indices in Figure 8. However, with introduction of the MODIS observations for the classification, PSRFM-21, PSRFM-31 and PSRFM-41 all produce an improved result to some degree (Figure 7b–d). This verifies that clustering with MODIS input leads to better results.
- (3)
- Comparisons between the algorithms PSRFM-i1 and PSRFM-i2 (i = 1, 2, 3, 4), i.e., the models with and without the RA, show that the RA for abrupt surface changes plays a more important role when only fine resolution images are used for clustering (Figure 7a PSRFM-11 vs. PSRFM-12). Abrupt surface changes such as flooding, may occur quickly, as such the clustering using only the two images cannot reflect the reality on the prediction date. With the improvement of the clusters by introducing the MODIS observations in different ways, the role of the RA becomes less critical and even unnecessary. For example, the output of PSRFM-22 is better than that of PSRFM-21 (Figure 7b PSRFM-21 vs. PARFM-22), while PSRFM-42 shows almost no improvement than PSRFM-41 (Figure 7d PSRFM-41 vs. PSRFM-42) because the clusters determined from the MODIS reflectance change ratios may be closer to the reality. These visual comparison conclusions are also confirmed by the comparison of the quality indices in Figure 8.
- (4)
- If we compare the different approaches PSRFM-ij (i = 1, 2, 3, 4; j = 1, 2), we can conclude that PSRFM-22 and PSRFM-41 or PSRFM-42 are the better options because of their relatively good quality indices and higher computation efficiency with less input data for clustering. The result of PSRFM-32 looks comparable but it is less computationally efficient because it involves all the Landsat-5 and MODIS image bands for the clustering process which slows down the computations.
- (5)
- Comparison of the improved PSRFM algorithms such as PSRFM-22, PSRFM-32, PSRFM-41 and PSRFM-42 with the two published methods STARFM and ESTARFM reveals that all these improved PSRFM methods yield better results, especially the result of NIR band in this test case. The reason for the better result of NIR band might be due to the properties of water body related to NIR spectrum. In general, surface of water reflects a litter Near Infrared spectrum as well as in other infrared bands. As the result, water body has very small value in those bands and exhibits black in false color composite using those bands. However, water body may present some various colors due to different materials contained in the water, such as soils. This may cause difficulty for STARFM and ESTARFM algorithms to find similar pixels around a prediction pixel, or the values of some similar pixels identified from the calibration images were changed on the prediction date. For instance, in this study image, some areas of water body (including flood areas) show yellow instead of black in the false color composite of SWIR1-NIR-SWRI2 in Figure 7. Since the improved PSRFM considered the surface changes caused by the flood by including the MODIS image on the prediction date for the clustering, it may overcome this issue better and therefore achieve a better result. As the flood water effects on NIR band are more significant than that on the green and red bands, the fusion result of NIR band with the improved PSRFM looks also better. Between STARFM and ESTARFM, STARFM looks a little better according to their quality indices in this case (Figure 8).
4.2. Test Case with Forest Fire
- (1)
- STARFM results (Figure 9c and Figure 11c) look not considerably affected by the areas of no MODIS observations because its prediction is based on the similar pixels around the prediction pixel. This can be an advantage for some special cases. For example, when some MODIS pixels of a small area are not available due to cloud contamination but their surrounding pixels have a valid value and a similar land-cover type that does not change between the dates of the calibration and the prediction images, these similar pixels can be used for prediction. In this test case, although pixel values can be predicted for those pixels of no MODIS observations, the predicted pixels bear some bias from the actual value because the land surface of some similar pixels based on the calibration image is changed by the fire on the prediction date. For this reason, its quality indices are the worst among all the methods.
- (2)
- The current implementation of ESTARFM excludes predictions for all pixels of no MODIS observations or with an invalid value, even only a single band has no observation or invalid value. Therefore, the predicted result shows a big black area in the predicted image as displayed in Figure 9d and Figure 11d. In comparison to STARFM, ESTARFM has the similar issue in selecting the similar pixels from the calibration image. However it excluded all pixels where the MODIS image does not have valid data. Its corresponding quality indices look better than that of STARFM in this test case.
- (3)
- Comparisons between the observed Sentinel-2 MSI image and the images predicted from the different algorithms PSRFM-i1 and PSRFM-i2 (i = 1, 2, 3, 4), with and without the RA show a similar result as the first dataset, i.e., the RA plays a more important role when the clusters are based on the fine-resolution images only. With the improvement of the clusters by introducing MODIS observations on the prediction date in different ways, the results are improved. For example, PSRFM-21 and PSRFM-31 are better than PSRFM11 (Figure 11g,i vs. Figure 11e); and PSRFM-41 is better than PSRFM21 and PSRFM31 (Figure 11k vs. Figure 11g,i). With the RA, PSRFM-22 is much better than PSRFM-21, while PSRFM-42 shows much less and even no improvement than PSRFM-41 because the clusters determined from the MODIS reflectance change ratios with PSRFM-41 is closer to the reality.
- (4)
- If we compare the different approaches PSRFM-ij (i = 1, 2, 3, 4; j = 1, 2), we can draw the same conclusion as the first test case, i.e., PSRFM-22 and PSRFM-41 or PSRFM-42 are the better options because of their relatively better quality indices (Figure 12) and higher computation efficiency. The result of PSRFM-32 looks comparable but its computation is less efficient because it involves all the fine- and coarse-resolution image bands for the clustering process which slows down the computation.
- (5)
- Comparison of the improved PSRFM algorithms such as PSRFM-22, PSRFM-32, PSRFM-41 and PSRFM-42 with the two published methods STARFM and ESTARFM reveals once again that all these improved PSRFM methods yield better results. This can be evidenced obviously from the evaluation scores of the quality indices against the observed Sentinel-2 MSI reflectance in Figure 12. Different from the first test case, the improved results are equally visible for all three bands NIR, SWIR1 and SWIR2 (Figure 8 vs. Figure 12).
5. Discussion
- (1)
- The method PSRFM-41 might introduce some unexpected side effect for some land-cover types such as water when their reflectance values are small/or close to zero. Since their reflectance change ratios cannot be determined accurately due to their small values and minor changes, their predicted values may become negative or an unreasonable value. This is evident from the result of PSRFM-41 in Figure 9k. Some pixels in the river (at the bottom right corner) show different colors from the black color as the results from other methods. Similarly, the Residual Adjustment (RA) may also cause a similar problem for some same land-cover types when their reflectance values are very close to zeros because the RA may adjust these pixel values and result in negative values. To avoid such prediction errors, we kept these pixels from the original fine images unchanged, i.e., ignored the predicted values if they become negative or invalid.
- (2)
- If the dataset does not have significant abrupt surface changes, the RA is not necessary and the PSRFM11 and PSRFM41 are relatively simple and can achieve reasonably good results. If the RA is applied, the results may become worse because of the possible RA interpolation errors. To avoid the negative impact of possible RA interpolation errors, the same criteria used for optimizing the clusters, i.e., the rule of Maximizing Correlation but Smaller Residuals (MCSR) [10], was used to determine whether RA process is needed or not. If the result with RA shows an improved correlation between the differences of the fine-resolution images and the differences of the coarse-resolution (MODIS) images but still no increase of the sum of the residual squares of the predicted fine-resolution-like image considerably, it is acceptable. Otherwise, the RA is not necessary.
6. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Gao, F.; Masek, J.; Schwaller, M.; Hall, F. On the blending of the Landsat and MODIS surface reflectance: Predicting daily Landsat surface reflectance. IEEE Trans. Geosci. Remote Sens. 2006, 44, 2207–2218. [Google Scholar]
- Zhu, X.; Chen, J.; Gao, F.; Chen, X.; Masek, J.G. An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions. Remote Sens. Environ. 2010, 114, 2610–2623. [Google Scholar] [CrossRef]
- Hilker, T.; Wulder, M.A.; Coops, N.C.; Linke, J.; McDermid, G.; Masek, J.G.; Gao, F.; White, J.C. A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODIS. Remote Sens. Environ. 2009, 113, 1613–1627. [Google Scholar] [CrossRef]
- Zhukov, B.; Oertel, D.; Lanzl, F.; Reinhackel, G. Unmixing-based multisensor multiresolution image fusion. IEEE Trans. Geosci. Remote Sens. 1999, 37, 1212–1226. [Google Scholar] [CrossRef]
- Huang, B.; Zhang, H. Spatio-temporal reflectance fusion via unmixing: Accounting for both phenological and land-cover changes. Int. J. Remote Sens. 2014, 35, 6213–6233. [Google Scholar] [CrossRef]
- Zhu, X.; Helmer, E.H.; Gao, F.; Liu, D.; Chen, J.; Lefsky, M.A. A flexible spatiotemporal method for fusing satellite images with different resolutions. Remote Sens. Environ. 2016, 172, 165–177. [Google Scholar] [CrossRef]
- Hazaymeh, K.; Hassan, Q.K. Spatiotemporal image-fusion model for enhancing temporal resolution of Landsat-8 surface reflectance images using MODIS images. J. Appl. Remote Sens. 2015, 9, 096095. [Google Scholar] [CrossRef]
- Kwan, C.; Budavari, B.; Gao, F.; Zhu, X. A Hybrid Color Mapping Approach to Fusing MODIS and Landsat Images for Forward Prediction. Remote Sens. 2018, 10, 520. [Google Scholar] [CrossRef]
- Wang, J.; Huang, B. A Rigorously-Weighted Spatiotemporal Fusion Model with Uncertainty Analysis. Remote Sens. 2017, 9, 990. [Google Scholar] [CrossRef]
- Zhong, D.; Zhou, F. A Prediction Smooth Method for Blending Landsat and Moderate Resolution Imagine Spectroradiometer Images. Remote Sens. 2018, 10, 1371. [Google Scholar] [CrossRef]
- Sekrecka, A.; Kedzierski, M. Integration of Satellite Data with High resolution Ratio: Improvement od Spectral Quality with Preserving Spatial Details. Sensors 2018, 13, 4418. [Google Scholar] [CrossRef] [PubMed]
- Wang, Q.; Atkinson, P.M. Spatio-temporal fusion for daily Sentinel-2 images. Remote Sens. Environ. 2018, 204, 31–42. [Google Scholar] [CrossRef] [Green Version]
- Gevaert, C.M.; García-Haro, F.J. A comparison of STARFM and an unmixing based algorithm for Landsat and MODIS data fusion. Remote Sens. Environ. 2015, 156, 34–44. [Google Scholar] [CrossRef]
- Emelyanova, I.V.; McVicar, T.R.; Van Niel, T.G.; Li, L.T.; van Dijk, A.I.J.M. Assessing the accuracy of blending Landsat–MODIS surface reflectances in two landscapes with contrasting spatial and temporal dynamics: A framework for algorithm selection. Remote Sens. Environ. 2013, 133, 193–209. [Google Scholar] [CrossRef]
- Hirvonen, R.A. Adjustments by Least Squares in Geodesy and Photogrammetry; Ungar: New York, NY, USA, 1971; 261p, ISBN1 0804443971. ISBN2 978-0804443975. [Google Scholar]
- Wolf, P.R. Survey Measurement Adjustments by Least Squares. In The Surveying Handbook; Springer: Berlin/Heidelberg, Germany, 1995; pp. 383–413. [Google Scholar] [CrossRef]
- Wu, M.; Niu, Z.; Wang, C.; Wu, C.; Wang, L. Use of modis and landsat time series data to generate high-resolution temporal synthetic landsat data using a spatial and temporal reflectance fusion model. J. Appl. Remote Sens. 2012, 6. [Google Scholar] [CrossRef]
- Wu, M.; Huang, W.; Niu, Z.; Wang, C. Generating daily synthetic landsat imagery by combining landsat and modis data. Sensors 2015, 15, 24002–24025. [Google Scholar] [CrossRef] [PubMed]
- Ei Rahman, S.A. Hyperspectral Image Classification Using Unsupervised Algorithms. Int. J. Adv. Computer Sci. Appl. 2016, 7, 198–205. [Google Scholar]
- Emelyanova, I.; McVicar, T.; Van Niel, T.; Li, L.; Van Dijk, A. Landsat and MODIS Data for the Lower Gwydir Catchment Study Site; v3.; CSIRO: Canberra, Australia, 2013. [Google Scholar] [CrossRef]
- Zhou, W.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Trans. Image Process. 2004, 13, 600–612. [Google Scholar] [Green Version]
- Pa, J.; Hegdeb, A.V. A Review of Quality Metrics for Fused Image. Aquat. Procedia 2015, 4, 133–142. [Google Scholar]
- Zhang, Y. Methods for Image Fusion Quality Assessment—A Review, Comparison and Analysis. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2008, XXXVII, 1101–1109. [Google Scholar]
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhong, D.; Zhou, F. Improvement of Clustering Methods for Modelling Abrupt Land Surface Changes in Satellite Image Fusions. Remote Sens. 2019, 11, 1759. https://doi.org/10.3390/rs11151759
Zhong D, Zhou F. Improvement of Clustering Methods for Modelling Abrupt Land Surface Changes in Satellite Image Fusions. Remote Sensing. 2019; 11(15):1759. https://doi.org/10.3390/rs11151759
Chicago/Turabian StyleZhong, Detang, and Fuqun Zhou. 2019. "Improvement of Clustering Methods for Modelling Abrupt Land Surface Changes in Satellite Image Fusions" Remote Sensing 11, no. 15: 1759. https://doi.org/10.3390/rs11151759
APA StyleZhong, D., & Zhou, F. (2019). Improvement of Clustering Methods for Modelling Abrupt Land Surface Changes in Satellite Image Fusions. Remote Sensing, 11(15), 1759. https://doi.org/10.3390/rs11151759