Song et al. (2025) TIF: A time-series-based image fusion algorithm

Identification

• Journal: Remote Sensing of Environment
• Year: 2025
• Date: 2025-10-02
• Authors: Kexin Song, Zhe Zhu, Shi Qiu, Pontus Olofsson, C. S. R. Neigh, Junchang Ju, Qiang Zhou
• DOI: 10.1016/j.rse.2025.115035

Research Groups

• Department of Natural Resources and the Environment, University of Connecticut, Storrs, CT, USA
• NASA Marshall Space Flight Center, Huntsville, AL, USA
• Biospheric Sciences Laboratory, NASA Goddard Space Flight Center, Greenbelt, MD, USA
• Earth System Science Interdisciplinary Center, University of Maryland, College Park, MD, USA
• Science Systems and Applications, Inc (SSAI), Lanham, MD, USA

Short Summary

This study developed the Time-series-based Image Fusion (TIF) algorithm to generate 10-meter surface reflectance time series by synthesizing Landsat 8/9 and Sentinel-2 A/B data. TIF consistently outperformed state-of-the-art methods in accuracy and efficiency, offering a practical pathway for creating 10-meter Harmonized Landsat and Sentinel-2 (HLS) products for fine-scale Earth observations.

Objective

• To develop a novel Time-series-based Image Fusion (TIF) algorithm capable of generating 10-meter surface reflectance time series by synthesizing Landsat 8/9 and Sentinel-2 A/B data.
• To evaluate TIF's performance against state-of-the-art methods in terms of spectral and spatial fidelity, robustness to land surface dynamics, and efficiency for large-scale, operational applications, particularly for fine-scale change detection.

Study Configuration

• Spatial Scale:
  • Input data: 30-meter (Landsat 8/9), 10-meter (Sentinel-2 native bands), 20-meter (Sentinel-2 resampled bands), 60-meter (simulated coarser Landsat for benchmark).
  • Output data: 10-meter surface reflectance.
  • Study areas: Five HLS tiles (each 109.8 kilometers by 109.8 kilometers) across the Conterminous United States (CONUS). Validation sites consisted of five 6 kilometer by 6 kilometer test areas.
• Temporal Scale:
  • Data acquisition period: Landsat 8/9 (2013–2021), Sentinel-2 (2015–2021).
  • Observation matching window: ±16 days.
  • HLS revisit interval: Approximately 2 days.
  • Global analysis of observation pairs: 10-year period (2013–2024).

Methodology and Data

• Models used:
  • Developed: Time-series-based Image Fusion (TIF) algorithm (pixel-based linear regression with temporal K-means clustering and robust regression).
  • Benchmark: Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM), Flexible Spatiotemporal Data Fusion (FSDAF) 2.0, Sen2Like (PredictMode and CompositeMode), extended Super-Resolution Convolutional Neural Network (ESRCNN).
  • Classification: Random Forest (for change detection).
• Data sources:
  • Satellite imagery: Landsat 8/9 Operational Land Imager (OLI) and Sentinel-2 Multi-Spectral Instrument (MSI).
  • Products: NASA Harmonized Landsat-Sentinel (HLS) Version 2.0 30-meter surface reflectance (L30), Sentinel-2 10-meter BRDF-corrected surface reflectance (S10, derived from Sentinel-2 TOA reflectance).
  • Ancillary data: National Land Cover Database (NLCD) 2021 land cover maps, Google Earth images.
  • Preprocessing: Land Surface Reflectance Code (LaSRC) v3.5.5, Fmask 4.6, Automated Registration and Orthorectification Package (AROP), C-factor Bidirectional Reflectance Distribution Function (BRDF) normalization, Sen2cor 2.10.

Main Results

• TIF consistently outperformed state-of-the-art methods (STARFM, FSDAF 2.0, Sen2Like, ESRCNN) across five diverse U.S. validation sites.
• Quantitative Accuracy: TIF demonstrated a 24 % reduction in Root Mean Square Error (RMSE) and a 6 % increase in Structural SIMilarity (SSIM) compared to FSDAF 2.0 and ESRCNN. It significantly outclassed STARFM and Sen2Like across all spectral and spatial metrics.
• Spectral Fidelity: TIF achieved the best overall performance with a mean RMSE of 0.0196 across all validation sites, and the lowest RMSE of 0.0132 in homogeneous forested areas.
• Spatial Fidelity: TIF exhibited the smallest error in semivariance difference (below 1 x 10^-3 for all lag distances), indicating superior spatial detail preservation.
• Change Detection: TIF-predicted images achieved the highest mean F1 score of 0.70 and the lowest mean disagreement rate of 0.05 against reference change maps, demonstrating its effectiveness in capturing fine-scale land changes.
• Efficiency and Scalability: TIF generates reusable pixel-level coefficients, enabling an efficient "predict-mode" without model retraining. Coefficient generation processing time is 0.4–0.7 seconds per pixel per CPU core. Estimated global storage for TIF coefficients is approximately 153 terabytes.
• Global Applicability: Analysis showed that over 96 % of the world's land surface meets the minimum data requirement (≥6 observation pairs) for TIF, and over 93 % meets the optimal threshold (≥30 pairs) over a 10-year period.

Contributions

• Introduces TIF, a novel time-series-based image fusion algorithm that uniquely leverages the entire historical record of pixel-level Landsat-Sentinel-2 observations, eliminating the reliance on strict image pairs or ancillary data.
• Develops a fully pixel-based framework that builds per-pixel linear regression models, capturing dynamic spectral relationships between sensors while explicitly accounting for land surface changes through adaptive temporal K-means clustering and robust regression.
• Generates reusable, pixel-level coefficients, enabling a highly efficient "train once, predict many" paradigm for scalable 10-meter time-series generation without the need for model retraining for each new prediction date.
• Demonstrates superior performance in quantitative accuracy (spectral and spatial fidelity) and multi-date change detection compared to existing state-of-the-art fusion methods (STARFM, FSDAF 2.0, Sen2Like, ESRCNN).
• Offers a practical and computationally efficient pathway for creating operational 10-meter versions of NASA’s Harmonized Landsat and Sentinel-2 (HLS) products, significantly advancing capabilities for fine-scale, time-sensitive Earth observations.

Funding

• UConn Eversource Energy Center (Near Real-time Assessment of Forest Risk to Infrastructure Using Satellite Time Series)
• NASA (Improvements of QA Band and New Science Data Layers Proposed for the NASA Harmonized Landsat and Sentinel-2 Products, grant number 80NSSC23K0773)

Citation

@article{Song2025TIF,
  author = {Song, Kexin and Zhu, Zhe and Qiu, Shi and Olofsson, Pontus and Neigh, C. S. R. and Ju, Junchang and Zhou, Qiang},
  title = {TIF: A time-series-based image fusion algorithm},
  journal = {Remote Sensing of Environment},
  year = {2025},
  doi = {10.1016/j.rse.2025.115035},
  url = {https://doi.org/10.1016/j.rse.2025.115035}
}