Zhou et al. (2025) Remote sensing meta modal representation for missing modality land cover mapping: From EarthMiss dataset to MetaRS method

Identification

• Journal: Remote Sensing of Environment
• Year: 2025
• Date: 2025-11-26
• Authors: Yiheng Zhou, Ailong Ma, Junjue Wang, Zihang Chen, Yanfei Zhong
• DOI: 10.1016/j.rse.2025.115132

Research Groups

• State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan, China
• Graduate School of Frontier Sciences, The University of Tokyo, Kashiwa, Chiba, Japan

Short Summary

This paper introduces EarthMiss, a high-resolution multimodal dataset for land cover mapping with missing modalities, and proposes MetaRS, a meta-modal learning framework that disentangles features to significantly improve performance in such challenging scenarios.

Objective

• To address the issue of poor performance in land cover mapping when key remote sensing modalities are missing, by developing a new dataset and a novel meta-modal representation framework.

Study Configuration

• Spatial Scale: 0.6-meter high-resolution imagery (Optical and SAR), covering 13 cities across five continents, with 8 common land cover classes.
• Temporal Scale: Data simulates real-world missing modality scenarios; validation includes a 2023 Libya flood case study.

Methodology and Data

• Models used: MetaRS (Meta-modal learning framework), Meta-modal aware module, Meta-modal representation regularization training strategy.
• Data sources:
  • EarthMiss dataset: 3355 pairs of 0.6-meter high-resolution Optical and Synthetic Aperture Radar (SAR) images.
  • Four additional benchmark datasets (unspecified in the provided text).
  • Real-world 2023 Libyan-flood case study.

Main Results

• EarthMiss is established as a high-resolution multimodal benchmark dataset with 8 land-cover classes, representing the highest number of classes at this resolution for such a dataset.
• MetaRS significantly surpasses existing methods for missing modality land cover mapping across EarthMiss, four additional benchmarks, and a 2023 Libyan-flood case study.
• MetaRS effectively achieves meta-modal and specific-modal feature disentanglement, ensuring consistency of transferred knowledge.

Contributions

• Introduction of EarthMiss, a novel high-resolution (0.6 m) multimodal remote sensing land cover dataset comprising 3355 Optical and SAR image pairs across 13 cities and 5 continents, featuring 8 land cover classes.
• Proposal of MetaRS, a new meta-modal learning framework designed for missing modality land cover mapping, which includes a meta-modal aware module and a regularization training strategy.
• Demonstration of MetaRS's ability to disentangle meta-modal and specific-modal features by supervising the covariance matrix of multi-modal features, leading to improved performance.
• Comprehensive validation of MetaRS's superior performance against existing methods through extensive experiments on EarthMiss, other benchmarks, and a real-world flood case study.

Funding

• Not mentioned in the provided text.

Citation

@article{Zhou2025Remote,
  author = {Zhou, Yiheng and Ma, Ailong and Wang, Junjue and Chen, Zihang and Zhong, Yanfei},
  title = {Remote sensing meta modal representation for missing modality land cover mapping: From EarthMiss dataset to MetaRS method},
  journal = {Remote Sensing of Environment},
  year = {2025},
  doi = {10.1016/j.rse.2025.115132},
  url = {https://doi.org/10.1016/j.rse.2025.115132}
}