Diakogiannis et al. (2026) Tackling fluffy clouds: robust agricultural field boundary delineation from Sentinel-1 and Sentinel-2 satellite image time series

Identification

• Journal: Remote Sensing of Environment
• Year: 2026
• Date: 2026-10-01
• Authors: Foivos I. Diakogiannis, Zheng-Shu Zhou, Jeff Wang, Gonzalo Mata, David A. Henry, Roger Lawes, Amy Laura Parker, Peter A. Caccetta, Suzanne L. Furby, Rodrigo Ibata, Ondrej Hlinka, Jonathan Richetti, Kathryn L. Batchelor, Chris Herrmann, Andrew Toovey, John Taylor
• DOI: 10.1016/j.rse.2026.115695

Research Groups

• Technology, CSIRO, Kensington, WA, Australia
• A&F, CSIRO, Perth, Floreat WA, Australia
• A&F, CSIRO, Melbourne, VIC 3030, Australia
• CSIRO Space and Astronomy, Kensington, WA, Australia
• University of Strasbourg, Strasbourg, France
• IM&T CSIRO, Brisbane, Queensland, Australia
• Australian National University, School of Computing, ACT, Canberra, Australia

Short Summary

This paper introduces PTAViT3D, a deep learning architecture using 3D Vision Transformers for robust agricultural field boundary delineation directly from cloud-affected Sentinel-1 and Sentinel-2 satellite image time series, demonstrating state-of-the-art performance and global transferability across diverse agricultural environments.

Objective

• To develop a deep learning architecture capable of accurately delineating agricultural field boundaries directly from preprocessed, cloud-affected Sentinel-1 and Sentinel-2 satellite image time series, without requiring extensive manual cloud-free data curation or compositing.

Study Configuration

• Spatial Scale: Continental (Australia's cropping regions, approximately 100 km x 100 km Sentinel-2 MGRS tiles), global (24 countries across four continents for Fields-of-the-World), regional (France for PASTIS, Mekong Delta (Vietnam), Tonlé Sap (Cambodia), West-Friesland (Netherlands) for AI4SmallFarms). Pixel resolutions of 10 meters (Sentinel-2) and 5 meters by 20 meters (Sentinel-1 IW), with super-resolved Sentinel-2 imagery at 2.5 meters for AI4SmallFarms.
• Temporal Scale: Multi-temporal satellite image time series, typically spanning cropping cycles. Sentinel-2 data from 2019 and 2023 (ePaddocks), with 4 to 61 acquisitions per pixel, and up to 48 randomly sampled acquisitions for PASTIS. Sentinel-1 data from May to November 2019 (ePaddocks) and January to February 2021 (AI4SmallFarms), with routine coverage every 12 days.

Methodology and Data

• Models used:
  • PTAViT3D: A 3D Vision Transformer architecture for processing single-modality (Sentinel-1 or Sentinel-2) time series.
  • PTAViT3D-CA: An extension of PTAViT3D incorporating cross-attention mechanisms for fusing Sentinel-1 and Sentinel-2 datasets.
  • Baselines: U-Net with EfficientNet-B3 backbone (FoTW), U-TAE (PASTIS), FracTAL ResNet (ePaddocks), Delineate Anything (DA) (PASTIS).
• Data sources:
  • Satellite: Sentinel-1 (C-band SAR, dual-polarization VV+VH or HH+HV, IW mode, SLC and GRD formats), Sentinel-2 (Level L2A, bands B2, B3, B4, B8).
  • Observation/Benchmark:
    • ePaddocks™: CSIRO’s national, continental-scale agricultural field boundary product for Australia (2019 and 2023 versions).
    • Fields-of-the-World (FoTW): Global benchmark for agricultural field boundary segmentation.
    • PASTIS: Dataset for panoptic segmentation and crop type semantic segmentation with Sentinel-2 time series.
    • AI4SmallFarms: Dataset for field delineation in smallholder farms (Vietnam, Cambodia, Netherlands).
  • Ancillary: CLUM (Catchment Scale Land Use of Australia) for masking non-agricultural areas, 30 m Copernicus global DEM for Sentinel-1 terrain correction.

Main Results

• PTAViT3D and PTAViT3D-CA effectively delineate field boundaries directly from cloud-affected Sentinel-1 and Sentinel-2 time series, simplifying data preparation.
• Cloud Tolerance: PTAViT3D shows robustness to partially clouded Sentinel-2 imagery, with IoU degrading from approximately 0.79 (0-20% cloud tolerance) to 0.58 (20-50% cloud tolerance) when maximum time instances are limited to 16 or 32. Cloud-augmented training significantly reduces severe thin-cloud IoU loss (from 0.074 to 0.031) and severe mixed cloud-shadow IoU loss (from 0.032 to 0.015).
• Sensor Modality:
  • Sentinel-2 inputs generally provide a stronger signal for field boundary extraction than Sentinel-1 due to higher resolution.
  • PTAViT3D with Sentinel-1 inputs achieves excellent performance (IoU of 0.79) comparable to a 2D baseline trained on Sentinel-2, making it a viable solution in persistently cloudy regions.
  • Fusion of Sentinel-2 (super-resolved) and Sentinel-1 (PTAViT3D-CA) significantly improves performance for smallholder farms (AI4SmallFarms), increasing mIoU by 6-7 percentage points over Sentinel-2-only.
• 3D vs. 2D Modeling: Time series (3D) models consistently outperform their 2D counterparts. For ePaddocks, S2&S1 3D and S2 3D models achieve an IoU of 0.81, outperforming S2 2D (0.79) and S1 3D (0.79).
• Transferability:
  • Fields-of-the-World (FoTW): PTAViT3D outperforms the FoTW 2-class baseline in 19 out of 21 matched countries, increasing mean mIoU from 0.764 to 0.840 and reducing standard deviation from 0.137 to 0.091.
  • PASTIS: PTAViT3D achieves state-of-the-art performance for crop type classification with 24 randomly sampled Sentinel-2 acquisitions (0.847 Overall Accuracy, 0.649 mIoU, 0.796 MCC), improving on U-TAE.
  • AI4SmallFarms: The S2-SR&S1 model achieves an IoU of 0.925 for Cambodia and 0.799 for Vietnam (10 m resolution). The boundary F1 score (10 m) of 0.520 improves on the baseline (0.390) by 13.0 percentage points. PoLiS distance is 15.06 meters (10 m resolution), outperforming the baseline (26.7 meters).
• Continental-Scale Application: PTAViT3D was successfully deployed to create the 2023 ePaddocks national product for Australia, demonstrating its scalability and practical utility. Raster-level evaluation shows a mean IoU of 0.804 ± 0.194 and a very low False Discovery Rate (0.011 ± 0.011). Polygon-level evaluation shows a mean IoU of 0.893 ± 0.221, Mean Surface Distance of 39.076 ± 38.706 meters, and Hausdorff Distance of 224.998 ± 192.163 meters.

Contributions

• A 3D Vision Transformer architecture specifically tailored to process Satellite Image Time Series (SITS) data, expanding upon prior work in semantic segmentation (SSG2).
• A memory-efficient and effective 3D vision attention mechanism designed explicitly for satellite image time series.
• Two implementations: PTAViT3D for processing either Sentinel-2 or Sentinel-1 time series, and PTAViT3D-CA, integrating Sentinel-2 and Sentinel-1 data via cross-attention.
• Demonstration of model performance on partially cloud-affected Sentinel-2 sequences under operational scene-selection thresholds (up to 40% scene-level cloud cover) and controlled cloud/shadow perturbations, together with Sentinel-1 time-series processing as an all-weather alternative in persistently cloudy regions.
• Demonstration of transferability across regions, by benchmarking against publicly available datasets: Fields of the World (FoTW), PASTIS, and AI4SmallFarms, covering diverse agricultural landscapes and challenges.
• A roadmap application for large-scale agricultural boundary delineation, with the validation of PTAViT3D across the Australian cropping regions, and the creation of the 2023 ePaddocks national product of Australia.

Funding

• CSIRO IMT Scientific Computing (resources and expertise)

Citation

@article{Diakogiannis2026Tackling,
  author = {Diakogiannis, Foivos I. and Zhou, Zheng-Shu and Wang, Jeff and Mata, Gonzalo and Henry, David A. and Lawes, Roger and Parker, Amy Laura and Caccetta, Peter A. and Furby, Suzanne L. and Ibata, Rodrigo and Hlinka, Ondrej and Richetti, Jonathan and Batchelor, Kathryn L. and Herrmann, Chris and Toovey, Andrew and Taylor, John},
  title = {Tackling fluffy clouds: robust agricultural field boundary delineation from Sentinel-1 and Sentinel-2 satellite image time series},
  journal = {Remote Sensing of Environment},
  year = {2026},
  doi = {10.1016/j.rse.2026.115695},
  url = {https://doi.org/10.1016/j.rse.2026.115695}
}