Traba et al. (2026) Towards onboard thermal hotspots segmentation with raw multispectral satellite imagery

Identification

• Journal: International Journal of Applied Earth Observation and Geoinformation
• Year: 2026
• Date: 2026-01-15
• Authors: Cristopher Castro Traba, David Rijlaarsdam, Jian Guo, Roberto Del Prete, Gabriele Meoni
• DOI: 10.1016/j.jag.2026.105095

Research Groups

• Delft University of Technology, Netherlands
• Ubotica Technologies, Netherlands
• 𝛷-lab, European Space Research Institute (ESRIN), European Space Agency (ESA), Italy
• Advanced Concepts and Studies Office, European Space Research Institute (ESRIN), European Space Agency (ESA), Italy

Short Summary

This study introduces and validates the first onboard AI-based payload processing pipeline for low-latency segmentation of thermal hotspots using raw multispectral satellite imagery. The pipeline, leveraging a modified U-Net model (ResUnet-S2) and a newly created dataset (SegTHRawS), achieved thermal hotspot detection in 1.45 seconds with a peak power of 4.05 W on CubeSat-compatible hardware, significantly reducing detection latency compared to conventional methods.

Objective

• To design, develop, and validate the first onboard AI-based payload processing pipeline for low-latency segmentation of thermal hotspots (wildfires and volcanic activity) using raw multispectral satellite imagery on CubeSat-compatible hardware.
• To create the first publicly available dataset (SegTHRawS) for thermal hotspot segmentation in raw multispectral satellite imagery, specifically for the Sentinel-2 mission.

Study Configuration

• Spatial Scale: 20 meters spatial resolution for thermal event detection; input images of 1152 × 1296 × 3 pixels, processed as 20 non-overlapping 256 × 256 × 3 pixel patches; capable of detecting small-scale events (<50 meters).
• Temporal Scale: End-to-end execution time from image acquisition to event detection of 1.45 seconds, which is faster than the Sentinel-2 granule acquisition time (3.6 seconds); dataset covers globally distributed Sentinel-2 images from 2016 to 2022.

Methodology and Data

• Models used:
  • ResUnet-S2 (a modified Fully Convolutional Network derived from U-Net, optimized for fast on-device inference)
  • U-Net, Attention U-Net, U-Net 3+ (modified versions for comparison)
  • U-Net, U-Net++, DeepLabV3+ with MobileOne-S0, EfficientNet-B0, and MobileNetV2 encoders (for training comparison)
  • SuperGlue matching network (for band co-registration)
  • Contextual threshold methods (Schroeder et al., 2016; Murphy et al., 2016; Kumar and Roy, 2018; Massimetti et al., 2020) and novel Castro-Traba conditions (for segmentation mask generation via strict majority voting).
  • Focal Loss function (for addressing class imbalance during training).
• Data sources:
  • Raw multispectral satellite imagery from Sentinel-2 (Level-0, 13 spectral bands from Coastal Aerosol (~0.44 µm) to Short-Wave InfraRed (~2.2 µm), specifically Near InfraRed (B8A), Short-Wave InfraRed (B11, B12) for processing).
  • SegTHRawS (Segmentation of Thermal Hotspots in Raw Sentinel-2 data): A newly created, publicly available dataset comprising 17,817 co-registered raw multispectral images (256 × 256 pixels, 13 bands, 50.4 GB) with weakly-supervised segmentation masks.
  • THRawS (Thermal Hotspots in Raw Sentinel-2 data) dataset (Meoni et al., 2024): Source for the raw multispectral imagery.
  • de Almeida Pereira et al. (2021) dataset: An external dataset of Landsat-8 L1C reflectance-based imagery for thermal hotspot segmentation, used for model generalization validation.
• Hardware: Raspberry Pi 3B+ (Payload Data Handling Processor) and Intel Movidius Myriad X VPU (AI accelerator, on CogniSAT-XE2 board) for onboard validation.

Main Results

• The proposed onboard processing pipeline, using the ResUnet-S2 model on CubeSat-compatible hardware (Raspberry Pi 3B+ and CogniSAT-XE2), achieved an end-to-end execution time of 1.452 seconds for a 1152 × 1296 × 3 pixel image.
• The pipeline demonstrated low power consumption with a peak power of 4.062 W and a total energy consumption of 5.833 Joules (calculated) or up to 16 Joules (measured, including hardware initialization).
• The ResUnet-S2 model achieved an Intersection over Union (IoU) of 0.988 and an F-1 score of 0.986 on the SegTHRawS dataset.
• The SegTHRawS dataset, the first publicly available for thermal hotspot segmentation in raw multispectral satellite imagery, was created, comprising 17,817 co-registered raw Sentinel-2 images (50.4 GB).
• The ResUnet-S2 model demonstrated strong generalization capabilities, outperforming the original U-Net-Light (3c) model (IoU 0.814, F-1 0.897) on an external Landsat-8 dataset (de Almeida Pereira et al., 2021) after retraining, achieving an IoU of 0.835 and an F-1 score of 0.909.

Contributions

• Design and verification of the first onboard payload processing pipeline for low-latency thermal hotspot segmentation using raw multispectral satellite imagery, tested on CubeSat-compatible edge computing hardware.
• Creation of SegTHRawS, the first publicly available dataset for thermal hotspot segmentation in raw multispectral Sentinel-2 imagery.
• Development of a novel contextual algorithm (Castro-Traba conditions) for thermal hotspot segmentation in raw Digital Numbers (DNs), integrated into the mask generation process.

Funding

Funding information is not explicitly provided in the paper.

Citation

@article{Traba2026Towards,
  author = {Traba, Cristopher Castro and Rijlaarsdam, David and Guo, Jian and Prete, Roberto Del and Meoni, Gabriele},
  title = {Towards onboard thermal hotspots segmentation with raw multispectral satellite imagery},
  journal = {International Journal of Applied Earth Observation and Geoinformation},
  year = {2026},
  doi = {10.1016/j.jag.2026.105095},
  url = {https://doi.org/10.1016/j.jag.2026.105095}
}