Karpasitis et al. (2026) A new efficiency metric for the spatial evaluation and inter-comparison of climate and geoscientific model output

Identification

• Journal: Geoscientific model development
• Year: 2026
• Date: 2026-01-13
• Authors: Andreas Karpasitis, Panos Hadjinicolaou, George Zittis
• DOI: 10.5194/gmd-19-345-2026

Research Groups

Climate and Atmosphere Research Center (CARE-C), The Cyprus Institute, Nicosia, Cyprus

Short Summary

This study introduces the Modified Spatial Efficiency (MSPAEF) metric to improve the spatial evaluation and inter-comparison of climate and geoscientific model outputs. It demonstrates that MSPAEF consistently provides robust and intuitive performance, accurately capturing spatial patterns under diverse conditions, and addresses limitations of existing metrics by being sensitive to relative bias and scale-independent.

Objective

• To propose and evaluate a new spatial efficiency metric, Modified Spatial Efficiency (MSPAEF), designed to overcome limitations of existing metrics (SPAEF, WSPAEF, Esp) for assessing how well climate and geoscientific models reproduce observed spatial patterns of variables like precipitation and temperature.

Study Configuration

• Spatial Scale: Global scale for real-world application using CMIP6 models remapped to 1° resolution; 10x10 grid for synthetic data experiments.
• Temporal Scale: 30-year historical period (1981-2010) for annual mean precipitation and temperature.

Methodology and Data

• Models used:
  • New metric: Modified Spatial Efficiency (MSPAEF).
  • Compared metrics: Spatial Efficiency (SPAEF), Wasserstein Spatial Efficiency (WSPAEF), Spatial Pattern Efficiency metric (Esp).
  • Application: 33 Coupled Model Intercomparison Project phase 6 (CMIP6) models.
• Data sources:
  • Synthetic data: Generated with varying spatial correlation coefficients, biases, and standard deviation ratios, for both normal and skewed distributions.
  • Observational/Reanalysis data: ERA5 reanalysis dataset.

Main Results

• MSPAEF demonstrates robust and intuitive performance across a range of synthetic data scenarios (varying correlation, bias, standard deviation ratio) for both normally and skewed distributed variables.
• MSPAEF is sensitive to both spatial pattern agreement and relative mean bias, distinguishing it from bias-insensitive metrics (SPAEF, Esp) and metrics sensitive to absolute bias magnitude (WSPAEF).
• In synthetic case studies, MSPAEF consistently aligns with intuitive assessments of model performance, while other metrics exhibit limitations in specific scenarios (e.g., WSPAEF's high sensitivity to absolute bias, SPAEF/Esp's insensitivity to bias).
• When applied to global CMIP6 models (1981-2010) for annual precipitation and 2 m temperature, MSPAEF rankings show the most similarity to Esp (normalized absolute ranking difference of 2.8 for precipitation, 3.8 for temperature).
• CMIP6 models generally perform better in capturing the spatial distribution and magnitude of temperature (most MSPAEF values > 0.9) than precipitation (most MSPAEF values < 0.85).
• WSPAEF rankings for precipitation differ significantly from other metrics due to its high sensitivity to large absolute biases in precipitation (e.g., a 0.030 meter bias can lead to WSPAEF values exceeding 30). This discrepancy is reduced for temperature due to smaller absolute biases.

Contributions

• Introduction of the Modified Spatial Efficiency (MSPAEF) metric, a novel bias-sensitive and scale-independent spatial efficiency measure that integrates spatial pattern similarity and relative mean bias.
• Addresses key limitations of existing spatial evaluation metrics by improving sensitivity to spatial distribution and relative bias, and by eliminating reliance on user-defined parameters (e.g., histogram bins).
• Provides a balanced indicator that dynamically emphasizes spatial pattern similarity or bias depending on the data characteristics, offering a more comprehensive model evaluation.
• Enhances objectivity, consistency, and reproducibility of model evaluations across diverse datasets and analysis settings due to its scale-independence.

Funding

OptimESM project (European Union’s Horizon Europe research and innovation programme, grant agreement no. 101081193).

Citation

@article{Karpasitis2026new,
  author = {Karpasitis, Andreas and Hadjinicolaou, Panos and Zittis, George},
  title = {A new efficiency metric for the spatial evaluation and inter-comparison of climate and geoscientific model output},
  journal = {Geoscientific model development},
  year = {2026},
  doi = {10.5194/gmd-19-345-2026},
  url = {https://doi.org/10.5194/gmd-19-345-2026}
}