Karimi et al. (2026) Evaluating hv-block cross-validation for dependence-aware model selection in reference evapotranspiration estimation

Identification

• Journal: Scientific Reports
• Year: 2026
• Date: 2026-10-28
• Authors: Sepideh Karimi, Mostafa Sadeghzadeh, Naser Shiri, Jalal Shiri
• DOI: 10.1038/s41598-026-77384-x

Research Groups

• Water Engineering and Science Research Institute (WESRI), University of Tabriz, Tabriz, Iran
• Department of Water Engineering, Faculty of Agriculture, University of Tabriz, Tabriz, Iran
• Department of Civil Engineering, Sharif University of Technology, Tehran, Iran

Short Summary

This study empirically evaluated the hv-block cross-validation procedure for model selection in reference evapotranspiration (ETo) estimation under temporal autocorrelation. It found that while hv-block showed a higher over-selection rate in simulations, its practical impact on ETo estimation accuracy was negligible.

Objective

• To empirically assess the finite-sample selection behavior and over-selection bias of hv-block cross-validation under temporal autocorrelation.
• To determine if this bias persists with increasing sample size and whether it is linked to the procedure's structural design or autocorrelated noise.
• To evaluate the practical consequences of using hv-block cross-validation for real-world reference evapotranspiration (ETo) estimation accuracy with machine learning models.

Study Configuration

• Spatial Scale: Two automated weather stations in Iran (Rasht and Jask).
• Temporal Scale: Approximately 11 years of continuous daily records (4000 days) for real-world data. Monte Carlo simulations used sample sizes of 4000, 10000, and 20000 days.

Methodology and Data

• Models used:
  • Machine Learning: Random Forest (RF) for real-world ETo estimation.
  • Statistical: Ordinary Least Squares (OLS) linear regression for Monte Carlo simulations.
  • Model Selection Criteria: hv-block cross-validation (with PACF-based and cyclical exclusion windows), Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), standard 5-fold cross-validation, Out-of-Bag (OOB) error.
  • Time series analysis: Partial Autocorrelation Function (PACF) using Levinson–Durbin recursion, spectral analysis for cyclical patterns, Ljung-Box Q-test.
• Data sources:
  • Simulated data: Monte Carlo simulations with a known linear data-generating process (AR(1) noise, seasonal term, correlated predictors).
  • Observation data: Daily meteorological observations from two automated weather stations in Iran (Rasht and Jask) for approximately 11 years (4000 days). Measured variables included maximum air temperature (°C), minimum air temperature (°C), relative humidity (%), wind speed at 2 m (m s⁻¹), and solar radiation (MJ m⁻² day⁻¹). Daily reference evapotranspiration (ETo, mm day⁻¹) was computed using the FAO-56 Penman–Monteith equation.

Main Results

• In controlled simulations, hv-block cross-validation exhibited a higher over-selection rate (23–30%) compared to BIC (below 1%) and AIC (11.2–18.2%) across all tested autocorrelation levels.
• The over-selection bias of hv-block did not diminish as the sample size increased from 4000 to 20000 days, indicating persistence within the investigated range.
• Control experiments revealed that the over-selection bias is primarily associated with the structural design of the hv-block procedure (exclusion gap, fold geometry, replication imbalance), with autocorrelated noise playing a secondary role.
• Despite the observed over-selection bias in simulations, the resulting performance accuracy losses were negligible, with RMSE differences below 0.002 (in dimensionless units of the simulated response).
• In real-world ETo estimation using Random Forest, the RMSE differences between hv-block and temporally naive criteria (OOB, 5-fold CV) were consistently below 0.01 mm day⁻¹ at both study stations.
• The choice of exclusion window determination (PACF-based vs. cyclical) showed minimal differences in performance, demonstrating the robustness of the hv-block procedure to reasonable variations in the 'h' parameter.

Contributions

• Provides empirical evidence on the finite-sample selection behavior of hv-block cross-validation, addressing a gap in the theoretical debate regarding its consistency.
• Quantifies the over-selection bias of hv-block and investigates its persistence with sample size and its relation to structural design versus autocorrelated noise.
• Translates theoretical statistical concerns into practical hydrological relevance by assessing the impact of hv-block on real-world ETo estimation accuracy using a non-parametric model (Random Forest).
• Demonstrates that despite statistical over-selection bias, the practical consequences for operational ETo estimation are minimal, suggesting hv-block's suitability for hydrological applications.

Funding

Not applicable.

Citation

@article{Karimi2026Evaluating,
  author = {Karimi, Sepideh and Sadeghzadeh, Mostafa and Shiri, Naser and Shiri, Jalal},
  title = {Evaluating hv-block cross-validation for dependence-aware model selection in reference evapotranspiration estimation},
  journal = {Scientific Reports},
  year = {2026},
  doi = {10.1038/s41598-026-77384-x},
  url = {https://doi.org/10.1038/s41598-026-77384-x}
}