Xu et al. (2026) Integrating LSTM and Transformer for Improved Daily Runoff Prediction: A Parallel Computing Approach

Identification

• Journal: Modeling Earth Systems and Environment
• Year: 2026
• Date: 2026-02-28
• Authors: Dong-mei Xu, Yang-hao Hong, Wenchuan Wang, Miao Gu, Jun. Wang, Yanwei Zhao, Hong-fei Zang
• DOI: 10.1007/s40808-026-02738-3

Research Groups

North China University of Water Resources and Electric Power, Zhengzhou, China

Short Summary

This study proposes a novel LSTPencoder model, integrating LSTM and Transformer encoder via a parallel computing and information-sharing approach, to improve daily runoff prediction accuracy. Optimized by the AOAAO algorithm, the model effectively captures both local and global sequence features, demonstrating superior forecasting performance and offering a more effective explanation of complex causal relationships in runoff sequences.

Objective

• To address the challenges of high computational complexity and the tendency to overlook local trend characteristics in Transformer models when handling multimodal time-series data for daily runoff prediction.
• To develop a novel LSTPencoder model based on dual-architecture parallel computing and prediction information sharing that integrates a gating mechanism with the Transformer encoder to capture both local and global features.
• To optimize the model's hyperparameters using the AOAAO algorithm to achieve the best forecasting results for daily runoff.

Study Configuration

• Spatial Scale: Two hydrological stations:
  • Tangnaihai Hydrological Station (Yellow River Basin, Qinghai Province, China): Controlling drainage area of 122,000 square kilometers.
  • Dongjiang Hydropower Station (Yangtze River Basin, Hunan Province, China): Drainage area of 4,719 square kilometers above the dam site.
• Temporal Scale: Daily runoff prediction.
  • Tangnaihai: 1461 daily runoff data from 2015-01-01 to 2018-12-31.
  • Dongjiang: 1277 daily runoff data from 1999-01-01 to 2022-06-30.
  • Prediction uses a sliding window of 7 days to predict the 8th day's runoff.

Methodology and Data

• Models used:
  • Proposed: LSTPencoder (hybrid of LSTM and Transformer Encoder with a feature sharing module), optimized by the Adaptive-Operator-based Arithmetic Optimization Algorithm (AOAAO).
  • Benchmark models: Artificial Neural Network (ANN), Long Short-Term Memory (LSTM), Attention-LSTM, Transformer (Encoder only).
• Data sources: Univariate historical daily runoff observational data from Tangnaihai Hydrological Station and Dongjiang Hydropower Station.
  • Data split: 60% for training, 20% for validation, 20% for testing.
  • Data preprocessing: Normalization using min-max scaling to [0,1].

Main Results

• The proposed LSTPencoder model demonstrated superior forecasting performance compared to other models.
• At Tangnaihai Hydrological Station, LSTPencoder improved the coefficient of determination (R²) by an average of 44.79% compared to baseline models (LSTM, Attention-LSTM, Transformer).
• At Dongjiang Hydropower Station, LSTPencoder improved R² by an average of 16.59% compared to baseline models.
• After hyperparameter optimization with the AOAAO algorithm, the AOAAO-LSTPencoder model achieved R² values of 0.97 for Tangnaihai and 0.95 for Dongjiang on the test sets.
• Compared to the optimized Transformer and LSTM models, the R² values for AOAAO-LSTPencoder increased by 57.17% and 12.77% (Tangnaihai), and 31.79% and 5.28% (Dongjiang), respectively.
• AOAAO-LSTPencoder consistently achieved the lowest Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) values at both stations. For Tangnaihai, MAE was 71.72. For Dongjiang, RMSE was 23.27 and KGE was 0.95.
• The AOAAO algorithm effectively navigated the high-dimensional hyperparameter space, identifying non-intuitive optimal parameter combinations and accelerating model convergence.
• Feature space analysis showed that the Encoder module captures global long-term dependencies (multi-centered clusters), while the LSTM module preserves local temporal continuity (continuous, banded structures), demonstrating their complementary roles.

Contributions

• Proposed a novel LSTPencoder daily runoff sequence prediction model that replaces the Transformer's Decoder with a parallel computation module of multi-layer LSTM units, enabling simultaneous training and improved utilization of time series features for complex univariate prediction tasks.
• Introduced a multi-angle feature extraction method and a BatchNorm1d layer within the LSTPencoder model to optimize multi-directional correlations of time series features, enhance feature sharing, address internal covariate shift, and accelerate convergence.
• Applied the Adaptive-Operator-based Arithmetic Optimization Algorithm (AOAAO) for the first time to achieve end-to-end hyperparameter optimization of this hybrid model, effectively addressing the "black-box" tuning challenge and significantly enhancing model convergence speed and generalization capability.
• Demonstrated the model's high accuracy using only a single runoff time series as input, highlighting its potential for hydrological forecasting in regions with limited meteorological data or ungauged basins.

Funding

• Key Special Projects of National Key Research and Development Program on “Major Natural Disasters and Public Safety” (No: 2024YFC3012300)
• Henan Province Centrally Guided Local Science and Technology Development Fund Projects for 2024 (No: Z20241471017)

Citation

@article{Xu2026Integrating,
  author = {Xu, Dong-mei and Hong, Yang-hao and Wang, Wenchuan and Gu, Miao and Wang, Jun. and Zhao, Yanwei and Zang, Hong-fei},
  title = {Integrating LSTM and Transformer for Improved Daily Runoff Prediction: A Parallel Computing Approach},
  journal = {Modeling Earth Systems and Environment},
  year = {2026},
  doi = {10.1007/s40808-026-02738-3},
  url = {https://doi.org/10.1007/s40808-026-02738-3}
}