Zhao et al. (2025) Evaluation of climate prediction models in Yunnan, China: traditional methods and AI approaches

Identification

• Journal: Scientific Reports
• Year: 2025
• Date: 2025-12-08
• Authors: Junfan Zhao, Fan Zhao, Hang Deng
• DOI: 10.1038/s41598-025-27326-w

Research Groups

College of Big Data and Intelligent Engineering, Southwest Forestry University, Kunming, China

Short Summary

This study evaluates five artificial intelligence (AI) models (CNN, LSTM, Transformer, CNN-LSTM, LSTM-Transformer) against a traditional regional climate model (RegCM) for predicting daily temperature, precipitation, and relative humidity in Yunnan, China. The results demonstrate that AI models, particularly LSTM-Transformer and CNN-LSTM, significantly outperform RegCM, offering a data-driven basis for improved climate risk assessment in complex terrains.

Objective

• To systematically evaluate the performance of five advanced artificial intelligence (AI) models (CNN, LSTM, Transformer, CNN-LSTM, and LSTM-Transformer) in predicting daily temperature, precipitation, and relative humidity in Yunnan Province, China.
• To compare the predictive accuracy and stability of these AI models against a traditional regional climate model (RegCM) in a region with complex terrain and significant spatial heterogeneity.
• To reveal the advantages of AI models in capturing local climate nonlinear characteristics and explore effective pathways for climate modeling to support climate risk assessment and early warning applications.

Study Configuration

• Spatial Scale: Yunnan Province, China, covering 25 national meteorological stations across the Yungui Plateau with diverse topography (mountains, hills, basins). RegCM simulations used a horizontal resolution of 10 km.
• Temporal Scale: Daily meteorological observations from 2004 to 2018 (15 years) were used for training, validation, and testing. Models predicted the next day's target variable using a sliding window of 120 days to capture temporal dependencies.

Methodology and Data

• Models used:
  • Traditional Regional Climate Model (RCM): RegCM (specifically RegCM37, configured based on CORDEX-EA Phase I experimental framework).
  • Artificial Intelligence (AI) Models:
    • Convolutional Neural Network (CNN)
    • Long Short-Term Memory (LSTM)
    • Transformer
    • CNN-LSTM (hybrid model)
    • LSTM-Transformer (hybrid model)
• Data sources:
  • Observational Data: Daily meteorological observations from 25 national stations in Yunnan Province, China, obtained from the China National Meteorological Data Center (http://data.cma.cn), spanning 2004–2018.
    • Core Variables: Average, maximum, and minimum temperature; segmented precipitation (20:00–08:00 and 08:00–20:00) and total precipitation (20:00–20:00); average and minimum relative humidity; wind speed (average, maximum, and extreme); and sunshine duration.
    • Derived Features: Daily temperature range, sine and cosine encodings of month and day, and seasonal indicators (spring, summer, autumn, winter), totaling 18 features.
  • RegCM Input Data: ERA-Interim reanalysis data (initial field and lateral boundaries), NOAA OISST Sea surface temperature, and global baseline data (USGS topography, MODIS land cover).
  • Preprocessing: Z-score normalization, Principal Component Analysis (PCA) to reduce 18 features to 9 components explaining 93.2% of variance, sliding window (120 days), data augmentation, and chronological split into training (2004–2015, 80%), validation (2016, 10%), and testing (2017–2018, 10%) sets.
  • Evaluation Metrics: Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Pearson correlation coefficient (R).

Main Results

• Overall Performance: AI models consistently achieved higher accuracy and greater stability than the traditional RegCM across all predicted variables. Hybrid AI models (CNN-LSTM and LSTM-Transformer) generally showed the best performance.
• Temperature Prediction:
  • LSTM-Transformer achieved the highest accuracy: RMSE = 0.7410 °C, MAE = 0.5760 °C, R = 0.9938 (test set).
  • RegCM showed the lowest performance: RMSE = 3.5519 °C, MAE = 2.2880 °C, R = 0.8475 (test set).
• Precipitation Prediction:
  • CNN-LSTM was the most effective: RMSE = 4.7260 mm, MAE = 2.5422 mm, R = 0.8559 (test set).
  • RegCM showed the poorest results: RMSE = 4.8871 mm, MAE = 2.9323 mm, R = 0.4585 (test set).
• Relative Humidity Prediction:
  • LSTM-Transformer achieved the highest accuracy: RMSE = 3.7054 %, MAE = 2.8825 %, R = 0.9710 (test set).
  • RegCM showed the lowest accuracy: RMSE = 4.8233 %, MAE = 3.0280 %, R = 0.8443 (test set).
• Comprehensive Comparison: LSTM-Transformer attained the highest overall comprehensive score (0.7043), closely followed by CNN-LSTM (0.7022), indicating superior adaptability and stability across multiple climate variables.

Contributions

• Provides a systematic and comprehensive evaluation framework for comparing traditional regional climate models with advanced AI models for climate prediction in complex terrains.
• Demonstrates that AI models, particularly hybrid architectures like LSTM-Transformer and CNN-LSTM, significantly outperform traditional RegCM in predicting daily temperature, precipitation, and relative humidity in a region with high spatial heterogeneity.
• Highlights the potential of data-driven AI approaches to capture complex nonlinear climate patterns more effectively and efficiently than physically-based models in specific regional contexts.
• Offers a data-driven basis for developing more accurate and robust climate risk assessment and early warning systems, especially crucial for regions prone to climate-related disasters.
• Emphasizes the value of integrating temporal modeling (LSTM) with attention mechanisms (Transformer) and spatial feature extraction (CNN) for enhanced multivariate climate prediction.

Funding

• Yunnan Fundamental Research Projects, China (Grant No. 202301AT070223)
• Yunnan Reserve Projects for Young and Middle-Aged Academic and Technological Leading Talent (Grant No. 202405AC350034)
• National Natural Science Foundation of China (Grant No. 32160374)

Citation

@article{Zhao2025Evaluation,
  author = {Zhao, Junfan and Zhao, Fan and Deng, Hang},
  title = {Evaluation of climate prediction models in Yunnan, China: traditional methods and AI approaches},
  journal = {Scientific Reports},
  year = {2025},
  doi = {10.1038/s41598-025-27326-w},
  url = {https://doi.org/10.1038/s41598-025-27326-w}
}