2025 et al. (2025) CLLMate: A Multimodal Benchmark for Weather and Climate Events Forecasting

Identification

• Journal: Open MIND
• Year: 2025
• Date: 2025-10-10
• Authors: Association for Computational Linguistics 2025, Lau, Alexis Kai Hon, Li, Haobo, Qu, Huamin, Wang, Zhaowei, Wang, Jiachen, WANG, Yueya
• DOI: 10.48448/f85m-y402

Research Groups

Not explicitly available in the provided text, but mentions "Jiaxin Bai and 6 other authors".

Short Summary

This paper introduces Weather and Climate Event Forecasting (WCEF), a new task that leverages numerical meteorological raster data and textual event data to predict weather and climate events. To facilitate this, the authors present CLLMate, the first multimodal dataset for WCEF, and benchmark 32 existing models, revealing their advantages and limitations for this task.

Objective

• To propose Weather and Climate Event Forecasting (WCEF) as a novel task aimed at predicting weather and climate events by translating numerical meteorological variables into actionable textual narratives.
• To address the lack of supervised datasets and multimodal data alignment challenges by presenting CLLMate, the first multimodal dataset for WCEF.
• To systematically benchmark existing models on CLLMate to evaluate their performance and highlight the dataset's value for training and benchmarking in WCEF.

Study Configuration

• Spatial Scale: Not explicitly defined, but implied global coverage through the use of ERA5 reanalysis data.
• Temporal Scale: Not explicitly defined for the dataset's temporal span, but comprises 26,156 environmental news articles.

Methodology and Data

• Models used: 32 existing models, including closed-source, open-source, and fine-tuned models (specific model names not provided in the abstract).
• Data sources:
  • 26,156 environmental news articles (textual event data).
  • ERA5 reanalysis data (numerical meteorological raster data).

Main Results

• Introduction of CLLMate, the first multimodal dataset specifically designed for Weather and Climate Event Forecasting (WCEF), which aligns 26,156 environmental news articles with ERA5 reanalysis data.
• Systematic benchmarking of 32 diverse models on the CLLMate dataset was performed.
• Experiments revealed the advantages and limitations of existing Multimodal Large Language Models (MLLMs) when applied to the WCEF task.
• The study demonstrated the significant value of the CLLMate dataset for both training and benchmarking models in the context of weather and climate event forecasting.

Contributions

• Proposal of Weather and Climate Event Forecasting (WCEF) as a new and challenging task that bridges the gap between numerical meteorological predictions and actionable textual narratives of events.
• Creation of CLLMate, the first multimodal dataset for WCEF, providing a crucial resource for research in this emerging field by aligning environmental news articles with ERA5 reanalysis data.
• A comprehensive benchmark of 32 diverse models on the new CLLMate dataset, offering insights into the current capabilities and limitations of existing Multimodal Large Language Models (MLLMs) for multimodal environmental forecasting.

Funding

Not available in the provided text.

Citation

@article{20252025CLLMate,
  author = {2025, Association for Computational Linguistics and Lau, Alexis Kai Hon and Li, Haobo and Qu, Huamin and Wang, Zhaowei and Wang, Jiachen and WANG, Yueya},
  title = {CLLMate: A Multimodal Benchmark for Weather and Climate Events Forecasting},
  journal = {Open MIND},
  year = {2025},
  doi = {10.48448/f85m-y402},
  url = {https://doi.org/10.48448/f85m-y402}
}