Jeon et al. (2025) Multimodal Optical Biosensing and 3D-CNN Fusion for Phenotyping Physiological Responses of Basil Under Water Deficit Stress

Identification

• Journal: Agronomy
• Year: 2025
• Date: 2025-12-24
• Authors: Yu-Jin Jeon, Hyoung Seok Kim, Taek Sung Lee, Soo Hyun Park, Heesup Yun, Dae-Hyun Jung
• DOI: 10.3390/agronomy16010055

Research Groups

• Department of Smart Farm Science, Kyung Hee University, Yongin 17104, Republic of Korea
• Interdisciplinary Program in IT-Bio Convergence System, Kyung Hee University, Yongin 17104, Republic of Korea
• Smart Farm Research Center, Korea Institute of Science and Technology (KIST), Gangneung 25451, Republic of Korea
• Department of Biological and Agricultural Engineering, University of California, Davis, CA 95616, USA

Short Summary

This study developed a multimodal optical biosensing and 3D convolutional neural network (3D-CNN) fusion framework to non-destructively phenotype basil's physiological responses (normal, resistance, recovery) under water deficit stress, achieving 96.9% classification accuracy by integrating RGB, depth, and chlorophyll fluorescence imaging.

Objective

• To develop and evaluate a 3D-CNN-based multimodal optical biosensing framework for classifying basil's physiological responses (normal, resistance, and recovery) under water deficit stress.

Study Configuration

• Spatial Scale: Basil plants (Ocimum basilicum L.) were cultivated hydroponically in a custom-built growth and imaging chamber. Regions of interest (ROIs) corresponding to basil leaves were extracted and resized to 32 × 32 pixels. A 3D fusion cube of size 130 × 32 × 32 (spectral × height × width) was constructed for each ROI.
• Temporal Scale: Basil plants underwent 9 days of drainage and re-watering treatment. Data collection was performed three times a day (morning, afternoon, evening). Chlorophyll fluorescence (CF) imaging involved 20 minutes of dark adaptation, exposure to actinic light, and four saturating flashes to capture dynamic physiological responses.

Methodology and Data

• Models used:
  • Deep Learning: 3D Convolutional Neural Network (3D-CNN), 2D Convolutional Neural Network (2D-CNN).
  • Traditional Machine Learning: Logistic Regression, k-Nearest Neighbors (k-NN), Support Vector Machine (SVM), Light Gradient Boosting Machine (LightGBM).
• Data sources:
  • Multimodal optical biosensing data: RGB images, Depth images (Intel® RealSense™ LiDAR Camera L515), Chlorophyll Fluorescence (CF) images (Basler ace acA1300-60gm-NIR GigE camera with 6 mm C Series VIS-NIR fixed focal length lens and longpass OD4 650 nm 12.5 mm filter).
  • Derived data: 31 types of physiologically significant chlorophyll fluorescence parameters (calculated from 13 directly measured CF images).
  • Fusion data: 130 optical parameter layers (6 RGB-Depth channels, 93 RGB-mapped CF parameter layers, and 31 single-channel CF parameter maps) assembled into a 3D fusion cube.
  • Thermal images were acquired for physiological validation but not included in the fusion cube for 3D-CNN training.

Main Results

• The multimodal 3D-CNN fusion framework achieved a classification accuracy of 96.9% for phenotyping basil's physiological responses (Normal, Resistance, Recovery) under water deficit stress.
• This performance significantly surpassed traditional machine learning classifiers (maximum 60.77% accuracy for SVM) and a 2D-CNN model (76.79% accuracy).
• An ablation study confirmed that full multimodal fusion (RGB + Depth + CF) yielded the highest accuracy (96.90%), with chlorophyll fluorescence (CF) being the most informative single modality (84.53% accuracy).
• t-SNE visualization demonstrated that the 3D-CNN learned distinct and biologically meaningful latent representations, clearly separating Normal, Resistance, and Recovery states, with the Recovery state forming an intermediate continuum.
• ROC analysis showed consistently high Area Under the Curve (AUC) values for all classes (Normal: 0.90, Resistance: 0.93, Recovery: 0.92), indicating robust separability.
• The 3D-CNN model has approximately 1.73 million trainable parameters (model size of approximately 6.6 MB) and an average inference time of approximately 5 ms per region of interest (ROI) on an NVIDIA RTX-3090 GPU.

Contributions

• Developed a novel multimodal optical biosensing and 3D-CNN fusion framework for non-destructive, real-time monitoring of crop water stress.
• Integrated RGB, depth, and time-resolved chlorophyll fluorescence data into a unified 3D fusion cube, effectively capturing both spatial and temporal–spectral features of plant physiological state transitions.
• Demonstrated superior classification performance and learned biologically meaningful feature representations compared to existing traditional machine learning and 2D-CNN approaches.
• Provided a scalable and interpretable approach for adaptive irrigation control and intelligent environmental management, advancing AI-driven precision agriculture.

Funding

• Regional Innovation System & Education (RISE) program through the Gyeonggi RISE Center, funded by the Ministry of Education (MOE) and the Gyeonggi-do, Republic of Korea (2025-RISE-09-A07).

Citation

@article{Jeon2025Multimodal,
  author = {Jeon, Yu-Jin and Kim, Hyoung Seok and Lee, Taek Sung and Park, Soo Hyun and Yun, Heesup and Jung, Dae-Hyun},
  title = {Multimodal Optical Biosensing and 3D-CNN Fusion for Phenotyping Physiological Responses of Basil Under Water Deficit Stress},
  journal = {Agronomy},
  year = {2025},
  doi = {10.3390/agronomy16010055},
  url = {https://doi.org/10.3390/agronomy16010055}
}