Intelligent Recognition Method of Rice Growth Stage Based on Multimodal Fusion

doi:10.16819/j.1001-7216.2026.250703

Abstract

Abstract: 【Objective】The growth period of rice is an important reference indicator for irrigation, fertilization, and pest control measures in rice planting management. Researching an intelligent recognition method for rice growth period based on multimodal fusion, improving the accuracy of rice growth period recognition, is of great significance for optimizing rice planting management measures and increasing rice yield. 【Method】To address the vulnerability of RGB images to lighting, an H-color channel image is extracted, and a dual-branch rice growth stage image recognition model based on residual network and attention mechanism is established; Due to the influence of meteorological factors on the length of rice growth period, a rice growth period meteorological identification model based on three meteorological factors and a bidirectional long-short period network is established; Using decision trees for decision-level multimodal feature fusion, multiple models were trained and tested on a self-built rice growth period dataset.【Result】The improved rice growth period image recognition model achieved an accuracy of 84.25%, which is superior to traditional image recognition models such as ResNet50 and ConvNeXt. The multi-modality model based on images and meteorological achieved an accuracy of 87.66% in identifying 9 rice growth stages, which is superior to the single modality recognition model based on images and meteorology.【Conclusion】The multimodal recognition model based on image and meteorological fusion compensates for the shortcomings of single modal recognition methods. Compared with single modal recognition models, it has significantly improved the accuracy of identifying 9 rice growth stages.

Key words: rice growth period, multimodal, image modality, meteorological modality, decision tree

摘要： 【目的】水稻生育期是水稻种植管理过程中灌溉、施肥和病虫害防控措施的重要参考指标。研究基于多模态融合的水稻生育期智能识别方法，提高水稻生育期识别准确率，对优化水稻种植管理措施，提升水稻产量具有重要意义。【方法】针对RGB图像易受光照影响，提取H颜色通道图像，建立基于残差网络与注意力机制的双分支水稻生育期图像识别模型；由于气象因子影响水稻生育期长短，建立基于3个气象因子和双向长短期记忆网络的水稻生育期气象识别模型；利用决策树进行决策级多模态特征融合，在自建数据集上进行多种模型的训练和测试。【结果】改进的水稻生育期图像识别模型准确率达到84.25%，优于ResNet50、ConvNeXt等传统图像识别模型。基于图像和气象的多模态模型对9个水稻生育期识别准确率达到87.66%，优于单一图像和气象的单模态识别模型。【结论】基于图像与气象融合的多模态识别模型弥补了单模态识别方法的不足，相较于单模态识别模型，显著提升了对9个水稻生育期的识别准确率。

关键词: 水稻生育期, 多模态融合, 图像模态, 气象模态, 决策树

XU Yongwei, LIU Shuhua, FENG Zelin, LUO Ju, NI Zhaoxin, YANG Baojun, YAO Qing, LI Agen. Intelligent Recognition Method of Rice Growth Stage Based on Multimodal Fusion[J]. Chinese Journal OF Rice Science, 2026, 40(5): 663-678.

许永炜, 刘淑华, 冯泽霖, 罗举, 倪兆新, 杨保军, 姚青, 李阿根. 基于多模态融合的水稻生育期智能识别方法[J]. 中国水稻科学, 2026, 40(5): 663-678.

References

[1] 丁颖. 中国水稻栽培学[M]. 北京: 农业出版社, 1961. Ding Y. Rice Cultivation in China[M]. Beijing: Agriculture Press, 1961. (in Chinese) [2] Yoshida S. Fundamentals of Rice Crop Science[M]. Los Banos: International Rice Research Institute, 1981. [3] Counce P A, Keisling T C, Mitchell A J. A uniform, objective, and adaptive system for expressing rice development[J]. Crop Science, 2000, 40(2): 436-443. [4] Bouman B A M. ORYZA2000: Modeling Lowland Rice[M]. Los Banos: International Rice Research Institute, 2001. [5] Lampayan R M, Rejesus R M, Singleton G R, Bouman B A M. Adoption and economics of alternate wetting and drying water management for irrigated lowland rice[J]. Field Crops Research, 2015, 170: 95-108. [6] Dobermann A. Rice: Nutrient Disorders & Nutrient Management[M]. Los Banos: International Rice Research Institute, 2000. [7] 郝东川. 水稻病虫害诊断与防治技术[M]. 广州: 广东科技出版社, 2024. Hao D C. Diagnosis and Control Techniques for Rice Diseases and Pests [M]. Guangzhou: Guangdong Science and Technology Press, 2024. [8] 杜余能, 贾贤生, 陈永忠, 杨兵, 胡超, 程大流. 水稻稻曲病防治适期研究[J]. 安徽农业科学, 2004, 32(1): 39. Du Y N, Jia X S, Chen Y Z, Yang B, Hu C, Cheng D L. Study on the suitable time of the control of false smut of rice plant[J]. Journal of Anhui Agricultural Sciences, 2004, 32(1): 39. (in Chinese with English abstract) [9] 冯健昭, 潘永琪, 熊悦淞, 吴彻, 肖德琴. 基于mRMR-XGBoost的水稻关键生育期识别[J]. 农业工程学报, 2024, 40(15): 111-118. Feng J Z, Pan Y Q, Xiong Y S, Wu C, Xiao D Q. Rice key growth stage identification based on mRMR-XGBoost[J]. Transactions of the Chinese Society of Agricultural Engineering, 2024, 40(15): 111-118. (in Chinese with English abstract) [10] 张梦茹. 基于TCN-LSTM深度学习模型的水稻生育期卫星遥感识别[D]. 武汉: 湖北大学, 2022. Zhang M R. Satellite remote sensing identification of rice growth period based on TCN-LSTM deep learning model[D]. Wuhan: Hubei University, 2022. (in Chinese with English abstract) [11] 杨振忠, 方圣辉, 彭漪, 龚龑, 王东. 基于机器学习结合植被指数阈值的水稻关键生育期识别[J]. 中国农业大学学报, 2020, 25(1): 76-85. Yang Z Z, Fang S H, Peng Y, Gong Y, Wang D. Recognition of the rice growth stage by machine learning combined with vegetation index threshold[J]. Journal of China Agricultural University, 2020, 25(1): 76-85. (in Chinese) [12] 徐建鹏, 王杰, 徐祥, 琚书存. 基于RAdam卷积神经网络的水稻生育期图像识别[J]. 农业工程学报, 2021, 37(8): 143-150. Xu J P, Wang J, Xu X, Ju S C. Image recognition for different developmental stages of rice by RAdam deep convolutional neural networks[J]. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(8): 143-150. (in Chinese) [13] Qin J, Hu T, Yuan J, Liu Q, Wang W, Liu J, Guo L, Song G. Deep-learning-based rice phenological stage recognition[J]. Remote Sensing, 2023, 15(11): 2891. [14] Yang Q, Shi L, Han J, Yu J, Huang K. A near real-time deep learning approach for detecting rice phenology based on UAV images[J]. Agricultural and Forest Meteorology, 2020, 287: 107938. [15] 许轲, 孙圳, 霍中洋, 戴其根, 张洪程, 刘俊, 宋云生, 杨大柳, 魏海燕, 吴爱国, 王显, 吴冬冬. 播期、品种类型对水稻产量、生育期及温光利用的影响[J]. 中国农业科学, 2013, 46(20): 4222-4233. Xu K, Sun Z, Huo Z Y, Dai Q G, Zhang H C, Liu J, Song Y S, Yang D L, Wei H Y, Wu A G, Wang X, Wu D D. Effects of seeding date and variety type on yield, growth stage and utilization of temperature and sunshine in rice[J]. Scientia Agricultura Sinica, 2013, 46(20): 4222-4233. (in Chinese with English abstract) [16] 苏李君, 刘云鹤, 王全九. 基于有效积温的中国水稻生长模型的构建[J]. 农业工程学报, 2020(1): 162-174. Su L J, Liu Y H, Wang Q J. Rice growth model in China based on growing degree days[J]. Transactions of the Chinese Society of Agricultural Engineering, 2020(1): 162-174. (in Chinese with English abstract) [17] 蔡昆争, 骆世明. 不同生育期遮光对水稻生长发育和产量形成的影响[J]. 应用生态学报, 1999, 10(2): 193-196. Cai K Z, Luo S M. Effect of shading on growth, development and yield formation of rice[J]. Chinese Journal of Applied Ecology, 1999, 10(2): 193-196. (in Chinese) [18] Shimono H, Okada M, Kanda E, Arakawa I. Low temperature-induced sterility in rice: Evidence for the effects of temperature before panicle initiation[J]. Field Crops Research, 2007, 101(2): 221-231. [19] Yano M, Katayose Y, Ashikari M, Yamanouchi U, Monna L, Fuse T, Baba T, Yamamoto K, Umehara Y, Nagamura Y, Sasaki T. Hd1, a major photoperiod sensitivity quantitative trait locus in rice, is closely related to the Arabidopsis flowering time gene CONSTANS[J]. The Plant Cell, 2000, 12(12): 2473-2483. [20] Katsura K, Maeda S, Lubis I, Horie T, Cao W, Shiraiwa T. The high yield of irrigated rice in Yunnan, China[J]. Field Crops Research, 2008, 107(1): 1-11. [21] Yin X, Kropff M J, McLaren G, Visperas R M. A nonlinear model for crop development as a function of temperature[J]. Agricultural and Forest Meteorology, 1995, 77(1/2): 1-16. [22] Guo Y, Liu Y, Georgiou T, Lew M S. A review of semantic segmentation using deep neural networks[J]. International Journal of Multimedia Information Retrieval, 2018, 7(2): 87-93. [23] Geusebroek J M, van den Boomgaard R, Smeulders A W M, Geerts H. Color invariance[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(12): 1338-1350. [24] Kolkur S, Kalbande D, Shimpi P, Bapat C, Jatakia J. Human skin detection using RGB, HSV and YCbCr color models[EB/OL]. 2017: arXiv: 1708.02694. https://arxiv. org/abs/1708.02694 [25] Vaswani A, Shazeer N, Parmar N. Attention is all your need[J]. Advances in Neural Information Processing Systems, 2017, 30. [26] Le P, Zuidema W. Quantifying the vanishing gradient and long distance dependency problem in recursive neural networks and recursive LSTMs[EB/OL]. 2016: arXiv: 1603.00423. https://arxiv.org/abs/1603.00423 [27] Schuster M, Paliwal K K. Bidirectional recurrent neural networks[J]. IEEE Transactions on Signal Processing, 1997, 45(11): 2673-2681. [28] Yu Y, Si X, Hu C, Zhang J. A review of recurrent neural networks: LSTM cells and network architectures[J]. Neural Computation, 2019, 31(7): 1235-1270. [29] Ramachandram D, Taylor G W. Deep multimodal learning: A survey on recent advances and trends[J]. IEEE Signal Processing Magazine, 2017, 34(6): 96-108. [30] Loh W Y. Classification and regression trees[J]. WIREs Data Mining and Knowledge Discovery, 2011, 1(1): 14-23. [31] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90. [32] Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 7-12, 2015. Boston, MA, USA. IEEE, 2015: 1-9. [33] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016. Las Vegas, NV, USA: IEEE, 2016: 770-778. [34] Liu Z, Mao H, Wu C Y, Feichtenhofer C, Darrell T, Xie S. A ConvNet for the 2020s[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 18-24, 2022. New Orleans, LA, USA: IEEE, 2022: 11966-11976. [35] Dosovitskiy A, Beyer L, Kolesnikov A. An image is worth 16x16 words: Transformers for image recognition at scale[J]. ICLR, 2020: arXiv preprint arXiv: 2010.11929. [36] Tan S, Lu H, Yu J, Lan M, Hu X, Zheng H, Peng Y, Wang Y, Li Z, Qi L, Ma X. In-field rice panicles detection and growth stages recognition based on RiceRes2Net[J]. Computers and Electronics in Agriculture, 2023, 206: 107704. [37] van Oort P A J, Zhang T, de Vries M E, Heinemann A B, Meinke H. Correlation between temperature and phenology prediction error in rice (Oryza sativa L.)[J]. Agricultural and Forest Meteorology, 2011, 151(12): 1545-1555. [38] Blockeel H, Devos L, Frénay B, Nanfack G, Nijssen S. Decision trees: From efficient prediction to responsible AI[J]. Frontiers in Artificial Intelligence, 2023, 6: 1124553.