基于多模态融合的水稻生育期智能识别方法

doi:10.16819/j.1001-7216.2026.250703

摘要/Abstract

摘要：

【目的】水稻生育期是水稻种植管理过程中灌溉、施肥和病虫害防控措施的重要参考指标。研究基于多模态融合的水稻生育期智能识别方法，提高水稻生育期识别准确率，对优化水稻种植管理措施，提升水稻产量具有重要意义。【方法】针对RGB图像易受光照影响，提取H颜色通道图像，建立基于残差网络与注意力机制的双分支水稻生育期图像识别模型；由于气象因子影响水稻生育期长短，建立基于3个气象因子和双向长短期记忆网络的水稻生育期气象识别模型；利用决策树进行决策级多模态特征融合，在自建数据集上进行多种模型的训练和测试。【结果】改进的水稻生育期图像识别模型准确率达到84.25%，优于ResNet50、ConvNeXt等传统图像识别模型。基于图像和气象的多模态模型对9个水稻生育期识别准确率达到87.66%，优于单一图像和气象的单模态识别模型。【结论】基于图像与气象融合的多模态识别模型弥补了单模态识别方法的不足，相较于单模态识别模型，显著提升了对9个水稻生育期的识别准确率。

关键词: 水稻生育期, 多模态融合, 图像模态, 气象模态, 决策树

Abstract:

【Objective】The growth period of rice is an important reference indicator for irrigation, fertilization, and pest control measures in rice planting management. Researching an intelligent recognition method for rice growth period based on multimodal fusion, improving the accuracy of rice growth period recognition, is of great significance for optimizing rice planting management measures and increasing rice yield. 【Method】To address the vulnerability of RGB images to lighting, an H-color channel image is extracted, and a dual-branch rice growth stage image recognition model based on residual network and attention mechanism is established; Due to the influence of meteorological factors on the length of rice growth period, a rice growth period meteorological identification model based on three meteorological factors and a bidirectional long-short period network is established; Using decision trees for decision-level multimodal feature fusion, multiple models were trained and tested on a self-built rice growth period dataset.【Result】The improved rice growth period image recognition model achieved an accuracy of 84.25%, which is superior to traditional image recognition models such as ResNet50 and ConvNeXt. The multi-modality model based on images and meteorological achieved an accuracy of 87.66% in identifying 9 rice growth stages, which is superior to the single modality recognition model based on images and meteorology.【Conclusion】The multimodal recognition model based on image and meteorological fusion compensates for the shortcomings of single modal recognition methods. Compared with single modal recognition models, it has significantly improved the accuracy of identifying 9 rice growth stages.

Key words: rice growth period, multimodal, image modality, meteorological modality, decision tree

许永炜, 刘淑华, 冯泽霖, 罗举, 倪兆新, 杨保军, 姚青, 李阿根. 基于多模态融合的水稻生育期智能识别方法[J]. 中国水稻科学, 2026, 40(5): 663-678.

XU Yongwei, LIU Shuhua, FENG Zelin, LUO Ju, NI Zhaoxin, YANG Baojun, YAO Qing, LI Agen. Intelligent Recognition Method of Rice Growth Stage Based on Multimodal Fusion[J]. Chinese Journal OF Rice Science, 2026, 40(5): 663-678.

图/表 21

图1 基于多模态的水稻生育期智能识别研究技术路线

Fig. 1. Research technology roadmap for intelligent recognition of rice growth period based on multimodal data

图2 水稻9个生育期图像 A: 苗期; B: 分蘖期; C: 穗分化期; D: 抽穗期; E: 开花期; F: 乳熟期；G: 蜡熟期; H: 黄熟期; I: 完熟期。

Fig. 2. Images of nine growth stages in rice A, Seedling stage; B, Tillering stage; C, Panicle initiation stage; D, Heading stage; E, Flowering stage; F, Milk stage; G, Dough period; H, Yellow ripening stage; I, Full-ripening stage.

图3 乳熟期水稻图像数据增强 A: 原图; B: 添加高斯噪声; C: 旋转; D: 亮度增强; E: 对比度增强。

Fig. 3. Enhancement of image data of rice during milk ripening period A, Original image; B, Add Gaussian noise; C, Rotate; D, Brightness enhancement; E, Contrast enhancement.

图4 2024年水稻全生育期3种气象数据曲线 A: 每日太阳能辐射值; B: 每日平均温度值; C: 累积生长度日值。

Fig. 4. Curve charts of three meteorological data during rice entire growth duration in 2024 A, Daily solar radiation value; B, Daily average temperature value; C, Accumulated growing degree day value.

图5 BiIRiceGS模型结构

Fig. 5. Structure diagram of BiIRiceGS model

图6 不同亮度下不同生育期水稻图像在RGB通道的分布情况 A: 不同亮度下R通道像素平均分布; B: 不同亮度下G通道像素平均分布; C: 不同亮度下B通道像素平均分布。

Fig. 6. Distribution of rice plant images during various growth stages in RGB channels under different brightness levels A, Average distribution of R channel pixels under different brightness levels; B, Average distribution of G channel pixels under different brightness levels; C, Average distribution of B channel pixels under different brightness levels.

图7 水稻相邻生育期末期与初期图像 A：开花期末期；B：乳熟期初期；C：乳熟期末期；D：蜡熟期初期；E：蜡熟期末期；F：黄熟期初期；G：黄熟期末期；H：完熟期初期。

Fig. 7. Photographs of rice plants at successive late and early growth stages A, Late flowering stage; B, Early milky-ripening stage; C, Late milky-ripening stage; D, Early dough-ripening stage; E, Late dough-ripening stage; F, Early yellow-ripening stage; G, Late yellow-ripening stage; H, Early full -ripening stage.

图8 九个水稻生育期图像在不同通道像素分布的脊状图 A: RGB格式R通道; B: RGB格式G通道; C: RGB格式B通道; D: HSV格式H通道; E: HSV格式S通道; F: HSV格式V通道; G: LAB格式L通道; H: LAB格式A通道; I: LAB格式B通道。

Fig. 8. Ridge plots of pixel distribution in different color channels for random images at nine growth stages A, RGB format R channel; B, RGB format G channel; C, RGB format B channel; D, HSV format H channel; E, HSV format S channel; F, HSV format V channel; G, LAB format L channel; H, LAB format A channel; I, LAB format B channel.

图9 原始图像和颜色注意力增强图像

Fig. 9. Original image and color attention enhanced image

图10 基于气象数据的水稻生育期识别模型MDRiceGS网络结构

Fig. 10. MDRiceGS network structure diagram of rice growth period recognition model based on meteorological data

图11 LSTM单元结构

Fig. 11. LSTM unit structure diagram

图12 水稻生育期识别决策树模型结构

Fig. 12. Structure diagram of decision tree model for rice growth period recognition

表1 基于残差网络的BiIRiceGS模型消融实验

Table 1. Ablation experiment of BiIRiceGS model based on Resnet

颜色注意力模块 Color attention module	残差注意力模块 Residual attention module	准确率 Accuracy	精确率 Precision	召回率 Recall	F₁值 F₁ value
×	×	77.17	77.42	77.08	77.13
√	×	82.33	82.55	82.25	82.26
×	√	79.09	79.15	79.02	78.99
√	√	84.25	84.46	84.20	84.20

表2 不同模型对水稻生育期识别性能对比

Table 2. Comparison of recognition performance of different models for rice growth period

模型 Model	准确率 Accuracy	精确率 Precision	召回率 Recall	F₁值 F₁ value
AlexNet	72.76	73.44	73.06	72.80
GoogleNet	76.60	77.29	77.15	76.67
ResNet50	80.80	80.93	81.13	80.71
ConvNeXt	76.08	76.82	76.61	69.92
Vit-B-16	74.54	75.31	74.92	74.64
BiIRiceGS	84.25	84.46	84.20	84.20

表3 BiIRiceGS模型针对不同品种水稻的识别结果

Table 3. Recognition results of BiIRiceGS model for different varieties of rice

水稻品种 Rice variety	准确率 Accuracy	精确率 Precision	召回率 Recall	F₁值 F₁ value
黄华占Huanghuazhan	84.38	84.58	84.27	83.87
C两优华占C Liangyouhuazhan	83.21	83.56	82.97	82.71
甬优7872 Yongyou 7872	86.91	87.31	87.03	87.10
春优927 Chunyou 927	82.31	82.84	81.96	82.09

表4 不同BiLSTM层数的MDRiceGS对水稻生育期的识别结果

Table 4. Recognition results of rice growth period using MDRiceGS with different BiLSTM layers

BiLSTM层数 BiLSTM layer	准确率 Accuracy(%)	精确率 Precision(%)	召回率 Recall(%)	F₁值 F₁ value (%)	参数量 Parameter quantity
1	76.80	76.87	76.76	76.85	9.35M
2	77.51	77.49	77.44	77.43	9.74M
3	77.87	77.82	77.75	77.74	10.14M
4	77.34	77.31	77.25	77.24	10.53M

表5 MDRiceGS模型针对不同品种水稻生育期的识别结果

Table 5. Recognition results of MDRiceGS model for growth period of different rice varieties

水稻品种 Rice variety	准确率 Accuracy	精确率 Precision	召回率 Recall	F₁值 F₁ value
黄华占Huanghuazhan	77.78	78.36	77.50	77.78
C两优华占 C Liangyouhuazhan	77.86	77.84	77.67	77.51
甬优7872 Yongyou 7872	78.19	78.00	78.05	77.91
春优927 Chunyou 927	77.62	77.64	77.47	77.21
平均值 Mean	77.87	77.82	77.75	77.74

表6 不同决策树深度对水稻生育期的识别结果

Table 6. Recognition results of rice growth period based on different decision tree depths

决策树深度 Decision tree depth	准确率 Accuracy(%)	精确率 Precision(%)	召回率 Recall(%)	F₁值 F₁ value(%)	参数量 Parameter quantity
4	71.24	71.27	71.24	71.05	4.86 kb
5	86.53	86.43	86.42	86.49	7.25 kb
6	87.66	87.54	87.56	87.53	9.64 kb
7	87.66	87.54	87.56	87.53	12.8 kb

表7 不同融合算法对水稻生育期的识别结果

Table 7. Recognition results of rice growth period using different fusion algorithms

融合算法 Fusion algorithm	准确率 Accuracy (%)	精确率 Precision(%)	召回率 Recall(%)	F₁值 F₁ value(%)	融合模块参数量 Fusion module parameter quantity
多层感知机 Multilayer perceptrons	81.45	81.33	81.32	81.30	1.25 Mb
随机森林 Random forest	87.75	87.61	87.65	87.61	87.60 kb
决策树(深度为6) Decision tree (depth of 6)	87.66	87.54	87.56	87.53	9.64 kb

表8 MultiMRiceGS模型对四种水稻生育期的识别结果

Table 8. Recognition results of growth stages of four types of rice using MultiMRiceGS model

水稻品种 Rice variety	准确率 Accuracy	精确率 Precision	召回率 Recall	F₁值 F₁ value
黄华占 Huanghuazhan	88.19	88.20	88.07	87.96
C两优华占 C Liangyouhuazhan	87.50	87.30	87.30	87.21
甬优7872 Yongyou 7872	87.92	88.32	87.96	87.88
春优927 Chunyou 927	87.00	87.11	86.87	86.91

图13 不同模型对9个水稻生育期识别的混淆矩阵 A: 图像模型混淆矩阵; B: 气象模型混淆矩阵; C: 多模态模型混淆矩阵。

Fig. 13. Confusion matrix for recognizing 9 rice growth stages using different models A, Image model confusion matrix; B, Meteorological model confusion matrix; C, Multimodal model confusion matrix.

参考文献 38

[1]	丁颖. 中国水稻栽培学[M]. 北京: 农业出版社, 1961.
	Ding Y. Rice Cultivation in China[M]. Beijing: Agriculture Press, 1961. (in Chinese)
[2]	Yoshida S. Fundamentals of Rice Crop Science[M]. Los Banos: International Rice Research Institute, 1981.
[3]	Counce P A, Keisling T C, Mitchell A J. A uniform, objective, and adaptive system for expressing rice development[J]. Crop Science, 2000, 40(2): 436-443.
[4]	Bouman B A M. ORYZA2000: Modeling Lowland Rice[M]. Los Banos: International Rice Research Institute, 2001.
[5]	Lampayan R M, Rejesus R M, Singleton G R, Bouman B A M. Adoption and economics of alternate wetting and drying water management for irrigated lowland rice[J]. Field Crops Research, 2015, 170: 95-108.
[6]	Dobermann A. Rice: Nutrient Disorders & Nutrient Management[M]. Los Banos: International Rice Research Institute, 2000.
[7]	郝东川. 水稻病虫害诊断与防治技术[M]. 广州: 广东科技出版社, 2024.
	Hao D C. Diagnosis and Control Techniques for Rice Diseases and Pests[M]. Guangzhou: Guangdong Science and Technology Press, 2024. (in Chinese)
[8]	杜余能, 贾贤生, 陈永忠, 杨兵, 胡超, 程大流. 水稻稻曲病防治适期研究[J]. 安徽农业科学, 2004, 32(1): 39.
	Du Y N, Jia X S, Chen Y Z, Yang B, Hu C, Cheng D L. Study on the suitable time of the control of false smut of rice plant[J]. Journal of Anhui Agricultural Sciences, 2004, 32(1): 39. (in Chinese with English abstract)
[9]	冯健昭, 潘永琪, 熊悦淞, 吴彻, 肖德琴. 基于mRMR-XGBoost的水稻关键生育期识别[J]. 农业工程学报, 2024, 40(15): 111-118.
	Feng J Z, Pan Y Q, Xiong Y S, Wu C, Xiao D Q. Rice key growth stage identification based on mRMR-XGBoost[J]. Transactions of the Chinese Society of Agricultural Engineering, 2024, 40(15): 111-118. (in Chinese with English abstract)
[10]	张梦茹. 基于TCN-LSTM深度学习模型的水稻生育期卫星遥感识别[D]. 武汉: 湖北大学, 2022.
	Zhang M R. Satellite remote sensing identification of rice growth period based on TCN-LSTM deep learning model[D]. Wuhan: Hubei University, 2022. (in Chinese with English abstract)
[11]	杨振忠, 方圣辉, 彭漪, 龚龑, 王东. 基于机器学习结合植被指数阈值的水稻关键生育期识别[J]. 中国农业大学学报, 2020, 25(1): 76-85.
	Yang Z Z, Fang S H, Peng Y, Gong Y, Wang D. Recognition of the rice growth stage by machine learning combined with vegetation index threshold[J]. Journal of China Agricultural University, 2020, 25(1): 76-85. (in Chinese)
[12]	徐建鹏, 王杰, 徐祥, 琚书存. 基于RAdam卷积神经网络的水稻生育期图像识别[J]. 农业工程学报, 2021, 37(8): 143-150.
	Xu J P, Wang J, Xu X, Ju S C. Image recognition for different developmental stages of rice by RAdam deep convolutional neural networks[J]. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(8): 143-150. (in Chinese)
[13]	Qin J, Hu T, Yuan J, Liu Q, Wang W, Liu J, Guo L, Song G. Deep-learning-based rice phenological stage recognition[J]. Remote Sensing, 2023, 15(11): 2891.
[14]	Yang Q, Shi L, Han J, Yu J, Huang K. A near real-time deep learning approach for detecting rice phenology based on UAV images[J]. Agricultural and Forest Meteorology, 2020, 287: 107938.
[15]	许轲, 孙圳, 霍中洋, 戴其根, 张洪程, 刘俊, 宋云生, 杨大柳, 魏海燕, 吴爱国, 王显, 吴冬冬. 播期、品种类型对水稻产量、生育期及温光利用的影响[J]. 中国农业科学, 2013, 46(20): 4222-4233.
	Xu K, Sun Z, Huo Z Y, Dai Q G, Zhang H C, Liu J, Song Y S, Yang D L, Wei H Y, Wu A G, Wang X, Wu D D. Effects of seeding date and variety type on yield, growth stage and utilization of temperature and sunshine in rice[J]. Scientia Agricultura Sinica, 2013, 46(20): 4222-4233. (in Chinese with English abstract)
[16]	苏李君, 刘云鹤, 王全九. 基于有效积温的中国水稻生长模型的构建[J]. 农业工程学报, 2020(1): 162-174.
	Su L J, Liu Y H, Wang Q J. Rice growth model in China based on growing degree days[J]. Transactions of the Chinese Society of Agricultural Engineering, 2020(1): 162-174. (in Chinese with English abstract)
[17]	蔡昆争, 骆世明. 不同生育期遮光对水稻生长发育和产量形成的影响[J]. 应用生态学报, 1999, 10(2): 193-196.
	Cai K Z, Luo S M. Effect of shading on growth, development and yield formation of rice[J]. Chinese Journal of Applied Ecology, 1999, 10(2): 193-196. (in Chinese)
[18]	Shimono H, Okada M, Kanda E, Arakawa I. Low temperature-induced sterility in rice: Evidence for the effects of temperature before panicle initiation[J]. Field Crops Research, 2007, 101(2): 221-231.
[19]	Yano M, Katayose Y, Ashikari M, Yamanouchi U, Monna L, Fuse T, Baba T, Yamamoto K, Umehara Y, Nagamura Y, Sasaki T. Hd1, a major photoperiod sensitivity quantitative trait locus in rice, is closely related to the Arabidopsis flowering time gene CONSTANS[J]. The Plant Cell, 2000, 12(12): 2473-2483.
[20]	Katsura K, Maeda S, Lubis I, Horie T, Cao W, Shiraiwa T. The high yield of irrigated rice in Yunnan, China[J]. Field Crops Research, 2008, 107(1): 1-11.
[21]	Yin X, Kropff M J, McLaren G, Visperas R M. A nonlinear model for crop development as a function of temperature[J]. Agricultural and Forest Meteorology, 1995, 77(1/2): 1-16.
[22]	Guo Y, Liu Y, Georgiou T, Lew M S. A review of semantic segmentation using deep neural networks[J]. International Journal of Multimedia Information Retrieval, 2018, 7(2): 87-93.
[23]	Geusebroek J M, van den Boomgaard R, Smeulders A W M, Geerts H. Color invariance[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(12): 1338-1350.
[24]	Kolkur S, Kalbande D, Shimpi P, Bapat C, Jatakia J. Human skin detection using RGB,HSV and YCbCr color models[EB/OL]. 2017: arXiv: 1708.02694. https://arxiv.org/abs/1708.02694
[25]	Vaswani A, Shazeer N, Parmar N. Attention is all your need[J]. Advances in Neural Information Processing Systems, 2017, 30.
[26]	Le P, Zuidema W. Quantifying the vanishing gradient and long distance dependency problem in recursive neural networks and recursive LSTMs[EB/OL]. 2016: arXiv: 1603.00423. https://arxiv.org/abs/1603.00423
[27]	Schuster M, Paliwal K K. Bidirectional recurrent neural networks[J]. IEEE Transactions on Signal Processing, 1997, 45(11): 2673-2681.
[28]	Yu Y, Si X, Hu C, Zhang J. A review of recurrent neural networks: LSTM cells and network architectures[J]. Neural Computation, 2019, 31(7): 1235-1270.
[29]	Ramachandram D, Taylor G W. Deep multimodal learning: A survey on recent advances and trends[J]. IEEE Signal Processing Magazine, 2017, 34(6): 96-108.
[30]	Loh W Y. Classification and regression trees[J]. WIREs Data Mining and Knowledge Discovery, 2011, 1(1): 14-23.
[31]	Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[32]	Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 7-12, 2015. Boston, MA, USA. IEEE, 2015: 1-9.
[33]	He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016. Las Vegas, NV, USA: IEEE, 2016: 770-778.
[34]	Liu Z, Mao H, Wu C Y, Feichtenhofer C, Darrell T, Xie S. A ConvNet for the 2020s[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 18-24, 2022. New Orleans, LA, USA: IEEE, 2022: 11966-11976.
[35]	Dosovitskiy A, Beyer L, Kolesnikov A. An image is worth 16x16 words: Transformers for image recognition at scale[J]. ICLR, 2020: arXiv preprint arXiv: 2010.11929.
[36]	Tan S, Lu H, Yu J, Lan M, Hu X, Zheng H, Peng Y, Wang Y, Li Z, Qi L, Ma X. In-field rice panicles detection and growth stages recognition based on RiceRes2Net[J]. Computers and Electronics in Agriculture, 2023, 206: 107704.
[37]	van Oort P A J, Zhang T, de Vries M E, Heinemann A B, Meinke H. Correlation between temperature and phenology prediction error in rice (Oryza sativa L.)[J]. Agricultural and Forest Meteorology, 2011, 151(12): 1545-1555.
[38]	Blockeel H, Devos L, Frénay B, Nanfack G, Nijssen S. Decision trees: From efficient prediction to responsible AI[J]. Frontiers in Artificial Intelligence, 2023, 6: 1124553.