Evaluation of LIVED benchmark-based large language models in animal husbandry

YU Bing; WEN Jiwei; ZHANG Xiaopeng; DU Xiaoyong; LI Guoliang; LIU Chousheng; QIU Xiaotian; FANG Yaping

doi:10.13300/j.cnki.hnlkxb.2026.03.012

您当前的位置：

首页 >

文章列表页 >

Evaluation of LIVED benchmark-based large language models in animal husbandry

Agricultural artificial intelligence | 更新时间：2026-11-05

- Evaluation of LIVED benchmark-based large language models in animal husbandry
- Journal of Huazhong Agricultural University Vol. 45, Issue 3, Pages: 137-147(2026)
- 作者机构：
  
  1.作物遗传改良全国重点实验室/湖北洪山实验室/农业农村部智慧养殖技术重点实验室/ 农业智能技术教育部工程研究中心/湖北省农业大数据工程技术研究中心/ 华中农业大学信息学院，武汉 430070
  2.全国畜牧总站，北京 100125
- 作者简介：
- 基金信息：
- DOI：10.13300/j.cnki.hnlkxb.2026.03.012
  CLC： TP18
- Received：30 September 2025，
  
  Published：25 May 2026
- 稿件说明：
移动端阅览
禹冰，文纪威，张霄鹏，等.基于畜牧业LIVED基准的大语言模型评估［J］.华中农业大学学报，2026，45（3）：137‐147.

YU Bing,WEN Jiwei,ZHANG Xiaopeng,et al.Evaluation of LIVED benchmark-based large language models in animal husbandry[J].Journal of Huazhong Agricultural University,2026,45(03):137-147.
禹冰，文纪威，张霄鹏，等.基于畜牧业LIVED基准的大语言模型评估［J］.华中农业大学学报，2026，45（3）：137‐147. DOI： 10.13300/j.cnki.hnlkxb.2026.03.012.

YU Bing,WEN Jiwei,ZHANG Xiaopeng,et al.Evaluation of LIVED benchmark-based large language models in animal husbandry[J].Journal of Huazhong Agricultural University,2026,45(03):137-147. DOI： 10.13300/j.cnki.hnlkxb.2026.03.012.

摘要

为促进大语言模型更好地服务于畜牧业，系统评估了11个通用大语言模型和3个畜牧领域专用模型在专业化考试基准数据集中的表现。构建的LIVED（livestock knowledge evaluation dataset）数据集包含11 037道中文畜牧业知识评估数据集，涵盖选择题、判断题、填空题与问答题4种类型。评估结果显示，各大语言模型性能差异显著，综合性能排名前3的模型分别是DeepSeek-R1（准确率0.700）、Kimi K2（准确率0.681）和DeepSeek-V3.1（准确率0.675）。DeepSeek-R1的选择题准确率最高（0.868），GPT-5的判断题准确率最高（0.842），Kimi K2的填空题准确率最高（0.651），问答题Qwen3-235B-A22B的准确率最高（0.504）。畜牧业专用模型中，神农百晓大模型整体性能最佳（0.630），但仍落后DeepSeek-R1 7百分点。错题分析显示，当前大语言模型的错误主要富集在畜牧经济与管理、畜牧生产管理等方面。结果表明，大语言模型在专业知识准确性与推理能力方面仍存在明显不足，未来垂直大模型研发需优先解决专业知识与推理性能的短板。

Abstract

The performance of 11 general-purpose large language models （LLMs） and 3 specific LLMs in animal husbandry was systematically evaluated on the benchmark dataset of professional examination to promote the application of LLMs in animal husbandry.The constructed LIVED dataset included a Chinese animal husbandry knowledge assessment dataset with 11 037 questions covering multiple choice，judgment，completion，and open-ended questions.The results showed that there was significant difference in the performance of LLMs，with the top three models in terms of overall performance being DeepSeek-R1 （accuracy of 0.700），Kimi K2 （accuracy of 0.681），and DeepSeek-V3.1 （accuracy of 0.675）.DeepSeek- R1，GPT-5，Kimi K2 and Qwen3-235B-A22B had the highest accuracy in multiple-choice （0.868），judgment （0.842），completion （0.651），and open-ended questions （0.504）.Shennong Baixiao had the best overall performance （0.630） in the specialized livestock models，still lagging behind DeepSeek-R1 by 7.0 percentage points.The results of error analysis showed that the errors in the current LLMs were mainly concentrated in areas including the management of economy and production in animal husbandry.It is indicated that there are still significant shortcomings in the accuracy of professional knowledge and reasoning ability of LLMs.In the future，the development of vertical large models needs to prioritize addressing the shortcomings in professional knowledge and reasoning performance.

关键词

Keywords

references

OPENAI，ACHIAM J，ADLER S，et al.GPT-4 technical report［PP/OL］.arXiv，2023：2303.08774［2025-09-30］.https：//arxiv.org/abs/2303.08774.

MINAEE S，MIKOLOV T，NIKZAD N，et al.Large language models：a survey［PP/OL］.arXiv，2024：2402.06196［2025-09-30］.https：//arxiv.org/abs/2402. 06196.

WANG N，YANG H Y，WANG C D.FinGPT： instruction tuning benchmark for open-source large language models in Financial Datasets［PP/OL］.arXiv，2023：2310.04793［2025-09-30］.https：//doi.org/10.48550/arXiv. 2310.04793.

JIN Q，DHINGRA B，LIU Z P，et al.PubMedQA：a dataset for biomedical research question answering［C］//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing （EMNLP-IJCNLP）.Hong Kong.Stroudsburg：ACL，2019：2567-2577.

CHONG D D，GUO Z H，HUA Y N，et al.Benchmarking large language models on CMExam ：a comprehensive Chinese medical exam dataset［C］//Advances in Neural Information Processing Systems 36.December 10-16，2023.New Orleans：Neural Information Processing Systems Foundation，Inc.（NeurIPS），2023：52430-52452.

DAI Y F，FENG D Y，HUANG J M，et al.LAiW：a Chinese legal large language models benchmark［C］//Proceedings of the 31st International Conference on Computational Linguistics.Abu Dhabi：Association for Computational Linguistics，2025：10738-10766.

GUO X，XIA H T，LIU Z W，et al.FinEval：a Chinese financial domain knowledge evaluation benchmark for large language models［C］//Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics：Human Language Technologies （volume 1：Long Papers）.Albuquerque，Stroudsburg：ACL，2025：6258-6292.

HENDRYCKS D，BURNS C，BASART S，et al.Measuring massive multitask language understandingar［PP/OL］.arXiv， 2020 ：2009.03300［2025-09-30］.https：//doi.org/10.48550/arXiv.2009.03300.

MORI S，SUEN C Y，YAMAMOTO K.Historical review of OCR research and development［J］.Proceedings of the IEEE，1992，80（7）：1029-1058.

REIMERS N，GUREVYCH I.Sentence-BERT：sentence embeddings using Siamese BERT-networks［C］//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing （EMNLP-IJCNLP）.Hong Kong，China.Stroudsburg：ACL，2019：3980-3990.

SANDMANN S，HEGSELMANN S，FUJARSKI M，et al.Benchmark evaluation of DeepSeek large language models in clinical decision-making［J］.Nature medicine，2025，31（8）：2546-2549.

BROWN T B，MANN B，RYDER N，et al.Language models are few-shot learners［PP/OL］.arXiv， 2020 ：2005.14165［2025-09-30］.https：//doi.org/10.48550/arXiv.2005.14165.

HURST A，LERER A，GOUCHER A P，et al.GPT-4o system card［PP/OL］.arXiv，2024：2410.21276［2025-09-30］.https：//doi.org/10.48550/arXiv.2410.21276.

META LLAMA.meta-llama/llama3［EB/OL］.GitHub，（2025-03-13）［2025-09-30］. https：//github.com/meta-llama/llama3.

YANG A，LI A F，YANG B S，et al.Qwen3 technical report［PP/OL］.arXiv，2025：2505.09388［2025-09-30］.https：//arxiv.org/abs/2505.09388.

DEEPSEEK-AI，GUO D Y，YANG D J，et al.DeepSeek-R1：incentivizing reasoning capability in LLMs via reinforcement learning［PP/OL］.arXiv，2025：2501.12948［2025-09-30］.https：//arxi v.org/abs/2501.12948.

KAPLAN J，MCCANDLISH S，HENIGHAN T，et al.Scaling laws for neural language models［PP/OL］.arXiv， 2020 ：2001.08361［2025-09-30］.https：//arxiv.org/abs/2001.08361.

WU J，WANG Z H，QIN Y F.Performance of DeepSeek-R1 and ChatGPT-4o on the Chinese national medical licensing examination：a comparative study［J/OL］.Journal of medical systems，2025，49：74［2025-09-30］.https：//doi.org/10.1007/s10916-025-02213-z.

WANG P J，LI Z Z，YIN F，et al.Mv-math：evaluating multimodal math reasoning in multi-visual contexts［C］//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）.June 10-17，2025.Nashville：IEEE，2025：19541-19551.

HOGAN A，BLOMQVIST E，COCHEZ M，et al.Knowledge graphs［J］.ACM computing surveys，2022，54（4）：1-37.

MURPHY K，SCHÖLKOPF B，VAN MOFFAERT K，et al.Multi-objective reinforcement learning using sets of Pareto dominating policies［J］.The journal of machine learning research，2014：3483-3512.

戴高阳，孟小艳，张容祯.基于深度学习的畜牧业知识图谱构建［J］.计算机与数字工程，2024，52（6）：1746-1753.DAI G Y，MENG X Y，ZHANG R Z.Construction of knowledge graph of animal husbandry based on deep learning［J］.Computer & digital engineering，2024，52（6）：1746-1753 （in Chinese with English abstract） .

POTAMITIS I.ChatGPT in the context of precision agriculture data analytics［PP/OL］.arXiv，2023：2311.06390［2025-09-30］.https：//arxiv.org/abs/2311.06390.

吴华瑞，李静晨，杨雨森.基于大语言模型的个性化作物水肥管理智能决策方法［J］.智慧农业（中英文），2025，7（1）：11-19.WU H R，LI J C，YANG Y S.Intelligent decision-making method for personalized vegetable crop water and fertilizer management based on large language models［J］.Smart agriculture，2025，7（1）：11-19 （in Chinese with English abstract） .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Genome⁃wide identification and expression analysis of the CoSAD gene family in Camellia oleifera plants with three levels of ploidy

Design of an IoT-based monitoring system for small-scale rural biogas projects

Effects of nitrogen fertilization combined with straw returning to field on rhizosphere microorganisms in rice

Rhizosphere inoculation of biocontrol bacteria modulates pepper microbiota and enhances resistance to Phytophthora capsici

Metabolomic comparison of muscle quality in Chinese hooksnout carp （Opsariichthys bidens） between integrated photovoltaic-fishery and conventional pond aquaculture modes

Related Author

No data

Related Institution

No data

AI问答

⁰