5,438,287 AI resources indexedWorking toward the world’s largest AI resource index.

Browse resources 13 sections
PDF

Evaluating the Potential of Leading Large Language Models in Reasoning Biology Questions

Recent advances in Large Language Models (LLMs) have presented new opportunities for integrating Artificial General Intelligence (AGI) into biological research and education. This study evaluated the capabilities of leading LLMs, including GPT-4, GPT-3.5, PaLM2, Claude2, and SenseNova, in answering conceptual biology questions. The models were tested on a 108-question multiple-choice exam covering biology topics in molecular biology, biological techniques, metabolic engineering, and synthetic biology. Among the models, GPT-4 achieved the highest average score of 90 and demonstrated the greatest consistency across trials with different prompts

By Gong, Xinyu, Holmes, Jason, Li, Yiwei, Liu, Zhengliang, Gan, Qi, Wu, Zihao, Zhang, Jianli, Zou, Yusong, Teng, Yuxi, Jiang, Tian, Zhu, Hongtu, Liu, Wei, Liu, Tianming, Yan, YajunResearch paper

Resource information

Authors
Gong, Xinyu, Holmes, Jason, Li, Yiwei, Liu, Zhengliang, Gan, Qi, Wu, Zihao, Zhang, Jianli, Zou, Yusong, Teng, Yuxi, Jiang, Tian, Zhu, Hongtu, Liu, Wei, Liu, Tianming, Yan, Yajun
Research area
Artificial intelligence
Research topics
Computation and Language (cs.CL), Artificial Intelligence (cs.AI), FOS: Computer and information sciences, FOS: Computer and information sciences
Publication date
2023
DOI
10.48550/arxiv.2311.07582
Upstream last modified
Nov 14, 2023
License
arXiv.org perpetual, non-exclusive license
Last record update
Nov 5, 2026

About this resource

Recent advances in Large Language Models (LLMs) have presented new opportunities for integrating Artificial General Intelligence (AGI) into biological research and education. This study evaluated the capabilities of leading LLMs, including GPT-4, GPT-3.5, PaLM2, Claude2, and SenseNova, in answering conceptual biology questions. The models were tested on a 108-question multiple-choice exam covering biology topics in molecular biology, biological techniques, metabolic engineering, and synthetic biology. Among the models, GPT-4 achieved the highest average score of 90 and demonstrated the greatest consistency across trials with different prompts. The results indicated GPT-4's proficiency in logical reasoning and its potential to aid biology research through capabilities like data analysis, hypothesis generation, and knowledge integration. However, further development and validation are still required before the promise of LLMs in accelerating biological discovery can be realized.