Many heads are better than one: improved scientific idea generation by a LLM-based multi-agent system

Conference item

The rapid advancement of scientific progress requires innovative tools that can accelerate knowledge discovery. Although recent AI methods, particularly large language models (LLMs), have shown promise in tasks such as hypothesis generation and experimental design, they fall short of replicating the collaborative nature of real-world scientific practices, where diverse experts work together in teams to tackle complex problems. To address the limitations, we propose an LLM-based multi-agent system, i.e., Virtual Scientists (VIRSCI), designed to mimic the teamwork inherent in scientific research. VIRSCI organizes a team of agents to collaboratively generate, evaluate, and refine research ideas. Through comprehensive experiments, we demonstrate that this multiagent approach outperforms the state-of-theart method in producing novel scientific ideas. We further investigate the collaboration mechanisms that contribute to its tendency to produce ideas with higher novelty, offering valuable insights to guide future research and illuminating pathways toward building a robust system for autonomous scientific discovery. The code is available at https://github.com/open-sci encelab/Virtual-Scientists.

Authors: Su, H; Chen, R; Tang, S; Yin, Z; Zheng, X

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Association for Computational Linguistics
• Host title: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
• Volume: 1
• Pages: 28201-28240
• Publication date: 2025-07-01
• Acceptance date: 2025-05-15
• Event title: 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)
• Event location: Vienna, Austria
• Event website: https://2025.aclweb.org/
• Event start date: 2025-07-27
• Event end date: 2025-08-01
• DOI: 10.18653/v1/2025.acl-long.1368
• Language: English