ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical Reasoning

Summary

ReasonMed is the largest open-source medical reasoning dataset to date, containing 370K high-quality question-answer examples with multi-step chain-of-thought (CoT) rationales and concise summaries. The examples were distilled from 1.75M initial reasoning paths generated by three competitive LLMs (Qwen-2.5-72B, DeepSeek-R1-Distill-Llama-70B, and HuatuoGPT-o1-70B) through a rigorous multi-agent verification and refinement pipeline with an Error Refiner. The full dataset contains 1.11M rows across three variants (ReasonMed, CoTMed, ResponseMed). Models trained on ReasonMed (ReasonMed-7B) surpass prior best sub-10B models by 4.17% and exceed LLaMA3.1-70B on PubMedQA by 4.60%.

Keywords

medical reasoning chain-of-thought multi-agent qa healthcare distillation

Topics

Medical / NLP / Reasoning

Research notes

• Published at EMNLP 2025 (arxiv: 2506.09513). Code at github.com/YuSun-Work/ReasonMed. Uses an easy-medium-difficult (EMD) curation pipeline. The finding that integrating detailed CoT with concise answer summaries yields the best fine-tuning results is a key contribution.