Infinity Instruct

Summary

Infinity Instruct is a large-scale, high-quality instruction fine-tuning dataset (7M instructions) developed by BAAI, featuring data evolution strategies and model ability diagnosis for instruction generation. It includes a 7M Core version (1.4M instructions reaching 95.7% of full 7M performance) and has been used to fine-tune Llama3, Mistral, Qwen2, and Yi models, achieving favorable results on AlpacaEval 2.0 and MT-Bench without RLHF. Accepted to AAAI 2026.

Keywords

instruction-tuning sft large-scale chinese english baai aaai2026 instruction-evolution

Topics

NLP / General instruction following

Research notes

• Requires agreeing not to conduct harmful experiments. Associated with arXiv:2506.11116, arXiv:2507.06968, arXiv:2402.00530. Includes reward scores and instruction labeling types from Skywork-Reward-Llama-3.1-8B.