Recursive Multi-Agent Systems (RecursiveMAS)

Summary

Extends the recursive/looped-model scaling axis from a single model to multi-agent systems: the whole team is cast as a unified latent-space recursive computation. Each agent acts as a layer of a recursive LM, connected through a lightweight RecursiveLink module (inner link for in-agent latent thought generation, outer link for cross-agent latent state transfer); only the final round decodes text. Trained with an inner-outer loop algorithm that jointly co-optimizes all RecursiveLinks (~13M trainable params, 0.31% of the system) with stable gradients, unlike text-based recursive SFT. Evaluated across 9 benchmarks (math, science, medicine, search, code) under 4 collaboration patterns: +8.3% average accuracy over the strongest baselines, 1.2x-2.4x inference speedup, and 34.6%-75.6% token reduction.

Keywords

multi-agent · latent reasoning · looped transformers · recursive models · NeurIPS 2026

Topics

multi-agent systems, latent reasoning, recursive models

Research notes

• Discovery: shared by Shizhe Diao (@shizhediao, Thinking Machines; LMFlow author) quoting Jiaru "Rubin" Zou (@Jiaru_Zou), announcing NeurIPS 2026 acceptance: https://x.com/shizhediao/status/2104984897752601056
• Paper: https://openreview.net/forum?id=ML0KFwGcPG
• Interactive playground/demo on project page by Jindong Jiang.
• Method: RecursiveLink two-layer residual modules; inner-outer loop training; theoretical runtime/gradient-stability analysis.
• Accepted at NeurIPS 2026.