Agentic Discovery of Neural Architectures: AIRA-Compose and AIRA-Design
- Type
- other
- Venue
- arXiv / FAIR at Meta
Summary
Dual AIRS-Bench frameworks. AIRA-Compose recasts Composer NAS: agents arrange 16-layer strings over MLP/attention(/Mamba), then stretch/stack to 350M–3B. 14 AIRAformer/AIRAhybrid designs; at 1B / 37.5B tokens AIRAformer-D beats Llama 3.2 by 2.4 pp 0-shot avg and AIRAhybrid-D beats approx. Nemotron-2 by 3.8 pp; AIRAformer-C isoFLOP slope 54% steeper than Llama 3.2. AIRA-Design: agents write LRA attention (within 2.3/2.6 pp of human SOTA on retrieval/text) and Autoresearch train.py (Greedy Opus 4.5 +lit BPB 0.968, beating published min). No official code on abs.
Keywords
aira · nas · hybrid-llm · mamba · composer · lra · autoresearch · fair-meta · rsi · agentic-research
Topics
neural architecture search, LLM agents, hybrid models
Research notes
- Primary: arxiv abs (cs.AI). CC BY-NC-SA 4.0 on HTML. FAIR at Meta; Pepe/Lin joint first, C.-J. Wu/Bachrach joint last. Correspondence despoinam / yorambac@meta.com. No official code on abs. HF paper page 17 upvotes; no linked models/datasets. Discord posted PDF. Uses public MAD/BabiStories/DCLM/LRA/ClimbMix; agent-found architectures are described, not a standalone public corpus, so no datasets_local row. License field left blank per catalog convention.