Metis: Memory Foundation Model

Summary

Defines memory foundation models: a persistent parametric memory state plus native remember/forget/update procedures executed in forward computation rather than external RAG. Metis blocks add a local dense memory matrix and a hyper-memory updater (Gated Delta Network) fused via memory attention; online maintenance is a gradient-free forward pass with frozen weights. Mid-trained on 357,137 primary + 609,443 auxiliary synthetic multi-step trajectories from 27 public benchmarks (~406M primary tokens). Under no-context eval, Metis-27B leads Temp-LoRA and δ-Mem on MemOps, LoCoMo (Gold), and NextMem, still well below full-context Qwen3.5. Releases Metis-4B/9B/27B. Long-horizon compression loss and latent confusion remain.

Keywords

metis · memory-foundation-model · agent-memory · qwen3.5 · locomo · memops · memtensor · gated-delta-network

Topics

LLM agents, native memory, foundation models

Research notes

• Primary: arxiv abs (CC BY-NC-SA 4.0, cs.CL/LG). Code https://github.com/MemTensor/Metis (120 stars at check). Models Apache-2.0: IAAR-Shanghai/Metis-4B (1.9k dl, 4 likes), Metis-9B, Metis-27B; collection https://huggingface.co/collections/IAAR-Shanghai/metis. Correspondence lizy@memtensor.cn, xu.chen@ruc.edu.cn. Affiliations MemTensor, RUC, NUS, SJTU, Tongji. Discord posted AlphaXiv 2607.26760v1. Training data synthesized from public benchmarks; no standalone public training-dataset repo confirmed, so no datasets_local row.