PufferLib

Summary

PufferLib is an open-source (MIT) reinforcement learning library focused on fast, compatible simulation. It offers one-line wrappers making complex environments (NetHack, Neural MMO, Griddly) compatible with Gymnasium/PettingZoo-style libraries, drop-in vectorization, and Puffer Ocean — 12 C environments each simulating at over 1M steps/second on a single CPU core. A PPO demo trains Ocean envs at 300K–1.2M steps/second on a single RTX 4090.

Keywords

reinforcement learning · simulation · environments · open source · PPO

Topics

reinforcement learning, simulation, environments, open source, PPO

Research notes

• Key findings: PufferLib 2.0 paper: 'PufferLib 2.0: Reinforcement Learning at 1M steps/s' (Joseph Suarez); 30x faster than CleanRL on Atari (30K steps/s); Works with CleanRL and SB3; supports Atari, Procgen, NetHack, Neural MMO 3
• puffer.ai is the documentation site; the library is at github.com/PufferAI/PufferLib. Site fetch rate-limited; details from web search (RLJ 2025 paper summary, OpenReview, GitHub READMEs).