EsoLang-Bench
- Type
- dataset
- Venue
- Lossfunk
- Year
- 2026
- Source
- huggingface
- Access
- free
- Language
- en
- Added
- 2026-07-17T20:01:04.310335+00:00
- Verified
- 2026-07-17T20:01:04.310335+00:00
Summary
EsoLang-Bench is a contamination-resistant benchmark for evaluating large language models on out-of-distribution programming languages. It contains **80 algorithmic programming problems** organised into four difficulty tiers, each shipped with **6 hidden input-output test cases** for byte-exact scoring. Problems are language-agnostic and intended to be solved in five esoteric target languages whose pre-training presence is many orders of magnitude smaller than mainstream languages.
Keywords
hf-dataset language-modeling · -code · -reasoning json text datasets pandas polars mlcroissant code code-generation esoteric-programming benchmarking evaluation out-of-distribution contamination-resistance reasoning brainfuck befunge whitespace unlambda shakespeare
Topics
Language Modeling, Code, Reasoning
Research notes
- downloads=124; likes=13