The Optimal Choice of Hypothesis Is the Weakest, Not the Shortest

Summary

If A ⊂ B, generalisation is inferring from A a hypothesis sufficient to construct B. Shortest/MDL compression is neither necessary nor sufficient to maximise the probability of generalising. Under uniformly distributed tasks, no proxy matches weakness maximisation everywhere and beats it somewhere. In binary-arithmetic experiments, maximum weakness generalised at 1.1–5× the rate of MDL. Argues this helps explain why DeepMind's Apperception Engine generalises. AGI 2023 (LNCS 13921). Discord posted PDF v4 of arXiv 2301.12987.

Keywords

weakness · mdl · generalization · apperception-engine · agi · simplicity · kolmogorov

Topics

generalization, simplicity, AGI

Research notes

• Primary: arxiv abs (cs.AI; also cs.LG, math.LO). License CC BY-NC-ND 4.0 on HTML at check. DOI 10.1007/978-3-031-33469-6_5. Discord posted PDF v4. No official code on the abs query. Not a new hosted corpus, so no datasets_local row. Cataloged from the matching abs even though the leftover link was the PDF.