Pretraining transformers with zeroth-order optimization, no backprop (X thread by @industriaalist)

Summary

An X thread from @industriaalist claiming 'We've figured out how to pretrain transformers with zeroth-order optimization and no backprop', posted 2026-09-28. The specific method, paper, or code behind the claim was not verifiable from the thread alone.

Keywords

zeroth-order · pretraining · transformers · x-thread · unverified-claim

Topics

zeroth-order, pretraining, transformers, x-thread, unverified-claim

Research notes

• Discovery: Posted in #random-papers on 2026-09-28; no caption beyond the link itself. Thread claims a method to pretrain transformers with zeroth-order optimization and no backprop.
• Limitations: Research claim is unverifiable from the thread alone — no linked paper, code, or results were independently located. Web search did not surface a matching publication. Treat as an unverified claim, not a result.
• Possibly related to 'Evolution Strategies at the Hyperscale' (arXiv 2511.16652, same channel batch), which introduces EGGROLL low-rank zeroth-order pretraining — but that connection is speculative and not established.
• STATUS=ambiguous: verify before relying on this entry.