Here's another slightly longer TL;DR:

Humans solve hard problems by them down into parts and solving them part by part. We normally ask language models to solve algorithmic problems in one go (or if they revise their solutions, we expect them to revise everything). This has been known to be a problem for a while. It turns out, maybe unsurprisingly, that by asking language models to break problems down and then implementing subparts independently, we get way better results.

We do this by writing a programming language (basically, English with indentation plus a small amount of syntax for tests and references). We design an LLM-powered compiler around it to generate programs efficiently. We show it works on solving competitive coding problems, robotic task planning, and math theorem proving. We also show that it's decently robust - able to implement a bare-bones lisp compiler in a few dozen lines.