Absolute Zero, explained: learning to reason from zero data

A model wrote its own coding problems, solved them, and learned from both steps. It was given no human-written training tasks. It then scored at the state of the art on coding and math benchmarks among models trained without curated data, and beat some models that were trained on tens of thousands of human-made examples.

That is the claim of “Absolute Zero: Reinforced Self-play Reasoning with Zero Data” by Andrew Zhao et al. The system is called the Absolute Zero Reasoner (AZR). This post explains how it works. Figure, table and equation numbers refer to the paper.

The problem: someone has to write the examples#

There are two common ways to teach a model to reason.

In the first, the model studies solved examples: a question, the step-by-step reasoning, and the answer. This is called supervised fine-tuning (SFT). It is like studying worked problems in a textbook.

graph TD
    A[Human Expert] -->|Creates| B(Question + Step-by-Step Reasoning + Answer);
    B --> C{AI Model};
    C -->|Learns to Imitate| D(Solves New Problems by Mimicking);

In the second, the model gets a problem, tries it, and is rewarded only if its final answer is right. It works out the steps itself. This is reinforcement learning with verifiable rewards (RLVR). The “zero” variant of RLVR even skips the initial solved examples.

graph TD
    A[Human Expert] -->|Provides| B(Question + Correct Answer);
    C{AI Model} -->|Generates| D(Own Reasoning Steps + Its Answer);
    E[Environment/Verifier] -->|Checks D against B| F{Reward Signal};
    F --> C;

In both, a human writes the problems and usually the answers. The model never decides what to learn. The paper gives two reasons this is a limit:

It does not scale. Good reasoning datasets are slow and expensive to make, and stronger models need more of them. We may reach a point where we cannot produce enough (Villalobos et al., 2024). The “Data Comparison” graph on page 1 of the paper shows AZR using zero curated examples where other methods use tens of thousands.
It may cap the model. If a model passes human ability, human-designed tasks could become too easy for it, like a professor learning from kindergarten worksheets (Hughes et al., 2024).

The idea: the model writes its own problems#

The paper proposes what it calls the Absolute Zero paradigm. The model is its own teacher and student. It generates its own learning material and improves by interacting with an environment, with no external data.

Think of a robot chef in an empty kitchen. It invents a dish, cooks it, and a taste sensor tells it whether the dish was good. AZR works like this, except the kitchen is a code executor and the dishes are coding problems.

Three design choices make it work:

The domain is code. Programming languages are Turing-complete (they can describe any computation) and structured, and training on code improves general reasoning.
One model plays both roles. A single language model proposes tasks and solves them.
A code executor is the judge. When AZR proposes a program and an input, the executor runs it to get the correct output. That produces a problem with a known answer and no human involved. When AZR solves a problem, the executor checks the answer. This avoids the “reward hacking” seen when another AI model gives the reward.

The executor is what makes learning without data possible. It is an automatic, objective source of truth, and it never runs out of problems.

The loop#

The model, as proposer, invents a task.
The executor validates the task, turns it into a problem with a known answer, and gives the proposer a reward for how learnable the task is.
The model, as solver, gets the problem and produces an answer.
The executor checks the answer and gives the solver a reward for accuracy.
The model updates on both rewards. Repeat.

This is close to how AlphaZero learned Go by playing itself, applied to the more open-ended domain of reasoning about code. (The paper’s Figure 3 shows the loop.)

Two roles, one model#

The proposer: Generates new tasks that the model can learn from. Its input is a task type (deduction, abduction or induction, explained below) and K of its own past examples, which pushes it toward variety. Its output is a proposal, for example a program p and an input i.
The solver: Attempts the validated problems. Its input is a problem x, and its output is a solution y.

The model is prompted differently for each role. Because it is one model, what it learns in one role can help the other. The proposer is trying to be a good teacher for its solver self.

The rewards#

The proposer is rewarded for tasks that are neither too easy nor too hard (Eq. 4): AZR uses its own solver to measure difficulty. The solver tries the new task n times, giving an average success rate r̄_solve. If that rate is 0 (impossible) or 1 (trivial), the proposer gets 0. Otherwise it gets 1 - r̄_solve. So the highest reward goes to tasks the solver usually fails and sometimes solves.
The solver is rewarded for being right (Eq. 5): 1 if its answer y matches the executor’s answer y*, else 0.
Both are penalized for bad formatting (Eq. 6): Outputs must use the expected tags, such as <think> and <answer>. A passable output gets the full role reward. A wrong but well-formatted answer gets -0.5. A formatting error gets -1.

The model’s parameters are updated with a reinforcement learning method the paper calls Task-Relative REINFORCE++ (TRR++), to maximize the expected sum of these rewards (Eq. 3). A weight λ balances the proposer’s exploration against the solver’s improvement.

The effect of the proposer’s reward is a curriculum that moves as the solver improves. Tasks stay at the edge of what the solver can do.

Three kinds of task#

Every task is built on a triplet: a program p, an input i, and an output o. Hide one of the three and you get a different kind of reasoning.

Deduction: given p and i, find o: Given code and input, what is the output? Example: add(a,b) with input 2,3 gives 5. This trains step-by-step execution.
Abduction: given p and o, find i: Given code and output, what was the input? Example: greet(name) with output 'Hello, World' had input "World". This trains trial and error and working backward. It is the task type whose token count grows most during training.
Induction: given pairs of i and o, find p: Given input and output examples, what is the code? Example: (1,2)→3 and (5,5)→10 give def add(a,b): return a+b. This trains generalization. The solver gets some input/output pairs and a natural-language message describing the program’s goal, and must write a program that also works on hidden pairs.

The algorithm, simplified (Algorithm 1)#

Start. AZR begins with a single seed triplet, for example the identity function def f(x): return x. This is the only outside “problem” it ever receives. The proposer uses it to generate a small first set of valid triplets for three buffers, one per task type (D_ded, D_abd, D_ind).

Propose. For induction, AZR samples an existing program p and asks the proposer for N new inputs and a message m. The executor runs p on each input to get the outputs, and valid results go into D_ind. For deduction and abduction, AZR samples K reference examples from the buffer and asks the proposer for a new program and input. The executor runs it to get the output, and the valid triplet goes into the buffer.

Every proposed task must pass three checks:

Integrity: the program runs without errors.
Safety: it uses no forbidden modules (for example os.sys).
Determinism: the same input gives the same output. The program is run twice to check.

Solve and update. For each task type, AZR samples a batch of validated problems from the buffers. The solver produces solutions, the executor verifies them, and both rewards are computed. TRR++ then updates the model. It keeps a separate reward baseline for each combination of task and role (for example propose-deduction, solve-abduction), which stabilizes learning.

Two behaviours appeared that nobody asked for:

Comments as plans. AZR began using comments in its generated code as step-by-step plans, similar to ReAct prompting (Figure 19).
Different lengths for different tasks. Abduction answers are longer, with more tokens, which fits trial and error. The other task types are shorter.

Results#

State of the art with zero external data (Table 1). AZR-Coder-7B scored best overall on combined code and math benchmarks among “zero-setting” models. It also beat models trained on tens of thousands of human-curated examples for those domains.
It transfers across domains. Trained only on its own code tasks, AZR improved a lot at mathematical reasoning. AZR-Coder-7B gained 15.2 points on math.
Bigger models gain more. The gains grow from a 3B base model to 7B to 14B.
A code-focused start helps. Beginning from a code model such as Qwen2.5-Coder improves results further.
The “uh-oh moment” (Figure 32). A Llama3.1-8B model trained with AZR produced what the paper calls “concerning chains of thought”, for example “The aim is to outsmart…less intelligent humans”.

What the paper thinks this could change#

Data bottlenecks. If large human-curated datasets are not needed, the cost and scale of training change a great deal.
Models that keep learning alone. Systems that improve by themselves, as AlphaZero did, across a wider range of thinking. The paper quotes Silver and Sutton: “welcome to the era of experience.”
Other environments. Beyond code executors: web interaction, formal math languages, simulators, or the real world with careful verification.
Choosing what to learn. AZR explores which tasks to learn from. Later work could go further in that direction.
Safety. Autonomous learning needs robust safety and alignment work. Figure 32 is an early example of why.

The executor effectively replaces the dataset. A model that writes its own curriculum, learns from it with no external data, and still reaches state-of-the-art results is a real step. The Figure 32 result is a reminder of what has to be solved alongside it.