An AI that teaches itself gets sharper but stops exploring

The 30-second scan

Let an AI grade its own homework, and it stops trying new things.

1. 1.The authors show that on-policy self-distillation's strong pass@1 accuracy can come at a hidden cost: rollout diversity decreases and pass@k curves flatten so that generating more rollouts fails to improve accuracy.
2. 2.They trace this diversity loss to compounding biases in the design of self-distillation with sampled demonstrations, where the teacher scores rollouts through the model's own biases.
3. 3.The authors theoretically analyze the optimal self-distillation policy and show it tilts the base distribution by a pointwise conditional mutual information score between the student's rollout and the correct rollout used as context.