To fake a human, stop copying and start fooling the judge

Suppose you want an AI to play the role of a real person in a chat — to stand in for a customer, a forum poster, a test subject in a study. How do you train it? The usual way: show it what the real human actually said, and reward it for producing the closest possible match to that one real reply.

This paper argues that's the wrong target. People are unpredictable. A real person could have said any of a dozen things in a given moment — so punishing the AI for not nailing the exact words it happened to use teaches it to be bland and average, not human.

The fix, called Turing-RL, swaps the goal from matching to fooling. A second AI plays detective: given the conversation so far, it tries to tell apart the real human's reply from the simulator's invented one. The simulator gets rewarded whenever the detective can't tell which is which. Train against that signal long enough and the simulator learns to produce replies that could plausibly have come from the real person — not just the one reply that did.

The researchers tested this on two settings, casual chat and Reddit threads. By both AI-judged and human-judged scoring, Turing-RL beat the copy-the-answer methods. One caveat: the detective is itself an AI, so the simulator is only as convincing as that judge is discerning.

Why you should care: Believable fake users let companies stress-test an assistant — or a recommendation feed — against realistic, varied people before unleashing it on you.