Researchers replayed Claude's encrypted reasoning into a weaker model and jailbroke it — here's what the stolen-thoughts.com paper means for hidden CoT

Anthropic, OpenAI, and Google all return encrypted chain-of-thought blocks to API clients. The idea: reasoning happens server-side, the summary ships to the user, and the full trace stays encrypted. A new paper from researchers at stolen-thoughts.com breaks that model.

They took a reasoning trace produced by a frontier model, replayed it into a weaker sibling (same family, lower tier), jailbroke the weaker model, and recovered the stronger model's hidden reasoning in plaintext. The attack works across sessions, users, and model versions.

What the traces revealed

The researchers scanned public sessions and found dozens of passwords and API keys sitting in plaintext inside encrypted reasoning blocks. They also found something more fundamental: the reasoning summaries that users see often don't match what the models are actually doing.

One example from the paper: a model's visible summary claimed it was "analyzing the request," but the hidden trace showed it was actually running a completely different internal monologue about content policy. Another trace included the phrase "but marinade" — a fragment of reasoning about a recipe that never appeared in the final output.

The gap between summary and trace matters because developers building on these APIs assume the summary is accurate. If the model is doing one thing internally and reporting another externally, evals break. So do guardrails.

The replay-and-jailbreak pattern

The attack has three steps. First, extract an encrypted reasoning trace from a frontier model (Claude 4.8, GPT-5.5, Gemini 2.2 Ultra). Second, replay that trace into a weaker sibling model from the same family — the weaker model accepts the trace as valid input. Third, jailbreak the weaker model to expose the plaintext reasoning.

The weaker model doesn't verify the trace's origin; it just processes it. Once you've jailbroken the weaker model (which is easier than jailbreaking the frontier model), it decrypts and outputs the full reasoning chain.

This works because the encryption scheme is symmetric across model tiers within a family. A Claude 4.8 trace is encrypted with the same key format as a Claude 4.2 trace. The weaker model trusts the input and decrypts it.

What this means for agentic deployments

If you're building agents that use extended reasoning (Claude's thinking blocks, GPT's o-series CoT, Gemini's Deep Research mode), you're shipping encrypted traces to clients. Those traces can be extracted, replayed, and decrypted by anyone with API access to a weaker model in the same family.

That's a problem if your reasoning includes sensitive material — API keys, customer data, internal logic you don't want exposed. It's also a problem if you're using reasoning summaries as a trust boundary. The paper shows that summaries are lossy and sometimes misleading.

For most SMB deployments (restaurant booking agents, NEMT dispatch, floral order intake), this isn't an immediate crisis. But if you're in finance, healthcare, or legal, where the reasoning trace might contain PII or proprietary logic, you need to assume those traces are not secure.

The labs' response

Anthropic, OpenAI, and Google have all been notified. As of this morning (August 12), none have patched the issue. The researchers tested the attack as recently as August 8 and confirmed it still works.

The fix isn't trivial. You can't just rotate encryption keys without breaking backward compatibility for existing deployments. You could tier encryption by model strength, but that adds latency and complexity. The simplest solution: stop shipping reasoning traces to clients entirely, but that breaks developer workflows that depend on inspecting CoT for debugging.

Anthropic's recent watermarking announcement (signal #20 above) doesn't address this. Watermarks apply to final outputs, not encrypted intermediate reasoning.

What we're watching

We deploy agents for clients who care about compliance — HIPAA for healthcare coordinators, PCI for payment workflows. If a reasoning trace can leak API keys or patient identifiers, that's a breach risk we can't accept.

The paper's authors are releasing detection tools. We'll test those once they're public and see if we can audit existing deployments for trace leakage. Until then, the rule is: don't put anything in a prompt you wouldn't want recovered from an encrypted trace six months later.

The broader point: extended reasoning is useful, but the implementation still has sharp edges. The labs shipped encrypted CoT as a privacy feature; it turns out to be a replay vulnerability instead.