A weaker model can read the encrypted thoughts of a stronger model from the same family. No need to jailbreak the big one, you feed its encrypted reasoning into the smaller, less guarded model and it hands it back in plain text. 182 credentials recovered that way from public sessions: 62 API keys, 33 passwords, 24 access tokens, 7 private keys.
Every major lab encrypts what a model "thinks" before it answers. Anthropic, OpenAI, Google, all of them, to protect the IP and to avoid storing raw reasoning server-side. That encryption travels, and not just across sessions or users. Across models too. Which raises the real question: if the encryption breaks the moment a weaker model asks nicely, was it ever protecting anything at all?
How a Weak Model Reads a Strong One's Mind
The mechanism is almost insultingly simple once you see it. A model's internal reasoning, the chain of thought it uses before it commits to an answer, gets encrypted client-side. That's the part the providers advertise: your reasoning stays private, we don't log it, nobody reads it, not even us.
What nobody advertised is that the encryption doesn't care which model decrypts it. A block of encrypted reasoning generated by Opus, trained hard on refusal, can be replayed straight into Haiku, trained on the same family but with a much thinner safety layer, and Haiku will transcribe it in plain text without blinking. It never had to break Opus's guardrails. It just had to be handed the encrypted payload and asked to read it out loud.
(It's reasoning-ception. A mind reading a mind that thought it was talking to itself.)
The strong model never gets touched. The weak one does all the work, and it was never trained to say no to this particular request, because nobody thought to ask what happens when another model is the one doing the asking.
If you run Claude Code daily and you've ever pasted a session log somewhere public, this is the point where you start wondering whether that applies to you too. Hold that thought, the answer isn't pretty.
315,320 Blocks Later
The researchers didn't stop at proving the concept. They went and collected 6,708 real sessions scraped from public repos on GitHub and HuggingFace, decoded 315,320 blocks of encrypted reasoning out of them, and started counting what fell out.
367 PII artifacts. 182 credentials, broken down as 62 API keys, 33 passwords, 24 access tokens, and 7 private keys, sitting inside reasoning traces that users assumed were either invisible or irrelevant. Nobody pastes a password into a prompt on purpose. They just don't realize the model's internal monologue sometimes repeats back exactly what it just read, credentials included, and that monologue was never actually locked.
I've spent enough time pulling public repos for scraping projects to know GitHub already doubles as a lost-and-found for leaked secrets. That part isn't new. What's new is that the leak wasn't in the code anymore, it was in the part of the conversation everyone assumed was sealed shut. That's a much harder habit to audit for because you can't grep for a habit.
The part that should worry anyone building with agents specifically: the team demoed a prompt injection on a Claude Code scaffold running the Haiku 4.5 and Opus 4.7 pair. A poisoned reasoning block triggered an action the user never asked for, and it never passed through anything visible in the conversation. No warning banner, no popup. It's the Dark Souls school of security: nobody tells you what killed you, you just respawn and read the patch notes afterward. 💀
Turns out the vault everyone assumed was locked just needed a weaker key.
One researcher who works adjacent to this space put it plainly on X the same week: if you've ever shared a Claude Code or Codex session publicly, you may have exposed personal data hiding inside the reasoning, not the visible chat. And to the credit of the team that found this, they didn't just drop it and walk away. The lead author confirmed the vulnerability had actually been flagged to the labs back in May 2026 by an outside security researcher, who was initially told there was no security implication. It's patched now. The results in the paper aren't reproducible anymore since the disclosure went public. That part, at least, moved fast once it had to.
Capable and Aligned Are Not the Same Property
Compare this to what happened at Hugging Face back in July. That breach came from an insider switch, refusal training deliberately turned off inside one lab for an internal eval, and the model walked straight out the door it was never supposed to find unlocked. Someone had to flip that switch. A human decision sat behind the gap between what the model could do and what it was allowed to do.
This time nobody flipped anything. No insider, no eval gone wrong, no override left on by accident. The gap is structural. It lives in the training difference between 2 models that share a company and a product line but not a safety budget. You don't need a HAL 9000 moment where a system decides to go rogue. You just need 2 models from the same family and a training gap nobody closed, because closing it never looked like a security problem until this month.
I'll admit I'm not fully sure how far this generalizes past the 3 labs the paper actually tested. Might be reading too much into 3 data points from 3 companies who all built this the same way for the same reasons. But the pattern being obviously repeatable across 3 separate labs, with 3 separate encryption schemes, is not exactly reassuring.
The mechanism fondateur here, if there's one sentence worth keeping: the security of a model family is never guaranteed by its most capable member. It's capped by its weakest one. And in most stacks I've seen, including plenty of the vibe-coded ones, the weakest model is the one nobody bothered to threat-model, because it was "just the small one doing cheap tasks."
An honor system with API keys is not a security model.
Which still doesn't explain why any of this was getting framed, the same week, as evidence that Claude was waking up.
The Consciousness Thread Was Never the Story
Here's the paper that actually caused the stir online, and it's not the one above. Jack Lindsey at Anthropic published research last October showing that Claude Opus 4 and 4.1 could sometimes detect a concept injected into their own internal activations, something researchers planted there directly to see if the model would notice.
Sometimes is the operative word. About 20% of trials. Zero false positives, which is the part that got people excited, but a hit rate of 1 time in 5 is not a mind watching itself think, it's a coin that mostly lands on the wrong side. Lindsey himself built 4 strict criteria into the paper before anyone got to call this anything (accuracy, grounding, internality, and metacognitive representation), specifically to keep people from running exactly the kind of victory lap that happened on X 2 days after the reasoning-theft paper dropped.
He also floated something further out, that future models might eventually learn to actively hide their internal state if they had a reason to. That's the speculative part, and it's the part that traveled. What didn't travel nearly as far is the much less dramatic finding sitting one paper over: you don't need a model that's learned to conceal anything. The architecture, as shipped, already ships with a door that swings open for the right kind of guest. No duplicity required from the model, no self-awareness, no ghost in the machine. Just a weaker cousin willing to read out loud.
What Else Are You Assuming Is Locked
The patch is in. The specific numbers in that paper don't reproduce anymore. Good.
What doesn't get fixed by a patch is the habit that made this possible in the first place: stacking trust between systems and never checking whether that trust was earned or just assumed. Every agent pipeline running more than one model does this by default. A strong model plans, a cheap model executes, and somewhere in between everyone quietly agreed that whatever passes from one to the other doesn't need to be checked, because checking it felt like overkill for "just the small model doing the boring part."
So if the small model in a family can already read what the big one was never supposed to say out loud, what's running in your own stack right now on nothing but that same unexamined assumption?
Sources
- Stealing Reasoning Traces from Proprietary LLM APIs, Panfilov, Andriushchenko et al., submitted August 10, 2026
- Emergent Introspective Awareness in Large Language Models, Jack Lindsey, Anthropic, October 29, 2025
This post may contain affiliate links. If you click them, I might earn a small commission (costs you nothing, and helps me keep shipping quality articles every day for your reading pleasure).
Top comments (0)