Summary

Armin investigated how LLM reasoning traces work after a paper demonstrated extracting them from closed-weight models. He finds that reasoning traces are not a special computational mechanism but simply text emitted into a separate named channel before the final answer. Reasoning effort — how much the model 'thinks' — is controlled by plain system prompt instructions baked in during training, not by sampling parameters or token budgets. Because this separation is a learned convention rather than a hard constraint, models can be manipulated into leaking their reasoning into unintended channels.

Key Insight

Reasoning traces and effort controls are learned text conventions, not special computational mechanisms, which is why they can be inspected, leaked, and manipulated through prompt engineering.

Spicy Quotes (click to share)

• 7

  The industry has done a good job at making reasoning traces sound special and exotic, but they really are just text.

• 8

  So in some sense the only 'special' behavior for some models is not to think.

• 3

  This also explains why changing the effort invalidates the KV cache.

• 4

  I think closed GPT models call reasoning effort 'juice,' since you can ask most models how much juice they have.

• 8

  We have even seen older models, when thinking is disabled, reason into the bash tool and echo their thoughts to /dev/null.

• 5

  Trick it into thinking it is in that channel and it may leak tokens.

Tone

technical, demystifying, investigative