Summary

Armin uses a minor Twitter controversy — Pangram flagging David Sacks' tweet as AI — as a starting point to experimentally probe what 'AI-generated text' actually means. He prompts an LLM to write a Sacks-style tweet, which Pangram rates 100% AI, then manually rewrites every sentence from scratch without LLM help. Pangram still rates the rewritten text 100% AI. His conclusion: when an LLM establishes a text's structure, that structural fingerprint persists through human editing and will always trigger AI detectors. The post ends with genuine ambivalence about whether this is a fair judgment, and candid self-reflection about his own growing dependence on LLMs as writing aids.

Key Insight

Structure is the AI fingerprint — if an LLM scaffolds the skeleton of a text, no amount of human sentence-level rewriting escapes detection, because Pangram reads the shape of reasoning, not just the words.

Spicy Quotes (click to share)

• 5

  if you rely on an LLM to give your text structure, it will score badly on Pangram even if you do plenty of edits over it

• 6

  it's quite unlikely you're going to get a post that starts out as slop into a structure that will make it appear that it's not

• 3

  I did notice that I became both more reliant on those tools and that they have become much more aggressive editors and it gave me pause

• 5

  plenty of people will find a '100% AI' rating misleading when in fact the author has done plenty of editing. But maybe it's fair to have this to show up as entirely AI?

• 4

  I came to quite appreciate the existence of Pangram because at the very least it has made me quite aware of some of the effects that using LLMs for writing blog posts has

• 7

  I first came up with a prompt. And when I say I came up with that prompt I in fact used an LLM to propose to me from that tweet what I might want to say for the structure.

Tone

reflective, experimental, candid