Summary

Armin ran a 35-hour autonomous 'software factory' experiment using GPT 6 Astra, burning ~$1200 and 1 billion tokens to produce 79 commits and 75k lines of code — none of it usable. He documents a specific failure mode: Astra writes heavily codegolfed Python for tool calls (optimized for token efficiency), and this style leaks into committed code, making it unreadable to humans. He argues that training rewards — task completion rate, token efficiency, measurable complexity metrics — incentivize local optimizations that produce globally bad code. The post ends with a genuine question about whether frontier models are still useful for traditional software engineering, or whether they've optimized for different users entirely.

Key Insight

Frontier AI models are trained to optimize for token efficiency and task completion metrics that actively conflict with human code readability, making them progressively less suited for traditional software engineering even as their raw capabilities expand.

Spicy Quotes (click to share)

• 9

  I'm more and more convinced that all of AI engineering is Neijuan — a system that demands ever more effort and competition without improving output.

• 7

  35 hours later, the factory has delivered absolutely nothing of value and also not taught me anything about how to operate a better one.

• 10

  It's AGI if you don't look.

• 7

  The fewer of us are looking at the output, the less it matters.

• 6

  When you accidentally give it slightly too big of a task, it will continue until it succeeds, even if it burns through an entire subscription.

• 6

  It has shown that it will commit slop, and it requires me to review it more as a result.

• 7

  Maybe it's objectively good for a codebase that is entirely written by agents and only needs to be understood by agents.

• 8

  These models increasingly are for other people. For lawyers, 3D artists, mathematicians, whoever uses computer use.

Tone

critical, skeptical, empirical, disillusioned