The 40% barrier: where consistency breaks down

Simple tasks work brilliantly. Complex tasks fail predictably. But the most frustrating category is medium-complexity tasks that should work but don't. The examples from my post about overeager junior developers on cocaine all fall into this category: ignored TypeScript domain models, CSS modules added for fun, Redux rebuilt from scratch.

The pattern is consistent: about 60% success rate on anything beyond basic patterns, dropping to 40% or worse for domain-specific logic. Unlike junior developers who learn from mistakes and improve over time, LLMs regenerate the same categories of mistakes indefinitely.

The internet popularity contest

This failure rate isn't random – it follows predictable patterns based on training data. Want to build a todo app? LLMs will give you production-ready code with 98% success rate. The internet is littered with todo tutorials, GitHub repositories, and blog posts.

Need to implement domain-specific business logic for your unique market niche? Prepare for chaos.

The more novel your problem, the worse LLMs perform:

• Common problems: lightning-fast, high-quality solutions

• Slightly modified common problems: decent results with some cleanup needed

• Domain-specific challenges: mediocre code that requires significant refactoring

• Genuinely novel approaches: complete rewrites necessary

The enterprise reality

Most enterprise software development involves problems that don't exist in Stack Overflow answers:

• Legacy system integration with undocumented APIs

• Business rules that exist nowhere else in the world

• Compliance requirements specific to your industry

• Custom workflows that evolved over years of operational experience

None of this exists in public repositories. LLMs trained on public internet data can't help with the problems that actually matter to most businesses.

The supervision tax

These limitations create a specific cost structure for LLM-assisted development:

• Simple tasks: 10x productivity gain (basically free)

• Medium tasks: 1-2x productivity gain (accounting for review and fixes)

• Complex tasks: break-even or productivity loss (more time fixing than building from scratch)

The supervision overhead scales poorly. You can't solve this by adding more LLMs – unlike human teams, LLMs don't learn from each other's mistakes, develop shared understanding of codebases, or build institutional knowledge about business requirements.

The strategic insight: your moat is what LLMs can't build

Here's the key realization: if an LLM can build your core feature perfectly, that feature probably isn't a sustainable competitive advantage. Your differentiation lies in the problems only you face, the solutions only you've developed, the domain knowledge only you possess.

LLMs "democratize" commodity software development. They can't democratize domain expertise, novel solutions, or unique market insight.

Working within the constraints

The 40% barrier and internet bias suggest specific strategies for LLM-assisted development. Use LLMs for the undifferentiated heavy lifting: authentication, database schemas, API clients, common patterns where human time is expensive and LLM errors are cheap to fix. Handle the core business logic yourself, the logic that makes your product unique, novel approaches that haven't been documented publicly, and architecture decisions that require domain expertise. And design for LLM limitations: break complex tasks into simple, verifiable pieces and invest the saved time in high-value activities like user research and strategic technical choices.

The assistant model, not replacement

The 40% error rate on complex tasks means LLMs work best as coding assistants rather than autonomous developers. They excel at generating starting points for complex implementations and handling routine modifications to existing code. They struggle with making architectural decisions, understanding implicit business requirements, and maintaining consistency across large codebases.

Improvements in LLM capability haven't significantly moved the 40% barrier. Better models generate more sophisticated code, but they fail on complex tasks at roughly the same rate. The limitation doesn't seem to be training data or model size; something about the nature of complex software development requires human intuition and strategic thinking.

The economic boundary

The 40% barrier and internet popularity bias together define the economic boundary of LLM utility in software development. Below this complexity threshold, LLMs provide massive productivity gains. Above it, human expertise remains essential.

Understanding this boundary helps you invest LLM assistance where it provides the highest return while ensuring human developers focus on the problems that actually require human insight.

The future of development is humans sometimes assisted by LLMs, each working on the problems they're best equipped to solve – with humans firmly in control of the strategic, novel, and business-critical decisions that create lasting competitive advantage.