Lights-Off Software Factories Fail: Why Code Maintainability Still Requires Humans

Despite the hype, PR quality is down, incidents up, and codebases deteriorating. This isn't a tooling failure but a model training limitation. Benchmarks only reward test-passing, so models optimize for correctness, not maintainability.

The cost of bad architecture surfaces months later, making it impossible to propagate a reward signal back through reinforcement learning. No amount of post-hoc review agents or token-maxing can fix this, because the model never learns what good code design looks like. The result: codebases become harder to change over time.

Engineering leaders must accept that human oversight isn’t optional for brownfield systems. The sweet spot is upfront design: product reviews, architecture alignment, program design, and vertical slicing before generating code. This front-loaded alignment makes human review light and fast, while AI still handles the heavy lifting of implementation.

Giving up full automation fantasy enables sustainable velocity. The bottleneck isn't PR volume but review ease. Aligned teams find AI code a joy to inspect. Goal: own better code, not less.