Gemini 4 Argan looks great on benchmarks and fails where code matters

The release gap. Google shipped Gemini 4 Argan after an eight-month gap in Pro-line releases, priced at $2/$10 per million tokens, doubling to $4/$20 on December 1. The reviewer builds on the model in his own products and is paid by Google as a YouTuber, so the negative results are hard to dismiss. The first signal: the model looks cheap on paper but costs more once cache-hit rates around 70-80% and token-hungry output are counted.

The benchmark gap. On Artificial Analysis, Argan ranks near the top on intelligence and shows the lowest hallucination rate among high-scoring models, with 15% on their omniscience benchmark. The catch: the overall score drops to fourth once refusals and partial answers count. The reviewer's codebase audits tell a sharper story. In the TS Rust audit, 24 of 36 claims were false. In Lakebed, 56 of 78 findings were wrong. An architecture review produced more false findings than a competing model produced true ones.

The production failure. A PR Gemini 4 Argan created for the T3 code SQLite layer ran to 1,200 lines when the fix needed about 30. It read files synchronously, missed two existing PRs that fixed the problem, failed CI, and ignored review-bot comments. The reviewer closed it, handed it to Opus 5.5, then merged a 21-line fix. That sequence matters more than any benchmark: agentic coding value depends on converging to a correct, minimal change.

The useful bits. The model has strengths. Front-end generation was decent, especially with Anthropic's design skill, though below Fable 5.1. Unique Google capabilities matter for specific workflows: cheap YouTube transcript access, vision tasks, and localization are places where Gemini remains useful. But none of that changes the software-team conclusion. A model that is more likely to be wrong than right in audits should not drive code review, architecture analysis, or autonomous PRs without a strong verification layer.

The organizational lesson. The bigger story is workflow design. A model can rank high on synthetic benchmarks and still erode team velocity when every claim gets checked. The reviewer estimates he spent more time vetting Gemini output than writing the code with a smarter model. For engineering leaders, the question is not which model has the best leaderboard score, but what fraction of its output survives a real code review and at what cost in reviewer time and cache misses.