Why a faster model can make your users wait 8x longer

The proposal. MiniMax's Morgan Suo opens with a familiar question: a new model generates answers faster, so should you switch? His answer: test the whole job, not the model. The test job is an analyst research report covering 14 companies, where the model must find the right reports, align reporting periods, keep business segments separate, and explain the numbers. The standard is whether an analyst can open sources, check a calculation, and see what evidence could change the conclusion.

What changes. To make M3.1 flash faster, MiniMax uses sparse attention to select text blocks, quantizes the KV cache to 4 bits while queries stay at 8 bits, applies 4-bit weights and activations in the MoE experts, and adds speculative decoding. Each optimization reduces data movement or computation, and each one gets checked against the research job. The point is that these changes interact with the user's workflow in ways a benchmark table won't show.

The catch. The model's own evaluation table shows why. Average scores stay roughly flat, but the sign score drops 6.6 percentage points. A report on cash flow versus profit could live exactly in that regression. The speaker warns that an average hides a mistake a user cares about. This is the strongest part of the talk and worth carrying into any model evaluation.

The serving trap. The most concrete number comes from serving tests. When concurrency doubles from 32 to 64, total output per second grows about 1.9 times, but p95 time to first token rises from about 1.5 seconds to 11.3 seconds. Median latency can look fine while a real share of users feel an eight times longer wait. Throughput alone is the wrong metric for a product with users.

Reasoning budget. MiniMax also controls how much thinking the model does. On a coding test, the high reasoning setting used fewer output tokens than max reasoning, but accuracy fell from 80% to 68%. Fewer tokens did not mean fewer errors. The default reasoning level changes the amount of work you request and the quality you get, so teams need to record the setting as part of any evaluation.

What to do. Suo closes with three checks: can an analyst trust the report and verify it; what did the finished job cost including retrieval, tool calls, and human fixes; and can the result be repeated with saved model versions, prompt settings, sources, and failure examples. He suggests a small rollout with a rollback path. The talk is a vendor pitch, but the evaluation discipline is sound and applies to any model swap.