When model prices fall, a feature that felt too expensive yesterday may suddenly look feasible. That is the useful part of the September 2026 release cycle: stronger choices at a wider range of prices. It is also where teams get tempted to choose a model from a price table before they understand the job the model must do.
The headline price is only the beginning
Anthropic lists Claude Opus 5.5 at $4 per million input tokens and $20 per million output tokens, a 20 percent reduction from Opus 5. OpenAI offers GPT-6 Sol and Luna with different pricing tiers; its published table distinguishes standard input, cached input, output, processing modes and context length. A simple screenshot of one row is not a reliable total cost estimate. Check the current provider page before a purchasing decision.
- Record the exact model and pricing tier you intend to use.
- Separate input, cached input, output, tool and infrastructure costs.
- Measure cost per completed user task, rather than cost per million tokens alone.
Price one real task end to end
Consider a support assistant that reads a customer question, retrieves policy passages, drafts an answer and occasionally asks a second model to review the result. A single request can include the user message, system instructions, retrieved text, intermediate tool output and generated text. Output tokens may cost substantially more than input tokens, and a retry doubles part of the work.
Build a small spreadsheet from representative traces: prompts from actual use cases, typical document lengths, frequency of follow-up questions and the proportion of requests that need a second pass. Multiply each part by the corresponding published rate, then add retrieval, storage and any human review. The first useful result is a cost range per resolved question. Your second result is the point where a person or a simpler workflow becomes more economical.
- Measure a short, ordinary, and difficult request.
- Include failed attempts and retries in the estimate.
- Check the bill again after changing retrieval size or output limits.
Compare quality on your own work
Public benchmarks can reveal capabilities, but they do not tell you whether a model handles your policy documents, terminology or edge cases. Assemble a small blind set of the questions people actually ask. Include ambiguous requests, missing information and examples where the assistant should refuse to guess. Score usefulness, factual support, latency and ease of correction.
A smaller model may solve routine extraction reliably while a more capable model handles unusual cases. Routing those requests can be better than putting every task on one model. That design has its own complexity: you need a dependable way to recognize the unusual case and to know when the smaller model has failed. Start with one model and a repeatable evaluation before adding routing.
A comparison you can actually run
Suppose one hundred customer messages arrive each day. Sixty are routine, thirty require checking account context and ten are ambiguous. Take ten examples from each group. Run the same prompts and context through the models you are considering; hide the model names while someone reviews the outputs. Record how many answers are useful without editing, how long the user waits and how many extra calls are needed after the first response.
If a cheaper model needs repeated retries on the ambiguous requests, keep the ordinary requests on that model and test a selective escalation only when you can identify the difficult cases reliably. The result should be an ordinary-looking routing rule with a measurable failure rate, not a promise that one model wins every task.
- Use the same representative inputs for every provider.
- Blind-review a subset of outputs.
- Record both the provider charge and the cost of corrections.
The decision after the comparison
Choose the model and workflow that produces an acceptable result within the time and budget your users can tolerate. Keep a change log with the pricing date and a handful of benchmark examples. A provider may change its catalog; your evaluation set gives you a way to reconsider without starting from opinion every time.
What matters to a visitor is not that your application uses the newest name in the model picker. They notice whether the answer is useful, whether it arrives quickly and whether the product behaves honestly when it cannot help. That is the measure that should decide the architecture.