API cost control

Before You Add AI to Your App, Estimate the Token Bill

Adding an AI feature is easy. Keeping the bill predictable is harder.

Most AI cost problems start before the first production user. A team builds the feature, likes the demo, ships it, and only later asks the question that should have been asked earlier: how much does this cost per user?

That question does not need a perfect financial model. It needs a rough, honest estimate before launch.

The simple formula

For most LLM features, start with this shape:

monthly cost =
requests per user
* active users
* average tokens per request
* price per token

Then add a safety margin. The mistake is not being slightly wrong. The mistake is shipping with no model at all.

Estimate input and output separately

Do not use one token number for everything. Estimate input tokens, output tokens, system prompt tokens, retrieved context tokens, retry tokens, and background batch tokens separately.

A support summary feature can look cheap until every request includes a long conversation history. A coding helper can look cheap until retries and regenerations become normal user behavior.

Watch for hidden multipliers

AI costs often grow through multipliers that are easy to miss in a demo:

• One user action triggers multiple model calls.
• A retry system doubles requests during failures.
• A long system prompt is sent every time.
• Retrieval adds too much context.
• The feature runs in the background for inactive users.
• Logs or evaluations call another model.

One simple AI feature can become a chain. Price the chain, not the first link.

Calculate cost per successful outcome

Do not only calculate cost per request. Calculate cost per successful outcome.

If 100 requests create 60 useful results, your effective cost is higher than the raw API price suggests. Ask how many outputs are accepted, how many need regeneration, how many are ignored, and how many require human correction.

Put limits in the product

Before launch, define max tokens per request, max requests per user per day, max retries, cache strategy, model fallback rules, and when to use a cheaper model.

Limits are product design. They are not just engineering details. A useful AI feature should have a boundary before the invoice teaches the lesson.

A practical pre-launch checklist

1. What is the expected cost per active user?
2. What is the worst-case cost per active user?
3. Which part of the prompt grows with user data?
4. What happens if users retry five times?
5. Can a cheaper model handle most requests?
6. What usage limit protects the business?
7. What metric proves the feature is worth the cost?