The cost of AI

This page is an overview of what AI costs, from a single API call up to the data centers behind it. Prices change quickly, so the figures here are examples and orders of magnitude. The structure of the costs changes much more slowly, and it explains most pricing decisions you will meet.

Two kinds of cost: training and inference

Training is a one-time cost per model. A frontier training run occupies tens of thousands of specialized chips for weeks or months. Sam Altman said GPT-4, released in 2023, cost more than 100 million dollars to train, and newer frontier models cost far more. Behind the final run are years of research, failed experiments, data, and staff.

Inference is the cost of using the model, and it occurs on every request. Each answer requires computation on expensive hardware. As usage grows to billions of requests, inference becomes the larger share of the total. It is also the cost you pay directly as a customer.

The contrast became visible in early 2025, when DeepSeek reported that the final training run of its V3 model cost about 5.6 million dollars. The figure left out research, earlier experiments, and hardware purchases, but it showed that clever engineering can cut training costs dramatically.

Paying per token

API prices are quoted per million tokens, separately for input and output. Output costs several times more than input, because the model generates output one token at a time.

An example with illustrative prices of 3 dollars per million input tokens and 15 dollars per million output tokens: a support assistant handles 1,000 conversations a day, each with 2,000 tokens of input (instructions, retrieved documents, and the question) and 500 tokens of output.

That is about 400 dollars a month. The same workload on a small model can cost a tenth of that, and on the largest flagship model several times more.

Some factors make costs grow faster than expected:

• Long context. Every message in a chat resends the whole conversation, so long conversations get more expensive with every turn.
• Reasoning tokens. Reasoning models think before answering, and the hidden thinking is billed as output.
• Agents. An agent may call the model dozens of times for one task, each call with a growing context.
• Retries and failures. Requests that fail validation and are repeated cost twice.

Subscriptions

Chat assistants for consumers and teams are sold as monthly subscriptions. A common pattern is a free tier with limits, a standard tier around 20 dollars per user per month, and premium tiers of 100 to 200 dollars or more for heavy use, early features, and higher limits. Subscriptions hide token costs from the user. The provider makes money on light users and can lose money on very heavy ones, which is why the top tiers have usage limits.

Prices keep falling

The price of a given level of capability falls quickly. Performance that cost a premium when a model was new costs a small fraction of that a year or two later, through smaller models that reach the same quality, better hardware, and more efficient software. Measured per task, AI has become cheaper at a rate few technologies have matched.

Total spending still rises, for two reasons. People use AI for more and larger tasks, such as agents running for an hour instead of a single question. And the newest frontier models, which many users want, stay expensive.

Infrastructure and investment

Behind every API call is physical infrastructure: chips, servers, data centers, power lines, and cooling. The largest technology companies are each spending tens of billions of dollars per year on AI data centers, and projects such as OpenAI’s Stargate plan investments in the hundreds of billions. Nvidia’s chips are in such demand that supply, not price, has often limited growth.

This spending is a bet that demand will continue to grow. Whether revenues will justify it is one of the most debated questions in the industry, and some observers compare the situation to earlier technology bubbles.

Energy and water

According to the International Energy Agency, data centers used about 415 terawatt-hours of electricity in 2024, roughly 1.5 percent of global electricity consumption. The IEA expects that to roughly double to around 950 terawatt-hours by 2030, about 3 percent of global demand, with AI as the main driver. The growth is concentrated in a few regions, mainly the United States and China, where data centers put pressure on local grids.

Per request, the picture is different. The IEA notes that the energy needed per AI task has fallen by at least ten times a year recently, and that a simple text query uses less electricity than running a television for the same time. Video generation, reasoning, and agentic tasks, however, can use hundreds or thousands of times more energy than a simple text answer.

Data centers also use water for cooling, which matters in regions where water is scarce. Data center operators increasingly sign contracts for renewable and nuclear power, and site new facilities where power and water are available.

Keeping your own costs down

1. Pick the smallest model that is good enough. Test a small model first, and use a large one only where it makes a measurable difference.
2. Route by difficulty. Send easy requests to a cheap model and hard ones to a strong model.
3. Use caching. Keep long, repeated parts of the prompt, such as instructions and documents, at the start so the provider’s prompt caching applies.
4. Batch what can wait. Batch processing is often about half the price.
5. Send less context. Retrieve the relevant passages instead of whole documents, and summarize long conversations.
6. Limit output and loops. Set maximum output lengths, and limit the steps and budget of agents.
7. Measure. Log tokens and cost per feature, so you know where the money goes before the bill arrives.