AI production cost audit
Enter your monthly tokens, cache-hit share and cloud bills to estimate what routing, caching, batch APIs and right-sizing would save. Prices checked November 20, 2026.
1. LLM usage per month
2. Hosting, database and team
3. Assumptions (edit to match your workload)
Classification, extraction, routing and short answers usually qualify. Test on your own evals before you switch.
Stable system prompts, tool definitions and shared documents placed first in the prompt are what get cached.
Evals, backfills, enrichment and nightly reports can use the batch APIs.
Your estimate of idle or oversized capacity: always-on preview environments, oversized database instances, unused vector indexes.
| Lever | Per month | How |
|---|
How to use it
- Add each model you call in production with its monthly input and output tokens, in millions, and the share of input that is already cached. Your provider's usage dashboard has these numbers.
- Enter your monthly hosting, database and vector database bills, and how many engineers work on the product.
- Set the assumptions: how much traffic a smaller model can handle, the cache-hit share you can reach, how much work can wait for a batch job, and how much infrastructure is idle or oversized.
- Read the savings by lever, then download the refactor spec for your team.
How the estimate works
The levers are applied in order so savings are not counted twice. Routing moves your chosen share of each workload to the same provider's small model (GPT-6 Luna, Claude Haiku 5.5 or Gemini 3.5 Flash-Lite). Caching then raises the cached share of input to your target, prices it at the cached-input rate, and subtracts the cache-write premium, assuming one write for every ten reads. Batch applies each provider's listed 50% discount to the share of remaining work that can wait. Right-sizing applies your percentage to hosting, database and vector spend. List prices only: enterprise discounts, regional rates and long-context surcharges are not included.
The tables below are the data the calculator uses. Every price links to the provider page it came from, checked on November 20, 2026.
LLM API list prices per million tokens
Standard tier, short-context rates, in US dollars, as published on November 20, 2026. Batch prices are the listed batch discount applied to standard rates.
| Provider | Model | Input | Cached input | Output | Batch input | Batch output |
|---|---|---|---|---|---|---|
| OpenAI | GPT-6.2 Astra | $8.00 | $0.40 | $40 | $4.00 | $20 |
| OpenAI | GPT-6.2 Sol | $2.00 | $0.10 | $10 | $1.00 | $5.00 |
| OpenAI | GPT-6.1 Terra | $1.00 | $0.05 | $5.00 | $0.50 | $2.50 |
| OpenAI | GPT-6 Luna | $0.10 | $0.01 | $0.50 | $0.05 | $0.25 |
| Anthropic | Claude Fable 5.2 | $8.00 | $0.20 | $40 | $4.00 | $20 |
| Anthropic | Claude Opus 5.5 | $4.00 | $0.20 | $20 | $2.00 | $10 |
| Anthropic | Claude Sonnet 5.5 | $2.00 | $0.10 | $10 | $1.00 | $5.00 |
| Anthropic | Claude Haiku 5.5 | $0.10 | $0.01 | $0.50 | $0.05 | $0.25 |
| Gemini 4 Neon | $3.00 | $0.15 | $15 | $1.50 | $7.50 | |
| Gemini 4 Argon | $2.00 | $0.10 | $10 | $1.00 | $5.00 | |
| Gemini 3.8 Flash | $0.75 | $0.07 | $3.75 | $0.38 | $1.88 | |
| Gemini 3.5 Flash-Lite | $0.30 | $0.03 | $2.50 | $0.15 | $1.25 | |
| Gemini 2.5 Flash-Lite | $0.10 | $0.01 | $0.40 | $0.05 | $0.20 |
Prompt caching and batch rules by provider
| Provider | Cache read price | Cache write | Minimum cacheable prompt | Batch discount |
|---|---|---|---|---|
| OpenAI | 0.1x input on most current models (0.05x on some) | 30-minute write 1.25x input; 12-hour write 2.5x input (GPT-5.6 and later) | 1,024 tokens (GPT-5.6 and later) | 50%, 24-hour window |
| Anthropic | 0.1x input on most models; 0.05x on Opus 5.5 and Sonnet 5.5; 0.025x on Fable 5.1 and 5.2 | 5-minute write 1.25x input; 1-hour write 2x input | 512 tokens on current models (up to 4,096 on older ones) | 50%, expires after 24 hours; combines with caching |
| Listed per model (for example $0.075 on Gemini 3.8 Flash against $0.75 input) | No write fee listed; explicit caches bill storage per million tokens per hour; Gemini 4 models cache implicitly only | 4,096 tokens (Gemini 4, 3.8 and 3.6 Flash) | 50%, 24-hour target; caching supported in batch |
Worked example: 100M input and 20M output tokens a month
The same monthly workload on each model at list price. It shows why routing matters: the gap between a flagship and a small model in the same family is often more than ten times.
| Model | No caching | 50% of input cached | 50% cached, all batch |
|---|---|---|---|
| GPT-6.2 Astra | $1,600 | $1,220 | $610 |
| GPT-6.2 Sol | $400 | $305 | $152 |
| GPT-6.1 Terra | $200 | $152 | $76.25 |
| GPT-6 Luna | $20.00 | $15.50 | $7.75 |
| Claude Fable 5.2 | $1,600 | $1,210 | $605 |
| Claude Opus 5.5 | $800 | $610 | $305 |
| Claude Sonnet 5.5 | $400 | $305 | $152 |
| Claude Haiku 5.5 | $20.00 | $15.50 | $7.75 |
| Gemini 4 Neon | $600 | $458 | $229 |
| Gemini 4 Argon | $400 | $305 | $152 |
| Gemini 3.8 Flash | $150 | $116 | $58.12 |
| Gemini 3.5 Flash-Lite | $80.00 | $66.50 | $33.25 |
| Gemini 2.5 Flash-Lite | $18.00 | $13.50 | $6.75 |
Hosting and database price points
| Service | Published price | Source |
|---|---|---|
| Vercel Pro | $20 per month platform fee, includes 1 deploying seat and $20 of usage credit; extra seats $20 per month | Vercel docs |
| Vercel functions (Pro, iad1) | $0.60 per 1M invocations; Active CPU $0.128 per hour; provisioned memory $0.0106 per GB-hour | Vercel docs |
| Vercel data transfer (iad1) | Fast Data Transfer $0.15 per GB; CDN requests $2.00 per 1M | Vercel docs |
| Supabase plans | Pro from $25 per month, Team from $599; paid plans include $10 per month of compute credits | Supabase pricing |
| Supabase compute | Micro $0.01344 per hour (about $10 per month, 1 GB); Small $0.0206 per hour (about $15 per month, 2 GB); Medium $0.0822 per hour (about $60 per month, 4 GB); Large $0.1517 per hour (about $110 per month, 8 GB); XL $0.2877 per hour (about $210 per month, 16 GB); 2XL $0.562 per hour (about $410 per month, 32 GB); 4XL $1.32 per hour (about $960 per month, 64 GB) | Supabase docs |
| Supabase disk and egress | 8 GB disk included, then $0.125 per GB; 250 GB egress included, then $0.09 per GB | Supabase pricing |
| AWS Fargate (us-east-1) | x86 $0.04048 per vCPU-hour and $0.004445 per GB-hour; ARM $0.03238 and $0.00356; Fargate Spot up to 70% off | AWS Fargate pricing |
| Amazon RDS for PostgreSQL (us-east-1) | db.t4g.medium $0.065 per hour Single-AZ ($0.129 Multi-AZ); db.m7g.large $0.168 ($0.337 Multi-AZ) | Amazon RDS pricing |
| AWS Savings Plans | Compute Savings Plans up to 66% off; up to 50% on Fargate | AWS Savings Plans |
| Pinecone | Standard: $50 per month minimum, storage $0.33 per GB-month, read units from $16 per 1M, write units from $4 per 1M; Enterprise $500 per month minimum | Pinecone pricing |
Vector storage at 10 million embeddings
Ten million vectors at 1,536 dimensions in float32 are about 61.4 GB before index overhead. Storage is the smallest part of the bill on both options; read volume and the compute behind the index decide the total.
| Option | Raw vector storage | Storage cost per month | What else you pay for |
|---|---|---|---|
| Pinecone Standard | 61.4 GB | $20.28 (the $50 monthly minimum applies) | Read units (from $16 per 1M) and write units (from $4 per 1M), which scale with query and upsert volume |
| pgvector on Supabase Pro | 61.4 GB | $6.68 of disk beyond the 8 GB included | The compute size that keeps the index and working set in memory (see Supabase compute above), plus the plan fee |
What usually moves the bill
- Output length. Output is priced five times higher than input on most models in the table. Shorter answers and structured output cut the most expensive tokens.
- Prompt layout. Caches match on the start of the prompt. Put the system prompt, tool definitions and reference documents first and the user's question last.
- Retrieval size. Sending twenty chunks when three answer the question multiplies input cost on every call.
- Per-feature metering. Most teams cannot say which feature spends the money. Log model, tokens and cost per request so routing decisions are based on data.
- Idle infrastructure. Preview environments, oversized database instances and vector indexes nobody queries show up on every invoice.
Who does the work
These changes are engineering work: an eval harness before any model switch, prompt restructuring, batch pipelines with retries, and infrastructure changes with rollback plans. Our AI pod teams do this on your team, reporting to your leads, in your cloud and repos. If the product started as a prototype, the prototype-to-production runbooks and risk scorer cover the security and reliability side, and the Heroku to AWS runbook covers a platform move. For a single workload, the LLM cost calculator compares one prompt across models.
FAQ
How much can prompt caching save on an LLM bill?
It depends on how much of each prompt repeats. Cached input is billed at a fraction of the normal input price: for example, Claude Sonnet 5.5 lists $2 per million input tokens and $0.10 for cache hits, and GPT-6.2 Sol lists $2 and $0.10. Some providers also charge more to write the cache, so savings come from long, stable prefixes that are reused many times.
Do batch API discounts stack with prompt caching?
Anthropic's pricing page says the Batch API and prompt caching discounts can be combined. OpenAI's pricing page lists separate cached-input rates for batch requests, and Google's batch documentation says context caching is supported for batch requests. All three list a 50% batch discount with a 24-hour window.
What is model routing?
Model routing sends each request to the cheapest model that handles it well, and keeps the large model for the hard cases. In production that means an eval set built from real traffic, a router (rules or a small classifier), and a fallback to the larger model when confidence is low.
Is pgvector cheaper than Pinecone at 10 million embeddings?
On storage alone, 10 million 1536-dimension float32 vectors are about 61 GB, which costs less on Supabase disk than on Pinecone storage. The real comparison is the compute you need to keep a pgvector index fast versus Pinecone read and write units, so model both with your query volume.
How current are these prices?
Every price on this page was read from the provider's own pricing page on November 20, 2026, and the page shows the checked date. Prices change often, so confirm on the linked pages before you commit to a plan. Google has announced increases for some Gemini models from December 1, 2026 and January 1, 2027.
Want engineers to make the cuts?
Ryz Labs AI pods are senior engineers who build evals, routing and caching into AI systems in your cloud, on your team, reporting to your leads.
Start a conversation →