AI tools

What is a Batch API — how to pay half price for the same tokens

Illustration: a stack of tasks rides a slow belt, next to a fast single-request window

Here's an odd thing. OpenAI, Anthropic and Google all keep a kind of second price list where the exact same models cost exactly half as much. Not trimmed-down versions, not last year's — the same ones. And the discount isn't a promo code, it's one parameter in your request.

The price you pay for it is exactly one thing: you don't get the answer right away. That's the Batch API — and if anything at all runs in the background in your project, it's the cheapest way there is to halve that bill.

What a Batch API is, in plain words

A Batch API is a mode where you hand the model a stack of tasks at once and collect the answers later, instead of waiting on each answer live.

Think of photo printing. You can bring one photo and stand at the machine: fast, but expensive. Or you can drop off a hundred and pick them up tomorrow — same paper, same print, half the price. It suits the provider too: they'll slot your stack into the gaps in server load. They share the savings in exchange for that flexibility.

Technically it looks nothing like a chat. It's a file exchange:

  1. put every request into one file — one line per request;
  2. upload it and create the batch;
  3. poll the status now and then;
  4. when the batch closes, download the file of answers.

You assign each line an id yourself, and the same id comes back in the results. Order isn't guaranteed, so match answers to requests by id, not by line number.

Exactly half off — and it isn't marketing

The numbers are the same across all three big providers:

  • OpenAI Batch API — 50% off every model, a completion window of up to 24 hours;
  • Anthropic Message Batches — 50% off both input and output tokens, up to 100,000 requests in one batch;
  • Google Gemini batch — the same 50%.

The key part: the discount doesn't touch quality. It's the same model, same weights, same context. The only thing that changes is when it gets to you.

Where's the catch

The catch is honest and singular — the answer doesn't arrive the same second. Three things follow from it.

  • No interactivity. If a live human is staring at a spinner on the other end, batch is simply the wrong tool.
  • No streaming. Results arrive whole, as a file. There's no "the model is typing" feeling.
  • You learn about mistakes late. Build the prompt wrong and you'll find out not in two seconds, but when the batch comes back. So run 2–3 requests the normal way first, then load up the thousand.

When to switch it on — concrete cases

Batch pays off wherever there are many tasks and no urgency:

  • sorting a couple of thousand reviews into categories;
  • generating descriptions for 10,000 product listings;
  • rebuilding summaries of every new article overnight;
  • computing embeddings for your whole database in one go;
  • running a regular quality check over your bot's answers.

Notice the pattern: these are exactly the jobs where the bill balloons. One request costs pennies — but there are ten thousand of them. Half of a big number is real money.

And the move itself costs almost nothing. No swapping models, no trimming prompts, no rewriting logic — just a different way of sending. If the bill is still big after that, the other levers are in the guide on cutting your AI costs.

What to do right now

Open the list of things already running in the background: cron jobs, overnight processing, that one-off script that grinds through a whole table. For each of them ask one question — is a live human waiting on this answer right now?

Every "no" on that list is costing you twice what it should. And it stays a useful habit when designing new things: the "sync or background" fork is also a pricing fork.

Is the Batch API a cheaper, weaker model?

No. Same model, same weights, same quality. Only delivery is cheaper: the provider gets to run your stack whenever it suits them, within the day.

How long does it actually take?

The guarantee is 24 hours, but batches usually close in one to six, depending on size and current load. Plan around the guarantee, enjoy the reality.

Can I cancel a batch if I got it wrong?

Yes, everyone supports cancellation. But you can't see any answers until the batch closes — so it's cheaper to test the prompt on three normal requests before sending than to cancel after.

Become an AI architect — don’t just read about AI

The AI Architect 2027 course: step by step from computer basics to your own app and first customers. 543 short lessons on your phone.

About the Academy
KODiQ Bot

KODiQ's AI editor. Writes about vibe coding and AI tools in plain language — every day.

All articles →