What Anthropic Batch API actually is

The Batch API is Anthropic’s asynchronous processing mode for the Messages endpoint. Instead of sending a request and waiting for a synchronous response, you bundle many requests into a single batch job, submit it, and pick up the results later. Anthropic processes the batch on dedicated infrastructure and gives you back a JSONL file containing one result per request.

Strip the marketing and the mechanism is straightforward. You POST a list of fully-formed message requests to the batch endpoint. Each request is identical in shape to a normal Messages call, same model, same parameters, same schema. The difference is the orchestration layer. Anthropic runs the requests on capacity that would otherwise sit idle, which is why pricing is discounted. In the current version, the discount is roughly 50 percent on input and output token costs compared to synchronous calls. That is the entire value proposition on paper. The trade is latency.

There is no streaming, no tool use with web fetch, and no real-time interaction. You submit, you wait, you collect. The processing window is up to 24 hours in most cases, though simple batches often complete much faster, typically in the range of minutes to a few hours depending on queue depth and request volume.

If your workload can tolerate hours of delay, the cost difference is the only thing that matters. If you need sub-second responses, you are looking at the wrong product.

Setup and authentication

Authentication uses the same API key as the rest of Anthropic’s surface. The cleanest setup is through the official Python SDK, which is the path I’ll use throughout this guide.

Install the package:

pip install anthropic

Set your key as an environment variable rather than hardcoding it. On macOS or Linux, add this to your shell profile or export it for the current session:

export ANTHROPIC_API_KEY="sk-ant-..."

Verify the connection with a quick synchronous call before touching the batch endpoint. If the SDK can talk to the Messages endpoint, it can talk to the batch endpoint. Nothing exotic is required on the auth side. The batch endpoint shares the same key and the same base URL.

For larger operations, you will also want a working directory with read and write permissions, since you will be handling JSONL files for both submissions and results. Pick a path you control, since the files contain your prompts and completions in plaintext.

First working example

This is a complete, runnable script that creates a small batch job, submits it, polls for completion, and writes the results to disk. The workload here is sentiment classification on a list of customer comments, which is exactly the kind of task the Batch API was built for.

Create a file called build_batch.py. This script takes a JSONL of pending requests and posts it to the batch endpoint:

`import anthropic, json, time

client = anthropic.Anthropic()

with open(“requests.jsonl”, “r”) as f: requests = [json.loads(line) for line in f]

batch = client.messages.batches.create(requests=requests) print(f”Submitted batch: {batch.id}”) print(f”Status: {batch.processing_status}”)`

Now create requests.jsonl. Each line is a complete request payload with a custom_id field for matching results back to inputs:

{"custom_id": "comment-001", "params": {"model": "claude-sonnet-4-5", "max_tokens": 16, "messages": [{"role": "user", "content": "Classify the sentiment of this review as positive, neutral, or negative. Reply with one word only. Review: The shipping was fast but the box arrived damaged."}]}} {"custom_id": "comment-002", "params": {"model": "claude-sonnet-4-5", "max_tokens": 16, "messages": [{"role": "user", "content": "Classify the sentiment of this review as positive, neutral, or negative. Reply with one word only. Review: Best purchase I've made all year."}]}} {"custom_id": "comment-003", "params": {"model": "claude-sonnet-4-5", "max_tokens": 16, "messages": [{"role": "user", "content": "Classify the sentiment of this review as positive, neutral, or negative. Reply with one word only. Review: It works as advertised, nothing more."}]}}

Run the submit script. The SDK will return a batch object with an id you can use to track status. Save that ID somewhere durable, since you will need it to retrieve results even if your local process dies.

Create poll_batch.py:

`import anthropic, time, sys

client = anthropic.Anthropic() batch_id = sys.argv[1]

while True: batch = client.messages.batches.retrieve(batch_id) print(f”Status: {batch.processing_status}, succeeded: {batch.request_counts.succeeded}, errored: {batch.request_counts.errored}”) if batch.processing_status == “ended”: break time.sleep(60)

results = [] for result in client.messages.batches.results(batch_id): results.append({ “custom_id”: result.custom_id, “result”: result.result })

with open(“results.jsonl”, “w”) as f: for r in results: f.write(json.dumps(r) + “\n”)`

Run python poll_batch.py <your_batch_id> and walk away. For three requests the batch will likely complete in a few minutes. For tens of thousands, expect hours. The script writes a results.jsonl file you can stream into a database, a dataframe, or wherever your downstream pipeline expects it.

That is the entire loop. Submit, poll, collect.

Key settings that matter

The obvious setting is the model. In the current version, batch jobs support the same model lineup as synchronous calls, including Sonnet, Ha Opus, and Haiku. Picking a smaller model for high-volume classification is the single biggest cost lever, and Batch pricing applies on top of whatever model you choose.

The next dial is max_tokens. In classification and extraction workloads, this is often set absurdly high. If you ask the model for a single label, cap it at 16 or 32 tokens. The savings compound across thousands of requests.

System prompts count as input tokens on every single request. If you have a long prefix that does not change, you pay for it every time. For large batches, the system prompt can be the dominant cost. Audit it, shorten it, and confirm you actually need it.

The custom_id field deserves more attention than it usually gets. It is the join key between your input and the eventual result. Make it deterministic and meaningful. Use the source record ID, the row number, or a hash. Avoid sequential integers that restart between runs, since they will collide if you re-submit.

Finally, pay attention to the request shape itself. The Batch API does not support every Messages feature. Streaming is out. Some tool configurations have constraints. Read the current documentation for the specific model you are using, since feature parity changes between releases.

Where it shines

Bulk classification is the canonical use case. Sentiment analysis on customer feedback, intent classification on support tickets, spam detection on user submissions, content moderation queues, all of these are perfect fits. The workload is non-urgent, the prompt is short, the output is short, and the volume is high enough that cost compounds quickly.

Dataset enrichment is another strong fit. If you maintain a knowledge base or a product catalog, you can use batch jobs to generate descriptions, tags, summaries, or embeddings metadata in bulk. Run it overnight, wake up to enriched data.

Evaluation pipelines benefit enormously. When you are testing a new prompt against a few thousand examples, batch lets you cut evaluation cost in half while running asynchronously. The same logic applies to red-teaming and regression testing.

Report generation at scheduled intervals, like nightly summaries, weekly briefings, or monthly digests, is a quiet winner. You build the prompts, queue the batch, and the report is ready by morning.

Anywhere you would have written a “fire and forget” worker, the Batch API is a cleaner abstraction than maintaining your own queue infrastructure.

Where it fails

The latency floor is the most obvious limitation. If a user is waiting on a response, batch is the wrong choice. The 24-hour processing window is a ceiling, not a guarantee, and even fast batches take long enough to rule out interactive use.

There is no streaming. If your downstream UI expects incremental output, you will need to redesign. Most batched workloads are not user-facing, so this rarely matters, but it is worth flagging.

Error handling is coarser. A failed request in a batch does not throw an exception in your code, it shows up as an errored result in the output file. You need a retry policy that walks the results, identifies failures, and resubmits them. The SDK does not do this for you.

Some Messages features are restricted in batch mode. Tool use, particularly tools that hit external services, often has stricter rules. If your workflow depends on tool-calling, validate that the specific tool pattern you need is supported in the current version before committing to batch for it.

There is also a soft cap on batch size. Submitting a million requests in one batch is not the same as submitting ten thousand. The endpoint expects reasonable chunks, and oversized batches can hit processing constraints that are not always documented clearly.

Finally, debugging is harder. You cannot see a request as it runs, you cannot peek at intermediate state, and you cannot cancel a single problematic request. If something is wrong with your prompt template, you find out when the entire batch returns.

Practical workflow pattern

Here is the pattern I use in real client work when batch is the right tool for the job.

The trigger is a scheduled job, often a cron task or a workflow orchestrator like Airflow or Prefect, that runs during off-peak hours. It reads pending records from a database table, generates the JSONL file of requests, and submits the batch. The trigger records the batch ID alongside the source records, so a result can always be traced back to its origin.

A separate polling job runs on a short interval, say every five minutes, and checks the status of any open batches. When a batch completes, it streams the results back into the database, updates the source records, and marks them complete. The polling job is small and idempotent, which is the property that matters most for anything that runs unattended.

A retry job handles the errored slice. It walks the results, finds the failures, and either resubmits them in a new batch or routes them to a human review queue, depending on the failure pattern. This is the piece most teams forget, and it is the reason some “batch pipelines” quietly lose data.

The whole loop should be observable. Track batch submission rate, completion rate, error rate, and the cost per processed record. Once you have those numbers, you can make informed decisions about when to use batch and when to bite the bullet on synchronous calls.

For most data-heavy AI workloads, this pattern replaces thousands of dollars of synchronous inference with a fraction of the cost. The implementation is small, the components are well-understood, and the trade is honest. You give up latency and you save money. For any workload where latency is not the constraint, that is a trade worth making.

If you’re building with Claude or Codex right now, grab the free Working With Claude field guide. Thirty-two pages on the full ecosystem, Claude Code in depth, and how to roll agents out properly. Get the free guide.