What Helicone Actually Is

Helicone is an open-source observability layer that sits between your application and LLM providers. Technically, it is a proxy server. You point your existing OpenAI or Anthropic client at Helicone’s endpoint instead of the provider’s endpoint, and Helicone forwards the request, captures the response, and logs everything that happened.

What it captures per request:

• Full prompt and completion text
• Input and output token counts
• End-to-end latency
• Cost in dollars, calculated from token usage and the model’s published pricing
• Model identifier and provider
• HTTP status code
• Any custom properties you attach via headers
• Any user identifier you attach via header

What you get back is a dashboard at helicone.ai with filters, time-series charts, and request-level inspection. It is not a model router. It is not primarily a caching layer. It is a logging and analytics layer that happens to also offer caching, rate limiting, and prompt management as side features.

The architecture matters because the integration is minimal. Because it is a proxy, you do not need to install a vendor SDK to get the basics working. You change one URL and add one header. That is the entire integration surface for the core feature.

Helicone is open source on GitHub under the helicone/helicone repo. The cloud product at helicone.ai is the hosted version of that codebase. If you have compliance or data residency requirements, you can self-host the same stack.

Setup and Authentication

There are two paths: the cloud-hosted Helicone or self-hosted.

Cloud path, the one most teams should start with:

1. Sign up at helicone.ai using email or GitHub
2. Open Settings, then API Keys
3. Generate a Helicone API key for your workspace
4. In your OpenAI client, change the base URL to https://oai.helicone.ai/v1
5. Add the header Helicone-Auth with the value Bearer followed by your Helicone key
6. Keep using your OpenAI API key as normal in the standard api_key field

The auth model is worth understanding. Helicone uses your existing provider key unchanged. You are not giving Helicone long-term access to your OpenAI account or rotating credentials. The Helicone key only identifies your workspace to the proxy so it knows which dashboard to log to.

Self-hosted path:

• Clone the helicone/helicone repository
• The Docker compose file brings up the API, the web dashboard, Postgres, ClickHouse, and Redis
• You will need to provision these properly for production, including persistent volumes and backups
• Useful when you cannot send request data to a third party for compliance reasons

For most teams starting out, the cloud version is the right move. The free tier covers a reasonable volume for development workloads and small production traffic. Paid tiers unlock higher rate limits, longer retention, and team features.

Environment variables worth setting up from day one:

• HELICONE_API_KEY in your deployment environment
• OPENAI_API_KEY as before
• ANTHROPIC_API_KEY if you use Anthropic through the proxy at https://anthropic.helicone.ai/v1

First Working Example

Python with the official OpenAI SDK:

from openai import OpenAI

client = OpenAI(
    base_url="https://oai.helicone.ai/v1",
    api_key="<your-openai-key>",
    default_headers={
        "Helicone-Auth": "Bearer <your-helicone-key>"
    }
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}]
)

print(response.choices[0].message.content)

Run that script. Log into helicone.ai and you should see the request appear in your dashboard within a few seconds. If you do not see it, the most common causes are a missing Helicone-Auth header, a typo in the base URL, or the Helicone key belonging to a different workspace.

Node equivalent using the openai package:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://oai.helicone.ai/v1",
  apiKey: process.env.OPENAI_API_KEY,
  defaultHeaders: {
    "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`
  }
});

const response = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Hello" }]
});

To verify the integration is actually flowing through Helicone and not bypassing it, attach a custom property:

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
    extra_headers={
        "Helicone-Property-Environment": "dev",
        "Helicone-Property-User": "test-user-123"
    }
)

These properties become filter chips in the dashboard. If you can filter by them and see your request, the integration is confirmed working end to end.

Key Settings That Matter

Most teams set up Helicone and never touch these. They are where the value compounds.

Custom properties. Any header of the form Helicone-Property-* becomes a first-class searchable field in the dashboard. Use them for environment, user_id, feature name, A/B test variant, customer tenant, or anything else you want to slice by later. The dashboard supports filtering and grouping on these.

User tracking. The Helicone-User header ties requests to a specific end user. This unlocks per-user cost analysis, abuse detection, and the ability to see everything one customer has done in a session.

Caching. Set Helicone-Cache-Enabled: true along with Helicone-Cache-TTL of a duration in seconds. Helicone will return cached responses for identical request bodies within the TTL window. This saves real money on repeated queries like classification, extraction, and embedding generation. The cache key is an exact match on the full request body, so any whitespace or parameter change invalidates the match.

Rate limiting. Configure per-user or per-property rate limits in the dashboard. Useful for protecting against runaway agent loops or abusive tenants in a multi-tenant SaaS.

Request body logging toggle. If you handle sensitive data, you can disable logging of request and response bodies while keeping metadata. Set Helicone-Disable-Logging: true on specific requests to skip logging entirely. You still get the cost and latency, just not the prompt content.

Prompt versioning. The Helicone-Prompt-Id header ties a request to a prompt template stored in the Helicone prompt registry. You can version prompts, roll back, and A/B test by sending traffic to different prompt IDs.

Streaming support. Helicone works with stream=True on both OpenAI and Anthropic. The full assembled response is logged after the stream completes, so token-by-token latency is not available but the final output and total latency are.

The most underused feature is custom properties. Most teams log everything as anonymous requests and then cannot filter by feature or customer. Add properties from the first request you ship.

Where It Shines

Cost attribution across features. When you have multiple product features hitting OpenAI, custom properties let you see exactly what each feature costs. Without this, you get one bill at the end of the month and no idea where the money went.

Debugging production LLM issues. When a user reports the AI gave a weird answer, you can search by user_id, find the exact request and response, see the latency, see if it was a timeout or a model refusal. This beats grepping application logs every time.

Caching for deterministic workloads. Classification, extraction, embedding generation, and routing prompts are usually deterministic. Same input, same output. Helicone cache turns these into free requests, which compounds quickly when you have a chat product with repeated system prompts.

Multi-provider visibility. The same dashboard covers OpenAI, Anthropic, and other providers if you use their respective proxy endpoints. One place to see total LLM spend across vendors.

Compliance-sensitive logging. Self-hosting means request and response data never leaves your infrastructure. Useful for healthcare, finance, legal, and any regulated workload.

Prompt iteration tracking. Prompt versioning means you can see which prompt version produced which results, compare them, and roll back if a new version tanks quality.

Where It Fails

It is not a model router. Helicone does not pick the cheapest model for you or fall back automatically when one provider is down. You decide the model in your code.

Latency overhead. The proxy adds a network hop. Typical overhead is in the low tens of milliseconds, which is invisible for most applications. For latency-critical workloads like real-time voice or interactive agents, test before committing.

Limited analytics depth. Helicone gives you request-level data and basic aggregations. It is not a full analytics platform. If you need cohort analysis, funnel analysis, or custom dashboards, export to a warehouse and build there.

No built-in evaluation framework. Helicone logs what happened. It does not score quality. Pair it with something like Braintrust, Langfuse, or your own eval pipeline if you need automated quality scoring.

Cache invalidation is manual. If your prompt logic changes, cached responses from the old logic persist until the TTL expires. You can bust the cache by changing the Helicone-Cache-Seed header, which acts as a namespace for the cache key.

Self-hosting is non-trivial. The Docker compose setup works for development but production self-hosting needs attention to Postgres backups, ClickHouse scaling, and Redis sizing. Plan for ongoing operational work.

Streaming analytics are weaker. You get the final response logged but not token-by-token latency breakdowns. If you need that, look at a tool focused on streaming telemetry.

Practical Workflow Pattern

Here is how a real team slots Helicone into a production setup.

Stage 1 is drop-in observability. Add the Helicone proxy to your existing client. Add Helicone-User from your auth context and two or three custom properties for environment, feature, and tenant. Ship to production. You now have cost and latency visibility you did not have before, with zero changes to your application logic.

Stage 2 is cost controls. Identify the top three cost drivers from the dashboard. Add caching where the workload is deterministic. Set rate limits per tenant if you run multi-tenant SaaS. Add alerts for cost spikes so you find out before the invoice does.

Stage 3 is prompt management. Move your prompts into the Helicone prompt registry. Version them. A/B test by sending a percentage of traffic to different prompt IDs. Roll back if quality drops. This is where prompt iteration stops being a code deploy and starts being a config change.

Stage 4 is export to your warehouse. Set up Helicone’s data export to BigQuery, Snowflake, or Postgres. Build your own dashboards for product-level metrics. Helicone becomes the operational layer for live debugging and quick checks. Your warehouse becomes the analytical layer for deeper questions.

The pattern that fails is treating Helicone as a one-time install and never using the properties or prompt features. Teams get a dashboard, look at it once, and go back to console.log debugging. The properties are where the value compounds over months.

A practical tip: wrap your LLM client so it automatically injects Helicone-User from your auth context and Helicone-Property-Feature from your routing layer. Do not make every callsite remember to add headers, because they will forget.

Another practical tip: log the Helicone request ID alongside your application logs. Helicone returns a request ID in response headers. When something breaks, search Helicone by that ID and your logs by that ID and correlate the two views in seconds.

A final practical tip: do not log request bodies for sensitive workloads. Use Helicone-Disable-Logging on requests that handle PII, or scrub bodies before they hit the proxy. The metadata is usually enough for debugging without retaining the actual content.

The practical next step is the free Working With Claude field guide. Thirty-two pages covering the ecosystem, Claude Code, and how to govern a rollout properly. Get your copy.