What Langfuse Actually Is

Langfuse is an open-source observability platform built specifically for applications that call large language models. It captures the inputs, outputs, latency, token usage, and cost of every LLM call your application makes, then lets you inspect, score, and replay those calls from a web UI.

Under the hood it is a tracing system based on OpenTelemetry. Each LLM call becomes a span, and a chain of calls becomes a trace. The platform stores these traces in a database (Postgres plus ClickHouse in the self-hosted version, managed in the cloud version) and exposes them through a dashboard built with Next.js.

There are two deployment paths. The cloud version at langfuse.com gives you a managed instance with a free tier that covers small projects. The self-hosted version runs in Docker and gives you full control of the data, which matters if you are handling regulated workloads or want to keep prompt data on your own infrastructure.

The platform is not just a tracing viewer. It also includes prompt management with versioning, evaluation runs against datasets, user feedback collection, and basic analytics on cost and latency. Most teams adopt it for tracing first and then layer in the other features over time.

Setup and Authentication

For most readers the fastest path is the cloud version. Create an account at cloud.langfuse.com, then create a new project. The project page gives you three credentials you will need:

• LANGFUSE_PUBLIC_KEY
• LANGFUSE_SECRET_KEY
• LANGFUSE_HOST (defaults to the cloud URL but you can point it at a self-hosted instance)

Treat the secret key like any other API credential. Do not commit it to source control. Use environment variables or a secrets manager.

For self-hosting, the canonical setup is the docker-compose file in the Langfuse GitHub repository. The minimum stack is a Postgres database, a ClickHouse database for analytics, Redis for caching, and the Langfuse web and worker services. The compose file wires these together and exposes the web UI on a local port. From there the same SDK works against your local URL by setting LANGFUSE_HOST to that address.

Install the SDK for your language. Python is the most mature:

pip install langfuse

For TypeScript or JavaScript:

npm install langfuse

Both SDKs share the same conceptual model: a client object you initialize with your credentials, decorators or wrapper functions that mark code as traced, and explicit flush calls when running in short-lived scripts.

First Working Example

Here is a minimal Python example that traces a single OpenAI call. It assumes you have OPENAI_API_KEY and your Langfuse credentials set as environment variables.

from langfuse import Langfuse from openai import OpenAI

langfuse = Langfuse() client = OpenAI()

with langfuse.start_as_current_observation( as_type=“span”, name=“openai-call” ) as span: response = client.chat.completions.create( model=“gpt-4o-mini”, messages=[{“role”: “user”, “content”: “Explain tracing in one sentence.”}] ) answer = response.choices[0].message.content span.update(output=answer)

langfuse.flush()

Run the script. Open the Langfuse dashboard, click on Traces, and you will see the call listed with its input, output, latency, and token counts. If you have billing enabled on your OpenAI account, the cost field will populate automatically for supported models.

For a more realistic example, wrap a multi-step pipeline. A typical RAG application has a retrieval step, a prompt construction step, and an LLM call. Each step becomes its own span nested under a parent trace. The @observe decorator is the cleanest way to do this:

from langfuse import observe, Langfuse from openai import OpenAI

langfuse = Langfuse() client = OpenAI()

@observe() def retrieve(query): # replace with your vector store call return [“doc 1”, “doc 2”]

@observe() def generate(query, context): response = client.chat.completions.create( model=“gpt-4o-mini”, messages=[ {“role”: “system”, “content”: f”Use this context: {context}”}, {“role”: “user”, “content”: query} ] ) return response.choices[0].message.content

@observe() def pipeline(question): docs = retrieve(question) return generate(question, docs)

pipeline(“What is observability?”) langfuse.flush()

Each function now appears as a span in the trace, with the parent-child relationship preserved. Inputs and outputs are captured automatically from function arguments and return values.

Key Settings That Matter

The defaults work for most teams, but a handful of settings have outsized impact.

Sampling. For high-volume applications, tracing every call is expensive. The SDK accepts a sample_rate between 0 and 1, which controls what fraction of traces are sent to the server. A common pattern is to sample 100 percent in development and 10 to 20 percent in production, then turn sampling back up when investigating an issue.

User and session identifiers. Langfuse lets you attach a user_id and session_id to each trace. The user_id enables per-user analytics and the session_id groups related traces into a conversation. Without these, every trace looks anonymous and you lose the ability to debug a specific user’s experience.

Environment and release tags. The SDK accepts a release tag that gets attached to every trace. This is how you separate staging from production data in the dashboard. The same applies to environment labels.

Masking sensitive data. The SDK has a mask function you can register to scrub PII or secrets before they leave your application. This matters when prompts contain user data that you are not allowed to store.

Prompt management. The prompt management feature stores versioned prompt templates in Langfuse rather than in your codebase. You fetch the active prompt at runtime through the SDK, which means you can change prompt text without redeploying code. This is one of the most underused features of the platform.

Evaluation. Datasets let you define test cases with expected outputs, then run your pipeline against them and score the results. The score can be a manual rating, a heuristic like exact match, or an LLM-as-judge call to a separate model. Evaluation runs are stored and comparable across versions of your pipeline.

Where It Shines

Langfuse is at its best in three scenarios.

First, debugging a production LLM application. When users report that the chatbot gave a bad answer, you want to find the exact trace, see the prompt that was sent, the model response, the retrieved context, and the latency of each step. Langfuse gives you that view in a few clicks. Without it you are reading logs and hoping.

Second, iterating on prompts. The combination of prompt management, dataset evaluation, and trace inspection turns prompt engineering into a measurable workflow. You change a prompt, run the dataset, compare scores, and ship the winner. Teams that adopt this pattern typically move faster than teams that edit prompts in code.

Third, cost and usage analytics. The platform aggregates token counts and estimated cost across traces, users, and time periods. For a team running multiple LLM features, this is the easiest way to see which feature is burning the budget.

Where It Fails

Langfuse is not a complete observability solution. There are honest gaps.

It does not replace a general APM tool like Datadog or Honeycomb. If you need to trace a complex microservice architecture that happens to call an LLM, you still want a general tracer for the non-LLM parts. Langfuse is focused on the LLM portion.

The self-hosted version has operational overhead. ClickHouse and Postgres both need tuning, the worker service needs monitoring, and backups are your responsibility. For a small team without dedicated platform engineering, the cloud version is usually the right call.

Evaluation features are useful but not sophisticated. The LLM-as-judge setup works, but you have to bring your own judge model and prompts. There is no built-in regression detection or statistical significance testing across runs.

Search and filtering on traces is functional but not great for very large trace volumes. If you are sending millions of traces per day, expect to spend time on indexing and retention policies.

Finally, the SDKs are mature for Python and TypeScript but thinner for other languages. If your stack is Go, Rust, or Java, you will be relying on the OpenTelemetry exporter and writing more glue code.

Practical Workflow Pattern

A reasonable setup for a small team building an LLM product looks like this.

In development, run Langfuse cloud or a local Docker stack. Trace every call at 100 percent. Use the @observe decorator liberally on every function that touches the LLM or its dependencies. Attach user_id from your auth context and session_id from your conversation state.

In staging, run the same setup but use a release tag like staging-2026-07-01. Build a small evaluation dataset of 50 to 100 representative inputs with expected behaviors. Run the pipeline against this dataset on every pull request and post the score as a comment.

In production, sample at a lower rate (10 to 20 percent is typical for APIs at this tier) but always trace user-reported issues at 100 percent by passing a force_sample flag. Store prompts in Langfuse rather than in code