What Chroma Actually Is

Chroma is an open-source embedding database built primarily for Python developers working on retrieval-augmented generation, semantic search, and similarity-based applications. It is not a general-purpose database, and it is not trying to be one. The project focuses on three concerns: storing dense vector embeddings, persisting those vectors alongside the source documents and any metadata you want to attach, and providing a query interface that combines nearest-neighbor search with traditional filtering.

Under the hood, the current version uses a Rust-based core for the actual vector operations, which is a meaningful step up from the original pure-Python implementation. Storage is handled through a pluggable backend, with DuckDB-backed persistent mode being the default for single-node use. The project also ships a client-server mode where Chroma runs as a standalone process that your application talks to over HTTP.

Conceptually, a Chroma deployment has three building blocks. A Client connects you to a Chroma instance, either in-process or over the network. A Collection is the unit of organization, roughly analogous to a table, and it owns a single embedding function plus a set of records. A Record is the actual data: an ID, an optional document, an optional embedding, and an optional metadata dictionary. The query interface takes a text or vector input, embeds it, and returns the closest records.

The model is intentionally narrow. Chroma does not handle document parsing, chunking, or LLM orchestration. It expects you to bring your own pipeline and use Chroma as the storage and retrieval layer. This is a design choice, not a missing feature, and it keeps the surface area small enough that you can read the entire API in an afternoon.

Setup and Authentication

The fastest path into Chroma is a single pip install.

Running this command pulls in the Python client, the embedded server components, and the default embedding function, which is a small sentence-transformers model that runs locally. You do not need any API keys for this mode because the embedding model is bundled with the package. The first time you create a collection with the default function, the model downloads, which can take a minute or two depending on your connection.

If you want to use OpenAI, Cohere, or another hosted embedding provider, you set the relevant API key as an environment variable and pass the corresponding embedding function to the client. Chroma does not store credentials anywhere. The pattern looks like this.

import os
import chromadb
from chromadb.utils import embedding_functions

os.environ["OPENAI_API_KEY"] = "sk-..."

client = chromadb.PersistentClient(path="./chroma_store")
openai_ef = embedding_functions.OpenAIEmbeddingFunction(
    model_name="text-embedding-3-small"
)
collection = client.get_or_create_collection(
    name="documents",
    embedding_function=openai_ef,
)

For a team setup, the recommended approach is to run Chroma as a server. The command chroma run --path /var/lib/chroma starts an HTTP server on port 8000 by default, and the Python client points at it with chromadb.HttpClient(host="localhost", port=8000). In production this is the configuration you want, because it isolates the embedding workload from your application process and makes it easier to monitor and scale independently.

Authentication for the server mode is intentionally minimal in the open-source version. There is an experimental auth feature using bearer tokens, but the common production pattern is to run Chroma behind a reverse proxy that handles auth at the network layer. If you need a managed version with proper auth, RBAC, and SLAs, Chroma offers a cloud product, though the open-source path is the one most teams start with.

First Working Example

Here is a complete, runnable script that creates a collection, ingests a few documents, runs a query, and prints the results.

import chromadb

client = chromadb.PersistentClient(path="./chroma_store")
collection = client.get_or_create_collection(name="documents")

collection.add(
    ids=["doc1", "doc2", "doc3"],
    documents=[
        "Chroma is an embedding database for AI applications.",
        "Postgres handles structured data better than vector search.",
        "Retrieval augmented generation improves LLM answer quality.",
    ],
    metadatas=[
        {"source": "docs", "topic": "chroma"},
        {"source": "docs", "topic": "postgres"},
        {"source": "docs", "topic": "rag"},
    ],
)

results = collection.query(
    query_texts=["How do I store vector embeddings?"],
    n_results=2,
)

for doc, meta, dist in zip(
    results["documents"][0],
    results["metadatas"][0],
    results["distances"][0],
):
    print(f"[{dist:.3f}] {meta['topic']}: {doc}")

When you run this, the add call embeds the three documents using the default model, stores the vectors alongside the text and metadata, and persists everything to the ./chroma_store directory. The query call embeds the question, performs a nearest-neighbor search, and returns the top two matches. You should see the first and third documents come back ranked highest, because they are about Chroma and RAG, both of which are topically close to the question.

A few things to notice. You never explicitly created the embedding function in the default case. Chroma falls back to its bundled model when no function is specified. The IDs are strings you control, and using stable, deterministic IDs (like a hash of source URL plus chunk index) makes ingestion idempotent. The metadata dictionary is where you store everything you will want to filter on later.

Key Settings That Matter

The collection configuration is where most of the meaningful choices live, and these are the dials most people ignore.

The embedding function is the most consequential setting because it determines what “similar” means for your data. The default model is a 384-dimensional MiniLM variant that is fine for prototypes but rarely the best choice for production RAG. Higher-quality models from OpenAI, Cohere, or Voyage typically produce noticeably better retrieval, at the cost of API spend and latency. Switching the embedding function on an existing collection requires re-indexing, so choose early.

The distance function defaults to L2 (squared Euclidean), but cosine similarity is usually more appropriate for text embeddings because it ignores vector magnitude and focuses on direction. You set this at collection creation with a metadata key.

collection = client.get_or_create_collection(
    name="documents",
    metadata={"hnsw:space": "cosine"},
)

HNSW parameters control the speed-recall tradeoff. Chroma exposes them through collection metadata using keys like hnsw:M, hnsw:construction_ef, and hnsw:search_ef. The defaults are reasonable for collections under a few hundred thousand vectors. Past that range, you may need to tune them. Higher values mean slower indexing and queries but better recall, which is the usual pattern for ANN indexes.

Metadata filtering is one of Chroma’s underrated features. You can scope queries to a tenant, a date range, or any other field present in the metadata. The filter syntax is a small subset of SQL.

results = collection.query(
    query_texts=["embedding storage"],
    n_results=5,
    where={"topic": "rag"},
    where_document={"$contains": "LLM"},
)

The where clause filters on metadata fields, and the where_document clause filters on document content using operators like $contains. Combining vector search with these filters is where the real power shows up for production use.

Batch size during ingestion matters because embedding is the slow part. Calling add with 100,000 documents at once will blow up memory. Practical batches are in the 100-500 range, and you should stream from your source rather than loading the whole corpus.

Where It Shines

Chroma’s biggest strength is developer ergonomics for small to mid-scale projects. If you are building a personal RAG app, a prototype for a client, or a tool that ingests a few hundred thousand documents, the experience is hard to beat. The Python API is clean, the defaults are sensible, and you can go from zero to a working semantic search in a few minutes.

It also shines when the team is small and the deployment