What Vector Databases Actually Is

A vector database is a storage system built to index and query high-dimensional numeric arrays called vectors. These vectors are typically the output of an embedding model, a neural network that converts text, images, or audio into a fixed-length list of floating-point numbers. Dimensions range from 384 for compact models up to 4096 for larger ones, and each dimension captures some semantic feature of the input.

Strip away the marketing language and the technical premise is simple. A traditional database answers “find row where id equals X” or “find rows where the name column contains the word Y.” A vector database answers “find the K vectors most similar to this query vector, ranked by distance in the embedding space.” The data structure that makes this fast is an Approximate Nearest Neighbor index, most commonly HNSW (Hierarchical Navigable Small World) graphs, IVF (Inverted File) indexes, or variants that combine them.

The trade is between exact recall and speed. A brute-force scan comparing your query against every stored vector gives perfect results but becomes unusably slow past a few hundred thousand items. ANN indexes sacrifice a small percentage of recall (the typical range is 95 to 99 percent) to return results in single-digit milliseconds across millions of vectors.

Vectors can be dense (every dimension has a value) or sparse (mostly zeros, used by learned sparse retrieval models). Most beginner use cases involve dense vectors from sentence-transformer style models. The database doesn’t care which you use, as long as you configure the dimension and distance metric correctly.

Everything else in the product, from metadata filters to replication to API design, sits on top of that core indexing engine.

Setup and Authentication

The major vector databases in the current landscape fall into two camps: self-hosted and managed. Self-hosted options include Qdrant, Milvus, and Weaviate, all of which ship as Docker images or Helm charts. Managed options include Pinecone and the cloud versions of the self-hosted products.

For a developer getting started, the fastest path is Docker. Pull the image for whichever engine you’ve chosen, expose the REST port (6333 for Qdrant, 8080 for Milvus standalone, 8080 for Weaviate), expose the gRPC port if your client supports it, set an API key via the engine’s environment variable convention, and start the container. The standard invocation looks like docker run with the -p flags for port mapping, the -e flag for the API key, and the image name. Once running, you can hit the health endpoint to confirm everything started cleanly.

Authentication is usually an API key. The container reads it from an environment variable at startup, and your client library passes it in a header or as a constructor argument. For local development, a simple string is fine. For anything resembling production, store the key in a secrets manager and inject it at runtime.

The CLI client is installed separately, typically via pip or a package manager, and authenticates with a command like qdrant-cli login pointing at the URL and key. The Python and JavaScript clients are the most mature across all the major options, with Go, Java, and Rust generally close behind.

Network configuration is where beginners get stuck. If the database runs in Docker and your code runs on the host, localhost works. If both run in containers, you need to put them on the same Docker network. If the database runs in the cloud, you need to whitelist your client IP or use a VPC peering setup. Plan for this before you start.

First Working Example

A realistic first project is a semantic search over a small document collection. The shape transfers cleanly to production systems.

Start by installing the client library and an embedding model. The sentence-transformers Python package is the standard choice, and the all-MiniLM-L6-v2 model is a sensible default that produces 384-dimensional vectors. The full setup is pip install sentence-transformers followed by pip install for your chosen database client.

Write a script in three sections. The first section reads the API key from an environment variable, instantiates the client, and creates a collection with the matching vector dimensions and a distance metric (cosine is the right choice for normalized embeddings from sentence-transformers).

The second section defines a small list of documents, each a short paragraph of text along with metadata like the source file and a topic tag. Embed each document’s text using the model, then upsert the batch into the collection. Each upsert call needs a unique id, the vector, and a payload object holding the metadata and the original text.

The third section is the query. Take a natural language question, embed it with the same model, send it to the database with a top_k parameter of 5 or 10, and print the results. Each result should include the similarity score, the metadata, and the original text.

A good test query for a knowledge base of 50 to 100 documents is something specific like “how do I configure retention policies.” If the returned documents are clearly related, the system is working. If they look random, the usual suspects are: mismatched embedding models between insert and query, the wrong distance metric, or metadata filter syntax that accidentally excludes valid matches. Check those before touching anything more complex.

Store the script in version control, but keep the API key in a .env file that’s gitignored. This is the same hygiene as any database project, and the habit is worth building early.

Key Settings That Matter

The dials most people ignore are the ones that change performance and cost the most.

Distance metric is the first. Cosine similarity measures the angle between vectors and is the standard for text embeddings because the magnitude usually carries no useful information. Euclidean distance measures straight-line distance and matters when magnitude is meaningful. Dot product is fast but assumes vectors are already normalized, which sentence-transformer models do but custom models might not. Picking wrong doesn’t throw an error. It silently degrades quality.

HNSW parameters are the second. The M parameter controls connections per node in the graph. Higher M means better recall and more memory usage. The efConstruction parameter controls the search breadth during build. Higher values mean slower builds but higher quality indexes. The defaults are reasonable for small datasets but underperform at scale. For collections past a million vectors, you generally want M around 16 to 32 and efConstruction around 200 to 400.

Quantization is the third. Scalar quantization (int8) reduces memory by roughly 4x with minimal recall impact for most use cases. Product quantization reduces it further, sometimes 16x or more, but parameter tuning is harder. If you’re on a managed plan that charges per vector stored, turning on scalar quantization is essentially free money.

Payload indexing is the fourth. By default, metadata filters trigger a scan of all payloads in the candidate set. If you filter on a field heavily, like a tenant_id in a multi-tenant system, create a payload index on that field. Without it, a filter matching 1 percent of your data can take longer than the vector search itself, which defeats the point.

Sharding and replication matter for production. A single shard is fine for development. For production workloads, two replicas are typical for availability, and enough shards to keep each under a few hundred thousand vectors in active memory is a common rule of thumb. The exact numbers depend on vector size, query volume, and latency targets, so benchmark rather than guess.

Where It Shines

Semantic search over unstructured text is the clearest fit. Customer support knowledge bases, internal documentation, legal contract archives, and research libraries all benefit from retrieval by meaning rather than keyword. A user searching for “how do I cancel my subscription” should find documents that don’t contain the word cancel if those documents answer the question well.

Retrieval Augmented Generation is the second. RAG is the pattern where you embed a corpus, store the embeddings, and at query time retrieve the most relevant chunks to feed into an LLM context window. The vector database is the memory layer that grounds model responses in your actual data. Without it, the model hallucinates from its training data. With it, you get answers that cite the documents you provided.

Recommendation systems work well when items and users can be embedded into the same space. Similarity search becomes the recommendation engine. The same pattern handles job matching, content suggestions, and product discovery in e-commerce.

Image and audio similarity is a fourth use case. Embedding models exist for images (CLIP and its descendants) and for audio, and the database doesn’t care whether the input was text or pixels. Reverse image search, near-duplicate detection, and audio fingerprinting all use the same machinery.

Anomaly detection on embeddings is a fifth. Normal patterns cluster in one region of the vector space. A query landing far from any cluster is a signal. This is the architecture behind modern fraud detection, content moderation, and security monitoring at companies operating at scale.

Where It Fails

Full-text search is the first honest limitation. Vector databases do not do exact phrase matching, boolean operators, or keyword highlighting well. If your users need those, pair the vector database with a dedicated search engine like Elasticsearch, Meilisearch, or Typesense. Hybrid retrieval that runs both and merges results is the common production pattern.

Relational queries are the second. There is no equivalent of a SQL JOIN across multiple vector collections. If your data has heavy relational structure, keep the source of truth in a relational database and replicate the relevant slices into the vector store. Treating the vector database as the system of record is a mistake that becomes painful quickly.

Cost is the third. Managed vector databases typically charge per vector stored and per query. At scale, particularly for high-write workloads, this becomes meaningful. Self-hosting shifts cost to infrastructure and operational complexity, which is its own trade, but the per-vector economics change dramatically.

Domain mismatch is the fourth. The default embedding model is trained on general web text. It handles everyday queries well but struggles with specialized vocabulary in medicine, law, engineering, and other technical fields. No amount of index tuning fixes a model that doesn’t understand your domain. The answer is either a fine-tuned model or a model specifically chosen for the domain. Most beginners assume the default will work and wonder why results are poor.

Latency at extreme scale is a fifth. Even with ANN indexes, querying 100 million vectors in under 50 milliseconds is hard. If you need both, the typical answer is tiered storage, with hot data in memory and cold data on disk, plus aggressive quantization. Plan for this if your use case is in that range.

Practical Workflow Pattern

The pattern that holds up in production looks like this.

Keep the source of truth in a regular database or file storage. The vector database is a derived index, not the canonical store. This separation makes reindexing, schema changes, and disaster recovery tractable.

Chunk documents before embedding. Embedding models have context limits, and even when they don’t, smaller chunks give more precise retrieval. Typical chunk sizes range from 200 to 800 tokens with 10 to 20 percent overlap to preserve cross-boundary context. The exact numbers depend on the document type and the queries you expect.

Embed with a model that fits your domain. Generic models are fine for general use. Specialized domains need specialized models. The cost difference is usually small compared to the quality difference.

Store the original text alongside the vector. The metadata should let you trace any result back to the source document, the section, the chunk position, and the embedding model version. Auditability matters more than people expect when someone asks why a particular result was returned.

Plan for reindexing from day one. Embedding models improve, your documents change, and your schema will evolve. A scheduled job that rebuilds the full corpus, typically quarterly, is standard. Make this part of the design rather than a retrofit.

Monitor recall in production. Log queries, the returned results, and any user feedback signals. Feed that data into an evaluation pipeline that measures whether the index is actually returning what users need. Without this loop, you have no way to know if your system is performing.

Separate dev, staging, and production indexes. Use the same schema but different endpoints. Test schema changes and embedding model swaps in staging before promoting them.

Enterprise DNA put together a free field guide on exactly this: the full Claude ecosystem, Claude Code, and how to roll agents out without breaking things. Get the guide.