What Dify actually is

Dify is an open-source platform for building applications on top of large language models. It packages together the parts you would otherwise assemble yourself: a prompt IDE, an orchestration layer for chaining model calls, retrieval augmented generation with vector storage, agent frameworks with tool calling, logging, monitoring, and a deployment layer that exposes your work as an API endpoint or a chat interface.

Under the hood, Dify is a Python web application, Flask on the backend and React on the frontend, that talks to model providers through a normalized interface. You can self-host it via Docker, or you can pay for the managed cloud. It does not train models. It does not replace your model provider. What it does is remove the repetitive plumbing work of getting from “I have a prompt” to “I have a deployed, observable, version-controlled LLM application.”

The cleanest way to think about it is as a Backend-as-a-Service for LLM apps, with an LLMOps layer glued on top. Those two concerns are usually sold separately in the wider ecosystem. Dify bundles them into one product, which is its main value proposition and also the source of some of its design compromises.

Setup and authentication

You have two paths: cloud or self-hosted. Cloud is faster to evaluate. Self-hosted is what production teams usually end up with for data residency reasons.

For the cloud, sign up at dify.ai, create a workspace, then go to Settings, then Model Providers, and add credentials for whichever model vendor you want to use. OpenAI, Anthropic, Azure OpenAI, Google, Mistral, and a long tail of others are supported through a plugin system. You paste an API key, name the connection, and it becomes available across every app in your workspace.

For self-hosting, the standard approach is Docker Compose. Clone the public repository, set the environment variables in a .env file (database URLs, secret keys, storage backends, vector database settings), and run docker compose up -d. On a 4-core, 8GB machine the stack starts in a few minutes. After it boots, you reach the UI on port 80 or 443 depending on how you fronted it with a reverse proxy. The bundled services include PostgreSQL, Redis, and a vector store, all of which you can swap for external instances once you move to production.

Authentication inside Dify itself is by user account. Each workspace member has a role: owner, admin, editor, or viewer. For the model providers, you authenticate Dify to the provider using provider-specific keys, and Dify passes those through on every call. You do not need OAuth for most providers, just a plain API key. If you self-host, you can also wire Dify into an external identity provider through OIDC for SSO.

First working example

Let us build a working chat application that answers questions about a PDF you upload. This is the canonical Dify demo and for good reason, it exercises the prompt layer, the knowledge base, and the chat front end in one application.

In the studio, click Create Application. Pick Chatbot. Give it a name. Pick a model from the dropdown. The dropdown only shows models for which you have configured credentials, so if it is empty, go back to Model Providers and add one. Pick the cheapest reasonable model for your first build. You can swap models later without rewriting anything.

Now click into Knowledge. Add a new knowledge base. Upload a PDF or a folder of text files. Dify will chunk the document, generate embeddings using whichever embedding model you have configured, and store the vectors in its built-in vector database. Qdrant is the default, though you can swap this out for Weaviate, Milvus, or pgvector. For most documents the default chunking strategy is fine. You can tune chunk size and overlap in the dataset settings, but for a first pass, leave them alone.

Back in the orchestration canvas, your application already has a system prompt, a context variable, and a chat history variable wired up. Open Variables and make sure the context variable is set to point at your knowledge base. Save. Click Publish, then Preview.

Type a question about your PDF. You should get a grounded answer with citations pointing to specific chunks. If you do not, the most common reasons are that the context variable is not connected to the knowledge base, the retrieval is returning empty results because the query is too narrow, or the embedding model is not configured.

That is your first deployed LLM application, with retrieval, an API endpoint, and a chat UI. No code was written. The application is now reachable as a REST endpoint and as an embeddable iframe, both configured under the application’s Publish settings.

Key settings that matter

Most people leave these at defaults and pay for it later in cost or quality.

Temperature is set per application in the model parameters panel, not per workspace. A customer support bot should run at 0 to 0.2. A creative writing assistant can run at 0.7 to 0.9. The default of 0.7 is a creative-writing default applied to everything, which is rarely what production needs.

Max tokens bounds the response length. Without a cap, a model can chew through thousands of output tokens on a single question. For chat applications, 500 to 1000 is a typical range. Set this based on your cost ceiling per request, not on what the model can technically output.

Top-p and frequency penalty are best left at defaults unless you have a specific failure mode. They interact in non-obvious ways and are over-tuned by beginners.

Retrieval settings are where the real quality work happens. Under the knowledge base configuration, the top-k value (number of chunks retrieved) is the single biggest lever. Values of 3 to 5 work for most document Q&A. Going higher adds noise without improving recall. The similarity threshold matters more for hybrid search setups. The rerank model option is worth turning on if you have a rerank provider configured. It noticeably improves answer quality on messy documents because it re-scores the initial retrieval results with a more expensive model.

System prompt is the place where most quality work happens. Be explicit about what the model should and should not do, what format to return, and what to do when it does not know. Two to three sentences is the sweet spot. Longer system prompts often degrade performance because they compete with the retrieved context for attention budget.

Conversation opener and suggested questions are small touches that change whether users actually engage with your application. Set them.

Where it shines

Dify genuinely excels at three categories of work.

Internal tools and prototypes. If you need to give a non-technical stakeholder a working chat-with-your-data app by Friday, Dify is the fastest path that does not involve buying a SaaS subscription you cannot customize. The visual builder means they can iterate on the prompt themselves without filing a ticket.

RAG-heavy applications. The knowledge base layer is mature. It handles ingestion, chunking, embedding, retrieval, and citation. It supports multiple file formats and connects to external sources through plugins. For document Q&A over a few hundred to a few thousand documents, it is production-ready out of the box.

Multi-model orchestration where you want a router or fallback logic. Dify supports workflows that branch on conditions, call multiple models, and merge results. For example, route easy questions to a cheap model and hard questions to a strong one, with classification done by a third model. This kind of logic in raw code is hours of work. In Dify it is a drag-and-drop canvas.

Where it fails

Honest list. Dify is not the right tool in several common situations.

Highly customized UX. The built-in chat UI is functional but generic. If your application needs a tailored interface, custom components, branded design, or embedded interactions, you will end up fighting the front end. The API is solid, so the common workaround is to use Dify as the backend and build your own UI. At that point, you are paying for the visual builder for your own engineers and getting no value from it.

Complex agentic workflows with many tools. Dify supports tool calling and agents, but when the workflow has more than ten steps with branching, parallel execution, and conditional retries, the visual canvas becomes hard to follow. At that scale, code in a framework like LangGraph beats a visual tool every time on both clarity and debuggability.

Strict latency requirements. Dify adds orchestration overhead on top of the model call. For typical RAG apps this is a few hundred milliseconds, which is fine. For real-time voice or sub-200ms applications, that overhead is meaningful.

Data residency and compliance. Self-hosting works, but you own the entire stack including security patching, backups, and version upgrades. For organizations in regulated industries, the operational burden is real and the managed cloud tier may not meet the residency requirement.

Heavy multi-tenancy. Dify’s permission model is workspace-based, not application-based. Building a SaaS product on top of Dify where each customer needs isolated data, billing, and branding is awkward and not the design goal.

Practical workflow pattern

Here is how teams that get value from Dify actually use it.

A small team uses the cloud tier for prototyping and evaluation. They build the prompt, the retrieval, and the workflow in Dify because iteration speed is the priority. They connect a logging destination and watch real conversations. They evaluate the answers against a set of golden examples they curate themselves, typically a few dozen questions with expected answers. The Logs and Annotations tabs in the studio are the workflow’s heart, not the canvas.

Once the application is working and they understand the failure modes, they decide what to do next. If the production app is a simple chatbot, they keep using Dify’s hosted chat UI or call the Dify API from a thin custom front end. If the production app needs a custom UI or tighter integration, they export the orchestration as an API call from their own backend, leaving Dify as the orchestration engine. The response includes token counts, latency, and the retrieved chunks, which makes it easy to monitor in your own observability stack.

For data-sensitive deployments, the team migrates the same Dify setup to self-hosted, often on Kubernetes in their own VPC. The same applications, the same prompts, the same knowledge bases, just running on their own infrastructure. The migration is a few hours of work because the configuration is portable as long as you exported datasets and prompts.

The mistake to avoid is treating Dify as a final product. It is a development environment and a deployment target. It is not your application. Your application is the chat surface your users see, the data store you own, and the business logic you maintain. Dify is the part in the middle that handles the LLM orchestration. Keeping that mental model clean is what keeps teams from getting locked in.

When you are ready to extend, the workflow APIs (called Workflow in the app type) let you wrap a multi-step orchestration as a single callable endpoint. You can trigger it from a webhook, call it from your own code, or chain multiple workflows together. This is the integration point that makes Dify usable as a building block rather than a closed system. Pair it with version control on your prompts and a small evaluation harness, and you have a credible foundation for shipping LLM features into a real product.

Enterprise DNA put together a free field guide on exactly this: the full Claude ecosystem, Claude Code, and how to roll agents out without breaking things. Get the guide.