What AI Support Bot Actually Is

Strip away the marketing and AI Support Bot is a hosted retrieval-augmented generation layer with two surfaces. The first is a chat widget you drop into a website, the second is a REST API you can call from any backend. Underneath, it is doing four things in sequence when a user asks a question.

The ingestion step takes whatever sources you point it at. PDFs, Notion pages, public URLs, help center articles, raw text, CSV files of past tickets. Each source is split into chunks in the range of 200 to 800 tokens depending on the setting you choose. The chunks are embedded using a model the platform picks, typically a mid-size embedding model that balances cost and recall.

The storage step puts those embeddings into a vector index alongside the raw text and metadata like source URL, title, and last updated timestamp. This index is what gets queried at runtime.

The retrieval step takes the user question, embeds it with the same model, runs a nearest neighbour search, and pulls the top K chunks. The default K in the current version is around 4, which is a reasonable starting point for most knowledge bases.

The generation step feeds those chunks to a large language model with a system prompt you control. The model is told to answer only using the supplied context and to refuse when the context does not contain the answer. The response comes back to the widget or API caller with citation metadata pointing at the source chunks.

That is the whole pipeline. The interesting part is what you can tune, and that is what the rest of this guide covers.

Setup and Authentication

Sign in to the dashboard and create a workspace. Workspaces are isolated, so if you run support for more than one brand you want a workspace per brand rather than trying to mix them. Each workspace gets its own knowledge base, widget config, and API key.

Once the workspace exists, the first real step is grabbing an API key. Go to Settings, then API Keys, and create one. Treat this key the way you would treat any production secret. Put it in your environment manager, not in your front-end bundle. The widget uses a separate publishable key that is safe to expose, but the API key is not.

Next, install the source connector for wherever your knowledge lives. Most teams will use one of three patterns. A Notion or Confluence connector for an internal wiki, a URL crawler for a public help center, or a direct upload for PDFs and CSVs. Pick the connector, paste the URLs or invite the integration, and run an initial sync. The first sync on a typical mid-size help center takes somewhere between 5 and 20 minutes depending on the number of pages.

After the sync finishes, go to the Test panel and ask a few questions you already know the answers to. This is the fastest way to see whether the chunking is working before any customer ever sees the bot.

For the widget, the embed snippet is a single script tag plus a div with a known ID. The publishable key goes into a data attribute on the script. Drop it into your site’s footer template and you are live. For the API, the base URL is in the dashboard and authentication is a bearer token in the Authorization header.

First Working Example

Here is a runnable example that hits the chat completions endpoint with a real request body. It assumes you have an API key in an environment variable and a workspace with at least one source indexed.

The first thing the request needs is the workspace identifier, which is in the URL of the dashboard. The second is the model selector. Most teams use the default model for the first deployment and switch later if cost or quality becomes an issue. The third is the messages array, which is just a system message plus the user question. The system message is what you would write directly in the dashboard, and the API lets you override it per request if you want different behaviour for different surfaces.

A typical request body looks like this. The model field accepts a string identifier. The messages field is an array of objects with role and content. The stream field, when set to true, returns server-sent events so the widget can render tokens as they arrive. The retrieval field is where you can pin the source IDs you want the model to draw from, which is useful when you want to keep an answer scoped to a particular product line.

The response payload contains the assistant text plus a citations array. Each citation has a source ID, the chunk text, and a relevance score. The relevance score is on a 0 to 1 scale and anything below around 0.4 is usually noise. You can use this threshold client-side to decide whether to show citations at all or to add a “Was this helpful” prompt only on the marginal answers.

If you are integrating this into a custom front-end, the typical pattern is to call the API from a thin server route, never directly from the browser, both for key safety and for caching. A 5 minute in-memory cache on the question text is usually enough to absorb duplicate traffic from users who mash the send button.

Key Settings That Matter

The settings most teams leave on default are the ones that cause most of the bad answers. Here are the dials worth understanding.

Chunk size and overlap trade off recall against precision. Larger chunks give the model more context per retrieval hit, which helps with questions that span multiple paragraphs. Smaller chunks give you more granular matches, which helps with very specific questions. Overlap prevents information from being cut in half at a chunk boundary. A good starting point is 500 tokens with 50 token overlap. If you see answers that quote the wrong section, try smaller chunks. If you see answers that miss obvious context, try larger.

Top K is the number of chunks fed to the model. Four is a fine default. Going to 6 or 8 helps when a single question genuinely needs to draw from two distinct documents, but it also burns more tokens per request and can introduce contradictions if the sources disagree. Going to 2 makes the bot more confident but also more brittle.

The system prompt is the single highest-leverage setting. A weak prompt looks like “You are a helpful support assistant.” A strong prompt names the product, names the audience, lists the out-of-scope topics, gives a fallback phrasing, and tells the model how to handle citations. The fallback phrasing matters more than people think. Without it, the model invents its own refusal style, which is often vague. A specific phrase like “I do not have that information. Please contact support at help@example.com” gives you a consistent handoff to a human.

Temperature should usually be at or near 0 for support. You want the same question to produce the same answer, not a creative rewrite. Some teams go as high as 0.2 if they want the bot to rephrase boilerplate answers slightly to match the user’s wording, but anything above 0.3 starts introducing factual drift.

The confidence threshold is a setting that controls when the bot is allowed to answer at all. If the top retrieval score falls below the threshold, the bot returns the fallback instead of attempting an answer. This is your main guardrail against hallucination. Set it too low and the bot guesses. Set it too high and it refuses on questions it could have answered. The right number depends on your content, but 0.55 to 0.65 is a reasonable range to start in.

Citation visibility is a UX dial, not a quality dial. Showing citations builds trust and gives users a way to verify, but it also clutters the chat. Many teams show citations only when the confidence is below a certain bar, so high-confidence answers feel clean and low-confidence answers feel transparent.

Where It Shines

The strongest fit for this kind of tool is high-volume, low-stakes questions that have a clear written answer somewhere in your knowledge base. Think “how do I reset my password”, “what is the refund policy”, “which plan includes SSO”. These questions are tedious for humans, well documented, and benefit from 24/7 coverage.

It also shines in the first 30 to 60 seconds of a support interaction, where most users are looking for a self-serve answer before they are willing to wait for a human. Even a partial answer cuts ticket volume meaningfully for most support teams in the range of 20 to 40 percent of inbound traffic.

Internal IT and HR support is another strong fit because the source material is finite, the questions repeat heavily, and the cost of a wrong answer is usually recoverable. A bot that says “open a ticket” is often more useful than a human who takes 6 hours to say the same thing.

For SaaS products with a public help center, the bot doubles as a search layer. Users who would otherwise bounce off the help center because the search is bad will ask the bot in natural language and get a real answer. This is where the citation back to the help center article becomes valuable, because it teaches users that the help center is actually usable.

Where It Fails

The tool is not a replacement for a human on edge cases. Anything that requires reading between the lines of a customer’s situation, anything emotional, anything that touches billing disputes or compliance, and anything where the user is venting rather than asking should escalate fast. The bot does not have a way to detect tone reliably, so this escalation has to be a rule you build around it, usually by detecting certain keywords or by giving the user an obvious “talk to a human” button.

It also fails badly when the knowledge base itself is bad. If your help center is contradictory, out of date, or scattered across multiple sources that disagree, the bot will surface those contradictions to your customers. Garbage in, garbage out applies harder here than in almost any other AI application, because the model will use whatever context you give it without judgment.

Multi-step troubleshooting is another weak spot. The bot can answer “how do I reset my password” but it struggles with “my internet is down and I have tried X and Y and Z, what next.” The context window is large enough to hold the conversation, but the retrieval is per-turn, so it will keep pulling the same generic troubleshooting chunk instead of advancing through a diagnostic flow. For these cases you want a structured flow engine layered on top, not pure retrieval.

Languages the source material does not cover are also a failure mode. If your help center is English and a user asks in French, the bot will sometimes answer in French using thin translations of the English content, which is worse than refusing. The fix is to either restrict the bot to the languages your content actually supports or invest in translated sources.

Practical Workflow Pattern

The way this usually slots into a real team looks like this. Someone owns the knowledge base, usually a support lead or a technical writer, and they are responsible for keeping the source material current. The bot is a thin layer over that material, not a separate system that someone else maintains.

Ingestion should run on a schedule, not just on demand. A nightly or on-publish sync keeps the bot from going stale. Most connectors support a webhook from your help center platform so the sync triggers immediately on publish, which is the better pattern.

Every week, review the conversations the bot had and look for two patterns. The first is questions it answered confidently and incorrectly, which means your content is missing something or your chunking is splitting it badly. The second is questions it refused that it could have answered, which usually means a chunk is sitting just below your confidence threshold and a small adjustment to either the threshold or the chunking will recover it.

The handoff to a human is the part most teams under-design. A bot that confidently says “I do not know” and then opens a ticket is genuinely useful. A bot that says “I do not know” and leaves the user stranded is not. Build the human handoff as a first-class feature with context carried over, not as an afterthought.

Finally, treat the bot’s analytics as a product feedback loop. The questions it cannot answer are a roadmap for your help center. The questions it gets most often are a roadmap for your UI. Either of these signals is usually more valuable than the support hours the bot itself saves.

If you’re deciding where to start with agents, start here. The free Working With Claude field guide walks through the ecosystem, Claude Code, and a real rollout plan. Get your copy.