Enterprise DNA
Guide Intermediate General

DSPy Tutorial for Building and Optimizing LLM Programs

Learn how to set up DSPy, define signatures and modules, evaluate a language model program, and optimize it with a practical example.

Sam McKay |
DSPy Tutorial for Building and Optimizing LLM Programs

DSPy is a Python framework for building language model programs from defined inputs, outputs, and tasks. Instead of hand-writing and repeatedly tweaking a long prompt, you define what a program should do, test its results, then use an optimizer to improve how it handles the task. This DSPy tutorial walks through setup, signatures, modules, evaluation, and a small ticket-classification program.

For a business, the practical value is a workflow you can inspect and measure. You can change the task definition, model, or examples, then compare the results against a held-out test set. DSPy doesn’t guarantee accurate answers, but it gives you a repeatable way to develop and assess a language model feature before putting it in front of staff or customers.

If you’re looking for a broader overview, our DSPy guide takes a different angle. Here, we’ll build and evaluate a small program step by step.

What you’ll build

The example classifies incoming support tickets into a small set of categories:

  • billing
  • technical
  • account

This is a useful first project because the desired output is clear and easy to check. You can apply the same pattern to tasks such as routing sales enquiries, extracting fields from forms, or assigning internal requests to a team.

You’ll need Python, a DSPy-compatible language model provider, and an API key for that provider. The code uses a generic model identifier. Replace it with the identifier and environment variable required by your provider. Don’t put a live key directly into a notebook or source file.

1. Install DSPy and configure a model

Install the package in your project environment:

pip install dspy

Set your provider’s API key as an environment variable, then configure DSPy:

import os
import dspy

lm = dspy.LM(
    "provider/model-name",
    api_key=os.environ["PROVIDER_API_KEY"],
    temperature=0.0,
)

dspy.configure(lm=lm)

Replace provider/model-name with a model identifier supported by your provider and DSPy. PROVIDER_API_KEY is a placeholder, too. Use the actual environment variable name for your account.

A temperature of 0.0 is a sensible starting point for a category-labeling task because you want consistent, constrained outputs rather than creative variations. It doesn’t make answers guaranteed or deterministic across every provider and setup. Check the provider’s documentation for the settings it supports.

Keep credentials outside your code, and make sure the environment used for evaluation is configured the same way as the environment you intend to deploy. If you’re new to connecting business workflows to models, the Enterprise DNA resources library has more material to help you plan the implementation.

2. Define a signature

A DSPy signature describes the inputs and outputs of a task. It provides the structure the program should follow, without requiring you to write a complete prompt yourself.

class ClassifyTicket(dspy.Signature):
    """Assign a support ticket to one of the allowed categories."""

    ticket = dspy.InputField()
    label = dspy.OutputField(
        desc="Choose exactly one: billing, technical, or account"
    )

The input field is the ticket text. The output field is the category label. The description gives the model a clear constraint, which matters here because your evaluation expects one of three exact values.

A signature isn’t a substitute for testing. If the categories overlap, or the description leaves important business rules out, the program can still produce unsuitable classifications. Define the categories in terms your team agrees on. For example, decide whether a refund request belongs under billing even when the customer mentions a login problem.

3. Create a module

A module uses a signature to perform a task. Start with dspy.Predict, which asks the model to produce the output described by the signature.

class TicketRouter(dspy.Module):
    def __init__(self):
        super().__init__()
        self.classifier = dspy.Predict(ClassifyTicket)

    def forward(self, ticket):
        return self.classifier(ticket=ticket)

program = TicketRouter()

The module wraps the predictor in a regular Python class. That makes it easier to expand the program later, for example by adding a separate step to extract a customer ID or draft a response. For a simple classification task, one predictor keeps the logic easy to inspect.

DSPy also includes modules such as dspy.ChainOfThought. That module asks a model to reason through a task before producing its answer. It can help with tasks that require several steps, but it isn’t automatically better for a straightforward classification problem. Start with the simplest module that fits the work, then compare alternatives using the same evaluation set.

4. Prepare examples and an evaluation metric

DSPy works with structured examples. Each example needs the task input and the expected output. Mark the input fields with .with_inputs() so DSPy knows what to pass into the program.

trainset = [
    dspy.Example(
        ticket="I was charged twice for my monthly subscription.",
        label="billing",
    ).with_inputs("ticket"),
    dspy.Example(
        ticket="The dashboard returns an error whenever I open a report.",
        label="technical",
    ).with_inputs("ticket"),
    dspy.Example(
        ticket="Please change the email address on my account.",
        label="account",
    ).with_inputs("ticket"),
    dspy.Example(
        ticket="I need a copy of last month's invoice.",
        label="billing",
    ).with_inputs("ticket"),
]

devset = [
    dspy.Example(
        ticket="I can't reset my password because the link has expired.",
        label="account",
    ).with_inputs("ticket"),
    dspy.Example(
        ticket="The export stops before it finishes downloading.",
        label="technical",
    ).with_inputs("ticket"),
]

These small examples are enough to show the workflow, not to establish production accuracy. For a real rollout, gather representative cases from the actual process and have someone who understands the business label them. Include borderline cases, such as a billing question that also mentions an account change.

Now define a metric. This one checks whether the predicted label matches the expected label after trimming whitespace and converting to lowercase.

def exact_label_match(example, prediction, trace=None):
    expected = example.label.strip().lower()
    actual = prediction.label.strip().lower()
    return actual == expected

Exact matching is appropriate when the output must be one of a fixed set of categories. For generated text, you may need a different metric, such as a structured check for required fields or a human review of answer quality. Choose a metric that reflects the cost of a wrong result, not just one that is convenient to calculate.

5. Measure the baseline

Evaluate the unoptimized program before changing it. This gives you a reference point for deciding whether an optimization helped.

evaluator = dspy.Evaluate(
    devset=devset,
    metric=exact_label_match,
    num_threads=1,
)

baseline_score = evaluator(program)
print(baseline_score)

Inspect individual predictions as well as the score. A single aggregate value can hide a serious problem, such as consistently confusing account and billing. You can run the program on a development example directly:

result = program(
    ticket="I can't reset my password because the link has expired."
)

print(result.label)

For business use, keep a record of the test examples, expected labels, model configuration, and evaluation results. That creates a simple comparison point when someone changes a category definition, the training examples, or the model.

6. Optimize the program

DSPy optimizers can use labeled examples and program outputs to adjust how a program is prompted. For this example, use BootstrapFewShot to compile a version of the program from the training data:

optimizer = dspy.BootstrapFewShot(
    metric=exact_label_match,
    max_bootstrapped_demos=4,
    max_labeled_demos=4,
)

optimized_program = optimizer.compile(
    program,
    trainset=trainset,
)

Then evaluate the compiled program against the same development set:

optimized_score = evaluator(optimized_program)
print(optimized_score)

The optimizer can select or produce examples to guide the program. This is not the same as changing the underlying model’s weights. It also doesn’t promise that the score will improve. With a tiny training set, results can be unstable or fail to represent the real mix of tickets.

Keep the development set separate from the examples used to compile the program. If you repeatedly tune against the same cases, you’ll gradually build a solution that fits those cases rather than one that generalizes. Once you’ve settled on an approach, assess it on a separate test set that you haven’t used during development.

For a larger project, DSPy has other optimization approaches. Pick one based on the quality and size of your examples, your model access, and the time you can spend running evaluations. The key point is to compare optimizers on the same task and data split, not to assume one is best for every workflow.

7. Review and prepare for deployment

Before using this program in a live business process, check more than whether the code runs:

  • Confirm category rules. Write down what each label means and how to handle ambiguous tickets.
  • Test difficult examples. Include typos, short messages, mixed requests, and cases that don’t fit any category.
  • Decide what happens on invalid output. A result outside the allowed labels should be flagged or sent for human review, not silently routed.
  • Measure the cost of mistakes. A misrouted low-priority request may be manageable. A missed complaint or account-security issue may need a stricter review process.
  • Keep a human fallback. Start with suggestions for staff to approve before automating a consequential action.
  • Re-evaluate after changes. A new category definition or model configuration can change behavior, so rerun your test set.

If your team wants to map this kind of workflow into a wider operating process, browse our business insights and operations resources for related guidance. The DSPy examples and evaluation approach can help you decide what to validate before connecting a model program to other systems.

Common DSPy mistakes

Treating a signature as a guarantee. A signature describes the task, but the model can still return an invalid or incorrect result. Validate outputs in your application and provide a fallback.

Using too few or too similar examples. A handful of examples demonstrates the mechanics, but it won’t represent every real ticket. Build a dataset with the variety your team actually sees.

Evaluating on the training data. A good result on examples used for optimization doesn’t tell you how the program will handle unseen cases. Keep development and test data separate.

Changing several things at once. If you change the signature, examples, model, and metric together, you won’t know which change affected the result. Make controlled changes and rerun the same evaluation.

Assuming optimization means fine-tuning. DSPy optimizers can improve how a program uses prompts and examples. They don’t necessarily update the model’s internal weights.

DSPy is most useful when you can describe a task clearly, collect examples of acceptable outputs, and measure whether the results meet your requirements. Start with one narrow workflow, establish a baseline, and only automate more after the tests and review process support it. If you’d like to discuss where this approach fits in your business, book a call with Sam.