Structured conversation for voice agents

Structured conversation is AR Labs' pattern for an AI conversation that has to get somewhere: the model talks, and code tracks what is collected and when to move on. The language model handles the talking: phrasing questions, following the person, responding naturally. Code handles the progress: which stage the conversation is in, what information each stage needs, what has been collected so far, and when the conversation may move on or end. The split gives a voice agent or chatbot the flexibility of a language model and the reliability of a program.

The problem it solves

A conversation driven only by a prompt is easy to build and hard to trust. The model may skip a question, invent an answer the person never gave, decide a stage is finished when it is not, or never end the call. Each failure is small, but in a sales discovery call, an intake interview or a support flow, the purpose is to come away with specific information, and a model left to judge its own progress will sometimes judge wrongly.

The opposite extreme, a fixed script or a form read aloud, is reliable but feels like a phone menu. Structured conversation sits between them: the model may say whatever serves the conversation, but it cannot change what counts as done.

How it works

The conversation is divided into gates. A gate is a point the conversation cannot pass until a condition is met; here, a stage with a list of named fields it must fill first. The fields are defined once in code, and that single definition generates both the instructions the model sees for the stage and the schema of the tool it uses to record answers, so the questions and the validation cannot drift apart.

stateDiagram-v2
  state "Gate 1, situation" as Gate1
  state "Gate 2, problem and vision" as Gate2
  state "Read back and confirm" as Closing
  state "Goodbye, code ends the call" as WrapUp
  [*] --> Gate1 : connect
  Gate1 --> Gate1 : record findings
  Gate1 --> Gate2 : code finds all fields valid
  Gate2 --> Gate2 : record findings
  Gate2 --> Closing : code finds all fields valid
  Closing --> WrapUp : person confirms
  WrapUp --> [*]

Within a gate, the cycle runs like this:

1. The model converses and, as answers emerge, calls a recording tool with whatever it has learned, even if only one field.
2. Code validates each value, rejecting blanks, placeholders and unknown fields, and merges the valid ones into the conversation's state.
3. A deterministic check, plain code with no model involved, decides whether every required field is now filled.
4. Only when it is does the handler move the conversation to the next gate, which brings new instructions and a new recording tool.

The state diagram above shows the order of the gates across a whole call. The figure looks inside one gate: its single field list feeding both the model's instructions and the recording tool's schema, and below it the record, validate and check loop that opens the next gate.

Two safeguards keep the conversation moving. If a gate is still unfinished after a fixed number of exchanges, code moves the conversation on with a prepared transition. And the end of the call is triggered by code when the final stage is entered, rather than by asking the model to remember to hang up, which proved unreliable as its last act.

Because progress is held in code, it can also be shown. The person sees a rail of stages that advances when a gate actually completes, a panel of what the agent is currently finding out, and a one-line statement of why it is asking what it asks.

Where it comes from

The two gates in our implementation follow SPIN Selling, Neil Rackham's 1988 model of consultative sales questioning: Situation questions establish facts, Problem questions surface a difficulty, Implication questions explore what it costs, and Need-payoff questions let the buyer state the value of solving it.

The engineering draws on decades of task-oriented dialogue research, which we reviewed in building the pattern. Early systems were finite-state machines or slot-filling forms. Later frameworks addressed their known limits. CMU's RavenClaw (Bohus and Rudnicky) separates a task tree, the plan, from a runtime stack that tracks where the conversation is, so that a side topic can be explored and then returned from. The Information State Update approach, built in the TrindiKit toolkit, keeps a structured record of what is known and pending as a first-class object, updated by rules separate from the conversation itself.

How we apply it

Salesbot, our live browser voice agent, is built on this pattern using Pipecat and its pipecat-flows library. It runs a short discovery conversation about how a visitor uses AI and where it could help, reads back what it heard, and closes with a summary built from the recorded answers.

The design went through two versions of who decides completion. In the first, the model submitted every field at once in a single call. In the second, a separate evaluator model judged completion in the background. Live comparison showed the background evaluator could force the conversation forward at the wrong moment and could infer fields the visitor never stated. The current design, where the model records findings as they emerge and code decides completion, proved smoother.

What it does not yet do

Gates that couple talking to data capture have a limit we have observed in testing: the agent cannot easily follow a need the visitor volunteers out of order, return to an earlier topic, or decide which question matters most given what was just said. The dialogue research points to a next design: hold the stages as a goal tree the model can navigate freely, use a stack for side discussions, and run extraction of structured answers as a separate background process that watches the transcript. That design is research and has not been built.

Related

Structured conversation is a small-scale instance of the harness paradox: the model is more useful because code holds the decisions that must not drift. Our Discovery Copilot applies a related idea to live human calls.