Salesbot: a Pipecat voice agent for sales discovery

Salesbot is AR Labs' browser voice agent for sales discovery, built on Pipecat. It holds the short conversation a consultant has with a prospective client before anyone talks about price, and it is for anyone who wants to see how a voice agent can run a structured interview to the end. A visitor presses Connect and speaks or types. The agent asks how they use AI today and where it could help them, reads back what it heard, and closes with a summary built from their answers.

We built it as a working demonstration of structured conversation: the model talks, and code tracks what has been collected and when to move on. An open-ended chatbot has no defined end; Salesbot has to reach one. Like any AI agent, it is a language model run in a loop with tools, so it can act as well as reply; here the main tool records the visitor's answers.

The problem

Every first conversation with a prospective client covers the same ground: what they use today, what gets in their way, and what fixing it would be worth. Someone has to ask those questions, and a web form gets short answers that nobody has thought about. A voice agent could hold that first conversation, but only if it reliably reaches the end: it asks every question, records only what the visitor said, and finishes.

Left to steer a conversation by itself, a language model does not do that. It repeats a question, skips a stage, fills in an answer nobody gave, or never hangs up. Salesbot is built around that weakness: the model talks, and code decides what has been collected and when the conversation moves on.

How it works

The figures use the C4 model, Simon Brown's diagrams that zoom from a system's containers (separately running programs and stores) into their components.

Containers

Salesbot is two programs we wrote, a web app and a voice server, with a SQLite store beside them. Speech recognition, the language model, speech synthesis and email are rented services reached over the network, and the browser and the voice server meet on a call hosted by Daily.

The web app's first request creates a call room and starts a pipeline for it. During the call, the two sides talk only through Daily, over WebRTC, the browser standard for real-time audio; updates for the page use RTVI, Pipecat's message protocol, on the same connection.

Inside the voice server

The voice server is one Python process. Pipecat's runner provides the route that creates rooms; everything else in the figure is code we wrote around the pipeline.

The stage machine is built with Pipecat Flows, an extension that treats a conversation as a graph of nodes, each with its own instructions and tools. One table in code defines every stage and its fields and generates both the model's instructions and its recording tool, so questions and checks cannot drift apart. The turn recorder watches the pipeline without sitting in it, writing a trace row and counting exchanges after each turn. The message guard refuses every browser message except typed replies, and the close sequence ends a call exactly once.

One turn

Each turn of the conversation makes a round trip through a streaming pipeline built on Pipecat, an open-source Python framework for real-time voice agents. A small voice-activity model (Silero) and a turn-ending model (Smart Turn) decide when the visitor has finished speaking. Deepgram transcribes the speech as it arrives, Claude Haiku 4.5 writes the reply word by word, and Cartesia turns those words into audio. Every stage streams into the next, so the agent starts speaking before it has finished deciding what to say, and the visitor can interrupt it at any point.

The conversation design

The conversation is a state machine of four stages, following a simplified form of SPIN selling (Situation, Problem, Implication, Need-payoff), Neil Rackham's method for consultative sales conversations. The first two stages are gates: points the conversation cannot pass until a condition is met, here a list of answers collected.

AI landscape asks what the visitor uses AI for today, which tools they use, and the one use they would miss most. Project discovery asks for one concrete problem, what it costs, how they imagine AI solving it, what would change, and what it would look like day to day. Inside a gate the model has one tool: it can record a finding, in its own short wording, whenever the visitor says something that answers one of the questions. After every recording, ordinary code checks whether the gate's list is complete. If it is not, the agent stays in the stage and is told which items are still missing. When the list is complete, the code moves the conversation on.

The split gives each side the work it does well. A language model is good at conversation and at noticing that something the visitor said answers one of the questions. It is unreliable at keeping track, across a dozen turns, of whether all of a stage's answers are in, and that bookkeeping is exactly what code does well.

The record handler drops blanks, placeholders such as "unknown" and fields the stage does not have. It stores nothing for a stage until the visitor has replied in it, because a late answer to one stage's last question once filled the next. A refusal returns a note telling the model to ask, never an error, since a tool that fails teaches the model to stop calling it. The list of missing fields comes from the function that decides completion, so the two cannot disagree.

Confirmation and close. In Closing the agent reads the answers back and asks whether they are right; if any answer is missing, it instead gives a short honest account that names what went unanswered. When the visitor agrees, the model calls a confirmation tool, its last decision in the call. Wrap-up has no tools: the agent says goodbye, and code ends the call once the goodbye has been spoken. An earlier design asked the model to hang up after its goodbye, and it sometimes left the call open.

Safety limits. A gate still open after 20 exchanges (an agent turn and a visitor turn) moves on anyway, and one background call on the same model recovers answers the visitor already gave there. Closing ends with a fixed farewell after eight exchanges without confirmation. From any stage before Wrap-up, the model can end a call politely if the visitor wants to stop.

A summary built from records. Code assembles the closing summary from the recorded findings rather than asking a model to summarise, so every line traces back to something the visitor said. A model asked to summarise a conversation after the fact tends to smooth it, embellish it and occasionally invent.

Turn-taking and interruption

A turn starts when the voice-activity model hears speech or a finished transcript arrives; provisional transcripts do not count, since they carry no words to answer. A turn ends when Smart Turn judges at a pause that the visitor has finished, with a five-second fallback. That judgement is a trade-off a visitor can hear: when someone trails off mid-thought, waiting is right if they are pausing to think and wrong if they have stopped, and no voice agent gets it right every time.

A voice agent that cannot be interrupted feels like an answering machine, because the visitor has to wait out every sentence. Interruption is a turn start, which cancels the reply and audio in progress. If a turn ends with no words while the visitor's last line is unanswered, a reply guard asks the model to answer it, once.

Typed replies enter where speech enters. A typed line is placed in the pipeline exactly where a finished transcript would arrive, so the agent treats it as speech, interruption included. Because the page is public, the server enforces 500 characters a line, one line a second and 60 lines a call.

Latency as a design budget

People are fast at conversation. Across languages, the most common gap between one speaker finishing and the next starting is about 200 ms, shorter than it takes to plan a sentence, so listeners predict where a sentence will end and prepare their reply while it is still being spoken (Stivers et al., Universals and cultural variation in turn-taking in conversation (PNAS, 2009)). Salesbot does not predict: it starts work once it judges the visitor has finished, so every step below is added after the visitor stops.

The design keeps a latency budget for each turn. The model's figure was measured on recorded replays; the others are typical published figures for each service.

Step	Typical time
Visitor's audio reaches the server	30–200 ms
Deepgram's first transcript	250–300 ms
Claude Haiku 4.5's first words	about 610 ms median
Cartesia's first audio	150–400 ms
Agent's audio reaches the visitor	30–200 ms

Added together, these come to roughly 1.1 to 1.7 seconds from the moment the visitor stops speaking, against a design target of under 1.5 seconds. Most of the spread is network distance between the voice server and the speech and model services. Each turn calls three of those services one after another, so the distance is paid three times, which is why the move to hosting nearer those services was decided.

Nothing slow in the voice path. Tool handlers return at once; database writes and email run as background tasks, because a blocking handler is dead air the visitor hears.

Persistence and logs

The server writes a stage's findings to a SQLite database as soon as the stage is complete, and the full record with the transcript when the agent closes the call, so a conversation the visitor leaves part-way is stored up to its last completed stage. It also saves each call's transcript to a log file every few seconds and once more when the visitor disconnects. A trace file gets one row per turn. How AR Labs demos use data lists what is stored and which outside services see it.

The web page

The page is a small Vite and React app. While the call runs, the page shows a progress rail of the stages, a status window that says in plain words what the agent is finding out now and which answers it already has, a running transcript, and a box for typed replies. The closing screen adds a link to request a meeting, planned features to tick and an email form; the server records one follow-up per call and emails the lab. The page's rules are plain functions outside React, which only draws, and contract tests keep its stage list and limits identical to the server's.

Decisions

The reasoning is recorded in 22 architecture decision records, each stating one decision, its context and its consequences. The main ones:

Decision	Why	Rejected
Build from zero, one version at a time	Every part must be explainable by its owner	Adapting a much larger earlier codebase
Model inside the pipeline	Tools are single actions; Pipecat runs the tool loop	A separate model service; an agent SDK
Pipecat	Its processors fit streaming audio; built-in Daily support	LiveKit Agents, Vocode, hand-written asyncio
Daily, with RTVI for page messages	Rooms by API; page updates reuse the call	LiveKit, Twilio, Agora; a second WebSocket
Deepgram and Cartesia	Streaming, with first text in about 300 ms and first audio in about 150 ms	Whisper, AssemblyAI; ElevenLabs
Pipecat Flows	Code owns the state; each stage sees only its own prompt and tool	Hand-built gate logic, LangGraph, prompting alone
The model records, code decides	The stage change stays in Pipecat's tool-call path and never interrupts a turn	One all-fields call; a second model judging completion
Claude Haiku 4.5	Asked for missing points on replays; first words in a median 611 ms against 1,101 ms	GPT-4o-mini, which recorded answers nobody gave; GPT-4.1
SQLite, written in the background	Nothing to run; enough for one writer per call	Airtable, hosted Postgres, JSON files
Email typed into a form, sent by Resend	Spoken email addresses transcribe badly	Collecting the address by voice; AWS SES

The completion decision mattered most. Early versions asked the language model to judge when a stage was finished, and conversations looped, skipped ahead or ended with questions unasked. A later design used a second model running beside the conversation to judge completion, and in side-by-side tests it moved the conversation on over the visitor and inferred answers nobody gave. It survives only as a comparison setting.

Deploy checks for a small service. One deploy script refuses an uncommitted tree or a call in progress, checks that development tools are left out of the public page, deploys, tests the running service, and restores the previous version if any check fails.

What exists today

The second version of Salesbot is live at salesbot.arlabs.tech. It runs on a single rented server and works in a desktop or mobile browser, with a microphone or with typed replies. A completed call ends on a closing screen with the summary and a form where the visitor can leave an email address to ask for follow-up.

Automated tests and scored replays cover the conversation layer. In a controlled typed session against the live demo on 2 November 2026, a call completed both gates, asked for a missing answer, read back, closed after confirmation and stored its record. People have listened to calls, but speech recognition, interruption, latency as heard and voice quality have no recorded, repeatable test yet; testing a browser voice agent gives the detail.

Salesbot is a browser demonstration of structured voice conversation. It is not a telephone agent: it has no phone numbers, call transfer, CRM or calendar integration.

Not built yet

Some things are planned and not built: emailing visitors their own summary, a deeper conversation with the full four SPIN stages, recognising a returning visitor, booking a call directly, and answering questions from a knowledge base. We have also decided to move the voice server to managed Pipecat hosting closer to the speech and model services, which would cut the pauses between turns; that move has not happened yet.

Two limits are known. Closing has no recording tool, so a correction during the read-back does not change the stored answers or the summary. And because gates couple talking to recording, the agent cannot easily follow a need raised out of order.

Next design

The design documents record where the conversation would go next, as design thinking rather than a schedule:

• The full SPIN sequence. Project discovery would split into Problem, Implication and Need-payoff gates. Stages are rows in one table, so this changes data and prompts, not the handler.
• Gates that observe rather than drive. The conversation would follow the visitor's needs while capture runs alongside; structured conversation covers the research.
• Sharper criteria. Examples of sufficient answers would replace general instructions.
• Checked quotes. Code could require each stored answer to match the visitor's words.

How it was built

The current Salesbot replaced a much larger earlier implementation and was built from zero one version at a time: a bare voice loop, then a prompt-only two-stage conversation, then the gated structure described here. The rule was that every part must be explainable by the person who owns it. Each change went through our coding pipeline, the route every code change takes: design, a written spec (what to build and how each criterion is proved), test-first implementation, independent review by a model from a different vendor, and a person's merge. The review comes from a different model family because the model that made a mistake is the least likely to spot it, and a change counts as finished only with evidence a reader can check: tests, a written walkthrough of why each change was made, and a hands-on run of the feature.

The idea underneath, a conversation in stages whose code decides when each stage is done, goes back to the core loop of Coherence, our earlier voice coaching app, and the essay managing two-way conversations inside a core loop that set out its reasoning.

Related projects

Voice UI is the generic browser voice interface taken from the first Salesbot. VOX Presenter uses the same browser and Pipecat pattern to discuss documents aloud, and Discovery Copilot applies the same SPIN checklist to a live call between two people rather than to a conversation with an agent.