VOX Facilitator

VOX Facilitator is AR Labs' design for a voice agent that runs a working session over a folder of documents, for a person who needs to turn a pile of project material into decisions. Like any AI agent, it would be a language model run in a loop with tools, so it can act as well as reply; here it is one you speak with. Point it at a project folder and it reads everything inside, works out what the project is trying to achieve and where it stands, and then holds a spoken conversation that moves from shared understanding to decisions to concrete next actions. While it talks, a browser panel shows the document, or the paragraph, that supports what is being said. A session produces decisions written into the project's own documents, a plan both parties can see and edit, and tasks ready for our coding pipeline, the route every code change takes: spec, test-first build, second-model review and a person's merge.

It exists to counter a common failure in knowledge work: analysis piling up faster than the questions that would settle it get asked. The Facilitator is biased the other way. It prefers asking an unanswered question or settling a decision to producing another summary.

This is a design. Nothing described here is built as a Facilitator yet, though most of the pieces it relies on exist in our other projects, above all VOX Presenter, our voice sessions about single documents.

How it would work

The design has five parts. Three of them extend things we already have.

flowchart LR
  DIR["Project folder"] --> CA["Cartographer"]
  CA -->|"map and agenda"| BR["Facilitator brain"]
  BR <-->|"conversation"| VL["Voice loop"]
  VL <-->|"show, scroll, highlight"| EP["Evidence panel"]
  BR -->|"dispatch"| WK["Research and task workers"]
  BR -->|"decisions and plan"| DIR

• Cartographer. A headless agent run, using the Claude Agent SDK, that fans out reader agents over the folder. It records each document's role, the stated goal, the current stage, open questions and blockers, with an index pointing to the evidence for each. It builds a layered summary, from the whole folder down to section and paragraph, so the session can fetch detail without stalling the voice.
• Facilitator brain. A slower, more capable model that owns the method: it drafts the agenda, runs the session's phases, decides what to show and what to ask, and writes the results. It is kept out of the audio path.
• Voice loop. The existing VOX Presenter backend: speech-to-text, a fast conversational model, and text-to-speech, with interruption built in.
• Evidence panel. The VOX Presenter browser page grown into a workspace with a document view, a live plan, progress cards for running research, and later a view of the task graph (all tracked tasks and the dependencies between them). The voice points at content on screen rather than reading it out.
• Workers. Research agents and task-creation steps the brain can start mid-conversation; a progress card appears and the conversation moves on until the answer is ready.

A session runs through six phases, each with an exit test:

stateDiagram-v2
  state "Orient (agree the picture)" as O
  state "Frame (agree the session goal)" as F
  state "Elicit (one question at a time)" as E
  state "Structure (build the plan)" as S
  state "Commit (write decisions and tasks)" as C
  state "Execute (track the work)" as X
  [*] --> O
  O --> F
  F --> E
  E --> S
  S --> C
  C --> X
  X --> [*]

In the elicit phase the Facilitator asks one question at a time and states its recommended answer, so the person reacts to a proposal rather than starting from a blank. In the structure phase it proposes a plan branch by branch, the person objects to specific parts, and it revises. Every resolved item is saved to a small state file in the folder, so a session can stop mid-agenda and resume later.

What exists today

The design document and two research reports exist: one comparing agent orchestration frameworks, and one on prior art for planning assistants, structured planning methods, citation interfaces and voice-first failure modes. The supporting pieces exist elsewhere in our work: the voice loop and two-agent pattern in VOX Presenter, the one-question-at-a-time interview method, research agents, and the coding pipeline that would execute the tasks a session produces.

The design is a proposal, not an active build. The proposed first slice tests the method in plain text, with no voice, before any real-time work.

Decisions

Voice carries momentum; the screen carries evidence. Research on voice interfaces is consistent that pure voice fails for dense information work, because speech is fleeting and cannot be skimmed. So the voice asks, proposes and confirms, and anything longer than a sentence goes on screen with a highlight.

A small state machine instead of a graph framework. LangGraph offers the strongest durable-state model of the frameworks surveyed, but adopting it would add a second orchestration framework to manage six states. The session state lives in a file the brain owns, designed so a later move to a graph framework stays possible.

Stay on Pipecat and RTVI. The voice layer already speaks RTVI, the real-time voice protocol used by Pipecat, and its interface commands for navigating, scrolling and highlighting cover what the panel needs. Adopting a second agent-to-interface protocol would add surface for no gain.

A cascaded voice pipeline instead of a speech-to-speech model. Separate speech-to-text, language model and text-to-speech stages keep the reasoning on our chosen model and already reach conversational speed in VOX Presenter.

Prove the method first. The riskiest part is whether the conversation feels like facilitation rather than a form. The build plan tests that cheaply, in text, before adding voice and screen.

Related projects

Discovery Copilot covers the other side of the same problem: it assists a person who is running a planned conversation with someone else, rather than running the session itself.