VOX: voice in, concepts out

VOX is AR Labs' working system and the idea behind it: a voice-first interface to AI agents (language models run in a loop with tools, so they can act, not only reply) in which a person talks and the system answers with structure they can see. Conversation goes in; concepts, maps and the shape of the work come out. The aim is a way of working with AI that respects the limits of attention in people and the limits of context in language models, so that neither side is asked to hold more than it can.

The idea

Most AI tools still ask people to work the way computers prefer: type a precise instruction, read a long block of text, decide, type again. We think conversation will become a main way people work with computers. Speech is how people already express intent, nuance and doubt, and a voice interface lets someone think out loud while an agent listens for what they mean rather than only what they said.

Speech has a weakness as an output channel, though. A spoken answer is linear and fleeting, and a long one overloads working memory, the small store of attention a person uses to hold things in mind while thinking. Text has the same problem at greater length. So the second half of the idea is to answer with pictures of the thinking: mind maps, concept maps, goal trees and structured summaries that a person can look at, navigate and return to. Conversation carries intent. Visualisation carries structure and complexity.

flowchart LR
  P["Person speaks"] --> V["Voice pipeline: speech to text"]
  V --> A["Agents with the person's context"]
  A --> S["Spoken reply"]
  A --> M["Visual map: concepts, goals, work"]
  S --> P
  M --> P

Limits on both sides

The design treats the person and the model as two kinds of mind with different constraints.

A person can attend to only a handful of things at once. Long lists of questions, dense analyses and ten options at a time produce overload, and the overload shows up as poor decisions. The interface therefore chunks what it presents, asks one thing at a time, and puts the rest in a picture rather than in a paragraph.

A language model has a context window, the text it can see when it responds, and that window is also a kind of working memory. Fill it with irrelevant material and the model's attention drifts; in our research, agents followed instructions less reliably when too much was loaded at once. The architecture therefore gives each agent a small, deliberate context and lets it load detail only when the work reaches that detail. This is context engineering, choosing what goes into a model's context window at each step, and it rests on the same cognitive science as our founding framework, cognitive ecology.

Branching conversations

A spoken conversation with an agent wanders. A side question opens, the side question raises another, and soon the original goal is buried. Our design answer is to show the conversation's goal tree on screen and let the person act on it: fork the conversation to explore a side topic, step back up to a clean parent, continue downward from there, and fork again, always seeing where in the hierarchy they are.

Part of this already works in text. When a coding agent hands a subtask to a subagent, the subagent explores in its own context and returns only its result, so the parent conversation stays clean. What the design adds is the visible tree, the ability to step back up, and a lasting record of the branches. It is a requirement we design against, not yet a built feature.

From conversation to an operating layer

Applied to a whole organisation's work, the same idea becomes VOX OS, our design for an operating layer over agent work, in which the dependency tree of all the work is the main surface, voice is the main input, and each node of the tree can be opened as a live agent context that a person can steer, fork or close. We build the presentation and orchestration layers, the parts a person sees and uses to supervise agents, because we expect them to stay hard while much general agent tooling becomes commodity. We rent the rest: agent sessions, the task database, networking and code hosting come from existing tools.

What exists today

The pieces exist separately; the unified interface does not yet.

• Voice. Salesbot, a browser voice agent built on the Pipecat framework, is live and runs a structured spoken conversation with a visible progress rail. VOX Presenter, an early internal prototype, holds a spoken discussion of a document while a second agent writes analysis into a panel on screen, which is the voice-in, concepts-out pattern at small scale.
• Visual maps of work. Vox Canvas draws our task graph in the browser as a navigable tree, and VOX Torch shows the same graph in the terminal.
• The bridge. Connecting a live voice conversation to the full working system, with its goals, projects, memory and tools, has been researched and not built. The research compared three routes: give a voice pipeline a snapshot of context at the start of a call; route each spoken request through a full agent session with live tool access; or pair a fast voice-facing agent with a slower agent that fetches context in the background. It also recorded the constraint that shapes every option: a speech-to-text, language-model and text-to-speech chain adds delay, and conversation feels natural only when replies come quickly.

Why we work this way

We think AI should extend people, not replace them, and that a person and an AI agent working together can do what neither does alone. An interface decides which of those outcomes is likely. One that hands people walls of text pushes them to skim and defer; one that asks a model to hold everything invites it to lose the thread. VOX is our attempt at an interface shaped to both minds.