VOX Memory: research into memory for AI agents

VOX Memory is AR Labs' research into long-term memory for AI agents, language models run in a loop with tools so they can act as well as reply. It is written for engineers weighing memory frameworks for their own agents. Its aim is an associative layer over our own documents, so that an agent asked "what do we know about visual project management?" gets back a connected set of documents, people, tools and decisions rather than a list of keyword matches. The project produced a comparative evaluation of memory frameworks and one real experiment (a spike, a short trial build), which we rejected after measuring its output. The knowledge graph that followed, a database of documents, entities and the links between them that agents would query, is designed but not built. Knowledge-graph memory remains an open research interest for us; the trial is paused, and plain files serve today.

The problem

We keep our knowledge as markdown documents arranged in a hierarchy that loads general context first and detail on demand: each project has an outcome, a strategy and a delivery record. That works well for retrieving a known document. It does not answer questions that cross documents: which research informed a decision, how thinking on a topic changed over months, or which five files in three folders are about the same idea. Keyword search finds documents that share words, not documents that share concepts.

What was tried

flowchart TD
  R["Framework evaluation"]
  H["Spike 1: conversation memory"]
  X["Rejected: mostly noise"]
  M["Pivot: graph over the documents themselves"]
  G["Spike 2 designed: Graphiti"]
  P["Paused"]
  R --> H --> X --> M --> G --> P

Framework evaluation. Across several rounds we compared Graphiti, Mem0, Hindsight, Letta, Mastra's observational memory, Honcho, memory-lancedb-pro, neo4j-agent-memory, TrustGraph and others, at the level of their code where that mattered. The comparison weighed bi-temporal validity (what was true at a given time), typed relationships between entities, keyword and semantic retrieval, and whether an agent could query the store natively over MCP, the Model Context Protocol. One investigation found that a published benchmark claim for another memory project rested on a category error in its metric.

Spike 1: conversation memory. In April 2026 we deployed an always-on agent with Hindsight, which scored well on a public long-memory benchmark. We preprocessed 119 of our own Claude Code session transcripts, reducing 245 MB of raw logs to 4.4 MB of signal, and ingested them. Of 1,195 extracted memories, about 78% were operational noise: narration of what an agent had done, echoes of the agent's own instructions, schedule entries. After aggressive filtering about a fifth survived, and even those were fragments disconnected from the documents they described. Deployment also met seven separate infrastructure failures before it ran.

What we learned. A benchmark score is not fitness for a particular deployment, and a spike should be judged on time to a first useful result. More fundamentally, extracted conversation fragments create a second body of knowledge to maintain alongside the documents, and it goes stale the moment the documents change. The documents are the knowledge. What we lacked was cross-links between the documents we already had.

The pivot: a graph over the documents. The revised design adds a relationship layer on top of the existing files without copying their content. Nodes are documents and the entities they mention; edges are explicit references, the folder hierarchy and semantic similarity, with typed relationships such as "researched", "implements" and "supersedes". Updates would be triggered by changes in git rather than by ingesting conversations, and agents would query the graph from Claude Code through Graphiti's MCP server. The spike plan, with its design document and two architecture decision records (Graphiti as the framework, Neo4j as the database), was written; the infrastructure, ingestion and retrieval milestones were not started.

What exists today

The research corpus, the evaluation and the rejected spike's findings exist. No memory system is running. In our design for VOX OS, an operating layer over agent work, a memory graph sits among the later items: it comes forward once a written post-mortem of the attempt exists and an append-only event log is available to feed it. Related designs held for later include a tier of facts promoted only by a person's approval, which every agent can list and fetch, and lessons an agent files for itself, its department or the whole workspace.

Our agents today rely on the document hierarchy, notes kept on tasks in the task tracker, and small memory files loaded at the start of each session.