AI agent memory for changing facts: append-only records versus memory systems

What form of memory lets an AI agent recall both which facts currently hold and how they came to change? AR Labs researched that question in October 2026 for the marketing manager agent in our multi-agent system, which has to track company goals, decisions its owner has accepted, and its own bets. The findings sit beside our wider work on agent memory. The report reviews published papers and benchmarks on agent memory for changing facts (LongMemEval, LoCoMo, HaluMem, Zep, Mem0, Hindsight), reported experience with memory providers on the Hermes agent harness, and judges a proposed append-only records model against both.

The agent in question is an AI marketing-manager agent running on Nous Research's Hermes agent harness. A harness is everything around the model that shapes what an agent does: the agent program, its instructions, skills, hooks, tools and checks. It needs to recall three kinds of thing: external facts that change (company goals and direction), decisions its owner has accepted, and its own bets. Recommendations here are inputs to the design, not decisions.

The model under review

The design proposed that the agent keep its own representation of company facts, with their history, rather than relying only on an external company repository. The proposal had not been accepted when this research was done.

• The company goal record stays a stable identity and pointer with no content of its own.
• Each time the agent reads the goal, it appends a reading to its evidence store, labelled user-provided. A reading holds the exact wording, the measure, the deadline, its sources (the OKR file at a named commit, a message by file name), the date read, and the agent's interpretation in its own words, kept apart from the quote.
• When the goal changes in the world, a new reading names the one it replaces. Readings are append-only, so the chain is the agent's history of the goal. The design already used this pattern for decisions.
• A reading that was recorded wrongly, such as a misquote, is marked superseded_by its correction. The model thus separates "changed in the world" from "recorded wrongly".
• A driver (the agent's bet that a marketing goal moves a company goal) cites the specific reading it bet against. A driver whose cited reading has been replaced is stale and goes to the owner for review.

Records live in one YAML file, the register, read directly by the agent with no search index.

Two terms recur below. Valid time is when a fact held in the world. Recorded time (also called transaction time) is when the system learned or recorded it. A chain head is the newest record in a replaces chain, the one that holds now.

Finding

The proposed model is the form the evidence favours at this agent's scale. It needs six small additions, none of which is a database, a graph or a search index. Our own earlier experiments with a temporal knowledge graph (a fact database an agent queries) and an extraction-based memory service pointed the same way: the hard part in each was keeping a model-extracted copy aligned with its authoritative source.

1. Published results favour curated, time-stamped, structured records read in full over model-extracted memory. On the LongMemEval benchmark, a model given only the relevant sessions scored 87 per cent against 61 per cent for the full history and below most memory systems' results. Below a few tens of thousands of tokens, reading everything beats memory systems on accuracy (Mem0's own paper: 72.9 against at best 68.4 per cent). Extraction-based systems mostly fail by omitting facts and silently skipping updates, and they keep no quote against which an error could be told apart from a real change.
2. No Hermes memory setup in use replaces the register. No Hermes deployment has published a long-running account of its memory working. The 73 memory providers in the plugin catalogue are all community tier, with reported problems that are about installation and identity, not retrieval quality. The only reported practice resembling this model is a user's hand-built chain of dated, append-only documents beside a small pointer in MEMORY.md. Extraction-based providers would at most add fuzzy search on top of the register, and their mirroring of Hermes's built-in memory would create a second, paraphrased copy.

The gaps and their smallest fixes:

Gap	What goes wrong without it	Smallest addition
Only recorded time	A goal amended on 13 October but read on 20 October looks as if it changed on the 20th; "what was the goal on date D" cannot be answered	An optional effective date on a reading, taken from the source when it states one. Valid-until is derived from the successor's effective, never stored
superseded_by is written onto the old record	Correcting a record edits an existing one, breaking append-only and the one-way flow of links	The correction carries corrects: <id>; superseded_by is derived on read, so every link lives on the newer record
No record that a fact was re-checked and still holds	A goal nobody has re-read for weeks looks as current as one read today	A short confirms: <id> reading, plus a rule that flags a chain head not confirmed within an agreed interval
One kind of stale bet	A driver resting on a since-corrected misquote and a driver resting on an old but accurately recorded goal need different handling	Two flags: stale (its reading was replaced) and unfounded (its reading was corrected)
Nothing checks the links	Supersession fails silently when a current flag is set but the links behind it are never written	A validator run after every write: each replaces, corrects and citation resolves; exactly one head per subject; IDs never reused
No way to find the current reading without walking chains	Fine at tens of records; slow and error-prone at hundreds	A stable subject key on each reading, and later a generated heads-only view of the register

Search by meaning is not a gap yet. The evidence places the point where reading everything stops winning somewhere between about 26,000 and 115,000 tokens of history. The register starts empty. Plain text search comes before embeddings, and any later index should take register records as structured items rather than run model extraction over them.

1. What the published evidence shows

Labels: (P) published evidence with numbers, (V) vendor or author claim, (I) inference. Most benchmark numbers are published by the authors of the system being measured.

1.1 Bitemporal and validity-window models

(P, author-published) Zep and its open-source engine Graphiti keep two timelines on every fact edge: valid time (t_valid, t_invalid) and recorded time (t'_created, t'_expired), the latter serving "the traditional purpose of database auditing" (Zep paper, arXiv 2501.13956, January 2025). When a new fact contradicts an overlapping old one, a model sets the old fact's t_invalid to the new one's t_valid; nothing is deleted. On LongMemEval with GPT-4o, Zep scored 71.2 per cent against 60.2 for the full history, using 1,600 tokens of context instead of 115,000. By category: knowledge update 83.3 against 78.2, temporal reasoning 62.4 against 45.1. With GPT-4o-mini, knowledge update fell slightly, from 76.9 to 74.4.

(P) The distinction comes from temporal databases: Snodgrass, Developing Time-Oriented Database Applications in SQL (1999, PDF), standardised in SQL:2011 as application-time and system-versioned tables (Kulkarni and Michels, SIGMOD Record 2012). (V) XTDB is bitemporal by default; Datomic keeps recorded time only and models valid time as ordinary attributes.

(V) Hindsight's authors say it does not model when a fact stopped being true, and refer users who need that to Zep (from their May 2026 consolidation post). (P) Their paper (arXiv 2512.12818) does store an occurrence interval and a separate mention time on each memory, but describes no rule for ending a fact, and lists "time-aware belief revision" as future work.

(I) A replaces chain ordered by date read is a recorded-time history. Validity windows add three things: a fact's effective date when it differs from when it was learned; facts with a known end; and the ability to ask "what was true on D" separately from "what did the agent believe on D". The second question is the one a stale-bet audit asks. Zep's figures suggest the gain is mostly on temporal reasoning, not on knowing the latest value.

1.2 Event sourcing and corrections

(V) In event sourcing, current state is a projection replayed from an immutable log; a wrong or late event is handled by appending a reversing or correcting event, never by editing the log (Fowler, Event Sourcing and Retroactive Event; Young, Versioning in an Event Sourced System, 2017). Practitioners distinguish a compensating event, which records a real reversal in the domain, from a correction event, which says an earlier event was recorded wrongly. Both are appended; projections may treat them differently.

(V) Architecture decision records mark a replaced record "Superseded by ADR-N", which in practice edits the old record's status line (Nygard, 2011).

(I) The proposed split between replaces and superseded_by is the compensating-versus-correction distinction. The difference is where the link lives: event sourcing puts it on the new event, while the proposal writes superseded_by onto the old record.

1.3 Episodic and semantic memory in agent designs

• (V) CoALA (Sumers et al., arXiv 2309.02427) divides agent memory into working, episodic, semantic and procedural. It is a taxonomy, not a result.
• (P) Generative Agents (Park et al., arXiv 2304.03442) keep an append-only stream of time-stamped observations ranked by recency, importance and relevance, with periodic reflections stored back into the stream citing their sources. Its ablations measured believability as judged by people, not factual accuracy.
• (V) MemGPT, now Letta (arXiv 2310.08560), separates core memory in context from searchable recall and archival stores.
• (V) A-MEM (arXiv 2502.12110) keeps linked notes that the model rewrites over time. (P) In Mem0's re-run on the LoCoMo benchmark it scored lowest, 48.4 per cent.
• (P, author-published) Mem0 (arXiv 2504.19413) reports on LoCoMo: full history 72.9 per cent, Mem0 with a graph 68.4, Mem0 66.9, Zep 66.0, the best retrieval baseline 61.0. Full history wins on accuracy; memory wins on latency (1.4 against 17.1 seconds at the 95th percentile) and tokens. Base Mem0 deletes a contradicted memory; its graph variant marks it invalid instead.
• (V) The figures are disputed. Zep says Mem0 misconfigured it and reports 75.1 per cent when run correctly (Zep blog). Letta reports 74.0 per cent for a plain agent with the history in files and grep and open tools, above Mem0 (Letta, August 2025). Secondary analyses report the same product scoring between 58 and 95 depending on who ran it, and an audit claiming much of LoCoMo's answer key is wrong; neither was verified here.
• (P, author-published) Hindsight separates world facts, the agent's own experiences, opinions and entity summaries. On LongMemEval with an open 20-billion-parameter model it reports 84.6 per cent on knowledge update and 79.7 on temporal reasoning, against 60.3 and 31.6 for the full history. Its retrieval budgets are left as placeholders in the paper.

(I) Two features recur in the stronger systems: a time stamp on every record, and a separation between facts about the world and the agent's own beliefs and acts. Graphs, embeddings and self-rewriting notes are not shown to help by themselves. The proposed model already separates readings (world facts), drivers (beliefs) and decisions (agreements).

1.4 Benchmarks on changed facts

(P) LongMemEval (Wu et al., ICLR 2025) scores knowledge update and temporal reasoning as separate categories.

Setting	GPT-4o	Llama 3.1 70B
Only the sessions holding the evidence	87.0% (92.4% with note-taking before answering)	74.4%
Full history, about 115,000 tokens	60.6%	33.4%

It also found that indexing by extracted facts improved recall by 9.4 per cent; that time-aware query expansion helped temporal questions only with a strong model; and that presenting retrieved items as structured JSON with notes worked best, while a poor reading format cost about ten points even with perfect retrieval.

(I) Curated, relevant-only context is the best condition in the literature. At about 26,000 tokens (LoCoMo), the full history beats every memory system; at about 115,000 (LongMemEval), it loses to good ones by ten to forty-five points. Knowing the latest value degrades least as history grows; temporal reasoning degrades most. A small register of structured, dated records read in full sits in the best condition.

1.5 What extraction does to provenance

(P) HaluMem (arXiv 2511.03506, v3 January 2026) tests memory systems stage by stage. On its medium dataset Mem0 extracted 42.9 per cent of the facts it should have, applied 25.5 per cent of updates correctly and omitted 74.0 per cent; the best system, MemOS, applied 62.1 per cent. On the long dataset Mem0's correct-update rate fell to 1.5 per cent. Every system answered under 70 per cent correctly.

(I) Extraction fails mostly by omission and by updates that never happen, rather than by invention. Once a memory is a paraphrase with no quote attached, nothing in the store distinguishes an error from a real change. Keeping the exact quote beside the interpretation, as the proposal does, is the defence these systems do not apply.

2. Memory setups in other Hermes deployments

Labels: (R) reported experience from users or operators, (V) maintainer or vendor claim, (I) inference. Issues are on NousResearch/hermes-agent; their numbers and dates were gathered during the research and were not individually re-checked.

Few long-running accounts exist. No write-up was found of a marketing, operations or CEO-style Hermes agent running for months with a named memory provider and any measure of recall. Reported experience is almost all in GitHub issues, about the built-in files filling up and plugins breaking.

Built-in mechanisms

• (V) MEMORY.md (2,200 characters) and USER.md (1,375) load as a frozen snapshot at session start and never compact themselves. The tool's replace action was withdrawn because "models used it as a find-and-swap, which wiped the other facts in multi-fact entries". On chat gateways a single session can run for weeks, so the learning loop "almost never gets to fire" without new sessions (memory docs).
• (R) Files fill within weeks and the agent cannot save once full (#121764, #5320). Stale entries "persist indefinitely … and can actively mislead the model" (#90035). On v0.17 one user's file fell from 28 entries to 5 three times in a session while the tool reported success (#56464).
• (R) The closest reported pattern to the proposed model: a user keeps a few critical facts in MEMORY.md as an anchor and maintains "a separate chain of dated, append-only documents" by hand, asking for it to become first-class (#116564); an RFC followed (#143379).
• (R) Session search still returns a fact after it was removed from MEMORY.md, reproduced on v0.22.5 (#145902). It can also return other profiles' sessions (#147008).
• (R) The background review that updates memory and skills works from a snapshot of the conversation and can "overwrite / replace / remove newer memory entries" (#9055). (V) The curator archives agent-created skills unused for 30 days unless pinned or referenced by a cron job (curator docs).

Memory-provider plugins

(V) The catalogue on Hermes's current main branch lists 73 memory providers, all community tier, including the vendors' own Honcho, Hindsight, Mem0 and Supermemory plugins; one external provider is active at a time, and writes to the built-in files are mirrored into it (memory-providers docs). Zep, Graphiti, Letta and Basic Memory are not in the catalogue; Graphiti and Basic Memory have been connected by users through other routes (#70067, #23296).

• (R) Honcho: reports are bugs, not quality: runaway queries, multi-bot prompts stored as the user's words, senders stored under the wrong identity, silent unreachability (#135728, #141333, #137328, #137329).
• (R) Hindsight: "breaks after every toolchain-refreshing update" (#142988); a migration left it disabled behind a misleading status (#135771). (V) Vectorize claims 94.6 per cent on LongMemEval in its own comparison (Vectorize).
• (R) Mem0 self-hosted "breaks on every update" (#135347).
• (V, no reported users found) Several small providers claim the behaviour the proposal needs: inspeximus ("a correction retires the old value by key", with revert), severian ("canonical append-only store", memories citing their observations, "truth resolution when a newer fact replaces an older one"), bilinc ("when it changed and why"), mnemosyne (a temporal knowledge graph).
• (R) One operator ran Hermes for about three months with memory kept separate from the model, so switching models kept what he had taught it; long-running actions still lose context between runs (comment on Artificial Code, May 2026).

Files, wikis and MCP

• (V) Nous's optional llm-wiki skill keeps an immutable raw/ folder of sources, an append-only log.md, and page front matter with updated, sources, contested and contradictions. Two community providers, hwiki and stack, keep memory as Markdown in git.
• (R) One operator's agent "manages my llm-wiki at night", without metrics (Artificial Code, May 2026). A user template archives sessions to an Obsidian vault and promotes "only durable, human-approved facts" into MEMORY.md (hermes-brain-template).
• (R) Graphiti and GBrain have been used over MCP; no reports were found of the reference MCP memory server or OpenMemory with Hermes.

Which setup supports the proposed model

(I)

• Extraction-based providers (Mem0, Honcho, Hindsight as a plugin, Supermemory) store model-written facts, not exact wording; their provenance points to a session, not to the OKR file or message; their dates are ingestion times, not reading dates; and supersession happens inside the provider with no replaces link to audit. A driver needs a stable record ID to cite, which these stores do not promise. They could add fuzzy search over the register and could not replace it. Because Hermes mirrors built-in memory writes into the active provider, routing register content through the memory tool would create a second, paraphrased copy.
• Inspeximus comes nearest to superseded_by, but hides the retired value on recall, whereas the proposal needs a replaced reading to stay citable by the drivers that bet against it. Severian matches on paper but has one maintainer and no reported users, and would move the supersession rules into its own code.
• File and wiki approaches keep wording and structure under git. The best fit found is the reported anchor pattern: the YAML register as the single source, a one-line pointer to it in MEMORY.md, and at most a text-search index over it later.
• Two cautions for any retrieval layer. Session search and providers will surface replaced readings from old transcripts with no status attached, as #145902 shows for removed facts, so the agent must check a reading's status in the register before relying on one it recalled any other way. And the skill that maintains the register should be pinned, so the curator cannot archive it and a late background review cannot rewrite it.

3. The proposed model judged against the findings

(I) What holds:

• Exact quote kept beside interpretation. This is the property extraction-based memory lacks and the one that separates a recording error from a real change (section 1.5).
• Append-only, dated records with explicit replaces links. This matches event sourcing and the design's existing pattern for decisions, gives the history directly, and needs no model on write.
• Two kinds of change link. replaces and the correction link are event sourcing's compensating and correction events.
• Separate world facts, beliefs and agreements. Readings, drivers and decisions mirror the separation the stronger agent-memory designs share.
• Read in full, no index. At the register's expected size this is the best-performing condition in the literature.
• The driver cites a reading rather than holding the goal. Several drivers can bet against one goal without copying it, and a replaced reading flags exactly the drivers that depended on it.

What it lacks, with the smallest addition for each, is the table under "Finding". Three items are worth expanding.

Effective date. A reading records when the agent read the goal, which is recorded time. A goal amended on 13 October and read on 20 October would date the change a week late. Add an optional effective date when the source states one. Do not add a stored valid_until: it can be derived from the next reading's effective, and storing it would mean editing the older record.

Where the correction link lives. The proposal marks the wrong record superseded_by its correction, which edits an existing record. Event sourcing's form, which keeps the register strictly append-only and every link pointing backwards, is for the correcting reading to carry corrects: <id>. A reader or the validator derives superseded_by. The same change would apply to the correction rule for all record types, not only readings.

The validator. Supersession links that are never written fail silently unless something checks them. Extending the design's existing parent, cycle and link checks to replaces, corrects and citations, with exactly one chain head per subject and no reused ID, is the guard against that failure.

Two further additions are optional and can wait for use: a resolves record that closes a driver with its outcome, which would later let the agent judge how good its bets were; and allowing replaces to take a list, for a goal that splits into two.

When to add search, and what kind. When the register stops fitting comfortably in one read, which the evidence places somewhere in the tens of thousands of tokens, first generate a heads-only view from it, then add plain text search. Consider search by meaning only after that, index register records as structured items with their IDs and dates rather than through model extraction, and treat the index as a retrieval aid whose results are always checked against the register.

Limits of this research

• The Hermes plugin catalogue was read from the GitHub main branch (commit 50d2adf), not from the installed v0.22.5 copy. Entries and tiers may differ slightly at v0.22.5.
• GitHub issue numbers and quotations in section 2 were spot-read, not each re-verified.
• Classic sources (Snodgrass, SQL:2011, Fowler, Young, Nygard, Park, MemGPT, A-MEM, CoALA) were cited from their canonical locations without re-reading them in this pass.
• Most benchmark figures are published by the authors of the systems measured, and LoCoMo comparisons across vendors are unreliable.
• Reddit, Discord and X returned nothing indexed on Hermes memory.