A model call on its own is stateless: it sees a prompt, produces a reply, and carries nothing into the next call. What semantic memory AI agents remember across sessions — that you prefer Python type hints, that your staging environment lives in a particular account, or that a service deprecated an API last month — has to be stored outside the model and assembled into each new prompt. Semantic memory is the part of long-term memory that holds those facts about the world and the user: durable, declarative statements that answer “what is true” rather than “what happened” or “how to do it”. This article explains what it is, how it differs from the agent’s other memory types, and how it is built, retrieved, and kept accurate.
What semantic memory is
Semantic memory is the agent’s store of general knowledge: facts and preferences that stay true across sessions until something changes them. “The user prefers concise answers with sources” is a semantic memory; so are “the team deploys on AWS” and “the staging environment lives in account Z”. These are statements about the world that the agent can recall and reason with later — not transcripts of what happened, and not instructions for how to behave.
The term is borrowed from cognitive science, where the episodic/semantic split within declarative memory is associated with Tulving, and the declarative/procedural distinction with Squire. That lineage matters because the categories are not vendor-invented. LangChain’s documentation adopts the same semantic, episodic, and procedural classification for long-term memory, and LangMem, its memory SDK, organizes memory around exactly those three types. When you design an agent’s memory around this taxonomy, you are aligning with the shared vocabulary of the field, which makes the design reviewable and the documentation legible to people who did not build the system.
One honest caveat: no framework exposes a standalone “semantic memory” feature flag. Semantic memory is always one type inside a broader memory API, alongside episodic and procedural memory, so the taxonomy is a design lens, not a product catalog.
Semantic memory vs episodic and procedural memory
The three types differ in what they hold and, crucially, in how they are written. Episodic memory records events: what happened, in which session, in what order. It is appended, not updated. Procedural memory records how to do things: skills, routines, and lessons about what worked. In LangMem it can take the form of a self-updated system prompt, so the agent’s behaviour adapts from experience. Semantic memory records facts, and updating a fact means superseding the old one — which is what makes staleness and contradiction handling central to its design.
LangMem’s own framing makes the distinction concrete. Semantic memory stores facts and preferences (“user works in Pacific time, prefers Python examples”). Episodic memory stores past interactions distilled as few-shot examples (“last time the agent solved X by doing Y”). Procedural memory encodes system behaviour and response patterns through rules and prompt optimization. The labels matter because the write path, the retrieval query, and the invalidation rule all depend on which type a piece of memory is: you update a fact, you append an episode, you refine a procedure.
How persistent agents build semantic memory
Building semantic memory is a three-step lifecycle: extract, consolidate, store. Extraction happens after a turn, when the agent — or a separate extraction step — decides which statements from the conversation are worth keeping as facts. LangMem formalizes this as an extract-and-consolidate loop driven by the model itself. In code, the extraction step looks like this:
from langmem import create_memory_manager
manager = create_memory_manager(
"anthropic:claude-3-5-sonnet-latest",
instructions="Extract user preferences and facts",
enable_inserts=True,
)Consolidation turns extracted statements into a durable fact store rather than a pile of near-duplicates: statements are deduplicated and merged with existing facts instead of stored verbatim. Mem0’s Dream runs two consolidation actions automatically as memories are added: Supersede, which marks a fact replaced by a newer one, and Merge, which combines duplicate facts. A third action, Synthesis, distills pattern memories on a schedule in the background — but it is opt-in and available on Pro plans and above, unlike Supersede and Merge, which run automatically for everyone. Letta’s Dreaming does the same with background subagents that review recent conversations, consolidate lessons, and update memory without interrupting active work.
One thing consolidation is not: knowledge distillation in the machine-learning sense. Teacher-student model training is not a mechanism any production agent framework uses for semantic memory; what frameworks actually do is LLM-driven extraction plus deduplication, merging, and promotion of facts. Research is formalizing the step: a 2026 paper describes consolidation as a deterministic function from an episodic event log to a separately addressable semantic knowledge layer, and a Microsoft paper proposes sleep-phase consolidation, an offline pipeline that deduplicates, clusters, and promotes valuable memories to a slow semantic store.
Storage takes more than one shape. Two supported approaches are embedding vectors in a vector store, which give similarity retrieval — a fact is stored with its embedding and fetched by closeness to the current query — and fact edges in a temporal knowledge graph, which give structured and temporal queries: a fact is a relationship between entities with timestamps, so the system can ask what was true at a given date. Graphiti is the documented example of the graph approach, storing facts as bi-temporal edges. The LlamaIndex fragment below is a different thing: a block-composition example showing how static, extracted-fact, and vector blocks are combined in one memory, and it assumes llm, vector_store, and embed_model already exist:
from llama_index.core.memory import (
StaticMemoryBlock,
FactExtractionMemoryBlock,
VectorMemoryBlock,
)
blocks = [
StaticMemoryBlock(
name="core_info",
static_content="My name is Logan, and I live in Saskatoon. I work at LlamaIndex.",
priority=0,
),
FactExtractionMemoryBlock(
name="extracted_info",
llm=llm,
max_facts=50,
priority=1,
),
VectorMemoryBlock(
name="vector_memory",
vector_store=vector_store,
priority=2,
embed_model=embed_model,
similarity_top_k=2,
),
]The FactExtractionMemoryBlock is the semantic layer in this setup: an LLM pulls facts out of flushed chat history, up to a configured limit. The VectorMemoryBlock stores batches of messages in the vector store for similarity retrieval. The StaticMemoryBlock is pinned information that is always inserted — the “always in context” end of the spectrum.
How semantic memory is retrieved at run time
Retrieval is retrieval-augmented generation applied to the agent’s own past. The agent embeds the current turn, searches stored facts by semantic similarity, and injects the hits into the prompt before the model call. That is the standard run-time pattern across frameworks: LlamaIndex’s VectorMemoryBlock retrieves message batches by similarity, Letta’s archival memory is searched with archival_memory_search, and LangMem exposes a search tool over namespaced memories. The point that matters most is a negative one: a fact that is stored but never retrieved is invisible to the model. Storage and visibility are different things.
Not all facts are retrieved the same way. Facts the agent needs constantly can be pinned in context, with no retrieval step at all: LlamaIndex’s StaticMemoryBlock and Letta’s core memory blocks (editable in-context sections, each around 2000 characters, holding the persona and human blocks) are always present. Everything else is fetched on demand, which keeps the prompt lean but adds a retrieval step that can fail.
Similarity alone is not enough to rank what comes back. Recency, importance, and contradiction status all shape which facts should surface: LlamaIndex orders memory blocks by priority and truncates by priority when the token budget is exceeded, and Graphiti combines three retrieval signals — dense vector embeddings, BM25 keyword search, and graph traversal from seed entities. Retrieval quality deserves its own evaluation, separate from the model’s reasoning, because a bad retriever silently starves the model of the facts it needs. LongMemEval makes the stakes concrete. The benchmark holds 500 questions over histories of roughly 115,000 tokens in its S subset, but the published comparison of commercial systems covered a smaller set of 97 questions, and the package does not establish the history length behind those 97 questions. On those 97 questions, ChatGPT running GPT-4o scored 0.5773 versus 0.9184 for offline reading of the same history — a 37% drop — and Coze scored 0.3299, a 64% drop. The authors conclude that “even the most capable long-context LLMs currently require an effective memory mechanism to manage an ever-growing interaction history.”
Keeping semantic memory accurate over time
Facts go stale and contradict each other, and the system must handle that deliberately. Graphiti handles this explicitly: every fact edge carries two timelines. The valid-time axis (valid_at and invalid_at) records when the fact was true in the world; the system-time axis (created_at and expired_at) records when the system learned or invalidated it. When new information contradicts an existing fact, the system closes the old fact’s validity window rather than deleting it, and records the new fact. As Zep puts it, “when new information contradicts an existing fact, the system closes the old fact’s validity window (sets ‘valid to’) rather than deleting it, and records the new fact.” The agent reasons over the current state while the history stays queryable — including point-in-time questions such as “what did the agent know as of date X?”
Other systems reach a similar outcome by different means. Mem0’s Supersede action marks the old fact replaced, and Merge combines duplicate facts; nothing its Dream consolidation does is destructive. Invalidation preserves an audit trail and lets you answer historical questions, while rewriting keeps the store small — but rewriting without retaining prior versions or a separate history loses the record of what the agent used to believe, so the trade-off is about what you keep alongside the rewrite, not an inevitable consequence of rewriting itself.
Staleness needs the same deliberate treatment. Temporal metadata records when a fact was learned and when it was true, and some systems support TTL-style expiry — Graphiti’s expired_at timestamp exists for retention management. But metadata only helps if something acts on it. One practical check is to periodically ask the agent to restate key stored facts and compare them against the source turns: silent overwrites show up as confidently wrong answers. Checking stored facts against their source turns is worth doing regularly, whether or not a benchmark has documented the failure mode.
Semantic memory AI agents in practice: frameworks and patterns
Four frameworks illustrate the design space. LangMem names the three types most explicitly: semantic memory as facts and preferences, episodic as distilled interaction examples, procedural as self-updated prompts. Its create_memory_manager extracts semantic memories such as user facts and preferences, while procedural memory is handled separately through prompt optimization; memories are commonly scoped to per-user namespaces, so one user’s facts do not leak into another’s.
Zep’s open-source engine Graphiti stores facts as bi-temporal edges in a knowledge graph; contradiction invalidates rather than deletes, and hybrid semantic, keyword, and graph retrieval adds a graph-traversal signal that pure vector search does not provide. LlamaIndex composes three memory blocks — static, fact-extraction, and vector — with priorities and a token budget, backed by a vector store such as Qdrant, Chroma, Weaviate, or Milvus. Letta takes the operating-system approach: the agent manages its own memory through tool calls, with core memory pinned in context, recall memory as searchable conversation history, and archival memory as a vector database. Letta’s Dreaming adds a second mechanism alongside those tool calls: background subagents that review recent conversations, consolidate lessons, and update memory — so agent-operated tools and background consolidation coexist rather than excluding each other.
As an illustrative application design — not a documented framework mechanism — consider a six-step loop: load context, budget tokens, act, observe the results, extract durable facts, and store them. The frameworks differ in where each step happens: extraction by a background pipeline or by the agent itself through tool calls, storage as vectors or graph edges, retrieval by similarity alone or by a hybrid of signals. To see the loop in action, follow one illustrative fact through it: after a user says they deploy on AWS, extraction stores the fact “team deploys on AWS”; when a later turn says the team migrated to GCP, the new fact supersedes it, so retrieval returns the GCP fact while the AWS fact stays on record as superseded; the next prompt is assembled with the current fact, not the stale one. The loop itself is a way to think about the design, not a feature any of the frameworks ships.
Common mistakes and limits
The most common mistake is treating semantic memory as a plain database: storing verbatim turns instead of extracted facts. Raw turns bloat retrieval and degrade recall, because similarity search over whole conversations returns the wrong granularity — a paragraph when the model needed a fact. Extract first and store facts; the raw history belongs in recall memory.
The second mistake is trusting stale facts. A memory system that never re-validates stored facts lets confident wrong answers accumulate, and the failure is silent: the agent answers fluently from a fact that stopped being true months ago. Temporal metadata, contradiction handling, and periodic restatement checks exist for exactly this reason.
The third is over-retrieval. Every retrieved fact is injected into the prompt and costs tokens on every call, and a prompt stuffed with marginally relevant facts can bury the instruction the model actually needs. Retrieval needs a budget and a ranking policy, not just a top-k.
The fourth is forgetting that persistent facts about a user need retention limits and a way to forget. Not every fact deserves to persist. Mem0 exposes update, delete, and delete-all per user, and teams add namespace retention policies with staleness thresholds and archive-then-delete actions. A memory layer without a deletion path is a privacy liability, not a feature.
Finally, mind the measurement gap. There is no benchmark that isolates semantic memory recall for agents. LongMemEval is the closest, covering five abilities including memory updates, temporal reasoning, and multi-session question answering — and on its 97-question commercial-system comparison, ChatGPT lost 37% of its accuracy to the memory pipeline. That is one reported comparison under specific conditions, not a verdict on every system, so treat it as evidence that the pipeline matters rather than as a universal loss rate. Evaluate your own memory layer on your own tasks: retrieval recall, staleness handling, and contradiction resolution are the three numbers that decide whether semantic memory is helping your agent or quietly misleading it.
