An agent that cannot remember starts over on every call. It re-asks answered questions, repeats corrected mistakes, and loses work spanning more than one session. This guide shows how to design ai agent memory systems that keep the right information, for the right amount of time, without letting old or wrong memories steer every future decision.
Storing text is easy. The hard parts are deciding what is worth keeping before you know what will matter, keeping stored memories accurate as the world changes, and loading only what the next decision needs. The evidence comes mainly from Anthropic’s 2026 work on agent memory, the Generative Agents architecture, and the persistence mechanisms in LangGraph, the OpenAI Agents SDK, and Letta.
What AI agent memory is — and what it is not
Anthropic defines agent memory as knowledge that Claude gains from experience, including interactions with users and assigned tasks, that it retains between sessions, saved as context it can read later. The definition is narrow: it excludes changes to the model’s weights, knowledge gained within a single conversation, and compaction, the summaries that replace older turns so work can continue in a new context window. Memory, here, is what survives between sessions as readable context.
That matters because language models alone are stateless: they retain nothing unless something outside the model stores it and brings it back. Anthropic’s augmented LLM is a model extended with retrieval, tools, and memory. Memory keeps state across turns and sessions; retrieval pulls context into the prompt; tools let the model act. Memory overlaps with a knowledge base, which holds an organisation’s durable facts. The two differ in role: a knowledge base centres on general reference material, while agent memory centres on what the agent itself has experienced, decided, and learned, and a design may keep session-derived records in either, as long as scope and provenance are explicit. Memory is also distinct from a tool, and from the prompt itself. Anything that must outlive one call needs a home outside the prompt, with rules for writing, reading, updating, and removing it.
The four memory layers every design needs
Organise memory into four layers. Working memory is what the agent holds right now: the current conversation, the task in flight, and any scratchpad. It lives in the context window and vanishes unless deliberately saved. Episodic memory is what happened: past sessions, decisions, and corrections, kept as transcripts, notes, or summaries. Semantic memory is what is true in general: durable facts about users, systems, and the world. Procedural memory is how work is done: reusable skills, playbooks, tested rules, and lessons learned.
Those layers are a design lens, not a vendor standard. Anthropic describes a complementary storage hierarchy: transcripts as a searchable event log, then memory notes, then representations such as wikis of linked pages, and finally a small injected context loaded into every session. Letta’s tiers differ again: core memory blocks in every prompt, recall memory for conversation history, and archival memory for saved records. The bridge from episodic to semantic memory is reflection. In Generative Agents, reflection periodically read recent memories in a timestamped memory stream and wrote higher-level inferences back into it. Anthropic’s dreaming and Letta’s sleep-time compute follow the same pattern: raw experience in, checked and consolidated knowledge out.
Step 1: Map what ai agent memory systems must remember, for how long, and for whom
Before choosing storage, list every memory need with its lifetime and owner. A turn-level need needs only the working set. A session-level need, such as a correction made five minutes ago, must survive across turns in that session. A task-level need must survive across sessions until the task ends. An indefinite need, such as a stated preference or verified fact, must survive until deliberately changed or removed. Writing lifetimes down prevents building a permanent store for information that goes stale in hours, and relying on conversation history for facts that must outlive the session.
Scope is a safety decision. Decide whether each memory belongs to one user, one team, or a shared pool, and what must never cross that boundary. If a memory could cause harm read by the wrong session, enforce scope in storage, by a per-user or per-tenant key, not by hoping the model will not mention it. Then rank needs by the cost of the two opposite failures. Anthropic frames memory as a prediction: a written rule predicts the pattern will matter again, and a verified fact predicts a future session will rely on it without double-checking. Forgetting means redoing work. Remembering wrongly is worse: a wrong note, trusted unchecked, steers every session that reads it. High-confidence items earn the small always-loaded layer; uncertain items stay in the searchable long tail, where they can be checked first.
Step 2: Choose storage and retrieval for each layer

Match storage to the query each layer must answer. Similarity search over a vector store suits notes where the question is what resembles this situation. Structured stores suit facts needing exact answers, histories needing audit order, and records needing update or deletion by key. Plain files suit small, human-readable memory people must inspect and edit, as with Anthropic’s memory notes. A transcript log with search suits the foundation layer: a complete record other layers are checked against.
Retrieval designs that rank candidate memories need explicit scoring. Generative Agents scored each memory by recency, with exponential decay of 0.995 per in-game hour; importance, rated 1 to 10 at creation; and relevance, embedding cosine similarity to the current situation. Those three signals are one option for retrieving experience or notes: a design can use them, explicit lookup, or exact-fact retrieval where the query suits it. Similarity-only retrieval risks letting old, trivial memories crowd out recent, important ones. Framework mechanisms cover only part of the problem: LangGraph checkpointers snapshot graph state at each super-step, organised by thread, for resuming and human inspection, while OpenAI Agents SDK sessions prepend stored history before each run and persist new items after it, with a limit on how much is fetched. Both cover run and conversation state, not cross-session learning, which still needs the episodic, semantic, and procedural layers.
Step 3: Write, update, and forget

Decide when the agent writes: at decision points, when a correction arrives, and at session end, when lessons can be summarised. Anthropic advises simple guidance for frontier models: create a note when a person asks or something is clearly worth remembering, and leave salience judgment to the model. Its tests support that: telling Opus 5 and Opus 4.7 to record only failures cost them much of memory’s performance boost, while prompting an older model, Sonnet 4.6, to read its memory and test its theories nearly doubled its score on Anthropic’s database task. Steering should match the model, and the default should be purpose, not a rigid rulebook.
Consolidation happens between sessions, off the critical path. Anthropic calls it dreaming: reflecting on logs to fix mistakes and produce new notes and representations. Letta calls it sleep-time compute: using idle time to reorganise context into learned context. Dreaming can correct a note against its transcript, build a wiki, fill gaps such as dead links, and promote stable facts into the injected layer. Set its cadence by how fast the agent’s world changes and how often memories are accessed, since consolidation costs compute and unchecked notes drift. Updating and forgetting complete the lifecycle: a contradicted record must be invalidated, not left to compete with its replacement, and every record type needs an expiry or review trigger. Anthropic’s Pokémon test illustrates memory selection and clutter: Claude Sonnet 3.7 wrote 413 files across 225 folders, mostly fast-changing game state, and earned one badge, while Claude Fable 5, keeping nine files plus one dashboard file and recording only strategy and tested rules, earned four. The comparison shows what each model judged worth saving; it does not isolate forgetting as the cause of the difference. Expiry and invalidation remain design recommendations, supported by the risk that unbounded growth costs more to search than the work it saves.
Step 4: Wire memory into the agent loop
Place read and write points deliberately. Read points load stored material before reasoning: the injected layer at session start, retrieved notes for the task, and run state restored from a checkpoint. Write points capture new material: salient events tagged as they happen, corrections recorded on arrival, and the session-end summary. If a model never reads what was written, the store is only disk usage; Anthropic found an older model often did not open its memory directory, and the fix was prompting it to read memory before answering, not a larger store.
Keep the injected layer small, because injected memories use tokens and a wrong note there is carried into every session. Claude Code’s auto memory loads only the first 200 lines or 25 KB of its memory index at conversation start, as an index pointing to detail kept in separate files. Set a similar budget, order material by what the next decision needs, and link to the long tail instead of copying it into every prompt. Give humans review over high-stakes writes: LangGraph checkpointers let a person inspect, interrupt, and approve steps from a saved snapshot, and the same inspect-and-reverse principle should apply to shared facts, safety rules, and promotions into the always-loaded layer, with the transcript kept as ground truth.
Failure modes, privacy, and safety limits
The most dangerous failure is a wrong memory trusted as fact. A note can be wrong from the start or drift as the environment changes. Anthropic’s tests show both failure and repair: a wrong note about which side of a mountain a building stood on sent the agent looping over a one-way ledge three times, and checking the note against the transcript corrected it. Keep provenance, the transcript or source behind each memory, and focus checking on information likely to change. Unbounded growth makes retrieval slower and noisier; retrieval misses leave the right memory unfound; context crowding lets old notes take space the current task needs.
Privacy and retention are design obligations. Memory crossing a scope boundary, one user’s facts in another user’s session, is a storage-design failure, and scope decisions from Step 1 are the control. A memory kept indefinitely is a liability if the fact is personal or sensitive. This research verified no vendor privacy, tenant-isolation, or retention controls for agent memory, so verify in your platform’s documentation what isolation, access control, and deletion it actually enforces, and design scope and expiry to match. Finally, memory cannot fix a missing tool, a bad plan, or ungrounded reasoning. Anthropic’s advice applies: use the simplest solution that works, and add agentic memory only when simpler approaches fall short.
A worked memory design checklist
Consider a support agent serving repeat customers. Working memory is the current ticket and conversation. Episodic memory is a searchable log of past tickets, decisions, and corrections, per customer. Semantic memory is verified facts, plan, systems, and policies, each with a source and check date. Procedural memory is a playbook of rules written only after a rule has worked more than once. Storage follows the layers: ticket log in a structured store keyed by customer, facts in a queryable store, playbook as editable files, transcript log as the foundation. Writes happen on correction and at ticket close; consolidation runs nightly, checking notes against transcripts and promoting only stable facts into an injected index holding plan, open issues, and current rules, and nothing else.
Checklist: lifetime stated for every need; scope keyed to one user, team, or shared pool; storage and retrieval fitted per layer, choosing signals appropriate to each query, including explicit lookup where suitable, and considering recency, importance, and relevance for experience or notes retrieval; contradicted records invalidated; an expiry or review trigger for every record type; human inspection and reversal for important memories; a small, ordered, indexed injected layer. Then test with fresh sessions that can only succeed by reading memory, plant a contradiction and check the old note no longer surfaces, and run a forgetting drill to confirm an expired record is gone from retrieval, the injected layer, and every index. Anthropic’s evaluations show the possible value: a wiki built by dreaming lifted performance on a large document corpus from 13 percent to 48 percent of graded criteria over 128 sessions, and search plus wiki reached 61 percent against 53 percent for search alone. Those are Anthropic’s results on its own evaluations, not a promise. Your recall, contradiction, and forgetting tests are the numbers that tell you whether your design works.












