Agentic RAG vs Agent Memory: What Each One Stores
Agentic RAG and agent memory both put outside information into a model's context, so teams often treat them as one thing. They answer different questions. Agentic RAG asks what the documents say about this task. Agent memory asks what this agent has already learned from working with you. If you build or run agents, you have to decide where each fact lives and who updates it. By the end you will know which store a given fact belongs in.
What is agentic RAG, in one sentence?
Agentic RAG is retrieval-augmented generation in which an agent plans the retrieval itself, deciding what to search and when the results are good enough to answer.
Plain RAG came first. The 2020 paper by Lewis et al. paired a language model with what it called non-parametric memory, a dense vector index of Wikipedia searched by a neural retriever. The model's weights held one kind of knowledge, and the index held another that could be swapped or refreshed.
The agentic version moves control to the model. The survey by Singh et al. was first posted in January 2025 and revised in April 2026. It describes it as embedding autonomous agents into the RAG pipeline. Those agents use four design patterns: reflection, planning, tool use and multi-agent collaboration. In practice the agent can rewrite its query or pick a different index when the first pass comes back thin.
What is agent memory?
Agent memory is information an agent writes for itself, from its own sessions, so a later session can use it.
Typical entries are a user's preferences and the corrections the agent got on a project.
The MemGPT paper from 2023 framed this as an operating system problem. It moved data between fast and slow memory tiers to give the model the appearance of more context than its window holds. Shipping products are simpler. Claude Code's memory docs describe two layers. CLAUDE.md files hold instructions you write, and auto memory holds notes Claude writes about your corrections and preferences. Both load at the start of every session, with auto memory capped at the first 200 lines or 25KB.
How does agentic RAG differ from agent memory?
| Agentic RAG | Agent memory | |
|---|---|---|
| Source of the content | Documents written by people or other systems | The agent's own sessions and corrections |
| Who writes it | Your ingestion pipeline | The agent, sometimes with you editing |
| Typical question | "What does the refund policy say?" | "How does this user want refunds handled?" |
| How it changes | Re-index when documents change | Updated after each session that teaches something |
| Scope | Shared corpus, often the whole company | Per user, per project or per agent |
| Main failure | Retrieves the wrong passage | Remembers something wrong, or stale |
Both often use embeddings and a retriever, so the code alone will not tell you which one you have. Check who writes the store.
When should you use agentic RAG?
Use it when the answer already exists in a body of text that the agent did not write. Product docs and support tickets are typical. The corpus is usually large and changes on its own schedule. Agentic RAG beats plain RAG when a question needs more than one lookup. The agent can check its results and search again.
The cost is latency and tokens. Each reflection or re-query is another model call. If your questions are simple lookups, a single retrieval step is cheaper and easier to debug.
When should you use agent memory?
Use it when the knowledge comes from the interaction itself and no document holds it yet. Examples are the flag a test suite needs on this machine, or the deploy procedure that took three tries last Tuesday.
Memory has its own risks. A wrong note keeps getting retrieved until someone fixes it. The Pydantic AI memory docs call memory model-written and untrusted. They add that a delimited prompt block is not a hard defense against prompt injection.
Can you combine agentic RAG and agent memory?
Yes. A support agent, for example, retrieves the policy from the knowledge base and recalls that this customer already got an exception last month. A coding agent searches the repository and recalls that the team bans a certain library.
Keep the stores separate. Mixing agent-written notes into a document index makes it hard to tell a vetted policy from an agent's guess. Give each store its own retrieval step, and label what comes back so the model knows which is which. When a memory turns out to be a durable fact for everyone, promote it into the documents on purpose and delete the note.
The bottom line
Use agentic RAG for knowledge that exists in documents and changes when the documents change. Use agent memory for what the agent learns from its own work with a user or a project. If you can build only one first, look at your agent's repeated mistakes. Missing facts from the docs call for retrieval. Repeated corrections call for memory.
How do you give coding agents memory that RAG cannot provide?
You can point Claude Code or Codex at your docs and repository, which is retrieval. What the agent learned yesterday, such as the flaky test, sits in that one host's session logs. Switch tools and it is gone. Even in the same tool, Claude Code loads only the first 200 lines or 25KB of auto memory.
memU is an agent-driven memory system for this case. A scheduled background task slices each host's new session logs into jobs. The agent writes what is worth keeping as memory or skill Markdown. memU's MemoryService makes no LLM or chat calls. It stores and embeds that Markdown, and a standing instruction in CLAUDE.md or AGENTS.md tells the agent to retrieve before answering. All hosts share one backend, and you can fix a wrong note by hand.
The hosted version is free and syncs across devices. Get an API key from memu.so, then send your agent one message:
Read https://memu.pro/SKILL.md, follow its instructions to install and configure memU, API Key is
YOUR_MEMU_API_KEY.
The self-hosted install and the support matrix are in the memU GitHub repository.
FAQ
Can RAG be used as agent memory?
Partly. You can index an agent's notes and retrieve them with RAG code. The memory part is the write path: the agent must decide what to save from its own sessions and update it when it turns out wrong.
What is the difference between RAG and agentic RAG?
Plain RAG runs one retrieval step before the model answers. Agentic RAG lets the model plan its own searches and try again when the results are weak. The Singh et al. survey names reflection and planning among its patterns.
Does agent memory need a vector database?
Not always. Claude Code's auto memory is a Markdown file loaded at session start. Larger memory stores usually add embeddings so the agent can retrieve only the relevant notes.
What is short-term vs long-term memory in AI agents?
Short-term memory is the current context window and conversation history. Long-term memory is anything stored outside the window that a later session can load or retrieve.