An agent built on the basic agent loop forgets everything the moment the program exits. For one-shot tasks, that's fine. For an agent that should remember users, accumulate knowledge, or pick up where it left off, you need a memory strategy — and the right one is usually simpler than you think.
Start with in-context memory (you already have it)
In-context memory is the messages list you pass with every API call — Claude re-reads the full history each time it's called. The ceiling is the model's context window: roughly 200,000 tokens, about 150,000 words. That is a lot of memory. For most agent tasks, in-context memory is all you need.
The rule: only reach for external memory when tasks span multiple sessions, or the knowledge is too large to fit in context. Adding memory infrastructure an agent doesn't need is the most common way beginners overcomplicate their first build.
External memory: a JSON file is a real memory system
The simplest persistent memory is a JSON file — good for facts, preferences, or state that must survive between runs:
import json
from pathlib import Path
MEMORY_FILE = Path("memory.json")
def load_memory() -> dict:
if MEMORY_FILE.exists():
return json.loads(MEMORY_FILE.read_text())
return {"facts": [], "context": ""}
def save_memory(memory: dict) -> None:
MEMORY_FILE.write_text(json.dumps(memory, indent=2))
def memory_to_context(memory: dict) -> str:
if not memory["facts"]:
return ""
facts = "\n".join(f"- {f}" for f in memory["facts"])
return f"\n\nWhat you already know:\n{facts}"
# Inject into the system prompt at the start of each run:
memory = load_memory()
system = SYSTEM_PROMPT + memory_to_context(memory)That's the whole trick: load at startup, fold the facts into the system prompt, and the agent "remembers" across runs.
Let the agent decide what's worth remembering
The step that makes it feel like real memory: give the agent a save_memory tool, so it chooses what to keep — you don't hand-curate:
# Tool definition
{
"name": "save_memory",
"description": "Save an important fact to long-term memory. "
"Use for information worth keeping across sessions.",
"input_schema": {
"type": "object",
"properties": {
"fact": {"type": "string",
"description": "The fact to remember, stated clearly."}
},
"required": ["fact"]
}
}
# In execute_tool:
elif name == "save_memory":
mem = load_memory()
mem["facts"].append(tool_input["fact"])
save_memory(mem)
return f"Saved: {tool_input['fact']}"Now "remember that I prefer summaries under 100 words" works exactly the way you'd hope: the model calls the tool, the fact lands in the file, and every future run starts already knowing it.
When a JSON file stops being enough
Thousands of facts, or recall that needs to be semantic ("what do I know about X?") rather than exact — that's when you graduate to a vector store and retrieval. It's a genuinely different tier of memory engineering: embedding, chunking, ranking, and knowing what to inject when.