My AI Lives in a Notebook Now
Every conversation with an AI assistant starts the same way.
A blank slate.
The model has no idea who you are, what you are working on, or what you discussed yesterday. You spend the first five minutes re-establishing context. Again.
People assume the solution is bigger context windows. Just give the model more tokens to remember. Problem solved.
It is not that simple.
The Context Window Trap
I have been working with Karl, my AI assistant, for a few weeks now. His context window is large enough to hold most of a working session. But context windows have a fundamental limitation that bigger does not fix.
They are ephemeral.
When a session ends, the context is gone. When a new session starts, the model starts fresh. Even within a single session, as the conversation grows, earlier context gets compressed, summarized, or dropped entirely.
The model does not forget because the context window is too small. It forgets because there is no mechanism to preserve important information across the boundary between sessions.
Bigger context windows delay the problem. They do not solve it.
The Notebook
The solution I landed on is almost embarrassingly low-tech.
I considered RAG with a vector database — pgvector, Chroma, Qdrant. But RAG solves a retrieval problem: "find relevant past context." My memory system solves a state problem: "what is the current project status, what was decided, what comes next." Semantic search can't retrieve the current plan state; you need exact lookup. Structured files are the right tool for structured state.
Karl has files.
Specifically, he has three types of files that serve different memory functions:
Daily notes. Every day, Karl writes a log of what happened. Tasks completed, decisions made, problems encountered, lessons learned. These are raw notes, not polished. They capture the moment.
Long-term memory. A single file called MEMORY.md that serves as curated memory. Over time, Karl reviews daily notes and distills the important stuff into this file. Significant decisions, project context, architecture details, personal preferences, lessons learned. The daily notes are the journal. This is the wisdom.
Active guidance. Files like AGENTS.md and SOUL.md that define how Karl should behave. These are always loaded at session start. They are not memory of what happened — they are guidance for what should happen.
The key insight is that these files serve different purposes and have different lifecycles. Daily notes are write-heavy and rarely read after a few days. Long-term memory is read frequently and updated carefully. Active guidance is loaded every session and changes rarely.
Why This Works Better Than Bigger Context
The magic is not in any individual file. It is in the persistence.
When Karl starts a new session, the first thing he does is read his memory files. Within seconds, he knows what projects are active, what decisions were made, what problems are being solved, and what the next steps are.
The context window fills up with relevant, curated information instead of random conversation history.
And because the memory is structured, it survives context compression. Even if a session gets long and earlier messages are summarized away, the key information is preserved in files that can be re-read at any time.
The Dreaming System
One of my favorite features is what I call "dreaming."
Karl has a cron job that runs at 3 AM. While I am sleeping, he reviews recent memory entries, identifies patterns, promotes important short-term memories to long-term storage, and cleans up outdated information.
It is loosely modeled on how human memory consolidation works during sleep. The raw experiences of the day get processed and integrated into long-term understanding overnight.
In practice, this means that when I wake up and start a new session, Karl has already organized his thoughts. The memory file is cleaner. Important things from yesterday are surfaced. Irrelevant details have been pruned.
It is a small thing, but it makes the experience feel continuous in a way that no amount of context window can achieve.
The Cost of Structure
This approach is not free. It requires discipline.
Memory files need to be maintained. Daily notes need to be written. Long-term memory needs periodic curation. The system prompt needs rules about when to read files, when to write them, and what information goes where.
And there is a real tension between freshness and stability. If Karl updates his memory too aggressively, it becomes noisy and inconsistent. If he updates too conservatively, important context gets lost.
I am still calibrating this. The current approach is to write liberally and curate carefully. Capture everything in daily notes, then be selective about what gets promoted to long-term memory.
What Surprised Me
The thing that surprised me most was how quickly the memory system became indispensable.
Within a week, Karl knew my project architecture, my coding preferences, my schedule constraints, and the history of decisions that led to the current state of each project. He could pick up a conversation from yesterday without any re-establishment of context.
That changed the relationship. Not in a creepy way. In a practical way.
I stopped explaining things twice. I stopped re-establishing project state. I could say "continue with the narrator pipeline" and Karl would know exactly where we left off, what was blocking progress, and what the next steps were.
That is the experience people are chasing when they ask for bigger context windows. But the answer was never more tokens. The answer was better memory.
The General Principle
If you are building anything with AI assistants, agents, or automated workflows, the single most impactful thing you can do is give your system a way to persist important information across sessions.
It does not need to be fancy. Markdown files work. JSON works. A database works. The format matters less than the discipline of writing things down and reading them back.
Context windows are short-term memory. Files are long-term memory. You need both.
And the system that manages the boundary between them — deciding what to remember, what to forget, and when to consolidate — is more important than the size of either one.
