Nobody Reviewed Your Agent's Memory

Agents can “remember” things. Mine keeps a directory of notes about how I work, what the project is for and which mistakes I told it not to repeat. But I think most of what is in there belongs somewhere else.

The memory feature is a directory of text files the agent loads again at the start of the next session. The agent writes them too. It decides what is worth keeping as it works, in its own words and mostly without asking me first.

Recently I wrote that feedback needs somewhere to land and that every loop which used to close inside somebody's head now has to close outside it. A memory file looks exactly like the answer to that. It is outside someone's head and it survives the end of a session.

One machine, one person

My memory directory is on my laptop. It records what my agent concluded from working with me.

Nobody else on the team can see any of it. If a colleague's agent works out the same convention, it writes its own version on its own machine and nothing connects the two.

You can check this on your own team. Ask two people to have their agent explain one of your conventions and compare the answers.

I do not think you will get the same answer twice. You will get two versions of the house style, both stated with complete confidence. Each one is built from whatever that agent happened to see. Neither person has a reason to suspect anything, because each agent is consistent with itself.

The loop closed on one machine instead of inside someone's head and the only thing that changed is that there is now a file.

Nobody had to agree to it

Suppose the tooling improved tomorrow and every memory file synced to every machine on the team. It would still be a directory of conclusions nobody agreed to.

Team knowledge gets its authority from somebody proposing it and somebody else arguing with it. That happens in a pull request or in the meeting where two people disagreed about naming for twenty minutes and eventually agreed on a name. Which mechanism you use is not the point. I reach for pull requests because most teams have them. A pair who argue it out while they write the code are doing the same thing.

That argument is what makes the conclusion trustworthy. A convention in a repository is worth trusting because people who had reason to object read it and agreed to it.

A memory file has none of that behind it. It was written by an agent, from one session, about one person's work and then saved. Nobody proposed it and nobody objected to it. Nobody is accountable for it being wrong.

Nothing in the file tells you any of that.

We have seen this before

I opened this whole run of posts by claiming that you cannot prompt your way to quality and that what does the damage is accumulation. The damage is a pile of small things nobody fully reviewed rather than one dramatic failure. Each one is reasonable on its own and each one makes the next problem harder to see.

That claim was about code. The same thing happens to what a team knows.

A memory directory grows one file at a time and nobody prunes any of them, because pruning means having an opinion about every file and nobody had one about any of them.

Six months in, your agent starts every session with a set of instructions nobody wrote on purpose and nobody has read in full.

You can find out where yours stands in about five minutes.

What's actually in there

The idea for this comes from Olly Styles, who I met at a meetup. He has written about a similar exercise. The prompt below is my own version of it.

Start a new session in your project and give your agent this:

Read every file in this project's memory directory, including the index and each individual memory file. Do not skim.

Verify each entry against the current state of the repository, then sort it into exactly one bucket:

• Helpful: true today and it would change what you do on a real task.
• Harmful: false, out of date or it would lead you to do the wrong thing.
• Useless: true but inert. It restates what the repo already makes obvious or it is too vague to act on.

Be harsh. You wrote these notes in earlier sessions, so treat them as a stranger's claims you have to check, rather than conclusions you already trust.

Give me a count per bucket, then list every entry under its bucket with one line saying why. Do not edit or delete anything.

The prompt has now been run on six memory directories. I ran two of them and other people ran the rest. Four come from one repository, a client where four of us work on the same code.

None of us can see anybody else's directory and it shows.

Six directories is not a study and I would not want you to read it as one. But not one of them came back clean.

Client B's sixteen harmful notes are the ones I would worry about. They are wrong and the agent loads them at the start of every session and works from them on a codebase the team is shipping.

They are rarely wrong in a way you would notice. Here is one from my own run, on a client repository. A note recorded that DNS was the last remaining blocker for getting HTTPS working. DNS had shipped. The records were in the infrastructure code, the domain was configured and it had been live for a while. Acting on that note, my agent would have told me HTTPS was still blocked on work I shipped myself.

That note was true on the day it was written. The work landed and nobody went back to correct the note. Most of the harmful pile looks like that. They are facts with an expiry date that nobody set, rather than mistakes.

The buckets are not a verdict. The agent sorts, you decide. What comes back is a shortlist of lines worth reading and for most of these directories that will be the first time anybody has had an opinion about any of them.

Harmful and useless are the easy ones. Delete them. What is left is the helpful pile and there are two kinds of notes in it. Some of it is about you and how you work. The rest is about the project and more than one person might need it.

Move it into the repository

There are products that will sell you a shared memory layer. I am not going to recommend one. That is not because they are badly built. Buying one means adopting somebody else's model of what your team knows and how it should be stored, before you have worked out your own.

You do not need to buy anything. Take the knowledge more than one person needs and move it into the repository.

A memory note was never a different kind of thing. It is text an agent reads at the start of a session and weighs against everything else in front of it, which is what a request is. Moving it into the repository does not change what it is. It changes who has seen it.

It is still a request, so keep going. Move it into the linter and it becomes a rule. Move it into a hook and it gets applied without anyone having to remember it. Write a check for anything you have caught twice, which is what I was late to.

Every one of those steps lives in the repository, so somebody reviewed it. A lint rule you added had a reviewer. So did the type that makes the bad state impossible. The memory directory on your laptop did not.

The first of those steps has a practical problem and you may be thinking about it already. An instructions file long enough to hold everything is too long to be read carefully. Keep it short and point at the rest, so the detail sits in separate files the agent loads only when the work needs them. A linked file is still a request and it competes for attention like any other, but it competes as one page instead of twelve.

None of this has anything to do with agents. Knowledge the team can rely on is knowledge the team has argued with. Everything else is one person's note, wherever it happens to be stored.

Write it down yourself

Plenty of what an agent records about me is mine. One note is about how I like prose written. Another is that I do not need the fundamentals explained to me. A third is that I get annoyed when it offers the same next step twice. None of that belongs to a team and committing it to a repository would be worse.

None of it belongs in a memory directory either. Those are all decisions I made, so they belong in a file I wrote and can edit, not in notes an agent took about me. For me that is ~/.claude/CLAUDE.md and the other tools have their own equivalent. A convention about how a codebase handles errors is not a preference at all and it belongs in the repository. If that convention lives only in my memory directory, my colleague's agent is free to decide otherwise and be just as confident.

Do all that and the memory directory is empty. If your tool lets you switch the feature off, I think you should. Not every tool does, and where you cannot, run the prompt above from time to time and treat what comes back as a list to clear.

This does not cover everything. Some of what a team knows cannot be turned into a rule at all. That knowledge lives in people. You learn it by working alongside them. I do not have a trick for that and I am suspicious of anyone selling one.

You can still tell the two apart. If it can be checked, check it and let the check go through review like everything else. If it cannot, then it was never going to survive in a text file on one laptop either.

Read it before you trust it

A memory file is a draft nobody reviewed. It is one person's notes, useful to that person and authoritative for nobody. As soon as something in there affects more than one person, it needs to leave the memory directory and go somewhere people can argue with it.

That means opening a pull request, same as everything else.