Load personal memory on every turn, and rewrite it whole

A user says, "My account in the issue tracker is this ID. From now on, use it to find my issues." The model answers, "I'll remember that." The next day, in a new conversation, the user asks "show me my issues," and the model either asks for the account again or searches with someone else's account that has a similar name. An LLM API keeps no state, so when a conversation ends its context does not carry into the next one. The model said it would remember but stored nothing, and the only thing that survives to the next conversation is what the system wrote down somewhere.

So the facts the model always needs go into one text block per user, and that block goes into the system prompt on every turn. A turn is one exchange: one question in, one answer out. In the same turn where the model says it will remember, it rewrites the whole block and saves it. Past conversations are stored separately and come out only when the model searches for them. This post covers the criteria for splitting memory into those two, and the rules for writing and editing the block that is always loaded.

#A fact you always need gets missed when it sits behind search

The common approach is to put every fact pulled from conversations into a searchable store and load the memories that resemble the question into the prompt. It has no limit on volume. The trouble comes when a fact you always need does not resemble the question. "Summarize the ones I own from last week's deployment" contains nothing that looks like an account ID, yet finding the user's issues requires that account. If search does not bring it back, the model builds its answer without knowing the account. Going the other way and loading every past conversation into the prompt overflows the context window quickly.

So the two kinds of memory reach the model by different paths.

All personal facts go into the one block that is always loaded. Observed in production, even the longest block stayed under 500 characters. With that much room left in the block that is always loaded, there is no reason to split personal facts again into an always-loaded part and a searched part.

Facts about data belong to table notes (“Corrected column meanings, kept as table notes for the next user”) and to the organization and personal glossaries (“Revenue differs by team, so questions carry the approved definition”), and isolation between organizations is covered in “RLS guards org boundaries; the table classification decides deletion”.

#The personal memory block goes into the system prompt, not the first message

The block could also be slipped into the conversation's first user message. We chose the system prompt for two reasons.

The first is compaction. When a long conversation nears the context window, compaction summarizes or drops older turns. It shrinks only the conversation history and leaves the system prompt alone. A block in the first message gets blurred in a summary or cut off, and the longer the conversation runs, the more likely that becomes.

The second is freshness. The system prompt is rebuilt on every turn. Edit the block in another conversation, and the next turn of this one already has the change.

The block's header shows the current character count against the limit. Invisible characters are stripped when the block is loaded. This block is the only path by which a sentence the model read in a document or web page can be copied into the system prompt.

#The turn that says "I'll remember" also saves

The chat agent's base system prompt addresses this gap directly. When the user gives or corrects the name they want to be called, the prompt tells the model to record it with the memory write tool in that same turn, and it says why. Saying you will remember stores nothing; without the write, it is gone when the conversation ends and the user has to say it again.

Accounts in outside systems come with an extra condition. The model writes an account to personal memory only when it has been confirmed, either by that service's current-user tool or by a lookup keyed on the user's whole login address. It does not write a guessed ID or one spelled almost the same. Once written, the account is what every later conversation uses to interpret "my issues," so a wrong entry becomes a lasting error that shows someone else's issues as the user's own.

The tool description also says what not to write. Anything tied to one date, one ticket or one question stays out. The block is loaded into every future conversation, so a line like "focusing on project A this week" keeps being loaded as fact long after it stops being true.

#Edits rewrite the whole block

There is one tool for editing personal memory, and its only argument is the block. The model sends the entire block it wants to end up with.

A tool that appends one line at a time is cheap to call and clear in intent, but it never looks at what is already there. Similar lines pile up with small differences, and lines tied to a moment never get removed. When the model has to rewrite the whole block, it reads the block already in front of it, edits it and writes it back. Merging duplicates and dropping stale lines happen in the same write. The tool description asks for that too: if a line already covers what you are about to add, edit that line instead of adding a new one.

The risk of a whole rewrite is dropping a line by accident. We expose that rather than prevent it. Along with the saved block, the tool result returns the lines this write removed and the lines it added, and the model checks the removed lines to confirm each removal was intended.

Three edge cases are fixed in advance.

• An empty block is not saved. The existing content stays and a warning comes back. In a whole rewrite an empty string means "delete everything," and that must not happen because of a model mistake.
• Deleting everything takes a separate delete operation, and it removes the row itself. That keeps "nothing remembered" from taking two shapes, an empty-block row and no row.
• When two writes collide, the later save wins.

The alternative for that last case is optimistic locking with a version value. It is stricter, because it stops lost updates at the storage step. But observed in production, personal memory writes came a few times per several thousand turns. Making the model carry a version value on every write cost more than the rare conflicts it would prevent.

Every write goes to the audit log. The store records it in one place, rather than the tool and the settings screen each doing it, and it requires the caller to say whether the writer is the model or the user. A rewrite records only the changed lines, truncated; a full delete records only how many lines were removed. Keeping deleted content in the audit log would make the user's deletion untrue.

#The limit is 4,000 characters, and a block over it is rejected

A block over the limit is rejected, not truncated. Silent truncation would make the end of the block disappear without anyone noticing. A model that gets the rejection warning merges or trims lines and writes again. The model decides when to clean up by reading the character count in the header.

The block is unstructured text, so "one line is one entry" is a habit, not a contract. If the model adds a Markdown heading, that line counts as an entry too. Rather than count something that cannot be counted, we measure characters only.

The character count is computed the same way, with leading and trailing whitespace stripped, when the limit is checked, when the prompt header is written and when the settings screen shows it. If the methods differed, a single trailing newline from the model would make the screen and the prompt disagree, with no way to tell which is right.

#Users edit the exact text the prompt carries, in settings

The block reaches the model in the user's name on every turn. So Memory in settings shows the raw block and lets the user edit it or write it from scratch. The string on the screen is the string in the prompt. Saving from the screen goes through the same save function and the same limit check as the tool.

The settings screen can stay open for minutes, and in the meantime the model may change the block in another conversation. The server accepts whichever save comes last, so the screen handles this conflict.

1. When the user clicks save, the screen rereads the server's content and compares it with the content from when editing began.
2. If they differ, it does not save and shows "Memory was changed elsewhere." The text being edited stays as it is.
3. Clicking save again overwrites with the edited text; clicking Load server copy discards the edits and loads the latest content.

Saving with an empty text box does not save and points the user to Clear all instead. Clear all asks for confirmation once more and cannot be undone.

#Past conversations come out only when the search tool is called

When a turn ends, a background job splits the turn's question and answer into 4,000-character chunks and stores them. Each boundary moves back to a whitespace position so that no word is split in half across two chunks, and a turn is indexed up to 10 chunks (40,000 characters). No LLM is involved. Summarizing every turn would cost that much on every turn, and a word left out of a summary could never find that conversation again.

Stored chunks are not loaded into the prompt automatically. When the model calls the conversation search tool, it runs keyword search and semantic search together over that user's chunks and returns the matches. Only the best-matching chunk from each turn is kept, so one long turn cannot fill the results. Chunks that look relevant are not preloaded on every turn, because most turns need no past conversation and an unrelated chunk muddies the current answer. When a question points back to an earlier conversation, such as "that sales query we wrote last time," the model is more accurate if it recognizes that and goes looking.

The search scope is fixed to the user. If the user's identity cannot be confirmed, no search runs and an empty result comes back. Chunks from a conversation the user deleted are excluded from search even before they are actually deleted.

#Design choices

Personal memory is loaded on every turn instead of searched. The reason is that a fact you always need is not missed when it does not resemble the question; the cost is paying for the block's tokens on every turn. That is why the block is capped at 4,000 characters. What happens to the prompt cache when the block changes is covered in “One system prompt edit zeroes cache reads, so changing blocks go last”.

What to remember is left to the model through the system prompt, because whether a sentence is worth remembering depends on the conversation's context. The cost is that the model can misjudge, and we absorb it by letting the user see and edit the block in settings at any time.

Editing is narrowed to a single whole rewrite, and the risk of dropped lines is absorbed by showing the removed lines in the tool result. Lost updates from concurrent writes are not prevented on the server. The model side, where writes are rare, lets the later save win, and only the settings screen, which can stay open for minutes, catches conflicts by comparing right before saving.

Past conversations come out only through the search tool. That avoids loading them on every turn, at the cost that they come out only when the model goes looking.

This design fits when what needs remembering is short and durable, like one person's preferences, preferred name and outside accounts. Read, edit and rewrite only works if the block fits under the limit and is loaded on every turn. If each user has thousands of characters of facts to remember, first decide which facts belong in the block and which should be retrieved by search.