OpenAI stores responses by default, so we resend the conversation

The OpenAI Responses API stores every response on the provider's servers unless the request sets store, and the official documentation says a stored response stays for at least 30 days (checked 2026-11-03). A stored response is convenient: the next request can continue from it with a single ID. But business conversations mix in database query results, internal documents and meeting minutes. If that content also sits on the provider's servers, a security review asking where conversations live and for how long now has two places to answer for. Then deleting a conversation in OwlAI would not delete the provider's copy.

So OwlAI sets store=False on every call and uses neither previous response IDs nor OpenAI's conversation objects. The original conversation context lives in one place, OwlAI's database, and every call assembles the full context from that database and sends it. This post covers what that design protects, what reasoning models need on top of it, and what it costs.

#There are two ways to leave a conversation with the provider, and both require storage

OpenAI offers two ways to keep conversation state on its servers. The first chains by response ID: put the earlier response's ID in previous_response_id on the next request, and the server stitches the stored input and output onto it. The second is a conversation object (the Conversations API). Conversation objects and the items attached to them are not subject to the 30-day retention limit.

The advantages are real. The client sends only the new input, so request bodies stay small. You need no code to store or assemble the conversation context. If the connection drops mid-stream, you can fetch the result again by response ID.

Both approaches only work if the provider stores the conversation. A response created with store=False is not kept on the server, so the next request cannot chain from its ID. The documentation's own stateless example resends the full earlier conversation. Turning storage off and leaving the conversation with the server are mutually exclusive.

#With storage off, the conversation lives in one place: OwlAI's database

OwlAI turns storage off. Every Responses API call carries store=False, so no OwlAI conversation is kept in the provider's conversation store. The conversation context exists only in OwlAI's database, which in an on-premises installation is PostgreSQL on the customer's own servers. Where a conversation lives and for how long can be answered by looking at that one database.

The setting's scope is narrow. store=False does not mean the provider never sees the content. When you use a model from an external provider, every call sends the conversation context and the question to that provider. The setting decides only whether the response is kept in the provider's conversation store. The provider also retains other records of its own. OpenAI keeps abuse-monitoring logs for all API usage for up to 30 days by default, and Zero Data Retention (ZDR), which keeps customer content out of those logs, applies only to organizations or projects approved in advance (checked 2026-11-03). Which terms apply is set by the customer's contract and account settings with the provider.

For work where data must not leave the customer's network at all, use a model from a provider inside that network. Data marked "Internal models only" is not visible to conversations running on external models (“External models never see internal-only data, not even its name”).

#Reasoning is conversation state too, so it comes back encrypted and goes out unchanged

Reasoning models add one more thing to carry over. The reasoning a model does before calling a tool is part of the conversation state, and OpenAI recommends passing earlier reasoning items back in the next request after a tool call. If the server does not store the response, the client is the only place left to carry that reasoning.

For calls that are not stored, OpenAI returns the reasoning encrypted inside the response. OwlAI sends include=["reasoning.encrypted_content"] on every call and stores the encrypted reasoning items in its database along with the rest of the response. The next call puts those items back in its input exactly as received. Only the provider can decrypt the block; OwlAI only keeps it. Reasoning blocks from Anthropic models work the same way: OwlAI resends them exactly as received so the signature attached to each block stays valid.

#Resending every time is the only path that works across all three providers

Leaving the conversation on the server works only with OpenAI. Anthropic's Messages API is stateless, and its documentation tells you to send the full conversation on every request. vLLM accepts the Responses API format but ignores store by default. If you enable it in the server settings, vLLM keeps responses only in process memory and never removes them until the server shuts down (vLLM source comment). Nothing is written to disk, so a restart loses them, and memory grows with everything stored.

That makes holding the conversation context on OwlAI's side the only approach that works with all three providers. Whatever the provider, the conversation context is assembled from the database the same way, and the only difference is the step that converts it into that provider's format just before sending. The stored conversation stays as received, and only the outgoing request is adjusted, so switching models within a conversation keeps the earlier conversation (“A 400 that sticks: add tool results on write, fix shapes on send”). Had the conversation been left on OpenAI's servers, moving to an Anthropic model would leave no way to bring it along. Models can be switched only within the provider tier the conversation started in (“External models never see internal-only data, not even its name”).

#OwlAI's database shows what the model saw when it answered

Because the conversation context is in OwlAI's database, you can check what the model saw when it gave a particular answer without asking the provider. Each call leaves these records.

These records stay in OwlAI's database regardless of the provider's retention period or account permissions.

Retries start from the same conversation context. When a failed call goes out again on the same model, or when a user's personal key fails authentication and the call switches once to the shared key, the request already carries the full conversation context. Nobody has to work out which key created the earlier response.

#Resending makes the request body larger, but input tokens are billed the same

The cost of this design is the request body. Every call sends the full conversation context instead of one response ID, so requests grow as the conversation does. Input-token billing, though, is the same either way. OpenAI's documentation says that even with previous_response_id, all earlier input tokens in the chain are billed as input tokens, because the server stitches the earlier conversation in and the model reads it.

Input piling up with every call costs the same wherever the conversation lives; “A turn's input tokens grow with the square of its rounds”, “One system prompt edit zeroes cache reads, so changing blocks go last” and “Compaction thresholds come from the input limit, not the context window” cover how to hold that cost down.

#Design choices

We paid in request body size and fixed the conversation to one place. The original conversation context lives only in OwlAI's database, and models from every provider are called through the same path.

We gave up one way of recovering a broken stream. Fetching a result by response ID only works if the response is stored on the server, and OwlAI stores none, so there is nothing to fetch. When a stream ends without a completion event, OwlAI does not attempt the lookup and classifies the failure as a stream disconnect right away. The lookup would return a 404, that 404 would be classified as an invalid-request error, and it would hide the real cause, the dropped stream.

Server-side compaction leaves a portability cost. The compaction results that OpenAI and Anthropic return are encrypted blocks that only models from the provider that produced them can read. According to OpenAI's documentation, server-side compaction retains no data on calls with store=False, so compaction still works with storage off. Moving to another provider, however, means expanding the pre-compaction original and sending that, so the first request after a model switch is larger (“A 400 that sticks: add tool results on write, fix shapes on send”).

This design fits when business data flows through conversations and your security review has to explain where that data stays and for how long. If conversations are short, data location doesn't matter and you use a single provider, leaving the conversation with the provider means less code.