Mid-turn messages sit in a DB row, and a row lock picks who takes them

A deep research turn runs for up to an hour by default. A turn is one exchange: a question comes in and an answer goes out. Say four researchers, the default limit for those running at once, are gathering material on competitors when the user sends "Only look at domestic companies." Three parties can touch that one message at almost the same moment. The user may send another line or withdraw it with the cancel button, the worker may load it into the model's input on the next round, and the turn may end right then. If the three cross, the message vanishes without reaching any turn, a withdrawn message reaches the model, or the same words are loaded twice.

“One agent loop for every feature, with the differences in hooks” and “A 400 that sticks: add tool results on write, fix shapes on send” cover where the message goes: after every tool call from the previous round has its result, just before the next model call. This post covers the decisions after that: where the message is kept and who takes it, what happens at points where it cannot be loaded, and how the turn's limits are counted once it is in.

#The message is saved on arrival, and one queue row per conversation holds its state

The simplest design is a queue in the memory of the worker that runs the turn. But the web server receives the message and the worker runs the turn, so they are separate processes. With an in-memory queue, a worker restart (“Worker restarts: every running turn answers or records its failure”) loses the message, and a page refresh loses the bubble the user just sent.

So the web server saves the message to the conversation as soon as it arrives. A paired queue row holds the delivery state. There are only two states, queued and delivered, and each conversation has at most one queued row. If more messages arrive while one is waiting, they are appended to the existing message instead of creating new rows. Text is joined with one blank line, and attachments and @mentions are merged, keeping each file or target once. Send "Only domestic companies," then "Limit it to listed ones," and the model gets both lines as one message.

The worker reads the row straight from the database at the head of a round, the point where every call from the previous round is paired with its result. That one database row is the only place the message lives.

#Appending, taking and withdrawing lock the same row

All three actions lock the row with SELECT … FOR UPDATE, under the condition that it is still queued. The first to lock it makes its change and commits. The one that comes next reads the committed state and decides what to do.

An append that adds an attachment needs one more step. Messages with attachments are not loaded into the running turn (see below), and while the take waits for the lock, an append can add an attachment and commit. If the take judged attachments from the locking statement's condition, it would see the snapshot from when that statement started, before the attachment was added. So the take reads the message again after it holds the lock, and judges from that.

The remaining race is the moment a turn ends. If a message is accepted while the ending turn concludes "nothing is left" and lets go, the message is stranded with no turn to run it. A second lock closes that gap. Each conversation has a run row that records the one turn currently running, and both accepting a message and ending a turn lock this row first, in the same transaction, before looking at the queue row. That leaves two outcomes. If the accept comes first, the ending turn sees the message and starts a follow-up turn. If the end comes first, the accept is rejected because no turn is running, and the client resends the same text as an ordinary question.

When the user presses the stop button, a stop flag goes on the run row first, and then the waiting message is withdrawn and returned to the input box. Once the flag is set, no new message is accepted, so nothing can slip in after the withdrawal.

At the moment a message is loaded, its stored timestamp moves to the delivery time. The conversation view draws messages in time order. Without the move, a message accepted while the previous turn was still writing its answer would appear above that answer. The order the model received is the order on screen.

#A take that takes longer than 2 seconds at the round head is skipped

A worker runs many turns on one event loop. Running the take directly on that loop would stall every other turn on the worker whenever the database is slow. So the take runs on a separate thread, and its transaction gets a statement_timeout of 2,000 ms. It changes one row under a condition, so it normally finishes in a few milliseconds.

If it times out or the database errors, that round proceeds without the message and logs a warning. The message stays queued and is taken at the next round head, and if it never gets in, it becomes the request of a follow-up turn. Failing a turn that has run for 30 minutes because one added line could not be fetched would cost the user the answer to the original question too.

In deep research, the lead waits a long time outside the round head, because it does not call the model while waiting for researchers' results. During that wait the lead only reads, every 2 seconds, whether a message is ready to load; if one is, it ends the wait and takes the message at the next round head. There is a reason it does not take the message where it waits. If the turn is cancelled or hits its time limit after the wait ends and never reaches a round head, the taken message is marked delivered but loaded nowhere. A message sent while the user is reviewing the research plan card is read the same way and becomes a revision request, and the lead rewrites the plan.

#Answer-writing rounds and messages with attachments go to a follow-up turn

A turn has rounds that remove the tools and only write the answer: the round that writes an answer from what has been gathered once the iteration limit is reached, the round that retries an empty response once, and the round that asks again for a final answer whose format broke. Loading a new request there would force the model to answer it without tools, with the "wrap up now" instruction placed ahead of the user's message. So messages are not loaded in these rounds.

A message that misses them stays queued and becomes the request of a follow-up turn in the transaction that ends the turn. The follow-up turn is a new turn with every tool. The input box hint shows the difference. While the model is using tools it reads "Anything to add?", and once the answer text starts streaming it reads "Anything to add? It is sent as a new question after this answer." If the turn fails, any leftover message takes the same path to a follow-up turn, and the server runs the follow-up turn even if the browser tab is closed.

Messages with attachments or @mentions also go to a follow-up turn. The query scope a mention sets and the preparation of attached files are fixed when a turn starts. The attachment limit is counted on the merged message. Counted per message, two messages each within the limit could merge into one turn over it. If the merged count goes over, the message is rejected and the client shows the limit.

The follow-up turn runs as an ordinary chat turn even if the previous turn was deep research. One line that missed its round should not start another hour of research. If research is needed again, the user picks deep research again. If the follow-up turn cannot be put on the job queue, the message goes back to the queue, and any message that arrived meanwhile is appended after it, which keeps the order they were sent in.

#A loaded message is a new request, so the turn's limits start over

A chat turn has an iteration limit of 40 rounds and a time limit of 40 minutes. If a message lands on round 39 and the count carries on, the model gets the wrap-up instruction one round later. For a request that needs fresh queries under a new condition, that means half an answer. A new message from the user is a new request, so from the round where it is loaded, the turn counts as if it had just started.

What starts over: the round count, elapsed time, the tool call count, consecutive failures of the same tool, and the once-per-turn recovery mechanisms, such as retrying an empty response only once. If the old limits already sent a "wrap up" warning, that warning stays in the conversation context, so a line saying the warning no longer applies is added after the loaded message. Without that line, the model rushes the new request too.

The following do not change with a new message.

• The conversation context and the step numbers in Thinking
• Tokens and cost used
• Database permissions and mention scope granted in this turn
• Approvals the user denied, and the record of actions already run
• The image generation limit (a cost cap per user message, so we chose not to reset it)

#Thinking is split and stored where the message was loaded

When a message is loaded, the turn records a boundary step in Thinking, the reasoning view above the answer, and sends the client an event marking the boundary. The client splits Thinking at that boundary: steps before it fold under the original question, and steps after it fold under the added message. The boundary step is also saved in the turn's step record, so reopening a past conversation splits it in the same place, and you can still tell which steps answered which message.

To change a waiting message, withdraw it with the cancel button next to its bubble. The text and attachments return to the input box, and the user edits and sends again. A message that has already been loaded cannot be withdrawn.

#Design choices

We chose a database row over an in-memory queue. An in-memory queue is faster and simpler because it skips a database round trip each round. But it needs the web server and worker in one process, and it loses messages on worker restarts and page refreshes. With a database row, the row lock settles the races, and the cost is one database round trip of a few milliseconds per round.

Instead of inserting a message the moment it arrives, we wait for the round head. The cost is latency (“A 400 that sticks: add tool results on write, fix shapes on send”). If a slow tool is running, the message is loaded after that tool finishes, and if the answer text has already started streaming, it goes to a follow-up turn after the answer ends.

Each loaded message restarts the turn's limits. If the user keeps adding messages, one turn runs that much longer and costs that much more. So accepting a message goes through the same usage limit check as a new question.

We chose withdraw-and-resend over editing a waiting message. An edit would add one more action that changes the same row, and the race table would grow with it. Withdrawing and resending are two actions that already exist, and the table above settles every race between them.

If you are building a chat where the agent runs for minutes and accepts user input in the meantime, start by checking whether the side that puts messages in and the side that takes them out are different processes. If they are, the place for the message is a database row both can lock.