Rerunning a chat repeats its mistakes. Keep the steps and working SQL

Say a chat once went well: it pulled last week's sales from the database, grouped them by department, and built an Excel file. That conversation holds more than the result. It also holds the quotes table the model queried first by mistake, the department grouping the user changed halfway, and a tool call that failed once. Replay the conversation next week and the model either goes through the same trial and error again or mixes last week's numbers, still sitting in its context, into this week's answer. Keep only a summary of the final answer and you lose the execution knowledge instead: which table, which conditions.

OwlAI turns a conversation like this into a playbook, a procedure that runs its fixed steps at a scheduled time with no one at the keyboard. Conversion keeps exactly two things from the conversation: the list of steps worth repeating, and the SELECT statements that actually succeeded in it. Everything that changes from run to run, such as dates, data, the run time, and whether there is anything to process today, is decided again at run time. This post covers what is kept, what is decided at run time, and where we drew that line.

#Replaying the chat breaks, and so does pinning the SQL

There are three common ways to automate repeated work. Feeding the whole conversation back as a prompt is easy to build, but as above it makes the model reread the wrong query and the changed requirement. Summarizing the conversation into a single instruction removes the trial and error, but table names and conditions tend to drop out of the summary, and the first run starts over with schema exploration.

The last way is to pin the SQL that worked and run it every time. It costs nothing and returns the same shape of result each time. But if a column is renamed, or a date condition is baked into the statement, the pinned SQL returns wrong results without an error. A report whose "last week" always means the same week piles up as a success every Monday.

Playbooks keep both the steps and the SQL, but treat the SQL as a suggestion for the next run, not as the answer.

#Conversion keeps a step list and the SELECTs that succeeded in the chat

Conversion starts from Create Playbook above the chat input. A conversation that never used a tool has no execution traces to carry over, so the button is disabled. Mobile screens do not show this button; there, you ask for a recurring task in chat and the model calls a tool that creates the playbook.

The conversion model produces a list of steps. Step types are fixed: reading documents, database query, organizing content, creating documents, work in an external service (a tool call such as a Jira comment), approval, and a few more, and each type has inputs that must be filled. Steps run top to bottom in a single line, with no branches. A database query step holds a natural-language question such as "sales by department last week" and the target database, not SQL. The model's output is saved only after it passes structural validation; the retries that fix validation errors are covered in “Keeping old answers out of edited questions with a conversation tree”.

SQL is handled separately. Code collects the read-only queries this conversation ran from its record, keeping only statements whose result came back as a success (up to 20). The model picks one statement from that list for each database step. If the statement the model attaches is not on the list, code drops it, because an edited or invented statement would be SQL that was never confirmed. A statement that survives is checked once more for being read-only and becomes that step's saved SQL. Nothing is executed again during this process.

So a playbook converted from a conversation does not redo schema exploration and SQL writing from scratch, even on its first run.

#Saved SQL rides into the next run as a suggestion

When a database step runs, its saved SQL goes to the model along with the step's instructions, together with when it was saved and how many times it has been reused. The model can run it as is, edit it, or write a new one. Execution always uses the data as of that moment, and permissions are checked again against the person running it every time.

When it finishes the step, the model also submits the SQL for the next run. If it used the saved SQL unchanged, it just says so; if it changed anything, it submits the new statement. In a new statement, values that change from run to run, such as dates, become placeholders of the form {{name}}, each with a type and an example value. There are five types: integer, number, string, date, and boolean. These are the types that have a clear rule for turning a value into a safe SQL literal.

The engine does not execute the submitted SQL. It checks only these three things:

• No placeholder is wrapped in quotes
• Every placeholder has a description
• The statement, built with the example values, is a single read-only SELECT

A quoted placeholder turns '{{start}}' into ''2026-09-01'', a syntax error. That error would surface only on the next run, so it is blocked at submission.

Turning off Pin the SQL on the edit screen stops both the suggestion and the saving. That step writes its SQL fresh on every run.

#Schedules are stored as wall-clock rules in the time zone the user picked

The run time is also computed again for every run. Turning "every Monday at 9" into an exact time is not left to the model. The conversion model emits only the recurrence type, the time, and the days of the week, and code computes the first run time in the schedule's time zone. Recurrence is an interval plus a unit (hours, days, weeks, months, years), and weekly schedules also pick days of the week.

Convert the rule into a single UTC cron and register it with a queue, and it breaks wherever daylight saving time (DST) applies. A job registered for 9 a.m. daily in New York runs at 8 a.m. local time once the clocks fall back, because one cron cannot express a local time whose offset from UTC changes with the seasons. Converting only the time to UTC and keeping the user's chosen weekdays is also wrong. Weekdays at 08:00 Korea time is 23:00 UTC the previous day, so registered that way the job runs Tuesday through Saturday.

So the recurrence rule is stored as a wall-clock rule in the time zone the user picked (the date and time that region's clocks show), and the next run time is computed in that time zone. A single job that runs every minute decides what runs:

1. It finds schedules whose next run time has passed.
2. It locks the schedule row and rechecks the next run time under the lock.
3. It compares against the previous run time and stops if that occurrence has already run.
4. It creates the run row and recomputes the next run time from the wall-clock rule.

Steps 2 and 3 make one occurrence run once, even with several workers or a retried job. If workers stop and an occurrence slips by more than 10 minutes, that occurrence does not run. A recurring schedule moves on to its next occurrence, and a one-time schedule simply ends. A report that runs long after the time the user picked is not what the user asked for. For the same reason, enabling a schedule after its start time makes a recurring schedule start from the next occurrence, and a one-time schedule cannot be enabled.

#A day with nothing to do is not a failure

The last thing that changes from run to run is whether there is anything to process today. When it finishes a step, the model reports one of three outcomes. If it did the work, the step is "Done" even with zero results, and the run moves to the next step. If the remaining steps are pointless, "Stopped" ends the run, and that does not count as a failure. Only when it could not do what it had to is the step "Failed." "Stopped" is available only on steps where May end the run here is turned on in the edit screen, so the person who built the playbook, not the model, decides where a run may end.

If a database step reports "Done" without a single successful query, the engine asks once more. Usually the model then actually runs the query. If it says the same thing a second time, the step is accepted with a "zero processed" result and the run continues. Failing a step that had nothing to query because the previous step returned zero rows would fail the whole run.

When a run ends, the person who ran it (the schedule's owner for a scheduled run) gets a push notification. Notifications come in the same three kinds: completed, stopped, and failed. Announcing a stopped run as finished would make the owner believe the report was produced. A failure notification includes the last progress sentence the model wrote, so you can tell where an unattended scheduled run stopped.

A scheduled run blocked by the pre-run check also triggers a failure notification. Without it, the owner would not learn that a schedule is blocked every day until opening the run log. The notification is sent only after the scheduler's transaction commits, so no notification points to a run that was rolled back and never existed. If the owner is over the usage limit or the definition is missing required inputs, no run is created; the playbook is turned off automatically and the edit screen shows why.

For the same reason, the enable, schedule, and run paths check more strictly than saving does. During editing, a step with empty required inputs can be saved, but a playbook with empty required inputs or no steps at all cannot be enabled. An empty playbook, once enabled, would finish every run as a success after doing nothing, and the owner would believe the work was running. While a playbook is enabled, its edit screen is read-only. If it could be edited while on, a definition that never went through the enable-time check would stay on the schedule.

#Design choices

Execution order is fixed to a single list order. A branching graph could express a different path for each condition, but someone would have to edit and validate the edges and conditions. The model makes condition-dependent decisions inside a step, and the person who builds the playbook marks where a run may end early with "May end the run here."

SQL from the conversation is saved verbatim, without passing through the model, so a statement whose results the user checked in the conversation does not change. The cost: if the chat SQL contains a literal date, the first run receives a statement with that date in it as its suggestion. Once the model, seeing the suggestion, fixes the date when it runs and submits a new statement with a placeholder, that statement is what later runs receive. Code also checks only whether a statement succeeded in this conversation. A query that succeeded against the wrong table is on the list too, and the model decides which statement goes with which step. Turning that judgment into code rules would make the rules wrong for every conversation shaped differently. Instead, the saved SQL is always a suggestion, so the model on the next run can fix it by looking at the question and the results.

Submitted SQL is not executed; only its shape is checked. Running it once more at submission would catch meaning errors earlier, at the price of one more query against the database for every step. Whether the statement means the right thing is judged by the model from the results on the next run.

Recurrence accepts only values it can express exactly. Hourly intervals must divide 24 (1, 2, 3, 4, 6, 8, 12, or 24 hours), and a 5-hour interval is refused. Storing a period that is bound to drift at the day boundary as some nearby value would make the schedule run on a period the user did not pick, with no error.

Sending results to other people is a step in the playbook. The model calls an email or messenger tool the administrator has connected, and that call goes through the same approval as any other external write tool. We chose this over a feature where the product takes a recipient list and sends results on its own. With such a feature, there would be a second way for data to leave besides tool calls, and approval and personal-information masking would have to be enforced in two places. The cost is that sending has to be written down as a step, and if the tool requires approval, it is reviewed when the playbook is first enabled and again whenever the definition changes.

If you are building a feature that turns work that went well once in chat into a recurring job, keep only the successful execution traces rather than the whole conversation, and treat those traces as suggestions the next run can fix. The more your data and schemas change between runs, the more that matters.