Auditing an AI agent: log every query it runs, not the conversation

Open your internal databases and documents to an AI agent, and after an incident a security officer will ask: who on the sales team used the AI to query the customer table last month, and did that person have permission to see it? Code blocks out-of-permission queries before a tool runs (“Permissions are enforced in code, judgment is left to the model”). But you still need to show later that something was blocked, and which queries weren't. Keeping full conversations as evidence gives administrators a way to read other people's conversations, and the evidence disappears the day the conversation is deleted. The source database's own log records every query under a single connection account, so it can't tell you who asked.

So the audit log records one call in which the AI touched an asset, not a conversation. One SQL execution, one document search, and one connector tool call each get one line, and each line says who touched which database or connector, when, and with what result (success, denied, or failed). Questions, answers, and query result values are not recorded. This post covers what is recorded and in what shape, how administrators opening those records and permission changes are recorded, and how we made sure the app can't alter the log.

#Conversations and source database logs can't say who queried what

Archiving whole conversations is simple, and it even keeps why the model queried what it did. But a conversation holds the asker's own words and the query result values as they are. Opening a conversation to audit it means the administrator reads content that isn't under audit. Conversation lifetimes don't match audit needs either. A conversation is deleted one year after its last activity, or 30 days after the user deletes it. The audit log has to outlive that.

The source database's audit features (pgAudit for PostgreSQL, unified auditing for Oracle, and so on) are trustworthy because they record what the database actually ran. But OwlAI connects to a directly connected database with one connection account for the whole organization. The database log records every user's queries under that same account. What OwlAI does instead is attach two markers so a DBA can pick out the statements OwlAI sent.

• Every statement the executor sends to a customer database starts with the comment /* OwlAI generated */. DBAs find the AI's statements in pg_stat_statements or V$SQL by this comment.
• The connection announces the client name owlai-dbexec. It shows up as application_name in PostgreSQL and program in Oracle, so OwlAI's sessions are recognizable in session views.

The markers only say that a statement came from OwlAI. Who caused it to run is answered by OwlAI's audit log. Match a statement a DBA found by its marker against the SQL execution record from the same time, and you get the person who ran it.

#Each SQL execution is one line, linked to its conversation only by value

Every call to the SQL execution tool writes one line to the audit log. Running the same SQL twice in one turn writes two lines. Each line holds the following.

• The time and the person who ran it
• The database connection
• The result: success, denied, or failed. Denied means a permission check or the read-statement check stopped it before execution; failed means it ran and the database returned an error.
• The number of rows returned (on success only)
• The original SQL text (truncated if long, with a mark saying so)
• The list of tables extracted from the SQL
• The source: chat, or a run of a particular playbook

The values in returned rows are not recorded, so the log never becomes another copy of the data.

Recording denials is the heart of this log. When the model sends SQL to a database it has no permission for, the execution tool refuses before running it, and the attempt leaves one line marked Denied. Security officers see blocked attempts in the same list as successful queries. A sudden rise in one person's denials signals either that their role doesn't fit their job or that someone is trying to use the AI to query beyond their permissions.

The line refers to the conversation, turn, and playbook run only by value. There's no foreign key to the conversation, so the line stays for the audit retention period even after the conversation is deleted. The table list is an index extracted by scanning the SQL. We chose to record it without sending another query to the source database, so a query through a view or synonym is recorded under that name, not the underlying table. The authoritative record of what ran is the SQL text stored alongside it.

Administrators see this log in the SQL executions tab of the activity log screen. They filter by period, user, database connection, and result, search the SQL text and table names, and export matching rows to CSV, up to 5,000 rows. Questions and conversation titles aren't in this tab, so search covers only the SQL text and tables.

Jobs that run without a user identity don't appear in this log. A schema analysis an administrator starts is one. Analysis isn't a person's query but work to understand that database, and starting it is recorded as an administrative event with who started it on which database.

#Document searches and connector calls record only what they touched

Document searches and connector (an MCP server your company connects to OwlAI) tool calls get one line each in the same shape. What is and isn't recorded differs by record.

Connector tool arguments are not recorded because they contain email bodies, issue comments, and customer data. Recording them would pile that content up in the audit log and turn it into a place where administrators read other people's mail. A denied connector call also covers calls stopped before they reach the remote server, such as a call blocked at execution because the role doesn't allow that connector.

Each decision a person makes on a tool that requires approval (“Told to confirm first, the model still sends the email, so approval happens right before execution”) also gets one line: approve, "Always allow," reject, and expiry in the approval prompt; calls approved without asking because of an "Always allow" record; calls blocked without asking after repeated rejections of the same call; revoking "Always allow"; and granting and revoking playbook pre-approval. These decisions are audit events and appear on the same tab as the viewing events below, and for the same reason as connector calls, tool arguments are not recorded.

#Permission changes record what was opened and what was closed

A denial has to be read against the permissions in force at the time. So permission changes are recorded too. When an administrator changes a role's database, document collection, or connector permissions, one line lists the targets that request opened and the ones it closed. For a database permission, it also says whether it covers the entire database or selected tables.

Permissions also change without a person.

• The upgrade that moved role permissions to the connector level deleted the old role-specific tool rules (“Customers know their connector tools' risks, so tool policy is per install”). It left one line per deleted rule with the role, the target tool, the deleted decision (allow, require approval, or block), and the reason. Deleting and recording happen in one SQL statement, so a rule can't be deleted without being recorded.
• When a connector server adds or removes tools, what the AI can use changes without anyone editing a role. So every real change to the tool list records the tool names added and removed and what triggered it (who was connecting when the tools came in, or whether a cleanup job removed them).

#Administrators opening someone else's records is recorded too

The people who can see the audit log are administrators, and administrator screens show other people's activity. Playbooks are shared automation, so run details show node outputs and parameters, and answer feedback comes with the conversation the user sent along with it. So an administrator opening someone else's records is itself recorded as a Viewed event.

This covers the SQL execution list and details, the playbook run list and details, the user list, the answer feedback list and feedback conversations, and the share link list. Each event line holds the viewer, the time, the IP address and browser they connected from, and the kind of resource opened, plus the resource's owner when a single item was opened. The content viewed is not included. Opening a list writes one line per request, with the number of rows returned and the filters applied. One line per row would turn a single page into dozens of lines and make the audit screen unreadable. Viewing only your own records isn't recorded.

These fields follow the five items Korea's Standards for Personal Information Security Measures require in an access log (identifier, access time, access location, the data subjects processed, and the task performed). On-premises, you find them as the Viewed type on the System events tab of system status; in OwlAI SaaS, under the activity log's Admin events tab, as the Viewed type.

When an OwlAI SaaS operator opens a customer organization's feedback conversation, the bar is higher. The view is recorded both in the platform audit log and in that organization's activity log, and if either write fails, the conversation is not returned. The organization's administrators see on their own screen what the operator opened and when.

#The app can't alter or delete the audit log

If the app that writes the log could delete it, one bug or one compromised account in the app would be enough to erase the evidence. So the database decides who can touch the audit tables. The database role the app uses gets only INSERT and SELECT on the audit tables, never UPDATE or DELETE. Only a dedicated retention role can delete rows, and an RLS policy limits even that role to rows older than 1,095 days (3 years). The screen has no way to delete records.

You can lengthen the retention period with observability.accessLogRetentionDays. The default is 1095 days, and the server refuses to start if the value is shorter. The check runs at startup rather than when the retention job first runs, so no installation operates believing records are deleted on a shorter schedule than they are. Three years matches the retention period the same standards require for records of granting, changing, and revoking access rights. “RLS guards org boundaries; the table classification decides deletion” covers how organization closure keeps the audit log instead of deleting it.

#Design choices

We chose per-call records over conversation transcripts. Keeping conversations would show even why the model picked a given table. The cost would be an audit log that lets administrators read other people's conversations, and copies that stay in the audit tables after a conversation is deleted. Per-call records can't answer "what did that email say?" They do answer "who ran which SQL on which database, and what was blocked?" even after the conversation is gone.

We read the table list from the SQL text instead of asking the source database. Querying the database's execution plans or dependency information would reveal the real tables behind views, but it means sending another query to the customer's production database just to write one audit line. So the list is a search index, and the SQL text is the authoritative record.

We record administrators' views instead of blocking them. On-premises, the administrator is the person who creates database connections and roles, so a mechanism that blocks their views has no real effect. We chose not to block them, but not to let them hide either.

The retention floor lives in two places: a configuration check and a database policy. With only the configuration check, a job connected directly to the database could delete rows inside the floor; with only the database policy, a setting shorter than the floor would lose its effect without any notice.

If you run an AI agent where many people query a database through one connection account, the source database log can't tell those people apart. Record person, asset, and result for each tool call, put denials in the same list, and record who opened that record, and you can show the AI stayed within permissions without keeping a single conversation.