Get started

Engineering NotesDatabase connections and OwlDB

OwlDB connections hold a database ID instead of credentials

To open a company database to the AI, an administrator enters the host, port, account and password into OwlAI. Those values are a copy of the original, so when the database server moves, the account password is rotated or the cluster fails over, the next query fails to connect. At a company whose database management platform already tracks connection details for dozens of databases, the same information gets written down once more in OwlAI, and every change to the original means bringing two places back in line.

An OwlDB connection does not make that copy. OwlDB is a database management platform, and an OwlDB connection is one you pick from the list of databases OwlDB manages. On this connection, OwlAI holds only OwlDB's database identifier and list metadata; the connection details and execution belong to OwlDB. This post covers what an OwlDB connection receives, what it passes along at execution time, and which database-side changes OwlAI is left to follow as a result.

When connection details change, you fix the same connection

A direct connection is one where an administrator typed in the connection details by hand. Here, every change on the database side becomes a setting to fix in OwlAI:

  • Move the server, and you fix the address and enter the password again.
  • Rotate the password, and you fix the stored password.
  • Fail over, and you point the connection at the new primary node.

Until someone fixes it, the AI's queries fail. It is worse when the old node the connection points to is still up. Queries succeed and answer from stale data, with no failure to warn anyone (“AI queries pick their database node at run time, not at registration”).

Deleting the connection and creating a new one does not solve this. Each connection carries its schema analysis results, the permissions granted to each role and its table notes, and deleting the connection deletes them too. So when connection details change, the administrator edits the fields of the same connection. OwlAI has no way of knowing the original changed, so keeping track of when to edit is also on the administrator.

Imported connections fill in the address, but the password remains

A common way to cut down on copying is to import the database list from a cloud account. OwlAI can import Supabase and Amazon RDS databases this way, without copying addresses by hand. When an administrator approves OwlAI in Supabase or creates a CloudFormation stack in the AWS console, OwlAI reads the database list, fills in the address, port and TLS settings of the database the administrator picks, and locks them. Mistyped addresses go away, but two things remain:

  • The administrator still enters the account and password. Supabase does not reveal database passwords, and OwlAI does not read database passwords from AWS. OwlAI stores the password the administrator enters with envelope encryption and decrypts it to connect (“Database backups carry stored passwords, so the key lives elsewhere”).
  • When the address changes, the administrator uses Import again in the edit window to approve the same database again and fill in the new address, then enters the password and saves. Analysis results and permissions stay as they are.

Importing saves you from copying addresses. A copy of the password still lives inside OwlAI, though, and when the original changes, a person fixes that copy.

An OwlDB connection receives only an identifier and list metadata

When an OwlDB connection is created, what OwlAI receives from OwlDB is the metadata of its database list.

Received Not received
Database identifier, display name, engine type Host, port, service name, account, password

The administrator picks from this list the databases to open to the AI, and OwlAI records OwlDB's database identifier on the connection. Schema analysis results, role permissions and table notes attach to this connection. When a database managed by OwlDB moves to another server or its node roles change, OwlDB's database identifier stays the same, so the connection and everything attached to it stay as they are.

The engine type also comes from the metadata. Both the prompt and the executor look at the engine to settle the SQL dialect: row-limit syntax, quoting, date functions. On a direct connection, the administrator picks the engine, and OwlAI refuses to save if the connection check finds a different engine on the actual server. An OwlDB connection has no engine field to pick. It takes the engine OwlDB already knows, so the two products cannot disagree about which engine a database runs. The display name, too, is used exactly as OwlDB provides it.

What passes between OwlAI and OwlDB, and what stays inside OwlDB
Figure 1. What passes between OwlAI and OwlDB, and what stays inside OwlDB

At execution time, too, OwlAI passes identifiers

When the AI calls the SQL tool, OwlAI first checks whether this user's role may use that database and whether the statement is a single read. Which tables within that database it can read is set by the permissions of the per-role database account (“Per-role database accounts enforce table permissions, not SQL checks”). These checks run exactly as they do for direct connections (“Two SQL executors, one set of checks and one result format”). Once they pass, OwlAI asks OwlDB to execute. The request carries four values:

The request carries no password either. OwlDB connects with the database account that matches the account identifier, and it also picks the node to run on from the node roles and replication lag at that moment (“AI queries pick their database node at run time, not at registration”). The response carries the result and the node that actually ran the query. When OwlDB sends a query back without running it, a reason code comes with it; whether that reason is shown to the AI or OwlAI retries is covered in “Heavy SQL is judged by OwlDB, which knows the database's state, not by a cost threshold” and “AI queries pick their database node at run time, not at registration”.

When querying through an OwlDB connection, OwlAI neither stores nor decrypts the customer database password. A direct connection decrypts its stored password when it connects, but an OwlDB connection has no password to decrypt in the first place.

The only changes OwlAI follows are changes to the database list

Put the changes from the first section side by side for the two connection types:

Database-side change Direct connection OwlDB connection
Server move Administrator fixes the address and password Handled entirely inside OwlDB
Password rotation Administrator fixes the password Handled entirely inside OwlDB
Failover Administrator fixes the connection target OwlDB picks a new node to run on
Rename Administrator renames it in OwlAI The new name from OwlDB appears
New database Administrator registers a new connection It appears in the list, and the administrator decides whether to open it

On an OwlDB connection, what OwlAI follows are changes to the list: a database appearing or being renamed. OwlAI holds no connection details, so a change to the connection details never creates a setting to fix in OwlAI.

Design choices

We chose not to receive connection details. Holding them would let OwlAI build a path that connects straight to the database when OwlDB does not respond. That path, however, sits outside the table permissions of the per-role database accounts OwlDB maintains (“Per-role database accounts enforce table permissions, not SQL checks”) and outside OwlDB's checks on heavy SQL (“Heavy SQL is judged by OwlDB, which knows the database's state, not by a cost threshold”). Keeping passwords only in OwlDB means rotation and revocation finish in that one place. The cost is that queries on an OwlDB connection do not run when OwlDB does not respond.

We set the connection source per connection rather than for the whole installation. The connection source records where a connection was created: direct entry, Supabase, Amazon RDS or OwlDB. Sources differ in whether an executor can be opened, whether roles can get table-level permissions, whether an SSH tunnel is available and whether the connection target can be edited by hand, and the registration screen, the API, the executor and role permissions all read one set of rules defined per source (“Two SQL executors, one set of checks and one result format”). Switching the whole installation into an "OwlDB mode" would be simpler, but a company that also uses databases OwlDB does not handle would then have to give up one or the other. With the source set per connection, adopting OwlDB does not shrink the set of databases you can use.

We chose to use the list metadata as is instead of rewriting it on OwlAI's side. If OwlAI overwrote names or engines, the two products would call the same database by different names, and someone would have to settle which is right. The original lives in OwlDB alone. The cost is that renaming a database on OwlAI's screens means renaming it in OwlDB.

If you have only a few databases and their connection details rarely change, direct connections are enough, and for Supabase or Amazon RDS databases, importing saves you from copying addresses. If OwlDB already manages your database list and connection details, and server moves, password rotations and failovers are frequent, connecting through OwlDB keeps the connection details in one place.