Blog

Engineering Notes

Design decisions and measurements from building an on-premises AI for work — how the principle of enforcing with code, guiding with instructions, and verifying with audits is implemented.

Part 1

Design principles and architecture

The design principles behind the whole series, and the execution structure every AI feature shares.

Engineering Notes · Design principles and architecture

Permissions are enforced in code, judgment is left to the model

OwlAI's agent reads internal databases and documents, and it also reads web pages and results from connectors…

October 31, 20267 min read

Engineering Notes · Design principles and architecture

One agent loop for every feature, with the differences in hooks

Five features in OwlAI have a model working by calling tools: chat; playbooks, which run a defined procedure w…

October 31, 202610 min read

Engineering Notes · Design principles and architecture

RAG can't count or sum rows, so we ask the source database in SQL

Ask "How many orders have gone unprocessed for more than 30 days?" through document search (RAG), and the mode…

October 31, 20268 min read

Part 2

Structured data and NL2SQL

Turning natural-language questions into accurate SQL against business databases.

Engineering Notes · Structured data and NL2SQL

Wrong-table SQL still runs, so search narrows the candidates first

A business database with thousands of tables often keeps backup and working copies such as ORDMSTBAK and ORDMS…

October 31, 202610 min read

Engineering Notes · Structured data and NL2SQL

DDL doesn't say what a column means, so we read its values first

The order table's STATUSCODE CHAR(1) holds S, D and C.

October 31, 202610 min read

Engineering Notes · Structured data and NL2SQL

Revenue differs by team, so questions carry the approved definition

Ask for "last month's revenue" and sales thinks of booked orders, while finance thinks of invoiced amounts net of returns.

October 31, 20269 min read

Engineering Notes · Structured data and NL2SQL

Read-only and query cancellation have to reach the database server

OwlAI's SQL execution tool runs SQL written by the model against a customer's production database.

October 31, 20269 min read

Engineering Notes · Structured data and NL2SQL

Schema analysis goes stale, so only changed tables get reanalyzed

Suppose that after schema analysis (“DDL doesn't say what a column means, so we read its values first”) finish…

October 31, 20269 min read

Engineering Notes · Structured data and NL2SQL

Query results go to a file so the model never retypes them

When a user says "Give me the result you just queried as a CSV file," the model has to write the rows it alrea…

October 31, 202610 min read

Part 3

Database connections and OwlDB

How the connection to a customer database is opened and protected: SSH tunnels through a bastion, encryption in transit and certificate checks, importing Amazon RDS and Supabase connections on SaaS, and how execution works for databases connected through OwlDB.

Engineering Notes · Database connections and OwlDB

The bastion gets one key line that only forwards to the database, not an account

When a production database sits in a private network behind a bastion host (an SSH server that leads into that…

October 31, 20269 min read

Engineering Notes · Database connections and OwlDB

Certificate errors are fixed with the right CA and the right name, not by turning off verification

When OwlAI runs on a corporate network and connects over TLS to a cloud LLM or a customer database, certificate errors come from two places.

October 31, 20269 min read

Engineering Notes · Database connections and OwlDB

Customer RDS lists are read with a 60-minute role, not an access key

To connect a customer's Amazon RDS or Aurora database from OwlAI SaaS, OwlAI needs the endpoint, port and engi…

October 31, 20269 min read

Engineering Notes · Database connections and OwlDB

Supabase connection addresses are filled by OAuth approval, not copied by hand

The Supabase dashboard lists three addresses for the same database: the project's direct address, port 6543 on…

October 31, 20269 min read

Engineering Notes · Database connections and OwlDB

OwlDB connections hold a database ID instead of credentials

To open a company database to the AI, an administrator enters the host, port, account and password into OwlAI.

October 31, 20267 min read

Engineering Notes · Database connections and OwlDB

Per-role database accounts enforce table permissions, not SQL checks

Say an administrator opens only the customer and order tables to the sales role and closes the salary table in the same database.

October 31, 20268 min read

Engineering Notes · Database connections and OwlDB

Heavy SQL is judged by OwlDB, which knows the database's state, not by a cost threshold

When AI-written SQL holds a production database for a long time, the order screen and the settlement batch tha…

October 31, 20267 min read

Engineering Notes · Database connections and OwlDB

AI queries pick their database node at run time, not at registration

To keep AI queries off the primary node, you can register the connection with a standby's address.

October 31, 20268 min read

Engineering Notes · Database connections and OwlDB

Two SQL executors, one set of checks and one result format

OwlAI has two ways to run SQL against a customer database.

October 31, 20269 min read

Part 4

Memory and search

Memory design that lets individual and team knowledge improve answers, plus document search, deep research, and source design.

Engineering Notes · Memory and search

Load personal memory on every turn, and rewrite it whole

A user says, "My account in the issue tracker is this ID.

October 31, 202610 min read

Engineering Notes · Memory and search

Corrected column meanings, kept as table notes for the next user

The status column STATCD in a customer table holds 1, 2, and 9.

October 31, 202610 min read

Engineering Notes · Memory and search

Korean particles break full-text search, so we join words with OR

A table note says 「반품 주문은 ORDRTN 테이블에 따로 쌓인다」 ("returned orders pile up separately in the ORDRTN table"), and…

October 31, 20268 min read

Engineering Notes · Memory and search

One system prompt edit zeroes cache reads, so changing blocks go last

Every time an agent calls the model, the request carries the tool definitions, the system prompt, and the conv…

October 31, 202613 min read

Engineering Notes · Memory and search

RLS guards org boundaries; the table classification decides deletion

In OwlAI SaaS, many organizations share one PostgreSQL database, and one person can belong to two of them.

October 31, 202610 min read

Engineering Notes · Memory and search

When document search ranks lookalike chunks first, rerank them

Ask "How many days of annual leave can I carry over to next year?" against a company's policy documents with e…

October 31, 20269 min read

Engineering Notes · Memory and search

Deep research returns reports with coverage and gaps, not raw text

A question like "Summarize last quarter's incident root causes across our three products, and compare how the…

October 31, 20269 min read

Engineering Notes · Memory and search

Every answer source is a URL, and the server builds the list

When one answer draws on web pages, internal documents, records queried from a database, and tickets returned…

October 31, 20268 min read

Engineering Notes · Memory and search

To total a table in a document, load it into a temporary SQL table

Upload a Hangul document with a 300-row price list and ask for "the average unit price by product group," and…

October 31, 20267 min read

Part 5

Conversation engine

The internals that keep conversations running: multi-turn, caching, compaction, and messages sent mid-turn.

Engineering Notes · Conversation engine

A turn's input tokens grow with the square of its rounds

A model API remembers nothing between calls.

October 31, 20268 min read

Engineering Notes · Conversation engine

OpenAI stores responses by default, so we resend the conversation

The OpenAI Responses API stores every response on the provider's servers unless the request sets store, and th…

October 31, 20266 min read

Engineering Notes · Conversation engine

Keeping old answers out of edited questions with a conversation tree

Say a user asks for "last month's top 10 customers by revenue," and the model answers from a table that holds only domestic customers.

October 31, 202610 min read

Engineering Notes · Conversation engine

Mid-turn messages sit in a DB row, and a row lock picks who takes them

A deep research turn runs for up to an hour by default.

October 31, 20269 min read

Engineering Notes · Conversation engine

A 400 that sticks: add tool results on write, fix shapes on send

Model APIs remember nothing between calls, so an agent resends the whole conversation context every round.

October 31, 20269 min read

Engineering Notes · Conversation engine

Compaction thresholds come from the input limit, not the context window

As a conversation grows, its requests eventually reach the input limit the model will accept.

October 31, 20269 min read

Engineering Notes · Conversation engine

Shrink giant tool results before the provider cuts them, and say so

Sometimes a single tool result blows past the model's input limit in one go: a query that returns hundreds of…

October 31, 20269 min read

Engineering Notes · Conversation engine

When every tool call succeeds but the results repeat, stop the agent

Sometimes every tool an agent calls succeeds and the work still goes nowhere.

October 31, 20268 min read

Part 6

Tools and execution

Workflow automation, data movement, tool design, document and diagram generation, and subagents.

Engineering Notes · Tools and execution

Rerunning a chat repeats its mistakes. Keep the steps and working SQL

Say a chat once went well: it pulled last week's sales from the database, grouped them by department, and built an Excel file.

October 31, 202610 min read

Engineering Notes · Tools and execution

Tool calls a script makes still get permission and approval checks

Take a request like "For each issue opened last week, look up the customer's contract, and email the owner onl…

October 31, 20269 min read

Engineering Notes · Tools and execution

Tools copy the form and load the guide; models fill the arguments

Making a quote from a company form takes two steps: copy the form to a new path, then fill in the copy.

October 31, 202610 min read

Engineering Notes · Tools and execution

Block Office files that won't open, then show the model its pages

When a model writes code that builds an Excel file, the run can exit with code 0 and the model can answer "Don…

October 31, 202611 min read

Engineering Notes · Tools and execution

185 tool definitions cost 99K tokens, so we list names and load picks

To give an agent a tool, you put that tool's definition, its name, description, and argument schema, into the request.

October 31, 20269 min read

Engineering Notes · Tools and execution

Model-drawn bars can miss their numbers, so the screen draws charts

Ask a model to draw a bar chart as SVG and it converts each value to a height on its own scale and writes the coordinates.

October 31, 202610 min read

Engineering Notes · Tools and execution

Parent-picked subagent tools skip approval, so code decides them

The usual way to add subagents is to give the parent model a tool that launches them and let it decide the pro…

October 31, 20269 min read

Part 7

Voice and meetings

Meeting recording, speech recognition, meeting glossaries, minutes, and audio retention.

Engineering Notes · Voice and meetings

Transcripts misspell internal names; the minutes model fixes them

When a meeting recording is transcribed, internal system names and product names come out as the common word that sounds closest.

October 31, 202610 min read

Engineering Notes · Voice and meetings

Overlapping meetings stall transcripts, so split queues by resource

In an on-premises installation, several meetings share one speech recognition model running on a single GPU.

October 31, 20269 min read

Engineering Notes · Voice and meetings

Ending a meeting drops the last remarks; audio waits for transcription

When a meeting ends, some of its sound has not been transcribed yet.

October 31, 202610 min read

Engineering Notes · Voice and meetings

A dropped meeting recording pauses the meeting instead of ending it

A meeting recording runs for an hour or more in a single browser tab.

October 31, 202610 min read

Engineering Notes · Voice and meetings

Choose meeting speech recognition by speaker turns and retention

Choose a meeting speech recognition service from a single error-rate table and two things go wrong.

October 31, 202610 min read

Part 8

Security and audit

How the principle of blocking deterministically, guiding with instructions, and confirming through audit is implemented: egress control, injection defense, personal data masking, a shared output boundary, credentials, sandboxing, approval gates, external tool policy, and permissions and audit.

Engineering Notes · Security and audit

External models never see internal-only data, not even its name

Put a model running inside the customer network and an external API model in one installation, and deciding wh…

October 31, 20269 min read

Engineering Notes · Security and audit

A URL in a tool result stays closed if the model wrote it first

Tools that read a web page or download a file from a URL only open addresses that appeared earlier in the conv…

October 31, 20268 min read

Engineering Notes · Security and audit

Scrubbing answers is too late for phone numbers. We mask tool results

"Find the customers who asked for a refund more than three times last week and summarize what they said" is no…

October 31, 20268 min read

Engineering Notes · Security and audit

Answer-only checks leak via Excel: one check for every exit path

In a conversation that analyzes customer inquiries, someone without permission to view originals sees the phon…

October 31, 20268 min read

Engineering Notes · Security and audit

Keeping presigned URLs out of the model with 19-character references

Connectors for cloud file services (external services, such as MCP servers, that a company connects to OwlAI)…

October 31, 20269 min read

Engineering Notes · Security and audit

Database backups carry stored passwords, so the key lives elsewhere

OwlAI's metadata database holds customer database passwords, LLM provider API keys, OAuth tokens for connector…

October 31, 20269 min read

Engineering Notes · Security and audit

Sandbox each script run, then prove isolation on the actual server

In OwlAI, Excel, Word, PowerPoint and Hangul documents are produced by python and node scripts the model write…

October 31, 20269 min read

Engineering Notes · Security and audit

Told to confirm first, the model still sends the email, so approval happens right before execution

You can write "Confirm with the user before sending email" in the system prompt, and the model can still call…

October 31, 202610 min read

Engineering Notes · Security and audit

Connector shows Connected but returns 401: refresh by response code

When you attach a connector (an outside service such as an MCP server that a company connects to OwlAI) with O…

October 31, 20269 min read

Engineering Notes · Security and audit

Connector OAuth callbacks finish at once only in the starting browser

OwlAI's per-user connectors (external services such as MCP servers that a company connects to OwlAI) attach through OAuth.

October 31, 20267 min read

Engineering Notes · Security and audit

Customers know their connector tools' risks, so tool policy is per install

Say a customer turns its internal e-approval system into an MCP server and attaches it to OwlAI as a connector…

October 31, 20269 min read

Engineering Notes · Security and audit

Auditing an AI agent: log every query it runs, not the conversation

Open your internal databases and documents to an AI agent, and after an incident a security officer will ask:…

October 31, 202610 min read

Part 9

Installation, operations, and quality

Technical reference for on-premises adoption, from installation and air-gapped transfer to updates and operational checks, plus the principles that keep an installable AI product's quality.