Wrong-table SQL still runs, so search narrows the candidates first

A business database with thousands of tables often keeps backup and working copies such as ORD_MST_BAK and ORD_MST_202312 next to the orders table. They have the same columns as the live table, so if the model picks a copy, the SQL runs without an error and the answer carries the numbers from the day the copy was made. Unlike a syntax error, the database does not reject it, so the model gets no chance to correct itself. Put the whole schema in the prompt and those copies sit in front of the model too, and the DDL for thousands of tables does not fit in the context window in the first place.

So we designed table selection as its own step, separate from writing SQL. The start of a conversation carries only database names. When the model calls table search with the question, semantic search, keyword search and table-note search gather candidates; tables outside the user's permissions are dropped, and tables judged to be backups move behind the live table. The model opens only one to three of the remaining candidates to check their columns, then writes SQL. This post covers what each step adds and what it filters out.

#The database reports a syntax error, but SQL on the wrong table becomes the answer

SQL with bad syntax is the less dangerous kind. The database refuses to run it and returns an error message, and the read-only execution tool hands that message to the model. The model fixes the SQL in the next round of the same turn. A turn is one question and its answer; a round is one model call plus running the tool calls in its response.

SQL on the wrong table is different. These are the shapes it commonly takes in business databases.

All three succeed as far as execution status goes, so the place to stop them is before the SQL is written. Reading the number of returned rows as a total, and the analysis that works out what columns and code values mean ahead of time, are covered in “RAG can't count or sum rows, so we ask the source database in SQL” and “DDL doesn't say what a column means, so we read its values first”. This post covers only the step that decides which table to look at.

#Load the whole schema or none of it, and the model still picks the wrong table

The simplest approach is to put all the DDL in the system prompt. With a few dozen tables, it really does work. With thousands, the DDL alone fills the context window and leaves no room for the conversation or tool results. Even a model with a generous window still pays to read hundreds of thousands of tokens every turn, and an installation running a small open model on an air-gapped network cannot choose this at all. With everything loaded, what the model faces is dozens of pairs like ORD_MST and ORD_MST_BAK whose names differ slightly, and the DDL does not say which one is live.

Load nothing, and the model either guesses table names or asks the user. Business users do not know table names. So the chat system prompt tells the model not to ask users for table or column names but to find them with tools, and to ask users follow-up questions only in business terms.

What remains is the space in between. Start with database names only, and give more detail each time the scope narrows.

#The start of a conversation carries only database names

The first turn's input lists up to 10 databases the user can query, each with its engine type such as MariaDB or Tibero. They are ordered by how many words they share with the question, and the top three get the database summary produced by analysis, cut to 120 characters. Databases the user has no permission for are listed separately, by name only. That way, when the user asks about one, the model says there is no permission instead of mistaking it for a database that does not exist. Databases without permission get no summary, because the summary alone reveals what business data the database holds.

Tables are found by search.

#Table search merges semantic and keyword search with RRF

The table search tool takes the question and returns 8 candidate tables by default, up to 30. It searches not table names but the text of the table cards that analysis produces (“DDL doesn't say what a column means, so we read its values first”). A card holds the table's purpose, a description of each column, foreign keys, what code values mean, columns that are easy to misuse, and the comments the customer wrote in the database. Even if a table is named ORD_DTL_H, a card that says "order detail history" lets a question about "order records" reach it.

The card text goes through two searches at once, an approach known as hybrid search.

The two rankings are merged with RRF (reciprocal rank fusion). The two scores are on different scales and cannot be added directly, but ranks can, and a table near the top of both lists comes first.

Tables with hundreds of columns are indexed in pieces. If one card were one vector, the meanings of hundreds of columns would blur into that single vector, and a question that matches only one column would have trouble finding the table. So the card is split into one head chunk (purpose, keys, foreign keys, columns to watch, the list of column names) and several chunks of 35 columns each. Search runs over chunks, and results are grouped per table by its closest chunk. Even if the question matches only the 300th column, the table becomes a candidate through the chunk that contains that column.

The third search is table notes. Notes are knowledge users leave during conversations, such as "this table has not been used since 2023" or "returns are stored separately in this table." A question phrased in words that appear only in a note, not in the card, still brings the noted table in as a candidate, along with an excerpt of the note that shows why it came up. When note hits are mixed in, a reranker reorders the candidates by relevance to the question.

If no database is specified and the user can query many, search picks databases first. It measures how relevant each database summary is to the question with embeddings (or by word overlap if embeddings are unavailable) and skips databases whose relevance falls far behind. The result says how many of how many databases were searched, so the model can name a database and search again if needed. Databases the model names, or that the user picked with a mention, are all searched.

#Backup and off-limits tables are filtered out of search results

Candidates are filtered three times before they reach the model.

First, tables the connection account can no longer see are dropped. These are tables deleted in the source database or tables the account lost permission for, and they are not kept as candidates even if their analysis results remain.

Next, tables the user's role cannot see are dropped. They leave the candidate list and appear by name only, in a separate field. With those names, the model answers that relevant data exists but the user has no permission to see it, and does not attempt SQL on those tables.

Last, backup, draft and staging tables are ranked lower. Which table is a copy of which is judged ahead of time by the analysis that looks across the whole database (“DDL doesn't say what a column means, so we read its values first”). A table judged to be a copy gets a lower score, and is moved behind the live table again even after the reranker reorders the list. The reranker orders only by relevance to the question, so it can put a copy, whose content matches the live table, in front. Partition tables hold real data split into parts, so they are not ranked lower.

When several databases are searched together, each candidate is marked with the database it came from. If a customers table comes up in both the CRM database and the billing database, the system prompt tells the model not to guess but to ask the user in business terms. Only the user knows whether "number of customers" means contracted customers or billed customers.

#The model opens cards for only one to three candidates

Each candidate carries the table name, database and schema, score and card text, plus the table role and backup status judged by analysis, what code values mean, columns that are easy to misuse, join candidates, and up to five notes attached to the table. The database's engine and version come along too. From this alone the model can discard most candidates.

The system prompt asks for two things together. Before writing SQL, open the table card for every table the SQL will use and check its columns. And stop at the one to three most promising search candidates instead of opening them all, because cards take up context too. A card gathers each column's type, key role, database comment, statistics, frequent values and meaning on one line. The mistake of looking for order status in REMARK, a free-text notes column, instead of STATUS_CODE is caught by the card's list of columns that are easy to misuse.

A table with a great many columns makes even one card large. When a card exceeds the tool result limit, it is shortened: every column's name, type and key stay, and the columns whose details were cut are listed by name. The model calls the card again with just the columns it needs to read their details.

When several tables must be joined, the model calls the join path tool. It returns foreign keys declared in the database separately from join candidates inferred by analysis. Inferred candidates come with their basis and a confidence level, and the system prompt tells the model to verify them before use. It also tells the model not to make up join keys beyond declared keys and verified inferred candidates.

The table name list tool is used only when search finds nothing. In a large database, a plain list of names is mostly noise.

Split this way, the schema the model sees in one turn is usually a few lines of database list, summaries of a few candidates, and one or two cards. The number of candidates and the number of cards opened are fixed, so this size barely changes as tables grow into the thousands.

#Design choices

We made search the first step of table selection. The model starts from the candidates search picks, so if search misses the live table, the model starts from the wrong candidates. That is why search can be called many times in one turn, and when the first candidates do not fit, the model rewrites the query or names a database and searches again. If that still fails, the table name list remains.

For keyword search we chose PostgreSQL's simple configuration. A configuration with a morphological analyzer can match Korean words with particles attached, but it ties search to one language. Table and column names are often English abbreviations like CUST_STAT_CD, and simple matches them the same way in any language. Semantic search covers expressions with particles attached.

Backup tables are ranked lower, not removed. Removing them would make it impossible to answer questions that ask about a copy on purpose, such as "the row count difference between last month's backup and the current table." The cost is that copies stay at the bottom of the candidate list, and the system prompt tells the model to use the live table unless the user asks about backups.

Ordering databases at the start of a conversation uses plain word overlap. It runs at the start of every conversation, so we chose fast over precise. If the ordering misses, databases within the top 10 still appear by name without a summary, and the model reaches databases outside the list through table search.

Opening only one to three cards costs rounds. When the first card does not fit, one more round runs, but the context is taken up by the tables the question actually uses.

If your database has hundreds of tables or more, with backup copies and several databases holding the same business data, and the SQL returns a wrong number, look at the candidate list search returned before looking at the SQL. Whether the live table was missing from the candidates or was there but behind a copy tells you where to fix it.