Every business AI looks alike in a demo. Ten questions to ask first

Ask any product a short question in a demo, such as "Who were our top customers by revenue last month?", and you get a plausible answer. That is why a demo tells you almost nothing about how business AI products differ.

The differences show up later: when you connect company data, when many people use it every day, and when it goes through a security audit. This post turns what the series covered into ten questions that test for those moments before you adopt. Each question lists, in parentheses, the posts that cover it in detail.

#Today's numbers have to come from the database at the moment you ask

1. Does it query your company's data as it stands right now? To answer about today's inventory and orders, it has to read the business database when the question is asked, not documents copied in advance. Also ask whether it can tie the terms each department uses differently to the data. (“Inventory right now: does the AI chatbot know today's count?”, “Departments count revenue differently. Whose number will the AI use?”)

2. Does the AI know when the data is too large? When a query result exceeds what the AI can read, some products read part of it and answer as if they'd read it all. Check that both the AI and the person asking can tell only part was read. (“Tens of thousands of orders: does AI really read them all?”)

#Your company decides where data goes and what it can open

3. Which AI model do your questions and data go to? Ask whether you're limited to external models, whether you can connect a model running on your own GPUs, and whether it works on a network with no internet access. See whether you can restrict which models receive data according to its classification. Answers aren't the only way out, either: check that generated files and tools that send data out, such as email, fall under one boundary and one approval. (“You want the newest AI models. The customer data can't leave.”, “Masked in chat, do national ID numbers still turn up in Excel?”, “Bringing AI to a network where ChatGPT won't load”)

4. Does the AI work only within the asker's permissions? Databases and documents a person can't access must stay closed even when they ask through the AI. Also check that it only reads from databases, and that code the AI writes runs only in an isolated space. (“Put AI on the production database, and security asks three questions”, “The AI runs its own code to tidy your Excel. Is it safe unreviewed?”, “Asking AI for data you aren't allowed to see: what happens?”)

#Risky actions need human approval, and every attempt is recorded

5. Does it ask a person before actions that can't be undone? Your company should be able to set each tool that writes to an external system to allowed, approval required, or blocked. Also ask whether the AI ignores instructions hidden in outside material. (“A hidden line in an email you asked it to summarize. Will the AI obey?”, “Email can't be unsent. So can you let an AI hit send?”, “Once the AI sends email and edits tickets, who approves it?”)

6. Is there a record of who queried what, and can you hand it to auditors? See whether executed queries, rejected attempts, security decisions such as tool approvals and personal-data masking, and administrators' viewing of records are all logged. Also check the retention period and how records are exported. (“Asking AI for data you aren't allowed to see: what happens?”)

#Installation and updates are also yours to check and roll back

7. Can you tell before installation whether your servers are ready? A tool that checks conditions in advance—operating system, security policies, internal proxies, certificates—cuts down trial and error on installation day. (“While IT and security wait on each other, the AI rollout drags on”)

8. Does your company approve updates, and can a failed one be rolled back? Ask whether only verified versions get in and whether it backs up automatically before applying. Also check that it returns to the previous version on failure, and that a server several versions behind can still upgrade in one step. (“Updating self-hosted AI: can it be done safely?”)

#The more you use it, the more the team keeps and the better it fits

9. Does what one person figures out become a team asset? You should be able to turn a good question and answer into recurring work, share it with the team, and get results in your company's own templates. (“Only the old hand knew the data rules. Will the AI still know them?”, “Could AI take over the weekly report end to end?”, “The AI wrote the document. Why paste it into the company template?”)

10. Does it adapt to your company the more you use it? Check whether it remembers users' preferences and work context while keeping that memory under company control, and whether internal terms come out more accurately in the minutes with each meeting. (“Report preferences you repeat every time: can AI remember them?”, “The account is locked. Does the AI still remember what they told it?”, “Minutes keep getting in-house terms wrong. Must you fix them by hand?”)

#Where the answer is vague, test it with real work in a pilot

OwlAI answers yes to all ten. If another product gives a vague answer on any line, we recommend running that line against real work during a pilot.