Ontologies for AI agents: what they are and when your company needs one

Lee este artículo en español.

In short. An ontology is a formal description of a business's concepts, how they relate to each other and the rules that let a system infer what is not stored in any table. It helps keep an AI agent from inventing relationships between data, but it is rarely the first thing the agent needs. Two simpler things come first: a well-documented data schema and a single definition of every business term.

Contents

• What an ontology is, in plain English
• Ontology, knowledge graph and semantic layer are not the same thing
• Why your agent gets your company's data wrong
• What we have seen in our own projects
• What the published tests show
• Three levels: only move up when you need to
• When an ontology is worth it
• What an ontology does not do
• What to do before you think about an ontology
• The question that matters
• Sources

You ask your AI agent how many invoices you have pending payment with a supplier. It answers in two seconds, with a figure and a flawless table. The figure is wrong, and nobody notices until month-end close.

When this happens, the fashionable answer is to add an ontology or a knowledge graph. Sometimes that is the solution. In the projects we see, the problem usually comes earlier and is far more boring: nobody has written down anywhere what "pending payment" means in your company.

Brijesh Dave makes this point in a recent article: many projects build ontologies they do not need, when well-documented data models, clear relationships and validation would be enough. He proposes three levels, and moving from one to the next only when the complexity justifies it.

What an ontology is, in plain English

A database stores information. An ontology describes what that information means, how it is related and what can be inferred from it.

Think of an electricity distributor. It deals with customers, contracts, supply points, transformers and incidents. Its database has a table for each, joined by keys. An ontology also writes down the business rules:

• A contract is linked to a supply point.
• Each supply point is identified by its CUPS, the universal code that tells it apart.
• A transformer feeds several supply points.
• Every transformer is a piece of electrical equipment.
• If a piece of equipment feeds a supply point and a contract is linked to that point, the contract depends on that equipment.

The last rule is the one that makes the difference. SQL can also find the contracts hanging off transformer T-103, but someone has to write that query, and repeat the logic in every report or agent that needs it. In the ontology, the rule is declared once and works for any question.

If you also declare that "feeds" chains (the substation feeds the line, the line the transformer and the transformer the supply point), the same rule finds the contracts affected by a fault anywhere in the network, without writing a new query for each level.

Original diagram · Montevive. Simplified example; identifiers are made up.

How a machine infers it

The most widely used standards for writing ontologies come from the W3C, the consortium that maintains web standards.

RDF stores each fact as a triple of subject, predicate and object: "T-103 feeds P-52". OWL adds the meaning: hierarchies ("every transformer is electrical equipment"), inverse relationships ("fed by" is the opposite of "feeds"), relationships that chain, and rules like the one about contracts.

A reasoner, a program that applies those definitions with formal logic, concludes that T-103 is electrical equipment and that C-881 depends on T-103, even though nobody stored either fact. This is not a language model's prediction: the same reasoning always gives the same result, and you can explain step by step why the system flagged a contract as affected.

Ontology, knowledge graph and semantic layer are not the same thing

The three terms are used as synonyms and they are not. Each one costs and solves different things.

Simplifying a lot, the ontology is the rules of the world, and the knowledge graph is the map of concrete facts in that world. In practice many graphs carry their ontology inside them, and many others are just connected data, with no rules.

Why your agent gets your company's data wrong

An agent built on a language model works wonderfully in the demo, with one clean table. Once it starts querying several databases, the ERP and a couple of APIs, the errors Dave describes show up:

• It confuses business concepts.
• It invents relationships between tables.
• It misreads the status of a transaction.
• It writes SQL queries that run and return the wrong thing.
• It does not know that two departments call the same thing by different names.

The visible problem is that the model makes things up. The real problem, in our experience, is that the meaning of the data is not written down: it lives in the heads of a few people and in old queries. The model fills that gap with whatever seems plausible.

What we have seen in our own projects

Two examples from the last few weeks, from agents and automations we build for clients.

In a debt collection project, the payment-default email had to state how much is owed. The head of the area changed the rule for what counts as "amount owed" three times in one week: which statuses are included, whether only overdue amounts count or future ones too, and which part of what is already written off. It is the same problem as the pending invoices at the start, and no model could have guessed the final version.

In another project, a platform that generates contracts called the same expense categories one thing in the application and another in the contract table, so the document ended up contradicting itself. We fixed it by settling on a single name per concept, agreed with the client and used everywhere.

Neither needed an ontology. They needed someone with authority to decide what each term means, and the system to use that definition and no other.

What the published tests show

A series of studies measures exactly this: how many business questions a language model answers correctly over a company's database, depending on the context you give it. They always use the same case, a fictional insurer with its database and a set of questions like the ones an executive would ask.

• 2023. Juan Sequeda, Dean Allemang and Bryon Jacob, of data.world, found that GPT-4 answered 16% of the questions correctly by querying the tables directly. With the same data represented as a knowledge graph, 54%.
• 2024. Allemang and Sequeda added an ontology-based check, with no language model involved, that reviews each query before it runs and asks the model to fix it if it does not fit. 72.6% of the answers were correct, 8% were "I don't know" and around 20% were wrong.
• 2026. dbt Labs took 11 questions from that case and ran each 20 times with two models, Claude Sonnet 4.6 and GPT-5.3-Codex. On the original database, letting the model write the query scored 64.5% with both, and the semantic layer 72.7%. Then they added three intermediate tables that organised the data for the business, and accuracy rose for both methods: 84% to 90% when the model wrote the query, and 98% to 100% with the semantic layer.

Original chart · Montevive, with data from the three publications cited. Models and setups change between studies: compare within each block, not across blocks.

These figures need to be read with care. It is a single case with few questions: with 11, each question moves the result by nine points. Data.world sells knowledge graphs and dbt Labs sells a semantic layer. And no study measures what it costs to build and maintain each layer.

Even so, they leave two takeaways. First: in this case, giving the agent the meaning of the business, as a graph, an ontology or a semantic layer, improved its answers a lot. Second, and less often quoted: in the dbt study, the biggest jump did not come from adding a layer, but from organising the data better.

The dbt authors stress one more thing: when the semantic layer cannot answer a question, it says so, instead of silently returning wrong data.

Three levels: only move up when you need to

Dave's proposal boils down to a staircase.

Original diagram · Montevive, based on the three levels proposed by Brijesh Dave. Relative cost is our own estimate.

This is our reading, building on Dave's, of where to start depending on the kind of agent:

The boundary is less rigid than a table suggests, because you can have a lightweight ontological model without RDF or OWL. What you should not do is skip the levels below.

When an ontology is worth it

According to Dave, and it matches what we see, a formal ontology makes sense when three things come together:

• A domain with complex hierarchies and classifications, such as regulation, insurance, healthcare or energy.
• Several systems or areas that need integrating, with knowledge you want to reuse across more than one agent.
• Conclusions that have to be auditable: it is not enough for the agent to be right, you have to be able to explain how it got there.

The clearest example is a compliance agent. To decide which obligations apply, it has to relate the type of data, the purpose of the processing, whether it is a special category, the applicable law and the category of the AI system. Those relationships change less than the data, have hierarchies, and are better not improvised by the language model on every question.

There, it works well to split the job into three pieces. The language model interprets what the person is asking. The ontology holds the concepts and their relationships. And a separate set of rules, independent of the model, decides what is allowed, required or forbidden.

It is the same split we apply in Moviwa, our security layer between teams and AI: the model detects what kind of data a message carries, and what happens to it is decided by the policies each company defines.

What an ontology does not do

• It does not fix bad data. It infers from what is there. If the data on which transformer feeds which point is wrong, the inference will be flawless and false.
• It does not maintain itself. Every change in the business or in the law has to be reflected, and a large ontology can end up with rules that contradict each other.
• It adds moving parts. Its own query language, a reasoner and someone who knows how to maintain both.
• It does not guarantee regulatory compliance. It helps apply rules, but obligations have to be kept up to date and applied in their legal context.
• It does not replace whoever decides what each term means. It only writes it down in a way a machine can use.

What to do before you think about an ontology

1. Write down the 20 questions your agent has to answer correctly. The real ones, the ones someone answers by hand today.
2. Define every business term in those questions, with an owner who signs off the definition.
3. Pick one source per concept. If "active customer" comes from two systems, decide which one rules.
4. Do not let the agent write free-form SQL against your systems. Give it predefined queries and metrics, with their permissions, and check what goes in and what comes out. It is a security issue too, as we explain in our article on agents and security.
5. Measure with those questions, and value "I don't know". An agent that admits it does not know is more useful than one that is confidently wrong. Repeat the measurement every time you change something.

If after that the agent still fails on questions that require inferring relationships that are not stored, it is time to evaluate a knowledge graph or an ontology.

The question that matters

The useful question is not "does my agent need an ontology?", but: what conclusions does the agent need to reach that are not explicitly stored anywhere, and what guarantees do I need about those conclusions?

If you are building an agent on top of your company's data, we can help you prepare what it needs before any technology. If you already have one running, our AI agent audit measures, with your real questions, where it goes wrong. See our services or get in touch with the three questions your agent answers worst, and we will tell you which level the problem is at.

Sources

• Brijesh Dave, "Do your AI agents actually need an ontology layer, or just a better schema?", Intellytics Solutions (Medium), 23 October 2026.
• Juan Sequeda, Dean Allemang and Bryon Jacob, "A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases", arXiv, November 2023.
• Dean Allemang and Juan Sequeda, "Increasing the LLM Accuracy for Question Answering: Ontologies to the Rescue!", arXiv, May 2024.
• Jason Ganz and Benoit Perigaud, "Semantic Layer vs. Text-to-SQL: 2026 Benchmark Update", dbt Developer Blog, 7 April 2026.
• W3C, "RDF 1.1 Concepts and Abstract Syntax", Recommendation, 2014.
• W3C, "OWL 2 Web Ontology Language Document Overview", Recommendation, 2012.