The Semantic Comeback: Why Enterprise AI Is Forcing a Rethink of Data Architecture

 As organisations move towards agentic AI, semantic technology is gaining renewed importance in helping machines interpret enterprise data consistently and within the right business context. Explore the challenges of implementing semantic layers, knowledge graphs and ontologies at scale, and why governance, domain ownership and architectural design will be critical to building reliable enterprise AI. 

Semantic technology is not new. But as organisations move from generative AI experiments to agentic systems that can act across business processes, giving machines a consistent, governed understanding of enterprise data is becoming an architectural priority.

The challenge is not simply adopting a semantic layer, building a knowledge graph, or publishing an ontology. It is making those capabilities work across fragmented systems, evolving business rules, and real-world decision-making.

AI needs business context

AI systems can retrieve information and generate convincing responses. That does not mean they understand the business context behind the data.

An agent may identify a revenue figure without knowing which calculation applies, which business unit owns the metric, which source is authoritative, whether the data is current, or whether it is permitted to use or act on it.

As AI moves from answering questions to executing processes, these gaps become consequential.

Semantic capabilities are complementary

Semantic layers, ontologies, and knowledge graphs solve different but related problems.

A semantic layer may define net revenue consistently across reporting and AI applications. An ontology may define what qualifies as a customer, contract, product, or entitlement. A knowledge graph can connect those entities across CRM, ERP, billing, support, and data-platform systems.

They are complementary—not interchangeable.

The real challenge is governance

The hard part is not modelling concepts. It is establishing shared meaning without pretending that every business function uses language in exactly the same way.

Finance, sales, operations, and compliance may have legitimate reasons to define “active customer,” “churn,” or “operational performance” differently. The goal is not one universal definition. It is clarity about which definition is authoritative, where exceptions apply, who owns the change, and how AI systems select the right interpretation.

A practical approach is a federated core plus domain extensions:

• Maintain a small set of shared enterprise concepts and core metrics.
• Allow domains to extend them with local rules and specialised definitions.
• Make mappings, ownership, versions, and exceptions explicit.
• Avoid forcing every function into one undifferentiated enterprise model.

Do not create another bottleneck

Most enterprises already have data warehouses, lakehouses, catalogues, APIs, legacy applications, and multiple cloud environments. Semantic capability should integrate with that landscape, not become another isolated central layer.

Start with a domain where inconsistent definitions already affect analytics, operations, compliance, or AI. Use existing governance, catalogue, identity, and data-platform capabilities where possible. Make semantic assets available through APIs and metadata standards, and avoid requiring every data product to pass through a single semantic platform.

If the semantic layer simply becomes another place where definitions are manually maintained, it will add cost rather than reduce it.

Semantics do not make AI trustworthy

A well-defined semantic model helps an agent interpret information. It does not guarantee that the underlying data is accurate, complete, current, or appropriate for a decision.

Organisations still need entitlement-aware access, lineage, data-quality signals, validation, monitoring, audit trails, and human approval for high-risk actions. An agent should not only know what “approved supplier” means; it should know whether it can use that classification, whether the source is authoritative, whether the information is fresh, and whether a human needs to approve the resulting action.

Semantic architecture should complement governance and security controls—not replace them.

A pragmatic starting point

Data and AI leaders should begin with business domains where inconsistent definitions or disconnected information are already creating friction.

Priorities include:

• Establish domain-owned semantic models with clear accountability.
• Define how shared concepts and domain-specific definitions coexist.
• Integrate semantic metadata with catalogues, governance, security, and data platforms.
• Use knowledge graphs where relationships genuinely add value.
• Introduce versioning, testing, change management, and impact analysis.
• Measure improvements in consistency, accuracy, traceability, policy compliance, and operational performance.

The objective is not to build the largest or most comprehensive knowledge graph. It is to establish the right level of semantic structure for the business problems AI is expected to solve.

The architectural question

The renewed interest in semantic technology reflects a broader shift: enterprises are no longer asking only how to make data available to AI. They are asking how to ensure AI interprets data consistently, within the boundaries of how the business actually operates.

For data leaders, this is a question of AI readiness and strategic investment. For architects, it is a question of introducing shared meaning without adding another layer of complexity.

The next challenge may not be giving machines more data. It may be ensuring they understand it consistently—and act on it responsibly.

For more insights and ideas from Australia's data and AI leaders and practitioners, join us at Data & AI Architecture Sydney 2027, Enterprise AI Sydney 2027, CDAIO Sydney 2027, and CDAIO Brisbane 2027.

Image credit: Perplexity AI