Understanding Data Sources Is Often a Real Bottleneck in Data Product Delivery

Liam De Lee

19 Nov 26

7min read

Business Analytics

When organizations talk about delays in delivering data products, the conversation usually turns into a lack of engineering capacity. More developers. More automation. Faster development.

But in our experience, the real bottleneck comes much earlier.

Before a single transformation is built, someone has to figure out what the source data actually means. Teams spend weeks reading API documentation, searching through hundreds of fields, tracking business owners, and interpreting poorly documented codes and relationships.

That work is slow, manual, and difficult to scale.

The good news? It's also exactly the kind of work that AI can help with.

On a recent enterprise platform, we used AI to accelerate source interpretation while keeping all business decisions and implementation under human control. The result: source onboarding dropped from the originally three weeks planned to one, and around 90% of the AI-generated field definitions were approved without changes.  

The lesson was simple: use AI to create the first draft, let people review it, and only then generate the technical implementation.

Why AI Adoption Creates a Trust Problem

AI adoption among developers is growing rapidly, but trust isn't keeping pace. Many developers use AI tools regularly yet still find themselves reviewing and correcting the output.

That's not necessarily a problem. For many business applications, being "mostly right" is good enough. For enterprise data products, it isn't.

A small mistake in a dashboard used for strategic decisions can create significant downstream issues. That's why we don't believe AI should be making business logic decisions on its own. Instead, organizations should focus on where AI adds value safely: helping people understand and document source systems faster.

Where Delivery Time Really Goes

Imagine a finance team requesting better spending visibility.

The data team opens the source system documentation and immediately faces dozens of resources, hundreds of fields, unclear relationships, and status codes with little or no explanation.

Questions quickly start piling up:

  • Which fields are still actively used?
  • What is the grain of each dataset?
  • Which columns contain personal or sensitive data?
  • Who owns the data?
  • Which relationships matter?

Finding the answers often means speaking with people across different departments, many of whom have limited availability.

This investigative work can consume weeks before any actual development starts.

Where AI Delivers the Most Value

One stage of the process is especially well suited to AI. It is not building the solution but preparing the first draft.  

When provided with source metadata such as API specifications, schemas, sample payloads, and documentation, a language model can quickly generate:

  • Descriptions of entities and fields
  • Proposed key relationships
  • Suggested owners
  • Potential personal or sensitive data classifications
  • Areas where documentation is unclear or incomplete

Instead of beginning with a blank page, architects and BI specialists start with a structured proposal they can review, correct, and approve. That's where much of the efficiency gain comes from.  

Human Review Remains Essential

AI can identify patterns. It cannot fully understand your business.

For example:

  • It can see that a status field contains nine values.
  • It cannot determine which values make an invoice legally payable.
  • It can suggest relationships between tables.
  • It cannot confirm whether those relationships align with business processes.

Those decisions require business knowledge and remain firm in human hands.  

The goal is not to replace expertise. The goal is to allow experts to spend less time collecting information and more time applying their knowledge.

A Practical Approach

The process we implemented follows six simple stages:

  1. Extract: Collect metadata from source systems such as schemas, APIs, and sample data.
  2. Review the Source Metadata: Store the extracted metadata in version control and validate it before moving forward.
  3. Generate a Proposal: AI creates a structured draft describing entities, fields, ownership, keys, and data sensitivity.
  4. Approve the Design: Architects and BI specialists review the proposal and make any required adjustments.
  5. Register the Design: The approved design is published to architecture and governance repositories.
  6. Generate and Deploy: Templates generate the models, tests, and documentation from the approved specification.

The key principle is simple: AI proposes. Humans decide. Automation is delivered.

An Unexpected Benefit: Governance Improves Automatically.

One surprising outcome was the improvement in governance.

Because ownership, descriptions, classifications, and lineage are captured before implementation starts, governance becomes part of the delivery process rather than an afterthought.  

The organization gradually builds a library of approved data definitions that can be reused across future projects.

It also creates visibility into source system changes. When fields are added, removed, or renamed, the differences become visible immediately instead of surfacing months later through broken reports.

Why This Matters for AI Agents

As organizations introduce AI agents, the importance of well-documented data grows even further.

An AI agent answering a business question needs more than access to data. It also needs context:

  • What does the table represent?
  • What is the level of detail?
  • What does each status mean?
  • Which data can be accessed?

Without that information, an agent is forced to guess.

The better your metadata and governance foundation, the more reliable future AI use cases become.  

Key Takeaways

  • The biggest delay in data product delivery is often understanding source systems, not writing code.
  • AI is most effective when used to accelerate source interpretation and documentation.
  • Human experts should review and approve designs before implementation begins.
  • Versioned metadata helps identify source changes before impact reporting.
  • Governance becomes a natural by-product of the delivery process.
  • Every approved design creates a reusable context for future analytics and AI initiatives.

In our recent implementation, source onboarding decreased from three weeks to one, while roughly 90% of AI-generated field descriptions were accepted without modification. The remaining 10% highlighted genuine ambiguities that required human judgement.

And that's where the real value lies: not replacing experts but helping them focus on the decisions that truly require expertise. Could your organization require an expert view or in need of help? Contact us, we are always open to listen and help.

Subscribe to our newsletter

Read, learn, adapt, grow.