AI agents still need employees to check their work

AI agents still depend on work they were meant to reduce

Deploying an AI agent can leave employees spending hours checking and correcting the work it was supposed to speed up. Agents are software systems that generate answers and take actions, but their ability to act can exceed their ability to judge the information they use. When employees must repeatedly supply that judgment, supervision remains an operating cost. Preparing an agent therefore means giving it data it can reach, context it can interpret, and rules it can apply before acting.

Human interpretation leaves a gap when agents take action

People often supplied this judgment in established data workflows. Van de Maele of Collibra explains that enterprise data systems largely served people reading dashboards and conducting analyses. When something looked wrong, a person questioned it and “supplied context the data couldn’t provide on its own.” Collibra sells software for defining data and controlling its use, so it has a commercial interest in enterprises investing in the definitions, context, and controls he recommends.

Moving from human analysis to agent action changes who must supply that context. A person can stop to question a definition or apply knowledge before deciding, whereas an agent may continue through the same uncertainty. “Agents change that,” Van de Maele said. “When a definition is ambiguous, or context is missing, an agent can still produce an answer and act on it with confidence. So the work isn’t simply cleaning up more data. Enterprises need to make context available in a form machines can understand and use: What does this data mean, can I trust it right now, and what am I allowed to do with it?”

Giving software access to those judgments requires more than correcting errors in a dataset. Even clean data can have unclear definitions, with permitted uses that depend on an employee’s knowledge. In Van de Maele’s account, deploying an agent exposes interpretation work that people previously did within the workflow. If that interpretation remains outside the system, employees must keep supplying it through checks, reviews, and corrections.

Recurring oversight adds to the cost of agent use

The reported frequency of those checks makes supervision an immediate operating concern. In a Collibra and Harris Poll survey, 87% of decision-makers said their teams regularly check whether the context available to AI agents remains accurate and current. These are separate questions: information can correctly describe an earlier situation yet be unsuitable for a decision now. Teams must check both the information itself and whether it still applies to the agent’s task.

Once an agent produces a result, another kind of work follows. More than half of respondents in the Collibra and Harris Poll survey said employees spend hours manually reviewing and correcting AI agent outputs. These reviews add work between the agent’s response and a result the business can use. Leaders assessing a deployment therefore need to account for employees’ time spent inspecting and repairing its output.

Teams can investigate this cost by examining why employees intervene in an agent’s work. Van de Maele’s recommendations give them specific questions about missing context and the rules applied when the agent acts. The survey describes employees’ checking and correction work, while this review can help a team identify the conditions behind supervision in its own deployment.

Data access and usable context address different constraints

Before an agent can interpret information, it must be able to reach it. An August report from Google Cloud and MIT Technology Review Insights found that AI could access an average of 45% of an enterprise’s data. The report identifies a strong data foundation and improved access as requirements for expanding agent use. Google Cloud sells cloud and AI services, so it benefits commercially when enterprises invest in the infrastructure needed to expand those deployments.

Greater access gives an agent more information, but it still needs to interpret what it retrieves. Van de Maele’s recommendations address that next step: establishing meaning, assessing whether data is trustworthy now, and checking permission to act. Access and interpretation are therefore separate requirements for deployment. Improving retrieval expands the available information; supplying usable context helps the agent judge how to use it.

Whether organizations trust the output is another question. Only half of organizations trusted that their AI agents’ outputs were relevant and accurate, according to the Google Cloud and MIT Technology Review Insights report. This finding measures confidence in results, while the Collibra and Harris Poll findings describe employees’ oversight work. When deciding whether to expand agent use, teams need to examine availability, confidence, and supervision separately because each affects how the business can use an agent.

Agents need checks for meaning, current trust, and permitted action

Making retrieved information usable begins with the definitions and rules attached to it. Van de Maele recommends aligned business definitions and encoded policy, rules written in a form software can check, as parts of a data foundation designed for agents. Structured context puts the information needed to interpret data into a form software can use. Runtime governance makes rules available for checking while the agent operates, including immediately before it takes an action.

Meaning comes first because an agent needs to establish what the information represents in its assigned task. Aligning business definitions reduces the ambiguity that employees previously resolved through questions and explanation. A deployment team can test this by asking whether the agent can interpret a piece of data using the information available within its workflow. If an employee must explain the definition each time, part of the decision still rests with that person.

A clear definition still leaves the question of whether the information is suitable now. The agent needs context to assess the data’s accuracy and currency at the moment of use. Teams report checking this judgment repeatedly, so it is a useful place to examine where human supervision enters the workflow. Van de Maele proposes putting the relevant context into a form the agent can use as it works.

Even trustworthy information requires a separate permission check because authority depends on the proposed use. Van de Maele recommends encoding policy so an agent can check whether an action is allowed just before it acts. The rule must therefore be available at the decision point and apply to the action being considered. If someone must interpret the policy separately each time, the workflow still depends on that person’s intervention.

Regulatory preparation creates further requirements for data foundations

Policy checks govern how an agent operates, while regulatory preparation also requires organizations to account for their data and its handling. According to the Collibra and Harris Poll survey, nine in 10 business leaders are actively preparing for AI regulations in the U.S. and globally. The survey also found that 51% of enterprise leaders are investing in data lineage and documentation. Lineage records where data originates and how it changes; documentation describes its characteristics and handling. Both give teams records they can use in preparation.

This preparation is taking place alongside debate about whether safeguards can keep pace with increasingly capable AI. Executives from OpenAI, Anthropic, Microsoft, and SpaceXAI have called for slower AI development, citing safety frameworks that are failing to keep up with frontier models, the most advanced models under development. These companies develop or commercialize AI, so their executives have a business stake in rules affecting its development and deployment. Their calls concern the pace and oversight of development, a broader issue than supervising an individual enterprise agent.

The broader debate is producing several kinds of policy response. Sen. Bernie Sanders, I-Vt., plans to introduce legislation that would temporarily pause advanced AI development. In California, Governor Gavin Newsom signed two AI bills establishing a framework for independent audits of large language models, which generate and process language. In Europe, the EU AI Act’s AI literacy obligations, which concern people’s understanding of AI and its use, and prohibitions on unacceptable use cases took effect in August.

These interventions create different preparation tasks because they address development, independent scrutiny, organizational knowledge, and permitted uses. Lineage and documentation can help an enterprise explain its data and handling practices. Audit preparation must address the audit requirements, while work on AI literacy and prohibited uses must address the knowledge and conduct those obligations require. Each preparation task needs a clear connection to the obligation it serves.

Key takeaways for decision-makers

• Budget for supervision: Agent deployments carry an ongoing cost when employees must check and correct their work. Deployment owners need to include that labor when assessing productivity gains and expansion plans.
• Make business context usable: Data owners need to turn definitions and judgments employees apply informally into context agents can interpret. Ambiguous definitions leave people responsible for resolving uncertainty as agents operate.
• Track why employees intervene: In the Collibra and Harris Poll survey, more than half of respondents reported hours spent reviewing and correcting agent outputs. Operations teams can examine those interventions to identify recurring gaps in context, data currency, and rules.
• Assess access and trust separately: AI could access an average of 45% of enterprise data, according to the Google Cloud and MIT Technology Review Insights report. Deployment reviews need separate checks for data availability, usable context, and confidence in results.
• Put checks at the decision point: Agent workflows need checks for data meaning, current trustworthiness, and permission to act. Platform and governance teams need to make definitions, context, and policies available when the agent uses information or takes action.
• Match preparation to specific obligations: Regulatory preparation spans documentation, audits, AI literacy, and permitted uses. Compliance owners need to connect each investment to the requirement it serves; lineage and documentation provide records of data origins and handling.