Back to Blog
Blog

How do you measure which human tasks an AI agent actually takes over?

How do you measure which human tasks an AI agent actually takes over?

The clearest way to measure which human tasks an AI agent actually takes over is to list the specific tasks in the process before the AI is introduced…

The clearest way to measure which human tasks an AI agent actually takes over is to list the specific tasks in the process before the AI is introduced, measure how long each one takes and who does it, then track which of those tasks the AI now completes and how often people need to correct or redo its work. Measuring at the task level avoids vague claims about time saved, which are easy to make and almost impossible to check.

Building the task list takes a little time but pays off, since a process like reviewing a procedure or preparing a permit usually breaks down into a dozen or more distinct activities, such as gathering reference documents, checking values against standards, drafting changes, routing for approval and recording the outcome. Timing a sample of real jobs, even roughly, gives a baseline that everyone can agree on, and it often shows that the time goes to different tasks than people assumed.

Once the AI is running, each task can be tracked in terms of how often the AI completes it, how often a person accepts the output as it is, how often they edit it and how often they discard it and do the work themselves. Those figures say far more than a single headline percentage, since an agent that drafts every change but has most of them rewritten has not really taken over the task, while one that handles a narrower set reliably may be delivering more real value.

It is also important to track new work the AI creates, like reviewing its output, since that time should be counted against the savings. Checking AI output can be quick when the source of every value is visible and slow when the reviewer has to verify everything from scratch, and it tends to fall on experienced people whose time is the most expensive and the most scarce, so it is worth measuring who does the reviewing as well as how long it takes.

The honest number is the time people spend on the process before and after, not just the time the AI spends working. Repeating the measurement a few months in is worthwhile too, since review effort often drops as people learn where the AI is reliable, and sometimes rises again if the AI is extended to tasks it handles less well.