Opinion
AI agents can’t become experts if every task is their first
"Two agents can use the same foundation model and still become very different workers," writes Nimble CEO Uri Knorovich. "The difference is the experience that accumulates around the job."
The first time an AI agent performs a difficult task, it may need to explore. Imagine asking it to find restaurants near a conference venue with private dining rooms. It has to identify the right websites, navigate menus, render pages, click into private-events sections and work out where the useful information actually lives.
That first run can require substantial reasoning and compute. Once the agent has discovered the path, however, the second run should not have to begin in exactly the same place.
For the past few years, most of the industry’s attention has centered on the model. A new model arrives, benchmarks move, context windows grow and reasoning improves. Those advances matter enormously. But an agent is more than its foundation model. It is a system of tools, retrieval, memory and rules built to perform work repeatedly. That repeated work creates the possibility of something the model alone does not provide: experience with the job.
An expert does not simply know more facts. An expert knows where to look, which sources deserve attention, what is probably noise, what evidence is still missing and when an answer is good enough to act on. AI agents will need to develop the same kind of specialization.
Consider a bank building an agent for risk assessment. A general-purpose model may know a great deal about companies, markets and finance. That does not mean it knows which evidence the bank requires for a particular decision, which sources count as authoritative, how fresh the information must be or which missing field makes the result unusable.
The same distinction appears in healthcare. Suppose an agent is mapping radiologists in New York who accept a particular insurance plan. A list of ten correctly identified doctors could seriously understate the available options if many relevant doctors were missed. Every fact in the answer may be true while the picture it gives is misleading.
A sales team building a prospect list faces a different trade-off. It may prefer broad coverage even if some leads turn out not to qualify, because a person can check them before making contact. The agent needs to know which omissions matter, which claims require verification and where human review makes an error tolerable. Those standards come from the job it has been given.
Increasingly, that intelligence sits around the foundation model. An agent can accumulate useful search paths. It can remember which sources produced reliable answers, learn that one website requires a particular navigation sequence, recognize that another is often stale, and reuse previous work instead of treating every research request as a blank page.
We see this when building web agents at Nimble. A first run may use computer vision and more expensive reasoning to work out how to navigate a complex site. Once the system understands the route, later visits to the same site can reuse that knowledge and perform much of the extraction with a smaller, cheaper model. The reusable asset is the route to the information. The restaurant’s current menu and availability still need to be checked.
Turning repeated work into expertise also takes feedback. When we work with enterprises, we encourage teams to put the agent’s findings in front of the people who know the business before letting it act. Their corrections help establish which sources and results are useful for the job. The system needs to retain those corrections alongside the results of its own checks, then use them to change the next search plan. A correction about one source should not become a rule for every source without further testing. Otherwise, a system may simply repeat the same mistake more efficiently.
The economics depend on the job, too. A routine monitoring task may need inexpensive compute and a predictable workflow. A high-value investment decision may justify much deeper research, more sources and a larger reasoning budget. Some jobs should take seconds; others may be better run asynchronously or overnight.
Response time alone tells us little about research quality. A fast answer may draw on a reliable, well-tested route; a slow one may waste effort on irrelevant pages. What matters is whether the system gathered the evidence the decision requires at a cost the business can justify.
This raises a question for companies deploying agents: what useful knowledge remains after an agent has performed the same job thousands of times? If all that remains is the original prompt, there may not be much accumulated advantage. If the system has learned which sources matter, which routes work, which evidence tends to be missing, what failure looks like and how much work different cases deserve, it has begun to acquire institutional knowledge.
That knowledge may eventually matter as much as the choice of foundation model. The leading models will keep improving, and companies will continue switching among them as price, performance and capabilities change. The harder question is whether the expertise accumulated around the work survives that switch.
Companies can test that advantage by giving the same model comparable new tasks, with and without the accumulated knowledge. Does the agent need fewer corrections? Does it find evidence it previously missed? Does it recognize when a familiar source or route has stopped working? Progress can mean maintaining quality at lower cost or producing a more complete answer whose value justifies the extra work.
Many competitors will have access to the same powerful models. Their agents will diverge through the knowledge they retain, the corrections they incorporate and the judgment built into their workflows. That is where repeated work begins to become a lasting business advantage.
Uri Knorovich is co-founder and CEO at Nimble.
