AI Automation
AI agent development
AI agent development is what you need when a process is too messy for rules but too repetitive for a person. I build agents that read unstructured input, decide what to do, act inside your systems, and hand off to a human the moment they are not confident. Every decision is logged in language you can audit.
- Runs in your accounts
- Every decision logged
- Human escalation built in
Sound familiar
Why most agent projects stall
The demo was great. In production it made things up.
We cannot tell why it did that, so we cannot trust it.
It handles the easy cases, which were never the problem.
What AI agent development actually involves
An agent is a program that decides. That is the whole distinction. A workflow follows a path you defined in advance; an agent looks at the situation, picks an action from a set of tools it has been given, does it, looks at the result, and decides what to do next.
That capability is genuinely useful for a narrow class of problem: processes where the input is unstructured, where the right next step depends on the content, and where the volume is too high for a person but the variation is too high for rules. Enquiry triage, document extraction, support ticket routing, research and enrichment tasks, first-pass qualification. Those are the jobs where agents earn their keep.
For everything else, an agent is a slower, more expensive and less predictable version of a function call. A large part of my job is telling people which of the two they actually have.
The failure mode nobody plans for
Agent demos work because the demo input is clean. Production input is not. A real inbox contains forwarded threads with six replies, attachments referenced but missing, two people asking different questions in one message, and the occasional automated bounce. An agent that was tested on tidy examples will confidently do something wrong with all of those.
The answer is not a better prompt. It is confidence handling: the agent must have a defined way to say “I am not sure about this one” and route it to a person, and the threshold for that has to be tuned against real data rather than guessed. In the systems I build, the review queue is a first-class feature, not an error path.
How I build production agents
Tools before prompts
An agent is only as good as the actions available to it. I define the tool surface first: the specific, narrow, well-validated functions the agent can call, each of which validates its own input and refuses nonsense. If the agent hallucinates an argument, the tool rejects it and the agent gets a usable error back rather than corrupting a record.
Narrow tools also make behaviour legible. “Called find_patient(email), got no match, called create_review_task” is a decision trail a human can follow. A single do-everything tool is not.
Bounded autonomy
Every agent I ship has a hard limit on how many steps it can take, an explicit list of actions it may never take without approval, and a spend cap. Unbounded loops are the most expensive bug in this field, and they are entirely preventable.
The actions requiring approval are chosen with the client and they are usually the irreversible ones: sending an external email, issuing a refund, deleting anything. An agent that can draft but not send is dramatically more useful than one that cannot be trusted to run at all.
Evaluation against your data
Before an agent goes live it runs against a set of real historical cases with known correct outcomes. That gives an accuracy figure that means something, and more importantly it shows which categories it gets wrong. Almost always the errors cluster, and the cluster is fixable.
This evaluation set stays. When the underlying model changes, or when I change a prompt, the suite runs again. Without it, every improvement is a guess and every model update is a risk you cannot quantify.
Observability from day one
Every run records the input, the tools called with their arguments, the model responses, the confidence signals and the final action. When someone asks in three months why a particular record was routed the way it was, the answer takes thirty seconds to find. Systems without this get switched off the first time they make a visible mistake, because nobody can defend them.
What I will talk you out of
Agents that write directly to customers without review. Agents doing arithmetic that a query would do exactly. Multi-agent architectures where one agent and three tools would work, which is most of the time. Replacing a person entirely when removing eighty percent of their busywork is cheaper, faster and less risky.
I would rather build the smaller thing that works than the impressive thing that gets turned off.
What you get
Included in every agent build
-
Validated tool surface
Narrow, well-tested functions the agent can call, each validating its own input so a bad argument fails cleanly instead of corrupting data.
-
Evaluation suite on your data
Real historical cases with known outcomes, giving an accuracy figure that means something and a regression test for every future change.
-
Human review queue
A tuned confidence threshold and a place for uncertain cases to go, designed as a feature rather than an error path.
-
Full decision logging
Input, tools called, responses and final action recorded for every run, so any decision can be explained months later.
The process
From messy inbox to running agent
-
Sample the real input
A few hundred genuine cases, including the ugly ones. This defines the tool surface and sets the confidence threshold.
-
Build and evaluate
Tools first, then the agent loop, then the evaluation run against historical cases with known answers.
-
Shadow run
The agent runs alongside the human process without acting, so you can compare its decisions to theirs before it touches anything.
-
Go live, bounded
Live with step limits, spend caps and approval gates on irreversible actions. Gates come off as confidence is earned.
Questions
AI agent development questions
How is an AI agent different from an automation?
An automation follows a path you defined in advance. An agent chooses its next action based on what it finds. Agents are worth the extra complexity only when the input is unstructured and the right step genuinely varies. When the path is knowable, a plain workflow is faster, cheaper and more predictable, and that is what I will recommend.
How do you stop an agent making things up?
Three ways, and none of them is prompt wording. Tools validate their own inputs and reject invalid ones. The agent is given the data it needs rather than being asked to recall it. And anything below the confidence threshold goes to a human queue instead of being acted on. The goal is not a model that never errs, it is a system where an error cannot do damage.
Which models do you build on?
Whichever fits the task, and the code is written so the model is swappable. Model quality and pricing move constantly, and anything hard-wired to one provider will be a liability within a year. The evaluation suite is what makes swapping safe: change the model, run the suite, see exactly what moved.
How long does an agent build take?
A focused agent handling one process is usually shadow-running within two to three weeks and live shortly after. The variable is not the building, it is how quickly I can get a representative sample of real input. Projects that stall almost always stall there.
Next step
Send me a hundred real examples
The fastest way to find out whether an agent is the right answer is to look at your actual input. Describe the process and I will tell you if it needs an agent, a workflow, or neither.
- Reply within 24h
- Honest about fit
- No retainers required