What makes an AI agent production-ready? The harness, not the model.

By José Antonio Mijares | 2026-10-02 | 6 min read

A demo agent takes a weekend. A production agent takes evals, tools, memory, permissions, and logs. Here is the checklist.

A demo agent takes a weekend. It answers the five questions you tried, in the order you tried them, on the data you had open.

A production agent has to survive the sixth question. And the tenth person. And a Tuesday when an API returns an empty list.

The model is not what decides that. The harness is: everything around the model that makes it dependable.

The five parts of a harness

Evals. A fixed set of real tasks with known good answers. Run them on every prompt change, every model change, every tool change. No evals means every change is a guess.

Tools. The agent acts through functions you wrote: look up an order, draft a reply, create a ticket. Each tool does one thing, validates its input, and returns an error the model can read. A tool that fails silently is a tool the agent will lie about.

Memory. What the agent keeps between steps and between sessions, and what it throws away. Decide it on purpose. An agent that remembers everything slowly fills its context with noise.

Permissions. Which actions run alone, which need a person to approve, which never run. Refunds, deletes, and anything sent to a customer start in the second group. You can move them later, with evidence.

Observability. Every run logged: the input, each tool call, each result, the final output, the cost. When something goes wrong you read the trace. You do not guess.

How to tell a demo from a production agent

Ask four questions about the agent in front of you.

1. What happens when a tool fails? If nobody knows, it is a demo.
2. How do you know the last change did not break anything? If the answer is "we tried it," it is a demo.
3. Who approves the risky actions? If the answer is "the agent is careful," it is a demo.
4. Where is yesterday's run? If nobody can open it, it is a demo.

A production agent has a boring answer to each one.

Start smaller than you want to

Pick one task a person does today. Write down its inputs, what done looks like, and which decisions get escalated.

Then ask the honest question: does this need an agent at all? A fixed sequence of steps with no judgment is a script. Scripts are cheaper, faster, and easier to test. Use an agent where the path changes based on what it finds.

Build the evals before the agent. Twenty real cases from last month's work beat two hundred invented ones.

What not to buy

A platform that promises agents with no evals. A demo built on your best five examples. An "autonomous" agent that can send, pay, or delete on day one. A vendor who cannot show you a trace.

Frequently Asked Questions

Q: What is an AI agent harness?

Everything around the model that makes the agent dependable: evals, tools, memory, permissions, and observability. The model reasons. The harness decides whether that reasoning is safe to act on.

Q: Which model should I use for an agent?

Start with the best one available, get the evals passing, then try cheaper ones against the same evals. The evals tell you when a cheaper model is good enough. Opinions do not.

Q: How many eval cases do I need?

Enough real cases to cover the common path and the failures you already know about. Twenty real cases from last month is a good start. Add every new failure as a case.

Q: When should an agent act without approval?

When the action is reversible, low cost, and the evals show it gets it right. Refunds, deletes, and customer messages start behind approval.

Q: Is an agent the same as AI implementation?

No. Implementation is deciding where AI belongs in a business and wiring it into the stack. Building an agent is one way to do that. How AI pilots reach production covers the first part.

Jamak builds production agents with the harness included. We run agents in our own products first. The agent work is AI Agents. The adoption work is AI Implementation. Talk to us. No rate card.