Clients keep asking me: “How do we make the agent give us the same answer every time?”
You can get closer. But that is the wrong question to obsess over.
AI agents are non-deterministic. Give one the same messy problem twice and it may take two different paths to solve it. Different SQL. Different retrieval order. Different reasoning steps.
Teams spend a lot of time trying to engineer that away with prompts, examples, temperature settings, and increasingly complicated system instructions. That work is useful. It just does not turn an agent into a spreadsheet.
The more useful definition of reliability is simpler: the path can change. The test it has to pass should not.
Different path, valid result
Take a data agent. Ask it, “How many customers churned last quarter?” Run it twice and it may write completely different SQL.
One version might use CTEs. The other may nest fourteen subqueries like it gets paid by the parenthesis.
I do not really care … if both answers pass the same checks.
- Did it use the approved definition of churn?
- Did it query the trusted source tables?
- Did it exclude test accounts?
- Does the math reconcile?
- Does the result match known checks for that data?
If those are true, the agent had room to solve the problem its own way without turning a different thought process into a different business rule.
This is how people are managed too
We do not expect two analysts to write the exact same SQL. We give them judgment, let them solve the problem, and check their work against the same standards.
We may have accidentally invented AI agent governance decades ago. We just called it management.
The important part is that the check cannot be another vague model opinion. Asking a second LLM, “Does this look right to you?” gives you a non-deterministic system grading a non-deterministic system. It can be confidently wrong twice.
Instead, give the workflow something objective: a calculation, a policy, a database rule, a system of record, or a known-good result.
Build the deterministic loop around the agent
For higher-consequence work, that loop can include schema checks, business rules, evaluation cases, approval thresholds, monitoring, and a recovery path when the result fails.
The goal is not to force every agent down the same route. It is to make the destination measurable before the agent is allowed to act.
I do not need my agent to give me the exact same answer every time.
I need every answer to have to pass the same test.
