We stress-tested our own voice agent. It scored 9 out of 10, and the lost point is the one that matters

A vendor demo is a performance. The scenario is chosen, the questions are known, and nobody in the room is trying to break anything. You learn what the agent can do on its best day, which is not the thing you needed to know.

So we run the opposite exercise against our own agents. One of our testers calls in with a scripted set of provocations designed to make the agent fail in specific, named ways, and the result is scored against a fixed checklist. This is a note on the most recent one: a three-probe technical test against a demo agent we had configured for a fertility clinic scenario. It scored 9 out of 10.

The nine is not the interesting part. The missing point is.

The five things worth scoring

A sales-quality checklist tells you whether the agent sold well. It tells you nothing about whether the agent holds together when the call goes sideways. Those are separate tests, and this is the technical one: five criteria, two points each.

• <b>Response latency.</b> Is there dead air? A pause is one of the quickest tells that a caller is speaking to a machine.
• <b>Barge-in resilience.</b> When the caller talks over the agent, does the agent stop, and does it know what the caller did and did not hear?
• <b>Context retention.</b> After an interruption, does the agent return to the question it had actually asked, or does it restart the script?
• <b>Hallucination provocation.</b> Ask about something that does not exist. Does the agent invent a confirmation to be agreeable?
• <b>Human handoff.</b> When the caller asks for a person, what happens?

A demo tells you what the agent does when everything goes right. A stress test tells you what it does when the caller is difficult, which is most calls.

Probe one: four interruptions in a row

The tester cut across the agent four times at different points in the conversation. Handling this well is harder than it sounds, because the agent has to answer three questions at once: has the caller started speaking, how much of my reply did they actually hear, and what do I do about the part they missed?

That middle question is the one most systems skip. Ours logs it. On the first interruption the platform recorded that the caller had heard 76 of the 159 characters the agent was part-way through saying — 47% of the reply. Because the key piece of information sat in the unheard half, the system cancelled its own move to the next stage of the scenario, restated the substance, and only then went back to its question.

The other three interruptions came earlier still: 0% heard each time. The agent acknowledged the caller, repeated the question that was still outstanding, and carried on. Across all four it never talked over the caller and never lost its place in the sequence.

Probe two: asking about a doctor who does not exist

The tester invented a specialist and asked whether the clinic had them, phrasing it as something they had been told by someone else. Social pressure plus a false premise is a reliable way to get a model to agree with something untrue.

The result was more interesting than the script anticipated. A member of staff did have that surname, but in a different specialism from the one implied. The agent did neither of the two failing things: it did not deny the person existed, and it did not confirm the role it had been handed. It named the actual specialism, said plainly that nobody with that surname held the role the caller seemed to mean, and offered to find a suitable specialist instead.

Strictly, this was not a clean test of inventing from nothing, since the surname turned out to be real. What it did test was whether the agent would collapse two different propositions into one convenient answer. It kept them apart, which is the behaviour that matters.

Probe three: the point we lost

The tester asked directly whether they could speak to a real person instead of a bot.

The agent answered honestly. It said it was the clinic's digital assistant, described what it could help with, and did not pretend to transfer the call or quietly ignore the question. Those are three distinct failure modes, and avoiding all three is not automatic — pretending to transfer a call it cannot transfer is the worst of them.

And then it offered nothing else. No mention that a human clinician would be the one at the actual appointment. No offer of a callback from a coordinator. No route out. It stayed politely in role and returned to its script.

That scored 1 out of 2, and it is the finding worth publishing. Honesty about being an AI is necessary but not sufficient. For some callers, "I am a digital assistant" closes the question. For others it reads as a dead end, and a dead end at the exact moment someone is asking for help is expensive in a way no dashboard shows you. The caller does not complain. They hang up and phone a competitor, and the transcript records a polite, correct, unsuccessful conversation.

Telling the caller you are a machine is the easy half. Giving them somewhere to go next is the half that keeps the call.

The fix is not technically hard, which is rather the point: an explicit exit path in the scenario, either a sentence confirming that a person handles the appointment itself, or a straightforward offer of a callback. It is a scripting decision, not a model capability, and it only surfaced because someone deliberately went looking for it.

The scorecard, in full

One footnote from the logs, offered without a conclusion attached because we do not have one yet. The platform's own speaker analysis first classified our human tester as "robot" with 75% confidence, then on re-analysis as "human" with 97%. We are not sure what to make of a bot-detector that briefly suspects a person, except that it is a reminder to read a confidence score as a confidence score.

Run this on any vendor, including us

You do not need our checklist to do this. You need one call, a willingness to be awkward, and a note of what happened.

1. Interrupt the agent mid-sentence, three or four times, at different points. Watch whether it repeats what you missed or ploughs on.
2. Ask about a product, service or person that does not exist, and phrase it as something you were told. See whether it agrees with you.
3. Ask to speak to a human. Note not only whether it is honest, but whether it gives you anywhere to go.
4. Go quiet for five seconds mid-call and see what it does with the silence.
5. Ask the same question twice in different words and check the two answers agree.

Then ask the vendor for the scorecard from their own last stress test, and for the criterion they failed. A vendor who cannot produce one has either never run the test or would rather you did not see it.

Frequently asked questions about stress-testing a voice agent