Short answer
Voice AI for clinic intake is viable, but it is harder engineering than chat-based AI: a chat interface can pause while the model thinks, and a voice agent cannot, because the pause becomes a dead line and callers hang up. On our first deployment, for a clinic group with four branches, language and latency were the binding constraints. Recognition accuracy on Tagalog-dominant and mixed-language speech was significantly lower than on English-dominant speech in our initial tests, so we added a pre-classification step that detects the likely primary language from the first few seconds and routes accordingly, with a fallback re-prompt when recognition confidence falls below a threshold. We set an internal target of under two seconds from caller end-of-speech to agent start-of-speech for the majority of exchanges, which required minimizing round-trip time across the call platform, speech-to-text, the language model, and text-to-speech, and rewriting every response pattern to be as concise as the information allowed.
Voice AI for clinic intake is a harder problem than it looks from the outside. A chat-based AI intake can pause while the model thinks. A voice agent cannot - the pause becomes a dead line, and dead lines make callers hang up. This is the account of our first voice AI deployment for a multi-branch clinic, what the technical constraints actually looked like, and how we evaluate whether the system is working.
The Problem It Was Solving
The client was a clinic group with four branches. Their front desk staff were spending a significant portion of each day handling inbound calls for appointment booking, rescheduling, basic FAQs, and referrals to the right branch for specific services. Staffing the phone line across branch hours was expensive and inconsistent - after-hours calls were missed, and busy periods created hold times that callers did not stay for.
The brief was a voice agent that could handle first-contact calls: collect caller information, identify the purpose of the call, book appointments directly into the clinic's scheduling system for straightforward cases, and escalate to a human for anything complex or urgent.
Tagalog, English, and the Real Caller Population
Our first challenge was language. The caller population for a Philippine clinic is not monolingual. Some callers speak English. Many speak Filipino or Tagalog. Most mix languages mid-sentence in ways that do not follow a clean code-switching pattern.
We tested multiple speech recognition approaches against real call recordings the clinic provided. Recognition accuracy on Tagalog-dominant speech and mixed-language speech was significantly lower than on English-dominant speech in our initial tests. The gap was large enough that a single-model approach produced an unacceptable error rate on the mixed-language inputs.
Our solution was a pre-classification step that detected the likely primary language of the first few seconds of speech and routed accordingly, combined with a fallback that prompted the caller to repeat if recognition confidence fell below a threshold. This added latency to the opening of the call but produced better overall accuracy.
The fallback prompt was tuned carefully. "Sorry, could you repeat that?" repeated twice in a row on an intake call kills the experience. We limited to one re-prompt per exchange and had the agent reframe rather than repeat verbatim.
Latency Is the User Experience
In a voice interaction, the response time between a caller finishing speaking and the agent responding is everything. If the gap is too long, callers assume the call has dropped or that they are talking to a broken system. The tolerance is narrow - measured in seconds, not seconds plus a few more.
We set an internal target of under two seconds from caller end-of-speech to agent start-of-speech for the majority of exchanges. Hitting that target required work on both the infrastructure side and the prompt side.
On the infrastructure side: we minimized round-trip latency between the call platform, the speech-to-text layer, the language model, and the text-to-speech synthesis. Each of those steps adds time. They cannot run in parallel because each step depends on the previous output, so the total time is additive.
On the prompt side: shorter responses help. A voice agent that generates a 200-word response takes longer to synthesize and delivers worse caller experience than one that generates 40 words. We rewrote every agent response pattern to be as concise as possible without losing the information the caller needed.
Evaluating Quality
We use three metrics to evaluate call performance, reviewed on a weekly basis with the clinic.
Containment rate: the percentage of calls the agent handled without escalation. A high containment rate is good only if the escalations that do happen are the right ones - genuinely complex cases, not cases where the agent failed.
Escalation appropriateness: we sample escalated calls and categorize why they escalated. This is manual review, not automated. We look for patterns: is the agent escalating things it should be able to handle? Are callers forcing escalation by expressing frustration?
Caller sentiment signals: we flag calls where the caller used specific language patterns that correlate with frustration - repeated "hello," repeated questions, explicit requests for a human. These are not definitive measures but they surface calls worth listening to.
The first month of live deployment produced a containment rate in the range we had targeted. The escalation patterns showed one category of call the agent was not handling well - insurance and HMO verification questions that required looking up information the agent did not have access to. We expanded the integration scope to include that lookup in the second iteration.
Voice AI for clinic intake is viable. It requires more careful engineering than chat-based AI features and more active monitoring post-launch. If your operation has a phone intake problem worth solving, the conversation is worth having.
Questions · 04
Frequently asked questions
What kinds of calls was the agent built to handle?
First-contact calls: collect caller information, identify the purpose of the call, book appointments directly into the clinic's scheduling system for straightforward cases, and escalate to a human for anything complex or urgent. The front desk had been spending a significant portion of each day on inbound booking, rescheduling, basic FAQs, and referrals to the right branch for specific services, with after-hours calls missed and hold times during busy periods that callers did not stay for.
What happens when the agent does not understand a caller?
It re-prompts once, and the wording of that prompt matters as much as the model behind it. "Sorry, could you repeat that?" delivered twice in a row kills an intake call, so we limited the agent to one re-prompt per exchange and had it reframe the question rather than repeat itself verbatim. Anything beyond that goes to a human.
How do you tell whether a voice agent is actually working?
Three metrics, reviewed weekly with the clinic. Containment rate is the percentage of calls handled without escalation, and it only counts if the escalations that do happen are the right ones. Escalation appropriateness is manual review — we sample escalated calls and categorize why they escalated, looking for cases the agent should have handled or callers forcing escalation out of frustration. Caller sentiment signals flag calls with patterns like repeated "hello," repeated questions, or explicit requests for a human; not definitive, but worth listening to.
What did the first month of live deployment show?
Containment landed in the range we had targeted. The escalation patterns surfaced one category the agent was not handling well — insurance and HMO verification questions that required looking up information the agent did not have access to. We expanded the integration scope to include that lookup in the second iteration.
