When Voice AI Transfers a Call, What Does Your Screen Say?

Picture a caller spending two minutes with your voice agent. She gives her name, her account number, the service address and a callback number, and explains that the technician who came on Monday didn't finish the job. The agent recognizes a complaint it shouldn't handle and transfers the call. Your office manager picks up. Her screen shows a phone number and nothing else, so the first thing she says is "Can I get your name?"

That moment decides how the caller remembers the whole call. The two minutes with the AI went fine, and the transfer made them feel wasted. Most voice AI projects spend their design effort on the conversation and treat the handoff as a phone feature. It's the part I would design first.

The free guide OpenAI Realtime API: Building AI Customer Conversations That Convert covers the whole build, from architecture to compliance. This post is about one piece of it, the one the guide calls handoff design: what the human sees when the AI passes the call along.

The transfer moves the call, and only the call

OpenAI describes its Realtime API as the way to build speech-to-speech voice agents (OpenAI documentation). For phone calls, its API reference includes a method that transfers an active SIP call to a new destination, and the documented request takes a destination such as a phone number (OpenAI API reference). Nothing in that request tells the person who answers what the caller already said. The call moves. The conversation stays behind unless you build something to carry it.

That's true of most call transfers, AI or not, and it's why a good warm transfer between two people starts with one of them briefing the other. When the first party is software, the briefing has to be written down and put in front of the human before they say hello.

A note on currency, since this area moves quickly. OpenAI's documentation now also describes GPT-Live and includes a guide for migrating to it from the Realtime API (OpenAI documentation). Ask your developer which one they're building on before either of you follows a tutorial, this post included. The handoff question is the same on both.

What should be on the screen

Think of the screen as a note the AI leaves before it steps away. It should be readable in the time it takes the phone to ring twice. For most businesses that means five fields:

• Who is calling, matched to a record if the number or the details they gave match one, and marked as unmatched if not.
• Why they're calling, in one sentence written for a person rather than pasted from a transcript.
• What they already gave: account number, address, order number, whatever you would otherwise ask for again.
• Why the AI transferred: the caller asked for a person, a complaint, a payment question, a failed identity check or a topic outside its scope.
• What the AI told them about what happens next, so the human doesn't contradict it.

The last field is the one teams skip, and it causes the most trouble. If the agent said "I'm connecting you with someone who can book the return visit", the person who answers needs to know that promise was made.

Where the note lives

The summary has to appear where the person answering already looks. For a small team that's usually the CRM record: the agent writes a note on the contact before the transfer, and the record opens when the call rings through. Teams that answer on mobile phones may do better with a text message to the person receiving the call a few seconds before it connects. Either way, test it on the device that person really uses, at the speed a real call arrives.

If neither is possible yet, fall back on speech. The agent can tell the caller what it's passing on, or your phone system may be able to play the person answering a short spoken summary before the call connects. It's clumsier than a screen, and it still beats a blank one.

What the caller hears in between

The seconds between the AI's last sentence and the human's first are dead air unless you plan them. Have the agent say who the caller is going to and what it has passed on, for example: "I'm connecting you with our office manager, and I've sent her your account details so you won't have to repeat them." Only let it say the second half if the note really arrives. A promise that the person already knows, followed by "Can I get your name?", lands worse than no promise at all.

The note also carries personal details, so treat it like any other customer record. Decide who can see it and how long the call summary is kept, and write that down next to the triggers below. It's a dull decision to make before launch and a painful one to make after a customer asks.

Decide the triggers before the first call

A handoff you designed is better than one the AI improvises. Write the triggers down before launch, and keep them conservative at first:

• The caller asks for a person, in any wording. Don't make them ask twice.
• The call involves money the AI isn't cleared to handle, such as refunds or a dispute over a charge.
• The caller is upset. Complaints go to a person early, before the AI has tried a second time.
• The AI can't confirm who the caller is, and the request needs that.
• The conversation drifts outside the topics the AI is meant to cover.

Decide as well what happens when nobody picks up. A transfer to an empty desk is worse than no transfer. The agent should know the hours when a person is available, and outside them it should take a message with the same five fields and tell the caller honestly when someone will call back.

Test the seam before the conversation

Most pre-launch testing checks whether the AI answers questions correctly. Test the handoff separately. Have someone play a caller with a complaint, get transferred, and watch what arrives on the screen and how long it takes. Then have the person answering try to finish the call without asking the caller to repeat anything. Every question they have to ask is a field missing from the note.

Run the same test outside business hours, with a caller who won't give a name and with one who asks for a person in the first sentence. These are the calls that embarrass a system in its first week when nobody tried them beforehand.

If you're evaluating voice AI now

Ask any vendor or developer to show you a transfer before they show you a conversation. Watch what the receiving person sees and how fast it arrives. Then ask what happens when nobody answers. I'll grant that a smooth demo conversation is genuinely hard to build and worth something. The transfer tells you more about how carefully the rest was built.

The free Realtime API guide covers the rest of the build: use case qualification, the architecture with telephony and CRM connection, disclosure and compliance, conversation design for AI, and testing before go-live. Its chapter on handoff design is the one I'd read first, with your own call flow open beside it.

← Back to all articles