1. Concepts
  2. Live Voice

Concepts

Live Voice

Explains how a stub holds a live AI voice call on webchat, WhatsApp or the phone, and where to find the tasks, notifications, feedback, models and voices involved.

Overview

A live voice call puts a caller in a real-time spoken conversation with an AI voice assistant. The call belongs to a stub: the stub starts it, steers it while it's live, and hears back from it during and after the call. A stub can have one live voice call at a time.

The voice assistant talks; the stub does the work. When the assistant needs something it can't answer on its own, such as account details, it hands the request to the stub. The stub handles it, for example with a GPT Chat Task and its tools, and sends the answer back for the assistant to tell the caller.

Channels

Channel How the call starts
Webchat The caller starts a call in the webchat widget. Start Voice Chat starts the call, and a notification on the same action joins the widget to it.
WhatsApp A WhatsApp call arrives with _update_from_whatsapp. Answer WhatsApp Call picks it up.
Phone A call arrives through a SIP connection with _update_from_call. Answer Call picks it up, and Make Call calls out.

A call from start to end

  1. An action runs Start Voice Chat with the model, instructions and voice. The caller's channel joins the call.
  2. While the call is live, the stub steers it with voice notifications or the Voice Command task: answer the assistant's requests, give it new instructions, mute the caller, or hang up.
  3. The call reports back with the _update_from_voice feedback action: when the assistant delegates a request, at a regular interval with the latest transcript, and when the call ends.
  4. Every call is recorded. The recording arrives with one more _update_from_voice shortly after the call ends.

Models

Live voice calls use a live voice model, such as gpt-live-1. Use the stubber-default-voice model alias to always get the model Stubber recommends. See Live Voice Models for the full list.

Voices

Choose the assistant's voice with audio.output.voice on Start Voice Chat. The default is marin.

For the voices available on gpt-live-1, with the language and regional accent of each, see OpenAI's GPT-Live voice options.

The voice is set when the call starts and can't be changed during the call. Each voice has its own timbre, pitch and pacing built in. Use the call's instructions to shape how the assistant speaks, eg its tone, speed or accent.