Tasks

Start Voice Chat

Starts a live AI voice call for a stub. Useful for voice assistants on webchat, WhatsApp calls and phone calls that talk to the caller in real time and hand work back to the stub.

The task connects to the live voice model straight away, so the assistant is ready before the caller speaks. It returns a voice_session_uuid that the caller's channel uses to join the call.

While the call is live:

• The stub steers the assistant with voice notifications or the Voice Command task.
• The call reports back to the stub with the _update_from_voice feedback action: when the assistant needs the stub's help, at a regular interval with the latest transcript, and once when the call ends.

A stub can have one live voice call at a time.

# Basic usage

# Parameters

# model

required

string

The live voice model to use, eg gpt-live-1. You can also use the stubber-default-voice Stubber model alias, which currently points to gpt-live-1.

See Models for a full list of supported models.

# instructions

optional

string

The assistant's prompt: who it is, how it should speak, and when it should hand work back to the stub.

The assistant can't look anything up or take actions itself. Tell it to delegate anything that needs the stub, and to tell the caller it is checking while it waits.

# chat_name

optional

string

A name for the call. It is sent back in every _update_from_voice.

# channel

optional

string

The channel the caller joins on. The channel picks the default audio format.

# greeting

optional

string

Instructions given to the assistant as soon as the caller joins, so the assistant speaks first. Eg "Greet the caller now in English, then listen."

Without a greeting, the assistant waits for the caller to speak.

# update_interval_seconds

optional

number

How often the call sends the latest transcript to the stub with _update_from_voice, while the call is live. An update is only sent when there is something new.

Between 5 and 3600, or 0 to turn interval updates off.

Default: 30

# api_region

optional

string

The data residency region the call runs in. One of eu or us.

Default: eu

# audio.output.voice

optional

string

The assistant's voice, eg marin, quartz or vesper.

The baseline "personality" of each voice—its timbre, pitch, and standard pacing—is built into the vocal model itself.

Default: marin

# audio.format

optional

object

The audio format of the call, as { "type": "audio/pcm", "rate": 24000 }. Supported formats are audio/pcm at 16000 or 24000, and audio/pcmu or audio/pcma at 8000.

Only set this when you need a specific format. The channel's default is the best choice in almost all cases.

The exception is phone calls: set it to the call's format, "~~stubpost.data._incoming_dial_data.call_data.audio_format". Phone calls use μ-law or A-law, whichever the phone system prefers, and the voice call has to match. See Answer Dial Call.

A phone call only works in μ-law or A-law at 8000. When channel is dial and audio.format is anything else, eg it's missing from the stubpost, the call starts with the most likely phone format instead, and the result's audio_format_converted says what changed. It may still not match the call's format, so always pass the call's audio_format.

Default: the channel's format, see channel

# input

optional

array

Earlier conversation to start the call with, eg the chat history before the caller started the call. At most 128 messages.

Each message has a role of developer, user or assistant, and exactly one text part in content:

        [
  {
    type: "message",
    role: "user",
    content: [{ type: "input_text", text: "I want to book a table." }],
  },
  {
    type: "message",
    role: "assistant",
    content: [{ type: "output_text", text: "Sure, for when?" }],
  },
];

      

# Result

# Properties

# payload.voice_session_uuid

string

The id of the voice call. Pass it to the caller's channel so it can join the call:

• webchat: see Join the call from webchat
• WhatsApp calls: Answer WhatsApp Call
• phone calls: Answer Dial Call

The channel has to join within 60 seconds. If it doesn't, the call ends as abandoned and the stub gets a session_ended _update_from_voice.

# payload.audio.format

object

The audio format of the call. Pass it to the caller's channel along with voice_session_uuid.

# payload.audio_format_converted

object

Only when the audio.format of a dial call wasn't a phone format: what it was changed to, eg { "from": { "type": "audio/pcm", "rate": 24000 }, "to": { "type": "audio/pcmu", "rate": 8000 } }. Check the template passes the call's audio_format.

# payload._stubbucks

array

The stubbucks charged to start the call. Like with other tasks, these are added to the stubpost's transaction history. The charge for the length of the call follows when the call ends, with the final _update_from_voice.

# Errors

# Examples

# All parameters

# Join the call from webchat

Add a webchat notification to the same action that runs start_voice_chat. It passes the new call to the caller's webchat, which then joins it:

start_voice_chat in stubpost.tasks.start_voice_chat is the name of the task in the action.

# Answer a phone call

On the action that receives _update_from_dial, start the voice call on the dial channel with the call's audio format, then answer the call with Answer Dial Call:

# Run the call in the US

# A typical voice flow

1. The caller starts a call, and an action runs start_voice_chat.
2. The caller's channel joins the call with payload.voice_session_uuid: a notification on the same action for webchat (see Join the call from webchat), or the answer task for WhatsApp calls and phone calls.
3. When the assistant needs help, the stub gets _update_from_voice with delegated: true. Run a GPT Chat Task with the messages.
4. When the answer arrives, send it back with a commentary voice notification and the delegation_id. The assistant tells the caller.
5. End the call with hangup, or wait for the caller to hang up. Either way, the stub gets a final _update_from_voice with event: "session_ended".