Feedback Actions

Update From Voice Feedback Action

Explains the Voice feedback action that runs during and after a live AI voice call. Useful for handling requests from the voice assistant, keeping the conversation history, and wrapping up when the call ends.

Overview

A live voice call is started with the Start Voice Chat task. While the call runs, and once when it ends, the _update_from_voice feedback action is triggered on the stub.

stubpost.data._incoming_voice_data.event tells you why the action ran:

`event`	`delegated`	When
`delegation`	`true`	The assistant needs the stub's help, eg to look something up or make a booking
`interval`	`false`	Every `update_interval_seconds` of the call, when there is new transcript. A delegation restarts the interval
`session_ended`	`false`	Once, when the call has ended, however it ended

Each update only carries what was said since the previous update. A sentence that was still being said when an update was sent continues in the next one.

The transcript since the previous update is set to the message of the stubpost, i.e. stubpost.message, as lines like:

        Caller: Hi, do you have a table for two this Thursday at seven
Assistant: Let me check that for you.

For session_ended, the message starts with a Voice session ended: <reason> line.

All the details are available in the stubpost.data._incoming_voice_data data path.

NOTE

A delegation doesn't say what the assistant needs. Work it out from the messages, eg by running a GPT Chat Task with them.

Handling the events

delegation

Run a GPT Chat Task with the messages.
When its answer arrives, send it to the call with a commentary voice notification and the delegation_id:

While it waits, the assistant tells the caller it is checking.

interval

Store the messages in the chat history, so the agent has the whole conversation the next time it runs.

session_ended

Wrap up, eg save a summary of the call or send a follow-up. session_ended.reason tells you how the call ended.

The charge for the length of the call comes with this update, and is added to the stubpost's transaction history like the charges of tasks.

Data Structure

Feedback Action Data

Example data that is passed to the _update_from_voice feedback action for a delegation:

        "_incoming_voice_data": {
  "event": "delegation",
  "delegated": true,
  "voice_session_uuid": "ae2ca8c6-ddf3-4840-9255-d9520a9cac7e",
  "chat_name": "Restaurant booking",
  "channel": "webchat",
  "messages": [
    { "role": "assistant", "content": "Hi, this is the restaurant assistant. How can I help?" },
    { "role": "user", "content": "Hi, do you have a table for two this Thursday at seven" }
  ],
  "transcript": [
    { "speaker": "assistant", "delta": " Hi,", "start_ms": 1000, "end_ms": 1200 },
    { "speaker": "assistant", "delta": " this", "start_ms": 1200, "end_ms": 1400 }
  ],
  "delegation_id": "item_ETSXQ2Sev4SEHt5NNoRmx",
  "offset_ms": 6200
}

When the call has ended:

        "_incoming_voice_data": {
  "event": "session_ended",
  "delegated": false,
  "voice_session_uuid": "ae2ca8c6-ddf3-4840-9255-d9520a9cac7e",
  "chat_name": "Restaurant booking",
  "channel": "webchat",
  "messages": [
    { "role": "assistant", "content": "We're closing the line now, goodbye." }
  ],
  "transcript": [],
  "session_ended": {
    "status": "closed",
    "reason": "stub_hangup",
    "usage_seconds": 52,
    "usage_confirmed": true
  }
}

Properties

event
string

Why the action ran: delegation, interval or session_ended.

delegated
boolean

true when the assistant needs the stub's help, i.e. when event is delegation.

voice_session_uuid
string

The id of the voice call, from Start Voice Chat.

chat_name
string

The chat_name the call was started with.

channel
string

The channel of the call, eg webchat.

messages
array

What was said since the previous update, as chat messages. The caller is user and the assistant is assistant. This is the same format as the GPT Chat Task messages.

Empty when nothing new was said, eg for a session_ended straight after a delegation.

transcript
array

The timed fragments behind the messages. Each fragment has a speaker (caller or assistant), the delta text, and start_ms and end_ms, the time in milliseconds from the start of the call.

delegation_id
string

delegation only. Pass it back with the answer, so the assistant knows which request it answers.

offset_ms
number

delegation only. When the assistant delegated, in milliseconds from the start of the call.

session_ended.status
string

session_ended only. closed, or abandoned when the caller's channel never joined the call.

session_ended.reason
string

session_ended only. How the call ended:

`reason`	Meaning
`coms_hangup`	The caller hung up
`coms_disconnected`	The caller's connection dropped
`stub_hangup`	The template ended the call with `hangup`
`join_timeout`	The caller's channel didn't join within 60 seconds
`expired`	The call reached the 120 minute limit
`content`	A safety filter ended the call
`connection_lost` / `remote_hangup`	The connection to the AI provider was lost
`gateway_shutdown`	The voice service restarted during the call

session_ended.usage_seconds
number

session_ended only. The billed length of the call in seconds. Time before the caller joined isn't billed.

session_ended.usage_confirmed
boolean

session_ended only. true when the AI provider confirmed the final usage_seconds. When false, usage_seconds is the last known value.

JSONSchema

The JSONSchema of the stubpost.data._incoming_voice_data is:

        {
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Incoming Voice Data",
  "type": "object",
  "properties": {
    "_incoming_voice_data": {
      "type": "object",
      "properties": {
        "event": {
          "type": "string",
          "enum": ["delegation", "interval", "session_ended"],
          "description": "Why the action ran."
        },
        "delegated": {
          "type": "boolean",
          "description": "True when the assistant needs the stub's help."
        },
        "voice_session_uuid": {
          "type": "string",
          "description": "UUID of the voice call."
        },
        "chat_name": {
          "type": ["string", "null"],
          "description": "The chat_name the call was started with."
        },
        "channel": {
          "type": ["string", "null"],
          "enum": ["webchat", "dial", "whatsapp", null],
          "description": "The channel of the call."
        },
        "messages": {
          "type": "array",
          "description": "What was said since the previous update, as chat messages.",
          "items": {
            "type": "object",
            "properties": {
              "role": { "type": "string", "enum": ["user", "assistant"] },
              "content": { "type": "string" }
            },
            "required": ["role", "content"]
          }
        },
        "transcript": {
          "type": "array",
          "description": "The timed fragments behind the messages.",
          "items": {
            "type": "object",
            "properties": {
              "speaker": { "type": "string", "enum": ["caller", "assistant"] },
              "delta": { "type": "string" },
              "start_ms": { "type": "number" },
              "end_ms": { "type": "number" }
            },
            "required": ["speaker", "delta"]
          }
        },
        "delegation_id": {
          "type": "string",
          "description": "delegation only. Pass it back with the answer."
        },
        "offset_ms": {
          "type": "number",
          "description": "delegation only. When the assistant delegated, in ms from the start of the call."
        },
        "session_ended": {
          "type": "object",
          "description": "session_ended only.",
          "properties": {
            "status": { "type": "string", "enum": ["closed", "abandoned"] },
            "reason": { "type": "string" },
            "usage_seconds": { "type": "number" },
            "usage_confirmed": { "type": "boolean" }
          },
          "required": ["status", "reason", "usage_seconds", "usage_confirmed"]
        }
      },
      "required": ["event", "delegated", "voice_session_uuid", "messages", "transcript"]
    }
  },
  "required": ["_incoming_voice_data"]
}

Links

Options

Update From Voice Feedback Action

Overview

Handling the events

delegation

interval

session_ended

Data Structure

Feedback Action Data

Properties

event
string

delegated
boolean

voice_session_uuid
string

chat_name
string

channel
string

messages
array

transcript
array

delegation_id
string

offset_ms
number

session_ended.status
string

session_ended.reason
string

session_ended.usage_seconds
number

session_ended.usage_confirmed
boolean

JSONSchema

On this page

Update From Voice Feedback Action

# Overview

# Handling the events

# delegation

# interval

# session_ended

# Data Structure

# Feedback Action Data

# Properties

# event string

# delegated boolean

# voice_session_uuid string

# chat_name string

# channel string

# messages array

# transcript array

# delegation_id string

# offset_ms number

# session_ended.status string

# session_ended.reason string

# session_ended.usage_seconds number

# session_ended.usage_confirmed boolean

# JSONSchema

On this page

Overview

Handling the events

delegation

interval

session_ended

Data Structure

Feedback Action Data

Properties

event
string

delegated
boolean

voice_session_uuid
string

chat_name
string

channel
string

messages
array

transcript
array

delegation_id
string

offset_ms
number

session_ended.status
string

session_ended.reason
string

session_ended.usage_seconds
number

session_ended.usage_confirmed
boolean

JSONSchema