Tasks

Summarize GPT Chat Task

Summarize long-running LLM conversations to manage context and improve performance.

This task takes an existing GPT Chat Task conversation and creates a comprehensive summary of the messages. The original messages are replaced with the summary, helping to:

• Reduce token usage in long conversations
• Maintain context while staying within model limits
• Improve response quality by condensing historical information
• Manage conversation length for better performance

# How It Works

1. The task selects messages from an existing chat based on your criteria
2. These messages are sent to an LLM with instructions to create a comprehensive summary
3. The summary replaces the original messages in the chat
4. Future interactions in the chat will use the summary as context instead of the full message history

# Basic Usage

Summarize a chat when it reaches 50 messages, excluding the main system message:

# Parameters

# Identifier Parameters

# gptchattaskuuid

optional

string

The unique identifier of the GPT chat to summarize. If not provided, it will be generated from stubref and chat_name.

Default: Generated from stubref and chat_name

# stubref

optional

string

The stub reference containing the chat. Used to identify the chat when gptchattaskuuid is not provided.

Default: Current stub's stubref

# chat_name

optional

string

The name of the chat to summarize. This should match the chat_name used in the GPT Chat Task.

Default: ""

# Message Selection Parameters

Control which messages from the chat are included in the summarization.

# message_selection.system_message_operations.main_system_message

optional

string

Controls whether the first system message in the chat is included in the summarization.

• "include": Include the main system message in summarization
• "exclude": Exclude the main system message (recommended - keeps your instructions intact)

Default: "exclude"

# message_selection.system_message_operations.intermediate_system_messages

optional

string

Controls whether system messages that appear after the first message are included in the summarization.

• "include": Include all intermediate system messages
• "exclude": Exclude intermediate system messages (but preserve previous summaries)

Default: "exclude"

# message_selection.system_message_operations.all_system_messages

optional

string

A convenience parameter that sets both main_system_message and intermediate_system_messages to the same value.

Default: Not set (individual parameters take precedence)

# message_selection.skip_messages.first

optional

integer

Number of messages to skip from the beginning of the chat (after excluding the main system message if configured).

Use this to preserve recent context that shouldn't be summarized yet.

Default: 0

# message_selection.skip_messages.last

optional

integer

Number of messages to skip from the end of the chat.

Use this to keep the most recent messages as full messages instead of summarizing them.

Default: 0

# Summarization Parameters

# summarization.model

optional

string

The LLM model to use for generating the summary. Can be different from the model used in the original chat.

See LLM Models for available models.

Default: Uses the same model as the chat being summarized

# summarization.summary_instruction

optional

string

Additional custom instructions for the LLM when creating the summary. These are added to the default comprehensive summarization instructions.

Example:

Default: null

# summarization.temperature

optional

number

Controls randomness in the summary generation. Lower values (0.1-0.3) produce more focused, deterministic summaries. Higher values (0.7-0.9) produce more creative summaries.

Default: Model default

# summarization.max_tokens

optional

integer

Maximum number of tokens the summary can contain. Use this to control the length of the generated summary.

Default: Model default

# summarization.include_summary_timestamp

optional

boolean

Whether to prepend a timestamp to the summary indicating when it was generated.

Example: "Summary generated on 2025-02-04T10:30:00.000Z"

Default: true

# summarization.include_previous_summaries

optional

boolean

Whether to include previous summaries when creating a new summary. This allows you to summarize a chat multiple times while maintaining the context from earlier summaries.

• true: Previous summaries are included in the new summarization
• false: Previous summaries are treated like regular messages and can be summarized again

Default: true

# summarization.include_default_summary_prefix

optional

boolean

Whether to add a default prefix to the summary explaining what it represents.

Example prefix:

        CHAT SUMMARY:
This summary encapsulates the key points from the last 25 messages in your conversation with John.
It is intended to provide context for future interactions, so please review it carefully.

      

Default: false

# Injection Parameters

# injection.insert_summary_as_role

optional

string

The role to assign to the summary message when it's inserted into the chat.

• "system": Inserts as a system message (recommended - the user doesn't have context of the summary)
• "assistant": Inserts as an assistant message

Default: "system"

# Activation Parameters

# activation.message_count

optional

integer

Minimum number of messages required in the chat before summarization will occur. If the chat has fewer messages than this threshold, the task will skip summarization.

This is useful for automatically triggering summarization only when needed.

Example:

If the chat has 35 messages, summarization is skipped. If it has 40 or more, summarization proceeds.

Default: 0 (always summarize)

# Result

# When Summarization Occurs

        {
  "success": true,
  "message": "Response received",
  "payload": {
    "chat_summarized": true,
    "prompt": {
      "model": "gpt-4o",
      "messages": [
        // Messages that were sent to the LLM for summarization
      ]
    },
    "response": {
      "id": "chatcmpl-...",
      "object": "chat.completion",
      "model": "gpt-4o",
      "choices": [
        {
          "index": 0,
          "message": {
            "role": "assistant",
            "content": "Summary generated on 2025-02-04T10:30:00.000Z\n\nThe conversation began with..."
          }
        }
      ],
      "usage": {
        "prompt_tokens": 2450,
        "completion_tokens": 312,
        "total_tokens": 2762
      }
    },
    "messages_before": [
      // Array of messages before summarization
    ],
    "messages_after": [
      // Array of messages after summarization (with summary inserted)
    ],
    "activation": {
      "total_message_count": 45,
      "activate_message_count": 40
    }
  }
}

      

# When Activation Criteria Not Met

# Properties

# chat_summarized

boolean

Indicates whether the chat was actually summarized. Will be false if activation criteria weren't met.

# prompt

object

The prompt that was sent to the LLM, including the model and messages array.

# response

object

The full response from the LLM provider, including the generated summary and token usage details.

# messages_before

array

The complete message array before summarization occurred.

# messages_after

array

The complete message array after summarization, with the summary message inserted.

# activation

object

Details about the activation criteria and current message count.

# Examples

# Summarize Every 30 Messages, Keep Last 5

Automatically summarize when the chat reaches 30 messages, but keep the 5 most recent messages as full messages:

# Custom Summary Instructions

Create a summary with specific focus areas:

# Use a Different Model for Summarization

Use a faster, cheaper model for summarization while using a more powerful model for the actual chat:

# Multiple Summarizations with Context Preservation

Summarize multiple times in a long conversation while preserving previous summaries:

This configuration:

• Triggers every 60 messages
• Keeps the last 10 messages as full messages
• Preserves previous summaries
• Adds timestamps to track when each summary was created
• Provides special instructions for maintaining chronological context

# Best Practices

# When to Summarize

Consider summarizing when:

• Conversations exceed 30-50 messages
• Token costs are becoming significant
• Model performance degrades due to long context
• You want to maintain long-term memory across many interactions

# What to Preserve

Use skip_messages.last to keep recent messages unsummarized:

• Recent messages often contain the most relevant context
• Summaries can lose nuance - keep critical recent exchanges as full messages
• Recommended: keep 5-10 recent messages

# System Messages

Set main_system_message: "exclude" to:

• Preserve your original instructions to the AI
• Avoid summarizing static configuration
• Keep your prompts intact across summarizations

# Custom Instructions

Use summary_instruction to:

• Focus on domain-specific information
• Ensure critical data types are captured
• Guide the summary format for your use case

# Model Selection

Consider using a cheaper model for summarization:

• Summarization is often simpler than generation
• gpt-4o-mini or claude-haiku-4-5-20251001 work well
• Save costs while maintaining quality