GPT Chat Task (LLM Task)

Context Compaction

Explains automatic context compaction for GPT Chat Task. Useful for long-running AI conversations that would otherwise grow too large for the model, cost too much, or slow down.

As a chat grows, every call sends more history to the model. Compaction reduces that history automatically, so long conversations keep working without you having to schedule a separate summarization step.

There are two kinds of compaction. A chat should use one or the other, not both.

Both are sticky: set them once on a GPT Chat Task and they apply to every later call in that chat. Each kind has its own enabled flag, so you can switch it on or off without resending its settings, and you can update a single setting without resending the rest.

# Summary compaction

Summary compaction runs the Summarize GPT Chat Task automatically, inside the chat. It accepts the same message_selection, summarization, injection and activation settings, so everything on that page applies here too.

# When it runs

On each call that goes to the model, summary compaction checks its activation settings before the model is called. When they are met, it summarizes the chat and the model then sees the shorter history on that same call.

• It runs on the existing chat history, before the new messages in that call are added.
• It does not run on calls that don't go to the model (submit_to_model: false), or on calls that bring in a tool result. Tool results are added to the chat as they come back from tool calls, and summarizing at that moment could separate a result from its call. Compaction simply runs on the next normal call instead.

# Activation

Whichever is reached first triggers compaction. The current context size is measured from the most recent model call (its prompt plus completion tokens).

# Things to know

• Summaries are permanent. The summarized messages are replaced by the summary, exactly as with the Summarize GPT Chat Task.
• It uses stubbucks for the summary call, on top of the chat's own call.
• If summarization fails, the chat task fails. Check that summarization.model is a valid model before relying on it, because the first summarization only happens once the activation threshold is reached.
• Set thresholds well below the failsafes. The stop_on_high_messages failsafe is checked before compaction runs, so a message threshold close to it may never get the chance to trigger.

# Provider compaction

Provider compaction uses the model provider's built-in compaction. Stubber passes your settings to the provider, and when the context crosses the threshold, the provider compacts it on its side.

• OpenAI models on the Responses API only. You can check which API a chat uses in _gpt_chat_details.api, which is "responses" for these models. For any other model the setting is ignored.
• compact_threshold must be a positive whole number of tokens. Underscores are allowed in a string value, so "200_000" is accepted. An invalid value fails the task and the setting is not stored.
• The stored chat history is not shortened. The provider compacts what it processes; the messages kept in the chat stay as they are.

# Seeing what happened

Every call on a chat with compaction settings reports what compaction did in _gpt_chat_details.compaction, even when nothing was compacted.

        {
  "summary": {
    "enabled": true,
    "activated": true,
    "message_count_before": 64,
    "message_count_after": 4,
    "gpt_summary_uuid": "..."
  },
  "provider": {
    "enabled": true,
    "compacted": false,
    "reason": "provider did not compact"
  }
}

      

summary only appears when summary compaction settings exist on the chat, and provider only when provider compaction settings exist.

# summary.enabled

boolean

Whether summary compaction is switched on for this chat.

# summary.activated

boolean

Whether the chat was summarized on this call.

# summary.message_count_before and summary.message_count_after

integer

The number of messages in the chat before and after summarizing. Only present when the chat was summarized.

# summary.gpt_summary_uuid

string

Identifies the summary record created by this compaction. Only present when the chat was summarized.

# provider.enabled

boolean

Whether provider compaction is switched on for this chat.

# provider.compacted

boolean

Whether the provider compacted the context on this call.

# reason

string

Why nothing was compacted, when nothing was. Present on both summary and provider.

# Turning compaction off

Set enabled to false for that kind of compaction. Its other settings are kept, so you can switch it back on later with enabled: true alone.