Template Techniques

LLM as a Task

Explains how to use a separate LLM call as a focused task or tool within a larger conversational-agent workflow.

Also known as LLM as a tool.
When a template defines a conversational agent, like a customer support agent for example, there might be actions that need simple LLM calls as part of an action.
It's important to differentiate between the main chat (conversational agent) and various gpt_chat_task or gpt_call tasks that use LLMs to extract data or perform small parts of the workflow.

# Detailed Example Structure

• A customer support conversational agent will typically have the chat_name set to main
• There might be an action called retrieve_customer_agreement which contains a task to retrieve the customer's agreement which is textual
• The action might then have a gpt_chat_task task to pass the textual agreement into an LLM
• The LLM would have a system prompt telling it to parse out certain aspects and response with JSON for example
• The gpt_chat_task would then be set to not interfere with any of the conversational parts of the flow with these settings :
  • disable_model_response : stops the model from running a feedback action
  • disable_model_action_execution : stops actions from being run
  • disable_action_exposure : stops actions being exposed to the model as tool options
• The chat_name of the llm as a task call is set to parse_agreement_{{{{skip}}}}{{#sugartime 'now'}}HH_mm_SS{{/sugartime}}{{{{/skip}}}} so that a new chat is save per call for debugging and inspection purposes.

Search the library for "GPT Chat Task implementing LLM as a task" for an example.