Guides

LLM Basics

How Large Language Models work and how to use them

Overview

Large Language Models (LLMs) are a type of AI that can be used to generate text.
They can generate text that is similar to the text they were trained on.
The effect of this is that they show reasoning abilities which can be harnessed to create AI Assistants.

How LLMs work

For the purposes of Stubber you need to have a simple understanding of LLMs and how they work.

Basics

Each LLM is customized (or fine-tuned) for a specific text input and output.
The models that Stubber primary uses are chat models. These models take text in the form of a conversation between an assistant and a user.
Some LLM models are trained only on text and these are used for different tasks.

We will focus on the chat LLM type.

Usage

To use a chat LLM you need to pass a chat log and the model is asked to complete the chat.
Each time you call the LLM you need to give it the chat log.
Stubber abstracts this from you and will automatically build up the chat log to give to the LLM.

Each call to a LLM is atomic and the LLM does not store any state. So when dealing with a LLM a tool that keeps the memory is very useful.

Roles

The chat LLMs are trained to understand "roles". The 2 most basic roles are "assistant" and "user".
This represents the most basic chat.
The next most important role is the "system" role. LLMs have been trained to adhere to the messages in the "system" role as instructions.
This allows you to give instructions to the LLM and pass data to the model outside of the user chat.