1. Guides
  2. LLM Basics


LLM Basics

How Large Language Models work and how to use them


Large Language Models (LLMs) are a type of AI that can be used to generate text.
They can generate text that is similar to the text they were trained on.
The effect of this is that they show reasoning abilities which can be harnessed to create AI Assistants.

How LLMs work

For the purposes of Stubber you need to have a simple understanding of LLMs and how they work.


Each LLM is customized (or fine-tuned) for a specific text input and output.
The models that Stubber primary uses are chat models. These models take text in the form of a conversation between an assistant and a user.
Some LLM models are trained only on text and these are used for different tasks.

We will focus on the chat LLM type.


To use a chat LLM you need to pass a chat log and the model is asked to complete the chat.
Each time you call the LLM you need to give it the chat log.
Stubber abstracts this from you and will automatically build up the chat log to give to the LLM.

Each call to a LLM is atomic and the LLM does not store any state. So when dealing with a LLM a tool that keeps the memory is very useful.


The chat LLMs are trained to understand "roles". The 2 most basic roles are "assistant" and "user".
This represents the most basic chat.
The next most important role is the "system" role. LLMs have been trained to adhere to the messages in the "system" role as instructions.
This allows you to give instructions to the LLM and pass data to the model outside of the user chat.

Some LLMs support a "function" role where the LLM can complete functions. The functions the model can choose to call are fed into the model by special "system" messages. This is functionality that is useful in the context of Stubber and allowing LLMs to run actions.