Guides

LLM Basics

How Large Language Models work and how to use them

Overview

Large Language Models (LLMs) are a type of AI that can be used to generate text.
They can generate text that is similar to the text they were trained on.
The effect of this is that they show reasoning abilities which can be harnessed to create AI Assistants.

How LLMs work

For the purposes of Stubber you need to have a simple understanding of LLMs and how they work.

Basics

Large Language Models (LLMs) like the ones Stubber uses are designed to understand and generate text based on specific input types. Stubber primarily uses chat models, where the model reads a conversation between a “user” and an “assistant” and responds accordingly. Other LLMs may work with different types of text inputs, but here, we’ll focus on chat-based ones.

Usage

To use a chat LLM you need to pass a chat log and the model is asked to complete the chat.
Each time you call the LLM you need to give it the chat log.
Stubber abstracts this from you and will automatically build up the chat log to give to the LLM.

Each call to a LLM is atomic and the LLM does not store any state. So when dealing with a LLM a tool that keeps the memory is very useful.

Roles

The chat LLMs are trained to understand "roles". The 2 most basic roles are "assistant" and "user".
This represents the most basic chat.
The next most important role is the "system" role. LLMs have been trained to adhere to the messages in the "system" role as instructions.
This allows you to give instructions to the LLM and pass data to the model outside of the user chat.