LLM Basics

Understanding AI

An introduction into using LLMs with Stubber as an AI Assistant

Introduction

Welcome to the "Understanding AI" section of Stubber's LLM Basics guide. This section serves as an introduction to using Large Language Models (LLMs) within Stubber, helping you harness the power of AI to enhance your workflows.

Artificial Intelligence, specifically LLMs, have transformed the way we interact with technology. They enable machines to comprehend and generate human-like text, opening up new possibilities for automating tasks, improving communication, and delivering smarter solutions. At Stubber, we integrate these advanced AI capabilities to provide you with a robust AI Assistant that can support a wide range of business needs.

This guide will explore the core concepts of LLMs, shedding light on how they function and their potential in Stubber. You’ll learn how to configure and deploy LLMs to enhance productivity, improve user experience, and achieve better outcomes for your users. Whether you’re aiming to streamline internal processes, enhance customer support, or drive efficiencies across your workflows, this guide will equip you with the knowledge to maximize AI’s impact within Stubber.

The Future Potential of LLMs

LLMs are rapidly advancing, becoming more powerful, adaptable, and context-aware. These developments are set to transform various industries and redefine how we interact with technology. Here’s a brief look at where LLMs are making an impact and what we can expect:

Translation and Localization: LLMs now handle cultural nuances and tone, allowing businesses to communicate authentically across languages.
Education and Tutoring: Virtual tutors powered by LLMs offer personalized learning, adapting to individual needs and making education more accessible.
Healthcare Support: LLMs assist in medical diagnostics, offer patient guidance, and improve healthcare access by processing patient information swiftly.
Customer Support: Beyond FAQs, LLMs handle complex queries with empathy and awareness, providing proactive and seamless customer service.
Creative Content: LLMs aid writers, marketers, and designers by generating ideas, writing copy, and even creating visual content.
Research and Knowledge Management: LLMs quickly analyze data, generate insights, and improve information retrieval for professionals.
Productivity Tools: Enhanced virtual assistants streamline task management, identify productivity gaps, and automate tedious tasks, boosting efficiency.

As these capabilities grow, LLMs will increasingly shape how we work, learn, and connect across industries and applications.

Understanding Context Windows

The context window is like the AI's working memory - it determines how much text the model can "see" and consider at once:

What It Means: The context window is the maximum amount of text (including your input and the model's responses) that can be processed in a single conversation. Different Models, Different Capacities:

GPT-3.5: ~4,000 words
GPT-4: ~48,000 words
GPT-4-40-preview: ~40,000 words
GPT-4o mini: 128,000 tokens (~96,000 words, depending on token-to-word ratio)
Claude: ~75,000 words

Practical Impact:

Longer context allows for more detailed conversations Better understanding of complex documents More accurate responses to questions about earlier parts of the conversation

Best Practices:

Break large tasks into smaller chunks when needed Prioritize most relevant information first Clear old context when starting new topics

Understanding Temperature

Temperature is a key setting that controls how your AI responds. Think of it as adjusting the AI's creativity level:

Low Temperature (0.1-0.3): Produces more focused, predictable responses. Ideal for tasks requiring accuracy like factual queries or technical documentation.

Example: If you ask, "How can Stubber help me with customer support?" a low temperature setting might respond with, "Stubber helps by providing instant, accurate responses to common queries."

Medium Temperature (0.4-0.7): Balances creativity with consistency. Perfect for general conversation and content creation.

Example: Asking "How can Stubber help me with customer support?" might yield, "Stubber is great for supporting customers by answering FAQs and personalizing responses."

High Temperature (0.8-1.0): Generates more creative and diverse outputs. Suitable for brainstorming and creative writing.

Example: At a high temperature, the response to "How can Stubber help me with customer support?" might become, "With Stubber, you have a virtual assistant that not only answers customer questions but also adds a friendly, conversational tone to interactions!"

Choosing the right temperature helps you get the most appropriate responses for your specific needs, whether you're looking for precise information or creative suggestions.

Understanding Tokens

Tokens are essentially pieces of words that an LLM processes when generating or responding to text. Each word or part of a word is split into tokens, which the model then interprets to produce a coherent response. This process allows LLMs to handle complex language patterns efficiently, as it doesn’t always interpret a word as a single unit—often breaking it down further.

Let’s go over how tokenization works in more detail, with an example of the word "Stubber."

How Tokenization Works

In many LLMs (like GPT-3 and GPT-4), tokenization is done using something called Byte Pair Encoding (BPE). BPE splits words into the smallest meaningful pieces based on patterns in the training data, so common words or phrases are often a single token, while less common or complex words are broken into multiple tokens.

For example:

Simple, common words like “and,” “it,” and “the” are usually just one token each.
Unique or compound words like “Stubber” might be broken down into several tokens if they’re not frequently seen in the training data.

Example of “Stubber” in Tokens

For many LLMs, the word “Stubber” would likely be represented by two tokens because it's not a common word in the model's vocabulary. Here's how it might break down:

"Stub" – 1 token
"ber" – 1 token

So, "Stubber" would use 2 tokens in this case.

Larger Example with 40 Tokens

To illustrate, let’s consider a 40-token example using the word "Stubber" in context:

“Stubber is a powerful AI-driven tool that leverages LLMs to automate workflows, improve customer service, and enhance productivity through advanced AI features. With Stubber, you can streamline repetitive tasks, analyze data efficiently, and even personalize responses to align with your company’s specific needs.”

This sentence contains about 40 tokens, depending on the exact model and tokenization rules. Each word or part of a word (like prefixes or suffixes) counts as a token, so using precise language and removing unnecessary words can help you manage token usage efficiently.

Why Tokens Matter

Token Limit: LLMs have a token limit in their context window, so understanding token usage helps ensure your inputs fit within that limit.
Cost Efficiency: Many AI platforms charge based on token usage, so concise inputs save costs.
Response Quality: Reducing tokens can help focus the model’s attention, leading to clearer and more relevant responses.

Understanding tokens helps you optimize prompts and responses, ensuring that you get the best performance from LLMs in Stubber!

Prompting

Prompt engineering is the art of crafting specific, clear prompts to get the best results from LLMs. Well-designed prompts lead to more accurate and helpful responses.

See more on prompt engineering : Prompt Engineering