Using GPT

GPT Tips and Tricks (LLM)

Some tips and tricks for working with LLMs

Prompting Tips

General

In order to get the best results from the LLM, it is important to give it the right information. Here are some tips to help you get the best results from the LLM:

1. Give very clear and concise instructions
2. Give examples
3. Give a persona and context
4. Give procedures, steps and ways of thinking, allowing the LLM to output it's thinking

Clear and Concise Instructions

When giving instructions to the LLM, it is important to be clear and concise. This means that you should avoid using ambiguous language and be as specific as possible.

• Try using different verbs or adjectives to get the LLM to output different results.
• Try shortening the prompt to get the LLM to output more concise results.
• Try splitting the prompt into multiple parts to get the LLM to output more detailed results.

Give Examples

When giving instructions to the LLM, it can be helpful to provide examples.
Providing examples is one of the surest ways to get the LLM to output the desired results.

When giving examples you could do it in two different ways :

1. System prompt then role play
2. Examples in System prompt

System prompt then role play

In this method of giving examples you explain the full instructions and then ask the LLM to role play the scenario.
You then add in "assistant" and "users" messages in the chat log to give the LLM an example of a good conversation.
Then you tell the LLM to stop the role play and start interacting with live users.

Examples in System prompt

In this method of giving examples you give the LLM examples inside of text blocks in the system prompt.
Here is a recommended format (32 equal signs are 1 token) GPT Tokenizer:

        Example 1
================================
Assistant: Could you please confirm the following information:
First Name : [user first name]
Last Name : [user last name]
Email : [user email]
Phone : [user phone number]
================================

      

Give a Persona and Context

When giving instructions to the LLM, it can be helpful to provide a persona and context.
For persona this could mean :

• give it a name
• tell it it's an expert
• years of experience
• projects completed
• places studied

For context this could mean :

• which company it works for
• where the company is situated
• the types of clients
• the general industry
• products and services

Give Procedures, Steps and Ways of Thinking

When giving instructions to the LLM, it can be helpful to provide procedures, steps and ways of thinking. This can help the LLM understand how to approach the task and what steps to take.

This could take the form of :

        Mode of operation :

- Ask for the user's name
- Ask for the user's email
- Ask for the user's phone number
- Run the submit_user_info function

      

Action descriptions

To get the best performance when describing your actions follow these guidelines:

Provide extremely detailed descriptions. This is by far the most important factor in performance. Your descriptions should explain every detail about the action, including:

• What the action does
• When it should be used (and when it shouldn't)
• What each field means and how it affects the action's behavior (use the help on the field settings)
• Any important caveats or limitations, such as what information the action does not return if the action name is unclear

The more context you can give about your actions, the better the AI will be at deciding when and how to use them. Aim for at least 3-4 sentences per action description, more if the action is complex.

Prioritize descriptions over examples. While you can include examples of how to use a action in its description or in a prompt, this is less important than having a clear and comprehensive explanation of the action's purpose. Only add examples after you've fully fleshed out the action description.

Hallucinations

There are a few ways to avoid hallucinations and detect them.
Avoiding hallucinations is the best way to prevent them from happening.
Detection mechanisms can assist you to detect and recover from hallucinations.

Avoiding Hallucinations

A good checklist to follow to avoid hallucinations is :

• [ ] The prompts are broken up into clear sections
• [ ] Each Stubber AI action has a clear description of when to use it
• [ ] Important fields on Stubber AI actions have a good help text
• [ ] IMPORTANT : All Stubber AI Actions have very clean return data (no AI actions should have the default return data)

Detecting Hallucinations

There is a set of 3 tasks that can be used in conjunction to detect hallucinations.
The first task retrieves the chat_log, the second tasks employs a different LLM to inspect and check the log for the data, and the third task parses this output to give you a boolean value to use to determine if the LLM is hallucinating.

The 3 tasks are in the tasks library on Stubber.