Prompt Engineering: Understanding How ChatGPT Works Under the Hood

ChatGPT, and other large language models (LLMs) like it, are revolutionizing how we interact with computers. They can translate languages, generate creative content, and answer your questions in an informative way, sometimes even mimicking human-like conversation. The most advanced versions of ChatGPT might seem, at face value, almost deterministic. It's tempting to assume that ChatGPT will answer your question accurately no matter how you phrase it, but this is false. The way you ask your question or instruct ChatGPT influences the answer you receive. To truly harness the power of ChatGPT and other LLMs, you need prompt engineering.

What is Prompt Engineering?

Prompt engineering is the art and science of crafting effective prompts to guide LLMs towards desired solutions. It's like providing the LLM with a detailed map and clear instructions to reach a specific destination. In essence, better prompts lead to better results. Here are a few key techniques to improve your prompting:

1. Context: Providing context is crucial for LLMs. This means giving the LLM relevant information before asking your question.
   Example:
   Copy

   
   <context> 
   The American Declaration of Independence was signed in 1776
   
   <question> 
   When was the Declaration of Independence signed? 
                       

   This technique is particularly effective when you want the LLM to answer questions about a specific text. Instead of relying on its general knowledge, the LLM can focus on the information you provide. However, there's a limitation: the context window. This refers to the maximum number of words (or tokens) the LLM can process at once. You've probably encountered this in ChatGPT, where you can only input a limited amount of text. While older models have smaller context windows, newer LLMs have significantly larger ones, sometimes exceeding 100,000 tokens. This allows for more detailed context, which is why the engineers behind these LLMs almost always increase the context window with each new release. Keep in mind that most LLM's APIs are billed based on the number of tokens sent to the LLM and how many it responds with. Extremely large contexts will make some projects that use LLMs financially infeasible.
2. Persona Pattern: This technique involves instructing the LLM to assume a specific persona or role.
   Example:
   Copy

   
   You are a senior software engineer with a vast amount of knowledge in SQL. How would I perform a left join on two tables?
                       

   By assigning a persona, you can guide the LLM to access and utilize specific knowledge domains within its vast network. While the exact mechanism is still being researched, it's likely that persona patterns activate different sections of the LLM's neural network, leading to more focused and relevant responses. You can read more about this pattern in the research paper: https://arxiv.org/pdf/2302.11382.
3. Chain of Thought: This technique encourages the LLM to reason through a problem step-by-step before arriving at a solution.
   Example:
   Copy

   
   How do I cook scrambled eggs? Think through your answer carefully and consider how the ingredients you suggest will interact with one another. Think about how heat or different types of pans will impact how the eggs cook.
                       

   While LLMs don't 'think' in the same way humans do, prompting them to articulate their reasoning process can lead to more accurate and comprehensive answers. This is because it forces the model to consider different aspects of the problem and build a logical chain of thought. You can find more information about this technique in this research paper: http://arxiv.org/pdf/2201.11903
4. Few-Shot Examples: Just like humans, LLMs often benefit from seeing examples. This technique involves providing the LLM with a few input-output pairs before asking your question.
   Example:
   Copy

   
   How many ounces are in a Starbucks Grande? Here are a few examples:
   
   * How many ounces are in a Demi? 3 oz.
   * How many ounces are in a Short? 8 oz.
   * How many ounces are in a Tall? 12 oz.
   * How many ounces are in a Grande? 16 oz. 
   * How many ounces are in a Venti? 20 oz. 
                       

   By providing examples, you help the LLM understand the pattern and apply it to your specific question. This is similar to providing context, but with the added benefit of explicit solutions, making it even easier for the LLM to answer correctly.

Prompt engineering is essentially the art of providing an LLM with a roadmap to the solution. Since effective prompt engineering is crucial for generating high-quality LLM output, it can be considered the programming language of LLMs. It might not be a traditional programming language like C or Python, but it serves the same purpose: instructing the machine to produce a desired output. Proficiency in prompt engineering will be an essential skill for professionals to eek the best performance out of their LLMs.