Large Language Models (LLMs)

Large Language Models (LLMs) are revolutionizing the way we interact with technology. These powerful models can understand, generate, and manipulate text in ways that were previously unimaginable. Let's dive into how they work and what makes them so special.

Understanding LLMs: Summarization

One of the most impressive capabilities of LLMs is their ability to summarize text. Think of it like a super-powered "tl;dr" function. These models can take a lengthy article or document and condense it into a concise summary.

For example, imagine you have a news article about a quirky entrepreneur selling snow from Boston online. An LLM can quickly generate a summary highlighting the key points:

• Kyle Waring ships Boston snow for $89 per 6 pounds.
• Snow is not shipped to New England states.
• Waring started the business after joking about shipping snow to warmer states.

The magic behind this is that LLMs have been trained on vast amounts of text where summaries often follow the "tl;dr" token. This allows the model to interpret this token as instructions to generate a summary.

Generating Text: Decoding Methods

To produce text, LLMs use a process called decoding. But how do they decide which words to generate?

Greedy Decoding

A simple approach is "greedy decoding," where the model always chooses the most likely word given the context. While straightforward, this method often leads to repetitive and generic text. Imagine a chatbot that keeps saying the same phrases over and over—that's greedy decoding in action.

Sampling Methods

To avoid repetition, LLMs often use sampling methods. These methods introduce more diversity into the generated text.

Random Sampling

In its simplest form, a word to generate is randomly selected according to its probability. While it may sound intuitive, it can produce bizarre results when the model picks less likely but odd words.

Top-k Sampling

Top-k sampling limits the model's choices to the top 'k' most likely words. This prevents the model from selecting highly improbable words, leading to more coherent output.

Nucleus or Top-p Sampling

Top-p sampling takes a different approach by selecting words that collectively make up a certain percentage ('p') of the probability mass. This method dynamically adjusts the pool of candidate words based on the context.

Temperature Sampling

Temperature sampling reshapes the probability distribution by adjusting the "temperature." Lower temperatures make the model more "greedy," favoring the most probable words, while higher temperatures flatten the distribution, encouraging more diverse outputs.

Training LLMs: Self-Supervision and Teacher Forcing

So, how do these models learn to generate text so effectively?

Self-Supervised Training

LLMs are trained using a technique called self-supervision. This means the model learns from raw text without needing explicit labels. The model is tasked with predicting the next word in a sequence, using the preceding words as context. It's like teaching a child by letting them read and guess what comes next.

Teacher Forcing

During training, LLMs use "teacher forcing." This means the model is always given the correct previous words to predict the next word, rather than relying on its own previous predictions. This helps the model learn more effectively.

Training Data: The Fuel for LLMs

LLMs require massive amounts of training data. This data is often sourced from the internet, including:

• Web text from Common Crawl
• Wikipedia
• Books
• The Pile

However, this data must be carefully filtered for quality and safety. Quality filters remove low-quality or irrelevant content, while safety filters aim to remove toxic or offensive material.

Some ethical concerns with large datasets include:

Concern	Description
Copyright	Much of the text used for training is copyrighted.
Data consent	Website owners are increasingly opting out of web crawling for LLM training.
Privacy	Large web datasets may contain private information like phone numbers and addresses.

Fine-Tuning: Adapting LLMs for Specific Tasks

While LLMs are powerful, they can be further improved for specific tasks through fine-tuning. This involves training the model on a smaller, more focused dataset.

There are different methods for fine-tuning which differs exactly which parameters are updated from the fine-tuning data:

• Retrain all the parameters.
• Freeze some of the parameters.
• Supervised fine-tuning.

Evaluating LLMs: Measuring Performance

How do we know if an LLM is any good?

Perplexity

Perplexity measures how well the model predicts unseen text. Lower perplexity indicates a better model.

Other Factors

Besides perplexity, we also consider:

• Accuracy on specific tasks (e.g., translation, summarization)
• Model size and training time
• Energy usage
• Fairness and bias

Dealing with Scale: Scaling Laws

The performance of an LLM increases with factors such as model size, dataset size, and the amount of training.

These factors are known as scaling laws. Scaling laws are useful in deciding how to train a model to a particular performance

KV Cache

At inference time, we iteratively generate the next tokens one at a time. For a new token that we have just generated, we need to compute its query, key, and values by multiplying by the respective matrices.

Whenever we compute the key and value vectors we store them in memory in the KV cache, and then we can just grab them from the cache when we need them.

Parameter Efficient Fine Tuning

Fine-tuning can be very difficult with very large language models because there are enormous numbers of parameters to train.

Parameter-efficient fine tuning selects a subset of parameters to update when fine-tuning.

An example is called LoRA where, instead of updating matrices during fine-tuning, we freeze these layers and instead update a low-rank approximation that has fewer parameters.

Potential Harms: Addressing the Risks

LLMs have potential downsides, including:

• Hallucinations (generating false information)
• Toxic language and stereotypes
• Bias and unfairness
• Privacy violations and misuse for malicious purposes

It's crucial to mitigate these harms by carefully analyzing training data and ensuring transparency through datasheets and model cards.

Summary

Large Language Models are powerful tools with the potential to transform many aspects of our lives. By understanding how they work, how they are trained, and the potential risks they pose, we can harness their power responsibly and ethically.