Index Ai Academy
Phase 6 · Modern LLM engineeringPost-training

Base models vs assistants

About 30 minLooking ahead: some prerequisites aren't done yet

Why this matters

Every assistant you use started as a base model. A base model is a document completer, trained on raw text. The parts you think of as "the model" come later. Answering questions, following instructions, taking turns, declining some requests, a steady voice: all of that is added in post-training. This phase is about that layer. It's also the layer you can change yourself, with fine-tuning.

  • Contrast a base model's behavior with an assistant's on the same prompt
  • Explain how a conversation becomes one token sequence through a chat template
  • List what post-training adds (and what it mostly doesn't) and the usual stages

Same model, different habits

Picture two people who have read the same huge pile of shop catalogs. The first has only ever read. Hand them a note that says "Describe the camp mug." They keep writing catalog text, as if the note were the start of a page. The second has also practiced as a shop assistant. They read the note as a request, answer it, and stop.

That's the gap between a base model and an assistant (also called a chat model). A base model comes straight out of pretraining, the stage from Phase 5. It continues text. An assistant is the same kind of model after extra training on conversations. That extra training is called post-training.

The academy's practice API serves two versions of the tiny GPT from Phase 5:

  • tiny-gpt is a base model, pretrained on the product catalog.
  • tiny-gpt-chat is the same model after a short round of supervised fine-tuning on chats. That means more training on example conversations. The next lesson shows how.

load_pretrained downloads one of them. It works like from_pretrained, which downloads a model from the Hugging Face Hub, which is the public library of open models. Here both models get the same prompts. Predict which one answers the request, then run it:

import torch
from tinygpt import load_pretrained

base, tok = load_pretrained("tiny-gpt")
chat, _ = load_pretrained("tiny-gpt-chat")
end = tok.token_id("<end>")

def complete(model, prompt, n=40):
    ids = torch.tensor([tok.encode(prompt)])
    out = model.generate(ids, n, temperature=0.7, stop_id=end)
    return tok.decode(out[0, ids.shape[1]:].tolist())

torch.manual_seed(0)
chat_prompt = "<user> Describe the Cove camp mug . <assistant>"
print("BASE, chat prompt:    ", complete(base, chat_prompt))
print("CHAT, chat prompt:    ", complete(chat, chat_prompt))
print("BASE, document prompt:", complete(base, "Our Cove camp mug is made from", 25))

The base model has never seen <user>, <assistant> or "Describe" in use. They are in its vocabulary, but they never appeared in its training text. So it doesn't treat the prompt as a request. It keeps writing catalog text from wherever the prompt stops. Nothing tells it when to stop, either.

The chat model knows the format. It answers about the product you asked about. Then it ends its turn with <end>. It doesn't always get this right. Run the cell again and it sometimes describes the wrong product or repeats itself. The RLHF and DPO lessons, which are the two post-training methods this phase teaches, train those slips away.

The base model isn't useless, though. Write the same request as the start of a document, and it gives a fluent completion.

Real models work the same way. Before chat models, people prompted GPT-3 by writing the start of the document they wanted, such as "Product description for the Kestrel tent:". Or they showed a few examples first.

Note: torch here is the course's in-browser version of PyTorch, cut down to what the lessons need. The tiny GPTs are far too small to be useful. They show the mechanics.