Index Ai Academy
Glossary · Phase 5 · Transformers

Linear probe

Also written: linear probes, linear probing

Freezing a pretrained encoder and fitting only a linear classifier on its features, to measure how useful they are.

After pretraining comes the test. Freeze the encoder, run the labeled photos through it, and fit a plain logistic regression on its output vectors. This is a linear probe: if a simple classifier does well on the encoder's features, the encoder has already done the hard part.

Try it now

The classifier is the same each time. Only the features it reads changed.

Learn it properly

Self-supervised learning: labels from the data itself · Pretrain an encoder with no labels by matching two views of each image, test it with a linear probe, and see what the augmentations remove, what collapse looks like, and how filling in masked parts differs. About 45 minutes.

Other terms from the same lesson

  • masked image modeling: Pretraining by hiding parts of an image and training a model to fill them in.
  • representation collapse: When an encoder gives nearly the same vector to every input, so its training loss looks perfect and its features are useless.
Keep learning

More in plain English.

1286 terms in the glossary, each with the lesson that teaches it.

1286 terms

The glossary

Every word you'll meet in AI and machine learning, explained in everyday language.

Phase 5 · lesson 45 min

Self-supervised learning: labels from the data itself

Pretrain an encoder with no labels by matching two views of each image, test it with a linear probe, and see what the augmentations remove, what collapse looks like, and how filling in masked parts differs.

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write