Index Ai Academy
Glossary · Phase 5 · Transformers

Inductive bias

Also written: inductive biases

What a model assumes before it sees any data, such as a CNN assuming that nearby pixels belong together.

Try it now

Press Next. Every box is a tensor, and its shape is written under its name.

Learn it properly

Build a vision transformer · Patch embeddings, a class token, position embeddings and an unmasked block, trained in the page; then two failures that explain it: no positions means no layout, and few images means losing to a CNN a fraction of its size. About 45 minutes.

Other terms from the same lesson

  • class token: A learned vector put in front of an image's patch tokens. It gathers information through attention, and the answer is read from it.
  • vision transformer: A transformer for images: it cuts the picture into small patches and treats each patch like a word.
Keep learning

More in plain English.

1286 terms in the glossary, each with the lesson that teaches it.

1286 terms

The glossary

Every word you'll meet in AI and machine learning, explained in everyday language.

Phase 5 · lesson 45 min

Build a vision transformer

Patch embeddings, a class token, position embeddings and an unmasked block, trained in the page; then two failures that explain it: no positions means no layout, and few images means losing to a CNN a fraction of its size.

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write