Index Ai Academy
Glossary · Phase 6 · Modern LLM engineering

OV circuit

Also written: OV circuits

What an attention head writes once it has looked at a token, as one table made by multiplying its value and output matrices with the embedding and the output layer.

Try it now

Five tensors of the copying model. The bars at the bottom are the singular values, largest first, and the sentence above them says how many hold 90% of the energy.

Learn it properly

Reading weights without running the model · Singular values, and an attention head's token-to-token circuit, read from the matrices alone and then checked against a measurement. About 35 minutes.

Keep learning

More in plain English.

1286 terms in the glossary, each with the lesson that teaches it.

1286 terms

The glossary

Every word you'll meet in AI and machine learning, explained in everyday language.

Phase 6 · lesson 35 min

Reading weights without running the model

Singular values, and an attention head's token-to-token circuit, read from the matrices alone and then checked against a measurement.

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write