Glossary · Phase 5 · Transformers
Query
In attention, what a token is looking for, like a search you type.
Think about searching a recipe website. You type "quick vegetarian dinner". That's your query: what you want. Each recipe carries a few tags, like "vegan", "20 minutes" or "pasta". Those tags are its key: what it can be found by. When a recipe matches, you don't get its tags back. You get the recipe itself. That's the value: what it hands over.
Try it now
Think of a library search: your query matches against labels (keys), and you take home a blend of the contents (values).
Learn it properly
Queries, keys and values · The three projections behind attention. About 30 minutes.
Other terms from the same lesson
- key: In attention, what a token offers to be matched on, like a page's index entry.
- value: In attention, the information a token passes along when it is matched, like the page you get back.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.