Index Ai Academy
Glossary · Phase 3 · Machine learning

Thompson sampling

A bandit method that draws a plausible success rate for each option from its record and plays the highest draw.

Try it now

Each step is smaller than the last, because each new reward is a smaller share of what has been counted.

Learn it properly

Bandits: explore or exploit · Learning from rewards with no labels: a running estimate for each option, why the greedy choice gets stuck, epsilon-greedy and an optimism bonus. About 35 minutes.

Other terms from the same lesson

  • epsilon-greedy: A strategy that takes the best-looking option most of the time and, with a small probability epsilon, picks one at random.
  • explore-exploit trade-off: The choice between trying an uncertain option to learn about it and taking the option that looks best now. You only learn about what you try.
  • multi-armed bandit: One decision repeated many times: choose among options with unknown payouts, see only the reward of the one chosen, and try to collect as much as possible.
  • regret: The reward given up compared with always choosing the best option. It measures what learning cost.
  • upper confidence bound: Scoring each option by its estimate plus a bonus that is large when it has been tried little, then taking the highest score.
Keep learning

More in plain English.

1286 terms in the glossary, each with the lesson that teaches it.

1286 terms

The glossary

Every word you'll meet in AI and machine learning, explained in everyday language.

Phase 3 · lesson 35 min

Bandits: explore or exploit

Learning from rewards with no labels: a running estimate for each option, why the greedy choice gets stuck, epsilon-greedy and an optimism bonus.

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write