Index Ai Academy
Glossary · Phase 3 · Machine learning

Return

In reinforcement learning, the total reward collected from a point on, often with later rewards discounted.

The agent's aim isn't the next reward. It is the total over the whole episode, called the return. One refinement turns out to matter a great deal: a reward that arrives later is counted for a little less.

Try it now

A reward k moves away is worth gamma to the power k of its face value today.

Learn it properly

Decisions in sequence: states, rewards and value · A Markov decision process on a grid world: discounted return, the value of a state, the Bellman equation, and value iteration to the best policy. About 40 minutes.

Other terms from the same lesson

  • Bellman equation: The value of a state is the reward for the next move plus the discount factor times the value of the state that move leads to.
  • discount factor: A number between 0 and 1, written gamma, that a reward is multiplied by for every step it arrives later. It sets how far ahead an agent cares.
  • episode: One run of an agent from a starting state to an end state.
  • Markov decision process: A problem described by states, actions, rewards and where each action leads, in which the next state depends only on the current state and action.
  • policy: A rule that picks an action for every state an agent can be in. When a language model is trained with rewards, the model itself is the policy.
  • value iteration: Finding the best policy by sweeping the Bellman equation over every state, taking the best action each time, until the values stop changing.
Keep learning

More in plain English.

1286 terms in the glossary, each with the lesson that teaches it.

1286 terms

The glossary

Every word you'll meet in AI and machine learning, explained in everyday language.

Phase 3 · lesson 40 min

Decisions in sequence: states, rewards and value

A Markov decision process on a grid world: discounted return, the value of a state, the Bellman equation, and value iteration to the best policy.

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write