SU-CS238 OCT172023

Notation

“state variables” represent the contents of the state; “state” is a complete assignment of state variables.

New Concepts

• Markov Decision Process
  • stationary Markov Decision Process
  • finite-horizon models + infinite-horizon models
• policy
  • stationary policy
  • optimal policy and optimal value function
  • policy evaluation and policy iteration
    • lookahead equation
    • Bellman Expectation Equation
    • action-value function and value-function policy
    • advantage function

Important Results / Claims

• policy evaluation methods
  • solving for the utility of a policy
  • finding the best policy
• policy iteration

Questions

• why is it d seperated

Interesting Factoids