Beyond curse of dimensionality
In grid method, we basically update the following:
In matrix, it is equivalent to
Since the maximization is performed column by column, we can calculate
from j=1 to
n
Here, 
is calculated
from 
and 
.
All we have to keep is vector v as well as vector k. We do not have to keep n x n matrix
in the calculation.
After getting vjf, we apply one of the methods from error bounds to modified policy
iterations mentioned before. We can select one of the following:
0: no
1:error bounds
2:policy iterations
3:policy iterations with error bounds
4:modified policy iterations
5:modified policy iterations with error bounds
We also implement collocation method. The calculation will be done much quicker.