Value function iterations with transition matrix

We continue to consider the following case with transition matrix

where

Today we implement policy iterations. Recall the following algorithm:

(1) Set a grid consisting of k and k' columnwise and rowwise respectively.

(2) Calculate utility for consumption as U using the grid matrix above.

(3) Starting from a certain v, update v1=U+beta*v' so as to maximize v1.

(a) Find the corresponding k' from (3) in value function iterations.

(b) Calculate utility from k and k' as r.

(c) Starting from a certain vp, update vp1=U+beta*vp where vp=(1/(1-beta))*r.

(d) Repeat the whole thing by setting vp=vp1 until some criterion is met.

(4) Repeat (3) by setting v=v1 for many times.

(5) Find the final corresponding value of k as k' according to the maximum value.

for a single dimension in 0829.

We now modify the algorithm above for multiple-dimension.

We find

rather than

by matrix inversion across dimensions all at once in each step.

That is implemented by

vp=(inv(eye(dim)-beta*tm)*r')';

for n x dim matrix of r.

The calculation of vp=(1/(1-beta))*r is a special case for single dimension.