Value function iterations with transition matrix

We continue to consider the following aggregate model with transition matrix

where

together with

where both A and Q change according to transition matrix.

Today we also implement both error bounds and policy iterations.

Again, the former is implemented by

where

across dimensions.

The latter is implemented by

(1) Set a grid consisting of k and k' columnwise and rowwise respectively.

(2) Calculate utility for consumption as U using the grid matrix above.

(3) Starting from a certain v, update v1=U+beta*v' so as to maximize v1.

(a) Find the corresponding k' from (3) in value function iterations.

(b) Calculate utility from k and k' as r for all dimensions.

(c) Starting from a certain vp, update vp1=U+beta*P*vp where

(d) Repeat the whole thing by setting vp=vp1 until some criterion is met.

(4) Repeat (3) by setting v=v1 for many times.

(5) Find the final corresponding value of k as k' according to the maximum value.

The resulting number of iterations are almost the same as that by the latter but the

value functions based on it are different from policy iterations alone.