Value function iterations with transition matrix

We consider the following case with transition matrix

where

together with

where both A and Q change according to transition matrix.

We implement

(1) Set a grid consisting of k and k' columnwise and rowwise respectively.

(2) Calculate utility for consumption as U using the grid matrix above.

(3) Starting from a certain v, update v1=U+beta*v' so as to maximize v1.

(a) Find the corresponding k' from (3) in value function iterations.

(b) Calculate utility from k and k' as r for all dimensions.

(c) Starting from a certain vp, update vp1=U+beta*P*vp where

(d) Repeat the whole thing by setting vp=vp1 until some criterion is met.

(4) Repeat (3) by setting v=v1 for many times.

(5) Find the final corresponding value of k as k' according to the maximum value.

as done yesterday.