Policy function calculation from value function iterations

We consider the same basic AK model with technology change:

and

where

and with depreciation

We can calculate which state we are in from the last point backward.

When we are in state 1, the previous state would be state 1 with probability p11 and

state 2 with probability (1-p11). When we are in state 2, the previous state would be

state 1 with probability p21 and state 2 with probability (1-p21). From a certain state,

we simulate it backward from the last to the first.