Policy function calculation from value function iterations

We consider the following general model with m states:

.

.

.

where

.

.

.

with probability matrix:

whose row sums are all zeros.

We also consider the following general switching model:

We consider the following general model with m states:

.

.

.

for the same c’s as above.