Value function iterations with transition matrix

We consider the following case with transition matrix

where

together with

where both A and Q change according to transition matrix as in 0921.

We continue to implement error bounds to speed up iterations. We adjust the value by

where

that is done across dimensions rather than dimension by dimension.