Beyond curse of dimensionality with transition matrix

We do the same model as in 0919. That is,

where and

for j=1,2,. . .,m.

We consider the following m x m transition matrix

where row sums are all ones. Here,

at probability of pij.

That is, we consider

.

.

.

or

in vector notation.

We have already calculated the vector notation column by column after multiplying

P and V and regard the result as vectors of value functions.

We now apply the column-by-column method to the part of utility as well.

We implement column-by-column calculation:

from j=1 to n

for each dimension. Here, is one of the column vectors calculated by (P*v’)’

including transition matrix P. If we have dim dimensions, we do the same thing as

above for dim times and get dim numbers of column for v.