Abstract
In this research healthcare software for personal health management is studied. Healthcare software gives appropriate advice based on personal health state. In previous research probability of staying in objective states is maximized under the condition that the true parameter of health state transition probability is known. But it is not always possible to know the true parameter. In this research a new method which maximizes the probability of staying in objective states under the condition that the true parameter is unknown is proposed. Markov decision processes with unknown state transition probability is used in modelling. In the proposed method dynamic programming is used in order to maximize the probability of staying in objective states. The effectiveness of the proposed method is shown by some computational examples. In the results probability of staying in objective states by the proposed method is greater than that of the comparison target.
