MEDICAL TREATMENTS USING PARTIALLY OBSERVABLE MARKOV DECISION PROCESSES

John E. Goulionis (Greece)

Received February 6, 2009

References:

[1] K. J. Aström, Optimal control of Markov processes with incomplete state information, J. Math. Anal. Appl. 10 (1965), 174-205. [2] D. P. Bertsekas, Dynamic Programming and Stochastic Control, Academic Press, New York, 1976. [3] D. P. Bertsekas, Stochastic Optimal Control. The Discrete Time Case, Academic Press, New York, 1998. [4] R. S. Braithwaite, A. C. Justice, C.-C. H. Chang, J. S. Fusco, S. R. Raffanti, J. B. Wong and M. S. Roberts, Estimating the proportion of patients infected with HIV who will die of comorbid diseases, Amer. J. Med. 118(8) (2005), 890-898. [5] R. M. D’Amato and R. T. D’Aquila, Management of antiretroviral therapy for HIV infection: modelling when to change therapy, J. Antiviral Therapy 3 (1998), 147-158. [6] E. V. Denardo, Contraction mappings in the theory underlying dynamic programming, SIAM Rev. 9 (1967), 165-177. [7] C. Derman, On optimal replacement rules when changes of state are Markovian, Symposium on Mathematical Optimization Techniques, Santa Monica, CA, 1960, pp. 201-210. [8] J. E. Goulionis, Structural properties for a two-state partially observable Markov decision process with an average cost criterion, J. Stat. Manag. Syst. 10(5) (2007), 715-733. [9] J. E. Goulionis and V. K. Benos, A maintenance policy under Markovian deterioration, Adv. Appl. Stat. 7(3) (2007), 357-388. [10] J. E. Goulionis and Dimitrios I. Stengos, The partially observable Markov decision processes framework in medical decision making, Adv. Appl. Stat. 9(2) (2008), 205-232. [11] Ohinish Masamitsu, Optimal minimal-repair and replacement problem of POMDP, Technical Report 91014, Kyoto University, 1991. [12] F. J. Palella, K. M. Delaney, A. C. Moorman, M. O. Loveless, J. Fuhrer, G. A. Satten, D. J. Aschman and S. D. Holmberg, Declining morbidity and mortality among patients with advanced human immunodeficiency virus infection, New England J. Med. 338(13) (1998), 853-860. [13] C. H. Papadimitriou and J. N. Tsitsiklis, The complexity of Markov decision processes, Math. Oper. Res. 12(3) (1987), 441-450. [14] E. J. Sondik, The optimal control of partially observable Markov processes, Ph.D. Dissertation, Stanford University, CA, 1971. [15] E. J. Sondik, The optimal control of partially observable Markov processes over the infinite horizon: discounted costs, Oper. Res. 26(2) (1978), 282-304. [16] N. L. Zhang and W. Liu, A model approximation scheme for planning in partially observable stochastic domains, J. Artificial Intelligence Res. 7 (1997), 199-230.

Keywords and phrases: Markov decision processes in medicine, decision analysis.