{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T05:59:37Z","timestamp":1773467977983,"version":"3.50.1"},"reference-count":35,"publisher":"Institute for Operations Research and the Management Sciences (INFORMS)","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Operations Research"],"published-print":{"date-parts":[[2022,5]]},"abstract":"<jats:p> Title: Sequential Decision Making Using Quantiles <\/jats:p><jats:p> The goal of a traditional Markov decision process (MDP) is to maximize the expectation of cumulative reward over a finite or infinite horizon. In many applications, however, a decision maker may be interested in optimizing a specific quantile of the cumulative reward. For example, a physician may want to determine the optimal drug regime for a risk-averse patient with the objective of maximizing the 0.10 quantile of the cumulative reward; this is the cumulative improvement in health that is expected to occur with at least 90% probability for the patient. In \u201cQuantile Markov Decision Processes,\u201d X. Li, H. Zhong, and M. Brandeau provide analytic results to solve the quantile Markov decision process (QMDP) problem. They develop an efficient dynamic programming procedure that finds the optimal QMDP value function for all states and quantiles in one pass. The algorithm also extends to the MDP problem with a conditional value-at-risk objective. <\/jats:p>","DOI":"10.1287\/opre.2021.2123","type":"journal-article","created":{"date-parts":[[2021,11,9]],"date-time":"2021-11-09T19:06:10Z","timestamp":1636484770000},"page":"1428-1447","source":"Crossref","is-referenced-by-count":7,"title":["Quantile Markov Decision Processes"],"prefix":"10.1287","volume":"70","author":[{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0001-6155-9068","authenticated-orcid":false,"given":"Xiaocheng","family":"Li","sequence":"first","affiliation":[{"name":"Department of Management Science and Engineering, Stanford University, Stanford, California 94305"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0002-2902-1644","authenticated-orcid":false,"given":"Huaiyang","family":"Zhong","sequence":"additional","affiliation":[{"name":"Department of Management Science and Engineering, Stanford University, Stanford, California 94305"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0001-9331-8920","authenticated-orcid":false,"given":"Margaret L.","family":"Brandeau","sequence":"additional","affiliation":[{"name":"Department of Management Science and Engineering, Stanford University, Stanford, California 94305"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"109","reference":[{"key":"B1","volume-title":"Constrained Markov Decision Processes","author":"Altman E","year":"1999"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1287\/opre.2014.1281"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1002\/sim.1851"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1007\/s00186-011-0367-0"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1111\/1540-6261.00455"},{"key":"B7","volume-title":"Dynamic Programming and Optimal Control","volume":"1","author":"Bertsekas DP","year":"1995"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1093\/aje\/kwu178"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1016\/j.frl.2008.10.002"},{"key":"B12","first-page":"3509","volume":"2","author":"Chow Y","year":"2014","journal-title":"Adv. Neural Inform. Processing Systems (Montreal, Canada),"},{"key":"B13","first-page":"1522","volume":"1","author":"Chow Y","year":"2015","journal-title":"Adv. Neural Inform. Processing Systems (Montreal, Canada),"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1145\/1323293.1294281"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1287\/opre.1080.0685"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.3905\/jod.1997.407971"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1109\/9.362904"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1191\/0961203303lu371oa"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1161\/JAHA.116.003411"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.18.7.356"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1287\/moor.2014.0689"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1287\/moor.2017.0872"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejor.2013.09.018"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1093\/cid\/cis731"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1137\/050622328"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.1287\/opre.1050.0216"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1287\/moor.2015.0747"},{"issue":"3","key":"B33","first-page":"645","volume":"35","author":"Piunovskiy AB","year":"2006","journal-title":"Control Cybernetics"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1016\/S0378-4266(02)00271-6"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-010-0393-3"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1017\/S0269964800001121"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejor.2012.08.022"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1287\/opre.1070.0480"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.1126\/science.1228160"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1109\/9.793723"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1287\/moor.1120.0566"},{"key":"B45","first-page":"6190","author":"Yang D","year":"2019","journal-title":"Adv. Neural Inform. Processing Systems"},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.1080\/09540121.2020.1719966"}],"container-title":["Operations Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/pubsonline.informs.org\/doi\/pdf\/10.1287\/opre.2021.2123","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,4,2]],"date-time":"2023-04-02T21:05:35Z","timestamp":1680469535000},"score":1,"resource":{"primary":{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/pubsonline.informs.org\/doi\/10.1287\/opre.2021.2123"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5]]},"references-count":35,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,5]]}},"alternative-id":["10.1287\/opre.2021.2123"],"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.1287\/opre.2021.2123","relation":{},"ISSN":["0030-364X","1526-5463"],"issn-type":[{"value":"0030-364X","type":"print"},{"value":"1526-5463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5]]}}}