Bounded parameter Markov Decision Processes with average reward criterion

Tewari, Ambuj & Bartlett, Peter L. (2007) Bounded parameter Markov Decision Processes with average reward criterion. Learning Theory, 4539, pp. 263-277.

View at publisher


Bounded parameter Markov Decision Processes (BMDPs) address the issue of dealing with uncertainty in the parameters of a Markov Decision Process (MDP). Unlike the case of an MDP, the notion of an optimal policy for a BMDP is not entirely straightforward. We consider two notions of optimality based on optimistic and pessimistic criteria. These have been analyzed for discounted BMDPs. Here we provide results for average reward BMDPs.

We establish a fundamental relationship between the discounted and the average reward problems, prove the existence of Blackwell optimal policies and, for both notions of optimality, derive algorithms that converge to the optimal value function.

Impact and interest:

4 citations in Scopus
Search Google Scholar™
5 citations in Web of Science®

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

ID Code: 43969
Item Type: Journal Article
Refereed: Yes
Additional Information: 20th Annual Conference on Learning Theory, COLT 2007, San Diego, CA, USA; June 13-15, 2007. Proceedings
Keywords: Markov Decision Processes, BMDPs, optimal policy
DOI: 10.1007/978-3-540-72927-3_20
ISBN: 9783540729259
ISSN: 0302-9743
Subjects: Australian and New Zealand Standard Research Classification > INFORMATION AND COMPUTING SCIENCES (080000) > INFORMATION SYSTEMS (080600)
Divisions: Past > QUT Faculties & Divisions > Faculty of Science and Technology
Past > Schools > Mathematical Sciences
Copyright Owner: Copyright 2007 Springer
Deposited On: 18 Aug 2011 02:03
Last Modified: 29 Feb 2012 14:34

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page