Hidden Markov Model for Human Decision Process in a Partially Observable Environment

Abstract

The environment surrounding us is inevitably uncertain; we cannot perceive all the information necessary for making optimal decision. Even in such a partially observable environment, humans can make appropriate decision by resolving the uncertainty. During decision making in an uncertain environment, resolving behaviors of the uncertainty and optimal behaviors to best suit for the environment are often incompatible, which is termed exploration-exploitation dilemma in the field of machine learning. To examine how we cope with the exploration-exploitation dilemma, in this study, we performed statistical modeling of human behaviors when performing a partially observable maze navigation task; in particular, we devised a hidden Markov model (HMM), which incorporates inference of a hidden variable in the environment and switching between exploration and exploitation. Our HMM-based model well reproduced the human behaviors, suggesting the human subjects actually performed exploration and exploitation to effectively adapt to this uncertain environment.