Learning in hidden Markov models with bounded memory

Monte, Daniel and Said, Maher (2010): Learning in hidden Markov models with bounded memory.

Abstract

This paper explores the role of memory in decision making in dynamic environments. We examine the inference problem faced by an agent with bounded memory who receives a sequence of signals from a hidden Markov model. We show that the optimal symmetric memory rule may be deterministic. This result contrasts sharply with Hellman and Cover (1970) and Wilson (2004) and solves, for the context of a hidden Markov model, an open question posed by Kalai and Solan (2003).

• Item Type: MPRA Paper
• Language: English
• Keywords: Bounded Memory; Hidden Markov Model; Randomization.
• Subjects: D - Microeconomics > D8 - Information, Knowledge, and Uncertainty > D82 - Asymmetric and Private Information ; Mechanism Design; D - Microeconomics > D8 - Information, Knowledge, and Uncertainty > D83 - Search ; Learning ; Information and Knowledge ; Communication ; Belief ; Unawareness; C - Mathematical and Quantitative Methods > C7 - Game Theory and Bargaining Theory > C72 - Noncooperative Games; C - Mathematical and Quantitative Methods > C7 - Game Theory and Bargaining Theory > C73 - Stochastic and Dynamic Games ; Evolutionary Games ; Repeated Games

References

Battaglini, M. (2005): "Long-Term Contracting with Markovian Consumers," American Economic Review, 95(3), 637-658.

Cole, H.L., and N.R. Kocherlakota (2005): "Finite Memory and Imperfect Monitoring," Games and Economic Behavior, 53(1), 59-72.

Compte, O., and A. Postlewaite (2009): “Effecting Cooperation,” Unpublished manuscript, University of Pennsylvania.

Cover, T., and M. Hellman (1971): “On Memory Saved by Randomization,” Annals of Mathematical Statistics, 42(3), 1075–1078.

Hellman, M., and T. Cover (1970): “Learning with Finite Memory,” Annals of Mathematical Statistics, 41(3), 765–782.

Kalai, E., and E. Solan (2003): “Randomization and Simplification in Dynamic Decision-Making,” Journal of Economic Theory, 111(2), 251–264.

Kocer, Y. (2009): “Endogenous Learning with Bounded Memory,” Unpublished manuscript, New York University.

Lipman, B. L. (1995): “Information Processing and Bounded Rationality: A Survey,” Canadian Journal of Economics, 28(1), 42–67.

Miller, D. A., and K. Rozen (2010): “Monitoring with Collective Memory: Forgiveness for Optimally Empty Promises,” Unpublished manuscript, Yale University.

Monte, D. (2009): “Learning with Bounded Memory in Games,” Unpublished manuscript, Simon Fraser University.

Mullainathan, S. (2002): “A Memory-Based Model of Bounded Rationality,” Quarterly Journal of Economics, 117(3), 735–774.

Romero, J. (2010): “Bounded Rationality in Repeated Games,” Unpublished manuscript, California Institute of Technology.

Rubinstein, A. (1998): Modeling Bounded Rationality. MIT Press, Cambridge.

Wilson, A. (2004): “Bounded Memory and Biases in Information Processing,” Unpublished manuscript, Harvard University.