Information Theory and Knowledge-Gathering

Murphy, Roy E (2006): Information Theory and Knowledge-Gathering.

Abstract

It is assumed that human knowledge-building depends on a discrete sequential decision-making process subjected to a stochastic information transmitting environment. This environment randomly transmits Shannon type information-packets to the decision-maker, who examines each of them for relevancy and then determines his optimal choices. Using this set of relevant information-packets, the decision-maker adapts, over time, to the stochastic nature of his environment, and optimizes the subjective expected rate-of-growth of knowledge. The decision-maker’s optimal actions, lead to a decision function that involves his view of the subjective entropy of the environmental process and other important parameters at each stage of the process. Using this model of human behavior, one could create psychometric experiments using computer simulation and real decision-makers, to play programmed games to measure the resulting human performance.

• Item Type: MPRA Paper
• Language: English
• Keywords: decision-making; dynamic programming; entropy; epistemology; information theory; knowledge; sequential processes; subjective probability
• Subjects: C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C61 - Optimization Techniques ; Programming Models ; Dynamic Analysis

References

1. Monograph: Arrow, K. (1961) “The Economic Implications if Learning by Doing,” Tech. Report No. 101, “Institute for Mathematical Studies in the Social Sciences,” Stanford Univ. Press 2. Journal: Arrow, K.,(1996) “The Economics of Information: An Exposition,” Empirica, Kluwer Academic Publishers, Netherlands, pp 119-128 3. Journal: Bellman, R., and Kalaba, R., (1960) IRE Trans. on Automatic Control, Vol. AC-5, pp 5-10 4. Book: Birkhoff, G. D., (1927), “Dynamical Systems,” AMS Colloquium Publications, N.Y., N.Y. vol. 9 5. Book: Brillouin, L., (1963) “Science and Information Theory,” Academic Press, New York, pp 1-23 6. Book: Carnap, R. (1950), “Logical Foundations of Probability,” Univ. of Chicago Press, Chicago 7. Book: Doob, J., (1952) “Stochastic Processes,” Chapter V. “Markov Processes – Discrete Parameter,” John Wiley & Sons, New York, pp 170-234 8. Book: Dretske, Fred, (1999) “Knowledge and the flow of Information,” CSLI Publications, Center for the Study of Language and Information, Stanford, Stanford CA. 9. Monograph: von Foerster, H., (1967) “Memory Without Record,” Artorga, Hampshire, England, pp 404-406 10. Journal: Foster, D.J and Wilson, M. A., (2006) “Reverse replay of behavioural sequences in hippocampal place cells during the awake state,” Nature 440, pp 680-683 11. Book: Good, I., “The Estimation of Probabilities – An Essay on Modern Bayesian Methods,” MIT Research Monograph No. 30, Cambridge, MA., pp 6-20 12. Book: Hadamard, J., (1954) “The Psychology of Invention in the Mathematical Field,” Dover Publications, N.Y., N.Y., pp 21-63 13. Book: Khinchin, A. (1957) “Mathematical Foundations of Information Theory,” (Translation of “The Entropy Concept in Probability Theory,” in Uspekhi Matematicheskikh Nauk, Vol. VIII, No. 3) Dover Publications, N.Y., N.Y. pp 1-28 14. Symposium: Kuhn, H.W. and Tucker, A.W., (1951) “Nonlinear Programming,” Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability. University of California Press, Berkeley, pp 481-492 15. Monograph: Marschak, J., (1960) “Remarks on the Economics of Information,” Cowles Foundation Paper No. 146, Cowles Foundation for Economic Reasearch in Economics, Yale, New Haven, 92-97. 16. Book: Murphy, R., (1965) “Adaptive Processes in Economic Systems,” Academic Press, N.Y., N.Y., pp 130-139 17. Book: Poincaré, H., (1952) “Science and Method,” Dover Publications, N.Y., N.Y., pp 46-63 18. Book: Prigogine, I., (1996) “The End of Certainty,” Free Press, N.Y., N.Y., pp 66-70 19. Book: Reichenbach, H. (1999) “The Direction of Time,” Dover Publications, N.Y., N.Y. 20. Journal: Shannon, C. (1948) “A Mathematical Theory of Communication, Bell System Tech Journal, pp 379-0423, pp 623-656 21. Book: Somenzi, V., (1965) “Entropy, Information, and the Mind-Body Problem,” in “Information and Prediction in Science,” Dockx, S. and Bernays, P. (Eds), Academic Press, New York, N.Y. pp 229-235 22. Book: Wagner, H., (1969) “Principles of Operations Research,” Prentice-Hall, New Jersey, pp 466-471 23. Journal: Walker, M., (2006) “Sleep to Remember,” American Scientist, Vol. 94, No. 4, pp 325-333