Mechanism Design for eliciting probabilistic estimates from multiple suppliers with unknown costs and limited precision

Papakonstantinou, A. and Rogers, A. and Gerding, E. H and Jennings, N. R. (2010): Mechanism Design for eliciting probabilistic estimates from multiple suppliers with unknown costs and limited precision. Published in: LNBIP: Agent-Mediated Electronic Commerce. Designing Trading Strategies and Mechanisms for Electronic Markets. E. David et al. (Eds.): AMEC/TADA 2009 No. 59 (2010): pp. 102-116.

Abstract

This paper reports on the design of a novel two-stage mechanism, based on strictly proper scoring rules, that allows a centre to acquire a costly probabilistic estimate of some unknown parameter, by eliciting and fusing estimates from multiple suppliers. Each of these suppliers is capable of producing a probabilistic estimate of any precision, up to a privately known maximum, and by fusing several low precision estimates together the centre is able to obtain a single estimate with a specified minimum precision. Specifically, in the mechanism’s first stage M from N agents are pre-selected by eliciting their privately known costs. In the second stage, these M agents are sequentially approached in a random order and their private maximum precision is elicited. A payment rule, based on a strictly proper scoring rule, then incentivises them to make and truthfully report an estimate of this maximum precision, which the centre fuses with others until it achieves its specified precision. We formally prove that the mechanism is incentive compatible regarding the costs, maximum precisions and estimates, and that it is individually rational. We present empirical results showing that our mechanism describes a family of possible ways to perform the pre-selection in the first stage, and formally prove that there is one that dominates all others.

• Item Type: MPRA Paper
• Language: English
• Keywords: Mechanism Design; Auction Theory; Multi-agent Systems; Scoring Rules
• Subjects: D - Microeconomics > D8 - Information, Knowledge, and Uncertainty > D81 - Criteria for Decision-Making under Risk and Uncertainty; D - Microeconomics > D4 - Market Structure, Pricing, and Design > D44 - Auctions; D - Microeconomics > D8 - Information, Knowledge, and Uncertainty > D82 - Asymmetric and Private Information ; Mechanism Design; D - Microeconomics > D8 - Information, Knowledge, and Uncertainty > D83 - Search ; Learning ; Information and Knowledge ; Communication ; Belief ; Unawareness

References

1. Hendrickson, A.D., Buehler, R.J.: Proper scores for probability forecasters. The Annals of Mathematical Statistics 42(6) (1971) pp. 1916–1921.

2. Savage, L.J.: Elicitation of personal probabilities and expectations. Journal of the American Statistical Association 66(336) (1977) pp. 783–801.

3. Selten, R.: Axiomatic characterization of the quadratic scoring rule. Experimental Economics 1(1) (1998) pp. 43–61.

4. Miller, N., Resnick, P., Zeckhauser, R.: Eliciting honest feedback: The peer prediction method. Management Science 51(9) (2005) pp. 1359–1373.

5. Zohar, A., Rosenschein, J.S.: Mechanisms for information elicitation. Artificial Intelligence 172(16-17) (2008) pp. 1917–1939.

6. Jurca, R., Faltings, B.: Reputation-based service level agreements for web services. In: In Service Oriented Computing, Amsterdam (2005) pp. 396–409.

7. Papakonstantinou, A., Rogers, A., Gerding, E.H., Jennings, N.R.: A truthful two-stage mechanism for eliciting probabilistic estimates with unknown costs. In: Proceedings of the 18th European Conference on Artificial Intelligence, Patras, Greece (2008) pp. 448–452.

8. DeGroot, M.H., Schervish, M.J.: Probability and Statistics. Addison Wesley (2002).

9. Jehiel, P., Moldovanu, B.: Efficient design with interdependent valuations. Econometrica, 69(5) (2001) pp. 1237-1259.

10. Arnold, B.C., Balakrishnan, N., Nagaraja, H.N.: A First Course in Order Statistics. SIAM (2008).