Does AHP help us make a choice? - An experimental evaluation

Ishizaka, Alessio and Balkenborg, Dieter and Kaplan, Todd R (2010): Does AHP help us make a choice? - An experimental evaluation.

Abstract

In this paper, we use experimental economics methods to test how well Analytic Hierarchy Process (AHP) fares as a choice support system in a real decision problem. AHP provides a ranking that we statistically compare with three additional rankings given by the subjects in the experiment: one at the beginning, one after providing AHP with the necessary pair-wise comparisons and one after learning the ranking provided by AHP. While the rankings vary widely across subjects, we observe that for each individual all four rankings are similar. Hence, subjects are consistent and AHP is, for the most part, able to replicate their rankings. Furthermore, while the rankings are similar, we do find that the AHP ranking helps the decision-makers reformulate their choices by taking into account suggestions made by AHP.

• Item Type: MPRA Paper
• Language: English
• Keywords: Decision analysis, Multiple Criteria Decision Aid, Analytic Hierarchy Process (AHP)
• Subjects: C - Mathematical and Quantitative Methods > C9 - Design of Experiments; C - Mathematical and Quantitative Methods > C4 - Econometric and Statistical Methods: Special Topics > C44 - Operations Research ; Statistical Decision Theory

References

Ahn B and Choi S (2007). ERP system selection using a simulation-based AHP approach: a case of Korean homeshopping company. J Oper Res Soc 59(3):322-330.

Akarte M, Surendra N, Ravi B, and Rangaraj N (2001). Web based casting supplier evaluation using analytical hierarchy process. J Oper Res Soc 52(5):511-522.

Al-Shemmeri T, Al-Kloub B and Pearman A (1997) Model choice in multicriteria decision aid, Eur J Opl Res 97(3):550-560.

Bana e Costa C, and Vansnick J (2008). A Critical Analysis of the Eigenvalue Method Used to Derive Priorities in AHP. Eur J Opl Res 187(3):1422-1428.

Banuelas R, and Antony J (2006). Application of stochastic analytic hierarchy process within a domestic appliance manufacturer. J Oper Res Soc 58(1):29-38.

Barzilai J (2001). Notes on the analytic hierarchy process. Proceedings of the NSF Design and Manufacturing Research Conference. Tampa. p 1-6.

Becker GM, De Groot MH, and Marschak J (1964). Measuring utility by a single-response sequential method. Behav Sci 9(3):226-232.

Beil R (1996). Laboratory Experimentation in Economic Research: An Introduction to Psychologists and Marketers. Psychol & Mark 13(4):331-340.

Belton V, and Gear A (1983). On a Shortcoming of Saaty's Method of Analytical Hierarchies. OMEGA 11(3):228-230.

Brugha C (2000). Relative measurement and the power function. Eur J Opl Res 121(3):627-640.

Brugha C (2004). Phased multicriteria preference finding. Eur J Opl Res 158(2):308-316.

Callaghan C, Gabriel A, and Sainty B (2006). Review and Classification of Experimental Economics Research in Accounting. J of Acc Lit 25:59-126.

Croson R, and Donohue K (2002). Experimental Economics and Supply Chain Management. Interfaces 32(5):74-82.

Dieckmann A, Dippold K, and Holger D (2009). Compensatory versus noncompensatory models for predicting consumer preferences. Judgm Decis Mak 4(3):200-213.

Dodd F, and Donegan H (1995). Comparison of priotization techniques using interhierarchy mappings. J Oper Res Soc 46(4):492-498.

Donegan H, Dodd F, and McMaster T (1992). A new approach to AHP decision-making. The Statician 41(3):295-302.

Dyer J (1990). Remarks on the Analytic Hierarchy Process. Mngt Sci 36(3):249-258.

Forman E, and Gass S (2001). The Analytic Hierarchy Process – An Exposition. Opns Res 49(4):469- 486.

Fukuyama H, and Weber W (2002). Evaluating public school district performance via DEA gain functions. J Oper Res Soc 53(9):992-1003.

Gigerenzer G, and Goldstein D (1996). Reasoning the fast and frugal way: models of bounded rationality. Psychol Rev 103(4):650-669.

Golden B, Wasil E, and Harker P (1989). The Analytic Hierarchy Process: Applications and Studies. Springer-Verlag: Heidelberg.

Guitoni A, Martel J-M (1998). Tentative guidelines to help choosing an appropriate MCDA method. Eur J Opl Res 109(2), 501-521.

Ho W (2008). Integrated analytic hierarchy process and its applications - A literature review. Eur J Opl Res 186(1), 211-228.

Hobbs B, and Meier P (1994). Multicriteria methods for resource planning: an experimental comparison. IEEE Trans. Pow. Sys, 9(4): 1811-1817.

Holder R (1991). Response to Holder's Comments on the Analytic Hierarchy Process: Response to the Response. J Oper Res Soc 42(10):914-918.

Huizingh E, and Vrolijk H (1997). Extending the Applicability of the Analytic Hierarchy Process. Socio-Econ Plann Sci 31(1):29-39.

Ishizaka A, and Labib A (2009). Analytic Hierarchy Process and Expert Choice: benefits and limitations. OR Insight 22(4):201–220, 2009. 22

Ishizaka A, Balkenborg D, and Kaplan T (2010). Influence of aggregation and measurement scale on ranking a compromise alternative in AHP, J Oper Res Soc, doi:10.1057/jors.2010.23(advance online publication).

Johnson C, Beine W, and Wang T (1979). Right-Left Asymmetry in an Eigenvector Ranking Procedure. J Math Psychol 19(1):61-64.

Karapetrovic S, Rosenbloom ES (1999). A quality control approach to consistency paradoxes in AHP. Eur J Opl Res 119(3): 704-18.

Karlan D (2005). Using Experimental Economics to Measure Social Capital and Predict Financial Decisions. Am Econ Rev 95(5):1688-1699.

Keeney R, Von Winterfeldt D, and Eppel T (1990). Eliciting public values for complex policy decisions. Mngt Sci 36(9):1011-1030.

Korhonen P, and Topdagi H (2003). Performance of the AHP in comparison of gains and losses. Mathl Comput Modelling 37(7-8).

Kornyshova E, and Salinesi C (2007). Selecting MCDM Techniques: State of the Art, Proceedings of the International IEEE Symposium on Computational Intelligence in Multicriteria Deicsion Making, Honolulu, 22-29.

Kumar S, and Vaidya O (2006). Analytic hierarchy process: An overview of applications. Eur J Opl Res 169(1):1-29.

Kwiesielewicz M, and van Uden E (2004). Inconsistent and Contradictory Judgements in Pairwise Comparison Method in AHP. Comput Opns Res 31, 713-719.

Lee C, and Kwak N (1999). Information resource planning for a health-care system using an AHPbased goal programming method. J Oper Res Soc 50(12):1191-1198.

Leung L, Lam K, and Cao D (2005). Implementing the balanced scorecard using the analytic hierarchy process & the analytic network process. J Oper Res Soc 57(6):682-691.

Li X, Beullens P, Jones D, and Tamiz M (2008). An integrated queuing and multi-objective bed allocation model with application to a hospital in China. J Oper Res Soc 60(3):330-338.

Liberatore M, and Nydick R (2008). The analytic hierarchy process in medical and health care decision making: A literature review. Eur J Opl Res 189(1):194-207.

Linares P (2009). Are inconsistent decisions better? An experiment with pairwise comparisons. Eur J Opl Res 193(2):492-498.

Millet I (1997). The effectiveness of alternative preference elicitation methods in the Analytic Hierarchy Process. J Multi-criteria Decis Anal 6(1), 41-51

Mingers J, Liu W, and Meng W (2007). Using SSM to structure the identification of inputs and outputs in DEA. J Oper Res Soc 60(2):168-179.

Omkarprasad V, and Sushil K (2006). Analytic hierarchy process: an overview of applications. Eur J Opl Res 169(1):1-29.

Payne J, Bettman J, and Johnson E (1988). Adaptive strategy selection in decision making. J Exp Psychol Learn Mem Cogn 14(3):534-552.

Pöyhönen M, Hamalainen R, and Salo A (1997). An Experiment on the Numerical Modelling of Verbal Ratio Statements. J Multi-criteria Decis Anal 6(1):1-10.

Saaty T (1977). A scaling method for priorities in hierarchical structures. J Math Psychol 15(3):234- 281.

Saaty T (1980). The Analytic Hierarchy Process. McGraw-Hill: New York.

Saaty T (2005). Making and validating complex decisions with the AHP/ANP. J Syst Sci and Syst Eng 14(1):1-36.

Saaty T (2006a). Rank from Comparisons and from Ratings in the Analytic Hierarchy/Network Processes. Eur J Opl Res 168(2):557-570.

Saaty T (2006b). There is no mathematical validity for using fuzzy number crunching in the analytic hierarchy process. J Syst Sci and Syst Eng 15(4): 457-464.

Saaty T, and Forman E (1992). The Hierarchon: A Dictionary of Hierarchies. RWS Publications: Pittsburgh.

Salo A, and Hamalainen R (1997). On the Measurement of Preference in the Analytic Hierarchy Process. J Multi-criteria Decis Anal 6(6):309-319.

Sha D, and Che Z (2005). Supply chain network design: partner selection and production/distribution planning using a systematic model. J Oper Res Soc 57(1):52-62.

Shim J (1989). Bibliography research on the analytic hierarchy process (AHP). Socio Econ Plan Sci 23(3):161-167.

Simpson L (1996). Do Decision Makers Know What They Prefer?: MAVT and ELECTRE II. J Oper Res Soc 47(7): 919-929.

Sturm B, and Weimann J (2006). Experiments in Environmental Economics and Some Close Relatives. J Econ Surv 20(3):419-457.

Tavana M (2005). A priority assessment multi-criteria decision model for human spaceflight mission planning at NASA. J Oper Res Soc 57(10):1197-1215.

Vargas L (1990). An overview of the analytic hierarchy process and its applications. Eur J Opl Res 48(1):2-8.

Webber S, Apostolou B, and Hassell J (1996). The sensitivity of the analytic hierarchy process to alternative scale and cue presentations. Eur J Opl Res 96(2):351-362.

Wheeler S (2005). An analysis of combined arms teaming for the Australian defence force. J Oper Res Soc 57(11):1279-1288.

Whitaker R (2007). Validation examples of the Analytic Hierarchy Process and Analytic Network Process. Math Comput Model 46(7-8):840-859.

Yeo G, Song D, Dinwoodie J, and Roe M (2009). Weighting the competitiveness factors for container ports under conflicting interests. J Oper Res Soc advance online publication:10.1057/jors.2009.1088.

Yüksel I, and Dagdeviren M (2007). Using the analytic network process (ANP) in a SWOT analysis - A case study for a textile firm. Inf Sci 177(16):3364-3382.

Zahedi F. (1986). The analytic hierarchy process: a survey of the method and its applications. Interface 16(4):96-108.