Wulf, David and Bertsch, Valentin (2016): A natural language generation approach to support understanding and traceability of multidimensional preferential sensitivity analysis in multicriteria decision making.

PDF
MPRA_paper_75025.pdf Download (1MB)  Preview 
Abstract
MultiCriteria Decision Analysis (MCDA) enables decision makers (DM) and decision analysts (DA) to analyse and understand decision situations in a structured and formalised way. With the increasing complexity of decision support systems (DSSs), it becomes challenging for both expert and novice users to understand and interpret the model results. Natural language generation (NLG) techniques are used in various DSSs to cope with this challenge as they reduce the cognitive effort to achieve understanding of decision situations. However, NLG techniques in MCDA have so far mainly been developed for deterministic decision situations or onedimensional sensitivity analyses. In this paper, a concept for the generation of textual explanations for a multidimensional preferential sensitivity analysis in MCDA is developed. The key contribution is a NLG approach that provides detailed explanations of the implications of preferential uncertainties in MultiAttribute Value Theory (MAVT). It generates a report that assesses the influences of simultaneous or separate variations of intercriteria and intracriteria preferential parameters determined within the decision analysis. We explore the added value of the natural language report in an online survey. Our results show that the NLG approach is particularly beneficial for difficult interpretational tasks.
Item Type:  MPRA Paper 

Original Title:  A natural language generation approach to support understanding and traceability of multidimensional preferential sensitivity analysis in multicriteria decision making 
Language:  English 
Keywords:  Decision support systems; Multiple criteria analysis; Preferential uncertainty modelling; Natural language generation; Multidimensional preferential sensitivity analysis 
Subjects:  C  Mathematical and Quantitative Methods > C6  Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling Q  Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q4  Energy > Q48  Government Policy 
Item ID:  75025 
Depositing User:  Valentin Bertsch 
Date Deposited:  15 Nov 2016 15:10 
Last Modified:  15 Nov 2016 15:11 
References:  AbuTaha, R. (2011). Multicriteria applications in renewable energy analysis: A literature review. paper presented at PICMET ‘11 Conference, July 31  August 04, 2011, Portland. DOI: 10.1007/9781447150978_2. Amgoud, L. and Prade, H. (2006). Explaining qualitative decision under uncertainty by argumentation. In Cohn, A. (Ed.), Proceeding AAAI 2006 proceedings of the 21st national conference on Artificial intelligence, Boston, MA, July 1620, 2006, AAAI Press, Menlo Park, CA, pp. 219–224. Bailey, D., Goonetilleke, A. and Deriche, M. (2011). A decision support system for site selection of largescale infrastructure facilities using natural language. paper presented at PICMET ‘11 Conference, July 31  August 04, 2011, Portland, available at: http://eprints.qut.edu.au/4223/ (accessed 12 January 2016). Basson, L. and Petrie, J.G. (2007). An integrated approach for the consideration of uncertainty in decision making supported by Life Cycle Assessment. Environmental Modelling & Software, Vol. 22 No. 2, pp. 167–176. DOI: 10.1016/j.envsoft.2005.07.026. Bélanger, M. and Martel, J.M. (2005). An automated explanation approach for a decision support system based on MCDA. In RothBerghofer, T.R., Schulz, S. and Woody, A. (Eds.), Explanationaware computing: Papers from the AAAI Fall Symposium, Arlington, VA, November 0406, 2005, AAAI Press, Menlo Park, CA, pp. 21–34. Bell, D.E. (1982). Regret in decision making under uncertainty. Operations research, Vol. 30 No. 5, pp. 961–981. DOI: 10.1287/opre.30.5.961. Bell, M.L., Hobbs, B.F. and Ellis, H. (2003). The use of multicriteria decisionmaking methods in the integrated assessment of climate change: Implications for IA practitioners. SocioEconomic Planning Sciences, Vol. 37 No. 4, pp. 289–316. DOI: 10.1016/S00380121(02)000472. Belton, V. and Stewart, T.J. (2002), Multiple Criteria Decision Analysis  An Integrated Approach, Springer US, Boston. Bertsch, V., Treitz, M., Geldermann, J. and Rentz, O. (2007). Sensitivity analyses in multiattribute decision support for offsite nuclear emergency and recovery management. International Journal of Energy Sector Management, Vol. 1 No. 4, pp. 342365. Bertsch, V. (2008). Uncertainty Handling in MultiAttribute Decision Support for Industrial Risk Management. University of Karlsruhe, Karlsruhe, 2008. DOI: 10.5445/KSP/1000007378. Bertsch, V. and Fichtner, W. (2016). A participatory multicriteria approach for power generation and transmission planning. Annals of Operations Research, Vol. 245 No. 1, pp. 177–207. DOI: 10.1007/s104790151791y. Bishara, A.J. and Hittner, J.B. (2012). Testing the significance of a correlation with nonnormal data: Comparison of Pearson, Spearman, transformation, and resampling approaches. Psychological methods, Vol. 17 No. 3, pp. 399–417. DOI: 10.1037/a0028087. Bloom, B.S., Krathwohl, D.R. and Masia, B.B. (1984), Taxonomy of educational objectives: The classification of educational goals Book 1: Cognitive Domain, Longman, New York. Brans, J.P. and Mareschal, B. (1994). The PROMCALC & GAIA decision support system for multicriteria decision aid. Decision Support Systems, Vol. 12 No. 45, pp. 297–310. DOI: 10.1016/01679236(94)900485. Brehmer, B. (1980). In one word: Not from experience. Acta Psychologica, Vol. 45 No. 13, pp. 223–241. DOI: 10.1016/00016918(80)900347. Broekhuizen, H., GroothuisOudshoorn, C.G.M., van Til, J.A., Hummel, J.M. and Jzerman, M.J. (2015). A review and classification of approaches for dealing with uncertainty in multicriteria decision analysis for healthcare decisions. PharmacoEconomics, Vol. 33 No. 5, pp. 445–455. DOI: 10.1007/s402730140251x. Browne, D., O’Regan, B. and Moles, R. (2010). Use of multicriteria decision analysis to explore alternative domestic energy and electricity policy scenarios in an Irish cityregion. Energy, Vol. 35 No. 2, pp. 518–528. DOI: 10.1016/j.energy.2009.10.020. Buchanan, B.G. and Shortliffe, E.H. (1984), Rulebased expert systems: The MYCIN experiments of the Stanford Heuristic Programming Project, The AddisonWesley series in artificial intelligence, AddisonWesley, Reading, MA. Butler, J., Jia, J. and Dyer, J. (1997). Simulation techniques for the sensitivity analysis of multicriteria decision models. European Journal of Operational Research, Vol. 103 No. 3, pp. 531–546. DOI: 10.1016/S03772217(96)003074. Carenini, G. and Moore, J.D. (2006). Generating and evaluating evaluative arguments. Artificial Intelligence, Vol. 170 No. 11, pp. 925–952. DOI: 10.1016/j.artint.2006.05.003. Clark, A., Fox, C. and Lappin, S. (2010), The handbook of computational linguistics and natural language processing, Blackwell handbooks in linguistics, WileyBlackwell, Chichester, West Sussex, Malden, MA. DOI: 10.1002/9781444324044. Cohen, J. (1988), Statistical power analysis for the behavioral sciences, 2nd ed., Lawrence Erlbaum Associates Inc., Hillsdale, N. J. Corrente, S., French, S., Greco, S., Kadziński, M., Knowles, J.D., Mousseau, V., Siebert, J. and Słowiński, R. (2014). Drafting a Manifesto for DMDSS Interaction (Working Group ‘DM Sense’). In Greco, S., Knowles, J.D., Miettinen, K. and Zitzler, E. (Eds.), Learning in Multiobjective Optimization: Report from Dagstuhl Seminar 2014, Dagstuhl Report, pp. 72–81. Dhaliwal, J.S. and Benbasat, I. (1996). The Use and Effects of KnowledgeBased System Explanations: Theoretical Foundations and a Framework for Empirical Evaluation. Information Systems Research, Vol. 7 No. 3, pp. 342–362. DOI: 10.1287/isre.7.3.342. Diakoulaki, D., Antunes, C.H. and Martins, A.G. (2005). MCDA and energy planning. In Figueira, J.R., Greco, S. and Ehrgott, M. (Eds.), Multiple criteria decision analysis: State of the art surveys, International Series in Operations Research & Management Science, Springer, New York, pp. 859–890. DOI: 10.1007/0387230815_21. Durbach, I.N. and Stewart, T.J. (2009). Using expected values to simplify decision making under uncertainty. Omega, Vol. 37 No. 2, pp. 312–330. DOI: 10.1016/j.omega.2007.02.001. Durbach, I.N. and Stewart, T.J. (2012). Modeling uncertainty in multicriteria decision analysis. European Journal of Operational Research, Vol. 223 No. 1, pp. 1–14. DOI: 10.1016/j.ejor.2012.04.038. Eisenführ, F., Weber, M. and Langer, T. (2010), Rational Decision Making, Springer, Berlin, London. DOI: 10.1007/9783642028519. Ellis, P.D. (2010). Effect sizes and the interpretation of research results in international business. Journal of International Business Studies, Vol. 41 No. 9, pp. 1581–1588. DOI: 10.1057/jibs.2010.39. Ellis, P.D. (2013), The essential guide to effect sizes: Statistical power, metaanalysis, and the interpretation of research results, 6th ed., Cambridge University Press, Cambridge. DOI: 10.1017/CBO9780511761676. Fisher, R.A. (1922). On the Interpretation of X² from Contingency Tables, and the Calculation of P. Journal of the Royal Statistical Society, Vol. 85 No. 1, p. 87. DOI: 10.2307/2340521. French, S. (1995). Uncertainty and Imprecision: Modelling and Analysis. The Journal of the Operational Research Society, Vol. 46 No. 1. DOI: 10.2307/2583837. French, S., Maule, J. and Papamichail, K.N. (2009), Decision behaviour, analysis and support, Cambridge University Press, Cambridge, UK, New York. Gardiner, P.C. and Edwards, W. (1975). Public values: Multiattributeutility measurement for social decision making. In Kaplan, M.F. and Schwartz, S. (Eds.), Human Judgment and Decision Process, Academic Press Series in cognition and perception, Academic Press, New York, pp. 1–37. DOI: 10.1109/TSMC.1977.4309720. Geldermann, J. (2010). Explanation Systems. In Ríos Insua, D. and French, S. (Eds.), EDemocracy: A group decision and negotiation practice, Advances in Group Decision and Negotiation, Vol. 5, Springer, Dordrecht, London, pp. 241–259. Geldermann, J., Bertsch, V., Treitz, M., French, S., Papamichail, K.N. and Hämäläinen, R.P. (2009). Multicriteria decision support and evaluation of strategies for nuclear remediation management. Omega, Vol. 37 No. 1, pp. 238–251. DOI: 10.1016/j.omega.2006.11.006. Gilovich, T., Griffin, D.W. and Kahneman, D. (2002), Heuristics and biases: The psychology of intuitive judgement, Cambridge University Press, Cambridge, UK, New York. Gosset, W.S. (1908). The probable error of a mean. Biometrika, Vol. 6 No. 1, pp. 1–25. DOI: 10.1093/biomet/6.1.1. Graves, S.B. and Ringuest, J.L. (2009). Probabilistic dominance criteria for comparing uncertain alternatives: A tutorial. Omega, Vol. 37 No. 2, pp. 346–357. DOI: 10.1016/j.omega.2007.03.001. Greco, S., Słowiński, R. and Zielniewicz, P. (2013). Putting Dominancebased Rough Set Approach and robust ordinal regression together. Decision Support Systems, Vol. 54 No. 2, pp. 891–903. DOI: 10.1016/j.dss.2012.09.013. Greef, H.P.d. and Neerincx, M.A. (1995). Cognitive support: Designing aiding to supplement human knowledge. International Journal of HumanComputer Studies, Vol. 42 No. 5, pp. 531–571. DOI: 10.1006/ijhc.1995.1023. Greening, L.A. and Bernow, S. (2004). Design of coordinated energy and environmental policies: Use of multicriteria decisionmaking. Energy Policy, Vol. 32 No. 6, pp. 721–735. DOI: 10.1016/j.enpol.2003.08.017. Greer, J.E., Falk, S., Greer, K.J. and Bentham, M.J. (1994). Explaining and justifying recommendations in an agriculture decision support system. Computers and Electronics in Agriculture, Vol. 11 No. 23, pp. 195–214. DOI: 10.1016/01681699(94)900086. Gregor, S. and Benbasat, I. (1999). Explanations from Intelligent Systems: Theoretical Foundations and Implications for Practice. MIS Quarterly, Vol. 23 No. 4, pp. 497–530. DOI: 10.2307/249487. Hämäläinen, R.P. and Alaja, S. (2008). The threat of weighting biases in environmental decision analysis. Ecological Economics, Vol. 68 No. 12, pp. 556–569. DOI: 10.1016/j.ecolecon.2008.05.025. Hammond, K.R., Stewart, T.R., Brehmer, B. and Steinmann, D.O. (1975). Social Judgment Theory. In Kaplan, M.F. and Schwartz, S. (Eds.), Human Judgment and Decision Process, Academic Press Series in cognition and perception, Academic Press, New York. Hauke, J. and Kossowski, T. (2011). Comparison of Values of Pearson’s and Spearman’s Correlation Coefficients on the Same Sets of Data. Quaestiones Geographicae, Vol. 30 No. 2, pp. 87–93. DOI: 10.2478/v1011701100211. Henrion, M. and Druzdzel, M.J. (1991). Qualitative Propagation and Scenariobased Explanation of Probabilistic Reasoning. In Bonissone, P.P., Henrion, M., Kanal, L.N. and Lenner, J.F. (Eds.), Uncertainty and artificial intelligence, 6th ed., pp. 17–32. Heo, E., Kim, J. and Boo, K.J. (2010). Analysis of the assessment factors for renewable energy dissemination program evaluation using fuzzy AHP. Renewable and Sustainable Energy Reviews, Vol. 14 No. 8, pp. 2214–2220. DOI: 10.1016/j.rser.2010.01.020. Hodgkin, J., Belton, V. and Koulouri, A. (2005). Supporting the intelligent MCDA user: A case study in multiperson multicriteria decision support. European Journal of Operational Research, Vol. 160 No. 1, pp. 172–189. DOI: 10.1016/j.ejor.2004.03.007. Hoffman, P.J., Earle, T.C. and Slovic, P. (1981). Multidimensional functional learning (MFL) and some new conceptions of feedback. Organizational Behavior and Human Performance, Vol. 27 No. 1, pp. 75–102. DOI: 10.1016/00305073(81)900404. Hogarth, R.M. (1987), Judgement and choice: The psychology of decision, 2nd ed., Wiley, Chichester, New York. Holtzman, S. (1988), Intelligent decision systems, AddisonWesley Longman Publishing Co., Inc., Boston, MA. Jessop, A. (2011). Using imprecise estimates for weights. Journal of the Operational Research Society, Vol. 62 No. 6, pp. 1048–1055. Jessop, A. (2014). IMP: A decision aid for multiattribute evaluation using imprecise weight estimates. Omega, Vol. 49, pp. 18–29. DOI: 10.1016/j.omega.2014.05.001. Jiménez, A., Mateos, A. and RíosInsua, S. (2005). Monte Carlo Simulation Techniques in a Decision Support System for Group Decision Making. Group Decision and Negotiation, Vol. 14 No. 2, pp. 109–130. DOI: 10.1007/s1072600524069. Kadziński, M., Corrente, S., Greco, S. and Słowiński, R. (2014). Preferential reducts and constructs in robust multiple criteria ranking and sorting. OR Spectrum, Vol. 36 No. 4, pp. 1021–1053. DOI: 10.1007/s002910140361z. Kahneman, D. and Knetsch, J.L. (1992). Valuing public goods: The purchase of moral satisfaction. Journal of Environmental Economics and Management, Vol. 22 No. 1, pp. 57–70. DOI: 10.1016/00950696(92)90019S. Kahneman, D., Slovic, P. and Tversky, A. (1982), Judgment under uncertainty: Heuristics and biases, Cambridge University Press, Cambridge, UK, New York. Kass, R. and Finin, T. (1988). The Need for User Models in Generating Expert System Explanations. International Journal of Expert Systems  Special Issue: Natural Language and Expert Systems, Vol. 1 No. 4, pp. 345–375. Kaya, T. and Kahraman, C. (2011). Multicriteria decision making in energy planning using a modified fuzzy TOPSIS methodology. Expert Systems with Applications, Vol. 38 No. 6, pp. 6577–6585. DOI: 10.1016/j.eswa.2010.11.081. Keeney, R.L. and Raiffa, H. (1976), Decisions with multiple objectives: Preferences and value tradeoffs, Wiley, New York. Kiker, G.A., Bridges, T.S., Varghese, A., Seager, T.P. and Linkov, I. (2005). Application of Multicriteria Decision Analysis in Environmental Decision Making. Integrated Environmental Assessment and Management, Vol. 1 No. 2, pp. 95–108. DOI: 10.1897/IEAM_2004a015.1. Kirkwood, C.W. (1997), Strategic decision making: Multiobjective decision analysis with spreadsheets, Duxbury Press, Belmont. Kowalski, K., Stagl, S., Madlener, R. and Omann, I. (2009). Sustainable energy futures: Methodological challenges in combining scenarios and participatory multicriteria analysis. European Journal of Operational Research, Vol. 197 No. 3, pp. 1063–1074. DOI: 10.1016/j.ejor.2007.12.049. Labreuche, C., Maudet, N., Mousseau, V. and Ouerdane, W. (2012). Explanation of the robust additive preference model by even swap sequences. paper presented at 6th Multidisciplinary Workshop on Advances in Preference Handling, Montpellier. Labreuche, C., Maudet, N. and Ouerdane, W. (2011). Minimal and Complete Explanations for Critical Multiattribute Decisions. In Brafman, R.I., Roberts, F.S. and Tsoukiàs, A. (Eds.), Algorithmic decision theory, Piscataway, NJ, October 2628, 2011, Springer, Berlin, Heidelberg, New York, pp. 121–134. Lahdelma, R., Hokkanen, J. and Salminen, P. (1998). SMAA  Stochastic multiobjective acceptability analysis. European Journal of Operational Research, Vol. 106 No. 1, pp. 137–143. DOI: 10.1016/S03772217(97)00163X. Lahdelma, R. and Salminen, P. (2001). SMAA2: Stochastic Multicriteria Acceptability Analysis for Group Decision Making. Operations Research, Vol. 49 No. 3, pp. 444–454. Levy, H. (1992). Stochastic Dominance and Expected Utility: Survey and Analysis. Management Science, Vol. 38 No. 4, pp. 555–593. Linkov, I., Varghese, A. and Jamil, S. (2004). Multicriteria decision analysis: A framework for structuring remedial decisions at contaminated sites. In Linkov, I. and Ramadan, A.B. (Eds.), Comparative risk assessment and environmental decision making, Kluwer Academic Publishers, Boston, MA, pp. 15–54. DOI: 10.1007/1402022433_2. Loken, E. (2007). Use of multicriteria decision analysis methods for energy planning problems. Renewable and Sustainable Energy Reviews, Vol. 11 No. 7, pp. 1584–1595. DOI: 10.1016/j.rser.2005.11.005. Loomes, G. and Sugden, R. (1982). Regret Theory: An Alternative Theory of Rational Choice under Uncertainty. Economic Journal, Vol. 92 No. 4, pp. 805–824. DOI: 10.2307/2232669. Lühn, T., Schlömer, G., Schmidtmann, G., Lehde, B., Schmiesing, J., Hofmann, L. and Geldermann, J. (2014). MultiCriteria Analysis of Grid Expansion Concepts on the Low Voltage Level. Zeitschrift für Energiewirtschaft, Vol. 38 No. 3, pp. 183–200. DOI: 10.1007/s123980140134z. Mao, J.Y. and Benbasat, I. (2000). The Use of Explanations in KnowledgeBased Systems: Cognitive Perspectives and a ProcessTracing Analysis. Journal of Management Information Systems, Vol. 17 No. 2, pp. 153–179. DOI: 10.1080/07421222.2000.11045646. Mareschal, B. and Brans, J.P. (1988). Geometrical representations for MCDA. European Journal of Operational Research, Vol. 34 No. 1, pp. 69–77. DOI: 10.1016/S03772217(00)000382. Mateos, A., Jiménez, A. and RíosInsua, S. (2006). Monte Carlo simulation techniques for group decision making with incomplete information. European Journal of Operational Research, Vol. 3 No. 174, pp. 1842–1864. DOI: 10.1016/j.ejor.2005.02.057. Matsatsinis, N.F. and Samaras, A.P. (2001). MCDA and preference disaggregation in group decision support systems. European Journal of Operational Research, Vol. 130 No. 2, pp. 414–429. Mavrotas, G. and Trifillis, P. (2006). Multicriteria decision analysis with minimum information: Combining DEA with MAVT. Computers & Operations Research, Vol. 33 No. 8, pp. 2083–2098. DOI: 10.1016/j.cor.2004.11.023. Morgan, M.G., Henrion, M. and Small, M. (1990), Uncertainty: A guide to dealing with uncertainty in quantitative risk and policy analysis, Cambridge University Press, Cambridge UK, New York. Morton, A. and Fasolo, B. (2009). Behavioural decision theory for multicriteria decision analysis: A guided tour. Journal of the Operational Research Society, Vol. 60 No. 2, pp. 268–275. DOI: 10.1057/palgrave.jors.2602550. Mustajoki, J., Hämäläinen, R.P. and Salo, A. (2005). Decision Support by Interval SMART/SWING  Incorporating Imprecision in the SMART and SWING Methods. Decision Sciences, Vol. 36 No. 2, pp. 317–339. DOI: 10.1111/j.15405414.2005.00075.x. Nunes, I., Miles, S., Luck, M. and Lucena, C. (2012). Investigating Explanations to Justify Choice. In Masthoff, J., Mobasher, B., Desmarais, M.C. and Nkambou, R. (Eds.), User modeling, adaptation, and personalization: 20th International Conference, UMAP 2012, Montreal, Canada, July 1620, 2012. Proceedings, Information Systems and Applications, incl. Internet/Web, and HCI, Vol. 7379, Springer, Berlin, Heidelberg, pp. 212–224. DOI: 10.1007/9783642314544_18. Ouerdane, W., Maudet, N. and Tsoukiàs, A. (2010). Argumentation Theory and Decision Aiding. In Ehrgott, M., Figueira, J.R. and Greco, S. (Eds.), Trends in Multiple Criteria Decision Analysis, International Series in Operations Research & Management Science, Vol. 142, Springer US, pp. 177–208. DOI: 10.1007/9781441959041_7. Pagano, R.R. (2013), Understanding statistics in the behavioral sciences, 10th ed., Wadsworth Publishing, Belmont, CA. Papamichail, K.N. and French, S. (2000). Decision support in nuclear emergencies. Journal of Hazardous Materials, Vol. 71 No. 13, pp. 321–342. DOI: 10.1016/S03043894(99)000862. Papamichail, K.N. and French, S. (2003). Explaining and justifying the advice of a decision support system  A natural language generation approach. Expert Systems with Applications, Vol. 24 No. 1, pp. 35–48. DOI: 10.1016/S09574174(02)000817. Papamichail, K.N. and French, S. (2005). Design and evaluation of an intelligent decision support system for nuclear emergencies. Decision Support Systems, Vol. 41 No. 1, pp. 84–111. DOI: 10.1016/j.dss.2004.04.014. Parikh, M., Fazlollahi, B. and Verma, S. (2001). The Effectiveness of Decisional Guidance: An Empirical Evaluation. Decision Sciences, Vol. 32 No. 2, pp. 303–332. DOI: 10.1111/j.15405915.2001.tb00962.x. Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Philosophical Magazine Series 5, Vol. 50 No. 302, pp. 157–175. DOI: 10.1080/14786440009463897. Pohekar, S.D. and Ramachandran, M. (2004). Application of multicriteria decision making to sustainable energy planning  A review. Renewable and Sustainable Energy Reviews, Vol. 8 No. 4, pp. 365–381. DOI: 10.1016/j.rser.2003.12.007. Reiter, E. and Dale, R. (1997). Building applied natural language generation systems. Natural Language Engineering, Vol. 3 No. 1, pp. 57–87. DOI: 10.1017/S1351324997001502. Ren, J., Fedele, A., Mason, M., Manzardo, A. and Scipioni, A. (2013). Fuzzy Multiactor Multicriteria Decision Making for sustainability assessment of biomassbased technologies for hydrogen production. International Journal of Hydrogen Energy, Vol. 38 No. 22, pp. 9111–9120. DOI: 10.1016/j.ijhydene.2013.05.074. Ribeiro, F., Ferreira, P. and Araújo, M. (2013). Evaluating future scenarios for the power generation sector using a MultiCriteria Decision Analysis (MCDA) tool: The Portuguese case. Energy, Vol. 52, pp. 126–136. DOI: 10.1016/j.energy.2012.12.036. Ríos Insua, D. and French, S. (1991). A framework for sensitivity analysis in discrete multiobjective decisionmaking. European Journal of Operational Research, Vol. 54 No. 2, pp. 176–190. DOI: 10.1016/03772217(91)902968. SánchezHernández, G. (2013). A contribution to the ranking and description of classifications. Dissertation, Instituto de Organización y Control de Sistemas Industriales, Universitat politécnica de Catalunya, Barcelona, 2013. Scholten, L., Scheidegger, A., Reichert, P., Mauer, M. and Lienert, J. (2014). Strategic rehabilitation planning of piped water networks using multicriteria decision analysis. Water research, Vol. 49, pp. 124–143. DOI: 10.1016/j.watres.2013.11.017. Scholten, L., Schuwirth, N., Reichert, P. and Lienert, J. (2015). Tackling uncertainty in multicriteria decision analysis – An application to water supply infrastructure planning. European Journal of Operational Research, Vol. 242 No. 1, pp. 243–260. DOI: 10.1016/j.ejor.2014.09.044. Scott, J.A., Ho, W. and Dey, P.K. (2012). A review of multicriteria decisionmaking methods for bioenergy systems. Energy, Vol. 42 No. 1, pp. 146–156. DOI: 10.1016/j.energy.2012.03.074. Silver, M.S. (1991a). Decisional Guidance for ComputerBased Decision Support. MIS Quarterly, Vol. 15 No. 1, pp. 105–122. DOI: 10.2307/249441. Silver, M.S. (1991b), Systems that support decision makers: Description and analysis, John Wiley information systems series, Wiley, Chichester, New York. Spiegelhalter, D.J. and KnillJones, R.P. (1984). Statistical and KnowledgeBased Approaches to Clinical DecisionSupport Systems, with an Application in Gastroenterology. Journal of the Royal Statistical Society. Series A (General), Vol. 147 No. 1, pp. 35–77. DOI: 10.2307/2981737. Stewart, T.J. (1992). A critical survey on the status of multiple criteria decision making theory and practice. Omega, Vol. 20 No. 56, pp. 569–586. DOI: 10.1016/03050483(92)90003P. Stewart, T.J. (2005). Dealing with Uncertainties in MCDA. In Figueira, J.R., Greco, S. and Ehrgott, M. (Eds.), Multiple criteria decision analysis: State of the art surveys, International Series in Operations Research & Management Science, Vol. 78, Springer, New York, pp. 445–466. DOI: 10.1007/0387230815_11. Streimikiene, D., Balezentis, T., Krisciukaitienė, I. and Balezentis, A. (2012). Prioritizing sustainable electricity production technologies: MCDM approach. Renewable and Sustainable Energy Reviews, Vol. 16 No. 5, pp. 3302–3311. DOI: 10.1016/j.rser.2012.02.067. Swartout, W.R. and Moore, J.D. (1993). Explanation in Second Generation Expert Systems. In David, J.M., Krivine, J.P. and Simmons, R. (Eds.), Second generation expert systems, Springer Berlin Heidelberg, Berlin, pp. 543–585. DOI: 10.1007/9783642779275_26. Tervonen, T. (2014). JSMAA: Open source software for SMAA computations. International Journal of Systems Science, Vol. 45 No. 1, pp. 69–81. DOI: 10.1080/00207721.2012.659706. Tervonen, T. and Figueira, J.R. (2008). A Survey on Stochastic Multicriteria Acceptability Analysis Methods. Journal of multicriteria decision analysis, Vol. 15 No. 12, pp. 1–14. DOI: 10.1002/mcda.407. Tintarev, N. and Masthoff, J. (2007). A Survey of Explanations in Recommender Systems. paper presented at IEEE 23rd International Conference on Data Engineering Workshop, April 1520, 2007, Istanbul. DOI: 10.1109/ICDEW.2007.4401070. Treitz, M. (2006), Production process design using multicriteria analysis, Universitätsverlag Karlsruhe, Karlsruhe. Tversky, A. (1972). Elimination by aspects: A theory of choice. Psychological Review, Vol. 79 No. 4, pp. 281–299. DOI: 10.1037/h0032955. Walker, M.A., Whittaker, S.J., Stent, A., Maloor, P., Moore, J.D., Johnston, M. and Vasireddy, G. (2004). Generation and evaluation of user tailored responses in multimodal dialogue. Cognitive Science, Vol. 28 No. 5, pp. 811–840. DOI: 10.1207/s15516709cog2805_8. Wang, J.J., Jing, Y.Y., Zhang, C.F. and Zhao, J.H. (2009). Review on multicriteria decision analysis aid in sustainable energy decisionmaking. Renewable and Sustainable Energy Reviews, Vol. 13 No. 9, pp. 2263–2278. DOI: 10.1016/j.rser.2009.06.021. Weiner, J.L. (1980). BLAH, a system which explains its reasoning. Artificial Intelligence, Vol. 15 No. 12, pp. 19–48. DOI: 10.1016/00043702(80)900211. Wybo, J.L. (2006). Editorial. International Journal of Emergency Management, Vol. 3 No. 2/3, pp. 99100. Ye, R.L. (1995). The value of explanation in expert systems for auditing: An experimental investigation. Expert Systems with Applications, Vol. 9 No. 4, pp. 543–556. DOI: 10.1016/09574174(95)000232. Zadeh, L.A. (1965). Fuzzy sets. Information and Control, Vol. 8 No. 3, pp. 338–353. DOI: 10.1016/S00199958(65)90241X. Zhou, P., Ang, B.W. and Poh, K.L. (2006). Decision analysis in energy and environmental modeling: An update. Energy, Vol. 31 No. 14, pp. 2604–2622. DOI: 10.1016/j.energy.2005.10.023. Zimmermann, H.J. (2000). An applicationoriented view of modeling uncertainty. European Journal of Operational Research, Vol. 122 No. 2, pp. 190–198. DOI: 10.1016/S03772217(99)002283. 
URI:  https://mpra.ub.unimuenchen.de/id/eprint/75025 