Credit scoring with boosted decision trees

Bastos, Joao (2008): Credit scoring with boosted decision trees. Forthcoming in:

Abstract

The enormous growth experienced by the credit industry has led researchers to develop sophisticated credit scoring models that help lenders decide whether to grant or reject credit to applicants. This paper proposes a credit scoring model based on boosted decision trees, a powerful learning technique that aggregates several decision trees to form a classifier given by a weighted majority vote of classifications predicted by individual decision trees. The performance of boosted decision trees is evaluated using two publicly available credit card application datasets. The prediction accuracy of boosted decision trees is benchmarked against two alternative data mining techniques: the multilayer perceptron and support vector machines. The results show that boosted decision trees are a competitive technique for implementing credit scoring models.

• Item Type: MPRA Paper
• Language: English
• Keywords: Credit scoring; Boosting; Decision tree; neural network; support vector machine
• Subjects: C - Mathematical and Quantitative Methods > C4 - Econometric and Statistical Methods: Special Topics > C44 - Operations Research ; Statistical Decision Theory; G - Financial Economics > G3 - Corporate Finance and Governance > G32 - Financing Policy ; Financial Risk and Risk Management ; Capital and Ownership Structure ; Value of Firms ; Goodwill

References

A. Asuncion and D.J. Newman. UCI machine learning repository, 2007. URL http://www.ics.uci.edu/~mlearn/MLRepository.html.

B. Baesens, T. Van Gestel, S. Viaene, M. Stepanova, J. Suykens, and J. Vanthienen. Benchmarking state-of-the-art classification algorithms for credit scoring. Journal of the Operational Research Society, 54:1028--1088, 2003.

L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone. Classification and regression trees. Wadworth International Group, Belmont, California, 1984.

J.N. Crook, D.B. Edelman, and L.C. Thomas. Recent developments in consumer credit risk assessment. European Journal of Operational Research, 183:1447--1465, 2007.

R.H. Davis, D.B. Edelman, and A.J. Gammerman. Machine learning algorithms for credit-card applications. IMA Journal of Management Mathematics, 4:43--51, 1992.

E. DeLong, D. DeLong, and D. Clarke-Pearson. Comparing the area under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics, 44:837--845, 1988.

Y. Freund and R.E. Schapire. A short introduction to boosting. J. Jpn, Soc. Artif. Intell., 14(5):771, 1991.

Y. Freund and R.E. Schapire. Experiments with a new boosting algorithm. in: Proceedings of the 13th International Conference on Machine Learning, pages 148--156, 1996.

J. Friedman. Recent advances in predictive (machine) learning. Proceedings of Phystat, Stanford University, 2003.

H.E. Frydman, E.I. Altman, and D-L. Kao. Introducing recursive partitioning for financial classification: the case of financial distress. Journal of Finance, 269--91:40(1), 1985.

W.E. Henley and D.J. Hand. A k-nearest neighbor classifier for assessing consumer risk. Statician, 44(1):77--95, 1996.

A. Hoecker, P. Speckmayer, J. Stelzer, F. Tegenfeldt, H. Voss, and K. Voss. TMVA -- toolkit for multivariate data analysis. arXiv:physics/0703039, 2007.

K. Hornik, M. Stinchcombe, and H. White. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359--366, 1989.

N.-C. Hsieh. Hybrid mining approach in the design of credit scoring models. Expert Systems with Applications, 28(4):655--665, 2005.

C.-W. Hsu, C.-C. Chang, and C.-J. Lin. A pratical guide to support vector classification, 2007. URL http://www.csie.ntu.edu.tw/~cjlin.

H.L. Jensen. Using neural networks for credit scoring. Managerial Finance, 18(6):15--26, 1992.

T.-S. Lee and I.-F. Chen. A two-stage hybrid credit scoring model using artificial neural networks and multivariate adaptive regression splines. Expert Systems with Applications, 28(4):743--752, 2002.

T.-S. Lee, C.-C. Chiu, C.-J Lu, and I.-F. Chen. Credit scoring using the hybrid neural discriminant technique. Expert Systems with Applications, 23(3):245--254, 2002.

C.-S. Ong, J.-J. Huang, and G.-H. Tzeng. Building credit scoring models using genetic programming. Expert Systems with Applications, 29(1):41--47, 2005.

A.K. Reichert, C.C. Cho, and G.M. Wagner. An examination of the conceptual issues involved in developing credit-scoring models. Journal of Business and Economic Statistics, 1(2):101--114, 1983.

R.E. Schapire. The boosting approach to machine learning: an overview. in: Proceedings of the 2002 MSRI Workshop on Nonlinear Estimation and Classification, Springer Verlag, pages 149--173, 2002.

D. West. Neural network credit scoring models. Computers and Operations Research, 27: 1131--1152, 2000.

D.West, S. Dellana, and J. Qian. Neural network ensemble strategies for financial decision applications. Computers and Operations Research, 32:2543--2559, 2005.

J.C. Wiginton. A note on the comparison of logit and discriminant models of consumer credit behavior. Journal of Financial and Quantitative Analysis, 15:757--770, 1980.