Completing correlation matrices of arbitrary order by differential evolution method of global optimization: A Fortran program

Mishra, SK (2007): Completing correlation matrices of arbitrary order by differential evolution method of global optimization: A Fortran program. Unpublished.

Abstract

Correlation matrices have many applications, particularly in marketing and financial economics. The need to forecast demand for a group of products in order to realize savings by properly managing inventories requires the use of correlation matrices.

In many cases, due to paucity of data/information or dynamic nature of the problem at hand, it is not possible to obtain a complete correlation matrix. Some elements of the matrix are unknown. Several methods exist that obtain valid complete correlation matrices from incomplete correlation matrices. In view of non-unique solutions admissible to the problem of completing the correlation matrix, some authors have suggested numerical methods that provide ranges to different unknown elements. However, they are limited to very small matrices up to order 4.

Our objective in this paper is to suggest a method (and provide a Fortran program) that completes a given incomplete correlation matrix of an arbitrary order. The method proposed here has an advantage over other algorithms due to its ability to present a scenario of valid correlation matrices that might be obtained from a given incomplete matrix of an arbitrary order. The analyst may choose some particular matrices, most suitable to his purpose, from among those output matrices. Further, unlike other methods, it has no restriction on the distribution of holes over the entire matrix, nor the analyst has to interactively feed elements of the matrix sequentially, which might be quite inconvenient for larger matrices. It is flexible and by merely choosing larger population size one might obtain a more exhaustive scenario of valid matrices.

• Item Type: MPRA Paper
• Institution: North-Eastern Hill University, Shillong (India)
• Language: English
• Keywords: Incomplete; complete; correlation matrix; valid; semi-definite; eigenvalues; Differential Evolution; global optimization; computer program; fortran; financial economics; arbitrary order
• Subjects: G - Financial Economics > G1 - General Financial Markets > G10 - General; C - Mathematical and Quantitative Methods > C8 - Data Collection and Data Estimation Methodology; Computer Programs > C88 - Other Computer Software; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods and Programming > C63 - Computational Techniques; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods and Programming > C61 - Optimization Techniques; Programming Models; Dynamic Analysis
• References: · Anjos, MF, NJ Higham, PL Takouda and H Wolkowicz (2003) “A Semidefinite Programming Approach for the Nearest Correlation Matrix Problem”, Preliminary Research Report, Dept. of Combinatorics & Optimization, Waterloo, Ontario.; · Barett, WW, Johnson, CR and Lundquist, M (1989). “Determinantal Formulae for Matrix Completions Associated with Chordal Graphs”. Linear Algebra and its Applications, 121:265–289.; · Barrett, WW, Johnson, CR and Loewy, R (1998). “Critical Graphs for the Positive Definite Completion Problem”. SIAM Journal of Matrix Analysis and Applications, 20:117–130. ; · Budden, M, Hadavas, P, Hoffman, L and Pretz, C (2007) “Generating Valid 4 x 4 Correlation Matrices”, Applied Mathematics E-Notes, 7:53-59. ; · Chesney, M and Scott, L (1989). “Pricing European Currency Options: A Comparison of the Modified Black-Scholes Model and a Random Variance Model”. Journal of Financial and Quantitative Analysis, 24:267–284.; · Glass, G and Collins, J (1970) “Geometric Proof of the Restriction on the Possible Values of rxy when rxz and ryx are Fixed”, Educational and Psychological Measurement, 30:37-39. ; · Grone, R, Johnson, CR, Sá, EM and Wolkowicz, H (1984).” Positive Definite Completions of Partial Hermitian Matrices”. Linear Algebra and its Applications, 58:109–124.; · Grubisic, I and R Pietersz (2004) “Efficient Rank Reduction of Correlation Matrices”, Working Paper Series, SSRN, http://ssrn.com/abstract=518563; · Helton, JW, Pierce, S and Rodman, L (1989). “The Ranks of Extremal Positive Semidefinite Matrices with given Sparsity Pattern”. SIAM Journal on Matrix Analysis and its Applications, 10:407–423. ; · Heston, SL (1993). “A Closed-form Solution for Options with stochastic Volatility with Applications to Bond and Currency Options”. The Review of Financial Studies, 6:327–343. ; · Higham, NJ (2002). “Computing the Nearest Correlation Matrix – A Problem from Finance”, IMA Journal of Numerical Analysis, 22, pp. 329-343.; · Johnson, C (1990). “Matrix Completion Problems: A Survey”. Matrix Theory and Applications, 40:171–198.; · Kahl, C and Günther, M (2005). “Complete the Correlation Matrix”. http://www.math.uni-wuppertal.de/~kahl/publications/CompleteTheCorrelationMatrix.pdf; · Kahl, C and Jäckel, P (2005). “Fast Strong Approximation Monte-Carlo Schemes for Stochastic Volatility Models”. Working paper, http://www.math.uni-wuppertal.de/_kahl/publications.html. ; · Laurent, M (2001). “Matrix Completion Problems”. The Encyclopedia of Optimization, 3:221–229.; · Marsaglia, G. and Olkin, I (1984). “Generating Correlation Matrices”. SIAM Journal on Scientific and Statistical Computing, 5(2):470-475.; · Mishra, SK (2004) “Optimal Solution of the Nearest Correlation Matrix Problem by Minimization of the Maximum Norm". http://ssrn.com/abstract=573241; · Mishra, SK (2006) “Global Optimization by Differential Evolution and Particle Swarm Methods: Evaluation on Some Benchmark Functions”. http://ssrn.com/abstract=933827 ; · Olkin, I (1981) “Range Restrictions for Product-Moment Correlation Matrices”, Psychometrika, 46:469-472.; · Pietersz, R and PJF Groenen (2004) “Rank Reduction of Correlation Matrices by Majorization”, Econometric Institute Report EI 2004-11, Erasmus Univ. Rotterdam.; · Rebonato, R and P Jäckel (1999) “The Most General Methodology to Create a Valid Correlation Matrix for Risk Management and Option Pricing Purposes”, Quantitative Research Centre, NatWest Group, http://www.rebonato.com/CorrelationMatrix.pdf; · Schöbel, R and Zhu, J (1999). “Stochastic Volatility With an Ornstein Uhlenbeck Process: An Extension”. European Finance Review, 3:23–46, ssrn.com/abstract=100831.; · Stanley, J and Wang, M (1969) “Restrictions on the Possible Values of r12, given r13 and r23” , Educational and Psychological Measurement, 29, pp.579-581.; · Storn, R and Price, K (1995) "Differential Evolution - A Simple and Efficient Adaptive Scheme for Global Optimization over Continuous Spaces": Technical Report, International Computer Science Institute, Berkley.; · Tyagi, R and Das, C (1999) “Grouping Customers for Better Allocation of Resources to Serve Correlated Demands”, Computers and Operations Research, 26:1041-1058.; · Xu, K and Evers, P (2003) “Managing Single Echelon Inventories through Demand Aggregation and the Feasibility of a Correlation Matrix”, Computers and Operations Research, 30:297-308.