The best estimation for high-dimensional Markowitz mean-variance optimization

Bai, Zhidong and Li, Hua and Wong, Wing-Keung (2013): The best estimation for high-dimensional Markowitz mean-variance optimization.

Abstract

The traditional(plug-in) return for the Markowitz mean-variance (MV) optimization has been demonstrated to seriously overestimate the theoretical optimal return, especially when the dimension to sample size ratio $p/n$ is large. The newly developed bootstrap-corrected estimator corrects the overestimation, but it incurs the "under-prediction problem," it does not do well on the estimation of the corresponding allocation, and it has bigger risk. To circumvent these limitations and to improve the optimal return estimation further, this paper develops the theory of spectral-corrected estimation. We first establish a theorem to explain why the plug-in return greatly overestimates the theoretical optimal return. We prove that under some situations the plug-in return is $\sqrt{\gamma}\ $\ times bigger than the theoretical optimal return, while under other situations, the plug-in return is bigger than but may not be $\sqrt{\gamma}\ $\ times larger than its theoretic counterpart where $\gamma = \frac 1{1-y}$ with $y$ being the limit of the ratio $p/n$.

Thereafter, we develop the spectral-corrected estimation for the Markowitz MV model which performs much better than both the plug-in estimation and the bootstrap-corrected estimation not only in terms of the return but also in terms of the allocation and the risk. We further develop properties for our proposed estimation and conduct a simulation to examine the performance of our proposed estimation. Our simulation shows that our proposed estimation not only overcomes the problem of "over-prediction," but also circumvents the "under-prediction," "allocation estimation," and "risk" problems. Our simulation also shows that our proposed spectral-corrected estimation is stable for different values of sample size $n$, dimension $p$, and their ratio $p/n$. In addition, we relax the normality assumption in our proposed estimation so that our proposed spectral-corrected estimators could be obtained when the returns of the assets being studied could follow any distribution under the condition of the existence of the fourth moments.

• Item Type: MPRA Paper
• Language: English
• Keywords: Markowitz mean-variance optimization, Optimal Return, Optimal Portfolio Allocation, Large Random Matrix, Bootstrap Method
• Subjects: G - Financial Economics > G1 - General Financial Markets > G11 - Portfolio Choice ; Investment Decisions; C - Mathematical and Quantitative Methods > C3 - Multiple or Simultaneous Equation Models ; Multiple Variables

References

Bai, Z.D., Chen, J. and Yao, J.-F. 2010. On estimation of the population spectral distribution from a high-dimensional sample covariance matrix. Australian & New Zeland Journal of Statistics, 52, 423-437

Bai, Z.D., Hui, Y.C., Wong, W.K., Zitikis, R., 2012. Evaluating Prospect Performance: Making a Case for a Non-Asymptotic UMPU Test, Journal of Financial Econometrics 10(4), 703-732.

Bai, Z.D., Li, H., Liu, H.X., Wong, W.K., 2011. Test Statistics for Prospect and Markowitz Stochastic Dominances with Applications. Econometrics Journal 122, 1-26.

Bai, Z.D., Liu, H.X., Wong, W.K., 2009. Enhancement of the Applicability of Markowitz's Portfolio Optimization by Utilizing Random Matrix Theory. Mathematical Finance 19(4), 639-667.

Bai, Zhidong, Huixia Liu and Wing-Keung Wong, (2009a), On the Markowitz Mean-Variance Analysis of Self-Financing Portfolios, Risk and Decision Analysis 1(1), 35-42.

Bai, Z.D., Liu, H.X., Wong, W.K., 2011. Asymptotic Properties of Eigenmatrices of A Large Sample Covariance Matrix. Annals of Applied Probability 21(5), 1994-2015.

Bai, Z.D., Miao, B.Q. and Pan, G.M. (2007). On the asymptotics of eigenvectors of large sample covariance matrix. Ann. Prob. 35.(.4), 1532-1572.

Bai, Z.D. and Jack W. Silverstein, 2010. Spectral Analysis of Large Dimensional Random Matrices. Second edition, Since Press, Beijing.

Bai, Z.D., Wang, K.Y., Wong, W.K., 2011. Mean-Variance Ratio Test, A Complement to Coefficient of Variation Test and Sharpe Ratio Test, Statistics and Probability Letters, 81(8), 1078-1085.

Blattberrg, R.C. and N.J. Gonedes (1974). A comparson of stable and student distribution as statistical models for stock prices. J. Bus. (47) 244-280.

Broll, U., Egozcue, M., Wong, W.K., Zitikis, R., 2010. Prospect Theory, Indifference Curves, and Hedging Risks. Applied Mathematics Research Express 2010(2), 142-153.

Broll, U., Wahl, J.E., Wong, W.K., 2006. Elasticity of Risk Aversion and International Trade. Economics Letters 91(1), 126-130.

Brown, S.J. (1978) The portfolio choice problem: comparison of certainty equivalence and optimal bayes portfolios. Commun. Stat. Simul. Comput. (7) 321-334.

Chan, C.Y., de Peretti, C., Qiao, Z., Wong, W.K., 2012. Empirical Test of the Efficiency of UK Covered Warrants Market: Stochastic Dominance and Likelihood Ratio Test Approach. Journal of Empirical Finance 19(1), 162-174.

Chan, J.C.C., Kroese D.P., 2010. Efficient estimation of large portfolio loss probabilities in t-copula models. European Journal of Operational Research 205(2), 361-367.

Clark, P.K. (1973). A subordinated stochastic process model with finite variance for speculative prices. {Econometrica} {37} 135-155.

Carpenter, G. A., Grossberg, S. and Reynolds, J. H. (1991). ARTMAP: Supervised real-time learning ad classification of nonstationary data by a self-organizing neural network. Neural Netw. 4:565-588.

Chopra, V.K., Hensel, C.R. and Turner, A.L. (1993). Massaging mean-variance inputs: returns from alternative global investment strategies in the 1980s. Management Science. 39(7) 845-855.

Clark, E., Jokung, O., Kassimatis, K., 2011. Making inefficient market indices efficient. European Journal of Operational Research 209(1) 83-93.

Egozcue, M., Wong, W.K., 2010. Gains from Diversification: A Majorization and Stochastic Dominance Approach. European Journal of Operational Research 200(3), 893-900.

Egozcue, M., Fuentes Garc\'ia, L., Wong, W.K., Zitikis, R., 2011. Do Investors Like to Diversify? A Study of Markowitz Preferences. European Journal of Operational Research 215(1), 188-193.

El Karoui Noureddine, 2008, Spectrum estimation for large dimensional covariance matrices using random matrix theory, The Annals of Statistics, Volume 36, Number 6 (2008), 2757-2790.

Elton, e.J., M.J. Gruber and M.W. Padberg (1976). Simple criteria for optimal portfolio selection. J. Finance 31(5) 1341-1357.

Elton, E.J., M.J. Gruber and M.W. Padberg (1978). Simple criteria for optimal portfolio selection: tracing out the efficient frontier. J. Finance 33(1) 296-302.

Fama, E.F. (1963). Mandelbrot and the stable paretian hypothesis. J. Bus. 36 420-429.

Fama, E.F. (1965). Portfolio Analysis in a stable paretian market. Manage. Sc. 11 401-419.

Fielitz, B.D. and J.P. Rozelle (1983). Stable distributions and mixtures of distributions hypotheses for common stock returns. J. Am. Stat. Assoc. 78 28-36.

Fong, Wai Mun and Wing-Keung Wong, (2006), The Modified Mixture of Distributions Model: A Revisit, Annals of Finance, 2(2), 167 - 178.

Frankfurter, G.M., Phillips, H.E., Seagle, J.P., 1971. Portfolio Selection: The Effects of Uncertain Means, Variances and Covariances. Journal of Financial and Quantitative Analysis 6, 1251-1262.

Fu, C., Lari-Lavassani, A., Li, X., 2010. Dynamic mean-variance portfolio selection with borrowing constraint. European Journal of Operational Research 200(1), 312-319.

Gasbarro, D., Wong, W.K., Zumwalt, J.K., 2007. Stochastic dominance analysis of iShares. European Journal of Finance 13, 89-101.

Hall, P., 1992. The Bootstrap and Edgeworth Expansion. Springer Series in Statistics. Springer-Verlag, New York.

Hensel, C. R. and A. L. Turner. 1998. Making Superior Asset Allocation Decisions: A Practitioner's Guide." In Worldwide Asset and Liability Modeling, Chapter 4, edited by W. T. Ziemba and J. M. Mulvey. Cambridge, United Kingdom: Cambridge University Press.

Jacobs, B.J., Levy, K.N., Markowitz, H.M., 2005. Portfolio Optimization with Factors, Scenarios, and Realistic Short Positions. Operations Research 53, 586-599.

Jansen, D.W., Li, Q., Wang, Z., Yang, J., 2008. Fiscal Policy and Asset Markets: A Semiparametric Analysis. Journal of Econometrics 147(1), 141-150.

Jorion, P., 1985. International Portfolio Diversification with Estimation Risk. Journal of Business 58(3), 259-278.

Jorion, P. (1992). Portfolio optimization in practice. Financial Analysts Journal. 48 68-74.

Kan, R., Zhou, G., 2007. Optimal Portfolio Choice With Parameter Uncertainty. Journal of Financial and Quantitative Analysis 42(3), 621-656.

Konno, H., Watanabe, H., 1996. Bond portfolio optimization problems and their applications to index tracking: a partial optimization approach, Journal of the Operations Research Society of Japan 39(3), 295-306.

Konno, H., Yamazaki, H., 1991. Mean-Absolute Deviation Portfolio Optimization Model and its Applications to Tokyo Stock Market. Management Science 37(5), 519-531.

Korkie Bob, Turtle Harry J., 2002. A Mean-Variance Analysis of Self-Financing Portfolios. Management Science, vol. 48, no. 3, pp. 427-443, 2002

Kroll, Yoram \& Levy, Haim \& Markowitz, Harry M, 1984. " Mean-Variance versus Direct Utility Maximization," Journal of Finance, American Finance Association, vol. 39(1), pages 47-61, March.

Laloux, L., Cizeau, P., Bouchaud, J.P., Potters, M., 1999. Noise Dressing of Financial Correlation Matrices. Physical Review Letters 83, 1467-1470.

Lam, K., Liu, T., Wong, W.K., 2010. A pseudo-Bayesian model in financial decision making with implications to market volatility, under- and overreaction. European Journal of Operational Research 203(1), 166-175.

Ledoit, O., Wolf, M., 2004. A Well-Conditioned Estimator for Large-Dimensional Covariance Matrices. Journal of Multivariate Analysis 88(2), 365-411.

Ledoit, O., Wolf, M., 2012, Nonlinear Shrinkage Estimation of Large-Dimensional Covariance Matrices, The Annals of Statistics, Vol. 40, No. 2, 1024-1060.

Leung, P.L.,Ng, H.Y., Wong, W.K., 2012. An Improved Estimation to Make Markowitz's Portfolio Optimization Theory Users Friendly and Estimation Accurate with Application on the US Stock Market Investment, European Journal of Operational Research 222, 85-95.

Leung, P.L., Wong, W.K., 2008. On Testing The Equality Of The Multiple Sharpe Ratios, With Application On The Evaluation Of Ishares. Journal of Risk 10(3), 1-16.

Li, W. M., Chen, J. Q., Qin, Y. L., Yao, J. F., Bai, Z. D., 2013, Estimation of the population spectral distribution from a large dimensional sample covariance matrix, working paper.

Lo, A.W., MacKinlay, A.C., 1988. Stock market prices do not follow random walks: evidence from a simple specification test. Review of Financial Studies 1, 41-66.

Ma, C., Wong, W.K., 2010. Stochastic Dominance and Risk Measure: A Decision-Theoretic Foundation for VaR and C-VaR. European Journal of Operational Research 207(2), 927-935.

Manganelli Simone, 2004. Asset Allocation by Variance Sensitivity Analysis, Journal of Financial Econometrics, Society for Financial Econometrics, vol. 2(3), pages 370-389.

Markowitz, H.M., 1952. Portfolio Selection. Journal of Finance 7, 77-91.

Markowitz, H.M., 1959. Portfolio Selection. New York: John Wiley and Sons, Inc.

Markowitz, H.M., and A.F. Perold (1981). Portfolio analysis with factors and scenarios. J. Finance 871-877.

Markowitz, Harry M, 1991. Foundations of Portfolio Theory, Journal of Finance, American Finance Association, vol. 46(2), pages 469-77, June.

McNamara, J.R., 1998. Portfolio Selection Using Stochastic Dominance Criteria. Decision Sciences 29(4), 785-801.

Michaud, R.O., 1989. The Markowitz Optimization Enigma: Is `Optimized' Optimal? Financial Analysts Journal 45, 31-42.

Muirhead, R.J., 1982. Aspects of Multivariate Statistical Theory. New York: John Wiley and Sons, Inc.

Pan, G.M. Comparision between two types of large sample covariance matrices. Accepted, Annales de l'Institut Henri Poincar\'{e}

Peccati, L., Cifarelli, D.M., 2003. VaR as a Risk Measure for Multiperiod Static Inventory Models. International Journal of Production Economics 81-82(1), 375-384.

Perold, A.F., 1984. Large-Scale Portfolio Optimization. Management Science 30(10), 1143-1160.

Ross, S.A., 1976. The Arbitrage Theory of Capital Asset Pricing. Journal of Economic Theory 13, 341-360.

Sharpe, W.F., 1964. Capital Asset Prices: A Theory of Market Equilibrium under Conditions of Risk. Journal of Finance 19, 425-442.

Sharpe, W.F., 1967. A Linear Programming Algorithm for Mutual Fund Portfolio Selection. Management Sciences 22, 499-510.

Sharpe, William F., 1971. A Linear Programming Approximation for the General Portfolio Analysis Problem. Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 6(05), pages 1263-1275, December.

Simaan, Y., 1997. Estimation Risk in Portfolio Selection: The Mean Variance Model versus the Mean Absolute Deviation Model. Management Science 43(10), 1437-1446.

Siverstein, J. W. (1995). Strong convergence of the empirical distribution of eigenvalues of large dimensional random matrices. J. Multivariate Anal. 5 331-339.

Silverstein, J.W. and Choi, S.I. (1995). Analysis of the limiting spectral distribution of large dimensional random matrices. J. Multivariate Anal. 54 295-309.

Stone, B.K.(1973). A linear Programming formulation of the general portfolio selection problem. J. Finan. Quant. Anal. 8(4) 621-636.

Soyer, R., Tanyeri, K., 2006. Bayesian portfolio selection with multi-variate random variance models. European Journal of Operational Research 171(3) 977-990.

Wong, F., Carter, C.K., Kohn, R., 2003. Efficient Estimation of Covariance Selection Models. Biometrika 90, 809-830.

Wong, W.K., 2007. Stochastic Dominance and Mean-Variance Measures of Profit and Loss for Business Planning and Investment. European Journal of Operational Research 182(2), 829-843.

Wong, W.K., Chan, R., 2008. Markowitz and Prospect Stochastic Dominances. Annals of Finance 4(1), 105-129.

Wong, W.K., Ma, C., 2008. Preferences over Meyer's location-scale family. Economic Theory 37(1), 119-146.

Wong, W.K., Phoon, K.F., Lean, H.H., 2008. Stochastic dominance analysis of Asian hedge funds, Pacific-Basin Finance Journal 16(3), 204-223.

Yu, M., Takahashi, S., Inoue, H., Wang, S., 2010. Dynamic portfolio optimization with risk control for absolute deviation model. European Journal of Operational Research 201(2), 349-364.

Zellner, A. and V.K. Chetty (1965). Prediction and descision problems in regression models from the bayesian point of view. J. Am. Stat. Assoc. 60 608-616.

Zhao, Y., Ziemba, W.T., 2008. Calculating risk neutral probabilities and optimal portfolio policies in a dynamic investment model with downside risk control. European Journal of Operational Research 185(3), 1525-1540.

Zymler, S., Rustem, B., Kuhn, D., 2011. Robust portfolio optimization with derivative insurance guarantees. European Journal of Operational Research 210(2), 410-424.