Finite Gaussian Mixture Approximations to Analytically Intractable Density Kernels

Khorunzhina, Natalia and Richard, Jean-Francois (2016): Finite Gaussian Mixture Approximations to Analytically Intractable Density Kernels.

Abstract

The objective of the paper is that of constructing finite Gaussian mixture approximations to analytically intractable density kernels. The proposed method is adaptive in that terms are added one at the time and the mixture is fully re-optimized at each step using a distance measure that approximates the corresponding importance sampling variance. All functions of interest are evaluated under Gaussian quadrature rules. Examples include a sequential (filtering) evaluation of the likelihood function of a stochastic volatility model where all relevant densities (filtering, predictive and likelihood) are closely approximated by mixtures.

• Item Type: MPRA Paper
• Language: English
• Keywords: Finite mixture, Distance measure, Gaussian quadrature, Importance sampling, Adaptive algorithm, Stochastic volatility, Density kernel
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C11 - Bayesian Analysis: General; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C63 - Computational Techniques ; Simulation Modeling

References

Adcock, Chris J. 2004. Capital asset pricing in UK stocks under the multivariate skew-normal distribution. Pages 191-204 of: Genton, Marc G. (ed), Skew-elliptical distributions and their applications. London: Chapman & Hall/CRC.

Adcock, Chris J. 2010. Asset pricing and portfolio selection based on the multivariate extended skew-Student-t distribution. Annals of Operations Research, 176(1), 221-234.

Ardia, David, Hoogerheide, Lennart, and van Dijk, Herman. 2009. Adaptive Mixture of Student-t Distributions as a Flexible Candidate Distribution for Efficient Simulation: The R Package AdMit. Journal of Statistical Software, 29(1), 1-32.

Azzalini, Adelchi, and Dalla Valle, Alessandra. 1996. The multivariate skew-normal distribution. Biometrika, 83(4), 715-726.

Basturk, Nalan, Hoogerheide, Lennart, Opschoor, Anne, and van Dijk, Herman. 2012. The R Package MitISEM: Mixture of Student-t Distributions using Importance Sampling Weighted Expectation Maximization for Efficient and Robust Simulation. Tinbergen Institute Discussion Paper 12-096/III.

Bornkamp, Bjorn. 2011. Approximating Probability Densities by Iterated Laplace Approximations. Journal of Computational and Graphical Statistics, 20(3), 656-669.

Bradley, Paul S., and Fayyad, Usama M. 1998. Refining Initial Points for K-Means Clustering. Pages 91-99 of: Shavlik, Jude W. (ed), Proceedings of the Fifteenth International Conference on Machine Learning. San Francisco, CA, USA: Morgan Kaufmann.

Bungartz, Hans-Joachim, and Griebel, Michael. 2004. Sparse grids. Acta Numerica, 13(5), 147-269.

Cameron, Stephen V., and Heckman, James J. 2001. The Dynamics of Educational Attainment for Black, Hispanic, and White Males. Journal of Political Economy, 109(3), 455-499.

Cappe, Olivier, Guillin, Arnaud, Marin, Jean-Michel, and Robert, Christian P. 2004. Population Monte Carlo. Journal of Computational and Graphical Statistics, 13, 907-929.

Cappe, Olivier, Godsill, Simon J., and Moulines, Eric. 2007. An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo. Proceedings of the IEEE, 95(5), 899-924.

Cappe, Olivier, Douc, Randal, Guillin, Arnaud, Marin, Jean-Michel, and Robert, Christian P. 2008. Adaptive Importance Sampling in General Mixture Classes. Statistics and Computing, 18(4), 447-459.

DeSarbo, Wayne S., Degeratu, Alexandru M., Wedel, Michel, and Saxton, M.Kim. 2001. The Spatial Representation of Market Information. Marketing Science, 20(4), 426-441.

Dominguez-Molina, J.Armando, Gonzalez-Faras, Graciela, and Ramos-Quiroga, Rogelio. 2004. Skew-normality in stochastic frontier analysis. Pages 223-242 of: Genton, Marc G. (ed), Skew-elliptical distributions and their applications. London: Chapman & Hall/CRC.

Douc, Randal, Guillin, Arnaud, Marin, Jean-Michel, and Robert, Christian P. 2007. Minimum Variance Importance Sampling via Population Monte Carlo. ESAIM: Probability and Statistics, 11(8), 427-447.

Doucet, Arnaud, de Freitas, Nando, and Gordon, Neil. 2001. Sequential Monte Carlo Methods in Practice. Springer New York.

Duffie, Darrell, and Pan, Jun. 1997. An overview of value at risk. The Journal of Derivatives, 4(3), 7-49.

Duong, Tarn. 2007. ks: Kernel Density Estimation and Kernel Discriminant Analysis for Multivariate Data in R. Journal of Statistical Software, 21(1), 1-16.

Everitt, Brian S., and Hand, David J. 1981. Finite Mixture Distributions. London: Chapman and Hall.

Ferguson, Thomas S. 1973. A Bayesian Analysis of Some Nonparametric Problems. The Annals of Statistics, 1(2), 209-230.

Fruhwirth-Schnatter, Sylvia. 2006. Finite mixture and Markov Switching Models. New York: Springer.

Gilks, Walter R., Roberts, Gareth O., and Sahu, Sujit K. 1998. Adaptive Markov Chain Monte Carlo through Regeneration. Journal of the American Statistical Association, 93(443), 1045-1054.

Giordani, Paolo, and Kohn, Robert. 2010. Adaptive Independent Metropolis-Hastings by Fast Estimation of Mixtures of Normals. Journal of Computational and Graphical Statistics, 19(2), 243-259.

Hamerly, Greg, and Elkan, Charles. 2002. Alternatives to the k-Means Algorithm That Find Better Clusterings. Pages 600-607 of: Nicholas, Charles, Grossman, David, Kalpakis, Konstantinos, Qureshi, Sajda, van Dissel, Han, and Seligman, Len (eds), Proceedings of the Eleventh International Conference on Information and Knowledge Management. New York, NY, USA: ACM.

Hamilton, James D. 1989. A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle. Econometrica, 57(2), 357-384.

Han, Bohyung, Comaniciu, Dorin, Zhu, Ying, and Davis, Larry S. 2008. Sequential Kernel Density Approximation and Its Application to Real-Time Visual Tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(7), 1186-1197.

Heiss, Florian, and Winschel, Viktor. 2008. Likelihood approximation by numerical integration on sparse grids. Journal of Econometrics, 144(1), 62-80.

Holmes, George K. 1892. Measures of Distribution. Publications of the American Statistical Association, 3(18/19), 141-157.

Hoogerheide, Lennart, Opschoor, Anne, and van Dijk, Herman K. 2012. A class of adaptive importance sampling weighted EM algorithms for efficient and robust posterior and predictive simulation. Journal of Econometrics, 171(2), 101-120.

Hoogerheide, Lennart F., Kaashoek, Johan F., and van Dijk, Herman K. 2007. On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: An application of flexible sampling methods using neural networks. Journal of Econometrics, 139(1), 154-180.

Hull, John, and White, Alan. 1998. Incorporating Volatility Updating into the Historical Simulation Method for Value-at-risk. Journal of Risk, 1, 5-19.

Kasahara, Hiroyuki, and Shimotsu, Katsumi. 2015. Testing the Number of Components in Normal Mixture Regression Models. Journal of the American Statistical Association, 110(512), 1632-1645.

Keane, Michael P, and Wolpin, Kenneth I. 1997. The Career Decisions of Young Men. Journal of Political Economy, 105(3), 473-522.

Kim, Sangjoon, Shephard, Neil, and Chib, Siddhartha. 1998. Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models. Review of Economic Studies, 65(3), 361-393.

Kon, Stanley J. 1984. Models of Stock Returns - A Comparison. Journal of Finance, 39(1), 147-165.

Kurtz, Nolan, and Song, Junho. 2013. Cross-entropy-based adaptive importance sampling using Gaussian mixture. Structural Safety, 42, 35-44.

Liesenfeld, Roman, and Richard, Jean-Francois. 2006. Classical and Bayesian Analysis of Univariate and Multivariate Stochastic Volatility Models. Econometric Reviews, 25(2-3), 335-360.

Marchenko, Yulia V., and Genton, Marc G. 2012. A Heckman Selection-t Model. Journal of the American Statistical Association, 107(497), 304-317.

Mazzuco, Stefano, and Scarpa, Bruno. 2015. Fitting age-specific fertility rates by a flexible generalized skew normal probability density function. Journal of the Royal Statistical Society Series A, 178(1), 187-203.

McLachlan, Geoffrey J., and Peel, David. 2000. Finite mixture models. New York: J. Wiley & Sons.

Moe, Wendy W., and Fader, Peter S. 2002. Using Advance Purchase Orders to Forecast New Product Sales. Marketing Science, 21(3), 347-364.

Newcomb, Simon. 1886. A Generalized Theory of the Combination of Observations so as to Obtain the Best Result. American Journal of Mathematics, 8(4), 343-366.

Ogundimu, Emmanuel O., and Hutton, Jane L. 2016. A Sample Selection Model with Skew-normal Distribution. Scandinavian Journal of Statistics, 43(1), 172-190.

Oh, Man-Suk, and Berger, James O. 1993. Integration of Multimodal Functions by Monte Carlo Importance Sampling. Journal of the American Statistical Association, 88(422), 450-456.

Omori, Yasuhiro, Chib, Siddhartha, Shephard, Neil, and Nakajima, Jouchi. 2007. Stochastic volatility with leverage: Fast and efficient likelihood inference. Journal of Econometrics, 140(2), 425-449.

Pearson, Karl. 1894. Contributions to the Mathematical Theory of Evolution. Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 185, 71-110.

Pitt, Michael K., and Shephard, Neil. 1999. Filtering via Simulation: Auxiliary Particle Filters. Journal of the American Statistical Association, 94(446), 590-599.

Richard, Jean-Francois, and Zhang, Wei. 2007. Efficient High-Dimensional Importance Sampling. Journal of Econometrics, 141(2), 1385-1411.

Ristic, Branko, Arulampalam, Sanjeev, and Gordon, Neil. 2004. Beyond the Kalman filter: particle filters for tracking applications. Boston, London: Artech House.

Scott, David W. 1992. Multivariate Density Estimation. New Jersey: Wiley.

Scott, David W, and Szewczyk, William F. 2001. From Kernels to Mixtures. Technometrics, 43(3), 323-335.

Smolyak, Sergey A. 1963. Quadrature and interpolation formulas for tensor products of certain class of functions. Soviet Mathematics, Doklady, 148(5), 1042-1045.

Titterington, Michael, Smith, Adrian F.M., and Makov, Udi E. 1985. Statistical Analysis of Finite Mixture Distributions. Chichester: Wiley.

Tucker, Alan L. 1992. A Reexamination of Finite- and Infinite-Variance Distributions as Models of Daily Stock Returns. Journal of Business and Economic Statistics, 10(1), 73-81.

Venkataraman, Subu. 1997. Value at risk for a mixture of normal distributions: the use of quasi- Bayesian estimation techniques. Economic Perspectives, 2-13.

Wang, Xuxu, and Wang, Yong. 2015. Nonparametric Multivariate Density Estimation Using Mixtures. Statistics and Computing, 25(2), 349-364.

Weldon, Walter F.R. 1892. Certain Correlated Variations in Crangon vulgaris. Proceedings of the Royal Society of London, 51, 1-21.

Weldon, Walter F.R. 1893. On Certain Correlated Variations in Carcinus maenas. Proceedings of the Royal Society of London, 54, 318-329.

West, Mike. 1992. Modelling with mixtures. Pages 503-524 of: Bernardo, Jose M., Berger, James O., DeGroot, Morris H., and Smith, Adrian F.M. (eds), Bayesian Statistics 4. Oxford, United Kingdom: Oxford University Press