Finite sample performance of kernel-based regression methods for non-parametric additive models under common bandwidth selection criterion

Martins-Filho, Carlos and yang, ke (2007): Finite sample performance of kernel-based regression methods for non-parametric additive models under common bandwidth selection criterion. Published in: Journal of Nonparametric Statistics , Vol. 19, No. 1 (2007): pp. 23-62.

Abstract

In this paper, we investigate the finite sample performance of four kernel-based estimators that are currently available for additive non-parametric regression models—the classic backfitting estimator (CBE), the smooth backfitting estimator, the marginal integration estimator, and two versions of a two-stage estimator of which the first is proposed by Kim, Linton and Hengartner (1999) and the second is proposed in this paper. The bandwidths are selected for each estimator by minimizing their respective asymptotic approximation of the mean average squared errors. In our simulations, we are particularly concerned with the performance of these estimators under this unified data-driven bandwidth selection method, since in this case both the asymptotic and the finite sample properties of all estimators are currently unavailable. The comparison is based on the estimators' average squared error. Our Monte Carlo results seem to suggest that the CBE is the best performing kernel-based procedure.

• Item Type: MPRA Paper
• Language: English
• Keywords: Additive non-parametric regression, Local linear estimation, Backfitting estimation, Smooth backfitting, Marginal integration
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C14 - Semiparametric and Nonparametric Methods: General

References

1. Barr, D.R. and N.L. Slezak, 1972, A comparison of Multivariate normal generators, Communications of the Association for Computing Machinery, 15, 1048-1049.

2. Buja, A., Hastie, T.J. and R.J. Tibshirani, 1989, Linear smoothers and additive models, Annals of Statistics, 17, 453-555.

3. Dennis, Jr., J.E., and R. Schnabel, 1983, Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, Englewood Cliffs, NJ.

4. Dette, H., C. V. Lieres undWilkau, S. Sperlich, 2005, A comparison of different nonparametric methods for inference on additive models, Journal of Nonparametric Statistics, 17 (1), 57-81.

5. Friedman, J.H. and W. Stuetzel, 1981, Projection pursuit regression, Journal of the American Statistical Association, 76, 817-823.

6. Fan, J. and S.J. Marron, 1994, Fast implementation of nonparametric curve estimators, Journal of Computational and Graphical Statistics, 3, 35-56.

7. Fushimi, M., 1990, Random number generation with the recursion Xt = Xt−3p � Xt−3q, Journal of Computational and Applied Mathematics, 31, 105-118.

8. Graybill, F.A., 1969, Introduction to matrices with applications in statistics, Wadsworth: Belmont, CA.

9. Horowitz, J. L. and Mammen, E., 2005, Oracle-efficient nonparametric estimation of an additive model with an unknown link function, working paper, Department of Economics, Northwestern University.

10. H¨ardle, W. and O. Linton, 1994, Applied nonparametric methods, in Robert Engle and Daniel McFadden eds. Handbook of Econometrics, v.4 . North Holland: Amsterdam.

11. Hastie, T.J. and R.J. Tibshirani, 1986, Generalized additive models, Statistical Science, 1, 297-318.

12. Hastie, T.J. and R.J. Tibshirani, 1990, Generalized additive models, Chapman and Hall: New York.

13. Jones, M.C., S.J. Davies and B.U. Park, 1994, Versions of kernel type regression estimators, Journal of the American Statistical Association, 89, 825-832.

14. Kennedy, W. and J. Gentle, 1980, Statistical computing, Marcel Dekker: New York.

15. Kim, W., O. Linton and N. Hengartner, 1999, A computationally efficient oracle estimator for additive nonparametric regression with bootstrap confidence intervals, Journal of Computational and Graphical Statistics, 8, 278-297.

16. Kinderman, A. and J. Ramage, 1976, Computer generation of normal random variables, Journal of the American Statistical Association, 71, 893-896.

17. Linton, O., 1997, Efficient estimation of additive nonparametric regression models, Biometrika, 84,469-474.

18. Linton, O. andW. H¨ardle, 1996, Estimation of additive regression models with known links, Biometrika, 83, 529-540.

19. Linton, O. and J.P. Nielsen, 1995, A Kernel method of estimating structured nonparametric regression based on marginal integration, Biometrika, 82, 93-100.

20. Mammen, E., Linton, O. and Nielsen J., 1999, The existence and asymptotic properties of a backfitting projection algorithm under weak conditions, The Annals of Statistics, 27, 1443-1490

21. Mammen, E., Marron, J.S., Turlach, B.A. and Wand, M.P., 2001, A general projection framework for constrained smoothing, Statistical Science, 16, No. 3, 232-248

22. Mammen, E. and Park, B.U., 2005, Bandwidth selection for smooth backfitting in additive models, The Annals of Statistics, 33, 1260-1294

23. Marsaglia, G., 1968, Random numbers fall mainly in the planes, Proceedings of the National Academy of Sciences, 61, 25-28.

24. Marsaglia, G., 1985, A current view of random number generators, in L. Billard editor Computer Science and Statistics: Proceedings of the 16th Symposium on the Interface, North Holland: Amsterdam.

25. Nielsen, J. P. and Sperlich, S, 2005, Smooth backfitting in practice, Journal of Royal Statistical Society, Series B, 67, 43-61

26. Newey, W., 1994, Kernel estimation of partial means, Econometric Theory, 10, 233-253.

27. Opsomer, J., 2000, Asymptotic properties of backfitting estimators, Journal of Multivariate Analysis, 73, 166-179.

28. Opsomer, J. and D. Ruppert, 1997, Fitting a bivariate additive model by local polynomial regression, Annals of Statistics, 25, 186-211.

29. Opsomer, J. and D. Ruppert, 1998, A fully automated bandwidth selection method for fitting additive models, Journal of the American Statistical Association, 93, 605-619.

30. Pagan, A. and A. Ullah, 1999, Nonparametric econometrics. Cambridge: Cambridge University Press.

31. Park, B. and J.S. Marron, 1990, Comparison of data-driven bandwidth selectors, Journal of the American Statistical Association, 85, 66-72.

32. Opsomer, J., and D. Ruppert, 1997, An effective bandwidth selector for least squares regression, Annals of Statistics, 25, 186-211.

33. Ruppert, D., S.J. Sheather, and M.P. Wand, 1995, An effective bandwidth selector for least squares regression, Journal of the American Statistical Association, 90, 1257-1270.

34. Sheather, S.J. and M.C. Jones, 1991, A reliable data-based bandwidth selection method for kernel density estimation, Journal of the Royal Statistical Society B, 53, 683-690.

35. Silverman, B.W., 1986, Density estimation for statistics and data analysis, Chapman and Hall: New York.

36. Simonoff, J.S., 1996, Smoothing methods in statistics, Springer: New York.

37. Sperlich, S., O. Linton, W. H¨ardle, 1999, Integration and backfitting methods in additive models - finite sample properties and comparison, TEST, 8, 1-39.

38. Staniswalis, J.G. and Severini, T.A, 2000, Fitting the additive model by recursion on dimension, Communications in Statistics, 29, 689-701.

39. Stone, C.,1980, Optimal rates of convergence for nonparametric estimators, Annals of Statistics, 8, 1348-1360.

40. Stone, C.,1985, Additive regression and other nonparametric models, Annals of Statistics, 6, 689-705.

41. Tjøstheim, D. and B. Auestad, 1994, Nonparametric identification of nonlinear time series projections, Journal of the American Statistical Association, 89, 1398-1409.

42. Wahba, G., 1990, Spline models for observational data. SIAM: Philadelphia, PA.