On the Equivalence of the Weighted Least Squares and the Generalised Least Squares Estimators, with Applications to Kernel Smoothing

Luati, Alessandra and Proietti, Tommaso (2008): On the Equivalence of the Weighted Least Squares and the Generalised Least Squares Estimators, with Applications to Kernel Smoothing.

Abstract

The paper establishes the conditions under which the generalised least squares estimator of the regression parameters is equivalent to the weighted least squares estimator. The equivalence conditions have interesting applications in local polynomial regression and kernel smoothing. Specifically, they enable to derive the optimal kernel associated with a particular covariance structure of the measurement error, where optimality has to be intended in the Gauss-Markov sense. For local polynomial regression it is shown that there is a class of covariance structures, associated with non-invertible moving average processes of given orders which yield the the Epanechnikov and the Henderson kernels as the optimal kernels.

• Item Type: MPRA Paper
• Language: English
• Keywords: Local polynomial regression; Epanechnikov Kernel; Non-invertible Moving average processes
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C13 - Estimation: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C14 - Semiparametric and Nonparametric Methods: General; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C22 - Time-Series Models ; Dynamic Quantile Regressions ; Dynamic Treatment Effect Models ; Diffusion Processes

References

Aitken, A.C. (1935), On the Least Squares and Linear Combinations of Observations, Proceedings of the Royal Society of Edinburgh, A, 55, 42-48.

Amemiya, T. (1985). Advanced Econometrics. Harvard University Press, Cambridge, U.S.

Anderson, T.W. (1948), On the Theory of Testing Serial Correlation, Skandinavisk Aktuarietidskrift, 31, 88-116.

Anderson, T.W. (1971), The Statistical Analysis of Time Series, Wiley.

Baksalary, J.K., Kala, R. (1983), On Equalities Between BLUEs, WLSEs and SLSEs, The Canadian Journal of Statistics, 11, 119-123.

Baksalary, J.K., Van Eijnsbergeren, A.C (1988), A Comparison of Two Criteria For Ordinary-Least-Squares Estimators To Be Best Linear Unbiased Estimators, The American Statistician, 42, 205-208.

Benedetti, J.K. (1977), On the Nonparametric Estimation of Regression Functions, Journal of the Royal Statistical Society, ser. B, 39, 248-253.

Cleveland, W.S. (1979), Robust Locally Weighted Regression and Smoothing Scatterplots, Journal of the American Statistical Association, 64, 368, 829-836.

Epanechnikov V.A. (1969), Nonparametric Estimation of a Multivariate Probability Density, Theory of Probability and Applications, 14, 153-158.

Findley, D.F., Monsell, B.C., Bell, W.R., Otto, M.C., Chen B. (1998). New Capabilities and Methods of the X12-ARIMA Seasonal Adjustment Program, Journal of Business and Economic Statistics, 16, 2.

Grenander, U., Rosenblatt, M. (1957), Statistical analysis of stationary time series, John Wiley and Sons, New York.

Hannan, E.J. (1970), Multiple Time Series, John Wiley and Sons, New York.

Henderson, R. (1916), Note on Graduation by Adjusted Average, Transaction of the Actuarial Society of America, 17, 43-48.

Hoskins, W.D., Ponzo P.J. (1972), Some Properties of a Class of Band Matrices, Mathematics of Computation, 26, 118, 393-400.

Kenny P.B., and Durbin J. (1982), Local Trend Estimation and Seasonal Adjustment of Economic and Social Time Series, Journal of the Royal Statistical Society A, 145, I, 1-41.

Kramer, W. (1980), A Note on the Equality of Ordinary Least Squares and Gauss-Markov Estimates in the General Linear Model, Sankhya, A, 42, 130-131.

Kramer,W. (1986), Least-Squares Regression when the Independent Variable Follows an ARIMA Process, Journal of the American Statistical Association, 81, 150-154.

Kramer, W., Hassler, U. (1998), Limiting Efficiency of OLS vs. GLS When Regressors Are Fractionally Integrated, Economics Letters , 60, 3, 285-290.

Kruskal, W. (1968), When Are Gauss-Markov and Least Squares Estimators Identical? A Coordinate-Free Approach, The Annals of Mathematical Statistics, 39, 70-75.

Jaeger A., Kramer, W. (1998), A Final Twist on the Equality of OLS and GLS, Statistical Papers, 39, 321-324.

Ladiray, D. and Quenneville, B. (2001). Seasonal Adjustment with the X-11 Method (Lecture Notes in Statistics), Springer-Verlag, New York.

Loader, C. (1999), Local regression and likelihood, Springer-Verlag, New York.

Lowerre, J. (1974), Some Relationships Between BLUEs, WLSEs and SLSEs, Journal of the American Statistical Association, 69, 223-225.

Macaulay, F.R. (1931), The Smoothing of Time Series, New York: National Bureau of Economic Research.

Magnus J.R. and Neudecker H. (2007), Matrix Differential Calculus with Applications in Statistics and Econometrics, Third edition, John Wiley & Sons.

McElroy F.W. (1967), A Necessary and Sufficient Condition That Ordinary Least-Squares Estimators Be Best Linear Unbiased, Journal of the American Statistical Association, 62, 1302-1304.

Meyer C.D. (2000), Matrix Analysis and Applied Linear Algebra, SIAM.

Muller, H.G. (1984), Smooth Optimum Kernel Estimators of Densities, Regression Curves and Modes, The Annals of Statistics, 12, 2, 766-774.

Muller, H.G. (1987),Weighted Local Regression and Kernel Methods for Nonparametric Curve Fitting, Journal of the American Statistical Association, 82, 231-238.

Nadaraya, E.A. (1964), On Estimating Regression, Theory of Probability and its Applications, 9, 141-142.

Phillips P. C.B. (1992), Geometry of the Equivalence of OLS and GLS in the Linear Model, Econometric Theory, 8, 1, 158-159.

Phillips P.C.B., Park J.Y. (1992), Asymptotic Equivalence of OLS and GLS in Regressions with Integrated Regressors, Journal of the American Statistical Association, 83, 111-115.

Priestley, M.B., Chao M.T. (1972), Nonparametric Function Fitting, Journal of the Royal Statistical Society, ser. B, 34, 384-392.

Puntanten S., Styan, G.P.H. (1989), The Equality of the Ordinary Least Squares Estimator and the Best Linear Unbiased Estimator, The American Statistician, 43, 3, 153-161.

Wallis, K. (1983). Models for X-11 and X-11 Forecast Procedures for Preliminary and Revised Seasonal Adjustments. In Applied Time Series Analysis of Economic Data (A. Zellner, ed.), pp. 3-11. Washington DC: Bureau of the Census, 1983.

Wand M.P. and Jones M.C. (1995), Kernel Smoothing, Monographs on Statistics and Applied Probability, 60, Chapman&Hall.

Watson, G.S. (1967), Linear Least Squares Regression, The Annals of Mathematical Statistics, 38, 1679-1699.

Watson, G.S. (1964), Smooth Regression Analysis, Sankhy¹a, A, 26, 359-372.

Zyskind, G. (1967), On Canonical Forms, Non-Negative Covariance Matrices and Best and Simple Least Squares Linear Estimators in Linear Models, The Annals of Mathematical Statistics, 38, 1092-1119.

Zyskind, G. (1969), Parametric Argumentations and Error Structures Under Which Certain Simple Least Squares And Analysis of Variance Procedures Are Also Best, Journal of the American Statistical Association, 64, 1353-1368.

Zyskind, G., Martin, F.B. (1969), On Best Linear Estimation and a General Gauss-Markoff Theorem in Linear Models with Arbitrary Non-negative Structure, SIAM Journal of Applied Mathematics, 17, 1190-1202.