Computer Simulates the Effect of Internal Restriction on Residuals in Linear Regression Model with First-order Autoregressive Procedures

Lee, Mei-Yu (2014): Computer Simulates the Effect of Internal Restriction on Residuals in Linear Regression Model with First-order Autoregressive Procedures. Published in: Computer Simulates the Effect of Internal Restriction on Residuals in Linear Regression Model with First-order Autoregressive Procedures , Vol. 3, No. 3 (October 2014): pp. 1-22.

Abstract

This paper demonstrates the impact of particular factors – such as a non-normal error distribution, constraints of the residuals, sample size, the multi-collinear values of independent variables and the autocorrelation coefficient – on the distributions of errors and residuals. This explains how residuals increasingly tend to a normal distribution with increased linear constraints on residuals from the linear regression analysis method. Furthermore, reduced linear requirements cause the shape of the error distribution to be more clearly shown on the residuals. We find that if the errors follow a normal distribution, then the residuals do as well. However, if the errors follow a U-quadratic distribution, then the residuals have a mixture of the error distribution and a normal distribution due to the interaction of linear requirements and sample size. Thus, increasing the constraints on the residual from more independent variables causes the residuals to follow a normal distribution, leading to a poor estimator in the case where errors have a non-normal distribution. Only when the sample size is large enough to eliminate the effects of these linear requirements and multi-collinearity can the residuals be viewed as an estimator of the errors.

• Item Type: MPRA Paper
• Language: English
• Keywords: Time series; Autoregressive model; Computer simulation; Non-normal distribution
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C15 - Statistical Simulation Methods: General; C - Mathematical and Quantitative Methods > C3 - Multiple or Simultaneous Equation Models ; Multiple Variables > C32 - Time-Series Models ; Dynamic Quantile Regressions ; Dynamic Treatment Effect Models ; Diffusion Processes ; State Space Models; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C63 - Computational Techniques ; Simulation Modeling

References

[1] B.H. Baltagi, Econometrics, Fifth edition, Springer: New York, 2011.

[2] C.F. Roos, Annual Survey of Statistical Techniques: The Correlation and Analysis of Time Series--Part II, Econometrica, 4(4), (1936), 368-381.

[3] G.E. Box, D.A. Pierce, Distribution of Residual Autocorrelations in Autoregressive-integrated Moving Average Time Series Models, Journal of the American Statistical Association, 65(332), (1970), 1509-1526.

[4] G.U. Yule, On the Time-Correlation Problem, with Especial Reference to the Variate-Difference Correlation Method, Journal of the Royal Statistical Society, 84(4), (1921), 497-537.

[5] J. Durbin, G.S. Watson, Testing for Serial Correlation in Least Squares Regression: I, Biometrika, 37(3/4), (1950), 409-428.

[6] J. Durbin, G.S. Watson, Testing for Serial Correlation in Least Squares Regression. II, Biometrika, 38(1/2), (1951), 159-177.

[7] M.Y. Lee, The Pattern of R-Square in Linear Regression Model with First-Order Autoregressive Error Process and Bayesian property: Computer Simulation, Journal of Accounting & Finance Management Strategy, 9(1), (2014a).

[8] M.Y. Lee, Limiting Property of Durbin-Watson Test Statistic, manuscript, (2014b).

[9] M.Y. Lee, The Conflict of Residual and Error Simulated in Linear Regression Model with AR(1) Error Process, manuscript, (2014c).

[10] T.S. Breusch, L.G. Godfrey, A review of recent work on testing for autocorrelation in dynamic economic models," University of Southampton, (1980).