A new method of robust linear regression analysis: some monte carlo experiments

Mishra, SK (2008): A new method of robust linear regression analysis: some monte carlo experiments.

Abstract

This paper elaborates on the deleterious effects of outliers and corruption of dataset on estimation of linear regression coefficients by the Ordinary Least Squares method. Motivated to ameliorate the estimation procedure, we have introduced the robust regression estimators based on Campbell’s robust covariance estimation method. We have investigated into two possibilities: first, when the weights are obtained strictly as suggested by Campbell and secondly, when weights are assigned in view of the Hampel’s median absolute deviation measure of dispersion. Both types of weights are obtained iteratively. Using these two types of weights, two different types of weighted least squares procedures have been proposed. These procedures are applied to detect outliers in and estimate regression coefficients from some widely used datasets such as stackloss, water salinity, Hawkins-Bradu-Kass, Hertzsprung-Russell Star and pilot-point datasets. It has been observed that Campbell-II in particular detects the outlier data points quite well (although occasionally signaling false positive too as very mild outliers). Subsequently, some Monte Carlo experiments have been carried out to assess the properties of these estimators. Findings of these experiments indicate that for larger number and size of outliers, the Campbell-II procedure outperforms the Campbell-I procedure. Unless perturbations introduced to the dataset are sizably numerous and very large in magnitude, the estimated coefficients by the Campbell-II method are also nearly unbiased. A Fortan Program for the proposed method has also been appended.

• Item Type: MPRA Paper
• Language: English
• Keywords: Robust regression; Campbell's robust covariance; outliers; Stackloss;Water Salinity; Hawkins-Bradu-Kass; Hertzsprung-Russell Star; Pilot-Plant; Dataset;Monte Carlo; Experiment; Fortran Computer Program
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C13 - Estimation: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C14 - Semiparametric and Nonparametric Methods: General; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C63 - Computational Techniques ; Simulation Modeling; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C15 - Statistical Simulation Methods: General; C - Mathematical and Quantitative Methods > C0 - General > C01 - Econometrics

References

• Aitken, A. C. (1935) "On Least Squares and Linear Combinations of Observations", Proceedings of the Royal Society of Edinburgh, 55: 42-48. • Andrews, D.F. (1974) "A Robust Method for Multiple Linear Regression" Technometrics, 16: 523-531. • Brownlee, K.A. (1965) Statistical Theory and Methodology in Science and Engineering, Wiley, New York. • Campbell, N. A. (1980) “Robust Procedures in Multivariate Analysis I: Robust Covariance Estimation”, Applied Statistics, 29 (3): 231-237 • Daniel, C. and Wood, F.S. (1971) Fitting Equations to Data. Wiley, New York. • Hampel, F. R., Ronchetti, E.M., Rousseeuw, P.J. and W. A. Stahel, W.A. (1986) Robust Statistics: The Approach Based on Influence Functions, Wiley, New York. • Hawkins, D.M., Bradu, D., and Kass, G.V. (1984) "Location of Several Outliers in Multiple Regression Using Elemental Sets", Technomenics, 26: 197-208. • Kashyap, R.L and Maiyuran, S. (1993) “Robust Regression and Outlier Set Estimation using Likelihood Reasoning”, Electrical and Computer Engineering ECE Technical Reports, TR-EE 93-8, Purdue University School of Electrical Engineering. http://docs.lib.purdue.edu/ecetr/33/ • Mahalanobis, P. C. (1936) “On the Generalized Distance in Statistics”, Proceedings of the National Institute of Science of India, 12: 49-55. • Plackett, R.L. (1950) "Some Theorems in Least Squares", Biometrika, 37: 149-157 • Rousseeuw, P.J., and Leroy, A.M. (1987), Robust Regression and Outlier Detection, Wiley. New York. • Rupert, D. and Carrol, R.J. (1980) "Trimmed Least Squares Estimation in the Linear Model," Journal of American Statistical Association, 75: 828-838. • Theil, H. (1971) Principles of Econometrics, Wiley, New York.