Low sample size and regression: A Monte Carlo approach

Riveros Gavilanes, John Michael (2019): Low sample size and regression: A Monte Carlo approach.

Abstract

This article performs simulations with different small samples considering the regression techniques of OLS, Jackknife, Bootstrap, Lasso and Robust Regression in order to stablish the best approach in terms of lower bias and statistical significance with a pre-specified data generating process -DGP-. The methodology consists of a DGP with 5 variables and 1 constant parameter which was regressed among the simulations with a set of random normally distributed variables considering samples sizes of 6, 10, 20 and 500. Using the expected values discriminated by each sample size, the accuracy of the estimators was calculated in terms of the relative bias for each technique. The results indicate that Jackknife approach is more suitable for lower sample sizes as it was stated by Speed (1994), Bootstrap approach reported to be sensitive to a lower sample size indicating that it might not be suitable for stablish significant relationships in the regressions. The Monte Carlo simulations also reflected that when a significant relationship is found in small samples, this relationship will also tend to remain significant when the sample size is increased.

• Item Type: MPRA Paper
• Language: English
• Keywords: Small sample size; Statistical significance; Regression; Simulations; Bias
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C15 - Statistical Simulation Methods: General; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C19 - Other; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C63 - Computational Techniques ; Simulation Modeling

References

Bühlmann, P., & Van De Geer, S. (2011). Statistics for high-dimensional data: Methods, theory and applications. Berlin: Springer-Verlag.

Bujang, M., Sa’at, N., & Tg Abu Bakar Sidik, T. (2017). Determination of Minimum Sample Size Requirement for Multiple Linear Regression and Analysis of Covariance Based on Experimental and Non-experimental Studies. Epidemiology Biostatistics and Public Health - 2017, Volume 14, Number 3. e12117, 1-9. Obtained from: https://ebph.it/article/download/12117/11431

Colquhoun, D. (2014). An investigation of the false discovery rate and the misinterpretation of p-values. Royal Society Open Science, Vol 1, Issue 3, Nov, N/A.

Faber, J., & Fonseca, L. (2014). How sample size influences research outcomes. Dental Press J Orthod. 2014 July-Aug;19(4), 27-29. Obtained from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4296634/

Forstmeier, W., Wagenmakers, E., & Parker, T. (2017). Detecting and avoiding likely false‐positive findings – a practical guide. Biological Reviews, Vol 92, Issue 4, 1941-1968. Obtained from: https://onlinelibrary.wiley.com/doi/full/10.1111/brv.12315

Holmes Finch, W., & Hernandez Finch, M. (2017). Multivariate Regression with Small Samples: A Comparison of Estimation Methods. General Linear Model Journal, 2017, Vol. 43(1), 16-30. Obtained from: http://www.glmj.org/archives/articles/Finch_v43n1.pdf

Lin, M., Lucas Jr, H., & Shmueli, G. (2013). Research Commentary—Too Big to Fail: Large Samples and the p-Value Problem. Articles in Advance, Information Systems Research, Vol. 24, No. 4, 1-12.

Mason, C. H., & Perreault, W. J. (1991). Collinearity, Power, and Interpretation of Multiple Regression Analysis. Journal of Marketing Research, Vol. 28, No. 3 (Aug., 1991), 268-280. Obtained from: http://www.ecostat.unical.it/Tarsitano/Didattica/Anamul/Papers_ADMD_FC/Collinearity.pdf

Speed, R. (1994). Regression Type Techniques and Small Samples: A Guide to Good Practice. Journal of Marketing Management, Vol 10, 1994, 89-104.

StataCorp. (2019). Stata Lasso Reference Manual Release 16. College Station, Texas: Stata Press Publication.