Arpino, Bruno and Varriale, Roberta (2009): Assessing the quality of institutions’ rankings obtained through multilevel linear regression models.
Download (230kB) | Preview
The aim of this paper is to assess the quality of the ranking of institutions obtained with multilevel techniques in presence of different model misspecifications and data structures. Through a Monte Carlo simulation study, we find that it is quite hard to obtain a reliable ranking of the whole effectiveness distribution while, under various experimental conditions, it is possible to identify institutions with extreme performances. Ranking quality increases with increasing intra class correlation coefficient and/or overall sample size. Furthermore, multilevel models where the between and within cluster components of first-level covariates are distinguished perform significantly better than both multilevel models where the two effects are set to be equal and the fixed effect models.
|Item Type:||MPRA Paper|
|Original Title:||Assessing the quality of institutions’ rankings obtained through multilevel linear regression models|
|Keywords:||Multilevel models, ranking of institutions, second-level residuals distribution|
|Subjects:||I - Health, Education, and Welfare > I2 - Education and Research Insititutions
C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C15 - Statistical Simulation Methods: General
|Depositing User:||Bruno Arpino|
|Date Deposited:||11. Jan 2010 01:51|
|Last Modified:||16. Feb 2013 02:42|
Aassve, A. & Arpino, B. (2007). Dynamic Multi-Level Analysis of Households' Living Standards and Poverty: Evidence from Vietnam. Working Paper of Institute for Social and Economic Research, paper 2007-10. Colchester: University of Essex.
Adab, P., Rouse. A., Mohammed, M.A., & Marshall. T. (2002). Performance league tables: the NHS deserves better. BMJ, 324, 95–98.
Aitkin, M., & Longford, N. (1986). Statistical modelling in school effectiveness studies (with discussion). Journal of Royal Statistical Society A, 149, 1-42.
Chiandotto, C. & Varriale, R. (2005). Un modello multilivello per l’analisi dei tempi di conseguimento del titolo nell’Ateneo fiorentino, in C. Crocetta (Ed.), Modelli di analisi della transizione università-lavoro. Vol.7, Cleup, Padova.
Downey, D.B., von Hippel, P.T., &Hughes, M. (2008). Are “failing” schools really failing? Using seasonal comparison to evaluate school effectiveness. Sociology of Education, 81(3), 242-270
Goldstein, H. & Healy, M.J.R. (1995). The graphical presentation of a collection of means. Journal of the Royal Statistical Society, 158, 175-177.
Goldstein, H. & Spiegelhalter, D.J. (1996). League tables and their limitations: statistical issues in comparisons of institutional performance. Journal of the Royal Statistical Society: Series A, 159, 385-443.
Goldstein, H., Rasbash, J., Yang, M., Woodhouse, G., Pan, H., Nuttall, D., & Thomas, S. (1993). A multilevel analysis of school examination results. Oxford Review of Education, 19, 425-433.
Jürges, H, & Sneider, K. (2007). Fair ranking of teachers. Empirical Economics, 32, 411-431.
Ladd, H. F., & Walsh, R.P. (2002). Implementing value-added measures of school effectiveness: getting the incentives right. Economics of Education Review, 21, 1-17.
Leckie, G. & Goldstein, H. (2009). The Limitations of Using School League Tables to Inform School Choice. The Centre for Market and Public Organisation 09/208, Department of Economics, University of Bristol, UK.
Maas, C.J.M. and Hox. J.J. (2004). Robustness issues in multilevel regression analysis. Statistica Neerlandica, 58(2), 127–137.
Maddala, G.S. (1971). The use of variance components models in pooling cross section and time series data. Econometrica, 39 (2), 341-358.
Neuhaus, J.M. & Kalbfleisch, J.D. (1998). Between- and within-cluster covariate effects in the analysis of clustered data. Biometrics, 54, 638-645.
Rabe-Hesketh, S. & Skrondal, A. (2005). Multilevel and longitudinal modeling using Stata. Stata Press, College Station, TX.
Rampichini, C., Grilli, L., & Petrucci, A. (2004). Analysis of university course evaluations: from descriptive measures to multilevel models. Statistical Methods and Applications, 13(3), 357-373.
Raudenbush, S.W., & Willis, J.D. (1995). The estimation of school effects. Journal of Educational and Behavioral Statistics, 20, 307-335.
Shaw, C.D. & Costain, D.W. (1995). League tables for health care. Journal of the Royal Society of Medicine, 88, 54-57.
Skrondal, A. & Rabe-Hesketh, S. (2004). Generalized Latent Variable Modelling: Multilevel, Longitudinal and Structural Equation Models. Boca Raton, FL: Chapman & Hall/CRC.
Snijders, T., & Bosker, R.(1999). An Introduction to Basic and Advanced MultileveModeling. Sage, London.
Tekwe, C.D., Carter, R.L., Ma, C., Algina, J. Lucas, M.E., Roth, J. Ariet, M., Fisher, T., &
Resnick, M.B. (2004). An empirical comparison of statistical models for value-added assessment of school performance. Journal of Educational and Behavioral Statistics, 29 (1), 11-35
Wößmann, L. (2008). Efficiency and equity of European education and training policies. International Tax and Public Finance, 15, 199–230