Explaining (in)efficiency in higher education: a comparison of parametric and non-parametric analyses to rank universities

Barra, Cristian and Lagravinese, Raffaele and Zotti, Roberto (2015): Explaining (in)efficiency in higher education: a comparison of parametric and non-parametric analyses to rank universities.

Abstract

In recent years more and more numerous are the rankings published in the newspapers or technical reports available, covering many aspects of higher education, but in many cases with very conflicting results between them, due to the fact that universities’ performances depend on the set of variables considered and on the methods of analysis employed. The aim of this study is to rank higher education institutions (HEIs) in Italy, comparing parametric and non-parametric approaches: we firstly apply a so-called double bootstrap Data Envelopment Analysis (DEA) to generate unbiased coefficients (Simar and Wilson, 2007) and then a Stochastic Frontier Analysis (SFA), modelling the production set through an output distance function, applying a within transformation to data as developed by Wang and Ho (2010), to evaluate which determinants have an impact on universities’ efficiencies. The findings reveal that, on average and among the macro-areas of the country, the level of efficiency does not change significantly among estimation methods which, instead, generate different rankings. This may guide universities’ managers and policymakers as rankings have a strong impact on academic decision-making and behaviour, on the structure of the institutions and also on students and graduates recruiters. Variables describing institution, market place and environment have an important role in explaining (in)efficiency.

• Item Type: MPRA Paper
• Language: English
• Keywords: Universities; Efficiency; Data Envelopment Analysis; Stochastic Frontier Analysis
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C14 - Semiparametric and Nonparametric Methods: General; C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling > C67 - Input-Output Models; I - Health, Education, and Welfare > I2 - Education and Research Institutions > I21 - Analysis of Education; I - Health, Education, and Welfare > I2 - Education and Research Institutions > I23 - Higher Education ; Research Institutions

References

Abbot, M. and Doucouliagos, C. (2003). The efficiency of Australian universities: a data envelopment analysis. Economics of Education Review, 22, 89-97.

Agasisti, T. (2009). Market forces and competition in university systems: theoretical reflections and empirical evidence from Italy. International Review of Applied Economics, 23 (4), 463–483.

Agasisti, T. (2011). Performances and spending efficiency in Higher Education: a European comparison through non-parametric approaches. Education Economics, 19 (2), 199-224.

Agasisti, T. and Dal Bianco, A. (2009). Reforming the university sector: effects on teaching efficiency. Evidence from Italy. Higher Education, 57 (4), 477-498.

Agasisti, T. and Johnes, G. (2009). Beyond frontiers: comparing the efficiency of higher education decision-making units across more than one country. Education Economics, 17(1), 59–79.

Agasisti, T. and Johnes, G. (2010). Heterogeneity and the evaluation of efficiency: the case of Italian universities. Applied Economics, 42 (11), 1365-1375.

Agasisti, T. and Wolszczak-Derlacz, J. (2014). Exploring universities’ efficiency differentials between countries in a multi-year perspective: an application of bootstrap DEA and Malmquist index to Italy and Poland, 2001-2011. IRLE Working Paper #113/14.

Agasisti, T., Catalano, G., Landoni, P. and Verganti, R. (2012). Evaluating the performance of academic departments: an analysis of research related output efficiency. Research Evaluation, 21 (1), 2-14.

Aigner, D., Lovell, C. K. and Schmidt, P. (1977). Formulation and estimation of stochastic frontier production function models. Journal of Econometrics, 6 (1), 21-37.

Alvarez, A., Amsler, C., Orea, L. and Schmidt, P. (2006). Interpreting and testing the scaling property in models where inefficiency depends on firm characteristics, Journal of Productivity Analysis, 25, 201-212.

Anderson, T. R., Daim, T. U. and Lavoie, F. F. (2007). Measuring the efficiency of university technology transfer, Technovation, 27, 306–18.

Arulampalam, W., Naylor, R., Robin, A. and Smith, J.P. (2004). Hazard model of the probability of medical school drop-out in the UK. Journal of the Royal Statistical Society, Series A 167 (1), 157-178.

Athanassopoulos, A.D. and Shale, E. (1997). Assessing the Comparative Efficiency of Higher Education Institutions in the UK by Means of Data Envelopment Analysis. Education Economics 5, (2), 117-134.

Battese, G.E. and Coelli, T.J. (1995). A model for technical inefficiency effects in a stochastic frontier production function for panel data. Empirical Economics, 20 (2), 325-332.

Battese, G.E., and G. Corra. (1977). Estimation of a production frontier model: with application to the pastoral zone of Eastern Australia. Australian Journal of Agricultural Economics, 21, 169-179.

Bini, M. and Chiandotto, B. (2003). La valutazione del sistema universitario italiano alla luce della riforma dei cicli e degli ordinamenti didattici. Studi e note di economia, 2, 29-61.

Boero, G., Mcnight A., Naylor, R. and Smith J. (2001). Graduates and graduate labour markets in the UK and Italy. Lavoro e Relazioni industriali, 2, 131-172.

Bonaccorsi, A., Daraio, C. and Simar, L. (2006). Advanced indicators of productivity of universities. An application of robust nonparametric methods to Italian data. Scientometrics, 66 (2), 389–410.

Brennan, S., Haelermans, C., and Ruggiero, J. (2014). Non parametric estimation of education productivity incorporating nondiscretionary inputs with an application to Dutch schools. European Journal of Operational Research, 234 (3), 809-818.

Buzzigoli, L., Giusti, A. and Viviani, A. (2010). The evaluation of university departments. A case study for Firenze. International Advances in Economic Research, 16, 24–38.

Carrington, R., Coelli, T. and Rao, D.S.P. (2005). The performance of Australian universities: conceptual issues and preliminary results. Economic Papers 24, 145-163.

Catalano, G., Mori, A., Silvestri, P., and Todeschini, P., (1993). Chi paga l’istruzione universitaria? Dall’esperienza europea una nuova politica di sostegno agli studenti in Italia, Franco Angeli, Milano.

Cave, M., Hanney, S. and Kogan, M. (1991). The use of performance indicators in higher education. A critical analysis of developing practices, London, Jessica Kingsley.

Chapple, W., Lockett, A., Siegel, D. S. and Wright, M. (2005). Assessing the relative performance of university technology transfer office in UK: parametric and non-parametric evidence, Research Policy, vol. 34, no. 3, 369–84.

Clarke, M. (2007). The impact of higher education rankings on student access, choice, and opportunity. Higher Education in Europe, 32 (1), 59–70. Coco, G. and Lagravinese, R. (2014). Cronyism and education performance. Economic Modelling, 38, 443-450.

Coelli, T. and Perelman, S. (2000). Technical efficiency of European railways, a distance function approach. Applied Economics, 32, 1967-1976.

Coelli, T. J. (2000). On the econometric estimation of the distance function representation of a production technology. Working Paper, 2000/042.

Curi, C., Daraio, C., and Llerena, P. (2012). University technology transfer: how (in)efficient are French universities? Cambridge Journal of Economics, 36, 629–654.

D’Este, P., Iammarino, S. (2010). The spatial profile of university-business research partnerships. Papers in Regional Science, 89: 335–350.

De Groot, H., McMahon, W.W. and Volkwein, J.F. (1991). The cost structure of American research universities. Review of Economics and Statistics, 73 (3), 424-431.

De Witte, K. and Hudrlikova, L. (2013). What about excellence in teaching? A benevolent ranking of universities. Scientometrics, 96, 337–364.

Del Barrio-Castro, T. and Garcia-Quevedo, J. (2005). Effects of University Research on the Geography of Innovation. Regional Studies, 39 (9), 1217-1229.

Desjardins, S.L., Ahlburg, D.A. and Mccall, B.P. (2002). A temporal investigation of factors related to timely degree completion. The Journal of Higher Education, 73 (5), 555-581.

Donina, D., Meoli, M., and Paleari, S. (2015). Higher education reform in Italy: Tightening regulation instead of steering at a distance. Higher Education Policy, 28(2), 215-234.

Dyson, R.G. (2000). Strategy, performance and operational research. Journal of Operational Research, 51, 5-11.

Emrouznejad, A. and Thanassoulis, E. (2005). A Mathematical Model for Dynamic Efficiency Using Data Envelopment Analysis. Applied Mathematics and Computation, 160, 363–378.

Etzkowitz, H. (2003). Research groups as ‘quasi-firms’: the invention of the entrepreneurial university. Research Policy 32 (2003) 109-121.

Florax, R.G.M. (1992). The university: A regional booster? Economic impacts of academic knowledge infrastructure. Avebury, Aldershot.

Florida, R., Mellander, C. and Stolarick, K. (2008). Inside the Black Box of Regional Development Human Capital, the Creative Class, and Tolerance. Journal of Economic Geography, 8 (5), 615-649.

Frey, B.S. and Rost, K. (2010). Do rankings reflect research quality? Journal of Applied Economics, 13, (1), 1-38.

Goldstein, H. A. and Renault, C. S. (2004). Contributions of universities to regional economic development: A quasi-experimental approach. Regional Studies, 38 (7), 733-746.

Greene, W. (2005). Reconsidering heterogeneity in panel data estimators of the stochastic frontier model. Journal of Econometrics, 126 (2), 269-303.

Haelermans, C., and Ruggiero, J. (2013). Estimating technical and allocative efficiency in the public sector: A nonparametric analysis of Dutch schools. European Journal of Operational Research, 227 (1), 174-181.

Halkos, G., Tzeremes, N.G. and Kourtzidis, S.A., (2012). Measuring public owned university departments’ efficiency: a bootstrapped DEA approach. Journal of Economics and Econometrics, 55 (2): 1-24.

Harvey, L. (2008). Assaying improvement, paper presented at the 30th EAIR Forum, Copenhagen, Denmark, 24–27 August.

Hashimoto, A. and Haneda, S., (2008). Measuring the change in R&D efficiency of the Japanese pharmaceutical industry. Research Policy, 37, 1829–1836.

Hazelkorn, E. (2007). The impact of league tables and ranking systems on higher education decision making. Higher Education Management and Policy, 19, 2.

Johnes, G. and Johnes, J. (1993). Measuring the Research Performance of UK Economics Departments: an application of data envelopment analysis. Oxford Economic Papers, 45, 332–347.

Johnes, J. (1996). Performance assessment in higher education in Britain. European Journal of Operational Research 89, 18–33.

Johnes, J. (2006). Measuring teaching efficiency in higher education: An application of data development analysis to economics graduates from UK Universities 1993. European Journal of Operational research, 174 (1): 443-456.

Johnes, J. (2014). Efficiency and mergers in English higher education 1996/97 to 2008/9: parametric and non-parametric estimation of the multi-input multi-output distance function. The Manchester School, 82, 465–487.

Johnes, J. and Yu, L. (2008). Measuring the research performance of Chinese higher education institutions using data envelopment analysis. China Economic Review, 19(4), 679-696.

Kao, C. and Hung, H.T. (2008). Efficiency analysis of university departments: an empirical study. Omega, 36, 653-664.

Kempkes, G., and Pohl, C. (2010). The efficiency of German universities–some evidence from nonparametric and parametric methods. Applied Economics,42 (16), 2063-2079.

Kraus, M. (2004) Schätzung von Kostenfunktionenfür die bundesdeutsche Hochschulausbil-dung: Einkonzeptioneller Ansatzimempirischen Test, ZEW Discussion Paper 4/36.

Kumbhakar, S.C. and Lovell, C.K. (2003). Stochastic frontier analysis. Cambridge University Press.

Lassibille, G. (2011). Student progress in higher education: what we have learned from large-scale studies. The Open Education Journal, 4, 1-8.

Laureti, T. (2008). Modelling exogenous variables in human capital formation through a heteroscedastic stochastic frontier. International Advances in Economic Research, 14 (1), pp. 76–89.

Laursen, K., Reichstein, T., and Salter, A. (2011). Exploring the effect of geographical proximity and university quality on university–industry collaboration in the United Kingdom. Regional studies, 45(4), 507-523.

Lewin, A.Y. and Lovell, C.A.K. (1990). Frontier analysis: parametric and nonparametric approaches. Journal of Econometrics, 46, 1-245.

Lovell, C. A. K., Richardson, S., Travers, P. and Wood, L. L. (1994), Resources and functionings: a new view of inequality in Australia, in Models and Measurement of Welfare and Inequality (Ed.) W. Eichhorn, Springer-Verlag, Berlin, pp. 787-807.

Madden, G., Savage, S. and Kemp, S. (1997). Measuring public sector efficiency: a study of Economics departments at Australian universities. Education Economics, 5 (2), 153–167.

Meeusen, W. and Van den Broeck, J. (1977). Efficiency estimation from Cobb-Douglas production functions with composed error. International Economic Review, 435-444.

Mowery, D. C., and Ziedonis, A. A. (2015). Markets versus spillovers in outflows of university research. Research Policy, 44(1), 50-66. OECD (2008). Tertiary Education for the Knowledge Society, OECD Publishing, Paris. Available at www.oecd.org/edu/tertiary/review.

Potì, B. and Reale, E. (2005). Changing patterns in the steering of the University in Italy: funding rules and doctoral programmes. Working Paper No. 11/2005, Ceris – CNR.

Restrepo-Tobón, D.A. and Kumbhakar, S.C. (2013). Profit efficiency of U.S. commercial banks: a decomposition, Documentos de Trabajo CIEF 010939, Universidad Eafit.

Shubert, T. and Kroll, H. (2014). Universities’ effects on regional GDP and unemployment: The case of Germany. Regional Science, DOI: 10.1111/pirs.12150.

Siegel, D., Wright, M., Chapple, W. and Lockett, A. (2008). Assessing the relative performance of university technology transfer in the US and UK: a stochastic distance function approach, Economics of Innovation and New Technology, vol. 17, 717–29.

Simar, L., and Wilson, P.W. (2007). Estimation and inference in two-stage, semi-parametric models of production processes. J. Econ. 136, 31–64.

Smith, J. and Naylor, R. (2001). Determinants of degree performance in UK universities: A statistical analysis of the 1993 student cohort. Oxford Bulletin of Economics and Statistics, 63, 29–60.

Thursby, J. G. and Kemp, S. (2002). Growth and productive efficiency of university intellectual property licensing, Research Policy, 31, 109–24.

Tofallis, C. (2012). A different approach to university rankings. Higher Education, 63 (1), 1–18.

Tomkins, C. and Green R. (1988). An experiment in the use of data envelopment analysis for evaluating the efficiency of UK university departments of accounting. Financial Accountability & Management, 4, 147–64.

Tyagi, P., Yadav, S.P. and Singh, S.P. (2009). Relative performance of academic departments using DEA with sensitivity analysis. Evaluation and Program Planning, 32: 168-177.

Wang, H.J. and Ho, C.W. (2010). Estimating fixed-effect panel stochastic frontier models by model transformation. Journal of Econometrics, 157 (2), 286-296.

Wang, H.J. and Schmidt, P. (2002). One-step and two-step estimation of the effects of exogenous variables on technical efficiency levels. Journal of Productivity Analysis, 18, 129-144.

Wilson, P.W. (2008). FEAR 1.0: A Software Package for Frontier Efficiency Analysis with R. Socio-Economic Planning Sciences, 42, 247-254.

Wolszczak-Derlacz, J., Parteka, A. (2011). Efficiency of European public higher education institutions: a two-stage multi-country approach. Scientometrics 89, 887–917.

Worthington, A. and Lee, B.L. (2008). Efficiency, technology and productivity change in Australian universities, 1998-2003. Economics of Education Review, 27, 285-298.