University departments evaluation: a multivariate approach

Monacciani, Fabiana (2010): University departments evaluation: a multivariate approach.

Abstract

Aim of the paper is to present a new model, based on multivariate statistic analyses, allowing to express a synthetic judgement on Departments activities by taking into consideration the whole set of indicators describing them both as aggregations of researchers and as University autonomous organs. The model, based on Principal Component Analysis and Cluster Analysis, allows both to explain the determinants of Departments performances, and to classify them into homogeneous groups. The paper shows the results obtained by testing the proposed model on University of Naples “L’Orientale” Departments, using data extracted by the 2007 assessment report to the Ministry of University and Research.

• Item Type: MPRA Paper
• Language: English
• Keywords: Evaluation, Departments, Multivariate statistics
• Subjects: D - Microeconomics > D0 - General; I - Health, Education, and Welfare > I2 - Education and Research Institutions > I23 - Higher Education ; Research Institutions; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C19 - Other

References

Adler R., Ewing J., Taylor P. 2008. Citation Statistics, Report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS), 6/11/2008.

CSS - Consiglio Italiano per le Scienze Sociali 2006. La valutazione della ricerca. Libro bianco. Venezia, Italy: Marsilio,.

Cugini A., Michelon G. 2009. La misurazione della performance nei Dipartimenti universitari. Economia & Management, n° 4 (July/August).

Dehon C, McCathie A, Verardi V. 2009. Uncovering Excellence in Academic Rankings. A case study on the Shanghai Ranking. Working Paper Mimeo.

Dubini P. 2005. Misurare la competitività dei sistemi Paese: il modello del World Economic Forum. Economia e Management, n°6.

EUA - European University Association 2009. Institutional Evaluation Programme: Guidelines for participating institutions. Belgium, Brussels: EUA Editions.

European Commission, Enterprise Directorate-General 2009, European Innovation Scoreboard (EIS). PRO INNO Europe Paper, n° 15, Bruxelles, Belgium.

Figà Talamanca A. 2009, E' lecito usare l'Impact Factor per valutare la qualità della ricerca?, relation presented to the conference The problem of evaluation in Social Sciences, Bergamo University, 25 November 2009, Bergamo, Italy.

Frey B.S., Rost K. 2008. Do Rankings Reflect Research Quality?, WP n. 390, Institute for Empirical Research in Economics, University of Zurich.

Gallo M. 2007. The scaling problems in service quality evaluation. Metodološki Zvezki, vol. 2, 165-176.

Gherghi M., Lauro C. 2004. Appunti di Analisi dei Dati Multidimensionali. Metodologia ed esempi. Naples, Italy: RCE Edizioni,.

Guan, J.; Wang, J. 2004. Evaluation and interpretation of knowledge production efficiency. Scientometric, n° 59, 131-155.

Hicks D. 2007. The role of quantitative research evaluation in developing international research excellence, relation presented to the conference organized by the Italian Society of Economists (SIE) on Research Evaluation: experiences and methods, Ancona University, 2-3 April 2007, Ancona, Italy.

Hirsch E.J. 2005, An index to quantify an individual's scientific research output, PNAS, vol. 102, n° 46, 16569-16572.

King D.A. 2004, The scientific impact of nations, Nature, 430, 15 July 2004.

Lippi M., Peracchi F. 2007. Il primo esercizio italiano di valutazione della ricerca: una prima valutazione, relation presented to the conference organized by the Italian Society of Economists (SIE) on Research Evaluation: experiences and methods, Ancona University, 2-3 April 2007, Ancona, Italy.

Liu N.C., Cheng Y. 2005. Academic Ranking of World Universities - Methodologies and Problems, Higher Education in Europe, 30, n°2.

Macerata University 2009. Se questa vi sembra una valutazione, Macerata, Italy: EUM - Edizioni Università di Macerata.

Marcuzzo M.C., Zacchia G. 2007, L’econlit e gli strumenti per la valutazione della ricerca economica in Italia. Rivista italiana per gli economisti, n°. 2., 279-310.

MIUR 1998. Indicazioni per la preparazione delle relazioni dei Nuclei di valutazione interna e insieme minimo di indicatori, doc 11/98.

MIUR 2009. D.M. 23 settembre 2009 prot. 45/2009, Decreto criteri di ripartizione del Fondo di Finanziamento Ordinario (FFO) delle Università per l'anno 2009.

MIUR, CUN 2010. Valutazione della ricerca, Prot. 612, 30/3/2010.

Moed, H.F. 2008. UK Research Assessment Exercises: Informed judgments on research quality or quantity?. Scientometrics, n° 74, 141-149.

Narin, F.; Hamilton, K.S. 1996. Bibliometric performance measures. Scientometrics, n° 36, 293-310.

Neary J.P., Mirrlees J.A., Tirole J 2003. Evaluating Economics Research in Europe: An Introduction. Journal of the European Economic Association, Vol 1 , n° 6, p. 1247.

OECD 1997. The evaluation of scientific research: selected experiences, Paris.

OECD 2009. Science Technology and Industry Scoreboard 2009, Paris

RAE - Research Assessment Exercise 2006. Assessment panel's criteria and working methods, doc. RAE 01/06, in http://www.rae.ac.uk.

Remler D. K., Pema E. 2009. Why do Institutions of Higher Education Reward Research While Selling Education?. NBER Working Paper No. 14974.

Rizzuto, C. 2003. Finanziamenti, progetti, centri di eccellenza, FAR, FISR, distretti tecnologici, in La ricerca universitaria: esperienze, modelli, proposte, eds.

Casciotti C. and Mosconi G., p. 96-115, Italy,Rome: CRUI editions.

Seglen P. O. 1997. Why the impact factor of journals should not be used for evaluating research, BMJ, 314-497.

Van Leeuwen T. 2006. Descriptive Versus Evaluative Bibliometrics. Monitoring and Assessing of National R&D Systems, in Handbook of Quantitative Science and Technology Research, part 3, 373-388, Netherlands: Springer.

Van Raan A. 2005. Challenges in Ranking of Universities, Invited paper for the First International Conference on World Class Universities, Shanghai Jaio Tong University, Shanghai, June 16-18, 2005

Vinke P. 2009. University Rankings, in Ranking Universities, eds. Dehon C., Jacobs D., Vermandele C., pp.11-25, Belgium, Bruxelles: Editions de l'Université de Bruxelles.

World Economic Forum 2009, The Global Competitiveness Report 2009-2010.