Metodi bibliometrici e revisione dei pari per la valutazione della ricerca: un confronto metodologico

Cicero, Tindaro and Malgarini, Marco and Nappi, Carmela Anna and Peracchi, Franco (2013): Metodi bibliometrici e revisione dei pari per la valutazione della ricerca: un confronto metodologico.

Abstract

The Italian Research Evaluation exercise for the period 2004-2010 has analyzed almost 185,000 among articles, books, patents and other scientific products submitted by Italian Universities and other public research bodies. In most cases, scientific publications have been peer reviewed; however, in hard sciences, medicines, engineering and economics, bibliometric indicators have also been used. For those areas, we have extracted a representative sample of scientific products, equal to the 10% of the reference population of submitted products, to be evaluated both with peer review and biblometric methods. Our analysis shows that peer review and bibliometric evaluations exhibit a level of concordance higher than that observed among two different reviewers of the same article. In almost any scientific discipline, however, there is a systematic difference among peer and bibliometric evaluations: more specifically, bibliometric scores are on average significantly higher than those obtained with the peer review. Overall, our results obtained fully support the choice adopted in the Italian exercise of using both evaluation techniques in order to assess the quality of Italian research institutions.

• Item Type: MPRA Paper
• English Title: Bibliometric and peer review methods for research evaluation: a methodological appraisement
• Language: Italian
• Keywords: Indicatori bibliometrici; revisione dei pari; valutazione della ricerca
• Subjects: I - Health, Education, and Welfare > I2 - Education and Research Institutions > I20 - General; I - Health, Education, and Welfare > I2 - Education and Research Institutions > I23 - Higher Education ; Research Institutions

References

ANVUR (2013), Rapporto Finale Valutazione della Qualità della Ricerca 2004-2010, luglio, http://www.anvur.org/rapporto/

Bornmann, L., Leydersdorff L. (2012), The validation of (advanced) bibliometric indicators trough peer assessments: A comparative study using data from InCites and F1000, Journal of Infometrics, vol. 7, II, Aprile

Cohen, J. (1960). "A coefficient of agreement for nominal scales". Educational and Psychological Measurement 20 (1): 37–46. doi:10.1177/001316446002000104.