Assessing scientific research performance and impact with single indices

Panaretos, John and Malesios, Chrisovaladis (2008): Assessing scientific research performance and impact with single indices. Forthcoming in: Scientometrics

Abstract

We provide a comprehensive and critical review of the h-index and its most important modifications proposed in the literature, as well as of other similar indicators measuring research output and impact. Extensions of some of these indices are presented and illustrated.

• Item Type: MPRA Paper
• Language: English
• Keywords: Citation metrics, Research output, h-index, Hirsch index, h-type indices
• Subjects: I - Health, Education, and Welfare > I2 - Education and Research Institutions

References

Adler, R., Ewing, J. and Taylor, P. (2008). Citation Statistics. Joint IMU/ICIAM/IMS-Committee on Quantitative Assessment of Research.

Amin, M. and Mabe, M. (2000). Impact Factor: Use and Abuse. Perspectives in Publishing, 1, 1-6.

Arencibia-Jorge, R., Barrios-Almaguer, I., Fernandez-Hernandez, S. and Carvajal -Espino, R. (2008). Applying Successive H indices in the Institutional Evaluation: A Case Study. Journal of the American Society for Information Science and Technology, 59(1), 155-157.

Ball, P. (2005). Index Aims for Fair Ranking of Scientists. Nature, 436, 900.

Banks, M.G. (2006). An Extension of the Hirsch Index: Indexing Scientific Topics and Compounds. Scientometrics, 69(1), 161-168.

Barendse, W. (2007). The Strike Rate Index: A New Index for Journal Quality Based on Journal Size and the h-index of Citations. Biomedical Digital Libraries, 4:3.

Batista, P.D., Campiteli, M.G., Kinouchi, O. and Martinez, A.S. (2005). Universal Behaviour of a Research Productivity Index. ArXiv:physics/0510142.

Beirlant, J. and Einmahl, J.H.J. (2007). Asymptotics for the Hirsch Index (available at: http://econpapers.repec.org/paper/dgrkubcen/200786.htm).

Bornmann, L. and Daniel, H.D. (2007). What Do We Know About the h Index? Journal of the American Society for Information Science and Technology, 58(9), 1381–1385.

Bornmann, L., Mutz, R. and Daniel, H.D. (2008). Are There Better Indices for Evaluation Purposes than the h-index? A Comparison of Nine Different Variants of the h Index Using Data from Biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 1-8.

Braun, T., Glänzel, W. and Schubert, A. (2005). A Hirsch-type Index for Journals. The Scientist, 19, 8.

Braun, T., Glänzel, W. and Schubert, A. (2006). A Hirsch-type Index for Journals. Scientometrics, 69(1), 169-173.

Burrell, Q.L. (1992). A Simple Model for Linked Informetric Processes. Information Processing and Management, 28, 637-645.

Burrell, Q.L. (2007a). Hirsch’s h-index: A Stochastic Model. Journal of Informetrics, 1, 16-25.

Burrell, Q.L. (2007b). On the h-index, the Size of the Hirsch Core and Jin’s A-index. Journal of Informetrics, 1, 170-177.

Chai, J-C., Hua, P-H., Rousseau, R. and Wan, J-K. (2008). The Adapted Pure h-Index. Proccedings of WIS 2008, Berlin Fourth International Conference on Webometrics, Informetrics and Scientometrics & Ninth COLLNET Meeting.

Chapron, G. and Husté, A. (2006). Open, Fair, and Free Journal Ranking for Researchers. BioScience, 56(7), 558-559.

Cole, S. and Cole, J.R. (1973). Social Stratification in Science. University of Chicago Press, Chicago.

Costas, R. and Bordons, M. (2007). The h-index: Advantages, Limitations and its Relation with Other Bibliometric Indicators at the Micro Level. Journal of Informetrics, 1, 193-203.

Cronin, B. and Meho, L. (2006). Using the h-index to Rank Influential Information Scientists. Journal of the American Society for Information Science and Technology, 59, 1275-1278.

Cozzens, S.E. (1989). What do Citations Count? The Rhetoric-first Model. Scientometrics, 15(5), 437-447.

Egghe, L. (2005). Power Laws in the Information Production Process: Lotkaian Informetrics. Elsevier, Oxford.

Egghe, L. (2006a). How to Improve the h-index. The Scientist, 20(3), 14.

Egghe, L. (2006b). An improvement of the H-index: the G-index. ISSI Newsletter, 2(1), 8-9.

Egghe, L. (2006c). Theory and Practice of the g-index. Scientometrics, 69(1), 131-152.

Egghe, L. (2008a). Modelling Successive h-indices. Scientometrics (to appear).

Egghe, L. (2008b). Dynamic h-index: The Hirsch Index in Function of Time. Journal of the American Society for Information Science and Technology (to appear).

Egghe, L. (2008c). Mathematical Theory of the h- and g-Index in Case of Fractional Counting of Authorship. Journal of the American Society for Information Science and Technology, 59(10), 1608-1616

Egghe, L. and Rao, R. (2008). Study of Different h-indices for Groups of Authors. Journal of the American Society for Information Science and Technology, 59(8), 1276-1281.

Egghe, L. and Rousseau, R. (2006). An Informetric Model for the Hirsch Index. Scientometrics, 69(1), 121-129.

Egghe, L., and Rousseau, R. (2007). An h-index Weighted by Citation Impact. Information Processing & Management, 44(2), 770-780.

Egghe, L., Rousseau, R. and Van Hooydonk, G. (2000). Methods for Accrediting Publications to Authors or Countries: Consequences for Evaluation Studies. Journal of the American Society for Information Science, 51(2), 145-157.

Evidence Report (2007). The Use of Bibliometrics to Measure Research Quality in the UK Higher Education System (A Report Produced for the Research Policy Committee of Universities, UK, by Evidence Ltd., a Company Specializing in Research Performance Analysis and Interpretation).

Gumbel, E.J. (1958). Statistics of Extremes. Columbia University Press.

Garfield, E. (1955). Citation Indexes to Science: A New Dimension in Documentation through Association of Ideas. Science, 122, 108-111.

Garfield, E. (2006). The History and Meaning of the Journal Impact Factor. Journal of the American Medical Association, 295, 90-93.

Glänzel, W. (2006). On the Opportunities and Limitations of the H-index. Science Focus, 1(1), 10-11.

Goldstein, H. and Spiegelhalter, D.J. (1996). League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance. Journal of the Royal Statistical Society, Series A. 159(3), 385-443.

Hirsch, J.E. (2005). An Index to Quantify an Individual’s Scientific Research Output. Proceedings of the National Academy of Sciences, USA, 102(46), 16569-16572.

Hirsch, J.E. (2007). Does the h-index have a Predictive Power? (available at: http://www.pnas.org/content/104/49/19193.full.pdf+html).

Iglesias, J.E., and Pecharromán, C. (2007a). Scaling the h-index for Different Scientific ISI Fields. Scientometrics, 73(3), 303-320.

Iglesias, J.E., and Pecharromán, C. (2007b). Comparing h-indices for Scientists in Different ISI Fields. Porceedings of the 11th Conference of the International Society for Scientometrics and Informetrics. Madrid, Spain: Spanish Research Council (CSIC), 862-863.

Imperial, J., and Rodriguez-Navarro, A. (2007). Usefulness of Hirsch’s h-index to Evaluate Scientific Research in Spain. Scientometrics, 71(2), 271-282.

Jacsó, P. (2008a). Testing the calculation of a realistic h-index in Google Scholar, Scopus, and Web of Science for F. W. Lancaster. Library Trends.

Jacsó, P. (2008b). The Pros and Cons of Computing the h-index Using Web of Science. Online Information Review, 32(5), 673-688.

Jin, B-H. (2006). H-index: An Evaluation Indicator Proposed by Scientist. Science Focus, 1(1), 8-9.

Jin, B-H., Liang, L., Rousseau, R. and Egghe, L. (2007). The R- and AR- indices: Complementing the h-index. Chinese Science Bulletin, 52, 855-863.

Kelly, C. D. and Jennions, M.D. (2006). The h Index and Career Assessment by Numbers. Trends in Ecology and Evolution, 21(4), 167–170.

Kinney, A.L. (2007). National Scientific Facilities and their Science Impact on Nonbiomedical Research. Proceedings of the National Academy of Sciences USA, 104(46), 17943-17947.

Kosmulski, M. (2006). A new Hirsch-type Index Saves Time and Works Equally Well as the Original h-index. ISSI Newsletter, 2(3), 4-6.

Kosmulski, M. (2007). MAXPROD – A New Index for Assessment of the Scientific Output of an Individual, and a Comparison with the h-index. International Journal of Scientometrics, Informetrics and Bibliometrics, 11(1).

Lehmann, S., Jackson, A.D. & Lautrup, B.E. (2006). Measures for Measures. Nature, 444, 1003 – 1004.

Lehmann, S., Jackson, A.D. & Lautrup, B.E. (2005). Measures and Missmeasures of Scientific Quality (to appear). (available at: http://arxiv.org/abs/physics/0512238)

Liang, L. (2006). H-index Sequence and H-index Matrix: Constructions and Applications. Scientometrics, 69(1), 153-159.

Liang L., Rousseau, R. and Egghe, L. (2007). The R- and AR-indices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855-863.

Liu Y., and Rousseau, R. (2007). Hirsch-type Indices and Library Management: The Case of Tongji University Library. Proceedings of ISSI 2007. Madrid: CINDOC-CSIC, 514-522.

MacRoberts, M.H. and MacRoberts, B.R. (1989). Problems of Citation Analysis: A Critical Review. Journal of the American Society for Information Science, 40, 342-349.

Martin, B.R. and Irvine, J. (1983). Assessing Basic Research: Some Partial Indicators of Scientific Progress in Radio Astronomy. Research Policy, 12, 61-90.

Meho, L.I. (2007). The Rise and Rise of Citation Analysis. Physics World, 29(1), 32-36.

Meho, L.I. and Yang, K. (2007). Impact of Data Sources on Citation Counts and Rankings of LIS Faculty: Web of Science vs. Scopus and Google Scholar. Journal of the American Society for Information Science and Technology, 58(13), 2105-2125.

Miller, C.W. (2006). Superiority of the h-index Over the Impact Factor for Physics. eprint arXiv:physics/0608183.

Moed, H. and van Leeuwen, T. (1996). Impact Factors can Mislead. Nature, 381, 186.

Molinari, J-F. and Molinari, A. (2008). A New Methodology for Ranking Scientific Institutions. Scientometrics, 75(1), 163-174.

Orbay, M., Karamustafaoğlu, O. and Öner, F. (2007). What does Hirsch Index Evolution Explain Us? A Case Study: Turkish Journal of Chemistry.

Panaretos, J. and Malesios, C.C. (2008). Assessing Scientific Research Performance and Impact with Single Indices. Technical Report no 235 (March 2008), Department of Statistics, Athens University of Economics and Business.

Persson, O., Glänzel, W. and Danell, R. (2008). Inflationary Bibliometric Values: The Role of Scientometric Collaboration and the Need for Relative Indicators in Evaluative Studies. Scientometrics (to appear).

Podlubny, I. (2005). Comparison of Scientific Impact Expressed by the Number of Citations in Different Fields of Science. Scientometrics, 64(1), 95-99.

Podlubny, I. And Kassayova, K. (2006). Towards a Better List of Citation Superstars: Compiling a Multidisciplinary List of Highly Cited Researchers. Research Evaluation, 15(3), 154–162

Prathap, G. (2006). Hirsch-type Indices for Ranking Institutions’ Scientific Research Output. Current Science, 91(11), 1439.

Publish or Perish User's Manual (2007). http://www.harzing.com/resources.htm#/pop.htm, accessed on June 12, 2007.

Roediger, H.L. (2006). The h-index in Science: A New Measure of Scholarly Contribution. Observer: The Academic Observer, 19(4), 37-40.

Rousseau, R. (2006a). Simple Models and the Corresponding h- and g-index. Science Technology Development (to appear).

Rousseau, R. (2006b). New Developments Related to the Hirsch index. Science Focus, 1(4), 23-25 (in Chinese). (English version available at: http://eprints.rclis.org/archive/00006376/)

Rousseau, R. (2007a). A Case Study: Evolution of JASIS’ Hirsch Index, Science Focus.

Rousseau, R. (2007b). The influence of Missing Publications on the Hirsch Index. Journal of Informetrics, 1, 2-7.

Saad, G. (2006). Exploring the h-index at the Author and Journal Levels Using Bibliometric Data of Productive Consumer Scholars and Business-related Journals Respectively. Scientometrics, 69(1), 117-120.

Saisana, M. and d’Hombres, B. (2008). Higher Education Rankings: Robustness Issues and Critical Assessment, JRC Scientific and Technical Reports 23487 EN.

Sanderson, M. (2008). Revisiting h Measured on UK LIS Academics. Journal of the American Society for Information Science and Technology, 59(7), 1184-1190.

Schreiber, M. (2007a). Self-citation Corrections for the Hirsch Index. EPL 78 30002, 1-6.

Schreiber, M. (2007b). The Influence of Self-citations on Egghe’s g Index.

Schreiber, M. (2008a). A Modification of the h-index: The hm-index Accounts for Multi-authored Manuscripts. Journal of Informetrics, 2(3), 211-216.

Schreiber, M. (2008b). To Share the Fame in a Fair Way, hm Modifies h for Multi-authored Manuscripts. New Journal of Physics, 10, 9.

Schubert, A. (2007). Successive h-indices. Scientometrics, 70(1), 201-205.

Schubert, A. and Glänzel, W. (2007). A Systematic Analysis of Hirsch-type Indices for Journals. Journal of Informetrics, 1, 179-184.

Sidiropoulos, A., Katsaros, D. and Manolopoulos, Y. (2006). Generalized h-index for Disclosing Latent Facts in Citation Networks. Scientometrics, 72(2), 253-280.

Symonds M.R., Gemmell N.J., Braisher T.L., Gorringe K.L., and Elgar M.A. (2006). Gender Differences in Publication Output: Towards an Unbiased Metric of Research Performance. PLoS ONE 1: e127, 1-5.

Tol, R.S.J. (2007). Of the h-index and its Alternatives: An Application to the 100 Most Prolific Economists.Working paper FNU-146.

Toro-Alves, N., Herculano, R.D., Terçariol, C.A.S., Filho, O.K. and Graeff, C.F.O. (2007). Hirsch’s Index: Case Study Conducted at the Faculdade de Filosofia, Ciências e Letras de Ribeirão Preto, Universidade de São Paulo. Brazilian Journal of Medical and Biological Research, 40, 1529-1536.

van Raan, A.J. (2005). Measurement of Central Aspects of Scientific Research: Performance, Interdisciplinary, Structure. Measurement: Interdisciplinary Research and Perspectives, 3(1), 1-19.

Van Raan, A.J. (2006). Comparison of the Hirsch-index with Standard Bibliometric Indicators and with Peer Judgment for 147 Chemistry Research Groups. Scientometrics, 67(3), 491-502.

Vanclay, J.K. (2007). On the Robustness of the h-index. Journal of the American Society for Information Science and Technology, 58(10), 1547-1550.

Vinkler, P. (2007). Eminence of Scientists in the Light of the h-index and other Scientometric Indicators. Journal of Information Science, 33, 481-491.

Wan, J-K., Hua, P-H., and Rousseau, R. (2007). The Pure h-index: Calculating an Author’s h-index by Taking Co-authors Into Account. Preprint. (available at: http://eprints.rclis.org/archive/00011401/01/pure_h.pdf)

Wu, Q. (2008). The w-index: A significant improvement of the h-index. (available at: http://arxiv.org/ftp/arxiv/papers/0805/0805.4650.pdf