Rethinking Research Metrics: The Performance Impact Score (PIS) as a Novel Alternative to the H-Index
DOI:
https://doi.org/10.59519/mper7101Keywords:
scientometrics, bibliography, researchers, evaluationAbstract
Providing an objective ranking of scientists based on their merit is a rather challenging task. Numerous factors complicate this endeavor, raising difficult questions about how such evaluations should be conducted. The goal of the present paper is to introduce a new measure for evaluating researchers’ performance, the Performance Impact Score (PIS), which combines both productivity and research impact. In this study, I compared the PIS with h-index scores for 108 researchers at the University of Sarajevo. More specifically, I examined the overlap between the top 20 researchers according to the PIS and the top 20 researchers according to Web of Science, Google Scholar, and SCOPUS h-indexes. The concordance rate was 65% for Google Scholar and Web of Science, and 55% for SCOPUS. The analysis highlights the importance of considering both productivity and impact when evaluating researchers’ rankings. It is also evident that the top 20 researchers vary across different metrics. The results further demonstrate that creating a fair and just ranking system requires going beyond the data available in bibliometric databases, particularly in cases where researchers rank highly by one metric but perform poorly when evaluated by another.
References
Akhtar, M. K. (2024). The H-Index Is an Unreliable Research Metric for Evaluating the Publication Impact of Experimental Scientists. https://doi.org/10.31219/osf.io/wh4ft
Barnes, C. (2017). The h-index Debate: An Introduction for Librarians. The Journal of Academic Librarianship, 43(6), 487-494. https://doi.org/https://doi.org/10.1016/j.acalib.2017.08.013
Bates, D. W. (2017). The ranking of scientists based on citations. Journal of Biomedical Informatics, 75, 128. https://doi.org/https://doi.org/10.1016/j.jbi.2017.10.012
Bornmann, L., & Daniel, H. D. (2005). Does the H-Index for Ranking of Scientists Really Work? Scientometrics, 65(3), 391-392. https://doi.org/10.1007/s11192-005-0281-4
Bornmann, L., Mutz, R., & Daniel, H. D. (2008). Are There Better Indices for Evaluation Purposes Than the h Index? A Comparison of Nine Different Variants of the h Index Using Data From Biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830-837. https://doi.org/10.1002/asi.20806
Chua, C., Cao, L., Cousins, K., & Straub, D. W. (2002). Measuring researcher-production in information systems. Journal of the Association for information Systems, 3(1), 145-215. https://doi.org/10.17705/1jais.00026
Costas, R., & Bordons, M. õ. (2007). The H-Index: Advantages, Limitations and Its Relation With Other Bibliometric Indicators at the Micro Level. Journal of Informetrics, 1(3), 193-203. https://doi.org/10.1016/j.joi.2007.02.001
Egghe, L. (2006). Theory and Practise of the G-Index. Scientometrics, 69(1), 131-152. https://doi.org/10.1007/s11192-006-0144-7
Friedmacher, F., Pakarinen, M. P., & Rintala, R. J. (2018). Congenital diaphragmatic hernia: a scientometric analysis of the global research activity and collaborative networks. Pediatric Surgery International, 34(9), 907-917. https://doi.org/10.1007/s00383-018-4304-7
Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences, 102(46), 16569-16572. https://doi.org/10.1073/pnas.0507655102
Hirsch, J. E. (2019). Hα: An Index to Quantify an Individual’s Scientific Leadership. Scientometrics, 118(2), 673-686. https://doi.org/10.1007/s11192-018-2994-1
Južnič, P., Pečlin, S., Žaucer, M., Mandelj, T., Pušnik, M., & Demšar, F. (2010). Scientometric indicators: peer-review, bibliometric methods and conflict of interests. Scientometrics, 85(2), 429-441. https://doi.org/10.1007/s11192-010-0230-8
Kelly, C. D., & Jennions, M. D. (2006). The h index and career assessment by numbers. Trends in Ecology & Evolution, 21(4), 167-170. https://doi.org/10.1016/j.tree.2006.01.005 Lane, J. (2010). Let's make science metrics more scientific. Nature, 464(7288), 488-489. https://doi.org/10.1038/464488a
Leydesdorff, L., Bornmann, L., & Opthof, T. (2019). Hα: The Scientist as Chimpanzee or Bonobo. Scientometrics, 118(3), 1163-1166. https://doi.org/10.1007/s11192-019-03004-3
Lüscher, T. F. (2018). Measuring the unmeasurable: assessing the quality of science and scientists. European Heart Journal, 39(20), 1765-1769. https://doi.org/10.1093/eurheartj/ehy295
Meho, L. I. (2020). Highly prestigious international academic awards and their impact on university rankings. Quantitative Science Studies, 1(2), 824-848. https://doi.org/10.1162/qss_a_00045
Memisevic, H., Ajnadzic, N., & Memisevic, M. (2022). Measuring Scholarly Impact in Bosnia and Herzegovina: The Picture Beyond the Numbers. Journal of Scientometric Research, 11(1), 112-117.
Memisevic, H., Pasalic, A., Mujkanovic, E., & Memisevic, M. (2019). In Search of a Silver Bullet: Evaluating Researchers’ Performance in Bosnia and Herzegovina. Journal of Scientometric Research, 8(3), 125-130.
Memisevic, H., Taljic, I., & Hadziomerovic, A. M. (2017). Making Use of H-index: the Shape of Science at the University of Sarajevo. Acta Informatica Medica, 25(3), 187-190. https://doi.org/10.5455/aim.2017.25.187-190
Nath, A., & Jana, S. (2021). A Scientometric Review of Global Altmetrics Research. Science & Technology Libraries, 40(3), 325-340. https://doi.org/10.1080/0194262X.2021.1918607
Rong, Z., Chang, R., Lu, L., Wang, X., & Zhang, Z. (2021). Research on the Index for Evaluating Academic Activeness of Researchers. Proceedings of the Association for Information Science and Technology, 58(1), 881-882. https://doi.org/10.1002/pra2.596
Sathian, B., Simkhada, P., & Sreedharan, J. (2014). Importance of the Authorship Order and Number of Co Authors in a Publication for Evaluation: A Necessary Enquiry. Nepal Journal of Epidemiology, 4(4), 384-385. https://doi.org/10.3126/nje.v4i4.11207
Selivanova, I. V., Kosyakov, D. V., & Guskov, A. E. (2019). The Impact of Errors in the Sсopus Database on the Research Assessment. Scientific and Technical Information Processing, 46(3), 204-212. https://doi.org/10.3103/S0147688219030109
Sternberg, R. J. (2018). Evaluating Merit Among Scientists. Journal of Applied Research in Memory and Cognition, 7(2), 209-216. https://doi.org/10.1016/j.jarmac.2018.03.003
Zerem, E. (2017). The ranking of scientists based on scientific publications assessment. Journal of Biomedical Informatics, 75, 107-109. https://doi.org/https://doi.org/10.1016/j.jbi.2017.10.007
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Haris Memišević (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
