Karolinska Institutet
Browse

Bibliometrics as a research assessment tool : impact beyond the impact factor

Download (1.09 MB)
thesis
posted on 2024-09-03, 02:19 authored by Jonas Lundberg

Introduction: While bibliometrics is frequently used as a tool for research assessment and is thought of as objective, quantitative and unobtrusive, the usability of bibliometric indicators has seldom been properly assessed and their value therefore been questioned. The aim of this thesis is to explore and develop the utility of bibliometrics as a research assessment tool. This is done through four studies that address the validity of bibliometrics as a research assessment tool. The issues that are further investigated are related with field delineation (Study I), collaboration (II) and research performance (I/III/IV).

Materials and methods: The thesis is primarily based on data from the citation indices (Cl) produced by Thomson Scientific. In the different studies bibliometric indicators are calculated based on 24 223 (I), 62 104 (II) 6 142 055 (III) and 596 (IV) publications. To assess the validity of bibliometric indicators calculated using the Cl, these are combined with data from PubMed (I) and compared with data from manual assessments (I), financial data (II) and a Swedish system for identification and early assessment of new methods in health care (IV). Three new indicators are developed based on theoretical reasoning (III).

Results: Frequently used bibliometric methods do not allow valid assessment of the development of research areas (I) or collaboration between academia and industry (II). There are also flaws associated with the current state-of-the-art performance indicator (III). At the same time, there are bibliometric methods available that can be used for identifying research areas (I) and there are certain types of collaboration between academia and industry that accurately can be described using bibliometric indicators (II). New performance indicators, which assign equal weight to all publications and control for the skewed and differing distribution of citations over publications, have been developed (III). Publications related to health care technologies that are deemed as very promising by clinical experts receive high scores using these indicators (IV).

Discussion: The results of this thesis show that uncritical assessments of research areas based on rudimentary article identification strategies or collaboration analyses based solely on coauthorship data may be misleading and thus provide incorrect information for decision-making. At the same time, a correct use of refined bibliometric indicators may provide valuable background information for decision makers.

List of scientific papers

I. Lundberg J, Fransson A, Brommels M, Skår J, Lundkvist I. (2006). Is it better or just the same? Article identification strategies impact bibliometric assessments. Scientometrics. 66(1): 183-197.
https://doi.org/10.1007/s11192-006-0013-4

II. Lundberg J, Tomson G, Lundkvist I, Skår J, Brommels M. (2006). Collaboration uncovered: Exploring the adequacy to measure university-industry collaboration through co-authorship and funding. Scientometrics. 69(3).
https://doi.org/10.1007/s11192-006-0170-5

III. Lundberg J (2006). Lifting the crown - citation z-score. Journal of Informatics. [Accepted]
https://doi.org/10.1016/j.joi.2006.09.007

IV. Lundberg J, Brommels M, Skår J, Tomson G (2006). Breakthroughs and bibliometrics: The sources of the Swedish rapid evaluation system of technologies in health care. [Submitted]

History

Defence date

2006-11-17

Department

  • Department of Learning, Informatics, Management and Ethics

Publication year

2006

Thesis type

  • Doctoral thesis

ISBN-10

91-7140-965-3

Number of supporting papers

4

Language

  • eng

Original publication date

2006-10-27

Author name in thesis

Lundberg, Jonas

Original department name

Department of Learning, Informatics, Management and Ethics (Lime)

Place of publication

Stockholm

Usage metrics

    Theses

    Categories

    No categories selected

    Keywords

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC