[GOAL] Irrational rationality: critique of metrics-based evaluation of researchers and universities

[Heather Morrison]

A unique contribution in my recently published book chapter that aims to advance critique of metrics-based evaluation of scholarly research:

Critique of the concept of "impact" that underlies all metrics-based evaluation of research and researchers (traditional and altmetrics) as inherently logically flawed. For example, by any kind of metrics, the retracted study by Wakefield et al. that falsely equated vaccination with autism has had tremendous impact - numerous academic and popular media citations both before and after retraction, arguably the real-world resurgence of diseases such as measles and inspiration for the anti-vaccination movement that is problematic in the fight against COVID. In other words, impact is one thing, and quality of research something else. We understand that impact can be negative when we use the phrase "environmental impact assessment"; why do we assume that impact is positive with respect to scholarly work?

This argument is developed in the latter portion of my book chapter
Dysfunction in knowledge creation and moving beyond from Stack's Global University Rankings and the Politics of Knowledge. Other chapters from this book may be of interest as the areas of research impact metrics and university rankings are very much inter-related.

Readers may also appreciate the pointer to an alternative approach to evaluation of research developed some time ago by the University of Ottawa and its Association of Professors (APUO) that does not rely on metrics and is flexible to address the wide variety of types of research outputs of different disciplines, from traditional (articles and books) to alternative (research-creation) and early recognition of emerging forms such as preprints.

Discussion and comments on list, via e-mail or the blog are welcome. The blogpost begins with a different approach to critique of university rankings per se and can be found at
https://sustainingknowledgecommons.org/2021/08/03/irrational-rationality-critique-of-metrics-based-evaluation-of-researchers-and-universities/

References

Morrison, H. (2021). Dysfunction in knowledge creation and moving beyond. In Stack, M. (ed.) (2021). Global University Rankings and the Politics of Knowledge. http://hdl.handle.net/2429/78483

Stack, M. (ed.) (2021). Global University Rankings and the Politics of Knowledge. http://hdl.handle.net/2429/78483

best,


Dr. Heather Morrison

Associate Professor, School of Information Studies, University of Ottawa

Cross-appointed, Department of Communication

Professeur Agrégé, École des Sciences de l'Information, Université d'Ottawa

Principal Investigator, Sustaining the Knowledge Commons, a SSHRC Insight Project

sustainingknowledgecommons.org

Heather.Morrison at uottawa.ca

https://uniweb.uottawa.ca/?lang=en#/members/706

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ecs.soton.ac.uk/pipermail/goal/attachments/20210803/09fd2010/attachment.html