June 27, 2023, #44
In a Science Insider post titled “‘Quietly revolutionary’ plan would shake up the way U.K. universities are evaluated”¹, Cathleen O’Grady writes that “United Kingdom’s four national funding bodies”¹ are changing “the way the labor-intensive Research Excellence Framework (REF) exercise defines ‘excellence’.”¹ Under these changes planned for the 2028 evaluation round “Research culture will get more weight, with less emphasis on publications.”¹
O’Grady writes that previous versions of the REF “have placed too great an emphasis on the number of ‘outputs’—peer-reviewed papers and other publications”¹, effectively prioritizing publication numbers “over long-term projects or contributions such as software development”¹ The new approach would alter the weighting so that “25% of each university’s score will rest on its assessment of ‘people, culture and environment,’ up from 15% in the 2021 REF.”¹ Publication numbers are not an ideal measure of quality, but they do have the virtue of being relatively transparent and equally clear criteria could be hard to find.
One of the lead persons for this change appears to be “Catriona Firth, policy lead for research and culture at the national funding body Research England”¹, who “says she and her colleagues will be looking for ‘robust indicators and evidence’ that universities can submit so the assessment ‘goes beyond institutions making claims about how good they are.’”¹ Looking for robust indicators is good, but actually naming them may be harder. Unequal treatment of staff appears to be a problem with the current system. According to Firth the current system “created inequalities and low morale within departments, and incentivized universities to carefully time the end of short-term contracts to keep certain staff out of their REF submissions.”¹
It is never easy to change an evaluation culture. Institutions that were successful in the old culture may be uncomfortable with the new criteria, especially if the criteria are vague. Nonetheless, this change could have positive effects, because many countries and institutions implicitly or explicitly use the REF as a model. Having the UK take the lead in moderating the pressure to publish may also reduce the number of dubious publications around the world.
1: Cathleen O’Grady, “Quietly Revolutionary” Plan Would Shake up the Way U.K. Universities Are Evaluated’, Science Insider, accessed 22 June 2023,https://www.science.org/content/article/quietly-revolutionary-plan-would-shake-way-u-k-universities-are-evaluated.