Alex Rushforth

@mob@akademienl.social Thanks. Does this mean people in the same place who don't travel to the conference meet up? If so, did the conference organisers arrange this? (can imagine it's difficult for conference organisers to coordinate if they're in a different country).

Criticisms of evaluative bibliometrics have been with us a long time. But over time their presence has increased. In the past ten years, efforts to reform professional practices around their uses in research(er) assessment have gained momentum. We describe these efforts as a professional reform movement. DORA, Leiden Manifesto, & Metric Tide are the best known initiatives, the latter coining the term ‘responsible metrics’.

We have submitted it to a journal and are awaiting peer review. But we would of course love to hear feedback and criticism of the Preprint. We would also encourage anybody who might want to leave feedback to deposit comments on a preprint peer review platform like PREreview

The movement is a diverse coalition, that unified under the mantra that indicators should support, not replace expert peer judgement (sometimes called informed peer review). But within the coalition there are fault lines. We put these down to different ways of framing ‘the metrics problem’:

There’s the professional-expert framing (more expert information about indicators is needed among evaluators); metrics-scepticism framing (critique and emancipation from quantitative measures are required); and reflexive framing (better dialogue and communication about purposes and limitations needs to be co-produced among research stakeholders).

We speculate that these three framings are likely to generate further frictions, especially as the mission of the reform movement has expanded considerably recently to incorporate agendas of parallel reform movements. We’ve seen already accusations of bibliometric denialism levelled against followers of CoARA and DORA. This is a clash, in our view, between the professional-expert framing and metric scepticism. https://link.springer.com/article/10.1007/s11192-023-04787-2

As per a recent contribution by Ioannidis and Maniadis, we call for ongoing debate about the purposes, benefits and limitations of quantitative indicators, as well as further debate about whether peer review should serve by default as the basis of all forms of assessment (including organizational assessments). https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3002408

In the meantime, there also needs to be more accessible forms of training about quantitative tools (this project seems promising: Developing Metrics Literacies: https://thebibliomagician.wordpress.com/2023/12/06/developing-metrics-literacies-competencies-dispositions-and-knowledge-for-the-critical-assessment-and-ethical-use-of-scholarly-metrics/

Finally, science studies should pay attention to the emergence of research assessment reform as trans-national professional movement. Given it is attempting to transform the reward system of science – a canonical topic within science studies – this seems a no-brainer. /Ends

@pol_eco_pub@openbiblio.social These breakout rooms are for social interaction (i.e. replacing tea and coffee break) for virtual guests?