In July 2018, draft guidance and panel criteria for REF 2021 were published for consultation. This follows the Stern review, which made the following recommendation about public engagement:
“Recommendation 7: Guidance on the REF should make it clear that impact
case studies should not be narrowly interpreted, need not solely focus on socioeconomic impacts but should also include impact on government policy, on
public engagement and understanding, on cultural life, on academic impacts
outside the field, and impacts on teaching.”
So, just how has this recommendation translated into the current draft guidance? Here are the ‘public engagement’ bits we could pick out (in bold):
“274. The panels also acknowledge that there are multiple and diverse pathways through which research achieves impact. Impact may be the result of individual or collective research (or a combination of these) within or between a range of organisations, within higher education and beyond, including collaboration beyond the UK. The associated impact may be achieved by a variety of possible models: from individuals, to inter-institutional groups, to groups including both academic and non-academic participants. The relationship between research and impact can be indirect or non-linear. The impact of research may be foreseen or unforeseen. It can emerge as an end product, but can also be demonstrated during the research process. Impact takes place through a wide variety of mechanisms. It may effect change or enrichment for local, national or international communities, groups or individuals. Consequently, public engagement may be an important feature of many case studies, as the mechanism by which the impact claimed has been achieved. ” (Page 62)
“288. Engaging the public with the submitting unit’s research (for example, through citizen science, patient and public involvement in health, or through public and community engagement), is an activity that may lead to impact. Sub-panels will welcome, and assess equitably, case studies describing impacts achieved through public engagement, either as the main impact described or as one facet of a wider range of impacts. Panels expect that case studies based on public engagement will demonstrate both reach (e.g. through audience or participant figures) and significance, and will take both into account when assessing the impacts. Examples of impacts arising from public engagement can be found as part of Table 1 (Annex A).” (Page 64)
The report states that more detailed advice on achieving and evidencing impact through public engagement can be found on the website of the National Coordinating Centre for Public Engagement: http://www.publicengagement.ac.uk/. However it also provides some indicative (but by no means exhaustive) examples of areas in which impact from research across the panels may be demonstrated to have had a positive influence on the quality of life of individuals and communities locally, nationally and internationally, which can be found in Table 1 (Page 94 onwards).
In short, the document doesn’t really tell us much that we don’t already know, but it is encouraging to see public engagement more explicitly referenced in relation to REF impact and there seems to be clear guidance in place for assessors to factor in both reach and significance when making judgments on the impactfulness of public engagement. Examples of different (1-4*) levels of significance are however missing, and so this is likely to be the area where most careful thought is needed. This is perhaps particularly true when thinking about how to best narrate ‘significance’ based on public engagement activity involving small numbers, such that it successfully conveys ‘representative snapshot’ evidence of what might be possible with larger numbers given more time, resource, manpower to scale up the activity etc….
The documents also suggests that there isn’t a silver bullet answer as to how to do this perfectly and that reviewing case studies from 2014 and their assessment might not be a bad place to start looking. I.e. we don’t necessarily need to scrap what people have done previously, we can instead improve on these approaches with new tools, resources or improved rigor.
The University of Birmingham’s Public Engagement with Research (PER) team will be mulling over the draft guidance and feeding in to the University’s consultation response via our colleagues in Research Planning, so if you’ve got thoughts you’d like to share with us, feel free to drop us an email at engage@contacts.bham.ac.uk with ‘REF consultation’ in the title of your email. The online consultation closes on 15 October 2018. Or why not come chat to us face to face on September 10th at the University Research Conference, we will be there all day manning our public engagement stand, so come and say hello (we promise we can talk about other things as well as REF)!
If you are interested to find out more about public engagement within the context of REF2021, the National Coordinating Centre for Public Engagement (NCCPE) are hosting an event on September 14th, 2018, which promises to:
- Provide an overview of the proposals and their implications for impacts arising from public engagement
- Present the NCCPE’s draft consultation response and its rationale
- ‘Road test’ tools and resources to help HEIs with their planning for 2021