Does the Research Excellence Framework (REF) Have a Sustainable Future?

The results of the Research Excellence Framework (REF) for 2021 have just been revealed – the first since 2014. REF is a UK-wide assessment carried out by four education funding bodies, assessing over 76,000 academics at 157 universities. The final ranking determines the amount of quality-related research (QR) funding that universities receive from the Government each year for the next seven years. 

While many have opted to celebrate the stronger diversity of nationwide results this year, the overall system has also been heavily criticised by unions and independent researchers. In the week that followed the results on 12 May, we analysed the trending areas of discussion among 133 journalists across 515 UK-based publications, alongside 34 journalists across 102 international media outlets.   

Strong Distribution of ‘World-leading’ Research  

84% of UK research assessed by REF has been dubbed as ‘world-leading’ or ‘internationally excellent’. Oxford University had the highest volume of world-leading research and made the largest submission of research compared with any other university, submitting more than 3,600 researchers in 29 subject areas. (Mail Online, 12.05.22).  

The term ‘northern powerhouse’ was trending in the first two days, with 162 articles referencing the strong representation of highly ranked universities in northern regions of the UK. The leading example was University of Northumbria, which was mentioned 189 times for soaring from 52nd to 28th in market share of future funding. Of this selection, 90% of all articles featured a quote from Andrew Wathey, Northumbria’s vice-chancellor, who commented that the achievement will move them ‘into territory formerly the preserve of the Russell Group’ (The Independent, 12.05.22).  As a result, the ‘golden triangle’ universities have lost 2.4 percentage points since the 2014 REF results, a statistic shared 85 times by both regional and national publications.  

Similarly, Scottish universities received strong prominence due to every institution in the region delivering ‘world-leading’ research (The National, 12.05.22). Among the 62 Scotland-focused articles across several UK news sources, 23% featured the terms ‘Scottish universities’ and ‘world-leading research’ in their headlines. 

Diversity and the ‘Levelling-Up’ Agenda 

As REF results are only released every seven years, a leading point of discussion in this cycle has been the ‘even spread’ of success in all four nations across the country (Mail Online, 12.05.22). This term has been used 155 times across UK and international media outlets, highlighting newfound diversity and funding allocated to universities outside the Russell Group.  

The Government has used this opportunity to outline how such results support the ‘levelling-up’ agenda, a seven-year plan to reduce (primarily economical) imbalances and increase public investment in areas outside the Southeast of England by 40%. The agenda has been mentioned 164 times by national news and business publications, as well as in a small percentage of coverage in North America, Australia and New Zealand. 96% of this coverage featured a quote from Steven Hill at Research England, and subsequent Chair of the REF steering group: ‘There’s lots of myths about where our research excellence is, but the truth is that it is more broadly distributed, as the results from this exercise show’, adding that the UK research system is ‘well placed to meet the Government’s ambitions for levelling up’.  (New Scientist, 12.05.22). An article by The Independent, titled ‘University research triumphs will help with levelling up agenda’, was syndicated 143 times by local and regional press such as The Reading Chronicle, Cotswold Journal and South Wales Argus. 

Share of Voice: Prominence of REF Opinions  

Hidden Layers of Inequity 

While almost 78% of coverage shares either a positive headline, themes or ‘levelling-up’, many unions and independent researchers have described the REF process as a ‘bureaucratic nightmare’ which can ‘entrench inequality’ given how the results impact funding allocations (Evening Standard, 11.05.22). Dr Jo Grady, General Secretary at University and College Union (UCU), has the strongest share of voice for those in opposition of REF. Grady’s critique has been quoted 93 times since the results were released on 12 May, also taking to Twitter to expand her reach: 

Trending Areas of Criticism in UK Media 

Multiple trending areas of controversy have emerged in and out of UCU, primarily being:                             

Poor Working Conditions  

With almost three in four researchers set to leave higher education, multiple unions across the UK have described the celebrations as ‘unfair’ and ignorant (Daily Mail, 12.05.22). UCU released a public statement describing vice chancellors as ‘utterly hypocritical’ after ‘severe cuts’ to researcher pensions, adding that they are ‘hijacking and seeking to capitalise on the hard work of research and academic related staff’ (UCU, 12.05.22).  

Excessive Cost on Public Funds 

While the ethics and structure behind REF have been heavily debated, one aspect that most parties agree with is the excessive administrative burdens and use of public funds. Between just the 2008 and 2014 exercises, costs increased from £66 million to £246 million. A combination of 31 university new sources, education journals and regional news publications have shared their concerns, leading some members of REF funding councils to recognise the ‘radical shake up’ that may be necessary to create a sustainable and ‘healthy research culture’ (Nature, 12.05.22). A professor at Cambridge University added that the costs exceed far beyond monetary value – the REF now ‘looms over the daily lives of institutions and individuals like a massive headache’ (HEPI, 10.05.22).   

Unfair Evaluation Methods 

Approaches used to assess university research have also been questioned, as many requirements are not attainable in all disciplines. For example, use of citation data is cautioned as an indicator of quality. Scholars in arts, humanities and many social sciences are particularly critical of this approach, not least because many publications in these fields extend well beyond traditional academic journals into books and physical objects (to include, for example, paintings). Catriona Firth, the associate director for research environment at Research England, is one of many who believes this has a negative impact on the type of work carried out at universities: ‘What institutes think is going to be valuable in the REF is what they encourage staff to do and what they invest in’ (Nature, 12.05.22). Another argued that the process was ‘dampening initiative and originality’, replacing ‘the object of desire (good research) with its proxy’ for the sake of a higher ranking (HEPI, 10.05.22).   

Political Engagement  

This year’s REF results have cut through to Parliament with a Early Day Motion being tabled to congratulate the University of Dundee on it’s performance by the SNP. Elsewhere, Paul Howell MP congratulated Northumbria University on its leap to No23 in the latest ranking. Daniel Zeichner MP spoke of the exercise as evidence of UK universities success, arguing this should be the focus of the Government instead of the ‘stoking [of] culture wars’ which has dominated debate in recent years.     

The framework received criticism in the House of Lords, with Viscount Hanworth referring to it as part of a burdensome audit culture. Blogs posted on the Higher Education Policy Institute reiterated that it is a ‘highly complex system of assessment’, but did caution that reforms should not aim not to throw the baby out the bath water.  

The University and College Union were also critical, drawing comparisons between universities celebrating their REF results with the reality for university staff, two-thirds of which they say ‘are considering leaving the sector’.  

MP Engagement  

Professor Geoff Rodgers noted the obligatory celebrations of Universities post- REF results are one of the most important aspects of the design of the whole exercise for the sector’s interaction with the Government. However, given only 13 MPs tweeted about REF in response to their local universities’ success, there may still be some way to go.   

The Future of REF: 2028 

The future of REF is uncertain. While steps towards the ‘levelling-up’ agenda suggest more diverse funding opportunities, Nick Hillman, director of the Higher Education Policy Institute, warned that there was ‘no consensus on the value of the REF’. The Labour Party has described the REF as ‘discredited’ and the current shadow minister for science, research and innovation, Chi Onwurah, has complained it ‘encourages a cut-throat environment’, calling instead for more ‘strategic direction from the Government’ and ‘a more equitable funding formula’ (University News, 13.05.22). Moreover, Peter Mandler, professor of Modern Cultural History at the University of Cambridge, believes REF is ‘no longer all that much about excellence or even about research’. In his blog with HEPI, he described the ‘sketchy’ assessment of university impact and environment as a ‘dirty secret’ within the REF system, adding that the whole process is less of a research assessment and ‘more of a public-relations assessment exercise’. Though a professor at one of the ‘golden triangle’ universities, Mandler is one of many in favour of the radical shake-up – or beyond that, to simply ‘rip up the rulebook and start again’ (HEPI, 10.05.22).  

Want to understand more about this story and data, or find out how the Vuelio Insights team can support you? Get in touch.