Skip to content

Research impact: A rising tide lifts all boats

How to measure the impact of research is a big issue at the moment for researchers and funders alike. As HEFCE’s The Metric Tide review of the use of metrics in research assessment and management is published, our Director of Strategic Evaluation and Impact Dr Ian Viney explains why funders like the MRC are interested in understanding how research leads to positive effects on health, wealth, culture and society — and that metrics are only a small part of this.

Dictionary entry for 'impact'

(Image: Nick Southall on Flickr under CC BY-NC 2.0)

 

The report out today, The Metric Tide, is so called to capture the view that like tides, the pressure to use metrics — quantitative aspects of research outputs and impact — to simplify evaluation is powerful and growing.  

However tides may of course be useful, as in the phrase “a rising tide lifts all boats”, which has been coined to refer to the spill-over benefit across the whole economy if leading industry sectors secure the support they need. [1]

The Metric Tide seeks to alleviate the pressure to measure research impact using inappropriate and over-simplified metrics, by setting out principles for the responsible use of such information. However there is of course still the requirement to efficiently tell the story of research progress, productivity and quality.

The report compiles thoughts from across the world on how funders are using quantitative indicators and their potential role in informing innovation policy. The recommendations will be important for guiding evaluation and assessment practices used by funders, universities, and publishers.

As someone who works to strengthen evaluation approaches in the research councils it was a privilege to represent RCUK on the steering group for the review, which brought together experts in scientometrics, research funding, research policy, publishing, and university administration.

One significant driver for the review was the recent Research Excellence Framework (REF), a huge exercise in 2014 which aimed to assess the current quality of research in UK universities and the impact realised in the past five years from research investments made over the past fifteen. A significant, and new, task in the REF was compiling from scratch 7,000 qualitative narrative reports of impact.

These narrative reports are a goldmine of information, but there is interest in whether there are more efficient ways to systematically and prospectively compile the story of research progress. It is highly likely that the impact component of the REF is here to stay, that there may be a need to update these case studies if they can be re-submitted to the next REF exercise, and the search for new examples of impact is undoubtedly already underway across the UK.

Compiling impact case studies involves connecting complex details of researcher/s, organisation/s, funding source/s and research outputs together, and so there is interest in how more consistent and prospective approaches to capturing these facts can help.

An excellent example of improving the way these facts can be connected has been the way the international community has ‘gone back to the drawing board’ concerning how researchers can be uniquely identified, resulting in establishing the ‘Open Researcher and Contributor ID’ or ORCID.

Similarly the research councils have led the development of an international approach to capture information on research output, now called Researchfish, which openly provides a rich set of outcome data linked to research council grants via the RCUK Gateway to Research.

In the past most attention has been paid in the STEM subjects to the analysis of publications, and even in bibliometrics which have been explored for more than 50 years, there are opportunities to improve the connection of this data.

For example, if funding sources were reliably recorded in publications at the stage of manuscript submission, the requirement from funders to add this data post publication would steadily reduce, the monitoring of open access to research findings would improve, and the value that funders, universities and publishers obtain from this information would be transformed.

Facts about people, outputs, papers and so on are a useful finding aid for impact stories, but what really motivates funders is the opportunity to use this evidence to learn more about how our work leads to impact. Understanding this may help us design more effective ways to support research, but also has the benefit of helping us explain how research can positively affect health, wealth, culture and society. This is of course very topical as we start another spending review in the UK.

To help us achieve this we are particularly interested in how we might support excellent “research on research”. Since 2010 the MRC has committed £1 million to seven new strategic studies to better understand impact, and we would like to fund more with other partners. This builds on landmark work that quantitatively demonstrated substantial economic returns from medical research [2], work that has shown that the funders of around £8 billion of UK medical research across the charity, public and private sectors are co-ordinating their investments in a highly complementary way. Work soon to be published examines the spill-overs from health research to the private sector. [4]

So, a tide is rising, and our evidence strongly suggests that a rising tide in research excellence will indeed lift all boats.

Ian Viney

The Metric Tide was commissioned by the former science minister, David Willetts, supported by HEFCE, and independently chaired by Professor James Wilsdon at the University of Sussex. Ian Viney was a member of the steering committee.

References

[1] Remarks of Senator John F. Kennedy, Municipal Auditorium, Canton, Ohio
September 27, 1960

[2] Health Economics Research Group, Office of Health Economics, RAND Europe. Medical Research: What’s it worth? Estimating the economic benefits from medical research in the UK. London: UK Evaluation Forum, 2008

[3] Exploring the Interdependency between Public and Charitable Medical Research. A report for CRUK, 2011

[4] Better measurement of the complementarity between UK public, charity and private medical research MRC grant to Kings College London (MR/L010801/2)

No comments yet

Leave a Reply

You may use basic HTML in your comments. Your email address will not be published.

Subscribe to this comment feed via RSS