We all know that European research is world class, but is it helping to cure Alzheimer’s, or easing Europe’s transformation to a digital economy? Impact assessments are nothing new, and as we move inexorably towards the successor to the Horizon 2020 EU Research and Innovation Programme, the ability to measure the impact of what has gone before becomes key to defining research projects and plans for the EU’s next Framework Programme for Research and Innovation (FP9).
So, it came as no surprise that participants at a recent event organised by the European Cooperation in Science and Technology (COST) concluded that a broader, longer-term assessment of impact, and input from stakeholder networks, could enhance the value of research and innovation (R&I) spending.
COST is an EU-funded programme helping researchers to network and learn from each other. It provides funds for organising conferences, meetings, training schools, short scientific exchanges and other networking activities across a wide range of scientific and humanities topics.
The event brought together COST researchers with representatives from the European Commission, European Parliament and funding, standards and research agencies to discuss how to better define impact.
Their discussion focused on how to increase the value R&I brings, both economically and to society, as well as on developing dialogue on the value of the EU’s scientific capacity and how networks can contribute to this value. The COST programme believes that they can help unlock the full potential of science by creating spaces where people and ideas can grow.
COST director, Ronald de Bruin, opened the event by reminding participants that just last year, European Commissioner for research, science and innovation, Carlos Moedas, had placed the matter of ‘impact’ high on the EU’s research policy agenda, saying it should be a key value in the upcoming Framework Programme.
De Bruin pointed out that this is a more complex task than might be imagined. “What is impact, and who measures it and how?” Researchers may argue that their work does not always have an immediate effect, while the Commission needs to account for taxpayers’ money.
The approaching FP9 proposals make this question increasingly pressing. COST’s event aimed to provide greater insight and input into the issues.
Participants called for further investment, greater impact and more trust in science, with the notion of impact to be broadened to include societal value. Here, COST Actions could help to ensure the right policies and frameworks are in place, they agreed.
Attendees also spoke of the need to better communicate the difference that research and innovation can make to citizens and pointed out that scientific, societal and economic impacts do not have to be contradictory.
One difficulty they discussed was that breakthroughs can take time; measurement frameworks need to widen their scope to improve understanding of where Europe stands on the path to creating impact.
With COST, researchers all over Europe set up or join research networks - also known as COST Actions - and organise their own networking activities on a topic of their choice.
By collaborating with both young and more experienced colleagues from different scientific backgrounds and holding different skillsets, researchers can develop new ideas in their field of science.
These ideas can mature into practical applications or products, new standards or, looking at long-term impact, even breakthroughs.
Another theme that emerged was that networks are strongly correlated to scientific impact. Here, there was recognition of the European Commission’s commitment to connecting scientists and research organisations, companies and governments.
Attendees noted that it is crucial to attract and engage the right partners in innovation and networks. There were calls to strengthen the link between education and research, and promote greater cross-disciplinarity between the arts, humanities, social science and science.
Quality research support also brings excellence, said Andjela Pepic of the COST Targeted Network for research administrators (BESTPRAC). Networking aids their work and ensures that innovation has impact.
However, focussing on impacts presents other problems. COST Action participant Marco Seeber, from the University of Ghent, outlined the “iceberg effect”, where the true dimension of a problem or impact is often misperceived (like an iceberg, only around a tenth of its true size or potential effect or impact is visible). This can lessen the good targeting of resources. Instead, suggested Seeber, evaluators should consider more how research helps promote innovation.
Jack Spaapen, Vice Chair of COST Action ‘the European Network for Research Evaluation in the Social Sciences and Humanities’, described how impact was not always linear, creating another source of potential misevaluation; what benefits one stakeholder does not always benefit another.
Spaapen, a member of the Royal Netherlands Academy of Arts and Sciences, said, “we should measure societal innovation” and create a collaborative impact evaluation model that avoids splitting problems or outcomes into silos and instead considers how different interests interact.
Another participant, speaking from an innovation-development perspective, added that it is important to correctly identify the target users when defining a research proposal, a position supported by Klaus Schuch of the COST Scientific Committee.
Following a series of workshops during the event, participants agreed that qualitative and quantitative measurement provided a more accurate picture of a programme’s impact.
There was renewed consensus on the need to focus on societal impacts, to consider the full complexity of possible results and to seek solutions through interdisciplinary research. Networks have a role to play in allowing cross-fertilisation of ideas, they added; there is a need to engage society, researchers and policy makers and mutually listen.
Several attendees welcomed the COST programme’s contribution to the FP9 debate, stressing how the innovation divide must change for Europe to stay competitive. COST was also praised for helping connect scientific communities across Europe. Many of those attending reiterated the need to seek solutions to societal challenges and advised EU decision-makers to use rigorous methodologies and avoid the search for quick fixes.
Concluding the discussion, attendees looked forward to the Bulgarian Council presidency and its expected focus on knowledge transfer and maximising long-term sustainability. COST plans to continue to explore the day’s ideas with stakeholders and future EU Council presidencies. “Impact is the art of managing expectations, reality and perceptions through dialogue,” concluded de Bruin.