Final evaluation of Horizon 2020

Horizon 2020 (H2020), the 8th Framework Programme (FP), ran from 2014 to 2020 with a budget of almost €80 billion. Final, ex-post evaluation has finally been published at the end of January. It is crucial for the preparation of the 10th FP running from 2028-2034.

During this period, it has supported more than 35,000 projects submitted by interested parties from 177 countries, and its results have resulted in almost 4,000 patents and trademarks. Scientists funded by the H2020 program have contributed to more than 276,000 peer-reviewed articles, and the programme has supported 33 Nobel Prize winners.

Globally, H2020 is also the second largest source of funding dedicated to climate change research with 64.4% of its budget invested in sustainable development, however, only 32% was dedicated to fighting climate change and the EC wishes for reaching the targetted 35% in the 10th FP.

The Commission claims that "each euro in costs linked to the programme will ultimately bring five euros in benefits to EU citizens by 2040".  Also, that "it fuelled a remarkable 20% additional growth in employment and a 30% increase in turnover and total assets for participating firms". And finally, that the H2020 "supported the mobility of close to 50,000 researchers across sectors and countries".

However, "the real-world impact of these achievements is hard to measure. Research can take years to deliver societal impact, making it difficult to measure progress. Plus, the review found Horizon 2020 impact monitoring used weak indicators and was too narrow in scope" (source: Science-Business).

Recommendations for improvement as based on the reports and major stakeholders commentaries:

• bigger budget - 74% proposals were judged as “above the quality threshold” in each call than could have been funded. The budget would have to be tripled to cover the needs (source: Science-Business). Some advise that the 10th FP should be 10 times bigger to meet the needs (source: Science-Business)

• broader participation - despite the Widening measures, half of the funding went to just 4 countries! DE, UK, FR and ES. Participation of Widening countries have risen only slightly in comparison to 7th FP.

• further simplification and reduction of the administrative burden, even though the average time to go from submitting a grant application to an award fell from 313 days in FP7 to 187 days during Horizon 2020. 

• reinforcement of the dissemination, exploitation and deployment of results

• support for the participation of women - the number of women on proposal evaluation panels was 42%, above the 40% target, but women accounted for 43% of those on scientific advisory panels, below the 50% target. The number of female researchers participating in projects was just 23%.

• enhancement of synergies with other initiatives at EU, national and regional level

• balance between fundamental and applied research - as the EU framework programmes become increasingly innovation and policy-driven, more and more funding is dedicated to applied research and innovation projects, at the expense of fundamental research and smaller projects 

• integration of social sciences and humanities and greater interdisciplinarity - critics mention "an overly technological focus of the programme" at the expense of the societal impact (source: Science-Business); what is more, the SSH´s potential was insufficiently explored in the evaluation

Fortunately, the interim report published in 2017 resulted in many improvements in the ongoing Horizon Europe (9th FP), especially in the realm of simplification and reduction of administrative burden (the increased use of the lump-sum-funding model) and dissemination of results. At the same time, the report warns, that "the error rate in expenditure reporting during projects was still too high", so the simplification should have its limits.

Altogether, the basis for the ex-post evaluation have been: over 1,000 interviews with project beneficiaries, national authorities and implementing bodies, as well as surveys of both successful and unsuccessful participants, combines extensive quantitative and qualitative analyses and builds on a large open public consultation with close to 2000 replies.

These reports are available:

Staff Working Document

Commission Report of the final evaluation of Horizon 2020

Annexes 1-6 of the final evaluation of Horizon 2020

Executive Summary of the final evaluation of Horizon 2020

Photo: pixabay