1. No category

 COMMITMENT and COHERENCE Ex‐Post‐Evaluation of the 7th EU Framework Programme (2007‐2013) Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 1
COMMITMENT and COHERENCE
essential ingredients for success in science and innovation Ex‐Post‐Evaluation of the 7th EU Framework Programme (2007‐2013) Chair: Rapporteur: Assistant: Louise O. Fresco André Martinuzzi Adele Wiman Members of the High Level Expert Group: Louise O. Fresco (The Netherlands) ‐ Chair André Martinuzzi (Austria) ‐ Rapporteur Maria Anvret (Sweden) María Bustelo (Spain) Eugenijus Butkus (Lithuania) Michel Cosnard (France) Arvid Hallen (Norway) Yuko Harayama (Japan) Sabine Herlitschka (Austria) Stefan Kuhlmann (Germany) Vesselina Nedeltcheva (Bulgaria) Richard Fowler Pelly (United Kingdom) picture © Marco2811 (Fotolia) Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 2
Content 1. Executive Summary ..................................................................................................... 4 2. Background and introduction .................................................................................... 12 3. Design, implementation and outcomes of FP7 .......................................................... 14 3.1. Aims and objectives of FP7 .......................................................................................... 14 3.2. Continuity between FP6 and FP7 ................................................................................. 16 3.3. Budget allocation in FP7 ............................................................................................... 18 3.4. FP7‐COOPERATION – stimulating EU wide collaborative research ............................. 19 3.5. FP7‐IDEAS – fostering EU wide research excellence .................................................... 20 3.6. FP7‐PEOPLE – building human resources, mobility and networks .............................. 21 3.7. FP7‐CAPACITIES – addressing specific needs in the innovation systems .................... 22 3.8. Agenda setting in FP7 ................................................................................................... 23 3.9. Success rates and success factors of FP7 proposals .................................................... 24 3.10. Organizations participating in FP7 ............................................................................... 26 3.11. Country participation in FP7 ........................................................................................ 32 3.12. Monitoring and evaluation of FP7 ............................................................................... 38 4. FP7 impacts on European excellence in science ......................................................... 40 4.1. Scientific excellence as a key concept of FP7 ............................................................... 40 4.2. Scientific output and impact ........................................................................................ 41 4.3. Cases of outstanding scientific achievement ............................................................... 42 4.4. European excellence on a global scale and the role of FP7 ......................................... 44 5. FP7 impacts on European research and innovation systems ...................................... 47 5.1. Contribution of FP7 to foster collaboration in research and innovation ..................... 48 5.2. Contribution of FP7 to strengthen research capability and capacity .......................... 50 5.3. Contribution of FP7 to mobility of researchers ........................................................... 51 5.4. Contribution of FP7 to international collaboration (with ‘Third Countries’) ............... 54 5.5. Contribution of FP7 to ERA and impacts on national innovation systems .................. 56 5.6. FP7 impacts on policy coherence ................................................................................. 57 6. FP7 impacts on value creation and economic growth ................................................ 59 6.1. Estimation of macro‐economic effects, growth and jobs ............................................ 59 6.2. Market impacts through successful implementation of outcomes ............................. 60 6.3. Effects on European Competitiveness ......................................................................... 61 6.4. Supporting European Industrial Base and Competitiveness ........................................ 63 6.5. Impact on the competitiveness of European SMEs ..................................................... 65 Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 7. 3
FP7 impacts on citizens and society ........................................................................... 66 7.1. The roles of citizens and civil society in FP7 ................................................................ 66 7.2. FP7 impacts on gender issues ...................................................................................... 70 7.3. Wider societal impacts of FP7 ...................................................................................... 73 8. Looking back and looking foreward ........................................................................... 76 8.1. Follow up on the FP7 mid‐term evaluation ................................................................. 76 8.2. Outlook to HORIZON 2020 ........................................................................................... 80 9. Annex ........................................................................................................................ 85 9.1. Profiles and CVs of the members of High Level Expert Group ..................................... 85 9.2. Data sources processed in preparing this report ......................................................... 88 9.3. List of references .......................................................................................................... 95 9.4. Experts consulted in preparation of this report ........................................................... 97 9.5. Abbreviations ............................................................................................................... 99 9.6. Twelve myths about FP7 ............................................................................................ 101 9.7. Institutional setting of FP7 ......................................................................................... 103 9.8. Who benefited from FP7 (by type of organization) ................................................... 105 9.9. Success rates and success factors of FP7 sub‐programmes ...................................... 107 9.10. Gender analyses ......................................................................................................... 114 9.11. Mobility of researchers in FP7‐PEOPLE ...................................................................... 119 9.12. International collaboration with ‘Third Countries’ in FP7 ......................................... 120 9.13. Annual Budget of FP6, FP7 and HORIZON 2020 by sub‐programmes ....................... 121 Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 1.
4
Executive Summary
The 7th EU Framework Programme (FP7) was one of the largest RTD programmes in the world. It accounts for the third largest share of the European Union (EU) budget and was the main financial instrument to build the European Research Area. FP7 was thus a major investment in knowledge, innovation and human capital in order to increase the potential for economic growth and to strengthen European competitiveness. This strong commitment to a European added value in research and innovation helped to build up excellent research networks, achieving outcomes faster and addressing problems from a range of perspectives, disciplines and research cultures. It is widely accepted in the business, science and education communities that, without this commitment, Europe would run the risk of losing a lot of excellent science and undermine its competitive position in innovation. It is widely accepted that a strong commitment to financing research and innovation as a long‐term investment is an indispensable condition for success and that coherence is a prerequisite for the design and implementation of effective and efficient policies and programmes. FP7 covered different themes and disciplines, addressed different stages of research and innovation chains and involved a broad diversity of stakeholders and societal groups. Given this broad scope, coherence within the programme and among its components was key. Furthermore, coherence with other policies and programmes at an EU‐level (e.g. the structural funds, growth and competitiveness policies) and at Member State level (e.g. national science and innovation programmes) is necessary to establish effective policy mixes. Following the principles of good governance regular evaluations play an important role in assessing coherence and ensuring that high impacts of publicly funded programmes materialise. The European Parliament Decision and European Council decision setting up FP7 stipulated that two years after the completion of FP7 the Commission shall carry out an external evaluation by independent experts of FP7 rationale, implementation and achievements. This report presents the findings of this ex‐post evaluation as well as recommendations for the next Framework Programme (HORIZON 2020) and RTD policies and programmes at the European and national level more generally. It informs the European Parliament, the Council, Member States, the Directorate General (DG) for Research and Innovation and various other DGs, the research community and the general public about the achievements of FP7 and challenges ahead. It aims to contribute to the continuous improvement of the design and implementation of the European Framework Programmes in general and HORIZON 2020 in particular. In contrast to the interim evaluation of FP7, this report puts a special emphasis on the impacts of FP7 on scientific excellence, economic growth, jobs and competitiveness, on the European innovation system and society at large. The findings and conclusions presented in this report are based on a range of sources of evidence. These include the programme structure of FP7, the EC budget allocations to different types of organisations and different regions, the success rates of proposals and the collaboration networks established by FP7 evaluated on the basis of CORDIS data (the Community Research and Development Information System) and partly confidential data provided by DG R&I (e.g. proposal data). Another important source was the 120+ reports of evaluation studies that were contracted by DG R&I and carried out by a number of professional evaluators and experts. For the first time, these evaluation reports were assembled in a structured repository that enabled synthesis of the evaluation findings from different sources. In addition, more than 50 experts from the EU Member States, the European Commission, umbrella organisations and national contact points were consulted. Last but not least, this report builds on the knowledge, experiences and expertise of the members of this High Level Expert Group. It also draws on the findings and recommendations of the FP7 mid‐term evaluation and addresses several new issues from an ex‐post perspective. Some impacts can already be assessed using quantitative data while many others can only be evaluated from a qualitative perspective or haven’t yet fully materialised to the extent that this evaluation could provide final conclusions. In these latter two cases a triangulation of different sources was used to provide an indication of trends and future pathways. After presenting the facts, figures and main achievements of FP7, this executive summary highlights the five key recommendations of the High Level Expert Group. More in‐depth analyses and elaborations on the recommendations can be found in the full version of this report. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 5
FP7 at a glance
FP7 was longer and larger than previous Framework Programmes. It represented a total voted budget of 55 billion euro, which accounts for an estimated 3% of total RTD expenditure in Europe or 25% of competitive funding. Consequently, it offered more stable and predictable funding opportunities for research and innovation on a European level than ever before. Over the seven years duration of FP7, more than 139.000 research proposals were submitted, out of which 25.000 projects of highest quality were selected and received funding. The most important groups among the 29.000 organizations participating in FP7 were universities (44% of the FP7 funding), research and technology organizations (27%), large private companies (11%) and SMEs (13%), while the public sector (3%) and civil society organizations (2%) played a minor role. FP7 was built on a vast experience of designing and implementing pan‐European research and innovation programmes and managed to balance continuity and adaptability. Similar to its predecessors, FP7 aimed to strengthen the European Research Area by co‐funding RTD projects with an explicit European added value; improving researchers' qualifications and supporting their careers by promoting their mobility; stimulating competitiveness and growth through joint initiatives of research organisations and the private and the public sector; and delivering positive societal impacts in a broad diversity of themes. At the same time a number of new features were implemented in FP7. For instance, academic research was reinforced by establishing the FP7‐IDEAS programme (ERC) that supported individual top‐level researchers from every scientific discipline carrying out excellent frontier research; and the needs of industry were addressed specifically by the Joint Technology Initiatives (JTI) that built on the preparatory work of the European Technology Platforms and allowed easier and more effective collaboration. From a participant's or applicant's perspective, FP7 was an open system that allowed more than 21.000 organisations, which had not participated in the previous FP6, to receive EU funding for RTD. At the same time, concentration effects in the RTD centres of Europe occurred, as is illustrated by the fact that the Top‐500 organisations in FP7 obtained 60% of the total EC contributions. The EU Member State participation patterns reflect the size, diversity and maturity of national science and innovation systems: high shares of EU funding are allocated to large, research intensive countries like France, Germany, the Netherlands, and the United Kingdom. These countries often host centres of excellence that have made substantial investments in acquiring and maintaining top‐level qualified human resources and professional support structures. In contrast, Mediterranean countries that suffer from the economic crisis and high unemployment rates reduced their public RTD expenditures. While FP7 could not compensate this loss, it still provided opportunities for researchers through mobility and cooperative projects. The share of FP7 funding for organizations from new EU Member States, as well as the success rates of proposals coordinated by researchers from these countries, were significantly lower. These lower shares were not caused by a bias against the new EU Member States, but rather by a comparably high number of weak proposals submitted by, or with partners from the EU‐13. However, since the science and innovation funds on national level are also substantially lower in these countries, FP7 played a more important role in relative terms, especially in competitive funding. Given the fact that FP7 only accounts for a small proportion of total RTD expenditure in Europe, its economic impacts are quite substantial. Through short‐term leverage effects and long‐term multiplier effects each euro spent by the European Commission on FP7 generated approximately 11 euro of estimated direct and indirect economic effects through innovations, new technologies and products. In total, the indirect economic effects of FP7 can be estimated at aprproximately 500 billion euro over a period of 25 years, giving an additional annual European GDP of 20 billion euro. When translating these economic impacts into effects on employment, FP7 directly created 1,3 million person‐years within the projects funded (over a period of ten years) and indirectly 4 million person‐years over a period of 25 years. There is also evidence of positive impacts in terms of micro‐economic effects with participating enterprises reporting innovative product developments, increased turnover, improved productivity and competitiveness. However, it is still too early to make a final assessment of the market impact of FP7 projects. Beyond economic effects and job creation, a number of qualitative impacts were also achieved by FP7. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 6
Main achievements
The key achievements of FP7, detailed below, are mirrored in quantitative data processed for preparing this report, while others are of qualitative nature and are based on the judgements of the expert panel and a number of additional experts. Areas of concern remain and are referred to in the conclusions and recommendations section as well as throughout the report in blue shaded sections. 1. Encouraged scientific excellence on individual and institutional level. FP7‐IDEAS demonstrated its ability to attract excellent researchers and become a benchmark of individual excellence. FP7‐PEOPLE has set a European standard for doctoral training of a new generation of excellent scientists. FP7‐COOPERATION facilitated transnational collaboration and thus provided a platform for the best minds to work together in order to contribute to solving major societal challenges. FP7‐CAPACITIES supported the involvement of excellent organizations from the SME sector, civil society, new EU Member States and developing countries in European research. 2. Promoted ground‐breaking research through a novel programme FP7‐IDEAS (ERC). The focus on supporting frontier research which, by definition, can be a risky endeavour, was enhanced. The number of publications in top rated scientific journals that acknowledge ERC funding, Nobel Prizes and Fields medals received by ERC grantees all attest to ERC grants becoming a mark of scientific excellence. 3. Engaged industry and SMEs strategically. Both, large corporations and SMEs have been involved extensively through increased public‐private‐partnerships, in particular the development of JTIs, and through a range of SME specific programmes. This has underlined FP7’s intended role of fostering Europe’s innovation‐based competitiveness. 4. Reinforced a new mode of collaboration and an open innovation framework. This was achieved through a more decentralized approach to the design, structure and direction of projects across the ERC, JTIs and the EIT. During the FP7 period, the European Commission has adapted the programme to the economic crisis and has responded to the a more generalised pursuit of open innovation. 5. Strengthened the European Research Area by catalysing a culture of cooperation and constructing comprehensive networks fit to address thematic challenges. A unique capability of cross‐border and cross‐sector cooperation was promoted,with organisations from on average of 6 countries collaborating in projects funded by FP7‐COOPERATION and FP7‐CAPACITIES. 6. Addressed certain societal challenges through research, technology and innovation. FP7‐COOPERATION included society‐relevant themes, such as Health, Energy, Transport and Security, whilst FP7‐CAPACITIES included a specific sub‐programme that was dedicated to "Science in Society". Furthermore, the focus on gender equality evolved from exclusively promoting individual female scientists to facilitating structural change in institutions. 7.
8.
9.
Encouraged harmonisation of national research and innovation systems and policies. In most EU Member States FP7 contributed to scientific excellence, focused on adressing societal challenges, and set standards for research funding mechanisms and selection processes. Through the sub‐programme FP7‐
ERA‐NET the cooperation and coordination of research activities carried out at national or regional level in the Member States and Associated States were intensified through networking of research activities, and the efforts to coordinate research programmes. Stimulated mobility of researchers across Europe. FP7‐PEOPLE has created the necessary conditions for an open labour market of researchers and supported their geographical mobility. Achievements during the FP7 period included fellowships gaining recognition as the best practice of doctoral training and the creation of attractive working conditions for geographically mobile researchers. Promoted investment in European research infrastructures. A combination of the support for the European Strategy Forum Initiatives for Research Infrastructures (ESFRI) and FP7‐CAPACITIES helped to achieve a more coherent and coordinated development and use of European research infrastructures. 10. Reached a critical mass of research across the European landscape and worldwide. Human and financial resources were made available to attract many organizations and individuals to collaborate with or work at European research institutions. Furthermore, a research programme of such scale has helped to put research on the public agenda and to show that research can be an instrument for economic and social development. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 7
Conclusions and recommendations
The ex‐post evaluation of FP7 makes the following recommendations to strengthen Europe’s position as a hub of global innovation and knowledge generation: (a)
(b)
(c)
(d)
(e)
Ensure focus on critical challenges and opportunities in the global context Align research and innovation instruments and agendas in Europe Integrate the key components of the Framework Programmes more effectively Bring science closer to the European people Establish strategic programme monitoring and evaluation More detailed suggestions are provided throughout the report in blue shaded sections. (a)
Ensure focus on critical challenges and opportunities in the global context Rationale: Stimulate economic growth and jobs in a future‐oriented dynamic European knowledge and innovation based economy. Background and analysis: FP7 has been the largest cooperative research and innovation programme world‐
wide, both in size and ambition. At the same time, FP7 funded research and innovation activities have been undertaken in an environment of “coopetition” (requiring the balancing of cooperation and competition imperatives). For HORIZON 2020 and its successor programmes it will be important to “think big” in focussing on the strategically important and critical challenges and opportunities of our times, while at the same time reinforcing the need for cooperation and recognising that global competition in key areas is getting fiercer. Developing the European model of a future‐oriented knowledge and innovation based economy requires Europe, by means of its respective programmes, to focus on a number of key strategic areas, as well as ensure lean and fast implementation procedures, that reflect the dynamics in key areas of global coopetition. Implementation: In order to reflect critical challenges and opportunities of our time, HORIZON 2020 and its successors should address overarching topics that help to further develop Europe’s profile as a dynamic and future‐oriented knowledge and innovation based economy in a global context more strategically. Future economic growth, jobs and social development in Europe depend on its leading competitive position in science and technologies and the effective exploitation of these discoveries. Excellent research, a vibrant innovation value chain and disruptive innovations open up fundamentally new paths of technological and product development. The main challenges for the years ahead are to identify a number of key areas in which Europe can play a truly leading role on a global scale, to ensure that the sources of this competitive advantage are strategically built up and developed, and to increase Europe's attractiveness for leading researchers and innovators in these areas. Europe should make the most of its immense potential by continuing to bring into science and innovation the different key actors in order to get the best value from the available talent, including women. The engagement of the private sector through large industrial players is critical to the success of an EU Research and Innovation programme. The EC should therefore establish a permanent mechanism of dialogue with the private sector, commit to continuous improvement during the lifetime of HORIZON 2020 and develop a strong European Innovation Strategy. In particular, the instrument of JTIs should be further strengthened and the contractual framework should be simplified. Improvements are required to ensure that SMEs play an increasingly important role in the innovation value chain. In addition to existing initiatives, the EC should encourage SME participation in national programmes as they are typically more appropriate to the needs of SMEs, as well as develop a range of indicators at European level to unlock the full potential of SMEs. Europe should build on research and innovation in a more targeted way to address the critical challenges and involve the civil society more broadly to build a socially, economically and environmentally sustainable future. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme (b)
8
Align research and innovation instruments and agendas in Europe Rationale: Increase synergies, effectiveness and efficacy of the European Science, Technology and Innovation System. Background and analysis: FP7 accounted for a small share of RTD investment and human resources in research in Europe. In order to achieve maximum effects with this given budget the Framework Programmes should not be perceived as an independent and detached funding system, but rather as a strategic intervention into the totality of research and innovation systems of the EU and its Member States. Instead of aiming at simplistic multiplier effects, the catalytic impacts of a European framework should be pursued. FP7 showed major progress in this regard: establishing the ERC was a successful intervention that increased competition, transparency and openness in European research. FP7‐PEOPLE helped to link science and industry via the European Industrial Doctorate Programme and the Industry Academia Partnership Programme. The ERA‐NET programme motivated national funding schemes to open up their programmes for applicants from other EU Member States. The FP7‐COOPERATION programme provided a reference scheme for cross‐border cooperation and introduced good management practices and impact orientation in a large number of European universities. The RSFF linked the EU Framework Programmes with funding instruments from the banking sector. At the same time, FP7 also showed some weaknesses: overall, the programme was oriented towards broad, general and rather obvious policy objectives (such as innovation, competitiveness and mobility), but lacked effective integration between them. Moreover, some of these were contradictory to a certain extent (such as project call requirements for highly efficient project structures, on the one hand, and coverage of as many EU countries as possible, on the other). Moreover, there were signs of inconsistencies, competition, lack of coherence and overlap of elements of FP7 and national programmes. This also occurred on the European level between the Framework Programmes and research and innovation efforts in other Directorates. Implementation: The Framework Programmes should combine strong policy objectives with decentralised and flexible implementation procedures. Implicit assumptions about how Framework Programmes work should be made explicit and published. Development of research themes and topics should focus on defining a number of concrete goals, while approaches and methods to accomplish these goals should be determined on a bottom‐
up basis. At the same time, it should be ensured that there is enough room for the unforeseen and social innovation, which may emerge both in fundamental and applied research.
In order to align the Framework Programmes with related policies and programmes at the European level the potential of a "Common Science, Technology and Innovation Policy" across the EU should be explored. Structural Funds should be used in a complimentary way to bring research facilities and salary levels of Eastern and Southern Europe to a competitive level; regional centres of excellence in these countries should foster specialization and provide attractive opportunities for researchers. A dedicated science, technology and innovation support fund within the Structural Funds is recommended. National and EU programmes should align their research priorities better using appropriate tools and incentives (such as pooling of funding in order to improve leverage effects, considering the innovation supply chain, shared databases and support of mobility). In a broader perspective also other policies and regulations should be more supportive towards innovation. The importance of quality standards for research is underlined. It is recommended that by establishing an EU‐wide quality stamp for outstanding scientific and enterprise driven proposals, successful proposers would be allowed to apply for funding at the national level in a streamlined manner.
(c)
Integrate the key components of the Framework Programmes more effectively Rationale: Efficiency, synergies and coherence of the Framework Programmes. Background and analysis: FP7 showed a complexity that stems from the long history of Framework Programmes and the expertise and interests of different DGs. Furthermore, special sub‐programmes (such as the SME, the International Cooperation and the Science in Society Programme in FP7‐CAPACITIES), as well as major programme parts (such as JTIs meeting the needs of industry or FP7‐IDEAS for science) were gradually introduced in response to particular needs of stakeholder groups. As a consequence, fragmentation and the Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 9
emergence of ‘silos’ have tended to threaten efficiency and coherence of the Framework Programmes in terms of compartmentalization and duplication of themes. In addition, some successful elements of FP7 were provided mainly through certain sub‐programmes, even though they would be equally useful in other sub‐
programmes. For example, mobility of researchers is promoted through FP7‐PEOPLE, but is not a key feature of FP7‐COOPERATION projects. On the other hand, some integration measures were implemented, e.g. the introduction of joint / coordinated calls and the PROOF OF CONCEPT scheme in FP7‐IDEAS, which supports ERC grant holders in generating innovative potential from their ERC‐funded project. Recent outsourcing of implementation, while needed, adds to the danger of further fragmentation as well as reduced transparency. Implementation: To increase efficiency and coherence of the Framework Programmes, synergy potentials should be assessed and implemented, while duplications between the different specific programmes and sub‐
programmes should be avoided in the future. The programme structure should allow budget transfers between programme years, particularly in those programmes that are open to all disciplines and themes, and that implement a bottom‐up approach (such as FP7‐IDEAS and FP7‐PEOPLE). Effective coordination processes between the agencies in charge of implementing HORIZON 2020 should be established to minimise fragmentation and ensure a high level of transparency. Funding instruments should be harmonised with a special emphasis on fostering linkages between the specific programmes (enabling the use of FP7‐PEOPLE funding opportunities for the preparation of FP7‐IDEAS proposals). Future Framework Programmes will benefit from making successful elements available across the programme (e.g. foster researcher exchanges in collaborative projects in addition to the Marie Curie Actions). (d)
Bring science closer to the European people Rationale: Increasing trust, acceptance, and ownership of research, and ensuring its relevance and creativity. Background and analysis: Research, industry, policy making and civil society combine essential complementary assets. In order to achieve a generally positive perception of science and better adoption of new knowledge and innovations, European research should increase citizen trust. FP7 has already addressed these challenges, but not in a substantial way. Two sub‐programmes addressed issues of high importance for citizens and society, but budgets of both were comparatively small: The Science in Society programme accounted for 0,65% of the FP7 budget, while 1,30% was allocated to the theme “Socio‐Economic Sciences and Humanities” in FP7‐
COOPERATION. Civil society organizations have been scarcely included in relevant decision‐making bodies, such as evaluation boards or expert groups. Furthermore, the gender balance and the representation of women on all levels (e.g. as grantees, leading researchers, coordinators, evaluators and experts), and the integration of gender into the research content was not substantially improved since the interim evaluation. During FP7 societal concerns regarding research and innovation (such as participation, open access, ethics) were integrated into a coherent and broadly accepted framework. Implementation: Future Framework Programmes should involve stakeholders more in building an evidence driven and science based society of the future and integrate civil society organizations in a more substantial way (for example by their inclusion in evaluation panels or by particular partnership programmes). Citizens and stakeholders should be engaged in a dialogue about the purpose and benefits of research and the way it is conducted. This entails the development of incentives for science communication as well as the establishment of particular support for more strategic measures of communication with different audiences. It should be encouraged to design projects in a way that supports the formation of linkages between researchers, citizens and policy‐makers. More tailored and targeted dissemination activities should be enforced and monitored. It is recommended to combine the current initiatives for agenda setting and stakeholder involvement in a sub‐
programme dedicated to “Visions and Agendas”. A European integrity code for scientists should foster people’s trust in science and innovation. Furthermore the dissemination of gender equality, diversity, ethics and participation should be fostered. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 10
(e)
Establish strategic programme monitoring and evaluation Rationale: Improve governance, accountability and performance of the Framework Programmes and enable better, evidence‐based and strategic decision making and foster continuous learning. Background and analysis: Given the size of research expenditures of FP7, evidence‐based decision‐making, in combination with adequate governance and institutional learning, are indispensable in order to ensure the continuous improvement of programme design and implementation. While evaluation activities have been established as routine procedures in recent years, their management was still organised in a very fragmented fashion. There are indications that evaluations do not fulfil their potential as instruments for facilitating continuous learning processes and the development of a solid strategic intelligence. There has been hardly any consolidation, validation, comparison or synthesis of information provided by the more than 120 independent evaluations of FP7 themes and areas. While data sets for monitoring (such as OpenAIRE, PIC codes or the improved data structure of SESAM) have been improved during the running time of FP7, these data sets have hardly been used as instruments for the systematic generation of knowledge and, more importantly, strategic intelligence. Some important information was still lacking (e.g. tracing careers of researchers, dissemination activities, activities to link research and policy making). High fluctuation rates in advisory bodies further impede continuous build‐up of knowledge at the individual level. Recent outsourcing of programme implementation activities to agencies has further increased the existing demand for monitoring, governance and control. The need to ensure and improve governance and learning capacities in HORIZON 2020 calls for the development of centralised competence and capacity in DG R&I. All these measures do not only aim at safeguarding accountability, but also at fostering evidence‐based decision‐making in further improving the programme design, implementation and outcomes, as such serving as basis for strategic decision making. Implementation: Considering that the Framework Programmes have consistently been the third largest budget of the European Union, a strategic and professional monitoring and evaluation system is required that increases transparency and serves as a comprehensive and trusted source of evidence‐based decision making. Based on precisely defined targets and a sound understanding of the theory of change, certain key data sets should be developed (e.g. tracing of individual researchers, gender monitoring and proposal evaluation results). It is also recommended that evaluation purposes, criteria, questions and report formats should be harmonized. The wide range of individual evaluations should be better planned and utilized to build up a coherent knowledge base that allows for continuous improvement of the Framework Programme. More focus should be given to quality control and standardisation of data sets in contracted evaluations to ensure they can be used as the evidence base for strategic decisions. Furthermore, a rigorous approach to evaluation syntheses and meta‐evaluations will enable systematic access to findings and ensure a better quality of evaluation studies. Establishing such a monitoring and evaluation system will require additional budget allocation and investment in personnel within DG R&I, but savings can certainly be made in the overall cost of evaluations in the long run. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 11
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 2.
12
Background and introduction
This report fulfils the obligation set out by the legal basis of FP7 to carry out an independent external evaluation of the rationale, implementation and the achievements of the Framework Programme two years after its completion. Since the last work programme of FP7 was implemented in 2013, the High Level Expert Group for the ex‐post evaluation of FP7 was set up in November 2014 and tasked with carrying out this evaluation. The group is comprised of 12 members, including a Chair and Rapporteur (for the full list and member’s profiles please see Annex 9.1.). The High Level Expert Group has drafted this report as a collective endeavour based on available and collected evidence (for a full list of data sources please see Annex 9.2.), hearings of internal and external experts (for full list of experts consulted in the preparation of this report please see Annex 9.4.) and the expertise and judgements of the members of the group. The objective of this report is to evaluate the implementation and effectiveness of FP7 and to summarize what FP7 has achieved and what shortcomings in its design and implementation could be identified. It aims to provide key insights that will help guide the future design of European Research Framework Programmes as well as Science, Technology and Innovation (STI) policies on the EU and national levels. The point of reference for this evaluation is the aims and objectives of FP7 as set out at the time of its adoption, wider EU policy goals regarding the European Research Area and the Europe 2020 Strategy. Consideration has also been given to the wider scientific, economic and societal impacts of FP7. It provides evidence‐based insights for the European Parliament and the Council, the European Commission and supports EU Member States, the research community, the general public, the media, civil society organisations and European industry, among other stakeholders, in the debate about the future of European research and innovation policy and the role of science and innovation as a stimulus for growth and global competitiveness. Although this report is an ex‐post evaluation, it is future looking and provides specific recommendations for more appropriate and effective implementation of future European Framework Programmes. Where appropriate, comparisons with the previous FP6 are made to illustrate trends. Certain aspects have been deliberately excluded from the scope of this report. The report does not assess the budget shares between different sub‐programmes, themes or areas, nor does it discuss the options for a completely different usage of the FP7 budget. It makes recommendations for further implementation of HORIZON 2020 and subsequent programmes but does not discuss details of HORIZON 2020 and its implementation (this will be the core of the mid‐term evaluation of HORZION 2020). It addresses FP7 as an STI programme and not the institutions that stand behind its design and implementation (e.g. DG R&I as the coordinating organisation or the implementing agencies). In contrast to many other evaluations, this report focuses on questions related to whether FP7 has been successful in achieving its aims and is thus not structured according to the specific programmes of FP7. Rather, it is structured according to key evaluation aspects such as the appropriateness of the programme design, implementation and programme; and the evidence of outcomes and impacts. The report contains 6 sections that are each dedicated to one of the key evaluation aspects. Each chapter discusses the rationale and objectives that are set in FP7, presents and discusses evidence and ends with conclusions and recommendations (highlighted in blue boxes). The Annex provides in‐depth information on a number of selected issues which are summarized in the following chapters. Chapter 3 focuses on the design, implementation and outcomes of FP7. It starts by summarizing and discussing the overarching goals and rationale of FP7 as a programme and then presents a comparison between FP6 and FP7 in more detail and an overview of thematic continuity all the way from FP3 to HORIZON 2020. Chapter 3 continues by analysing the main output of FP7 in terms of total budgets allocated, number of projects funded, number of participants who received funding and average EC contributions across FP7‐COOPERATION, FP7‐IDEAS, FP7‐PEOPLE and FP7‐
CAPACITIES. Further on, a number of programme design aspects are discussed, including the agenda setting process in FP7, eligibility, funding schemes and rates, project evaluation procedures. The chapter explores more deeply the success rates of participating organisations across four specific Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 13
programmes and offers an analysis of data on the different types of organisations that participated in FP7, how many of them where newcomers compared to FP6 and from which countries participants came. The degree of concentration of participating organisations and the analysis of the FP7 monitoring and evaluation system are discussed as well as the participation of new EU Member States in FP7. The chapter ends with a discussion on the monitoring and evaluation system and the overall design of the FP7 as a programme. Chapter 4 analyses the contribution of FP7 to fostering excellence in European science in terms of publication output, patent applications and other indicators of high scientific excellence in FP7 funded projects. Cases of outstanding scientific achievement are highlighted. The role of each of the specific programmes in promoting excellence is discussed. Chapter 5 offers insights in response to the question as to what extent FP7 has contributed to strengthening European research and innovation systems. The importance of fostering collaboration across national and institutional borders as well as the role of FP7 in strengthening European research capacities is discussed. The structuring effects and impacts of FP7 on national research systems across the EU and impacts on fostering research policy coherence are explored. The chapter ends with an analysis of FP7 outputs in terms of fostering international collaboration with countries outside the EU in pursuit of scientific excellence and economic competitiveness and an analysis of the impacts of FP7 on researchers’ mobility. Chapter 6 investigates the impacts of FP7 on value creation and economic growth in the EU. The chapter offers an estimation of GDP and job effects and examples of innovations significant for European competitiveness. Broader effects on progress in innovation and competitiveness indicators such as patents, innovation scorecards are presented. Apart from indicators, Chapter 6 presents an overview of the effects the innovative public‐private‐partnerships are having on the European industrial base. Lastly the role and impacts on European SMEs are presented. Chapter 7 captures the effects of FP7 on citizens and society. The relevance of FP7 for citizens and society in different roles are foremost discussed, leading to a discussion on the role of society in research including responsible research and innovation and focusing FP7 funded research on society‐related issues. Furthermore, the extent to which responsible research and innovation has gained traction in FP7 is addressed. FP7 impacts on gender equality and wider societal impacts in terms of addressing sustainable development in FP7 are evaluated. Chapter 8 looks back to the FP7 interim evaluation and to what extent the issues highlighted at the mid‐point of FP7 have been addressed in FP7 and later in HORIZON 2020. The chapter concludes by drawing special attention to the recommendations of this report that are the most relevant to be addressed by HORIZON 2020 in order to achieve the goals the FP7 successor European Research Framework Programme has set and to avoid the shortcomings that have limited FP7’s success. Several Annexes are made available at the end offering a deeper analysis of issues covered including extensive data tables, a map of the institutional setting for FP7 as well as debunking twelve widespread myths about FP7. Throughout this report comma (",") is used as decimal mark and point (“.”) as thousands separator. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 3.
FP7 Design, implementation and outcomes
3.1.
Aims and objectives of FP7
14
Research and innovation processes are characterized by complex interactions, feed‐back loops and variable. The 7th Framework Programme was the largest consolidated effort and investment in European research and innovation. Its overriding aim, as set out in the European Parliament Decision No 1982/2006/EC1, was to “contribute to the Union becoming the world’s leading research area”. FP7 has also been tasked “to strengthen industrial competitiveness and to meet the research needs of other Community policies”. In achieving this aim, the programme has focused “on promoting and investing in world class state‐of‐the‐art research, based primarily upon the principle of excellence in research”. Moreover, the legal basis of FP7 has set out that by investing in a “more stable foundation” for the European Research Area (ERA) a positive contribution to “the social, cultural and economic progress of all Member States” is expected. The role of research in promoting the strategic goal of the European Union to “become the most competitive and dynamic knowledge – based economy in the world, capable of sustainable economic growth with more and better jobs and greater social cohesion” has been acknowledged by the Lisbon Council and further reinstated by the European Council and the European Parliament (Resolution of 10 March 2005 on science and technology — Guidelines for future European Union policy to support research2). Thus, the following objectives of FP7 can be derived: 1. Promoting excellence in research 2. Fostering competitiveness and economic growth 3. Contributing to solving social challenges 4. Strengthening the human potential and researchers’ mobility 5. Fostering transnational research cooperation Research excellence has become an even greater focus in the 7th Framework Programme. The importance of promoting transnational cooperation and strengthening of human potential has remained as the continuous pillar from previous framework programmes and is central to the three key objectives of FP7. However, the 7th Framework Programme expanded on these key objectives by highlighting the need to enhance “the dynamism, creativity and excellence of European research at the frontier of knowledge” – the objective that underlines the new specific programme FP7‐IDEAS. The focus on competitiveness has also been strengthened in FP7. The economic crisis has dictated an even stronger move towards improving research performance in the pursuit of achieving the goals of the Europe 2020 Agenda. In search of new sources to fuel the European economy, the flagship initiative the “Innovation Union” was announced in 2011. Since Europe was lagging behind in R&D spending compared to its global competitors, such as the US and Japan, the Innovation Union called for more investment into strengthening the European knowledge base. The initiative aimed to improve “access to finance for research and innovation” and to “ensure that innovative ideas can be turned into products and services that create growth and jobs”3. As a result, FP7 shifted its focus even more to innovation as a means of fostering European global competitiveness mid‐programme. Supporting science as a means to reach European policy objectives related to societal wellbeing inherent in FP7 is not a new phenomenon compared to previous Framework programmes. However, FP7 marked a milestone by setting explicit expectations for science to contribute to solving some of the pressing challenges 1 European Parliament (2006) Decision No 1982/2006/EC of the European Parliament and of the Council of 18 December 2006 concerning the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007‐2013) 2 European Parliament (2004), European Parliament resolution on science and technology “Guidelines for future European Union policy to support research” (2004/2150(INI)) 3 European Commission (2010) Communication from the Commission on Europe 2020 Flagship Innitiative Innovation Union. SEC(2010) 1161. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 15
the EU faces today. The objectives of FP7 clearly stated its aim to contribute to finding solutions to “climate change and sustainability, the health of Europe’s population and the reinvigoration of the Lisbon strategy”. Strengthening the human potential and researchers’ mobility has been a continuous objective rooted in FP4. Top quality researchers are seen as both key to knowledge production and innovation in the EU, as well as fundamental to creating the ERA. Thus, promoting their mobility and network building is an important imperative. Fostering transnational research cooperation has underpinned FPs since their inception. Trans‐European and global cooperation in research is key to the knowledge exchange needed to solve the most complex of scientific challenges and to create European added value. Despite these far reaching goals, FP7 stressed its complementary role to EU Member States’ and European Industry’s efforts and other Community actions in support of the goals of the Union. It has thus not been the sole instrument of advancing European competitiveness and solving societal challenges. Yet, its role has been pivotal in supporting research that contributes to these aims. FP7, as well as predecessor FPs, is also a result of multi‐annual political negotiations. As such, a programme of this size navigates complex and often competing objectives. FPs in Europe are set to contribute to long term objectives, which cannot be reached within the time frame of one FP. Continuity and change are observed throughout the evolution from one FP to the next (for a discussion of continuity between FP6 and FP7 see Chapter 3.2.). Furthermore, the FPs had to adapt to changes in global society and economy – in the case of FP7 especially in the aftermath of the 2008 economic crisis. Thus, FPs have had to balance between continuity of long term objectives and adaptability to emerging global and local challenges. These evolutionary effects have to be considered and FP7 has to be assessed against the backdrop of the complexity and dynamism of the system which it aims to steer. FP7 has proven to be flexible enough to adapt to the changing environment caused by the financial and economic crisis. This need to adapt was possible to accommodate due to rather general commonly agreed aims. On the other hand, imprecise aims run the risk of not being attained, when they are not operationalized through concrete targets. Furthermore, several more concrete aims (such as contributing to European cohesion, promoting gender equality or supporting sustainable development) were set, without considering potential contradictions with the five explicitly stated broad aims of the programme. To avoid these constraints, HORIZON 2020 and its successor programmes should “think big” and focus on strategically important and critical challenges and opportunities. Programme aims should be made as specific as possible, concrete targets should be set and the coherence of the different explicit and implicit aims should be ensured. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 3.2.
16
Continuity between FP6 and FP7
FP7 was launched in a very different socio‐economic landscape than FP6 in 2005. The Lisbon strategy faced a mid‐term review resulting in the conclusion that most initial goals were not achieved. As a result, the strategy was refocused to promote growth and jobs in parallel to promoting sustainable development. At the same time, the end of FP6 marked the end of the largest process for the enlargement of the European Union to date, with 13 new Member States joining during the 2004 and 2007 enlargements. The changing policy environment was mirrored in the differences between FP6 and FP7 in terms of focus, structure and specific instruments. FP7 broadened the thematic focus compared to FP6. The largest part of FP7 was constituted by the specific programme FP7‐COOPERATION, which was structured around 10 priority themes that saw new themes emerging (e.g. Security), but also exhibited continuity with the themes in FP6. FP7 also demonstrated a stronger orientation towards the notion that research serves wider policy goals in comparison to FP6 in both the choice of thematic priorities and the introduction of new “impact statements” in project proposals. FP6 was much more focused on promoting scientific and technological advancement, strengthening and structuring of the ERA and promoting international cooperation. FP7 also marked a change in the structure of the programme. FP6 was primarily structured around three “instruments“: (1) focusing and integrating European research (including thematic priorities and specific activities targeted at policy support, SME involvement and international cooperation); (2) structuring the ERA (incl. stimulating innovation, transfer of knowledge, developing human resources, research infrastructures and science and society); and (3) strengthening the foundations of the ERA. FP7 was structured into four main programmes which reshuffled and incorporated specific parts of FP6. Promoting international cooperation was streamlined into FP7‐COOPERATION within ten specific thematic priorities. Developing human potential and strengthening research infrastructure, grouped together in FP6, were divided into two separate programmes in FP7 – FP7‐PEOPLE and FP7‐CAPACITIES respectively. The efforts in the ‘Science and Society’ stream were scaled up and continued in FP7 as the FP7‐CAPACITIES themes ‘Research Infrastructures’ and ‘Science in Society’. One of the most novel additions to FP7 was the FP7‐IDEAS programme aimed at exploratory, ambitious frontier research overseen by the European Research Council (ERC). Its predecessor in FP6 could be regarded as the “New and Emerging fields in Science and Technology” (NEST) programme, which relied on researchers to propose projects that could set new directions, in order to promote cutting edge research knowledge in new and interesting research avenues. In FP7, in line with a greater focus on growth, jobs and competitiveness, a new activity – the Joint Technology Initiatives (JTIs) – a form of public‐private partnerships (PPP), were established to implement a number of European Technology Platforms (ETP). These were regarded as an important mechanism to promote industrial research and foster European competitiveness and a central new component of FP7. On the administrative side the budget and the length of the programme was also increased in FP7. In the middle of the FP7 implementation period, a few measures aimed at simplifying the rules and procedures to improve programme effectiveness, attractiveness and accessibility were also introduced. FP7 thus builds on and goes beyond previous Framework Programmes. However, it did see significant changes in the more strategic structuring of the programme and better integration of wider goals (e.g. international cooperation). FP7 also broadened its thematic focus to bring research closer to the policy needs of the EU. New experimental funding modalities such as ERC and JTIs were also introduced. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 17
Thematic continuity between FP3 and HORIZON 2020 FP3
FP4
FP5
Information and
communication
technologies
RACE 2
Acts
ESPRIT 3
ESPRIT 4
TELEMATICS 1C
TELEMATICS 2C
FP7
H2020
Information and
communication
technologies
ICT
Security
Security
FOOD
Food, agriculture
and fisheries, biotechnology
Biotechnology
LIFESCIHEALT
Health
Health
Nanosciences, ‐
technologies, ‐
materials & new
production
techniques
Nano and new
materials
Space
Space
Transport
Transport
FP6
1. Focusing and
integrating European research
IST
IST
Life sciences and
technologies
BIOTECH 1
BIOTECH 2
BIOMED 1
BIOMED 2
AIR
FAIR
Quality of life
Industrial technologies
BRITE‐EURAM 2
BRITE‐EURAM 3
MAT
SMT
NMP
GROWTH
AEROSPACE
Transport
SUSTDEV
TRANSPORT
Food
Energy
Energy
Environment
Climate
Environment
ENV 1C
ENV 2C
MAST 2
MAST 3
Future and
emerging
technologies
EESD
NEST
ERC
FP7‐IDEAS (ERC)
Energy
JOULE 2
JOULE
Research potential of Convergence
Regions
THERMIE
Cooperations with
third countries and
international organisations
STD 3
INCO
International cooperation
CITIZENS
Socio‐economic
sciences and
humanities
SME
Research for the
benefit of SMEs
Innovation in SMEs
RSFF*
Access to risk
finance
FP7‐PEOPLE
(Marie Curie actions)
Marie Skłodowska‐Curie actions
Research infrastructures
Research infrastructures
Science in Society
Science with
and for
society
INCO 2
INCO
Targeted socio‐
economic research
IHP
SUPPORT
TSER
2. Structuring the ERA
ETAN
MOBILITY
Stimulation of training
& mobility of
resaerchers
INNOVATION
INNOVATION
INFRASTRUCTURE
HCM
TMR
SOCIETY
3. Strengthening the
foundations of the
ERA
Dissemination and
exploitation of results
POLICIES
INNOVATON
Inclusive societies
Support coherent
development of
research policies
Spreading
excellence
Regions of
knowledge
COORDINATION
RENA
Research and training
in the nuclear sector
Euratom
FUSION 11C
FUSION 12C
NFS 1
NFS 2
EURATOM
EURATOM
Coordination of
research activities
JRC
Knowledge in society**
EIT
EURATOM
EURATOM
FP7‐
COOPERATION
H2020 Societal
challenges
FP7‐CAPACITIES
H2020 Industrial leadership
H2020 Excellent
science
*RSFF: developed by the European Commission (EC) and the European Investment Bank (EIB), succeeded by H2020 specific objective “Access to Risk Finance”
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 3.3.
18
Budget allocation in FP7
Total FP7 voted budget was over 55 billion euro4. Compared to predecessor Framework Programmes, FP7 was also substantially longer ‐ the duration of FP5 and FP6 were 4 years each, while the duration of FP7 was 7 years. FP7 also offered a 66% higher annual European Commission funding compared to FP6; FP6 funding was 4,8 billion per year while FP7 was 8 billion euro per year. 81% of the voted budget (44,6 billion euro) was allocated to four specific programmes, namely FP7‐
COOPERATION, FP7‐IDEAS, FP7‐PEOPLE and FP7‐CAPACITIES. All four specific programmes were implemented through annual Work Programmes containing a number of competitive calls for proposals. These four specific programmes are in the focus of this ex‐post evaluation. The remaining 19% of the total FP7 budget was used to cover administrative expenditures of the European Commission (EC) associated with the implementation of FP7, as well as other instruments namely the Risk Sharing Finance Facility (RSFF) (in collaboration with the European Investment Bank), the International Thermonuclear Experimental Reactor (ITER), Nuclear Fusion and Fission Research (e.g. FP7 Specific Programme Euratom) and the Joint Research Center's (JRC) direct actions. These activities and associated costs are not within the scope of this evaluation. For the purpose of this evaluation study detailed data on FP7 research projects and participating organizations in the four specific programmes was made available by Directorate General for Research and Innovation (DG R&I), the eCORDA database. This dataset covers the following: 


approx. 44,6 billion euro of EC contribution to approx. 25.000 projects involving approx. 29.000 organizations. In addition to the data from eCORDA, additional data on FP6 funded projects and participating organizations, as well as data on proposals submitted for FP7 funding, was provided by DG R&I. This data was used to analyse potential overlaps between FP6 and FP7 in terms of participating organizations and for gaining insights into the potential success factors of research proposals. FP7 was substantially longer and larger than previous FPs. It offered more stable and predictable funding opportunities for research and innovation on European level than previously available. The increase in FP funding mirrored a substantial commitment of the European Union for promoting research and innovation. Collaborative research was the key element of FP7 since approximatley half of the voted budget and 64% of the research project funding was allocated to FP7‐COOPERATION. 4 European Commission (2013), Development of Community Research – Commitments 1984‐2013. Available at: http://ec.europa.eu/research/fp7/pdf/fp‐1984‐2013_en.pdf Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 3.4.
19
FP7-COOPERATION – stimulating EU wide collaborative research
64% of the EC contribution to research projects (approx 28 billion euro) was allocated to the specific programme FP7‐COOPERATION. In this specific progamme a total of 7.912 projects were funded. The project size ranged from 1,5 million euro to more than 50 million euro. The programme was constituted by ten thematic areas ("Themes") covering a broad variety of societal and policy relevant challenges (e.g. Health, Environment, Security) and areas in which major innovations were expected (e.g. ICT, NMP, Space). In addition, the ERA‐NET programme and the Joint Technology Initiatives (JTIs) can be subsumed into FP7‐COOPERATION (although they show different funding characteristics). The highest share of EU funding was allocated to projects in the field of ICT (18%) and Health (11%) followed by NMP (7%) and Transport (5%) themes. The average EC contribution per project in these themes lies between 3 and 5 million euro. The average EU contribution per partner organization was between 300.000 and 425.000 euro. The smallest share of the EC contribution, the lowest number of funded projects and the lowest EC contribution per project and per organization is found in theme “Social Sciences, Economics and Humanities” (for further details see Chapter 7.1.). The EC contribution to individual projects was allocated on the basis of competitive proposals addressing specific topics published in annual work programmes. In total, approx 3.200 topics were published in FP7‐
COOPERATION work programmes. Every proposed project had to clearly address one of these topics, involve partners from at least three different countries and focus on research, innovation, networking or dissemination. The thematic priorities, topic descriptions and budget allocation per topic was developed and decided upon by the European Commission in a top‐down process (for the details of the programming process see Chapter 3.8.). Submitted proposals were reviewed and awarded points by independent evaluators based on three criteria: (1) excellence; (2) implementation; and (3) potential impacts. FP7‐COOPERATION
Total EC contribution % of EC (in million contrib.
euro)
number of projects
Theme 01 ‐ Health
Theme 02 ‐ KBBE
Theme 03 ‐ ICT
Theme 04 ‐ NMP
Theme 05 ‐ Energy
Theme 06 ‐ Environment
Theme 07 ‐ Transport
Theme 08 ‐ SSH
Theme 09 ‐ Space
Theme 10 ‐ Security
ERANET
JTI
Subtotal FP7‐COOPERATION
4.792
1.851
7.877
3.239
1.707
1.719
2.284
580
713
1.295
313
1.966
28.336
1.008
516
2.328
805
368
494
719
253
267
314
104
736
7.912
11%
4%
18%
7%
4%
4%
5%
1%
2%
3%
1%
4%
64%
average EU number % of contribution of partici‐ projects
per project pations
(in 1000 euro)
4%
2%
9%
3%
1%
2%
3%
1%
1%
1%
0%
3%
31%
11.297
7.903
22.502
10.235
4.272
7.148
9.029
2.770
2.636
3.836
183
5.812
87.623
4.754
3.587
3.384
4.023
4.640
3.480
3.177
2.291
2.671
4.126
3.007
2.672
3.581
average average EU partici‐ contribution pations per per participation project (in 1000 euro)
11,21
15,32
9,67
12,71
11,61
14,47
12,56
10,95
9,87
12,22
1,76
7,90
11,07
424
234
350
316
400
241
253
209
271
338
1.709
338
323
FP7‐COOPERATION combined the objective of EU‐wide collaborative research with a logic of public procurement: major research areas and individual research topics were identified in a top‐down manner, proposals were selected by independent evaluators based on objective criteria and implemented by a large number of research and innovation projects. This collaborative approach strengthened the European Research Area by catalysing a culture of cooperation and constructing comprehensive networks fit to address thematic challenges. While a unique capability of cross‐border and cross‐sector cooperation was promoted, societal challenges were addressed such as Health, Energy, Transport and Security. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 3.5.
20
FP7-IDEAS – fostering EU wide research excellence
The specific programme FP7‐IDEAS was newly introduced in FP7 in order to increase research excellence in Europe and Europe’s attractiveness for the world renounded researchers. The programme targeted these goals through funding investigator‐driven research that stems from researchers’ own innitiative. 17% of the total FP7 EC contribution (approx. 7,7 billion euro) was allocated to this specific programme, in which 4.525 projects were funded. In order to implement the programme the ERC (European Research Council) was established, which, through the independed Scientific Council, was tasked with establishing a peer review process for project proposals and controling scientific quality. Most of the EC contribution in FP7‐IDEAS was allocated to two sub‐programmes with rather similar characteristics adressing researchers at different stages of their career: 
ERC Starting Grants addressed high potential projects, led by talented researchers at the stage of establishing their first research team or project. This sub‐programme offered research grants of maximum 1,5 million euro. 
ERC Advanced Grants supported excellent high‐risk frontier research projects led by established researchers and offered 2,5 million euro per grant. Additional funding opportunities were available for: (1) integration of small research groups working on the same project (Synergy Grants); (2) ERC grant holders supporting their efforts to transfer their research outcomes with innovation potential closer to market (Proof of Concept); and (3) grants for reserachers starting their own new individual reserch teams (Consolidator Grants). Three criteria were applied for the evaluation and selection of research proposals: (1) quality of the proposed research project; (2) the track record of the principle investigator; and (3) the research environment of the host organization. FP7‐IDEAS
Total EC contribution % of EC (in million contrib.
euro)
number of projects
ERC Starting Grants
ERC Advanced Grants
ERC other activities
Subtotal FP7‐IDEAS
3.115
3.708
851
7.673
2.315
1.700
510
4.525
7%
8%
2%
17%
average EU number % of contribution of partici‐ projects
per project pations
(in 1000 euro)
9%
7%
2%
18%
2.714
2.076
615
5.405
1.345
2.181
1.669
1.696
average partici‐ pations per project
1,17
1,22
1,21
1,19
average EU contribution per participation (in 1000 euro)
1.148
1.786
1.384
1.420 Compared to FP7‐COOPERATION the FP7‐IDEAS programme was different in several ways: o
In contrast to FP7‐COOPERATION in which only research consortia from at least three countries were invited to apply for funding, FP7‐IDEAS, invited individual researchers to submit their proposals. As a result, the average EU contribution per participating organization is five times higher than in FP7‐COOPERATION. o
ERC grants were rewarded to individual researchers (and not to the organizations they worked at) in order to strengthen their position and independance. As a result, the average contribution per participation (1,4 million euro) is five times higher than the average in the whole of FP7. Lastly, the average number of participations per project in FP7‐IDEAS is only 1,2. o
While FP7‐COOPERATION was programmed in a top‐down way, FP7‐IDEAS relied on a bottom‐up conceptulaization of research themes and priorities. As a result, every idea from every scientific discipline could be submitted for ERC funding. While FP7‐COOPERATION aimed to address grand societal challenges and ensuring high economic and societal impacts, FP7‐IDEAS aimed to increase scientific outputs, outcomes and impacts. FP7‐IDEAS primarily addressed the needs and logics of researchers in university environments. Instead of defining research topics ex‐ante and expecting researchers to collaborate following well‐defined work plans, it gave freedom and flexibility to the individual researcher to pursue his/her ideas. FP7‐IDEAS created a unique pan‐European research funding organization, established an open, direct international competition, and identified and supported the best scientists. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 3.6.
21
FP7-PEOPLE – building human resources, mobility and networks
The specific programme FP7‐PEOPLE aimed to improve the qualifications, mobility and networking of researchers all across Europe. The programme builds on nearly two decades of experience, since the origins of this specific programme are already present in FP4. While only 11% of the total budget of FP7 (approx 4,8 billion euro) was allocated to FP7‐PEOPLE, this specific programme accounts for 43% of the total funded projects. FP7‐PEOPLE funded both individual fellows and project consortia. Sub‐programmes "Initial Training", "Industry Academy Partnerships" and other activities were implemented in consortia, while sub‐programmes "Career Development" and "World Fellowships" mostly funded mobility of individual researchers through living and mobility allowances, as well as contributions to training and research costs, management activities and overheads. Besides funding the mobility of researchers, FP7‐PEOPLE also established European‐wide Doctoral degrees, such as the Innovative Doctoral Programme (IDP) and the European Industrial Doctorates (IED). FP7‐PEOPLE
Total EC contribution % of EC (in million contrib.
euro)
number of projects
Initial Training
Career Development
Industry Academia Partnerships
World Fellowships
other activities of FP7‐PEOPLE
Subtotal FP7‐PEOPLE
2.175
1.482
415
665
40
4.777
655
6.303
330
3.061
366
10.715
5%
3%
1%
1%
0%
11%
average EU number contribution % of of partici‐ per project projects
pations
(in 1000 euro)
3%
25%
1%
12%
1%
43%
5.611
6.442
1.402
4.473
1.587
19.515
3.321
235
1.257
217
110
446
average partici‐ pations per project
8,57
1,02
4,25
1,46
4,34
1,82
average EU contribution per participation (in 1000 euro)
388
230
296
149
25
245
Similar to FP7‐IDEAS, the specific programme FP7‐PEOPLE did not stipulate any themes or topics, but allowed submission across all disciplines and all themes. The proposals were evaluated according to five criteria: (1) scientific quality; (2) potential for transfer of knowledge and training; (3) the track record of the grantee; (4) synergies in implementation; and (5) expected impacts. FP7‐PEOPLE contributed to building up human resources by supporting the mobility of individual researchers on the one hand, and by supporting consortia and networks on the other. By funding individuals and not organizations FP7‐PEOPLE empowered rearchers to choose their topics and mobility paths. In order to achieve synergies it could be considered to put the responsibility for implementing the Marie Curie Actions in the same hands as the implementation of the ERC programme. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 3.7.
22
FP7-CAPACITIES – addressing specific needs in the innovation systems
The specific programme FP7‐CAPACITIES was set up with the purpose of strengthening research infrastructures, their use and development across Europe. While FP7‐CAPACITIES accounts only for 8% of the total FP7 budget and 8% of the toal funded projects, it addressed a broad variety of policy objectives and target groups: it supported the construction of new research infrastructures; the involvement of Small and Medium Sized Enterprises (SMEs) and of Society in EU wide research; aimed at creating regional networks; supported international cooperation; and the coherent development of research policies and research driven clusters. Due to comparably large size of consortia (especially in the sub‐programme "Research Infrastructure") and relatively low average total contribution per project (in all other sub‐programmes) the average contribution per partnering organization was only approx 200.000 euro and thus substantially lower than found in FP7‐
COOPERATION and FP7‐IDEAS. FP7‐CAPACITIES
Total EC contribution % of EC (in million contrib.
euro)
number of projects
Res. Infrastructure
Res. for the benefit of SMEs
Regions of Knowledge
Res.Pot. of Conv. Regions
Science in Society
Coherent dev. of res. policies
International cooperation
Subtotal FP7‐CAPACITIES
1.528
1.249
127
378
288
28
173
3.772
341
1.028
84
206
183
26
157
2.025
3%
3%
0%
1%
1%
0,1%
0,4%
8%
average EU number % of contribution of partici‐ projects
per project pations
(in 1000 euro)
1%
4%
0%
1%
1%
0%
1%
8%
5.267
9.124
1.005
307
1.820
131
1.393
19.047
4.482
1.215
1.508
1.834
1.576
1.087
1.105
1.863
average partici‐ pations per project
15,45
8,88
11,96
1,49
9,95
5,04
8,87
9,41
average EU contribution per participation (in 1000 euro)
290
137
126
1.230
158
216
124
198
Certain sub‐programmes of FP7‐CAPACITIES, namely ‘Research Infrastructures’, ‘Regions of Knowledge’ and ‘Science in Society’ followed a top‐down structure of ex‐ante defined themes and topics similar to FP7‐
COOPERATION. In the other sub‐programmes no thematic restrictions were set and proposals from all thematic areas and disciplines were accepted. Similar to FP7‐COOPERATION all submitted proposals were evaluated by independent experts based three criteria: (1) excellence; (2) implementation; and (3) potential impacts. FP7‐CAPACITIES provided incentives and research funding for a broad diversity of target groups, taking into account their organizational logics and adequately considering the complexity of the European innovation system. On one hand, this programme structure allowed FP7 to address a variety of policy objectives and to flexibly adapt to a variety of needs and demands. On the other hand, by targeting so many goals it ran the risk of becoming sub‐scale and notachieving significant impacts. As a result, FP7‐CAPACITIES helped to reach targets which were also guiding other specific programmes (e.g. of involving SMEs, Civil Society Organizations, partners from beyond Europe). A combination of the support for the European Strategy Forum Initiatives for Research Infrastructures (ESFRI) and FP7‐CAPACITIES helped to achieve a more coherent and coordinated development and use of European research infrastructures Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 3.8.
23
Agenda setting in FP7
Two specific programmes ‐ FP7‐COOPERATION and part of FP7‐CAPACITIES ‐ followed a top‐down approach in funding research. Within these programmes specific topics were called upon each year (in rare cases biannually) to which the European research community could respond to by submitting proposals. Given this top‐down approach the question on who defined the research agenda is central. First and foremost, the overarching agenda for FP7 funding goals was set in the legal base of the Framework Programme. The topics called for in annual Work Programmes (WPs) had to correspond to the boundaries and objectives set out in this legal base for each of the specific programmes. The process during which the topics in Work Programmes were agreed upon and adopted followed three phases of internal and external consultations. The whole process took place annually during a period of maximum 12 months. 1st phase: Consultation phase (12‐7 months before WP adoption): During this phase most of the external consultation on calls and themes took place. During this phase Advisory Groups were a key player in providing strategic input into annual WPs. These groups were set up for each of the themes in FP7‐COOPERATION and parts of FP7‐CAPACITIES. The members and reports of advisory group meetings were made public5. Notably, Advisory Groups did not work with the specific text of called topics. Advisory Groups may, however, have been presented with the draft texts as a point of information. At this stage, depending on perceived information needs of different Units, other types of external consultations took place, i.e. web based consultations, workshops or consultations with established groups Partnership for European Environmental Research (PEER). Each of the annual WP explicitly specified what approach was taken to internal and external consultation in the drafting of the calls. 2nd phase: Drafting of the Work Programme (4‐7 months before WP adoption): In the second phase, WPs were drafted by individual Commission Units and inputs from different Directorates in the European Commission were received and coordinated by DG R&I. At this stage the influence external stakeholders could have on the actual texts of the WP calls was smaller. 3rd phase: Adoption of Work Programme: At this stage the draft WPs were available and a formal opinion of the Programme Committee was considered. Programme Committees were typically composed of EU Member States’ representatives coming from different ministries, the scientific community, National Contact points and other stakeholders. They were chaired by the relevant European Commission Director for different parts of FP7. Programme Committees had significant influence over the text of the topics; however, a WP could still be adopted based on a negative opinion. Internally, a formal Inter‐Service Consultation (ISC) within the EC DGs also took place at this stage. After all of the opinions were considered the WPs were adopted and published. Each of the WP drafting processes had to balance between the objectives and focus specified in the legal base of a specific programme, Commission inputs and opinions of the Programme Committees. The role of external consultations remained advisory ‐ the final decision on calls and their final text in annual WPs remained with the European Commission. Notably, different thematic work programmes could end up having a different call structure, since different DGs had significant influence over the drafts of corresponding thematic work programmes and could amend them depending on foreseen needs. Allthough agenda setting and development of work programmes followed well developed procedures and several kinds of consultations took place, concerns remained about transparency and stakeholder involvement. In order to bring science closer to citizens and to consider their needs better in agenda setting it is recommended to combine the current initiatives for agenda setting and stakeholder involvement in a sub‐
programme dedicated to “Visions and Agendas”. This would allow to increase transparency about agenda setting in the top‐down parts of the EU Framework Programme and to make use of the currently scattered experiences and tools in stakeholder involvement and agenda setting processes. 5 See European Commission “Advisory Groups for FP7” at: http://ec.europa.eu/research/fp7/index_en.cfm?pg=eag Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 3.9.
24
Success rates and success factors of FP7 proposals
In total, approx. 140.000 proposals were submitted to apply for funding in FP7, with a total requested EC contribution of approx. 250 billion euro. All of them were evaluated by independent review boards according to criteria that reflected the key characteristics of the different specific programmes, sub‐programmes and funding schemes (see Chapter 4.1.). Four groups of proposals were distinguished: 
Proposals that were ineligible (because of missing obligatory criteria) or withdrawn by the proposers  in total ~ 3.900 proposals (=3%) with a total requested EC contribution of ~ 10 billion euro (48%); 
Proposals that ranked below the threshold (of evaluation)  in total ~ 67.000 proposals (48%) with a total requested EC contribution of ~ 129 billion euro (52%); 
Proposals that ranked above the threshold but did not receive any funding  in total ~ 43.000 proposals (31%) with a total requested EC contribution of ~ 66 billion euro (26%); and 
Proposals that reached the highest scores and received funding  the table below shows detailed data for the four specific programmes as well as two key indicators that allow a more comprehensive analysis of success rates: 
% of high quality proposals (i.e. the share of proposals that scored above threshold in the evaluation, were eligible and were not withdrawn) 
adjusted success rate (i.e. the share of funded proposals among the proposals that scored above threshold (high quality proposals) The share of high quality proposals was highest in FP7‐PEOPLE, lowest in FP7‐IDEAS and ranged somewhere in the middle in FP7‐COOPERATION and FP7‐CAPACITIES. The newly introduced IDEAS programme was highly attractive to a large number of researchers as it did not require collaborative research, was open to all disciplines and themes and offered highly attractive funding conditions. Especially in times of reduced national funding and limited career perspectives FP7‐IDEAS motivated a high number of first‐time proposers. As a consequence, a large number of submitted proposals did not reach the minimum quality criteria of this programme, so that only 26% were of sufficiently high quality. 48% of these high quality proposals in FP7‐
IDEAS received funding funded. number of proposals
FP7‐COOPERATION
FP7‐IDEAS
FP7‐PEOPLE
FP7‐CAPACITIES
total
Euro requested EC contribution (in million euro)
FP7‐COOPERATION
FP7‐IDEAS
FP7‐PEOPLE
FP7‐CAPACITIES
total
proposals proposals total above that number of threshold received EU proposals that did not funding
receive 42.026 7.912 11.575
36.283 4.525 4.850
50.168 10.715 23.950
10.815 2.025 2.802
139.292 25.177 43.177
18%
31%
proposals total proposals above requested that threshold EC received EU that did not contributi
funding
receive on funding
138.713 28.336 38.911
67.650 7.673 10.209
25.072 4.777 11.847
18.308 3.772 5.165
249.742 44.559 66.132
18%
26%
ineligible proposals adjusted high or below success quality withdrawn threshold
rate
proposals
proposals
20.685 25.937 14.921 5.493 67.036 48%
1.854 971 582 495 3.902 3%
41%
48%
31%
42%
37%
ineligible proposals adjusted high or below success quality withdrawn threshold
rate
proposals
proposals
64.659 48.284 8.222 7.613 128.778 52%
6.806 1.483 225 1.758 10.273 4%
42%
43%
29%
42%
40%
Remarks:
"High Quality Proposals" are proposals that scored in the evaluation above threshold, were eligible and were not withdrawn
"Adjusted success rate" is the % rate of funded proposals among the proposals that scored in the evaluation above threshold
46%
26%
69%
45%
49%
48%
26%
66%
49%
44%
CAPACITIES
PEOPLE
IDEAS
COOPERATION
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme proposals proposals above total that threshold number of Specific Programme
received EU that did not proposals funding
receive funding
Health
4.066 1.008 952
KBBE
2.951 516 1.203
ICT
16.784 2.328 3.951
NMP
2.677 805 424
Energy
1.666 368 424
Environment
2.736 494 935
Transport
3.105 719 933
SSH
2.746 253 1.159
Space
1.026 267 453
Security
1.827 314 605
ERANET
151 104 2
JTI
2.291 736 534
ERC Starting Grants
18.064 2.315 2.026
ERC Advanced Grants
12.743 1.700 2.294
other activities
5.476 510 530
Initial Training
5.088 655 2.485
Career Development
27.962 6.303 15.031
Industry Academia Partnerships 1.346 330 492
World Fellowships
15.057 3.061 5.870
other activities
715 366 72
Res. Infrastructure
944 341 275
Res. for the benefit of SMEs
5.629 1.028 1.216
Regions of Knowledge
468 84 114
Res.Pot. of Conv. Regions
2.279 206 868
Science in Society
831 183 170
Coherent dev. of res. policies
56 26 9
International cooperation
608 157 150
total
139.292 25.177 43.177
25
average multiple ineligible proposals adjusted high score of submissions or below success quality proposals of the withdrawn threshold
rate
proposals that gained similar proposals
EU funding proposals
1.994 1.109 9.663 1.411 804 1.166 1.317 1.179 289 883 6 864 13.263 8.398 4.276 1.901 6.290 513 5.949 268 246 3.241 196 1.109 432 21 248 67.036 112 123 842 37 70 141 136 155 17 25 39 157 460 351 160 47 338 11 177 9 82 144 74 96 46 0
53 3.902 51%
30%
37%
66%
46%
35%
44%
18%
37%
34%
98%
58%
53%
43%
49%
21%
30%
40%
34%
84%
55%
46%
42%
19%
52%
74%
51%
37%
48%
58%
37%
46%
48%
52%
53%
51%
70%
50%
70%
55%
24%
31%
19%
62%
76%
61%
59%
61%
65%
40%
42%
47%
42%
63%
50%
49%
88
88
84
85
86
89
85
90
87
86
80
78
95
95
88
93
90
86
90
81
83
86
83
91
82
75
83
89
0%
1%
3%
1%
1%
1%
4%
1%
4%
1%
0%
2%
3%
5%
1%
14%
4%
4%
3%
1%
2%
8%
1%
8%
2%
0%
2%
4%
The table above presents a more in‐depth look into the sub‐programmes (of each specific programme) and includes two additional indicators: 

the average score of successful proposals indicates the level of competition in the different sub‐
programmes (highest competition in SSH, FP7‐IDEAS, FP7‐PEOPLE, and Reg.Pot) the share of proposals that were submitted several times (the highest numbers in ITN, SME and Reg.Pot.) indicates those sub‐programmes in which a continuous improvement of proposals was possible (while in FP7‐COOPERATION, for example, most of the unsuccessful proposal could not be resubmitted). Success rates in the two subprograms that have individual grantees (FP7‐IDEAS and MCA in FP7.PEOPLE), had very different figures regarding the ratio of female‐male participation. Despite some disparities between countries and scientific fields, with a 37% of female MCA fellows6, the gender specific target of 40% female set by FP7 had almost been reached. However, taking into account all ERC grants together (Advanced, Consolidator and Starting grants) only 20% of the grants were awarded to women7 although 25% of the applicants were female. This suggests that female researchers faced a lower probability of success. In FP7‐COOPERATION, SSH showed comparably low adjusted success rates, a comparably high average score of successful proposals and very little chance for resubmission of proposals. This indicates an area in which a substantial number of high quality proposals did not receive funding and an increase of budget would be justified. ERA‐NETs and JTIs showed comparably high adjusted success rates and a rather low average score of successful proposals. Measures of quality control are recommended in order to ensure that the institutional commitment of partner organizations (which is of outstanding importance in these specific programmes) does not outweight other quality criteria. In FP7‐IDEAS two improvements are required: expectation management (to avoid being flooded by a high number of weak proposals) and training for applicants (to help high potentials to achieve better results). In FP7‐PEOPLE the threshold levels should be assessed and potentially raised in order to ensure the coherence of evaluation schemes across the different specific programmes. 6 7 Avramov, D. (2015), FP7 ex‐post evaluation People Specific Programme (2007‐2013): Rationale, implementation and achievements. ERC, Statistics on Gender Balance. Available at: http://erc.europa.eu/about‐erc/organisation‐and‐working‐groups/working‐
groups/gender‐balance
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 26
3.10. Organizations participating in FP7
Organizations that participated in FP7 funded research were divided into six distinct categories: universities, research and technology organizations, large private companies, small and medium sized enterprises, public authorities and others (for more details see Annex 9.8.): Universities (HES) form the most important FP7 target group having received a total of 44% (19,5 billion euro) of the total EC contribution across all four specific programmes. In total approximately 2.300 Universities participated in FP7. Universities were central to the logic of FP7‐IDEAS programme where 73% (5,6 billion euro) of the funding went to universities. They were also a central player in FP7‐
PEOPLE with a share of 62% (3 billion euro) of total FP7‐PEOPLE funding. Universities also played a substantial role in FP7‐COOPERATION receiving 35% (10 billion euro) of total funding. In FP7‐
COOPERATION universities were also the most prominent group of participants in some of the specific themes, namely in health research in relative terms (51%), and in ICT research, where they received the highest EC contribution ‐ 3 billion euro in total. In addition, they played a substantial role in socio‐economic research and humanities. Research and Technology Organizations (REC) account for 27% of the total EC contribution (10 billion euro). They received a rather similar share across all four specific programmes. In total approximately 3.600 Research and Technology Organizations participated in FP7. In FP7‐
COOPERATION they played a substantial role in environmental research (42% of total funding for the Theme Environment) and in space research (47% of total Theme Space funding). Within FP7‐CAPACITIES their share is the most significant in the theme Research Infrastructures (53% of total FP7‐CAPACITIES funding). Large private companies (PRC) account for 11% of the total EC contribution (5,1 billion euro). Their biggest share of funding came from FP7‐COOPERATION, where they gained 4,7 billion euro of EU funding. In terms of shares of total funding in different themes they played the most significant role in the themes Energy (31%), Transport (34%) and Security research (24%). Large private companies were also the key funding recipient of the Joint Technology Initiatives (32% of JTIs budget). In absolute terms, ICT was the theme with the highest EC contribution going to large private companies (1,6 billion euro). In FP7‐IDEAS only a very small share went to private companies. A similar situation is encountered in FP7‐CAPACITIES with an EC contribution totalling only 0,1 billion euro. Within FP7‐PEOPLE large companies only took part in the Initial Training Networks and the Industry Academia Partnerships, and the total EC contribution to PRCs in both activities (0,2 billion euro) is marginal compared to FP7‐COOPERATION. Small and Medium Sized Enterprises (SMEs) account for 13% of the total EC contribution (5,9 billion euro). The largest share of this funding for SMEs come from FP7‐COOPERATION (4,4 billion euro) and the SME programme of FP7‐CAPACITIES (1 billion euro). In absolute terms their involvement was the highest in ICT (1 billion euro), Health (0,85 billion euro) and NMP (0,71 billion euro) within FP7‐COOPERATION. In FP7‐CAPACITIES the sub‐
programme "Research for the benefit of SMEs" was focusing on the needs of this specific target group. In total the share of FP7 funding that went to the private sector was 25% (11 billion euro), with approximately 18.900 companies they form the largest group of organizations participating in FP7. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 27
Public Authorities played only a minor role in FP7, accounting for 3% of the total EC contribution (1,2 billion euro). They were mainly involved in FP7‐COOPERATION (0,85 billion euro), the career development sub‐
programme of FP7‐PEOPLE (0,15 billion euro) and in various FP7‐CAPACITIES sub‐programmes (total 0,15 billion euro). Their involvement mirrors the attempt to foster the links between research and policy making and to support the dissemination of research results into public policy. In total approximately 1.900 public authorities participated in FP7. Several different types of organizations could be found within the last category “Other”; these were mainly industry umbrella organizations, university networks and civil society organizations (CSOs). The total budget share of these organizations is 1 billion euro, however, this contains the support of one specific organization in the ERA‐NET programme of 0,25 billion euro. As a result the "real" share of other organizations is even smaller and mostly limited to the sub‐programme "Science in Society" in FP7‐CAPACITIES and their marginal involvement in FP7‐COOPERATION. Approximately 2.100 other organizations participated in FP7. FP7‐COOPERATION
28.3 billion euro
FP7‐IDEAS
7.7 billion euro
Universities
(HES)
FP7‐PEOPLE
4.8 billion euro
RTD Org.
(REC)
Universities
(HES)
RTD Org.
(REC)
Large private companies
(PRC)
Large private companies
(PRC)
small private companies
(SMEs)
small private companies
(SMEs)
Public Authorities
(PUB)
Public Authorities
(PUB)
other (incl. CSOs)
other (incl. CSOs)
Universities
(HES)
Universities
(HES)
FP7‐CAPACITIES
3,8 billion euro
RTD Org.
(REC)
RTD Org.
(REC)
Large private companies
(PRC)
Large private companies
(PRC)
small private companies
(SMEs)
small private companies
(SMEs)
Public Authorities
(PUB)
Public Authorities
(PUB)
other (incl. CSOs)
other (incl. CSOs)
The annual EC contribution to the Framework Programmes increased from 4,2 billion euro in FP6 to 6,4 billion euro in FP7, constituting an increase of 54%. Among the different types of participating organizations the highest increase was gained by universities (plus 71%), by private sector companies (plus 62%) and by public authorities (plus 93%, but in very low absolute terms). The increase of EC contribution to research organizations was below average (43%), while other organizations incl. CSOs had to face a decrease of annual EU funding of 41%. 7.000
other (incl. CSOs)
‐41%
+91%
Public Authorities (PUB)
6.000
Private Companies (PRC)
RTD Org. (REC)
5.000
Universities (HES)
+61%
4.000
+42%
3.000
2.000
+75%
1.000
0
FP6
FP7
The needs and logics of two large groups in STI were successfully addressed by FP7: The implementation of FP7‐
IDEAS led to a substantial increase of EC funding for universities and the establishment of JTIs (in FP7‐
COOPERATION) and SME (in FP7‐CAPACITIES) increased the annual contribution to private sector companies. Compared to FP6 the involvement of civil society organizations decreased substantially in FP7. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 28
Continuity between FP6 and FP7 regarding participating organizations
With an increased budget and changes in programme design interesting observations can be made in terms of how many organizations continued participating in FP7 funded projects and how many were newcomers. The analysis shows that from the 29.000 organizations that participated in FP7 25% had already participated in FP6, while 74% of organizations participated only in FP7 and not in FP6. The continuity of participation was the highest among universities (59% overlap) and RTD organizations (44% overlap) and the lowest among private companies (18% overlap) and other organizations (17% overlap). The high turnover of participating private sector companies was caused by a substantial number of SMEs who participated in FP7 for the first time. When taking into account the EC contribution that was received by participating organizations in FP6 and FP7 the picture looks very different: on average 85% of FP6 funding and 82% of FP7 funding was allocated to organizations that participated in both programmes. This continuity of EC funding was the highest among universities (97%), followed by RTD organizations (91%). A substantial share of FP7 funding also went to private sector companies who participated in both programmes ‐ 73% of FP6 funding and 49% of FP7 funding was received by companies participating in both programmes. Continuity FP6/FP7 (by number of org.)
(by EC contribution)
Universities
(HES)
Universities
(HES)
RTD Org.
(REC)
RTD Org.
(REC)
Private
Companies
(PRC)
Private
Companies
(PRC)
FP6 contribution to
organisations that
participated in FP6 and FP7
Public
Authorities
(PUB)
Public
number of organisations that only
participated in FP6
Authorities
FP7 contribution to
organisations that
participated in FP6 and FP7
other
(incl. CSOs)
other
number of organisations that only
participated in FP7
(incl. CSOs)
FP6 contribution to
organisations that only
participated in FP6
(PUB)
number of organisations that
participated in FP6 and FP7
0
10.000
20.000
number of organisations
30.000
FP7 contribution to
organisations that only
participated in FP7
0
10.000
20.000
30.000
million euro
FP7 showed a high degree of openness for organizations that have not participated in the previous Framework Programmes. This holds true for SMEs from all over Europe, but also for a substantial number of RTD organizations and even universities that participated in FP7 but not in FP6. This inherent openness is, however, not mirrored in terms of funding shares of newcomers and organizations that participated in both programmes. A very high percentage of the EC funding in FP7 was received by organizations that have already participated in FP6. This holds especially true for universities and research organizations, where a comparably small number of organizations managed to build up the qualifications and capacities for continuing to be key players in European‐funded research. FP7 can therefore be considered as having balanced the need for openness and concentration central to global competition. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 29
Degree of concentration of EU funding among the participating organizations
A high degree of concentration in a research and innovation system can have positive as well as negative effects on actors within the system. One the one hand, it can lead to increased global competitiveness and economies of scale and foster the emergence of centres of excellence. On the other hand, it can lead to unintended effects, such as the dominance of status over content, risks of elitist compartmentalisation or barriers against newcomers and actors from network peripheries. As there is no available data about the degree of concentration within the European Research Area, the following analysis is based on data on EC‐
funded projects and an ABC‐analysis of EC contribution in FP6 and FP7. 8 ABC Analysis
EC contribution to this group in million % EC euro
contribution
A‐Group (Top‐500)
B‐Group (> 100.000 € annual EC‐contribution)
C‐Group (< 100.000 € annual EC‐contribution)
total
27.098
13.051
4.769
44.917
60,3%
29,1%
10,6%
100,0%
no of no of partici‐ org. % of org. pations
500
5.455
22.917
28.872
1,7%
18,9%
79,4%
100,0%
58.964
41.730
32.921
133.615
EC contrib. % of per org in partici‐ partici‐ million pations pations
euro
per org.
44,1%
31,2%
24,6%
100,0%
54,20
2,39
0,21
1,56
117,9
7,6
1,4
4,6
EC contrib. per partici‐
pations in 1000 euro
459,57
312,74
144,85
336,17
The A‐Group contains ABC Analysis by type of organisation
the group of top‐500 organisations, which 0%
20%
40%
60%
80%
100%
were awarded the Univ.
highest amounts of EC (HES)
funding in FP7; ranging 283 org.
139 org.
56
A‐Group (Top‐500)
16,7 bio €
8,2 bio €
1,5
from around 800 million RTD Org.
euro to around 13 (REC)
million euro per organisation. This group Large private companies
B‐Group (> 100.000 € annual 680 1.118
includes large research 1.122 2.029 org.
(PRC)
2,5
2,6 bio
EC‐contribution)
3,4 3,3 bio €
organisations (such as the Centre National de small private companies
(SMEs)
la Recherche Scientifique, the C‐Group (< 100.000 € annual 5.360
10.360 org.
Public Authorities
Fraunhofer Gesellschaft 1,0
EC‐contribution)
2,4 bio €
(PUB)
and the Max Planck Gesellschaft), leading other (incl. CSOs)
universities (such as % of number of organisations per group
Oxford, Cambridge, University College London, ETH Zürich and Leuven) as well as some industry organisations (such as SAP, Thales, Siemens and Telefonica). While the A‐Group contains only 1,7% of organisations that participated in FP7, it received about 60% of total funding (more than 27 billion euro in total). On average, each A‐Group organisation participated in 120 FP7 projects and was awarded a share of about 460.000 euro per project. The A‐Group contains a relatively large share of universities (62%), which received a relatively large share of funding (57% of EC funding for the A‐Group in total). This can be interpreted as an effect of the specific programme FP7‐IDEAS, which was primarily targeted at top universities. Comparing this group of organisations by country shows that organisations from the UK were significantly over‐represented, while there were few organisations from Mediterranean countries and hardly any from the EU‐13. The degree of concentration for the A‐Group was highest in (sub‐) programmes FP7‐IDEAS, Infrastructures (FP7‐CAPACITIES), ITN (FP7‐PEOPLE) as well as ICT and HEALTH (FP7‐COOPERATION). 8 It is important to consider that the size of organisations involved in FP7 differs significantly and that the TOP‐500 organisations might well represent a substantial part of the total research system in Europe. As no indicators of organisational size are processed in the CORDA database, no relative figures could be included into this report.
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 30
The B‐Group contains about 4.000 organisations that euro received more than 100.000 annual EC contribution in FP7. They account for 19% of organisations that participated in FP7, and were awarded 29% of EC funding (about 13 billion euro in total). On average, each B‐Group organisation participated in eight FP7 projects, was awarded about 2,4 million euro in total and a share of about 312.000 euro per project. This group includes similar shares of research organisations, large companies, and SMEs; while universities were under‐
represented. A comparison by country shows that Italy, Spain and Portugal were over‐represented in this group, while the share of organisations from the UK was significantly smaller. The C‐Group contains all organisations that received less than 100.000 euro annual EC contribution in FP7. While this includes about 80% of all organisations that participated in FP7 (about 23.000 organisations in total), the C‐Group received only about 10% of total EC contribution (about 4,8 billion euro in total). In this group, private organisations were awarded the largest funding shares: SMEs received about 50% of total EC funding for this group (about 2,4 billion euro in total) and large companies received about 21% (about 1 billion euro in total). A comparison by country shows that organisations from Mediterranean countries and the EU‐13 were slightly better represented than on average across all three groups. To evaluate changes in the degree of concentration between FP6 and FP7, a separate ABC‐analysis was performed for FP6 and a comparison of the A‐Group between FP6 and FP7 was carried out. The analysis shows that the share of funds that were awarded to organisations in the A‐Group has slightly increased from FP6 to FP7: The top‐500 organisations received 58% of annual funding in FP6 and 60% in FP7. It also shows that the group of top‐500 organisations remained relatively stable: 369 organisations (74%) of the A‐Group in FP6 were also part of the A‐Group in FP79. This concentration effect becomes even more pronounced in a comparison of organisations by their position within the A‐Group: The top‐100 organisations were able to almost double their annual EC funding contribution from FP6 to FP7; some universities among this group even received the triple amount of funding in FP7 compared to FP6. Annual EC funding contribution for organisations ranked between 101 and 200 increased by 54% from FP6 to FP7, and was thus in line with the overall budget increase. Organisations ranked 201 and lower were able to obtain a comparatively low increase of 17% of annual EC funding contribution from FP6 to FP7; increases for this sub‐group were considerably lower than the overall budget increase (even though these organisations were part of the top‐500 organisations in FP6 as well as in FP7). A comparison of different types of organisations shows different trends: Among the group of higher education organisations, significant increases of annual funding were mainly received by leading universities in the programme FP7‐IDEAS. Some universities were awarded twice or three times as much funding in FP7 as they had received in FP6. This suggests that FP7 supported the emergence of European‐wide centres of excellence. At the same time, individual funding under FP7‐PEOPLE contained the danger of decreasing incentives for leading universities to cooperate with other, less prominent universities. It should be noted, however, that only a handful of universities among the A‐Group were awarded less annual funding in FP7 than in FP6. FP6 FP7 comparison HES
70.000.000
60.000.000
annual EC contribution in FP7

50.000.000
40.000.000
30.000.000
20.000.000
10.000.000
0
0
10.000.000
20.000.000
30.000.000
40.000.000
50.000.000
annual EC contribution in FP6
9 From FP6 to FP7, about 100 organisations moved from the A‐Group to the B‐Group or vice versa; about 30 moved between the A‐Group and the C‐Group. 60.000.000
70.000.000
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme Funding increases from FP6 to FP7 for research organisations among the A‐Group were consistent with the overall budget increase from FP6 to FP7. While budget increases were most pronounced for large organisations in this sub‐group (like in the sub‐group of higher education organisations), there were hardly any gains of more than 100%. It should be noted, however, that in FP7 annual funding amounts for the largest research organisations were almost double the amount awarded to the strongest universities. FP6 FP7 comparison REC
120.000.000
100.000.000
annual EC contribution in FP7

31
80.000.000
60.000.000
40.000.000
20.000.000
0
0
20.000.000
40.000.000
60.000.000
80.000.000
100.000.000
120.000.000
annual EC contribution in FP6
The sub‐group of private sector companies in the A‐Group showed a different development of funding increases from FP6 to FP7 than the other two sub‐groups. Among private sector companies, there were winners (primarily companies in the ICT industry) as well as losers (primarily companies in the automotive and transport industries). Instead of an increase in the degree of concentration, this sub‐group was characterized by a comparably strong fluctuation of participating organisations. FP6 FP7 comparison PRC
20.000.000
16.000.000
annual EC contribution in FP7

12.000.000
8.000.000
4.000.000
0
0
4.000.000
8.000.000
12.000.000
16.000.000
annual EC contribution in FP6
Concentration effects are a necessary precondition for success in a globalized competitive research and innovation system, but might also run the risk of unintended effects. As a consequence, future Framework Programmes should build up a useful data base for a long‐term analysis of participation and collaboration patterns. In addition, they would be well advised to ensure that there is a certain amount of diversity among research actors, that programmes and networks remain open to newcomers and that actors are able to make their way from network peripheries to the centres of cooperation. 20.000.000
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 32
3.11. Country participation in FP7
total EC contribution (in million euro)
Country (a)
United Kingdom
EU ‐ 15
(e) 3.595
1.709
1.086
1%
399
52
111
63
1.689
4%
1.133
280
182
94
1.060
2%
673
146
152
88
861
2%
633
113
51
63
5.059
11%
3.298
965
457
340
Belgium
1.785
4%
1.244
244
186
111
Netherlands
3.313
7%
2.108
691
320
195
Luxembourg
60
0%
43
1
11
5
7.079
16%
4.910
1.136
566
466
Austria
1.183
3%
812
186
118
67
Italy
3.568
8%
2.565
402
287
314
Spain
Germany
EU ‐ 13
519
3.234
7%
2.115
410
393
317
Greece
999
2%
701
54
88
156
Portugal
520
1%
346
55
59
37.946
85%
Poland
437
1,0%
252
19
45
120
Czech Republic
281
0,6%
183
16
29
52
Slovakia
77
0,2%
50
1
11
14
Hungary
287
0,6%
150
60
32
45
Slovenia
169
0,4%
117
2
14
35
Romania
141
0,3%
94
0
9
37
Bulgaria
97
0,2%
47
3
5
42
Croatia
90
0,2%
42
4
8
35
Estonia
94
0,2%
51
5
10
29
Lithuania
54
0,1%
30
0
5
20
Latvia
48
0,1%
26
1
3
18
Malta
21
0,0%
12
1
1
7
Cyprus
93
0,2%
46
14
13
20
sub-total EU-15
Associated Countries
FP‐COOP. FP7‐IDEAS FP7‐PEOPLE FP7‐CAPA. 16%
France
(d)
% 626
(b) Denmark
Finland
(c) FP7 6.909
Ireland
Sweden
24.575
6.443
173
61
2.860
sub-total EU-13
1.773
4%
1.043
2.021
4,5%
1.011
592
308
Norway
754
1,7%
507
88
66
92
Israel
875
2,0%
373
405
74
23
Turkey
189
0,4%
87
11
37
53
other Assoc. Countries
159
0,4%
92
2
19
46
447
111
3.997
9%
Russia
70
0,2%
59
0
4
7
Ukraine
23
0,1%
17
0
1
4
China
35
0,1%
30
0
2
3
India
39
0,1%
34
0
2
3
Japan
9
0,0%
7
0
0
2
USA
82
0,2%
76
3
1
2
Canada
12
0,0%
10
1
0
1
Brazil
32
0,1%
28
0
1
3
540
1,2%
388
17
22
113
other countries
sub-total Beyond Europe
Grand total
(a) …. Anglo‐Saxon Countries
(b) … Scandinavia
(c) … France and Benelux Countries
(d) … Germany & Austria
(e) … Mediterranean Countries
2.069
111
4.068
Switzerland
sub-total Assoc. Countries
beyond Europe
Organizations from all EU Member States, accession countries, associated countries and developing countries were eligible to apply for FP7 funding, while participants from other countries (such as Canada, Japan, the USA, and the BRIC countries) were only eligible under specific conditions (see Chapter 5.4.). 85% of the EC funding for FP7 research projects were allocated to organizations located in the EU‐15 (the “old” EU Member States) (in total 38 billion euro). The EU‐
13, i.e. countries that joined the EU during or after the 2004 accession (the "new" EU Member States) received only 4% of the total FP7 funding (in total 1,8 billion euro). Organizations from associated countries accounted for 9% of the FP7 budget (in total approx. 4 billion euro) and 2% was received by organizations from other countries beyond Europe (0,8 billion euro). Among the EU‐15 France, Germany and the United Kingdom received more than 5 billion euro of FP7 funding each, followed by Italy, the Netherlands and Spain with approx. 3 billion euro each. Among the associated countries Israel, Norway and Switzerland received a substantial share of the FP7 funding. 1.099
504
326
842
2%
649
21
32
140
44.559
100%
28.336
7.673
4.777
3.772
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme EC contribution per researcher per year
Country EU ‐ 15
United Kingdom
10 Average annual EU contribution per researcher across countries is approx. 3.900 euro. This is mainly accounted by 65% share of FP7‐COOPERATION funding and between 11‐12% of FP7‐
IDEAS, ‐PEOPLE and ‐CAPACITIES each. The highest average annual FP7 contribution per researcher is witnessed in the Netherlands, followed by Belgium, Greece and Ireland. In some of the largest EU Member States (such as France, Germany and United Kingdom) the average annual EU contribution per researcher was substantially lower (approx. 2.000 euro). FP‐
COOP. FP7‐
IDEAS FP7‐
FP7‐
PEOPLE CAPA. 3.873
2.015
958
609
291
6.201
3.952
518
1.104
627
Sweden
50.345
4.793
3.216
794
518
266
Denmark
37.298
4.059
2.577
560
583
339
Finland
40.260
3.054
2.246
402
180
225
France
242.988
2.975
1.939
567
269
200
Belgium
40.471
6.302
4.390
861
657
394
Netherlands
57.764
8.194
5.214
1.708
790
481
Luxembourg
2.523
3.415
2.407
57
643
308
327.258
3.090
2.143
496
247
204
36.261
4.662
3.201
731
466
265
Italy
104.121
4.895
3.520
552
394
430
Spain
351
Germany
128.952
3.583
2.343
454
435
Greece
24.370
5.857
4.112
316
515
915
Portugal
39.974
1.858
1.235
195
210
217
1.401.875
3.867
2.504
657
415
291
Poland
64.489
967
559
43
99
267
Czech Republic
30.546
1.312
858
73
138
244
Slovakia
14.105
778
507
12
114
145
Hungary
21.314
1.922
1.006
400
217
300
Slovenia
7.828
3.084
2.138
39
264
642
Romania
18.579
1.081
720
3
72
286
Bulgaria
11.573
1.198
581
34
68
515
Croatia
6.704
1.907
892
80
180
756
Estonia
4.223
3.183
1.724
153
341
965
Lithuania
8.423
917
504
0
82
331
Latvia
3.931
1.755
959
49
104
643
Malta
658
4.543
2.518
136
283
1.606
866
15.313
7.524
2.334
2.144
3.311
1.321
777
129
333
Cyprus
sub-total EU-13
191.715
82
4.500
FP7 EC contribution per researchers per year
o
FP7 14.411
sub-total EU-15
In order to carry out an in‐depth analysis the total number of researchers per country10 was processed and the average EU contribution per researcher and per year was calculated to illustrate the relative importance of FP7 funding in the research systems of each country: no of researchers
254.879
Ireland
Austria
EU ‐ 13
An in‐depth analysis of the four specific programmes showed that Israel, the Netherlands, Switzerland and the United Kingdom gained substantially larger shares in FP7‐IDEAS compared to other countries while the EU‐13 received above average shares of funding from FP7‐
CAPACITIES. The comparably low share of funding for organizations from beyond Europe can be explained by the specific funding schemes not allowing all organizations to participate in all research projects. The comparably high share of organizations from other countries beyond Europe is accounted for by the FP7‐CAPACITIES sub‐programme "International Cooperation", as well as by special eligibility criteria in FP7‐
COOPERATION that allowed partners from developing countries to participate on equal funding conditions as partners from EU Member States. Taking into account the comparable low staff costs in developing countries, the EC contribution to research projects were quite substantial for partners from these countries. 33
FP7‐CAPA.
4.000
FP7‐PEOPLE
3.500
FP7‐IDEAS
3.000
FP‐COOP.
2.500
2.000
1.500
1.000
500
‐
EU‐15
EU‐13
Source: Eurostat, “Total R&D personnel and researchers by sectors of performance, sex and NUTS 2 regions” for years 2007‐
2013 (table code: rd_p_persreg) Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme Looking at EU‐13 countries the average annual EU contribution per researcher across the new EU Member States was approx. 1.300 euro. The relative shares of FP7‐COOPERATION (59%) and FP7‐PEOPLE (11%) are similar to the EU‐15. FP7‐IDEAS and FP7‐
CAPACITIES are the programmes where most differences are observed: the share of FP7‐IDEAS (6%) is substantially lower and the share of FP7‐CAPACITIES is substantially higher (25%). This clearly indicates the high importance of FP7‐CAPACITIES sub‐
programmes for the EU‐13. It also points to the gap between EU‐13 and EU‐15 in the funding received from FP7‐IDEAS programme. annual FP7 contribution
Country United Kingdom
o
The annual FP7 contribution per inhabitant is 14 euro on average across the EU‐15, with large differences between countries (e.g. 20‐30 euro in Scandinavia, Austria, Belgium, Ireland and the Netherlands, 10‐20 euro in France, Germany, Greece, Spain and United Kingdom, and below 10 euro in Italy and Portugal). Within EU‐13 countries the annual FP7 contribution per inhabitant is less than a quarter of that in EU‐15. While Estonia and Slovenia come close to southern European EU‐15 countries (10‐12 euro of FP7 funding per inhabitant), the average FP7 contribution per inhabitant in other EU‐13 Member States is substantially lower (below 5 euro). per million euro RTD expenditures
16
20
33.962
Sweden
241
26
19.347
Denmark
151
27
21.476
Finland
123
23
18.097
France
723
11
16.562
Belgium
255
23
33.400
Netherlands
473
29
41.608
Luxembourg
9
17
14.890
1.011
12
14.099
169
20
21.047
Germany
510
9
26.125
Spain
462
10
33.056
Greece
143
13
100.552
Portugal
74
7
5.421
14
62
2
Czech Republic
40
4
17.272
Slovakia
11
2
26.108
23.775
Hungary
41
4
35.377
Slovenia
24
12
32.017
Romania
20
1
31.597
Bulgaria
14
2
67.094
Croatia
13
3
35.656
Estonia
13
10
49.447
Lithuania
8
2
29.288
Latvia
7
3
54.422
Malta
3
4
36.633
13
32
296.600
3
29.094
Cyprus
sub-total EU-13
270
o
In the countries that have been heavily hit by the economic crisis (such as Greece, Ireland, Italy, Portugal and Spain) the national RTD expenditures have been cut down, resulting in a comparable high contribution of FP7 to available RTD funds. o
In large EU‐15 Member States with well‐established national support schemes, the contribution of FP7 to the available RTD funding is comparably low (e.g. France, Germany). o
In other large EU‐15 Member States where high levels of competition are embedded in their national research and innovation systems, the FP7 share in comparison to national funding is substantially higher (e.g. in the Netherlands and United Kingdom). o
Due to substantially lower national RTD funding, FP7 funds substantially contributed to funding research in the EU‐13 (e.g. Bulgaria, Estonia, Latvia). Considering the total FP7 funding for research and innovation projects, the funding per inhabitant and per researcher, the EU‐15 gained a substantially higher share of funding than the EU‐13 in relative terms. However, FP7 funding was of lesser improtance for national research and innovation systems in countries with well‐
established national RTD funding schemes and well‐endowed national RTD budgets. Here, FP7 only contributing 1‐2% to the national RTD expenditures, while in countries lacking these budgets FP7 played a more prominent role. 30.059
22.436
Poland
Looking at FP7 funding in relation to the total annual RTD expenditures in different EU Member States shows the relative importance of FP7 for the national innovation systems and provides a totally different picture than above, as the EC financial contribution per million euro of national RTD expenditures is 30% higher in the EU‐
13 than in the EU‐15. More trends can be seen when analysing individual countries: 11 30.554
Italy
sub-total EU-15
EU ‐ 13
o
per inhabitant (in euro)
89
Austria
Going further into the analysis of FP7 funding in relation to country size, the shares of FP7 funding were analysed in relation to the number of inhabitants and annual national RTD expenditures per country (public and private)11: total
(in million euro)
987
Ireland
EU ‐ 15
o
34
Source: Eurostat “Total intramural R&D expenditure (GERD) by sectors of performance and NUTS 2 regions” for years 2007‐
2013 (table code: rd_e_gerdreg) Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 35
The role of researchers from the EU-13 countries in FP7
Share of Macro‐Regions by sub‐
programme (% EC contribution)
25%
50%
EC contribution to Macro‐Regions (mio €)
75% 100%
Health
KBBE
KBBE
ICT
ICT
NMP
NMP
FP7‐COOPERATION
Health
Energy
Environment
Environment
Transport
Transport
SSH
SSH
Space
Space
Security
Security
ERANET
ERANET
JTI
JTI
ERC Starting Grants
ERC Starting Grants
ERC Advanced Grants
ERC Advanced Grants
ERC other activities
ERC other activities
Initial Training
Initial Training
Career Development
Career Development
FP7‐IDEAS
Energy
FP7‐PEOPLE
FP7‐IDEAS
FP7‐COOPERATION
0%
FP7‐PEOPLE
FP7 had no explicit objective regarding the participation of the countries that have joined the European Union during the 2004 and 2007 enlargements (commonly referred to as EU‐
13 countries). In this way FP7 differed in its objectives from FP6. FP6 was being implemented just before the 2004 enlargement and has explicitly stressed the need and called for specific actions aimed at integrating the then candidate countries into the European research system FP7 had no explicit objective regarding the participation of the countries that have joined the European Union during the 2004 and 2007 enlargements (commonly referred to as EU‐
13 countries). However, FP7 data suggests that the divide between the participation patterns of EU‐15 and EU‐13 remained. Industry Academia Partnerships Industry Academia Partnerships
World Fellowships
0
2.250
4.500
6.750
EU‐13
World Fellowships
other activities of FP7‐PEOPLE
FP7‐CAPACITIES
FP7‐CAPACITIES
other activities of FP7‐PEOPLE
EU‐15
In total 1,8 billion euro of FP7 Res. Infrastructure
Res. Infrastructure
funding (4% of the total FP7 Res. for the benefit of SMEs
budget) was received by Res. for the benefit of SMEs
Assoc.
organisations based in the EU‐
Regions of Knowledge
Regions of Knowledge
13 countries. FP7‐
Res.Pot. of Convergence…Res.Pot. of Convergence Regions
beyond Europe
COOPERATION and FP7‐
Science in Society
Science in Society
PEOPLE mirrors the overall Coherent dev. of res. policies
Coherent dev. of res. policies
picture – in each programme International cooperation
EU‐13 represents about 4% International cooperation
share of the total programme budget. EU‐13 shares were above average in two themes in FP7‐COOPERATION ‐ in SSH (8%) and Security (7%). Only half of the average share of EU‐13 (2%) can be observed in FP7‐IDEAS, while in FP7‐CAPACITIES shows the highest, 13% share of total EC contribution received by EU‐13 countries. The latter can be accounted for by the specific sub‐
programmes such as SME, Regions of Knowledge, Research Potential of Convergence Regions and Science in Society. Furthermore, partners from the EU‐13 often played a minor role in FP7 projects and networks. For example, only 2% of the projects funded in FP7‐COOPERATION were coordinated by organisations from the EU‐13 countries. 9.000
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 36
There are differences between EU‐
(EU‐13)
share of org‐type (EU‐15)
15 and EU‐13 when it comes to the 20%
40%
60%
80%
0%
20%
40%
60%
80%
100% 0%
share of different types of organisations participating in FP7. FP7‐
FP7‐
Among the participants from EU‐13 COOPERATION
COOPERATION
there tends to be more SME and more public sector organisations FP7‐IDEAS
FP7‐IDEAS
involved, and smaller shares of large companies when compared to EU‐15. FP7‐PEOPLE
FP7‐PEOPLE
FP7‐COOPERATION shows similar FP7‐CAPACITIES
FP7‐CAPACITIES
shares of universities, research organisations and the business Universities (HES)
RTD Org. (REC)
sector across the macro regions Large private companies (PRC)
Small prive companies (SME)
with no significant differences Public Authorities (PUB)
other (incl. CSOs)
between EU‐13 and EU‐15. However there was a significantly higher share of SMEs participating among the organisations from the EU‐13 ‐ 24% of all participants, when compared to EU‐15 ‐ 16% of all participants were SMEs. FP7‐PEOPLE shows similar patterns ‐ SMEs represented 11% of participating organisations while only 6% of organisations from EU‐15 were SMEs. Overall, EU‐13 had more public sector organisations involved when compared to EU‐15 (especially in FP7‐
COOPERATION and FP7‐PEOPLE). The smaller share of SMEs from EU‐13 participating in FP7‐CAPACITIES is explained by the sub‐programme INFRASTRUCTURES where 9% of the EC contribution went to SMEs based in the EU‐15 and only 4% to SMEs in the EU‐13. Analysing the proposals submitted to the different specific programmes of FP7 provides insights into potential causes for the relatively low share of organisations from the EU‐13 in FP7 as a whole (see next page): o
The total number of participations in project proposals can be interpreted as an activity indicator. On average, 8% of the participations in FP7 proposals were accounted for by organisations from the EU‐13. This is more than twice as high as their share in FP7 funded projects. As a result, it was not a lack of activity that caused the relatively low shares of EU‐13 organisations among the participants of FP7. o
The share of participations in project proposals that scored above the threshold (=minimum) quality level can be interpreted as a quality indicator of proposals. The indicator shows how many proposals of sufficient quality and how many proposals of lower quality were submitted. While this quality indicator is on average 52% for the EU‐15 it is only 43% for the EU‐13. The biggest differences are found in sub‐
programmes where proposals were developed by individuals instead of consortia, e.g. in FP7‐IDEAS and FP7‐PEOPLE (this indication of a lower quality of proposals from EU‐13 is confirmed by an in‐depth analysis as well as shown in Annex 9.9.). o
The share of EC funded participations out of the proposals above threshold can be interpreted as a modified success indicator. The lower shares of EU‐13 is therefore caused not by a bias against the new EU Member States, but rather by a comparably high number of weak proposals submitted by, or with partners from the EU‐13. Some of most important reasons for the comparably lower share and lower success rates of the EU‐13 organisations are information and language barriers; lack of professional contacts and research networks; lack of leading Universities and Research organisations leaders in proposal matters; limited understanding of FP7; weak training in preparing successful proposals; insufficient motivation to participate in FP7; lack of practice in project management; little experience in cross‐country cooperation; generally low focus on R&D in policy and in business; few options for exploitation of research results at the national level. 100%
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme CAPACITIES
PEOPLE
IDEAS
COOPERATION
Specific Programme / sub‐programme
Health
KBBE
ICT
NMP
Energy
Environment
Transport
SSH
Space
Security
ERANET
JTI
ERC Starting Grants
ERC Advanced Grants
other activities
Initial Training
Career Development
Industry Academia Partnerships
World Fellowships
other activities
Res. Infrastructure
Res. for the benefit of SMEs
Regions of Knowledge
Res.Pot. of Convergence Regions
Science in Society
Coherent dev. of res. policies
International cooperation
total
37
activity indicator
quality indicator
success indicator
% of participations in proposals
% of participations in proposals above threshold
% of EC funded participations among participations in proposals above threshold
EU‐15
80%
75%
83%
82%
79%
73%
84%
71%
79%
81%
75%
90%
85%
83%
84%
86%
83%
81%
52%
48%
72%
79%
70%
40%
71%
70%
47%
78%
EU‐13
5%
8%
6%
7%
7%
8%
8%
15%
8%
10%
12%
5%
2%
3%
2%
5%
4%
9%
4%
38%
12%
12%
23%
39%
15%
20%
8%
8%
other
15%
17%
11%
10%
13%
18%
8%
14%
14%
9%
13%
6%
13%
15%
14%
9%
14%
10%
44%
14%
15%
9%
7%
21%
14%
10%
45%
14%
EU‐15
53%
66%
41%
49%
55%
59%
61%
58%
74%
56%
61%
70%
23%
31%
18%
65%
77%
63%
61%
65%
79%
42%
50%
43%
49%
64%
61%
52%
EU‐13
39%
54%
31%
46%
39%
49%
49%
50%
67%
49%
33%
61%
8%
9%
6%
54%
64%
45%
53%
75%
72%
38%
41%
46%
38%
46%
53%
43%
Diff
‐14%
‐11%
‐11%
‐3%
‐16%
‐10%
‐12%
‐8%
‐7%
‐7%
‐28%
‐9%
‐15%
‐22%
‐11%
‐11%
‐13%
‐18%
‐8%
10%
‐7%
‐3%
‐9%
3%
‐10%
‐17%
‐7%
‐9%
EU‐15
52%
33%
39%
68%
49%
38%
49%
19%
40%
35%
95%
72%
53%
42%
42%
21%
30%
41%
40%
82%
54%
45%
36%
27%
56%
55%
60%
41%
EU‐13
47%
31%
37%
66%
44%
33%
45%
16%
37%
34%
100%
68%
45%
50%
40%
19%
29%
36%
46%
89%
64%
42%
40%
25%
51%
64%
53%
39%
Diff
‐5%
‐2%
‐2%
‐2%
‐5%
‐5%
‐4%
‐4%
‐3%
‐1%
5%
‐5%
‐8%
8%
‐2%
‐2%
‐1%
‐6%
6%
7%
10%
‐3%
4%
‐2%
‐6%
9%
‐7%
‐2%
From a macro‐level perspective the performance of individual EU Member States in FP7 was strongly related to national RTD investments in an EU Member State. Therefore, in order to improve their shares of participation in the EU Framework Programmes, investments on a national level are needed. In addition, large countries can take advantage of existing internal networks for collaboration at a national level, and continue with the partnerships at EU level. Networking and cooperation skills may be as important as research expertise for collaborative research, which can be a barrier for newcomers from smaller countries who are not yet well connected to European networks. In order to overcome the differences in participation shares financial support measures were implemented and comprehensive awareness‐raising activities were organised by National Contact Points (NCPs). These operate in all EU‐13 countries, however differences exist between those where the system is well institutionalised (Central European EU Member States) and countries where only the first generation of NCPs have been organised (Eastern European EU Member States). In addition, FP7 has begun to inspire a process of aligning national research priorities with the European research priorities and encouraged a more strategic and applied approach to research in the EU‐13. It is frequently argued, that new EU Member States are not adequately represented in FP7 calling for respective measures for promoting their participation. However, instead of addressing regional coherence issues within the Framework Programmes and diluting the principle of excellence. The High Level Expert Group suggests strong efforts to use Structural Funds for excellence‐driven capacity building in the EU‐13 and to dedicate a specific fund for this purpose. The Smart Specialization Concept, tested in FP7 and implemented under HORIZON 2020, should help to prepare the ground in new EU Member States to increase their participation in HORIZON 2020 and beyond. In order to prevent a brain drain from the EU‐13, the mobility programmes of HORIZON 2020 should put a special emphasis on supporting EU‐13 researchers to return to their home countries and FP7‐
PEOPLE should help to further develop the scientific culture in these countries. It should be ensured that themes and topics of high importance for the EU‐13 are sufficiently addressed in the top‐down parts of the HORIZON 2020 and the possibility of some form of affirmative action favoring participants from the EU‐13 should be explored. At the national level, more information about rules of funding and opportunities should be provided via the NCPs and measures should be implemented in order to stimulate researchers to take leading roles in projects in order to increase their visibility, impact and budget share. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 38
3.12. Monitoring and evaluation of FP7
Monitoring and evaluation activities fulfil a range of different functions in the context of FP7: 
Legitimisation is the main function of monitoring and evaluation activities. The Framework Programmes absorb a significant amount of public finances and the allocation of these funds reflects political decisions. Therefore, accompanying monitoring and evaluation processes of use and impact are indispensable. Monitoring reports and evaluations at regular intervals were implemented to increase transparency vis‐à‐
vis the public and political decision‐makers. 
The potential of evidence‐based decision making was not fully leveraged in FP7. Decisions about programme design and implementation were mainly based on simplified indicators (such as success rates) and current problem situations; while innovation systems are characterized by complexity and uncertainty and thus call for adaptive decision‐making based on a long‐term perspective. Future Framework Programmes would benefit from a much more comprehensive data base for systematic use and knowledge sharing. 
Continuous learning could become the ultimate aim of monitoring and evaluation activities. In order to maximise their potential as generators of strategic intelligence, monitoring and evaluation activities need to be integrated in a system that enables continuous collection and combination of information. Periodic syntheses of compiled information would then significantly improve the basis for decision‐making. In addition, meta‐evaluations should be considered to ensure the quality of evaluations. Under FP7, opportunities for continuous learning were missed to a great extent. Monitoring
The monitoring of FP7 was primarily carried out at the programme level; i.e. based on data of proposals, funded projects, involved partner organisations and awarded EC contributions. The CORDIS (Community Research and Development Information Service) platform provides public access to information about funded projects, as well as additional information about proposal data, to which to EC staff, the National Contact points and selected experts have restricted access. SESAM, the European Commission Online Reporting Tool, contains extensive information about deliverables, milestones and dissemination activities of funded projects that are provided by the project coordinators – valuable data that has not been used for monitoring the implementation of FP7. In 2013, the openAIRE platform was set up to facilitate the monitoring of the scientific output of the Framework Programmes. openAIRE combines data on funded projects with data from a range of other data bases, and currently lists about 12 million entries. Occasionally, reports have also included patent and media data. Annual Monitoring Reports provided information on the number of proposals, projects and participating organisations under FP7. These reports were structured according to the programme structure of FP7, but did not address underlying policy aims. A monitoring system with particular focus on the EU Sustainable Development Strategy was implemented in 2009 and has been maintained since (https://www.fp7‐4‐sd.eu/). This system connects FP7 with aims and operational objectives of the EU Sustainable Development Strategy and a selection of Flagship Initiatives of the Europe 2020 strategy. Thus, it enables the analysis of FP7 data in relation to the Sustainable Development Goals (see Chapter 7.3). The application of this monitoring system to improve the governance of FP7 has been limited to providing information for drafting new work programmes. The following aspects of monitoring activities of FP7 and their data bases leave room for improvement: 1.
Only a small part of the data stock was continuously maintained and subjected to quality assurance. Consequently, the task to identify organisations that participated in both FP6 and FP7, for example, calls for elaborate manual comparison. In some cases, there are significant deviations between data stocks (e.g. between proposed EC contribution and actual budgets of funded projects). Furthermore, the categories of organisations in project proposals are inconsistent and often incorrect. 2.
Continuous monitoring was impeded by a range of missing pieces of information. There is, for example, no consistent collection of data at the level of the individual researcher. This makes it impossible to analyse the impact of the Framework Programmes on their mobility behaviour and careers. Furthermore, it is impossible to analyse gender issues to a satisfying degree. While detailed information about female participation and gender issues was collected systematically throughout FP6, such information was no Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 39
longer collected for FP7. Finally, there is no data on the participation in Framework Programme by organisational unit. This restriction renders analyses of actual cooperation and allocation of skills impossible, while social network analyses remain superficial. 3.
When data is adjusted or improved, this is neither transferred to the data stock used by DG R&I nor made available to subsequent evaluations. As a consequence, successive evaluations may build upon different data sets and thus reach divergent conclusions. 4.
The consistency in the precise meanings of certain key terms is insufficient and some indicators are misleading. “Success rates” are, for example, being used to indicate insufficient funding budgets (e.g. in FP7‐IDEAS) as well as insufficient quality of project proposals (e.g. in comparisons between regions). This report provides a first differentiated perspective on evaluation results (see Chapter 3.9). To maximise the utility of a professional and systematic monitoring system, it is essential to improve the scope and quality of data stocks concerning the Framework Programme and to systematically process monitoring results in a form that allows policy‐makers to actively use them for improved decision‐making. Evaluation
Although evaluations of FP7 were carried out on a regular basis, their form and quality varied substantially. Overall, about 140 evaluations and assessments were contracted on different themes and cross‐cutting issues of FP7 (see Annex 9.2. for the complete list) for an estimated budget of more than 20 million euro. Considering this budget, the varying levels of quality and lack of application of results is astounding: 1.
Rather than utilizing evaluations to support continuous learning and evidence‐based decision making, they have often been employed in micro‐political contests for resources and attention. There is no centralized award procedure for evaluations. Evaluations of specific programmes, themes and sub‐programmes were designed, tendered, awarded and managed by those units responsible for the respective programme, theme or sub‐programme. In this process, the evaluation unit in DG R&I only performs an advisory role. In this context, the recent detachment of this evaluation unit in DG R&I from the unit responsible for the CORDA data stock appears particularly counterproductive. 2.
There was little continuity of individual actors involved in evaluating the Framework Programmes; resulting in permanent losses of already acquired knowledge. This applies to the various units of DG R&I that award evaluations; the staff of the evaluation unit in DG R&I and CORDA; the pool of service contractors that carry out evaluations; and the high level groups that are summoned to evaluate and give advice. The recent outsourcing of programme implementation activities to agencies has further increased the existing demand for monitoring, governance and control. In addition aspects of diversity and gender have not been sufficiently taken into account in establishing evaluation panels and expert groups. 3.
Evaluation reports were structured differently, focused on different issues and used different data stocks. Furthermore, cross references between different evaluations are mostly absent. A centralised repository for FP7 data and corresponding evaluations was not available or systematically assessed before this report. The knowledge repository included in this report provided the possibility to scan evaluations for themes and key words and to systematically utilise their results for the first time. It should be used as a first step towards a more systematic way of storing and processing evaluations of Framework Programmes in the future. Given the size of research expenditures of FP7, a strategic and professional monitoring and evaluation system is required that increases transparency and serves as a comprehensive and trusted source of evidence‐based decision making. Key data sets need to be developed further, consistency of data needs to be ensured and evaluation methods, questions and report formats should be harmonized. The wide range of individual evaluations should be better utilized to build up a coherent knowledge base that allows for continuous improvement of the Framework Programme. A rigorous approach towards evaluation syntheses and meta‐
evaluations will enable the systematic access to findings and ensure a higher quality of evaluation studies. Establishing such a monitoring and evaluation system will require additional budget allocation and investment in personnel within DG R&I, as well as a more centralized approach to evaluation. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 4.
FP7 impacts on European excellence in science
4.1.
Scientific excellence as a key concept of FP7
40
Scientific excellence was an overarching aim of FP7. It is implemented across all four specific programmes and reflected in the evaluation and selection criteria of project proposals: 
In FP7‐COOPERATION and FP7‐CAPACITIES, collaborative research projects as well as coordination and support actions were evaluated along three equally weighted criteria: (a) Scientific and/or technological excellence: In order to achieve a high evaluation score a proposal had to demonstrate its potential to progress beyond the scientific state‐of‐the‐art. In addition, the fit of the thematic orientation of the proposal to the respective topic was evaluated. In the case of two proposals with an equally high score, in the absence of sufficient funds for both projects, the one with the higher score in excellence was funded. (b) Quality and efficiency of the implementation and management: This criterion was included to evaluate the project partners’ track records as well as budget allocation and management procedures. (c) Potential impact through the development, dissemination and use of project results: This criterion focused on the scientific well as economic and societal impacts. In FP7‐COOPERATION and FP7‐CAPACITIES, the funding scheme was applied in two stages: (1) In the first stage, the scientific and/or technological excellence was assessed; and (2) in the second stage, all three criteria listed above were considered. Other funding schemes such as ‘Research for the benefit of special groups’ also applied these three categories. In contrast, the scheme for evaluating and selecting proposals for JTIs applied a different set of criteria that put a much greater emphasis on innovation and market‐
related criteria. Overall, most of the evaluation schemes that were applied in FP7‐COOPERATION and FP7‐CAPACITIES considered scientific excellence as well as potential impacts and the efficiency of implementation. 
In FP7‐IDEAS, all types of proposals were evaluated according to three differently weighed criteria: (a) Potential of the principle investigator to become an independent research leader (45% of the total evaluation score): For assessing this criterion, the CV and publication list of the principle investigator were often decisive. (b) Quality of the proposed research project (45% of the total evaluation score): This criterion referred to the scientific excellence of the proposed project. (c) Research environment (10% of the total evaluation score): This criterion addressed the estimated support the principle investigator was likely to receive at the organization where he or she planned to carry out the proposed research. The percentage weighting shows that the outstanding quality of an idea was considered equally important to the track record of the principle investigator in FP7‐IDEAS. 
In FP7‐PEOPLE, scientific quality, implementation and impacts all played an equally important role in the evaluation and selection of proposals. Depending on the kind of proposed project, additional criteria were added: For individual fellowships, for example, the track record of the individual grantee was evaluated; while in partnership‐oriented projects additional aspects of training and knowledge transfer were considered. In the quest to ensure that scientific excellence is guaranteed across all specific programmes of FP7, there is another aspect to consider in addition to the criteria of evaluation and selection of proposals: the composition of the review and evaluation panels who apply these criteria. Because of the size of the panels and the number and diversity of the applicants, the selection process is less sensitive to local powers or coalitions of thematic experts than on the national level. However, scientific excellence seems to have differed across the specific programmes of FP7. While ERC representatives have repeatedly emphasized the scientific excellence of peer review evaluation panels for FP7‐IDEAS, panels for FP7‐COOPERATION have occasionally been criticized for their composition; particularly pointing out that economical and societal impacts were mostly assessed by researchers, due to the low shares of policy‐makers, industry and civil society representatives on review and evaluation panels. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 4.2.
41
Scientific output and impact
Research excellence necessitates proper evaluation criteria and impact measurement. There is no universal definition for research impact. It could be identified as changes in knowledge and understanding; changes in access to research; changes in attitudes and beliefs; changes in behaviour; and changes in outputs or citations. Academic impacts can be measured by citations in other academic work, international prizes, nominations to prestigious bodies (national academies) or the creation of a new scientific domain; while the external or non‐
academic impacts can be measured by creation of companies, by references in the “trade press or in government documents or by coverage in the mass media”. There are various sets of quality criteria in evaluating research excellence which vary in detail and approach. The conceptual elements include scientific merit, relevance, originality, etc. At the level of outcomes, the most frequently used indicators for scientific excellence are the number, quality and citations of scientific publications. Metrics such as publication rates, impact factors, and citation counts have gained significance in research evaluation even though these metrics are far from perfect indicators of research excellence. OpenAIRE is the central point of open access infrastructures and services for research, be it current or future projects funded by the European programs FP7 and HORIZON 2020. Through it over 50 organizations from all over Europe work to make publications and their respective research data easier to find and re‐use. The OpenAIRE portal includes over 11,5 million open access documents from over 600 data providers. Based on the most recent data12 on scientific publications documented in OpenAIRE more than 165.000 publications originated from FP7 projects. In total numbers the highest output could be found in FP7‐IDEAS and in the themes Health and ICT in FP7‐COPERATION. The highest average number of publications per million Euro was achieved in FP7‐IDEAS, as well as in the themes Health and Environment in FP7‐COPERATION. The comparably high numbers of publications per project in the Research Infrastructure programme can be explained by a high number of researchers using these infrastructures and mentioning this fact in their publications. average average total EC number of number of number of number of contribution publications publications projects
publications (in million (openAIRE)
per million per project
euro)
euro
7.834 86.726 11 28.336 3,06 FP7‐COOPERATION
Theme 01 ‐ Health
1.008 25.408 25 4.792 5,30
Theme 02 ‐ KBBE (Agriculture)
516 5.826 11 1.851 3,15
Theme 03 ‐ ICT
2.328 32.964 14 7.877 4,18
Theme 04 ‐ NMP (Nanotech)
805 6.640 8 3.239 2,05
Theme 05 ‐ Energy
368 1.761 5 1.707 1,03
Theme 06 ‐ Environment
494 9.870 20 1.719 5,74
719 1.015 1 2.284 0,44
Theme 07 ‐ Transport
Theme 08 ‐ SSH (Socio, Eco., Human.) 253 914 4 580 1,58
Theme 09 ‐ Space
267 1.730 6 713 2,43
Theme 10 ‐ Security
314 598 2 1.295 0,46
ERANET
26
no data
no data
313
no data
JTI
736
no data
no data
1.966
no data
4.525 62.084 14 7.673 8,09 FP7‐IDEAS
10.715 21.090 2 4.777 4,41 FP7‐PEOPLE
2.025 10.992 5 3.772 2,91 FP7‐CAPACITIES
Res. Infrastructure
341 8.410 25 1.528 5,50
Res. for the benefit of SMEs
1.028 525 1 1.249 0,42
Regions of Knowledge
84 14 0 127 0,11
Res.Pot. of Convergence Regions
206 1.784 9 378 4,72
Science in Society
183 162 1 288 0,56
Coherent dev. of res. policies
26 3 0 28 0,11
International cooperation
157 94 1 173 0,54
Specific Programme /
sub‐programme
Grand total
25.099
166.110
7
44.559
3,73
Grand total = unique publications (without double counts of publications that are mentioned in several FP7 projects)
Bibliometric analysis can be used to address the questions relating to scientific excellence – the extent to which the researchers are among the leaders in their field. The dissemination of results is therefore an integral part of the scientific endeavour. Furthermore, the ERC Scientific Council has encouraged and supported the provision of open access to the results of ERC funded research since the very beginning. At the level of impacts, the analysis of scientific excellence is not yet supported by quantitative indicators. FP7 has contributed to the increased publication records of European researchers and organizations, as well as to the strengthened position of European countries to compete internationally in terms of publications and 12 openAIRE download from Oct 27th 2015 Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 42
citations. Research output definitely increased during FP7, particularly publications in high ranking journals illustrating high quality of research. Between 2002 and 2012 the EU share of the top 1% most cited articles increased from 28,2% to 29,8% while the US share fell from 57% to 46,4%. China’s share went from 0,3% to 5,8%. To put this into perspective compared to these regions share of all publications, in 2012, articles with U.S. authors were among the top 1% most cited articles about 74% more often than expected, based on the U.S. share of all articles, compared with 85% in 2002. Between 2002 and 2012, EU‐authored articles became more influential on average. In 2002, they were cited 21% less often than expected among the top 1% most‐cited articles; in 2012, the EU improved to 6% less often. In 2012, China’s share of highly cited articles was 37% less than expected. For now there is still a significant gap. One indication of this is given in table 20 above which shows the location of the scientists rated as the most influential in 2014. The ERC uses two complementary sources to carefully track the publications produced by its funded projects. Firstly, the ERC automatically collects publications which acknowledge its funding from specialized bibliometric databases, and secondly, it records the publications reported by the Principal Investigators during the scientific reporting of their projects. Taken together these produce reliable datasets of publications funded by the ERC. As of 15 August 2014 over 29 000 publications acknowledging ERC support have appeared in the international, peer reviewed journals indexed by the Web of Science database. These publications report findings from both ongoing and finalized ERC projects. Because of the profile of the ERC’s budget over FP7, the large majority of ERC projects are still ongoing and therefore the final number of publications from ERC projects funded under FP7 will be substantially higher. The 314 projects already completed (187 Starting grants and 127 Advanced grants started in 2007‐2008) have produced 10.796 papers, or an average of 23 per grant in Life sciences, 48 in Physical and Engineering sciences, 18 in Social sciences and Humanities. With respect to patents, it is shown that out of 107 completed projects in Life sciences, 19 had at least one patent, with a total number of patents of 30, while out of 156 completed projects in Physical and Engineering sciences 34 had at least one patent, totalling 68 patents. Overall, 53 projects had at least one patent, or 20,2% of the total 263 completed projects. The EURECIA survey on grantees and a control sample has examined the impact on careers in‐depth. The most important benefit was autonomy; the ERC allowed talented researchers to have much more freedom in their research earlier and more substantially than the ordinary academic life in the national context. Second, there was a clear advantage in status (both within the academic community and at the institution). The ERC grant was a mark of excellent quality. However, in terms of career advancements, ERC grant holders only have marginal advantages, since the structure of careers at European universities and PROs was still rigid and has not adjusted to the novelty created by the ERC. The RI programme focused in supporting networks of existing research infrastructures, including the deployment of e‐Infrastructures. The analysis shows a high involvement of the research communities, which is in line with the primary function of research infrastructures. In contrast, there was little participation by actors in the private sector, suggesting a limited focus on innovation. The RI programme also managed to engage the best organisations in the different fields. Positive is also the relatively high participation and funding rate of organisations located in smaller EU‐15 and newer EU Member States, setting the base for a strong effect on European cohesion. According to the evaluation of RI, improved flow of knowledge was considered important, in particular for users located in the newer EU Member States, along with closer collaborations between scientists and ICT developers (e. g. in the energy, earth & environmental sciences). Industry participants stressed the importance of research industry collaboration. 4.3.
Cases of outstanding scientific achievement
Several outstanding scientific Prizes have been awarded to researchers who benefited from ERC grants: in 2014 alone three grantees received the Nobel Prize and two grantees received Fields Medals. Overall, there are 11 Nobel Prize winners and 5 Fields medallists among the grantees. The ERC projects are producing and disseminating a very substantial number of research findings, as well as having the most significant and highest research impact worldwide in the most prestigious journals. Publications in top level journals are a measure of impact, influence and prestige. High impact journals are defined to be the top 10% (in terms of SJR index) of all journals within a given scientific category and the top 1% most frequently cited publications. This analysis uses the methodology of the US National Science Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 43
Foundation and is based on all the publications acknowledging ERC funding and recorded in the Web of Science database. It showed that overall 12% of these publications were in the top 1% most frequently cited publications worldwide. On the same basis, the number of the publications in the top 10% was 855 out of 1996, or 43%. The table below shows the total number of publications acknowledging ERC funding including that have appeared in two of the most prestigious scientific journals (Nature and Science). Both of these analyses show, using different methods and data bases, that a substantial proportion of ERC funded articles are among the most frequently cited publications worldwide, including in the most significant top 1% category. Publications acknowledging ERC funding 2008 2009 2010 2011 2012 2013 2014 (as of Aug. 2014) Total Nature 2 13 30 51 70 102 75 343 Science 3 11 26 46 69 90 62 307 Total Nature and Science 5 24 56 97 139 192 137 650 All publications 51 592 1.944 4.114 7.041 10.504 6.073 30.319 There are already many discoveries from ERC funded projects which have been hailed as "landmark" or "exceptional advances". For example selected scientific publications acknowledging ERC‐funding have been featured by editorial boards of scientific journals or highlighted in post‐peer review systems such as Faculty of 1000. Given that the ERC is funding research of the very highest level, it is entirely possible that ERC funded research could lead to a heavy‐tailed impact. For example, the currently most frequently cited publication acknowledging ERC funding was published in Science in 2011 and has already been cited ca 3000 times (Science, 2011, Vol. 334, 629‐634). As is the case with a lot of cutting‐edge science the significance of this paper is difficult to grasp for the layman. However, Science itself summarised the impact as a paradigm shift in this area: “Dye‐sensitized solar cells have captured the imagination of a wide range of scientists and engineers who are striving to solve the world's energy problem.” The currently second and third most frequently cited publications acknowledging ERC funding (amongst many other funders) are the papers reporting the discovery of the Higgs Boson that appeared in 2012 (Physics Letters B, 2012, Vol. 716, 1–29; and 30‐36). The following four most frequently cited publications acknowledging ERC funding at the moment are a series of new findings on graphene and other similar and complementary materials, which appeared in Science or Nature (Science, 2009, Vol. 323, 610‐613; 2012, Vol. 335, 947‐950; Nat. Photonics, 2010, Vol. 4, 611‐622; Nat. Nanotechnol. 2011, Vol. 6, 147‐150) showing the intensity of the research effort in this new area finally leading to the EU’s biggest ever research initiative Graphene Flagship. The other frequently cited publications to acknowledge ERC funding reports findings on the large scale structure of the universe from the Sloan Digital Sky Survey (Astrophys. J. Suppl. S. 2009, Vol. 182, 543‐558), and on how cells store energy with implications for the treatment of Type 2 Diabetes, and describes an open‐source platform for biological‐image analysis (Nature, 2009, Vol. 458, 1056‐1060; Nature Methods 2012, Vol 9, 676‐682). Similar stories could be told of many of the publications reporting findings from projects that have been funded by the ERC. As described above, many hundreds of publications acknowledging ERC funding have already appeared in the world’s top journals and are being cited in the very top percentiles of research findings worldwide. We need to issue a warning against the request that frontier research must demonstrate its impact at a very early stage. By its very nature, frontier research is working at the edge of knowledge, where new knowledge is generated on a continuous basis with high levels of uncertainty. It seems that the issue of “closing the gap” between discovery, invention, and commercial exploitation (if possible) is not yet addressed systematically. The new funding schemes would be important to experiment with new solutions for making the results of ERC research available, understandable and exploitable to technological communities across Europe. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 4.4.
44
European excellence on a global scale and the role of FP7
The role of FP7-IDEAS for supporting Individual excellence
From 2007 to 2013, FP7‐IDEAS has awarded around 4.500 grants for 7,5 billion euro, selected from more than 43.000 proposals. Almost all grantees (99%) worked for a university or a research organisation. Each ERC grantee employed an average of six team members, thus contributing to train a new generation of excellent scientists. To date, the ERC has supported some 26.000 other team members, offering cutting‐edge research training for nearly 7.000 doctoral students and about 10.000 postdoctoral researchers. There was a strong concentration of beneficiaries. 12 organizations received more than 2 billion euro (26% of the FP7‐IDEAS). 60% of the grants are attributed to the first 100 organizations. More than 90% are attributed to less than 600 organizations. The first 11 countries received more than 90% of the grants. The correlation index between the number of grants per country and the Global Effort in Research and Development (GERD) of the country is 0,81, which indicates a strong correlation. There was also a very strong correlation of 0.97 between the number of grants per country and the number of publications within the 10% most cited. The success of FP7‐IDEAS and the ERC is an example of policy‐initiated change leading to the Europeanisation of science needed to overcome duplication and inefficient use of resources. There is also some evidence to suggest that the ERC is leveraging different research strengths through its competitive funding and that Europe is attracting and retaining outstanding researchers. However, the real impact on attractiveness and mobility is low since most of the grantees were already in their institutions when they made their proposal. ERC is a major breakthrough, stimulating ground‐breaking research. The European Research Council (ERC) has succeeded in attracting and funding world‐class research and is playing an important role in anchoring research talent. ERC advanced grant winners are world‐class excellent researchers. Several Nobel Prizes and Fields Medal winners are ERC grantees even though their career naturally started before they were awarded the grant. The complementarity of starting, consolidator and advanced grants is appropriate. The two other schemes, synergy and proof of concept, although too young to be evaluated in a rigorous way, appear less convincing and have to evolve in the future. The role of FP7-PEOPLE for supporting Individual excellence
FP7‐PEOPLE supported some 50.000 mobile researchers over 140 different nationalities and their research projects have been undertaken in more than 80 countries. This testifies to the world‐wide openness of the programme and its important contribution towards enhancing the knowledge transfer and the quality of research undertaken. Nearly 24% of MCA fellows were researchers from countries outside the EU Member States or Associated Countries. Thanks to the International Research Staff Exchange (IRSES), some 30.500 researchers could be seconded from third countries to EU‐28 to establish and strengthen scientific collaboration. To further enhance intersectoral collaboration and to involve the non‐academic sector in doctoral training, so that skills of researchers better match public and private sector needs, a pilot on European Industrial Doctorates has been introduced in 2012 as part of the ITN scheme. FP7‐PEOPLE supported 58 EID projects in which some 240 PhD candidates were trained. Each participating researcher must be enrolled in a doctoral programme and spend at least 50% of their time with industry partner(s). This is still a very low number of people to have any visible impact on a European scale. However, the concept is very interesting and should be developed in HORIZON 2020. FP7‐PEOPLE has proved highly relevant in terms of tackling the most pressing needs and challenges related to the implementation of research and innovation policy, as well as to the competitiveness and socio‐economic needs of Europe. FP7‐PEOPLE has been highly effective in terms of contributing to the single ERA and realising the goals of the Europe 2020 strategy. There is a positive influence of the programme in terms of enhancing employability and mobility of researchers in Europe and beyond, contributing to the free movement of knowledge and opening of the ERA to the world, increasing the number of researchers in Europe and creating more attractive opportunities and preconditions to choose a research career, as well as in terms of promoting excellence of research training in Europe. The added value of FP7‐PEOPLE was high in terms of providing beneficiary researchers with better career development and mobility opportunities, increasing the volume and scope of research, and providing the example of good practice for national authorities. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 45
Excellence at EU level
The EU is facing increasing world competition, in particular at the higher end of global value chains. In 2011, more than 70% of the world’s knowledge production was taking place outside the EU. Since 2008, developed Asian countries have gained increasing shares of global value chain income including income from medium‐
high and high‐tech products. However, Europe remains one of the main knowledge production centres in the world today, accounting for almost a third of the world’s science and technology production. The EU has managed to maintain its competitive knowledge position to a greater degree than the United States and Japan and is making progress towards its R&D intensity target of 3% by 2020. However, the US and Asian research and innovation efforts are often more strategically oriented. Science and technology development in Asia and the United States is more focused on transformative and pervasive technologies and more oriented towards emerging global markets. In comparison, the EU is less focused on strategic areas and tends to scatter its efforts on a wider range of scientific fields and technologies, with the risk of dominating none. FP7‐PEOPLE contributed actively to a knowledge‐based economy through a coherent framework addressing European needs for more researchers, better career opportunities in all research sectors and development of key skills to meet future challenges. 50.000 mobile researchers (including 10.000 PhD candidates) have been supported under FP7‐PEOPLE, all of them receiving high‐quality research training and excellent career opportunities in both public and private sectors. The principles guiding the doctoral training offered under FP7‐PEOPLE (ITN) have been recognised as best practice in Europe: international, intersectoral and interdisciplinary environment created by consortia from different countries and offering a significant exposure to industry, development of transferable skills to supported researchers, including on entrepreneurship, business skills and Intellectual Property Rights, as well as attractive working and employment conditions. The FP7‐PEOPLE programme has attracted international talent and contributed to the opening up of the European Research Area. Nearly 24% of the 50.000 FP7‐PEOPLE fellows were nationals of third countries. 46% of researchers coming to the EU from industrialised countries stayed in Europe after the end of their fellowship. 81% of the FP7‐PEOPLE beneficiaries stated that the programme provided attractive international mobility opportunities for researchers in their organisation. 76% indicated that it provided more opportunities to attract researchers to their organisation from abroad. 91% of all organisation beneficiaries considered that the MCA provided attractive opportunities to create new, or join existing international research networks. Enhancing cooperation between universities and industry in terms of knowledge sharing, training and broad skills development was a key element of FP7‐PEOPLE. FP7‐PEOPLE strengthened the resources of those institutions able to attract researchers internationally and thereby encouraged the spread of centres of excellence around the EU. FP7‐PEOPLE has gathered the best European and non‐European actors in research. In FP7 all the 100 best ranked European universities in the Shanghai ranking list have been actively involved in FP7‐PEOPLE. At the same time, the 65% of the outgoing European researches have carried out part of their research projects in the top 50 world universities. The FP7‐RESEARCH INFRASTRUCTURE programme contributed to the cohesion of the European research landscape and to the breadth and quality of the research infrastructure services. This was achieved particularly through the combined efforts of activities and the policy mix. European Added Value was an inherent part of the overall FP7‐RESEARCH INFRASTRUCTURE objectives, as it focused on activities and benefits difficult to achieve by actions of individual EU Member States alone. The RI programme made an important contribution to increasing international collaboration and opening up the European Research Infrastructure to the world, to the mutual benefit of the European and international research communities. The programme addressed the fragmentation of policies at national and European level and was successful in improving coherence of Research Infrastructure policy making based on the ESFRI roadmap and projects. The Research Infrastructures programme enhanced the optimal performance and use of existing, the development (or major upgrade) of new Research Infrastructure of pan‐European interest, and provided specific coordination and support actions for research infrastructures and their communities in Europe and worldwide. Due to its scientific nature, there was a high involvement of the research communities in the programme, and less participation by actors in the private sector. The e‐Infra activities were more relevant to industry and SMEs. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 46
Universities and research organisations were the principal beneficiaries of FP7 funding. This has been reinforced by the advent of the ERC, since nearly all the grants go to scientists attached to these organizations. The concentration of scientific stars in a small number of universities and research organisations leads to a better global ranking of these institutions. However, special care has to be given by ERC to the fact that this does not lead to a unique way of doing research and writing proposals. This is particularly important in humanities and social sciences where the domination of the English language and the “northern culture” has to be questioned. FP7‐IDEAS, ‐PEOPLE and ‐RESEARCH INFRASTRUCTURE have a strong positive impact on National Research Systems but no clear impact on National Innovation Systems. By awarding grants to highly competitive individual researchers through European‐wide competition, the ERC and MCAs have a strong direct positive impact on national research systems. This has been acknowledged by a large majority of European stakeholders. There is a strong need for more coherence. The various programmes for stimulating individual research excellence, mainly FP7‐IDEAS and ‐PEOPLE, as well as the Future and Emerging Actions (FET) actions, were processed independently and with a lack of common objectives. With the development of new programmes (EIT for example), the need for a more coherent strategy is stronger. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 5.
47
FP7 impacts on European research and innovation
systems
The overriding aim of FP7 was to develop a knowledge‐based society and economy and to contribute to Europe becoming the world’s leading research area. This chapter addresses whether FP7 has helped to strengthen the European research and innovation systems and by which means.13 FP7 showed a strong focus on promoting world class state‐of‐the‐art research and technological development. At the same time FP7 had to stimulate and strengthen the development of the ERA. The programme was supposed to meet these targets while since the early 2000s the global context of knowledge production, innovation and economic development has been changing considerably: the "triad" of North America, Japan, and Europe is no longer determining global economic development alone, competition has become multi‐
polar (including China, Korea, Latin‐American countries and others). Also, since 2008 the world economy – and in particular Europe – has been suffering from a serious economic crisis. Experts now anticipate that in the next two decades global multi‐polarity, economic and political instability, and major social and environmental challenges will determine the conditions for knowledge production, innovation, competitiveness, and sustainable welfare world‐wide and certainly in Europe 14 . It is in this context of challenges to the competitiveness of the European Research and Innovation System that FP7 was also expected to help to create an ‘Innovation Union’. Though difficult, FP7 was one of the few research and innovation funding programmes which kept its budget, contrary to many national programmes. This has had a positive effect on the perception of European programmes in the research community itself. Consequently, FP7 can be characterised as a highly relevant, though not a dominant sponsor in the European research and innovation system(s): in 2010, for instance, public and private sponsors in the EU Member States invested 245,6 billion euro into research and development, representing some 2,0% of GDP (an increase over the previous years); FP7 added about 6 billion euro, amounting to about 2,6% of all European R&D expenditures or about 8% of the publicly available resources in the EU Member States15. Comparing such figures, though, is difficult: Only limited shares of the total spending by EU Member States is allocated on a competitive basis. If competitive funding is compared, the FP7 budget represented circa 25% of R&D funding16 at national level, depending on the EU Member State. This is significant and has a significant impact at national level. Even so, given this relevant but limited role of FP7 vis‐à‐vis the other actors in European innovation systems, in order to be effective, strategic positioning is required. The final evaluation of FP6 (2009) had already suggested that FP7 “should act as a ‘coordinator’ or ‘lubricant’ for multi‐actor initiatives (like the ERA‐NETs)”17, in other words, it should work as a strategic facilitator of ERA, with strong leveraging effects. The High Level Expert Group found that FP7 clearly strengthened the European Research and Innovation System. Major initiatives in this context were the contribution of FP7 to capabilities of the knowledge‐based society and economy (5.1), the improvement of research capabilities and capacities (5.2), the use of dedicated 13 In this report in the notion ‘Research and Innovation System(s)’ is used to capture the research and innovation ‘landscape’ of related institutions, organisations and their interactions, while the notion ‘European Research Area (ERA)’ is used to denote European Union’s political concept and programme aiming to create “a unified area open to the world, in which scientific knowledge, technology and researchers circulate freely” (see http://ec.europa.eu/research/era/index_en.htm). 14 See e.g. the future scenarios suggested by the FP7 SIS project RIF – Research and Innovation Futures 2030: From explorative to transformative scenarios (project website: http://www.rif2030.eu/).
15
European Commission ERAWATCH, Research Funding Structure: Overview of funding. Available at:
http://erawatch.jrc.ec.europa.eu/erawatch/opencms/information/european_perspective/EU_Profile/eu_profile?section=Res
earchFundingStructure&subsection=OverviewFundingFlow
16 Estimation based on Ehardt‐Schmiederer, M.; Brücker, J.; Milovanović, D.; Postl, V.; Kobel, C.; Hackl, F.; Schleicher, L.; Antúnez, A. (2014): 7. EU‐Rahmenprogramm für Forschung, technologische Entwicklung und Demonstration (2007–2013), PROVISO‐
Bericht Frühjahr 2014, Wien 2014; Österreichischer Forschungs‐ und Technologiebericht / Austrian Research and Technology Report 2014 17 Rietschel, E., Arnold, E., Čenys, A., Dearing, A., Feller,I., Joussaume,S., Kaloudis, A., Lange, L., Langer, J., Ley,V., Mustonen, R., Pooley, D., Stame,N. (2009), Evaluation of the 6th Framework Programme for Research and Technological Development 2002‐
2006.
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 48
ERA‐instruments and the strengthening of national systems (5.3), a better coherence of national and transnational policy efforts (5.4), and finally through international collaboration with ‘Third Countries’ (5.5). 5.1.
Contribution of FP7 to foster collaboration in research and innovation
A central mechanism to meet the overarching targets of FP7 (‘excellence’, ERA, ‘Innovation Union’) was the stimulation of and the support for collaboration in research and innovation, across national and institutional borders. The study “European Added Value of EU Science, Technology and Innovation actions and EU‐Member State Partnership in international cooperation” analysed the impact of FP7 funding and identified a number of so‐
called ‘criteria for added value’18. The Expert Panel endorses these criteria and supports the conclusion that the Framework Programmes over the years, and FP7 in particular, have demonstrated improvements across mentioned criteria and beyond, as summarized below: Developing a culture of networking and cooperation. Over time the FPs helped and stimulated the development of a culture of cooperation and networking for which Europe has become unique as compared to other major regions or countries. Intrinsically, European researchers developed this capability, becoming obvious mainly if third country partners – not used to it ‐ joined projects. As such, FP projects managed to bring together individuals and organisations from different origins and societal sectors, as well as from research and research policy. This culture of cooperation fosters all advantages related to cooperation, pooling of resources, avoidance of duplication of similar activities, and enables the coordination of effort. Facilitating European excellence and capacity building. This area of added value mainly supports the researchers. It is the assumption that cooperation beyond national borders as well as internationally outside of Europe contributes to the increase of excellence because the best minds available work together to solve complex research questions, making use of the best methods and tools available. Thus, this capability contributes to the attractiveness of European research and innovation. Developing critical mass. Joining forces, increasing visibility and thus strengthening competitiveness in a global dimension is an important goal of FPs. An important rationale for EU action is that activities of such scale and complexity cannot be handled by any single Member State. The extent of the financial or human resources/expertise or infrastructure/equipment means that the work can only be carried out at European level in order to achieve and develop the critical mass, as well as to reduce the research or commercial risk for one single country or organisation. Fostering mutual learning and harmonisation in Europe, leading to standardisation and improved knowledge, also of international cooperation processes and practices, and addressing issues that have to do with the framework conditions for these activities, practical issues such as evaluation practices, project and programme management, and values associated with integrity issues and ethics at the level of the international cooperation. Avoiding redundancies and acting economically and effectively. International action at European level or EU/MS partnerships vis‐à‐vis third countries can result in efficiency gains by pooling of scarce public resources, leverage of public funding on private investment, and alignment of international STI cooperation priorities, allowing avoidance of duplication and rationalisation of efforts. International STI cooperation activities at EU level can contribute to the achievement of wider EU policy goals by realisation of greater economic and societal benefits and impacts. Fostering the strategic orientation of participants’ research and innovation activities, through the focus on a limited number of topics FP7 has ensured focus of efforts on topics of strategic European relevance. Participants in FP7 had to learn how to deal with this approach, over time, resulting in more strategic thinking and acting of participants involved in FP7 projects. Enhancing a culture of competition capability & excellence in Europe. The tough competition in the FPs over the decades has strengthened the know‐how on how to develop highly competitive proposals and projects. 18 Vullings, W., Arnold, E., Boekholt, P., Horvat, M., Mostert, B., Rijnders‐Nagle, M. (2014), European Added Value of EU Science, Technology and Innovation actions and EU‐Member State Partnership in international cooperation.
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 49
More importantly, it enhanced a culture of positive competition and excellence in Europe going beyond the respective “national ponds”. This push for excellence had also impacts on national systems. FP7 could achieve this at least with the help of a number of new, promising instruments. The importance of the ERC for research excellence has been addressed already in Chapter 4 of this report. Instruments such as the Joint Technology Initiatives (JTIs) have demonstrably helped to foster innovation. Set up as Public‐Private Partnerships the JTIs have been initiated in areas of strategic European interest19. JTIs represent the first experience with setting up PPPs in research at European level. They brought together EU, national and private resources, know‐how and research capabilities for a certain time with the aim of addressing major issues. This has been achieved by sharing pre‐competitive knowledge, generating critical mass, scale and scope in areas where global competitiveness is at stake. Thus JTIs were contributing to successfully position the EU globally with the development of breakthrough technologies of high innovation potential. Different types of JTIs have been set up and developed under FP7, also applying different configurations of the involvement of EU Member States. A more extensive analysis of the impact of the JTIs is provided in Chapter 6. The European Institute of Technology. The EIT was not formally part of FP7, however, due to its strategic nature, potential impact on the European Innovation System and the fact that it is partly funded by FP7 resources it is covered here. The EIT was set up as part of the renewed Lisbon Strategy for Growth and Jobs with the intention “…to reinforce our commitment to knowledge as a key to growth…to act as a pole of attraction for the very best minds, ideas and companies from around the World”. 20 As such the EIT has been set up to integrate all three sides of the Knowledge Triangle including higher education, research and business, thus contributing to Europe’s innovation agenda. Under FP7, three KICs (Knowledge and Innovation Communities), as functionally and legally separate units have been set up (“KIC Innoenergy”, future ICT “EIT‐
ICT Labs”, climate change mitigation and adaptation “Climate KIC”). So far, there is only one comprehensive evaluation of the EIT21 available, which was done early in the process of setting up the EIT and its KICs. Consequently, this evaluation has focused on the underpinning concept and structures being established to deliver the objectives and took a qualitative approach. Evidence is inevitably more limited in terms of scale and quality of KIC activities and outputs, effectiveness of partnerships and synergies, the benefits of different co‐location approaches, and wider effects on organisations and local, regional, national contexts. Overall, this evaluation came to the conclusion, that the EIT broadly met its key operational objectives in this early phase. This observation includes the fact that the EIT has succeeded in attracting businesses, educational and research partners with world‐class reputation to the KICs, even though many adopted a cautious approach to begin with. However, it is impossible to draw any conclusions with respect to the impact of the EIT on the European Innovation System. Under Horizon 2020, the EIT was allocated a budget of 2,7 billion euro, a Strategic Innovation Agenda was set up covering the period 2014‐2020, encompassing the development of further KICs as well as accompanying activities. To sum up, FP7 – with long‐standing as well as new instruments – was a key resource for the stimulation of and the support for collaboration in research and innovation, across national and institutional borders. Inevitably, this has created quite some instrumental complexity and the risk of establishing ‘silos’. Synergy potential between the different specific programmes and sub‐programmes should be assessed and duplications avoided. Effective coordination processes between the agencies in charge of implementing HORIZON 2020 have to be established. Funding instruments with a special emphasis on fostering linkages between the specific programmes should be harmonized (e.g. enabling the use of FP7‐PEOPLE funding opportunities for the preparation of FP7‐IDEAS proposals). Successful elements should be made available across the programme (e.g. foster researcher exchanges in collaborative projects in addition to the Marie Curie Actions). 19 Based primarily on JTI Sherpas' Group (2010), Designing together the ‘ideal house’ for public‐private partnerships in European research. 20 European Commission (2006), Communication from the Commission on Implementing the renewed partnership for growth and jobs. Developing a knowledge flagship: the European Institute of Technology, COM(2006)0077 final.
21 European Commission (2011), External Evaluation of the European Institute of Innovation and Technology. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 5.2.
50
Contribution of FP7 to strengthen research capability and capacity
Research capability of individual researchers, groups, networks and disciplines has been improved, in particular through joint research in the Cooperation‐Programmes22 and through Marie Curie projects and networks. A recently published study on network analysis examines the effectiveness of FP7’s network approach in achieving EU research policy objectives and fostering Europe’s international competitiveness in S&T 23 . In investigating the potential and limits of this approach, the study examines the effects of multidisciplinary, interdisciplinary, intersectoral and international collaborations on achieving the positive outcomes sought in FP7. The study’s methods included a network analysis, survey, case studies including in‐
depth interviews, a representative stakeholder workshop and regression analysis. The key findings reveal benefits from the continuity of research from FP6 to FP7. However, despite these demonstrated benefits and the increase in participants overall for FP7, the study also found a high attrition rate of organizations from FP6 to FP7. The report’s six recommendations include allowing existing networks to continue research in successive calls, while ascertaining that new members are included as this has a positive impact upon innovation. Clear guidelines outlining selection criteria expectations for the multi‐ or interdisciplinary and intersectorality of participants and the number of participating organizations, regions or countries in a project are recommended. It is also recommended the European Commission investigate why 65% of organizations present in FP6 did not participate in FP7. A study of the FP7 Marie Curie Fellowship funding concluded that it had definite beneficial impacts on a researcher’s career prospects. On several career and professional achievement indicators, Marie Curie former fellows score more positively than non‐fellows. Marie Curie‐related positive effects were more marked in particular for academic researchers, while there was room for improving collaboration and mutual benefits with the private sector. Overall, Marie Curie “enjoys a highly positive reputation and has frequently attracted talented EU researchers educated in prestigious universities. The degree of affiliation of former fellows remains high, even many years after the end of fellowship”24. Research capacities have been strengthened through the ESFRI measures. As summarized by the FP7 Interim Expert Group, FP7 was instrumental in coordinating European infrastructures activities, thus contributing to the next step of ERA development. FP7 has contributed to networking of a large number of national research infrastructures and opening them to European scientists through specific schemes. FP7 has initiated a vision for the future of European research infrastructures by harmonizing actions among MS through the ESFRI process and in particular the roadmap for research infrastructures. As a consequence, in many cases national roadmaps have been set up and MS have become receptive to hosting or participating in European research infrastructures. The European Strategy Forum initiatives for Research Infrastructures (ESFRI), the major structuring event in the FP7 period, has clearly helped to achieve more coherence and coordination across relevant research infrastructures in Europe, yet limited commitment of some EU Member States and sometimes insufficiently transparent policies by the European Commission need to be overcome in the future . As for eInfrastructures, according to the assessment by participants, the programme has been effective in the following ways: a) approximately 65% of the interviewed participants reported an impact of the Programme on an optimised functioning and development of research infrastructures in Europe, and about 50% half of them indicated an impact on a reduced fragmentation; b) the FP7 ICT eInfrastructure programme fostered and accelerated an improved structuring of the European research area and considerably enhanced European and international cooperation on research; c) industry actors that were involved in the FP7 ICT eInfrastructure programme were quite positive about the programme’s (future) effects25. 22 IDEA Consult; iFQ; PPMI (2013), Study on assessing the contribution of the framework programmes to the development of human research capacity.
23 Archambault, E., Campbell, D., Caruso, J., Côté, G., Hassan, E., Lavoie, R., Mitchell, B., Nievas, K., Rashid, M., Roberge, G., Lalonde, S.L., Ventimiglia, A., Kroll,H., Meyborg, M., Meyer, N., Møller, K., Light,D., Larsen,M., Bom, F., StryhnKoch, M., Hornnes,K. (2015), Network Analysis of FP7 participation.
24
25 Economisti Associati (2014), Marie Curie researchers and their long‐term career development: A comparative study. Mahieu, B., Hammerschmidt, R., Nooijen, A., Enzing, Ch. (2014), Evaluation of Pertinence and Impact of Research.
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 51
Under FP7 Smart Specialization was tested with limited dimension, particularly through the Regions of Knowledge Programme (RoK). Based on the idea to contribute to and foster regional knowledge based development, it represented FP7s “translation” of what has traditionally been covered by the Structural Funds. Considering that Smart Specialization strategies were relatively new, the final evaluation26 indicates that Smart Specialization has benefitted greatly from the RoK programme, in scale as well as in scope. Recommendations underline the importance of the involvement of regional authorities and enterprises as indication for the relevance of issues addressed. Future activities to be set up should focus on further developing existing Clusters and their relationships, specifically towards international relationships, rather than setting up new initiatives. 5.3.
Contribution of FP7 to mobility of researchers
FP7 had set an objective of strengthening the human research potential in Europe with the view of promoting the research profession, supporting Europe as an attraction for European and foreign researchers and thus ensuring that the European innovation systems can draw on the best talent. By doing so, FP7 aimed at strengthening research careers and improving research career prospects as well as promoting knowledge transfer between countries and sectors and supporting European innovation with competent researchers27. Promoting an open labour market for researchers in Europe is also one of the 5 ERA priorities28. FP7‐PEOPLE was the main instrument of FP7 aimed at investing in career development and researchers mobility that supported around 50.000 mobile researchers 29 . FP7‐PEOPLE, as opposed to other FP7 programmes, encouraged individual participation through a bottom‐up approach to funding through specific ‘Marie Curie Actions’ (MCA) and awarding mobility fellowships to individual researchers (MCA Fellows). Through various targeted actions the programme supported researchers from their initial training to life‐long learning and career development in academia and industry. The data on geographical mobility of FP7‐PEOPLE funded researchers30 showed EU wide mobility patterns (for more details see Annex 9.11.): 
The United Kingdom is obviously the most attractive country for MCA fellows with more than 3.600 incoming and only 600 outgoing researchers. The largest numbers of incoming researchers move to the UK from Italy (~700), Spain (~500), Germany (~500), France (~350), Poland (~220) and Greece (~200). The net gain of United Kingdom is therefore more than 3.000 researchers. 26 Based on: Debackere, K., Andersen, B., Dvorak, I., Enkel, E., Krüger, P., Malmqvist, H., Plečkaitis, A., Rehn, A., Secall, S., Stevens,W., Vermeulen, E., Wellen, D. (2014), Boosting Open Innovation and Knowledge Transfer in the European Union; Avice, E., Stevens, W., Mann, E., Figueiredo, J‐F., Maimets, T., (2013), Second interim evaluation of the RSFF; Campbell, D., Caruso, J., Archembault, E. (2013), Cross‐Cutting Analysis of Scientific Publications versus Other Science, Technology and Innovation Indicators. 27 European Parliament (2006) Decision No 1982/2006/EC of the European Parliament and of the Council of 18 December 2006 concerning the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007‐2013)
28 European Commission (2012), Communication from the Commission an A reinforced European Research Area Partnership for Excellence and Growth, COM(2012) 392 final. 29 Avramov, D. (2015), FP7 ex‐post evaluation People Specific Programme (2007‐2013): Rationale, implementation and achievements. 30 Analysis derived from data provided by FP7‐PEOPLE supporting expert (D. Avramov). Data used is based on researcher’s nationality rather than residency before the move. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 



Germany and France have experienced extensive exchange with other EU Member States, but do not show a significant net gain. In both cases the number of incoming researchers (Germany ~ 1.700, France ~ 1.200) is similar to the number of outgoing researchers (Germany ~ 1.500, France ~ 1.200). While most incoming researchers moved from the Mediterranean countries to Germany and France, most outgoing researchers of both countries moved to the United Kingdom. The Netherlands show a similar pattern but on a smaller scale and with a more prominent net gain: ~ 900 MCA fellows moved to the Netherlands (most of them from the Mediterranean countries, but also from Germany and the EU‐13 countries), while ~ 500 moved out of the Netherlands (many of them to the United Kingdom). 52
impacts of FP7‐PEOPLE on mobility
UK (3041 net)
IE (11 net)
SE (294 net)
DK (247 net)
FI (‐6 net)
FR (‐21 net)
BE (325 net)
NL (424 net)
LU (110 net)
DE (225 net)
AT (6 net)
IT (‐1568 net)
ES (‐643 net)
EL (‐436 net)
PT (‐311 net)
PL (‐678 net)
CZ (‐70 net)
HU (‐204 net)
SI (‐50 net)
RO (‐241 net)
SK (‐85 net)
BG (‐145 net)
HR (‐111 net)
EE (0 net)
LT (‐56 net)
The most significant net outflow LV (‐20 net)
is experienced by the CY (‐30 net)
Mediterranean countries: most MT (‐8 net)
of all Italy (~ 2.300 outgoing and 3.000
2.000
1.000
‐
1.000
2.000
3.000
~ 800 incoming researchers), Spain (~ 1.600 outgoing and ~ outgoing
incoming
returning / within country
900 incoming researchers), but is also relevant for Greece (~ 500 outgoing and ~ 200 incoming) and Portugal (~ 700 outgoing and ~ 300 incoming). Plausible explanations for these comparable high net outflows are the economic crisis, high unemployment rates and a lack of career prospects for young researchers. This tendency is also experienced by the majority of EU‐13 countries, but to a smaller extent: highest numbers are seen in Poland (~ 700 more outgoing researchers than incoming), Romania (~250) and Hungary (~200). These results confirm than FP‐PEOPLE significantly contributed to intra‐European mobility. Given that the available data only allows for an analysis of short term mobility, broader conclusions whether the outflow tendencies remain the same across time are unfortunately not possible. FP7‐PEOPLE also contributed to attracting researchers from outside Europe. In total around 24% of MCA fellows came from countries outside Europe, which is an increase compared to the share of 17% in FP6. Most of the researchers who received FP7‐PEOPLE funding came from China, Israel and Turkey. Even though no targets have been set by the FP7‐PEOPLE for the share of non‐European MCA fellows, significant strides have 4.000
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 53
been made during the course of FP7 in terms of improving the conditions for attracting more top talent from outside Europe by the adoption of the Scientific Visa Directive, creation of EURAXESS services in EU Member States and its information portal with a jobs marketplace31. However, FP7‐PEOPLE was only open for a pre‐
defined set of countries, mainly those that have signed an STI agreement with the EU, and Associated Countries. As a result FP7‐PEOPLE effects on attracting top research talent globally were limited by the design of the programme itself. FP7‐PEOPLE also made new efforts in promoting researchers mobility between academia and industry through mainly two activities ‐ the European Industrial Doctorates (IED) aimed at earlier stage researchers, and Industry‐Academia Partnerships (IAPP). In terms of main benefits that FP7‐PEOPLE funded researchers reported that mobility had on their careers, researchers have mostly valued access to “high quality research infrastructures and networks across all research fields, (…) access to a wider range of mobility options and to develop the kind of new research skills and competences that can support career progression and enhance employability“32. However, another survey based evaluation study of FP5, FP6 and FP7 funded researchers has concluded that the career effect experienced by MCA fellows was relatively small compared to a control group of researchers who received another mobility grant. For example MCA fellows were less likely to work in a for‐profit firm when compared to a control group; they showed a larger share of employment outside of their home country than the control group after their fellowship. In terms of sectoral mobility and disciplinary mobility over 90% in both MCA fellow group and control group did not move from one sector to another nor change their primary scientific discipline. Justifying the marginal differences between the control group and MCA fellows, the study suggests that long term career effects may take longer to materialize and are difficult to capture in the short term. Yet MCA fellows did report clear benefits in terms of prestige of an MCA fellowship. Interestingly, the same study also found out that the main reason for the control group not applying for an MCA fellowship was lack of awareness about its existence.33 In conclusion, the monitoring of MCA fellows provides only a fragmented view on the impacts of FP7‐PEOPLE on strengthening the human research potential in Europe and fostering researchers geographical mobility both in the EU and with other countries globally. Survey data that complements monitoring data provided as part of the contractual agreement by beneficiaries is also fragmented and based on very low response rates. FP7‐PEOPLE opened up the European research and innovation systems and gave a substantial number of researchers the opportunity to be trained in excellent institutions around Europe. Important strides were made towards an open labour market for researchers in Europe. Disparities across Europe exist and Mediterranean countries were seeing higher numbers of researchers leaving, while countries with strong research and innovation systems have been the main attraction points as destination countries, in the short run. In order to ensure that the programme promotes an active rotation of scientists, rather than the so called “brain drain” a better long term tracking of supported researcher’s careers has to be implemented. Furthermore, it has to be ensured that national research systems are open for returning researchers. Documenting the value of mobility, as well as implementing a tracer study to track and measure the impact of mobility on the employment status and career advancement of researchers, is necessary in order to ensure future mobility programmes contribute the most to mobility across Europe and globally. 31 Avramov, D. (2015), FP7 ex‐post evaluation People Specific Programme (2007‐2013): Rationale, implementation and achievements. 32 ECORYS (2012), FP7 Marie Curie Life‐long Training and Career Development Evaluation: Individual Fellowships and Co‐Funding Mechanism. 33 Economisti Associati (2014), Marie Curie researchers and their long‐term career development: A comparative study. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 5.4.
54
Contribution of FP7 to international collaboration (with ‘Third Countries’)
International cooperation was one of the most important objectives throughout the history of the EU Framework Programmes as well as a central objective of FP7. Promoting international collaboration was set as an objective in order to support European competitiveness globally, create contacts with scientists from outside Europe in order to provide them access to research networks, and address specific global challenges that affect third countries. In addition to a specific sub‐programme addressing international collaboration (the INCO programme in FP7‐CAPACITIES), international collaboration became an integrated issue within FP7‐
COOPERATION. FP7‐PEOPLE addressed the objectives related to strengthening human potential and attracting third country scientists. No specific goals were formulated for FP7‐IDEAS in terms of cooperation with partners from outside Europe. Promoting collaboration with partners from outside Europe aimed to foster excellence of European researchers and research organizations by cooperating with the brightest minds and frontrunner institutions around the world. Furthermore, societal challenges are best addressed by global networks. This especially incentivised collaboration with developing countries, where increased research collaboration contributed to EU development policy goals as well. On the other hand, international collaboration was limited by two other underlying logics. Firstly, promoting European competitiveness through FP7 requires protecting intellectual property rights and safeguarding the economic benefits of innovations. If the most important competitors would participate in joint research and innovation projects and share their background knowledge as well as the project outcomes, a unique competitive position would be difficult to achieve for European partners. Secondly, there was an implicit logic that FP7 budget stemming from European tax payers should mainly fund European researchers. FP7 achieved a balance between promoting international collaboration for scientific excellence and contributing to solving societal challenges on one hand, and protecting and fostering European competitiveness and European research funds on the other hand. No concrete targets for cooperating with “third countries” were set in FP7. Although FP7 stated to be open to any country willing to participate, international collaboration was limited by the funding eligibility criteria which differed for different non‐EU countries and remarkably lower than in previous Framework Programmes. As set out by the guiding notes and work programmes, FP7 funding was limited to EU Member Countries, Associated Countries34 and International Partner Cooperation Countries35, while countries classified as high income (e.g. USA, Japan, Taiwan) generally were not eligible to receive European funding for their research activities carried out within an FP7 funded project (with exceptions)36. As a result, funding eligibility remained the bottleneck for increasing extra‐EU cooperation, in particular with high income countries and FP7 budget was mainly allocated to EU Member States (90% of the total FP7 budget) and Associated Countries (9% of the total FP7 budget). FP6 ex‐post evaluation has called for specific strategies in collaborating with third countries, not only by differentiating the goals of collaboration with developing countries, emerging economies and major existing economies, but also by mainstreaming collaboration with the major or emerging economic powerhouses. This has been reinforced by the FP7 interim evaluation as well. In response, the European Commission has published a Communication in 2012 that called for a more strategic approach to international cooperation. This set out the approach to be taken in HORIZON 2020. This somewhat more strategic approach did not affect FP7 and shares of third country participants remained low. The majority of EC contributions across all programmes were allocated to EU Member States (90%). A comparably small share was received by organisations in Associated Countries (9%), ICPC Countries (1%) and High Income Countries (0,3%). The shares of participations from these countries mirror the funding patterns: 88% of participations from EU Member States, 8% from Associated Countries, 3% from ICPC Countries and 0,9% from High Income Countries. The differences between budgets and participations are caused by different EC contributions to the total project costs (around 70% for EU Member States, Associates and ICPC Countries; about 40% for High Income Countries) but also by different salary level are ICPC countries. As a result, the 34 For full list see: http://ec.europa.eu/research/participants/data/ref/fp7/116018/fp7‐third‐country‐agreements_en.pdf 35 For full list see: http://ec.europa.eu/research/participants/data/ref/fp7/206006/wp‐2013‐Annex‐1‐icpc‐list_en.pdf 36 For full list see: http://ec.europa.eu/research/participants/data/ref/fp7/90400/guideline‐third‐country‐participants_en.pdf Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 55
average EC contribution per participation was about 350.000 euro for EU Member States and Associated countries, but only around 100.000 euro for ICPC countries and High Income Countries. FP7 all specific programmes
EU member state (incl. overseas territories)
Associated countries (1)
ICPC countries
High income countries (2)
other countries
total
Participant Project Costs (million euro)
57.082
5.538
738
323
2
63.684
% EC EC contribution contribution to project % of total EC partici‐ (million euro)
costs
contribution pations
40.266
71%
90%
117.648
4.015
73%
9%
10.684
509
69%
1%
4.109
126
39%
0,3%
1.151
1
43%
0,0%
19
44.917
71%
100%
133.611
(1) according to Oct 2013 list
(2) allowed to participate but normally not eligible for funidng (exceptions apply)
EU % of contribution partici‐ per pations participation
88%
342.255
8%
375.831
3%
123.967
0,9%
109.214
0,0%
55.887
100%
336.179
When assessing international collaboration on the basis of allocated research funds, the level of collaboration with partners from ICPC and High Income Countries was marginal. This holds true for all the four specific programmes of FP7. Especially in FP7‐IDEAS, FP7‐PEOPLE and the JTIs few participants came from ICPC or High Income Countries. Only in FP7‐COOPERATION and in the INCO sub‐programme of FP7‐CAPACITIES the shares of these countries was slightly larger. Among Associated Countries Switzerland, Israel and Norway are the frontrunners in terms of EC contribution received and shares of participants. Among the ICPC countries the funds spread out more widely. The major players among ICPC partners were Russia, India, China, South Africa, Brazil and Ukraine. Among High Income Countries the highest share of EC contribution, by far, was received by organisations based in the USA (for an in‐depth analysis see Annex 9.12.). From a more qualitative point of view, studies have found third country organisations to be more willing to partner with EU organisations in FP7 projects than the other way around. For example, the importance of EU funding to start international collaboration was regarded lower among the third country partners compared to EU countries. A higher number of third country partners also reported that they participate in international collaboration even if there is no public funding provided when compared to EU coordinators. The main benefits of FP7 funding, in comparison to national funding, were reported to be access to international research communities, connection to leading scientists, better reputation, leverage for extra funding, and the formation of international consortia37. In general, academic organisations formed the core of international cooperation projects, with industry largely underrepresented among participants. The latter finding causes concern given that one of the major objectives of international cooperation in FP7 was increasing competitiveness. Furthermore, most of the survey respondents have highlighted that pre‐existing relations most often determined cooperation with third country partners. In terms of reported barriers for extra‐EU cooperation, some of the most prevalent ones in the aforementioned survey were lack of knowledge of the strengths of different countries, administrative burdens caused by international STI cooperation, lack of financial means often necessitated co‐funding, lack of trustworthy connections and political barriers in some countries. Even though there is a network of NCPs in third countries which has been found to be a useful source of information for third country participants in FP7, as well as EU organisations looking for partners; the capabilities of the different NCPs across the world differ greatly38. The shares of partners from outside Europe remained low in FP7. It was lowest in terms of partners from high income countries that could be strategic partners for Europe in fostering its scientific excellence and innovations. The lack of a more strategic approach to international collaboration persisted and led to an opaque situation with different, to some extent opposing logics. Even though the integration of international cooperation across the programme has been an important move in FP7, the lack of a strategic approach with clear objectives remained a weakness. Therefore, the High Level Expert Group sees an urgent need for a thematically 37 38 Vullings, W., Arnold, E., Boekholt, P., Horvat, M., Mostert, B., Rijnders‐Nagle, M. (2014), European Added Value of EU Science, Technology and Innovation actions and EU‐Member State Partnership in international cooperation.
Remotti, A., Damvakeraki, T., Plooder, M., Sterner, C. (2014), International Science and Technology Cooperation in the EU’s 7th Framework Programme: the specific programme ‘Cooperation’ and its thematic areas. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 56
differentiated strategy on international cooperation and increased efforts for bilateral agreements on STI collaboration. Investments in international cooperation have to be made by strategic involvement of partners from outside the EU in areas of key importance to European goals. This includes leadership in innovations, global societal challenges as well as science diplomacy. 5.5.
Contribution of FP7 to ERA and impacts on national innovation systems
FP7 was a key instrument of the Union’s efforts to boost the ‘European Research Area (ERA)’ vis‐à‐vis global competition: ERA aims to foster effective national research systems, to improve transnational co‐operation and competition, to achieve an open labour market for researchers and gender equality in research, and to facilitate the circulation of scientific knowledge. In 2010/11 the European Union indeed launched the ‘Innovation Union’, as an implementation of the Europe 2020 Strategy, complementing the ERA39. Did FP7 help to meet these aims? From a historical perspective, one can state that the subsequent European FPs (including FP7) have helped to transform and modernise the EU Member States’ research and innovation systems, in particular through their structuring effects, such as improved competitiveness of research, industrial and technological development, and policy‐making and coordination40. On the other hand a simplistic approach to innovation should be eluded. Innovation is dominantly a process of business engagement with markets, and research bodies should contribute to the intellectual, social and cultural resources of a region in ways that encourage inward investment of knowledge intensive business. Enquiry into the socio‐economic benefits of the ERA suggests that related FP7 initiatives had many positive consequences. Cross‐border cooperation, for instance, helped to reach critical mass, to foster research on societal challenges, to facilitate networked specialisation of research teams, knowledge sharing and transfer, to increase the visibility of research results, to reduce unnecessary duplication of efforts, to create a reliable environment fostering research by the private sector, and to promote economies of scope and administrative efficiency. Still, while the European Commission has launched numerous dedicated, often pioneering ERA measures (within or next to FP7, such as JPIs, ERA‐NETs, ERA‐NETs Plus and Article 185 networks) the implementation of ERA targets in the EU Member States appears to be slow and limited so far41. There are a number of examples for the structuring effects of FP7. FP7 played a proactive role in developing an excellence‐orientation in research (funding) institutions across all countries and regions in the ERA: The ERC with its relative independence and its excellence‐oriented criteria and procedures is increasingly serving as a role model for research funding organisations and universities in the EU Member States42; ERC‐awardees are free to bring their project to any academic location in Europe. The PEOPLE programme, too, served as a model for many EU Member State organisations, through its ex ante mode of evaluating awardees, its thematic openness, and the access to a broad range of hosts. Also, a network analysis found “that several regions were added to the network relative to FP6. Also, more co‐participations are observed in FP7 relative to FP6. Also, (…) regions from smaller countries with lower levels of participation experienced the strongest levels of integration in the network in FP7 relative to FP6.”43 A recent study of collaboration in ICT research networks found that the “cohesion across European member states and regions has been constantly promoted by European Union science policy, particularly through the Framework Programme”44. So FP7 has 39 40 41 42 43 44 European Commission (2010), Communication from the Commission on Europe 2020 Flagship Innitiative Innovation Union. SEC(2010) 1161. Arnold, E., Mahieu, B., Stroyan, J., Campbell, D., Carlberg, M., Giaracca, F., Horvath, A., Jávorka, Z., Knee, P., Meijer, I., Sidiqi, S., Wagner, C. (2011), Understanding the Long Term Impact of the Framework Programme. Dinges, M., Bouttier, R., Schiffbänker, H., Holzinger,F., Van der Giessen, A., Lehenkari, J. ,Deschryvere, M., Kuittinen, H., Rammer, Ch.(2014), Analysis of the state of play of the European Research Area in Member States and Associated Countries: focus on priority areas.
European Research Council (2012), Understanding and Assessing the Impact and Outcomes of the ERC and its Funding Schemes (EURECIA). Campbell, D., Ventimiglia, A., Archambault ,E. (2013), Scientific Output and Collaboration of Companies Publishing the Most in the ERA. Cecere, G. & Corrocher, N. (2015). The Intensity of Interregional Cooperation in Information and Communication Technology Projects: An Empirical Analysis of the Framework Programme, Regional Studies, 49, 2, pp. 204–218. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 57
fostered a wider participation of relevant actors and stakeholders at the regional level, hence also creating a stronger base for cooperation at the national level. Another, recent example of structuring effects of FP7 was the EU Commission’s attempt to introduce the vision of ‘Responsible Research and Innovation (RRI)’ as a guiding orientation in Europe45. RRI has been defined as “a transparent, interactive process in which societal actors and innovators become mutually responsive to each other with a view on the ethical acceptability, sustainability and societal desirability of the innovation process and its marketable products”. The ‘Science in Society’ programme under FP7 sponsored a number of projects aiming to define and mobilise RRI46. Meanwhile a growing number of RRI‐related initiatives have been launched in several (but certainly not all) EU Member States’ research and innovation systems – with different definitions, scope, and reach47. The FP6 Evaluation (2009) had concluded that the “role of the FP in the ‘policy mix’ at EU and EU Member State level is not yet well defined” and the FPs should use their European Added Value in a more strategic way (58). The Interim Evaluation of FP7 (2010) found that there is a very high likelihood that FP7 can contribute positively to ERA, and that for this purpose a sharper division of labour between what is done at EU level and what is undertaken in national programmes should be achieved and that a “well‐articulated innovation strategy” should be developed. Still, during the lifetime of FP7 key weaknesses of many EU Member States’ national innovation systems remained; they can be grouped around three blocks: (1) a lack of quality of the science base; (2) feeble contribution of the science base to the economy and society; and (3) inadequate framework conditions for business R&D and innovation48. FP7 and earlier FPs have clearly had a positive impact on the structure, working and performance of EU Member States’ research and innovation systems. Still, while diversity across system does and should persist, strengths and weaknesses remain to be very unevenly distributed across Europe. More cohesion and strengthening of weak areas is urgently needed – this is the crucial task of the Structural Funds, the Cohesion Fund and related measures! The FPs (resp. HORIZON 2020) should rather focus on a strategic mobilisation of knowledge, technology, and innovation themes where Europe is strong or should become strong – as an attractive hub an leader in global innovation networks, across national systems. A "Common Science, Technology and Innovation Policy" should be explored. As a first step, national and EU programmes should better align their research priorities using appropriate tools and incentives (such as pooling of funding in order to improve leverage effects, considering the innovation supply chain, shared databases and support of mobility). Common standards for research should be established. It is recommended that by establishing an EU‐wide quality stamp for outstanding scientific and enterprise driven proposals, successful proposers would be allowed to apply for funding at the national level in a streamlined manner. 5.6.
FP7 impacts on policy coherence
The landscape of European research and innovation actors and policies is highly complex; national and regional initiatives are complemented by manifold European‐level measures, next to the FPs also the research and innovation elements of the Structural Funds, the European Institute of Technology, and many more. In 45 46 Expert Group on the State of Art in Europe on responsible research and Innovation had recommended that RRI “has to be a key part of the research and innovation process and should be established as a collective, inclusive and system‐wide approach.” in Van der Hoven, J., Jacob, K., Nielsen, L., Roure, F., Rudze, L., Jack, S., Blind, K., Guske, A., Martinez Riera, C., (2013), Options for strengthening responsible research and innovation. Report of the Expert Group on the State of Art in Europe on responsible research and Innovation. Technopolis Group; Fraunhofer (2012), Interim evaluation & assessment of future options for Science in Society Actions. Interim evaluation. 47 For an overview see MoRRI, a web application for monitoring and visualizing data and information on Responsible Research and Innovation (RRI) in 16 European countries. MoRRI is part of the FP7 Res‐AGorA project; MoRRI builds upon and extends the work that was carried out in the ‘Monitoring Policies and Activities on Science in Society in Europe’ (MASIS, 2010‐11).
48 European Commission (2014), Research and Innovation performance. Innovation Union progress at country level in the EU 2014. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 58
this context, like earlier FPs the 7th Framework Programme certainly had an orientating function, thematically and in terms of programme designs, for many EU Member States. More particularly, various FP7 initiatives helped to improve ERA‐relevant policy intelligence and advice functions, e.g. measures by the Joint Research Centre (JRC) such as the explicit ERA monitoring procedure commissioned by the Research DG (ERAWATCH). A number of policy‐analytical and foresight research lines funded in the FP7 SSH area helped to better understand ongoing and futures challenges and options for European research and innovation and related policy. Still, this policy advice function of FP7 has been performed in an only weakly coordinated and strategic way. FP7 has seen quite a number of ERA‐NET (+) initiatives. In an ERA‐Net research funding, organisations and programme managers from EU Member States and Associated Countries combine financial and human resources in order to implement joint activities aiming to deepen ERA and to boost the efficiency of European research and research funding. Joint Programming (JP) is a process designed to ensure the optimisation of existing and future research policy efforts at the level of EU Member States. It aims to reinforce cross‐border cooperation and the coordination and alignment of national publicly funded research programmes in selected fields. An Expert Group found “that the Joint Programming process has got off to a good start, although the process can only reach its full potential if commitment and financial support from national level administrations continues”.49 The ERC turned out to be a relevant facilitator of institutional change and policy coherence in the landscape of research funding institutions50: The ERC provided a general and legitimate model of funding body for many EU Member States, concerning the funding of fundamental or bottom‐up research or individual researchers, the Europeanisation of activities, the importance of internationalisation in peer review, and overall, reinforced cross‐European competition. FP7 was a key element of the Union’s efforts to achieve policy coherence, horizontally and vertically, in the European research and innovation system. Still, the numerous initiatives taken appear often only loosely coordinated.. The European Treaty of 2012 (Articles 179‐190, in particular 181) legitimises the European Commission to initiate and coordinate strategic initiatives. Future FPs and especially HORIZON 2020 should address this mission in a more pro‐active way. 49 Acheson, H., Annerberg, R., Dammert, R., Klusacek, R., Kraus, W., Lock, J. (2012), Review of the Joint Programming Process. Final Report of the Expert Group. 50 European Research Council (2012), Understanding and Assessing the Impact and Outcomes of the ERC and its Funding Schemes (EURECIA). Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 6.
FP7 impacts on value creation and economic growth
6.1.
Estimation of macro-economic effects, growth and jobs
59
Research and innovation processes are characterized by complex interactions, feed‐back loops and variable time spans between research and development (R&D) activities and market penetration. R&D activities in basic research, applied research or design and development mark the beginning of innovation processes. They are succeeded by the integration of novel products and processes, the introduction of innovations into the market and subsequently market diffusion. The assessment of economic effects of R&D calls for sophisticated methods to connect increases in R&D to increases in productivity, domestic and external demand and, finally, growth and employment. While the outputs of innovation processes, such as e.g. publications, patents or prototypes, appear immediately, impacts in terms of increased sectoral competitiveness, GDP increases, increased employment or improved living conditions, take longer to become apparent. Impact assessment is further aggravated by the multitude of influences and diverging paths of influence between R&D and the impact on economy and society. Efforts to assess the economic impact of FP7 include seven Monitoring Reports51, an ex‐ante impact assessment (Delanghe and Muldur 2005), as well as three assessments of the impact of one year of FP7 funding respectively. While all these efforts provide valuable insights into short and long‐term effects of FP7, their explanatory power concerning economic effects is limited: the seven Monitoring Reports do not provide any data on economic impacts and the estimations of the ex‐ante impact assessment were thrown off track by the financial crisis of 2008‐2012. The best estimation of the economic impact of FP7, for the purpose of this evaluation, has so far been provided in the form of cumulative job and GDP increases induced by the FP7 2010, 2012 and 2013 calls for proposals. In these three assessments, impact is simulated in four phases over 15 years into the future in order to account for short‐, medium‐ and long‐term effects. Modelling involves three main mechanisms: (1) the leverage effect that enables determination of total R&D expenditure; (2) the spill over of knowledge describes knowledge transfers to other sectors and other countries; and (3) the economic performance of knowledge. Main limitations of these assessments include the time frame of only one FP7 funding year per study, the assumption that each year’s budget is allocated in form of a “one‐off shock”, as well as model limitations. The assessments further highlight the potential impact of the Risk‐Sharing Finance Facility (RSFF) as a leverage factor regarding GDP and employment increases. DG R&I is currently contracting an additional study with the aim to assess and quantify the economic impact of FP7 and the expected economic impacts of Horizon 2020. First results will become available in late 2016 at the earliest and could therefore not be processed in this evaluation. The best estimates of increases in GDP and employment that are directly connected to FP7 spending are provided by Fougeyrollas et al. (2012)52 and Zagamé et al. (2012)53 in their assessments of the programme years 2012 and 2013. We used their key indicators for a rough estimate of the economic impacts of the whole 7th Framework Programme:54 
Out of the total voted budget of 55 billion euro, the 50 billion euro of EC contribution to FP7 were taken as starting point for the estimation; taking the budgets allocated to the four specific programmes, the EC contribution to ITER, to Nuclear Fusion and Fission Research and the Joint Research Center's direct actions into account.55 51 All seven FP7 Monitoring Reports can be accessed at https://ec.europa.eu/research/evaluations/index_en.cfm?pg=fp7‐
monitoring
52 Fougeyrollas, A., Le Mouël, P., Zagamé, P., (2012), Consequences of the 2012 FP7 call for proposals for the economy and employment in the European Union. Report by ERASME 53 Zagamé, P., Fougeyrollas, A., Le Mouël, P. (2012), Consequences of the 2013 FP7 call for proposals for the economy and employment in the European Union.
54 In doing so, the limitations of such estimation are duly considered. Modelling of the economic impact of wide‐ranging government programmes over long periods is complex. The GDP impact is particularly sensitive to the leverage effect, which itself depends on the degree to which the private sector contributes to the specific programmes. Care should therefore be taken in interpreting the conclusions relating to economic growth effects in this evaluation.
55 We did not take into account the administrative expenditures of the European Commission (EC) associated with the implementation of FP7 (as they do not provide a substantial leverage or multiplier effect) and the contribution to the Risk Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 60

Both studies mentioned above estimated the leverage effect at 0,74, indicating that for each euro the EC contributed to FP7 funded research, the other organizations involved (such as universities, industries, SME, research organisations) contributed in average 0,74 euro. Based on the 50 billion euro mentioned above, the own contributions of other organizations to the funded projects can be estimated at 37 billion euro. In addition, the total staff costs for developing and submitting more than 139.000 proposals at an estimate of 3 billion euro were taken into account. In total, the contribution of grantees can be estimated at 40 billion euro. 
The total investment into RTD caused by FP7 can therefore be estimated at approximately 90 billion euro. 
For estimating the time scale of these investments the duration of FP7 (2007‐2013) plus the average project duration (3 years) was taken into account. Therefore, a total running time of ten years and an annual RTD expenditure of 9 billion euro covered by EC funding and own contributions of other organizations were calculated. 
Both studies mentioned above estimated a cumulative GDP multiplier of 6.5 for a period of 25 years. This consists of the total investment into RTD (90 billion euro) and the indirect economic effects (caused by new technologies, products and markets). Applying this estimation the indirect economic effects can be estimated at approximately 500 billion euro giving an additional annual GDP of approximately 20 billion euro for the next 25 years. 
Considering both ‐ the leverage effect and the multiplier effect ‐ each euro contributed by the EC to FP7 caused approximately 11 euro of direct and indirect economic effects. When considering the comparably high leverage effects of the RSFF, these figures are based on a rather conservative estimate and the real effects might be even higher. 
When translating these economic impacts into job effects, it was necessary to estimate the average annual staff costs of researchers (for the direct effects) and of employees in the industries effected by RTD (for the indirect effects). Based on estimated annual staff costs for researchers of 70.000 euro, FP7 directly created 130.000 jobs in RTD over a period of ten years (i.e. 1,3 million persons‐years). 
When estimating the indirect job effects it has to be considered that new technologies in some cases create new jobs while in other cases they might lead to job losses as well. This has already been taken into account in the two studies mentioned above. By applying their results to FP7 approximately 160.000 additional jobs are indirectly caused by FP7 over a period of 25 years (i.e. 4 million person‐years). The economic effects of FP7 have been quite substantial, given the fact that FP7 only accounts for 3% of RTD expenditure in Europe. Each euro contributed by the EC to FP7 caused approximately 11 euro of direct and indirect economic effects. In total FP7 stimulated an additional annual GDP of 20 billion euro over a 25 year period. Regarding job effects the results seem modest, especially in the context of the current unemployment rates in Europe. However, it has to be considered that FP7 was an instrument of research and innovation policy addressing excellence, competitiveness and societal challenges and not an instrument of job policy. 6.2.
Market impacts through successful implementation of outcomes
It is very clear that the timeframe for the achievement of market outcomes from any research funding programme is anything up to 15 years and hence the difficulty in reporting conclusively at this stage on the specific outcomes and impact of FP7. However, there are a wide range of case studies and examples to draw on and the conclusions of this section are based on a wide sampling. For example, a publication by DG Research and Innovation in 201256 highlights 15 case studies of projects funded by both FP6 (3 cases) and FP7 (12 cases) which clearly demonstrate success in market implementation as a result of substantial co‐funding by the EU. In their study (Evaluating the long term impact of Framework Programme research – Final Report), EPEC57 have also provided some comprehensive evidence of market implementation of outcomes from Framework Sharing Finance Facility (RSFF) (which provides a substantially higher leverage effect and would lead to biased results for the other programmes and activities).
56 European Commission (2013), Investing in European success: HORIZON 2020, Research and innovation to boost growth and jobs in Europe. 57 Arnold, E., Mahieu, B., Stroyan, J., Campbell, D., Carlberg, M., Giaracca, F., Horvath, A., Jávorka, Z., Knee, P., Meijer, I., Sidiqi, S., Wagner, C. (2011), Understanding the Long Term Impact of the Framework Programme. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 61
Programme support over time. They conclude that the FPs have been influential because they provide resources additional to those of the EU Member States and have been able to ‘leverage’ the use of those resources by encouraging coordination. A series of examples are cited 
In QIPC (Quantum Information Foundations and Technologies), the emergence of a new field of science and technology, helped it establish scientific and technological agendas, and to grow in Europe to such an extent that the EU is acknowledged to be fully competitive with the other world R&D leaders. 
In Stratospheric Ozone research, the Framework Programme has made a major contribution by growing and helping to coordinate the European research community. It has helped the European research community move from lagging far behind the USA to working at the global frontier. Research results have shaped the evolving Montreal Protocol requirements and have been so influential at the policy level that Europe has achieved the Protocol’s 2020 targets ten years ahead of schedule. 
In Solar Photovoltaics (PV) the Framework Programme has expanded the European research community and enabled it to work at the technological frontier. 
In the Automotive Industry, the Framework Programme’s role has been to sustain longer term research and development in areas such as fuel efficiency, emissions and safety. Exploiting the industry’s desire to self‐organise to define R&D directions and road maps has been a powerful way to coordinate the longer‐
term R&D effort and has supported a series of product and process innovations that help maintain Europe’s position among the global leaders in this industry. 
A clear example of commercial exploitation of research funded through FP7 is in the Security58 area, where 18% of the 61 completed projects that have been assessed were found to have ‘good innovation potential’, while 8% were completed with ‘substantial R&D breakthrough character’ and 5% with ‘outstanding use / exploitation of results’ (categories that are not mutually exclusive). 
Large‐scale bio medical research projects59, where most of the funding went to universities and public bodies based in the EU‐15, also produced a substantial number of commercially exploitable outputs. Concerning FP7 call topics, the activities were most frequently found in projects related to high‐throughput projects involving innovative therapeutic approaches and interventions (65%), anti‐microbial drug resistance (57%), large‐scale data gathering (56%) and detection, diagnosis and monitoring (56%). In conclusion, it is still too early to make a final assessment of the market impact of FP7 projects and the very nature of the Programme and its evaluation methodology may never enable a definitive analysis to be made. Notwithstanding this, FP7 is having a significant positive impact: FP‐funded research produces large numbers of patents, innovations and micro‐economic benefits. The FP has enabled certain company participants to report an increase in their turnover and profitability, raise their productivity, increase their market share, obtain access to new markets, reorient their research, technology and innovation strategies as well as commercial strategies, invest significant funds themselves and coordinate that in the larger context, and improve their competitiveness. 6.3.
Effects on European Competitiveness
A broader objective of FP7 was to improve the competitiveness of Europe as a whole and of its economy. In the latest EU Report on the State of the Innovation Union 201360, it is clearly stated that Europe remains the main knowledge production centre in the world, accounting for almost a third of the world’s science and technology production. The EU has managed to maintain its competitive knowledge position to a greater degree than the United States and Japan and, after a set‐back during the crisis, is making progress towards its R&D intensity target of 3% by 2020. The EU also remains a very attractive location for R&D investment. In 2011, the EU was the main destination of FDI in the world, receiving around 30% of FDI inflows worldwide, more than the United States or Japan, much of it R&D intensive. There are, however, weaknesses in Europe’s competitive position, which may threaten its medium‐term economic growth. Investment in knowledge is increasing faster in the 58 Technopolis Group (2015), Final Evaluation of Security Research under the Seventh Framework Programme for Research, Technological Development and Demonstration. 59 PPMI (2015), Ex‐post evaluation of the Health Theme in FP7: Preliminary Report. Preliminary analysis of FP7 projects portfolio and their outcome. 60 European Commission (2014), Innovation Union Competitiveness Report 2014. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 62
Asian economies than in Europe. In 2014, China’s R&D investment (expressed in purchasing power standards) may have exceeded that of all EU Member States together. At the same time, science and technology development in Asia and in the United States is often more strategic than in the EU. It is more focused on transformative and pervasive technologies oriented towards emerging global markets. The EU’s technology assets are more focused on its established and traditional industries, while its scientific specialisation does not sufficiently back up technology strengths. Determined reforms are needed to overcome fragmentation and develop a common long‐term strategic focus for Europe’s knowledge profile. A key measure of commercial impact is the extent to which durable start‐ups and spin‐offs have emerged from the Programme and EU‐funded projects in the area of ICT61 have led to spin‐offs, set up to commercialise products and services. Based on the information extracted from 821 final reports, 125 spin‐off companies were created as a result of the projects. However, although the Programme was largely research focussed, interviews carried out in the context of the support study revealed that FP7 ICT mechanisms were considered insufficient to help translate research results into innovative products, processes and services. Some participants mentioned the need for smaller projects and several rounds of funding, others expressed the view that the institutional capacity of research centres is not suited to promote innovation. Overall, most respondents pointed out that FP7 ICT by design targeted research more than innovation and that it was more successful at addressing research objectives than it was at supporting demand‐driven innovation. Innovation Scorecards and Output Indicators
The development over recent years of Innovation Output Indicators and Scorecards have enabled us to compare better the performance of the EU as a whole with other major economic blocks as well as to compare across EU Member States. In the Innovation Scorecard Exec Summary 201462, human resources and openness of the European research system have seen the highest growth in innovation performance. When looking at individual dimensions, open, excellent and attractive research systems contributed most to the overall innovation performance over the last eight years, followed by growth in Human resources. Looking at individual indicators, Community trademarks contributed most to the increase of the innovation performance, followed by Non‐EU doctorate graduates and International scientific co‐publications. Relatively good performance improvement was also observed in innovation collaboration of SMEs and commercialisation of knowledge as measured by license and patent revenues from abroad. Patents and Intellctual Property Rights
An indicator of the contribution to European Competitiveness is the development of protectable IPR and the overall number of registered IPR emanating from FP7 projects is around 1.700 (February 2015). Registered IPR concerns mainly patent applications (83%), however, trademarks, utility models and registered designs are equally occurring. In total the NMP and Health themes generated more than 50% of the IPR. This may appear low, but, while many FP7 projects have registered patents in the monitoring system, recent analyses suggest that the real number of patents is higher and the real new knowledge gained equally much higher than the number of registered patents suggests. According to the NMP patent analysis the database presents a non‐
trivial underestimate of the real patent output of projects. While 64% of the projects (185 out of 290) report no patenting activities, the survey results indicated several other forms of IP protection avenues such as trademarks (22%) and design registrations (22%). A large number of projects opted for secrecy (60%), defensive publishing (20%) or open source strategies (27% (ibid.). Open Innovation and Knowledge Transfer in the EU
We can also draw the conclusion that some of the most impactful elements of the FP7 are those that boost competitiveness through Open Innovation and Knowledge Transfer in the EU. An independent expert group reporting on this area 63 concluded that the EC, EU Member States, universities and public research 61 European Commission (2015), Ex‐post evaluation of ICT research in the Seventh Framework Programme. 62 everis (2014), Patent costs and impact on innovation: International comparison and analysis of the impact on the exploitation of R&D results by SMEs, Universities and Public Research Organisations. 63 Debackere, K., Andersen, B., Dvorak, I., Enkel, E., Krüger, P., Malmqvist, H., Plečkaitis, A., Rehn, A., Secall, S., Stevens,W., Vermeulen, E., Wellen, D. (2014), Boosting Open Innovation and Knowledge Transfer in the European Union. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 63
organizations, corporate sector, financial institutions, local communities and their citizens have no option but to advocate and to support open, networked and collaborative innovation‐led growth on which, in different ways, their own intellectual, operational and financial vitality will increasingly depend. European R&D policy had already propelled collaboration and Knowledge Transfer to the forefront before the current emphasis on Open Innovation took hold in Europe. However, there is still a long way to go before Europe can claim truly global competitiveness and commentators underline the importance of putting Open Innovation and Knowledge transfer in the spotlight, of stimulating innovative businesses and markets, and building innovation hubs and networks. There is consensus that Universities and PROs should also be incentivised to be more entrepreneurial, and that there should be smart integration of public and private capital into the ecosystem. The establishment of the European Institute for Innovation and Technology and its expanded funding under HORIZON 2020 demonstrate this commitment. Future value added will only be achieved (and be seen to be achieved) through the engagement, and leverage of the private sector. Clear conclusions are drawn from the PPP, JTI and SME oriented elements of FP7, which demonstrate leverage, impact, globally competitive discoveries and outcomes and which are being reinforced in HORIZON 2020. The EC has clearly been ready to improve the FP7 initiatives during the life of the programme, which has encouraged a reciprocal reaction from the private sector which has engaged more seriously both in terms of financial and intellectual capital. 6.4.
Supporting European Industrial Base and Competitiveness
Joint Technology Initiatives (JTIs) are public‐private partnerships at the European level in the field of industrial research. JTIs were introduced in FP7 as a means to implement the Strategic Research Agendas (SRAs) of a selected number of European Technology Platforms (ETPs) for which the scale and scope of the objectives meant that more intensive co‐ordination was needed. The five JTIs introduced in FP7 were Innovative Medicine Initiative (IMI) in pharmaceutical development, Clean Sky in the aeronautics industry, ARTEMIS in embedded systems, ENIAC in Nanoelectronics, and the Hydrogen and Fuel Cells Initiative (FCH). The JTIs were designed to go beyond traditional R&D funding and include measures to build an innovation ‘eco‐system’ in each of its technological fields, including standards promotion, identification of future skill requirements etc. In FP7, the JTIs represented a total Union contribution of EUR 3,12 billion, matched by an industry investment of EUR 4,66 billion, demonstrating a significant leverage effect. JTIs have proven to be successful in attracting a high level of industrial participation in their activities, including SMEs which represent about 28% of the participants. In addition, and even though the JTIs have only been fully operational for a limited time, the interim evaluations have acknowledged the progress made and the first signs of impact, albeit identifying some shortcomings. Evaluations note that public‐private partnerships in general and JTIs in particular, represent an innovative and effective way of implementing the Union's research and innovation policy. As probably one of the most important achievements, JTIs managed to bring together the frontrunners in terms of research and innovation in the industrial sectors concerned and motivated them to focus and align their efforts around strategic research and innovation agendas. The launch of JTIs was considered well justified on the basis of identified market failures, the long term nature of the required activities and the scale of the commitment needed to achieve the necessary breakthroughs. This effort was particularly notable with respect to the European‐wide discussions on Key Enabling Technologies and European knowledge‐based competitiveness. The development of the JTIs under FP7 was positively influenced by the Key Enabling Technologies (KET) High Level Expert group initiated/driven by DG Connect. This is an example of the effectiveness of coordinated approaches between different DGs and industry. The KETs high level expert group turned out to be highly effective both towards industry as well as the European Commission and eventually the EU Member States. An evaluation of the Innovative Medicines Initiative (IMI) Joint Undertaking64 concludes positively. IMI’s 2 billion euro budget for the period 2008‐2013 made it the largest life sciences PPP in the world and its main objectives have been to address the bottlenecks currently limiting the efficiency, effectiveness and quality of the drug development activities needed to bring innovative medicines to the market. Over the past two review periods, IMI has successfully demonstrated the feasibility of large, multi‐stakeholder PPPs for research and 64
Hunter, J., Szumowski, M., Andersen, T., Rosaria, M., Nucci, D., Wijnberg, B.(2013), Second Interim Evaluation. IMI ‐ Innovative Medicines Initiative Joint Undertaking. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 64
development in biomedicine. The new business model created by IMI is well established and has leveraged the research strengths across the European pharmaceutical industry, academia and SMEs. It has established over 40 public/private consortia which are delivering projects of high relevance to healthcare challenges. It is now perceived globally as the leading public‐private partnership (PPP) in healthcare. The study came to the conclusion that the IMI contributed to halting the decline private sector investment in European biopharmaceutical R&D and has even led to its increase over the past two years, unlike the US in the same period. Overall, the study shows that IMI has played a major role in consolidating the European pharmaceutical research base by acting as a "one stop shop" for biomedical research and development in Europe. This has contributed to reinforcing Europe's attractiveness for pharmaceutical R&D, stemming the flow of investment away from Europe to the USA and Asia. Two other JTIs, ENIAC and ARTEMIS65 have been recognised by their respective evaluations, as well as by several stakeholders consulted, as helping to keep the competitive position of Europe, because of their capacity to involve several key industry players in drafting a common research agenda. According to participants, the JTIs have further developed the trade‐offs between collaboration and competition, thereby accelerating discovery and development, as demonstrated in the set‐up of ENIAC pilot lines. The interim evaluation of the Clean Sky JTI66 confirmed that it is successfully stimulating developments towards its strategic environmental targets by focusing on radically new technological concepts. A survey of stakeholders involved in the Fuel Cells and Hydrogen JU (FCH)67 reveals strong positive impacts on investment, jobs and turnover. In total, it is estimated that: the number of patents granted in the EU to European companies for FCH showed a 16% annual increase compared to the average annual growth for all EU industries of 1,5% ‐ annual turnover increased by 10% per year, R&D expenditures by 8% and market deployment expenditures by 6% since 2007, whilst the number of jobs engaged in the projects has been increasing by about 6% per year since 2007. The study on the Fuel Cells and Hydrogen JU points out that, whereas the crisis saw a general tendency for research institutes and industry, across the globe, to withdraw from radical innovation and to focus on core business and incremental technology progress, the FCH JU has helped to counter this tendency, both by virtue of its stable funding and through the expression of a long‐term political commitment by the EU institutions that has given confidence to industry. The reports and interim evaluations also pointed towards some weaknesses in the current JTIs. This concerned in particular the need for stronger commitments from industry partners, with clearer measurement of these commitments and the associated leverage effect. There is also a need to provide more clarity on how JTIs are established, to equip them with clearer objectives and to ensure greater openness towards new participants. The report of the JTI Sherpa Group68 in addition made a number of recommendations to simplify and streamline the running of JTIs, including through a specific financial framework appropriate to their needs. Stakeholders have also raised concerns about the different rules and procedures that apply for each JTI and which may vary between JTIs and with those applicable under FP7. All of these concerns have been addressed in the proposed JTIs under Horizon 2020. However, for some JTIs further simplification measures are needed. JTIs have been instrumental and effective in bringing together the critical mass of relevant companies, addressing the most important industry needs and delivering on the high ambitions both in terms of content as well in leveraging additional private funding in a coordinated way. Thus, the instrument of JTIs should be further strengthened and procedures in the complex contractual framework should be simplified, with a view to providing necessary flexibility to fit the purpose of setting up and implementing JTIs as effective PPPs in European research. 65 Wild, A. (2014) ECSEL Joint Undertaking annual activity report, 2014 (27 June 2014–31 December 2014) 66 Bertolini, E., Eckardt, D., Hecker, P., Herrera, I., Horvat, M., Huguet, M., (2010), CLEAN SKY. 1st Interim Evaluation. 67 European Commission (2014), Annual Report on Programme Evaluation Activities 2013. 68 JTI Sherpas' Group (2010), Designing together the ‘ideal house’ for public‐private partnerships in European research. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 6.5.
65
Impact on the competitiveness of European SMEs
The competitiveness of European SMEs is well accepted as a critical success factor for the European economy. It is also well established that smaller enterprises will, by their nature, be more local and should largely be supported by national programmes for Research and Innovation. Successive European Programmes (FPs, MAP and CIP) have, however, also addressed the needs of innovative SMEs, and improved outcomes and value added. An evaluation of the participation of SMEs in the FP7‐COOPERATION and RSME under FP7‐CAPACITIES highlights two critical success factors of the FP7 focus on SMEs:69 
Additionality was very largely demonstrated. Only a very small percentage of SMEs state that they would have undertaken the project the same way without EC funding (full deadweight effect is only 2% in Cooperation and 4% in the RSME schemes). A significant minority of SMEs report that they would have proceeded with the project in some form, probably with a reduced scope at a later date or would have started searching for other public support. As much as 53% of SMEs in FP7‐COOPERATION and 62% of SMEs in the RSME schemes state they would not have been able to undertake the project at all without EC funding. 
Impacts are more difficult to measure but, the results of the various econometric analyses all show that SMEs participating scored much better than the control group with regard to employment growth and operating revenue for FP7 as well as for FP6. Also participating SMEs themselves report a range of positive tangible and intangible impacts, e.g. more cooperation, new knowledge gained, innovation competences improved, and these having a positive effect on their competitiveness. In FP7‐COOPERATION 54% of SMEs report an impact on turnover, for employment this was 50% and for exports 38%. Those SMEs reported an average increase of turnover of 22%, employment +25% and export +28%. In Research for SMEs. The Eurostars Joint Programme has also succeeded in accelerating the growth and innovative outputs of R&D‐
performing SMEs.70 The results of the final evaluation showed that Eurostars was relevant for the growth of R&D‐performing SMEs in Europe. The employment growth rate of R&D‐performing SMEs funded by Eurostars was nearly twice as high as that of applicant SMEs which were not funded. This can be causally attributed to Eurostars funding. The programme has accelerated the development and roll‐out of new and improved products, processes and services. The econometric evaluation established a positive and significant impact on the patent portfolio of funded firms relative to unfunded applicants. The programme has stimulated new cross‐
border collaborations that the members of the funded consortia intend to continue beyond the Eurostars funding period. There are clearly a number of challenges that remained in leveraging the potential of European SMEs. A clearer segmentation needs to ensure that the right programmes are addressed to the high potential participants. It remains complex and time‐consuming for a small business to participate despite steps to reduce bureaucracy, and the time to grant is far too slow for most commercial entities. In order to unlock the full potential of SMEs in European Innovation value chains it is suggested to develop suitable targets and indicators. Particularly for start‐ups and spin‐offs this would be extremely helpful and effective as they typically have trouble in technology/product development and getting to the market. SMEs do play an important and needed role in the “innovation pipeline”/”innovation value chain”. SMEs are particularly needed for their capabilities of coming up with new ideas, and their speed and flexibility in developing new concepts. However, they do not have the capacity and resources to go into product development, nor to get innovations quickly into the market. Thus, much closer interaction with large companies is needed. It is doubtful that a certain share of participation (15/20%) in the total Programme budget is therefore an indication of reasonable SME involvement in the innovation pipeline. It is unclear whether this arbitrary measure of share in participation and budget really reflects the real added value of SMEs. We suggest therefore to encourage the fostering of SME participation in national programmes as they are typically less complex, and to develop a range of indicators at European level in order to demonstrate the unlocking of the full potential of SMEs role in the innovation value. One effective measure could be the requirement for large companies to involve a specific number of SMEs in their EU projects. 69 70 Van Elk, K., Snijders, J., Prince, Y., Gibcus, P., Doove, S., Simmonds, P.,Warta, K., Good, B., Ruhland, S., Sheikh, S. (2014), Performance of SMEs within FP7. An Interim Evaluation of FP7 components. Makarow, M., Licht, G., Caetano, I., Czarnitzki, D., Elçi, S. (2014), Final Evaluation of the Eurostars Joint Programme. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 7.
66
FP7 impacts on citizens and society
Citizens and society are the major beneficiaries of all impacts mentioned in the earlier chapters of this report: scientific excellence is a prerequisite for providing novel insights and increasing the knowledge base of our society; value creation and economic growth lead to welfare, income and jobs. In addition to these indirect effects on society, FP7 explicitly expressed expectations for STI to contribute to solving some of the pressing challenges our societies face today and allocated substantial funding to themes of societal relevance such as health, environment, climate change and security. The ambition to involve researchers, enterprises, industries, and NGOs as well as citizens was addressed in the announcement of FP7 “Tomorrow’s answers start today” in 2007. During the Swedish presidency of the EU in 2009 the idea of steering both national and European research programmes to address societal challenges was further promoted with the Lund Declaration (July 2009) titled "Europe must focus on the grand challenges of our time". As a consequence, towards the end FP7 became more societal challenge driven and laid grounds for a substantial part of Horizon 2020 to be directed towards addressing grand societal challenges. The occurrence of these positive impacts depends on complex mechanisms along the innovation supply chain and can therefore not be quantified at this point in time. Therefore, certain proxies have been used to provide a qualitative assessment of FP7 impacts on citizens and society. We thus analysed the involvement of civil society organisations (CSOs) in FP7, budget share of society related research, communication of research results to the general public, FP7 impacts on mobility of researchers, gender equality, responsible research and innovation, as well as the wider societal impacts of FP7 in the backdrop of the newly adopted Sustainable Development Goals. 7.1.
The roles of citizens and civil society in FP7
Involving citizens into research and innovation boosts innovation potential, orientates innovation activities towards the most pressing societal problems, increases acceptance of new technologies and promotes societal trust in science. While in other world regions scientific progress and innovation are generally perceived as positive, many European citizens tend to share a more critical view. A major challenge of European research policy is therefore to increase citizen trust in science and innovation by bringing citizens and science closer together. Therefore, different ways of citizen engagement were tested, and citizens as well as civil society organisations were involved as recipients of communication about research. They served as a source of information and data, civil society organizations were involved as partners in research projects, and society in the broad sense was in the focus of research project contents. While these forms of involvement were broadly used in FP7‐COOPERATION and to a certain extent in FP7‐CAPACITIES projects, in contrast citizens and civil society organizations were barely involved in relevant FP7 programme decision‐making bodies, such as evaluation boards or expert groups. Informing citizens about research activities and outcomes
Communication is key in order to improve the process of translating scientific achievements into impacts and benefits for society. Without communication which is understandable for targeted audiences, societal trust for science in society will not be increased, and research risks to be seen as a cost rather than an investment. FP7 specifically set a task to promote communication of research in order to increase the use of its results by industry, policy makers and society 71 . In most of FP7 funded research projects, a broad variety of dissemination tools were implemented to inform citizens about research activities and outcomes. Most often used tools for communication, as reported by FP7 funded projects, were presentations, conferences, press releases, project websites, publications and workshops. Unfortunately, the information on the usage of these tools as currently reported in the project reporting system of the European Commission (SESAM) does not allow for a comprehensive analysis of tools used and results achieved of disseminating research results to the 71 European Parliament (2006) Decision No 1982/2006/EC of the European Parliament and of the Council of 18 December 2006 concerning the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007‐2013). Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 67
wider public. Furthermore, quantitative data was often missing from project reports as well.72 Evidence suggests that current dissemination and outreach activities, despite the large amount of efforts in FP7 funded projects, lacked in targeting and tailoring of these activities for different audiences with different purposes of communicating science outputs.73 The main European Commission information service for the public at large communicating EU funded research project results, web‐portal CORDIS could also benefit from improvements. Recent data shows that site visits of CORDIS been decreasing (total visits per annum of the CORDIS website were 16,4 million in 2008 and decreased to 5,5 million in 201374), furthermore according to the National Contact Points survey the CORDIS platform is still overly complex.75 During the course of FP7, grounds were laid for free circulation of scientific knowledge with the move towards an Open Access policy for publications and data resulting from FP funded research and the OpenAIRE platform for depositing peer reviewed scientific publications and datasets resulting from FP funded research was also implemented.76 In addition, every year a specific outreach event ‐ the “Researchers’ Night” was funded under FP7‐PEOPLE, offering the general public a night of entertainment and getting to know science. In total 332 “Researchers’ Night” projects were funded with a total EU contribution of approximately 25 million euro. By 2013 in total over 2,5 million people attended Researchers’ Night events that took place in over 350 cities around Europe. Citizens as a source of information and data
Citizens are important sources of information, especially in the social sciences, economics and humanities, but also in technology development and innovation projects. While no data is available about the involvement of citizens in FP7 projects in general, five FP7 funded projects77 developed and implemented a highly innovative approach of “Citizen Observatories”, involving citizens in information capturing, evaluation and communication, mostly in the area of environmental research. This approach was taken up in 2015 with a specific topic in HORIZON 2020 called “Growing a Low Carbon, Resource Efficient Economy with a Sustainable Supply of Raw Materials” (SC5‐17‐2015) aiming at strengthening environmental monitoring capabilities by utilising Citizen Observatories. Involving civil society organisations (CSO) as partners in research projects
Among the 29.000 organizations participating in FP7 only approximately 5% can be considered to be CSOs78. The share of the FP7 budget they received was even smaller (750 million euro, approximately 1,7% of the total FP7 budget). In major parts of FP7 hardly any CSOs have been a relevant target group and therefore were barely involved at all (e.g. JTIs, FP7‐IDEAS and the institutional funding schemes of FP7‐PEOPLE). Comparably higher involvement of CSOs could be found in FP7‐COOPERATION (they received approximately 400 million euro representing 1,5% of total budget), in the SME sub‐programme of FP7‐CAPACITIES (120 million euro, 9,3% budget share) and in the Science in Society programme of FP7‐CAPACITIES (52 million euro, 18% of budget). In two themes of FP7‐COOPERATION (Environment and SSH) a special funding scheme was tested, that aimed at involving and empowering CSOs. With a total budget of approximately 15 million euro and only 11 projects this funding scheme was marginal. Furthermore, CSOs encounter a number of difficulties in participating in FP779. Most frequently reported difficulties included: CSOs lack funding, time and human resources needed; perceptions of project objectives differ between researchers and CSOs; CSO regarded as of lower status in 72 Quantick, P., Neubert, H.J, Carrerras, J., Eberle, U., Holdsworth, M., Ohlsson, T., Vauchez, C., de Winter, K., (2011), Impact Assessment “Food”. 73 E.g. European Commission (2011), Impacts of EU Framework Programmes (2000‐2010) and Prospects for Research and Innovation in Food, Agriculture, Fisheries and Biotechnologies; Helming, K., Connolly, N., Amanatidou, E., Rem, P., Wilenius, M. (2014), Ex‐Post Evaluation of FP7 Cooperation Programme Theme: “Environment (including Climate Change)”.
74 European Commission (2014), Seventh FP7 Monitoring Report. Monitoring Report 2013. 75 European Commission (2013), Sixth FP7 Monitoring Report. Monitoring Report 2012.
76 European Commission (2012), Communication from the Commission on Towards better access to scientific information: Boosting the benefits of public investments in research. COM(2012) 401 final.
77 For details see www.citizen‐obs.eu 78 Current data makes it difficult to identify CSOs in FP7, as they are not marked as a separate type of organisation but rather administrated in a category “other organisations”. A draft analysis showed that a substantial number of these organisations do not fulfil the key characteristics of CSOs (such as mission orientation or providing services to a specific societal group), but are rather umbrella organisations of the business sector or research networks. 79 CONSIDER Project (2015), Civil Society Organisations in Designing Research Governance. Final Publishable Report.
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 68
consortia; insufficient clarity of tasks; views regarding methodology diverge not only between CSOs and academic researchers but also among academic researchers themselves (due to interdisciplinarity); commercial and scientific interests clash; academic researchers are sometimes concerned that CSO participation may weaken scientific legitimacy of research projects and jeopardize the potential to enhance their academic reputation. These difficulties typically occurred in certain project set ups, for example when only one CSO was involved in a project, the structure of the project excluded them from internal decision‐making and limited their capacity to affect outcomes, its involvement was not regarded as strategic and its value added was not recognized or CSO contributions were planned only for the end of the project. Focusing research on society related issues
Two sub‐programmes addressed issues of high importance for citizens and society, however budgets and the number of funded projects of both were comparatively small: Socio‐Economic Sciences and Humanities (SSH): SSH, a sub‐programme of FP7‐COOPERATION, accounted for the smallest budget share and smallest total number of projects across all themes of FP7‐COOPERATION with 253 funded projects and 580 million euro of funding (1,3% of the total FP7 budget).80 Within SSH the average EC contribution per project was 2,3 million euro and 210.000 euro per participating organisation – it is the lowest relative budget across all themes in FP7‐COOPERATION.81 Furthermore, the highest share of high quality proposals that could not and were not be funded is also found in SSH when compared to other themes of FP7‐
COOPERATION.82 Compared to other themes there was a comparably high share of small and medium sized research projects in SSH (25% in all of FP7, 39% in FP7‐COOPERATION, 60% in SSH). The degree of freedom for researchers to choose the best approach to an individual research project was also low83. The majority of organisations undertaking SSH research were Universities – more than half of particpating organisations in SSH were universities (43% in whole FP7, 35% in FP7‐COOPERATION, 63% in SSH). However, the involvement of the business sector in SSH was extremely low (25% in FP7, 8% in FP7‐COOPERATION, 4% in SSH). Economics and other social sciences are the most reported main or associated discipline, while Humanities disciplines represent only a small share. Organizations from France and Germany were under‐represented in SSH, while the involvement of partners from the EU‐13 and from countries beyond Europe was slightly higher than in the other themes of FP7‐COOPERATION. The research areas covered a broad variety of issues such as the knowledge economy, social cohesion, demographic changes, social trends and lifestyles, Europe’s changing role in the world, conflicts, peace and human rights, roles of citizens in Europe, socio‐economic and scientific indicators, as well as societal foresight activities. On the other hand, embedding SSH in other themes and areas of research has been modest. 80 Low share of Socio‐Economic Sciences and Humanities in FP7 is partly compensated by the SSH research share in FP7‐IDEAS, where 15% of the ERC grants were awarded to SSH disciplines according to the ERC Annual Report 2014. However, according to the report of the supporting expert for the evaluation of FP7‐IDEAS (Bonaccorsi, 2015), the 15% share for SSH in the disciplinary distribution of FP7‐IDEAS is reasonable, as a result an increase of SSH share in FP7‐COOPERATION would be justified.
81 This can be party explained by the fact that no major investments in infrastructure are required for carrying out SSH research. 82 2.746 proposals were submitted for funding in the SSH programme in total, out of which 51% were above threshold criteria for funding (46% on average in FP7‐COOPERATION). Out of these 1.412 proposals only 18% were funded (39% on average in FP7‐COOPERATION). On average a winning proposal had to score 90% of the maximum points in SSH (in other themes between 84% and 88% was sufficient to secure funding). As a result of budget constraints 11 proposals with more than 95% of points and 74 proposals with 90‐95% of point could not be funded. 83 Degree of freedom can be derived from the share of flexible funding schemes: in most themes of FP7‐COOPERATION two funding schemes were differentiated – ‘Research Projects’ and ‘Coordination and Support Action (CSA)’. In a few themes and areas a third more flexible, funding scheme was available – ‘Research or CSA’. While in ICT, Transport and Space this funding scheme was available to a certain extent, it was not possible in SSH. Furthermore, project sizes also had limited flexibility in SSH. In most themes FP7‐COOPERATION differentiated between small or medium scale focused research projects, and large‐
scale integrating projects research projects (the funding limits for the two types differ across themes). A third category with flexible budget size was used in 12% of the whole of FP7 (especially in themes Energy, Environment, Transport, Space and Security), but not in SSH. While scientists of other disciplines were allowed to define the appropriate type and size of research project for a specific topic, the European Commission predefined the type and size of SSH related research more than in any other theme. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 69
Science in Society (SiS): SiS was a sub‐programme of FP7‐CAPACITIES, accounted for 0,65% of the FP7 budget (in total 288 million euro) and funded 183 projects. With an average EC contribution of 1,6 million euro per project and 158.000 euro per participating organization the projects in SiS were even smaller than the ones in SSH. Although the budget share of Civil Society Organizations was 18% and therefore by far higher than in the other areas of FP7 (FP7‐COOPERATION 1,4%, FP7‐PEOPLE 1,5%, FP7‐IDEAS 0,06%), Universities (48%) and Research and Technology Organisations (20%) received the highest shares of SiS funding. With a budget share of only 8,6% the private sector was underrepresented in SiS .84 While partners from large leading EU countries (such as Germany, France, and the United Kingdom) are underrepresented in SiS, the share of organisations from Belgium (headquarters of many umbrella organisations and associations) and from the EU‐13 was above average. The budget allocated to SiS seems reasonable since 52% of the proposals that scored above threshold received funding.85 The draft ex‐post evaluation of SiS pointed out that SiS projects were comparably strong in terms of impacts on EU policies, media visibility and generating success stories, however weaker in scientific breakthroughs and production of information and knowledge. SiS overall demonstrated a clear European added value addressing science and psociaty relevant issues such as governance, ethics, public participation, awareness raising, gender equality, science education, open access to data, as well as dissemination of research and innovation. More that 50% of the SiS projects engaged with societal actors beyond the research community and more than 60% of the publications were published in open access journals or repositories. However, embedding of SiS aspects in the larger setting of FP7 has been modest. Responsible Research and Innovation (RRI)
The increased emphasis on Responsible Research and Innovation (RRI) towards the end of FP7 and in the beginning of Horizon 2020 enriched the traditional view of excellence in research with consideration of impact and sustainable development, coupled with societal responsibility and institutional change. RRI concept aims at aligning research and innovation outcomes to the values, needs and expectations of the European society along the following principles: 86 
Sensitivity to products, processes and purposes of Research and Innovation (R&I) to ensure they are ethically acceptable, sustainable, and socially desirable 
Anticipatory, i.e. proactive engagement with possible and likely consequences of R&I 
Reflexivity, i.e. critical reflection on assumptions underlying R&I activities 
Deliberative, i.e. open discourse on R&I activities with all stakeholders 
Responsiveness, i.e. inclusion of legitimate stakeholder opinions in R&I development RRI theme
Public engagement
Gender equality
Education
Open Access
Ethics
Governance
other
Total
number of projects
FP6 FP7 increase SaS SiS
in %
39
57
146%
33
23
70%
27
35
130%
0
9
x
30
24
80%
10
13
130%
20
15
75%
159 176
111%
EU contrib per project (in 1.000 €)
Partners per project
EU Contribution
FP6 increase in FP6 FP7 increase in increase SaS FP7 SiS
%
SaS
SiS
%
FP6 SaS FP7 SiS
in %
14,7 106,0
721%
377 1.860
493%
7,18 11,91
166%
16,5 35,0
212%
500 1.522
304%
5,36
7,7
144%
18,4 67,1
365%
681 1.917
281%
7,11 11,17
157%
‐
10,1
x
x
1.122
x
X
10,67
x
14,4 20,1
140%
480
838
174%
7,6
7,83
103%
5,0 16,1
322%
500 1.238
248%
6,1
5,77
95%
9,1 14,1
155%
455
940
207%
4,9
8,07
165%
78,2 268,4
343%
492 1.525
310%
6,52
9,81
150% While the number of projects addressing the components of RRI did not substantially increase from FP6 to FP7, the budget nearly tripled (however, compared to the average project size of FP7‐COOPERATION, RRI‐related projects are still comparably small). Building on the experiences of introducing RRI in FP6 and FP7 the concept 84 SiS is also broken down to areas and topics of research that rarely address business issues, from this point of view addressing science in society without involving business and SMEs is questionable. 85 Almost all proposals that scored more than 90% and half of the proposals that scored 80‐90% of possible points were funded in SiS. The average score of a winning proposal was 82%. 86 Owen, R., Heintz, M., Bessant, J. (Eds.), (2013), Responsible Innovation. Wiley Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 70
is now largely embedded into the framework programmes. However, similarly to other cross cutting issues it remains challenging to effectively integrate as a guiding principle in general and into call texts more specifically. Among all of the RRI themes ‘public engagement’ plays the largest role as the number of projects under this theme as well as the EU contribution increased highly above average. On the other hand, even if the budget for gender and ethics related research was increased, the number of funded projects in FP7 was reduced when compared to FP6. The transition from FP7 to Horizon 2020 marks a fundamental change in embedding RRI across European funded research. While so far RRI activities were concentrated in the ‘Science and/in Society’ sub‐programmes the principles of RRI will be integrated into the overall research strategy. Citizen involvement into European research projects aims at increasing trust, acceptance, and ownership of research, a positive perception of science, better adoption of new knowledge and innovations, and improving relevance and creativity of research outcomes. Future Framework Programmes should involve citizens and civil society organisations more substantially, e.g. by including them in evaluation panels or by particular partnership programmes. They should engage citizens and stakeholders in a dialogue about the purpose and benefits of research and the way it is conducted, create incentives for science communication and support more strategic measures of communication addressing different audiences, foster the linkages between researchers, citizens and policy makers. It is recommended to combine the current initiatives for agenda setting and stakeholders involvement in a sub‐programme dedicated to “Visions and Agendas”. Solid mechanisms to strategically embed SSH and RRI throughout Horizon 2020 are required. Goals for CSO participation should be considered. Targeted and tailored communication of science results to key audiences is crucial for ensuring that societal benefits of science and research materialise. Focusing on quality rather than quantity of dissemination activities and fostering sustained engagement of policy makers, researchers, innovators and other societal actors will be crucial to maintain and increase citizen trust in scientific innovations, ensure their take up, support the strategic transfer of research insights into European policy making and advance the wider societal impacts of Horizon 2020. 7.2.
FP7 impacts on gender issues
FP7 addressed two gender related objectives: 1) Equal opportunities for men and women in research, through encouraging equal participation in research teams at all levels, and creating working conditions and culture that allow men and women to have equally fulfilling careers and prevent a waste of talent87; and 2) Gender in research content, that is, equally addressing women’s and men’s realities as an integral part of the research to ensure innovation and the highest level of scientific quality by considering gender as a key analytical and explanatory variable in research and gender specific research to fill knowledge gaps88. The increasing knowledge about the complexity of gender issues in STI led the EC to a shift in focus from addressing women’s “problems” to targeting institutions in promoting change.89 In order to implement and foster a commitment for more gender equality in research a Toolkit on Gender in EU‐funded Research and associated gender trainings was developed by the European Commission90 and statistical data on women in research was started 87 “Europe simply cannot reach the level of SET resources needed for its development without finding ways to remove its anachronistic science gender imbalance” in Gago, J.M. (2004) Europe Needs More Scientists: Contribution by the High Level Group on Human Resources in Science and Technology to the EC Conference: Increasing Human Resources for Science and Technology. Brussels 2 April 2004
88
Buitendijk, S., Corda, D., Flodstrom, A., Holdcroft, A., Hunter, J. Pollitzer, E., Rees, T. Rice, C., Schiebinger, L., Schraudner, M., Sjorup, K. & Tarrach, R. (2011), Women in Science and Medicine, The Lancet, 377, 9768, p. 811; Rees, T. (2011), The Gendered Construction of Scientific Excellence, Interdisciplinary Science Reviews, Special Issue on Gender in Science, 36, 2, pp. 133–45..; Holdcroft, A., Snidvongs, S., & Berkley K.J., (2011), Holdcroft, A., Snidvongs, S., & Berkley K.J., (2011) Incorporating gender and sex dimension in medical research, Interdisciplinary Science Reviews, 36,2, pp. 180‐190. See also: European Commission (2012), European Commission (2012), Report on Structural Change in research institutions: Enhancing Excellence, Gender Equality and Efficiency in Research and Innovation.
89 European Commission (2012), European Commission (2012), Report on Structural Change in research institutions: Enhancing Excellence, Gender Equality and Efficiency in Research and Innovation.. 90 European Commission (2009), Toolkit: Gender in EU‐funded research. Yellow Window Management Consultants, Engender, Genderatwork. 2009. Brussels: European Commission. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 71
to be collected periodically.91 A target of 40% female participation was set for the FP7‐PEOPLE programme, as well as for Expert Groups, advisory boards, monitoring and assessment panels. While FP6 requested applicants to adopt Gender Action Plans for large projects, this prerequisite was lost in FP7. However, in the FP7 Negotiation Guidance Notes concrete examples of actions to be adopted by research teams to support gender equality’s commitment are given instead. Equal opportunities for men and women researchers
While the target of female participation in FP7 was set to Type of position % women 40%, the Framework Programmes have not made 19% substantial progress towards equal opportunities. At the Principal Investigators(*) 30% first glance the target seems to be met as the final reports Scientific coordinators indicate 38% of female participation in FP7 projects on Work Package leaders 29% average. When having a more in‐depth look there is strong Experienced researchers (PhD holders) 34% evidence that FP7 has not managed to overcome the ‘glass PhD students 45% ceiling effects’: the higher the position in a STI project, the lower is the share of women occupying that position. In For the total of FP7 projects which already submitted their final report at 26/03/2015. 2006 only 16‐17% of FP6 project coordinators were women. Source: CORDA, SESAM and RESPIR (Research Performance and Impact Six years later the share of female project coordinators in Reports) FP7 increased to 19,2%. The comparison of these figures (*) Extracted from the signed grant agreements corroborates the structural character of gender inequality in (6th FP7 Progress Report) STI, showing significantly higher differences in highest positions. In this regard FP7 projects show similarities to the general situation in Europe where in 2010 46% of all PhD graduates were women on average in the EU‐
2792. The picture is not uniform as FP7‐COOPERATION and FP7‐CAPACITIES showed substantial differences in women participation, ranging from 21% (JTIs) to 52% (HEALTH) (see Annex 9.10.). The ‘glass ceiling effect’ is smaller in themes with a generally lower share of women (such as Energy, Space and Transport), and higher in areas with a higher proportion of female general workforce (such as Health and KBBE). Future measures to foster women participation in STI should stress early education in the former areas while efforts should aim at ‘breaking the glass ceiling’ in the latter. FP7‐PEOPLE has nearly reached the 40% target with 37% female participation, although there are many disparities between countries and scientific fields. This success of gender balance in FP7‐PEOPLE was related to two types of measures: 1) transparent recruitment methods, which are beneficial for women and a powerful tool against hidden discrimination of women; and 2) actions to improve the work‐life balance of researchers (e.g. providing a dedicated family allowance for researchers with family obligations). All MCA projects had a dedicated family allowance for researchers with family obligations, a key issue in supporting early stage researchers and their mobility. The value of family support measures in MCA is attested by 42% of Individual Fellows (IF) and 44% of Industry‐Academia Partnerships and Pathways (IAPP) applying for this family allowance. Moreover, the 42% of Individual Fellows (IF) displayed an equal ratio of men to women.93 In FP7‐IDEAS only 30% of the proposed Starting and Consolidator Grants and only 15% of the Advance Grants were submitted by female researchers94 and the success rates of female applicants was significantly lower compared to their male colleagues. There was a particularly large gender gap at the top levels which discourages females to apply. This can be taken as an indication that the very concept of ‘excellence’ applied to frontier research is gendered itself. Responding to this concern the ERC “Gender Balance Working Group (GBWG)”95 has drafted two ERC Gender Equality Plans (2007‐2013 in FP7 and 2014‐2020 under HORIZON 91 European Commission (2013), She Figures 2012 Gender in Research and Innovation. Statistics and Indicators. 92 Data from European Commission (2013), She Figures 2012 Gender in Research and Innovation. Statistics and Indicators. 93 Avramov, D. (2015), FP7 ex‐post evaluation People Specific Programme (2007‐2013): Rationale, implementation and achievements. 94 European Research Council (2014) ERC Annual Report 2013, pp. 43‐44. 95 European Research Council (2014), On the way to the top: providing equal opportunities for men and women in science and technology. Working Group on Gender Balance, final summary, 2 December 2013. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 72
2020) and highlighted, that since stereotypes and implicit biases are still strong, there is a need to rethink the concept of an ‘ideal academic’ and the metrics of excellence.96 Gender issues in research content
In FP7‐COOPERATION 63 research projects (out of the total 6,967, which gives a share of 0,9%) representing a total EC contribution of 210 million euro that could be identified as relevant for addressing gender equality issues were funded (the other themes did not include any topics that could be deemed explicitly relevant for addressing gender issues): 
42 projects were funded in the theme HEALTH with a total budget of approximately 150 million euro covering issues such as prevention, diagnosis and treatment of HIV/AIDS, malaria and tuberculosis; reproductive health; access to health care and health systems. 
20 projects were funded in SSH with to total budget of approximately 60 million euro, conducting research on poverty; demographic change; unemployment; social innovation; and families. 
One coordination and support action was funded in SECURITY with a budget of 1 million euro dealing with Trafficking in Human Beings. In FP7‐CAPACTIES the sub‐programme SCIENCE IN SOCIETY funded 19 projects with a total EC contribution of about 30 million euro putting gender issues in the centre of their research content. These projects showed a strong focus on institutional structures, such as gender management in research organisations; study choices of women; involving research bodies in the debate on gender and research; and the promotion of gender equality in research institutions. Compared to the total budget of FP7 the share of projects focusing on gender issues in research content seems marginal, however compared to FP6 the dedicated budget for gender research more than doubled and the average EC contribution to gender related projects increased from 0,5 to 1,5 million euro per project. Despite these improvements data for assessing if and how the gender dimension was introduced in research content is still sparse and consists only of information provided answering three gender related questions in the final project reports template which even runs the risk of being differently interpreted by respondents. Some analysis of the data available: 
27% of the projects in FP7‐COOPERATION and 20% in FP7‐CAPACITIES stated that they carried out specific Gender Equality Actions in their project; 
15% of the projects in FP7‐COOPERATION and 8% in FP7‐CAPACITIES responded that a gender dimension associated with their research content was considered and addressed 
Regarding the types of gender actions carried out, the most frequently selected action was “Design and implement an equal opportunity policy” (20% in FP7‐COOPERATION, 16% in FP7‐CAPACITIES), “Set targets to achieve a gender balance in the workforce” (17% / 14%), “Actions to improve work‐life balance” (15% / 13%), and “Organise conferences and workshops on gender” (4% in both). Although the FP7 Interim Evaluation highlighted the importance of increasing female participation, the problem of gender inequality in FP7 in general, ‘glass ceiling effects’ and low participation and success rates in FP7‐IDEAS in particular persisted. Based on the success of FP7‐PEOPLE in reaching set targets, other specific programmes and sub‐programmes could be improved. However, focusing only on the individual (e.g. by training and promoting individual female researchers) might not be sufficient in addressing gender inequalities, as institutional structures, although seemingly neutral, keep on disadvantaging women in STI, especially in higher positions. Therefore, expanding the perspective from promoting individual female scientists to facilitating structural change in institutions is required and a structural strategy comprising of both, discursive and practical elements, is recommended. FP7 “Structural change projects”, aiming at structural change through fostering Gender Equality Plans in Universities and Research Organizations, can serve as a good practice case. 96 European Research Council (2014), On the way to the top: providing equal opportunities for men and women in science and technology. Working Group on Gender Balance, final summary, 2 December 2013 Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 7.3.
73
Wider societal impacts of FP7
FP7 has strongly reinforced the commitment of the EU to fund research that is of relevance to solving societal challenges. The overarching aim of the whole FP7‐COOPERATION programme, as a result, was set to “contribute to sustainable development”. Contribution to solving pressing challenges facing the European Union today manifest themselves to a varying degree in each of the specific themes in FP7‐COOPERATION either through calling for research for improving knowledge about preconditions for, or creating and improving technologies that increase societal wellbeing. At the time when FP7 was designed and decided sustainable development was high on the political agenda as the European Sustainable Development Strategy (EU SDS) was adopted in 2006. In FP6 a specific thematic priority “Sustainable development, global change and ecosystems” was dedicated to sustainable development with 2,12 million euro total budget allocated97 and focused specifically on renewable energy sources, transport and the sustainable management of terrestrial and marine resources in Europe. However no specific successor thematic priority was implemented in FP7, instead of one thematic priority sustainable development was integrated as an overarching priority for collaborative research. HORIZON 2020 took it even further by defining sustainable development as a cross‐cutting issue for collaborative research. Sustainable development thus is gaining more prominence as a guiding concept for research contributing to societal wellbeing. In September 2015 the UN General Assembly adopted 17 Sustainable Development Goals (SDGs) that are to guide global sustainable development efforts in the years to come. The 17 SDGs are a result of intergovernmental negotiations and mark a shift towards an integrated development agenda that reconciles the objectives of poverty eradication and sustainable development. Since for the first time the new generation of development goals is applicable to all countries developing and developed and it will also guide sustainable development efforts in the European Union. Research and technological innovation is also built in to the SDGs as means for developing country advancement, or means for solving some of the global social challenges such as sustainable use of marine resources, sustainable agriculture, and disease prevention. Since HORIZON 2020 is set to contribute to sustainable development, its contribution to global sustainable development goals is also in the forefront. The European Commission in its communication on 'A global partnership for poverty eradication and sustainable development after 2015” stated that science is an important driver for the implementation of the SDGs and states that HORIZON 2020 is committed to allocate 60% of its budget to sustainable development98. STI as a result has been defined as one of key tools for implementing SDGs globally. In order to assess the impacts of FP7 on sustainable development a monitoring system has been developed by DG Research and Innovation (www.FP7‐4‐SD.eu) designed to monitor the expected impacts of FP7‐
COOPERATION on sustainable development as defined by the aims and objectives of the EU SDS (project FP7‐4‐
SD” commissioned by DG Research and Innovation). Screenings of FP7‐COOPERATION topics showed that approximately 60% of the FP7‐COOPERATION funded projects and 76% of the EC contribution of could be identified to have had positive expected impacts on sustainable development. In response to the redefined global sustainable development agenda and to assess in how far the research funded in FP7 already addressed the 17 SDGs99, an in‐analysis was conducted100. The results confirm the analysis based on the EU SDS, as 4.872 projects (of 6.967 projects in FP7‐COOPERATION) and 19,4 billion euro (of 25,7 billion euro total EC contribution) addressed one or several of the 17 SDGs. In absolute terms the highest number of projects with positive impacts on one or more SDGs and the highest EC contribution to those can be found in Heath, ICT and NMP. In relative terms the share of SDG related projects was the highest in Health, Energy, Environment and Security. 97 European Union Council (2002), Decision 2002/835/EC: Council decision of 30 September 2002 adopting a specific programme for research, technological development and demonstration: "structuring the European Research Area" (2002–
2006) 98 European Commission (2015), Communication from the Commission on A Global Partnership for Poverty Eradication and Sustainable Development after 2015. COM(2015) 44 final. 99 SDGs as defined in as defined in the United Nations (2015), Zero draft of the outcome document for the UN Summit to adopt the Post‐2015 Development Agenda: Transforming Our World by 2030: A New Agenda for Global Action. 100 Dimitrova, A., Dimitrova, A., Hametner, M., Martinuzzi, A. (2015), FP7 and the SDGs ‐ How did research in the 7th EU framework programme address the Sustainable Development Goals (SDGs)?, Vienna: Institute for Managing Sustainability, Vienna University of Economics and Business. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme Number of projects by theme in FP7‐COOPERATION
74
EC contribution (in million euro)
Theme 01 ‐ Health (83%)
Theme 02 ‐ KBBE (69%)
Theme 03 ‐ ICT (55%)
Theme 04 ‐ NMP (78%)
Theme 05 ‐ Energy (87%)
Theme 06 ‐ Environment (94%)
Theme 07 ‐ Transport (56%)
Theme 08 ‐ SSH (76%)
Theme 09 ‐ Space (48%)
Theme 10 ‐ Security (97%)
‐
1.000
0
2.000
2.000
4.000
total EC contribution
6.000
8.000
total number of projects
EC contrib. to projects with positive exp.
impacts on SDGs
projects with positive expected impacts on SDGs
An in‐depth analysis showed that the highest contribution of FP7‐COOPERATION can be expected to SDG3 (‘Ensure healthy lives and promote well‐being for all at all ages’, 1.325 projects, 6,2 billion euro), to SDG 7 (‘Ensure access to affordable, reliable, sustainable and modern energy for all’, 1.378 projects, 5,8 billion euro), and to SDG 12 (‘SDG 12: Ensure sustainable consumption and production patterns, 1.749 projects, 7 billion euro). In contrast FP7 contribution to end poverty, education, gender equality, as well as to conserve and sustainably use the oceans was comparably low (less than 100 projects for each of these SDGs and less than 500 million euro for each of these in terms of EC contribution). Water and sanitation too has been the SDG that was only addressed by a few projects (105 projects). number of projects in FP7‐COOPERATION with expected positive impacts on SDGs
EU contribution (in million euro)
SDG 1: End poverty
SDG 2: End hunger
SDG 3: Healthy lives
SDG 4: Education
SDG 5: Gender equality
SDG 6: Water and sanitation
SDG 7: Access to energy
SDG 8: Growth, employment and decent work
SDG 9: Resilient infrastructure
SDG 10: Reduce inequality
SDG 11: Cities and human settlements
SDG 12: Sustainable consumption and…
SDG 13: Combat climate change
SDG 14: Oceans, seas and marine eco‐systems
SDG 15: Terrestrial ecosystems, biodiversity
SDG 16: Peaceful and inclusive societies
SDG 17: Global partnership for SD
‐
500
1.000
1.500
2.000
‐
2.000 4.000 6.000 8.000
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 75
By now there is no comparable monitoring of HORIZON 2020 implementation that would provide a public platform and comprehensive data on its expected impacts on sustainable development. Although the FP7‐4‐SD monitoring system offered a baseline to assess the impacts of FP7 by identifying the relevant topics and projects, no such thematic impact evaluation on one or several SDGs has been carried out by now. As a consequence, the knowledge and evidence on impacts of FP7 and HORIZON 2020 on society in general and on SDGs in particular is still very limited. FP7 did not equally address all of the sustainable development goals as defined by the United Nations. Some very key development goals remained under‐addressed e.g. poverty, education, water and sanitation. While not all SDGs equally depend on science for their attainment (e.g. reducing global poverty is rather a matter of implementing the right policies than poverty research), achieving some other SDGs is more of a matter of technological advancement and scientific breakthroughs (e.g. climate change, resilient infrastructures). Although in HORIZON 2020 sustainable development is addressed as a cross‐cutting issue and a target of 60% of the total budget was set, the lack of sustainability related research agendas and an effective monitoring system run the risk of losing track of which research areas can contribute the most to achieving sustainable development. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 8.
Looking back and looking foreward
8.1.
Follow up on the FP7 mid-term evaluation
76
The mid‐term evaluation of FP7101 concluded with 10 key recommendations for improvements to be made in the remaining years of FP7 or in the design and implementation of the successor programme. The European Commission has taken up a number of suggestions outlined in this mid‐term evaluation, as documented in the Communication from the Commission on the Response to the report of the Expert Group102. Certain limitations set by the legal base of FP7 limited the extent to which the recommendations could be implemented mid‐way through the implementation of FP7; as a result some of the recommendations were more extensively addressed in the design of HORIZON 2020 (HORIZON 2020). The recommendations and the measures taken by the Commission in the remaining years of FP7 and further in HORIZON 2020 are summarized below. Each of these sections contain a summary of what the mid‐term review recommended; how the Commission reacted during the course of FP7 and in the design of HORIZON 2020; and ends with the evaluative statement on whether the recommendations the interim panel made have been sufficiently addressed either by changes made during FP7 or the design of HORIZON 2020. Some of the recommendations made at the mid‐point of FP7 have also found a central role in the recommendations made by the ex‐post evaluation group. 1. Advance ERA and Innovation Union objectives The midterm evaluation recommended overcoming fragmentation in research; achieving a sharper division of labour between EU level and national programmes; addressing the ‘Grand Challenges’ at the heart of EU research policy; and ensuring three types of agenda setting (researchers setting the agenda in ‘Science for science’, industry setting the agenda in ‘Science for competitiveness’ and civil society actors setting the agenda for ‘Science for society’). During the second half of FP7 no substantial changes of the programme structure were implemented but several attempts at bridging research and innovation, making more interdisciplinary calls and responding to the financial crisis were made. But most of the recommendations of the mid‐term evaluation were not taken up immediately. In contrast, HORIZON 2020 marks a big change by bringing together formerly separate research and innovation programmes and a simpler architecture, centred on three pillars: 'Excellent Science', 'Industrial Leadership', and 'Societal Challenges'. Joint Programming and EC‐co‐funded Public‐to‐Public Partnerships (ERA‐NET and Art. 185) have also been redesigned in a way that allows for more strategic cooperation between the national and EU level and to speed up this cooperation. At the same time the need for more interdisciplinary and cross‐sectorial cooperation made it more demanding to identify the most relevant calls. Overcoming fragmentation and potential overlaps between national programmes and EU programmes remains a challenge and more work needs to be done in better coordination and achievement of synergies of national, transnational and EU‐programmes. Improvements have been made in terms of researchers setting the agenda through ERC and industry setting the agenda within JTIs / Joint Undertakings. Yet, civil society participation in setting the agenda remains marginal and civil society representatives have hardly been included in relevant decision‐making bodies, such as evaluation boards or expert groups. This report therefore concludes that proactive efforts to address civil society and citizens should be enforced. 2. Develop and implement high quality research infrastructures The mid‐term evaluation recommended more coherence between FP7‐CAPACITIES, ESFRI and capacity building efforts undertaken as part of Community Cohesion policy and what is being considered in the context of Joint Programming. The evaluation recommended more effort to boost the Integrated Infrastructure Initiatives (I3), 101 Annerberg, R., Begg, I., Acheson, H., Borrás, S.,Hallén, A.,Maimets, T.,Mustonen, R., Raffler, H., Swings, J‐P., Ylihonko, K. (2010), Interim Evaluation of the Seventh Framework Programme Report of the Expert Group.
102 European Commission (2011) Communication from the Commission on the Response to the Report of the Expert Group on the Interim Evaluation of the Seventh Framework Programme for Research, Technological Development and Demonstration Activities and to the Report of the Expert Group on the Interim Evaluation of the Risk‐Sharing Finance Facility.. COM(2011) 52 final.
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 77
establishing synergies between training instruments and utilisation of research infrastructures, and stimulating industrial and third country access. FP7 support for new research infrastructures targeted the preparatory phase for projects in the ESFRI Roadmap. For some of these projects, possible synergies with Cohesion Policy were worked out and information disseminated to project consortia. Support for I3 continued in the second half of FP7 and Work Programmes highlighted the opportunity for researchers from third countries to benefit from access to European research infrastructures. Several Research Infrastructure projects under the ESFRI Roadmap have been supported by FP7 and ESIF illustrate concrete synergies. Integrating activities in the FP7‐
RESEARCH INFRASTRUCTURE programme have been very successful when it comes to strengthening quality and relevance in national infrastructures – and at the same time contribute to transnational access which is at the heart of ERA. The FP7 contribution to support ESFRI projects has been very important. However, the funds available have been quite limited and not able to cover the real needs in a European context. Coherence remained an issue in the second half of FP7 and has continued to be an issue in HORIZON 2020, although several improvements have been implemented. There is no evidence available that industry or third country access to research infrastructures has actually improved. 3. Maintain the level of funding Responding to the first impacts of the economic crisis and the spread of austerity policy in Europe, the mid‐
term evaluation made a strong statement for maintaining the level of funding for FP7 and highlighted that the competitive challenges that the EU faces require sufficient investment in long‐term economic development. The evaluation advocated that the percentage of the total EU budget that FP7 will have when it ends should be regarded as a minimum for future programmes in order to ensure that excellent proposals are funded and some of the best researchers are not deterred from applying again. During the second half of FP7 the annual funding further increased, as had been planned from the onset, and the programme did not suffer from budget cuts. With total funding of approximately 70 billion euro the subsequent programme. HORIZON 2020, is one of the few areas of the EU's budget for 2014‐2020 to see a significant increase in resources, even if it was not as high as the budget foreseen in the initial Commission proposals. Even if there were proposals to cut the budget in order to strengthen other policy areas in the EU, by and large the FP7 budget has been protected and the programme reached its highest level of funding in its last year of 2013. 4. Develop a well‐articulated innovation strategy The mid‐term evaluation suggested defining a well‐articulated innovation strategy to ensure that instruments and priorities encourage participation of a broad spectrum of enterprises, universities and research organisations. It proposed to help European enterprises integrate in global innovation networks, channel financial support for research and innovation to areas of crucial importance for European competitiveness and reduce administrative burdens. Through continuous support of Joint Technology Initiatives and Public‐Private Partnerships during the second half of FP7, the industrial relevance of FP research was increased and helped whole industry sectors to align themselves behind shared research strategies. Significant efforts were made in enhancing the innovation impact of FP7 mainly through funding projects that took research results closer to market and putting additional emphasis on innovation impacts in evaluating proposals. In addition, the European Institute of Technology (EIT) started to fund Knowledge and Innovation Communities. The shift from research to research and innovation has been further fostered in HORIZON 2020 through the introduction of new funding schemes, the re‐arrangement of sub‐programmes, pilot and market replication projects, and industrial up‐scaling. However, instruments like JTIs and PPPs were initially not sufficiently developed to better grasp the importance of industrial challenges and the long term competitiveness of Europe. While the recommended shift towards innovation and competitiveness was to an extent taken up by a broad diversity of measures, by now no explicit innovation strategy has been developed. This lack of a strategic approach – also in the broader sense within the whole setup of the Framework Programmes – is the basis for one of the key recommendations of this ex‐post evaluation (see Executive Summary “Ensure focus on critical challenges and opportunities in the global context”). 5. Achieving a quantum leap in simplification The mid‐term evaluation called for a “quantum leap” in simplification and for increasing coherence of procedures and approaches between Commission Directorates General and the Executive Agencies; Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 78
considering the revision of the Financial Regulations as an opportunity to create more flexible conditions and a switch from a low‐risk/low‐trust attitude to a more trust‐based and risk‐tolerant approach. During the course of FP7 significant strides towards simplification were made by the redefining of the criteria for the acceptance of average personnel cost methodologies, simplification of SME owner’s remuneration, abandoning the obligation to report interest on pre‐financing and eligibility of non‐recoverable VAT. In addition, a single committee procedure was implemented ensuring uniform interpretation and application of the rules and procedures for implementing research grants. In the programme design of HORIZON 2020 several additional simplification measures were also implemented (see the following chapter). At the same time some critical views have been raised arguing that the Agencies have stricter rules than the DGs and that project officers at DG R&I had a more profound understanding of the research content than the Agencies’ staff. Coherence of procedures and approaches within HORIZON 2020 and beyond (e.g. regarding research funded directly by different DGs) remains an issue of concern. By now simplification was mainly about simplifying the administrative work of the European Commission and making it easier for the applicants. However, simplification has not been taken so far that a switch to the requested trust‐based and risk‐tolerant approach could be observed. 6. Striking a different balance between bottom‐up and top‐down approaches The midterm evaluation suggested changing the mix of funding measures and introducing more open calls in FP7‐COOPERATION during the second half of FP7. In addition, it highlighted the importance of education and research training. While top‐down programming remained a key principle of FP7‐COOPERATION, the relative importance of bottom‐up approaches increased in HORIZON 2020 due to a substantial increase of the ERC budget. In the successor programmes of FP7‐COOPERATION in HORIZON 2020 (i.e. ‘Industrial Leadership’ and ‘Societal Challenges’) the basic principle of top‐down programming was kept, but the orientation of the individual topics was changed from thematic programming to a challenges approach. On‐going work under the European Institute of Technology ensured that education remained a key component of the knowledge triangle and Marie Curie Actions (in FP7 as well as in HORIZON 2020) continued supporting the linkages of research, innovation and training. Instead of introducing bottom‐up approaches in areas that need top‐down approaches to address enabling technologies or societal challenges this evaluation suggests to put a special emphasis on the formulation of research topics. Instead of addressing vague policy aims or presenting lengthy lists of issues, research topics should focus on defining a number of concrete goals, while approaches and methods to accomplish these goals should be determined by the respective research proposal. 7. A moratorium on new instruments Due to the broad variety of funding instruments and the diversity of sub‐programmes, according to the expert panel, the mid‐term evaluation called for a moratorium on new instruments until the existing ones have been sufficiently developed and adequately evaluated. During the second half of FP7 no new funding instruments or sub‐programmes was introduced and the programme was implemented with the current legal base and the existing set of instruments. Joint Programming Initiatives and the European Institute of Innovation and Technology were introduced during this period, in FP7‐IDEAS two new sub‐programmes were introduced in 2011 (Proof of Concept and Synergy Grants). In HORIZON 2020 the whole structure of funding instruments and sub‐programmes was re‐arranged and the variety was reduced. In addition, the evaluation criteria for proposals was harmonized and reduced. 8. Increase female participation in FP7 The midterm evaluation highlighted the importance of increasing female participation in the second half of FP7 and of the leadership role of FP7, but suggested only a few concrete measures, such as a dedicated scheme under the Marie Curie actions and a sensitive but rigorous implementation of the 40% target for female participation in Programme and Advisory Committees. As a response, the monitoring in all stages of the project life‐cycle has been enhanced and new indicators for gender equality have been added. In FP7‐PEOPLE the role of a dedicated Career Restart Panel was reinforced, women applications systematically encouraged and equal opportunities in research careers promoted. In HORIZON 2020 three main objectives have been set: gender balance and equal opportunities in research teams, gender balance in panels and advisory groups and integration of the gender dimension in research and innovation content. The Commission raised the target of Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 79
the female advisory groups’ participation to 50%. However, a significant lack of data to assess or monitor female participation in EU funded research and their career pathways remains, and no dedicated sub‐
programme – neither in FP7 nor in HORIZON 2020 – has been created to boost female participation. Therefore gender balance remains an important issues and should be systematically evaluated and reinforced in HORIZON 2020. 9. Pave the way for increased participation from EU Member States that are underrepresented Aiming at an increased participation of organizations from the EU‐13, the midterm evaluation highlighted the importance of FP7‐PEOPLE and FP7‐CAPACITIES for developing the potential and the resources for their increased participation. In addition, an improvement of connections between Structural Funds and FP7 was suggested. As a response Synergies Expert Groups (SEG) were set up to find synergies between FP7, Structural Funds and the Competitiveness and Innovation Framework Programme. The importance of smart specialization strategies was not very large in FP7, but towards the end it gained a bit of momentum in preparation for Horizon 2020. Within FP7 especially RESEARCH POTENTIAL targeted the organisations from EU Member States that are underrepresented in EU research funding. In terms of connections between funds, the Europe 2020 flagship initiatives on Innovation Union and the Digital Agenda were perceived as an instrument for better alignment of EU policies and activities, including research, innovation and cohesion funding. For the first time, HORIZON 2020 includes a clear legal mandate to maximise synergies with the European Structural and Investment Funds and allocates approx. 800 million euro for the period 2014‐2020 through the specific Part IV dedicated to 'Spreading Excellence and Widening Participation'. 10. Open FP7 to international cooperation The mid‐term evaluation highlighted the importance of opening FP7 to international cooperation and improving the ability of European research and innovation to link up with other regions, markets and research and innovation agendas. It suggested conducting a review based on a thorough analysis of the current strategy towards international cooperation and recommended to integrate the international perspective into all programmes and instruments. As a response, the European Commission adopted a Communication on 'Enhancing and focusing EU international cooperation in research and innovation: a strategic approach' aiming at strengthening the Union’s excellence and attractiveness, tackling global societal challenges, and supporting the Union’s external policies. The first two‐year implementation report, published in autumn 2014, acknowledged the implementation of the strategy and also recommended better integration of international cooperation in the work programme development and refinement of the communication strategy towards the international communities. However, the preliminary results of the first HORIZON 2020 calls have shown a sharp decline in the participation of international partner countries compared to FP7. This is also due to the fact that BRIC‐M partners are no longer systematically funded. Redress measures are foreseen in the H 2020 work programmes 2016‐2017. Until today no quantitative targets for international cooperation and for the share of incoming researchers were set and there is no evidence on the level of impact that could demonstrate whether and to what extent FP7 funded research managed to promote European research links to global research networks. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 8.2.
80
Outlook to HORIZON 2020
HORIZON 2020 integrated the elements of FP7 and existing funding programmes (CIP and EIT). The total budget of HORIZON 2020 has also been increased to about 79 billion euro. This is nearly 50% larger than the budget of FP7 (for a detailed comparison of budgets of FP6, FP7 and HORIZON 2020 by sub‐programmes see Annex 9.13.). In 2015, the planned budget for HORIZON 2020 was cut by 2,2 billion euro to support the European Fund for Strategic Investments (EFSI)103. While the European Research Council (ERC), the Marie Skłodowska‐Curie Actions and the “Spreading excellence and widening participation” programme are not affected by these cuts, “Excellent Science” as a whole was cut by 209 million euro, “Industrial Leadership” by 549 million euro and “Societal Challenges” by 1 billion euro104. In HORIZON 2020, the four specific programmes have been re‐arranged into three major building blocks (Excellent Science, Industrial Leadership and Societal Challenge) that are accompanied by specific measures (Spreading Excellence, Science for Society, EIT, JRC and Euratom). This re‐arrangement increased the coherence of the programme structure compared to FP7, although most of the sub‐programmes were not substantially re‐
oriented or altered. Furthermore, several smaller sub‐programmes of FP7‐CAPACITIES were not prolonged (e.g. Regions of Knowledge, Research Potential Convergence Regions, Coherent Development of Research Policies and International Cooperation). In “Excellent Science” the former programmes FP7‐IDEAS, FP7‐PEOPLE and parts of FP7‐COOPERATION were brought together. Among the sub‐programmes FP7‐IDEAS / ERC shows the highest increase in funding from FP7 to HORIZON 2020. The importance of criteria that specified the so called European Added Value (e.g. cross‐
national collaboration and impact orientation) is substantially lower in this sub‐programme compared to criteria of scientific excellence. This new building block perfectly incorporates the key logic of science and universities: an orientation towards scientific excellence measured by track‐records and publication impacts; a bottom‐up approach allowing applications from any discipline; and a focus on outstanding individual researchers that decide with whom they want to collaborate. In “Industrial Leadership” the industry‐oriented sub‐programmes of FP7‐COOPERATION and the SME sub programme of FP7‐COOPERATION were brought together. In addition, the budget of the newly introduced Risk Sharing Finance Facility was substantially increased. This building block addresses the key logic of the private sector by focusing on innovation (rather than research), competitiveness and high leverage effects, while aiming to reduce administrative burdens. “Societal Challenges” brought together elements focused on societal concerns from FP7‐COOPERATION and retained an orientation towards a European Added Value. The implementation mechanisms of this building block show characteristics of public procurement (e.g. top‐down definition of research areas and topics, strong emphasis on efficient project management, detailed project implementation mechanisms and a strong focus on expected impacts). Here, the highest budget increases compared to FP7 were gained by the sub‐
programmes “Energy” and “Transport”. Compared to FP7, the following adaptations of the programme design were implemented in HORIZON 2020: 
HORIZON 2020 puts a stronger emphasis on innovation through the introduction of ‘close‐to‐market actions’ (e.g. prototyping, testing and demonstrating); an increase of the relative share of innovation projects among all the funded projects; a strengthened support for high‐tech SMEs; and exploring options for integrating venture capital and innovative forms of funding (e.g. inducement prizes, dedicated loan and equity instruments). 103 European Union Council (2015), Investing in European projects: Council and Parliament agree rules on European fund for strategic investments. Press Release. 104 European Parliament (2013), Regulation No 1291/2013 of the European Parliament and of the Council of 11 December 2013 establishing Horizon 2020 ‐ the Framework Programme for Research and Innovation (2014‐2020) and repealing Decision No 1982/2006/EC; Regulation (EU) 2015/1017 of the European Parliament and of the Council of 25 June 2015 on the European Fund for Strategic Investments, the European Investment Advisory Hub and the European Investment Project Portal and amending Regulations (EU) No 1291/2013 and (EU) No 1316/2013 — the European Fund for Strategic Investments. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 81

Easier access for participants is ensured through an increase of two‐stage‐proposal schemes and simplification measures in the proposal phase (e.g. broader acceptance of participants accounting practices for direct costs, flat rate for indirect costs, no time‐sheets for personnel working on a project full time, and the possibility of output‐based grants). Furthermore, simplification measures were also instituted in the implementation of the projects (e.g. participant portal, simplified grant agreements and guidance documents). Planning and management of proposal submissions were improved by the publishing of biannual work‐programmes and a better spread of submission deadlines. The time‐to‐grant period was reduced from 12 months in FP7 to 8 months in HORIZON 2020. 
SME involvement became a cross‐cutting issue in all areas of industrial leadership while the budget for the specific sub‐programme for SMEs was significantly reduced in comparison to FP7. 
In summer 2014, a guide on synergies between the European Structural Funds105, HORIZON 2020 and other innovation and competitiveness related EU programmes was published. It highlights the need for strategic choices and planning by the regions and EU Member States and emphasizes smart specialisation areas. Smart specialisation strategies should build on sound analyses of regional assets and technology, include an analysis of potential partners in other regions and avoid unnecessary duplication. The European Commission provides professional advice to EU countries and regions for the design of their research and innovation strategies for smart specialisation through the S3‐platform. This objective is supported by HORIZON 2020 by implementing its programmes in a synergies‐friendly manner in terms of raising awareness, providing information, engaging in communication campaigns, and connecting National Contact Points to national and regional ESIF policy makers and managing authorities as much as possible. 
Simplified and unified evaluation criteria (excellence, impact and implementation) were introduced except in the ERC (where excellence is the only criterion). 
The funding rates of direct costs were increased and harmonized in HORIZON 2020 following the principle ‘one project – one funding rate’. The contribution to overhead costs was set at 25% for all types of organizations, while in FP7 the contribution of overhead costs followed different types of remuneration schemes. VAT and article processing charges became eligible expenses. 
BRIC countries are no longer eligible for automatic funding under HORIZON 2020. Considering their total GDP, Brazil, Russia, China and India (as well as Mexico) have been given the status of “industrialised country or emerging economy”106. They are invited to participate in HORIZON 2020 with their own funding; funding for participants from these countries continues to be possible in exceptional cases. 
Open access to research publications became a general requirement in HORIZON 2020. A pilot action has been launched regarding Open Research Data aimed at improving and maximising access and re‐use of research data generated. 
The Organizational setting became more complex. While the ERC was established and gained a certain independency from DG R&I in FP7, the implementation of HORIZON 2020 was outsourced to agencies. This meant that the JTIs gained independency in terms of organising their own research agendas and awarding funding for projects on the basis of open calls. Moreover, the establishment of the EIT and its Knowledge and Innovation Communities (KICs) combined excellent research, education and innovation in one institution. Three areas of concern regarding HORIZON 2020 are currently discussed in the STI communities: 
The average success rates are substantially lower in HORIZON 2020 than in FP7 and different potential reasons for this are currently discussed (e.g. reduced funding at national level, back‐log between FP7 and HORIZON 2020, broader application of two‐stage proposal schemes, less prescriptive approach in drafting work programmes and topics). As discussed in Chapter 3.11. the High Level Expert Group recommends to 105 The budget for EU cohesion policy is approximately six times higher than the one for the EU Framework Programmes. During the period 2007‐2013 86 billion euro were allocated to innovation within the out of structural funds – a budget more than 50% higher than FP7.
106 European Commission (2012), Communication from the Commission on Enhancing and focusing EU international cooperation in research and innovation: A strategic approach. COM(2012) 497 final. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 82
take an in‐depth look into the reasons for lower success rates by differentiating between the share of weak proposals (below threshold) and the success rate among proposals of sufficient quality. 
Administrative burden remains an issue although several simplification measures were implemented. The High Level Expert Group recommends to take simplification as a continuous process with active involvement of all stakeholders. In general, a switch to a trust‐based and risk‐tolerant approach is needed. 
The 25% flat rate overhead cost scheme in HORIZON 2020 was critizised as less attractive compared to FP7 where in many cases 60% overhead costs and more were eligible. When discussing this issue, it has to be taken into account that in FP7 only 50% or 75% of the direct costs were funded, while in HORIZON 2020 the funding rate of direct costs has been increased to 70% and 100%. The reduction of overhead costs and the increase of funding of direct costs therefore often compensate each other and led only to a simplification but no net change in funding. 107 The High Level Expert Group believes that the structure of HORIZON 2020 is appropriate. There is a focus on Excellent Science, building on the successes of, amongst others, the ERC, FET and Marie Curie programmes. The second important dimension is Industrial Leadership. This was reinforced by the clear impact of the Public Private Partnership initiatives in FP7 and the potential shown by the JTIs and the engagement of SMEs, all of which have been strengthened and extended in the HORIZON 2020 Programme. Perhaps more fundamentally, there is shift towards a less prescriptive approach to funding Research and Innovation. There seems to be greater recognition of the need to encourage open innovation models, to partner with EU Member States under the Joint Programming Initiatives, and to thereby increase coordination and reduce duplication and fragmentation, while increasing the engagement of the private sector in a number of new ways. The changes from FP7 to HORIZON 2020 do meet the recommendations of this ex‐post evaluation only to a certain extent. Further improvements of the EU Framework Programmes, as well as STI policies, are therefore still high on the agenda: a)
Key areas in which Europe can play a truly leading role on a global scale and can increase Europe's attractiveness for leading researchers and innovators should be addressed. b) A more strategic approach and alignment with other DGs and EU Member States is required. c)
The externalisation of programme management to agencies has increased the fragmentation of project implementation – coordination is needed. d) HORIZON 2020 and subsequent FPs should address citizens’ concerns better and involve them in a more substantial role in the future. e) A truly strategic programme monitoring and evaluation culture should be established. 107 There are two areas where HORIZON 2020 funding is less attractive: (1) for organisations that charged overhead costs based on real indirect costs and that had charged higher than 60% rates, (2) for work packages that were not research and innovation oriented (e.g. dissemination) and already received 100% funding of direct costs. In addition, (higher) funding of direct costs and (reduced) funding of overhead costs decreases the freedom of the grantee in using the EC contribution. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 83
(a) Ensure focus on critical challenges and opportunities in the global context HORIZON 2020 followed a more focused approach in setting research agendas; identified a number of key technology areas and societal challenges; and widened the scope of instruments. The JTI model has been endorsed and resources have been increased. It is probable that the strong emphasis on SME participation will spill over to national programmes. Moreover, Eurostar's programme plays an important role in linking EU funding with national funding of SMEs. However, the need to identify a number of key areas in which Europe can play a truly leading role on a global scale and to increase Europe's attractiveness for leading researchers and innovators must remain an important objective. Interaction with national programmes and instruments is still very limited. This requires further attention since the interplay between R&I instruments and policy and legislative measures is crucial, and may need to be strengthened. Additionally, the contractual framework for JITs is still too complicated and it should be assessed if the JTI instrument ensures a sufficiently future‐
oriented approach in research and innovation priorities. (b) Align research and innovation instruments and agendas in Europe First steps towards aligning national and EU programmes were taken by the sub‐programmes Co‐fund, ERA‐
Net+, JPI and ESFRI. These amendments were mostly implemented within HORIZON 2020. However, there is still a lack of a more strategic approach and no alignment with other DGs and EU Member States was implemented. The call texts under HORIZON 2020 are more open and less prescriptive compared to FP7 and there is a stronger emphasis on impacts. Yet, greater emphasis should be put on HORIZON 2020 as a strategic intervention into the totality of research and innovation systems of the EU and its Member States. Steps towards establishing an EU‐wide quality stamp for outstanding scientific and enterprise driven proposals were implemented in the area of SMEs. However, there should be no restriction to certain types of organisations. Furthermore, outstanding proposals that could not be funded within HORIZON 2020 should receive priority by other national and EU programmes. Several guides and support platforms for aligning HORIZON 2020 and innovation actions of Structural Funds were already implemented. However, an overarching "Common Science, Technology and Innovation Policy" would be a major step forward in aligning the Framework Programmes with related policies and programmes. In addition, the creation of a dedicated science, technology and innovation support fund within the Structural Funds is recommended. (c) Integrate the key components of the Framework Programmes more effectively The restructuring of HORIZON 2020 along the three building blocks improved the coherence and communicability of the programme. However, the externalisation of programme management to agencies has increased the fragmentation of project implementation and made effective coordination even more difficult. A critical review of the division of roles and responsibilities between the Commission and the agencies is needed. Predictability has been increased and the peak load has been significantly reduced by bi‐annual work programmes in HORIZON 2020 and by spreading the submission deadlines over a longer period of time. Several target groups and policy aims that were addressed by individual sub‐programmes in FP7 (e.g. SMEs, Socio‐
Economic Sciences and Humanities, EU‐13 and International Collaboration beyond Europe) became cross‐
cutting issues and were integrated into the three building blocks of HORIZON 2020.108 If this integration has been successful should be assessed in the mid‐term evaluation of HORIZON 2020. Although several measures have been implemented to better integrate different parts of HORIZON 2020, avoiding silos, fragmentation and in transparency is still an important issue, as different units, agencies and DGs tend to promote „their“ issues and sub‐programmes. 108 It was not within the scope of this evaluation to assess if this integration was successful, but it is strongly recommended to put a special emphasis on this issue in the mid‐term evaluation of HORIZON 2020.
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme (d) 84
Bring science closer to the European people Responsible Research and Innovation became a cross‐cutting issue in HORIZON 2020; the sub‐programme “Science with and for Society” has a slightly higher budget than FP7‐SIS and the Citizen Science concept gained more attention during the last years. Nevertheless, many European citizens have a rather critical view of innovation and thus science runs the risk of losing contact to European citizens. Furthermore, the creative power of highly diverse teams of scientists is not exploited to its full potential, since the different types of organisations tend to collaborate in convergent groups. While HORIZON 2020 explicitly addresses “Societal Challenges”, the programme’s design follows a top‐down approach rather than involving citizens and society in a substantial manner. Although participatory elements have been strengthened, research agenda setting and work programme development is often perceived as taking place behind closed doors and highly important concerns of European citizens are only marginally addressed (e.g. social cohesion, European integration and combating unemployment). In addition, themes and topics often follow a technological fixing‐
the‐problem approach instead of addressing societal causes and major transformation processes. In order to promote a more positive public perception of science and better adoption of new knowledge and innovations, the Framework Programmes will have to address citizens’ concerns better and involve them in a more substantial role in the future. (e) Establish strategic programme monitoring and evaluation For HORIZON 2020 a set of performance indicators was newly established, comprising indicators for scientific excellence (e.g. publications, mobility patterns and usage of research infrastructure); industrial leadership (e.g. patent applications, share of SME participation and total investments mobilised via innovative investments instruments); and societal challenges (e.g. publications, patent applications and prototypes). At the same time, several indicators available FP7 are no longer obtainable due to simplification measures in reporting. It is therefore strongly recommended to ensure the continuity of datasets; improve the data on individual participating researchers (unambiguously identified); standardize data sets; align evaluation purposes, criteria and questions and make better use of evaluation results. The „Better Regulation Package“109 was published on May 19 2015. As a consequence, internal evaluations of FP7 and HORIZON 2020 will be carried out by DG R&I and will address effectiveness, efficiency, coherence, relevance and EU added value. It will depend on the implementation practice of this new procedure if this will merely constitute an additional administrative exercise or improve the systematic generation of knowledge and strategic intelligence of HORIZON 2020. In addition, regular evaluation syntheses and meta‐evaluations, more centralized evaluation guidance (including additional budget and staff), are urgently needed to enable better, evidence‐based and strategic decision‐
making and foster continuous learning in HORIZON 2020. 109 European Commission (2015), Communication from the Commission on Better regulation for better results ‐ An EU agenda. COM(2015) 215 final.
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 9.
Annex
9.1.
Profiles and CVs of the members of High Level Expert Group
85
Chair: Prof. Dr. ir. Louise Fresco (The Netherlands) President and Chairman of the Executive Board of Wageningen University and Research Centre Louise Fresco is the President of the executive board of Wageningen University and Research Centre, one of the leading research institutions worldwide in the field of Food, Agriculture and Life Sciences. She served as Assistant Director‐General for Food and Agriculture Organization of the United Nations in Rome, conducted field work in over 50 developing countries, served on the supervisory board of Rabobank and currently serves on the Board of Unilever. Louise Fresco is also a published author and documentary maker. For details see: www.louiseofresco.com. Raporteur: a.Prof. Dr. André Martinuzzi (Austria) Head of the Institute for Managing Sustainability, Associate Professor at Vienna University of Economics and Business For more than 15 years André Martinuzzi has been coordinating projects funded by the EU Framework Programmes and conducted research projects on behalf of a number of different EC Directorates General, Eurostat, the United Nations Development Programme and several national ministries. He was the rapporteur for preparation of the Austrian Sustainable Development Strategy, the CSR guiding vision and the Sustainable Forestry Programme in Austria. His main areas of research are corporate sustainability, sustainable development policies, monitoring and evaluation, knowledge brokerage and sustainability innovation. For details see: www.sustainability.eu. Rapporteur’s assistant: Adele Wiman, MSc. (Finland) Research Fellow at Institute for Managing Sustainability at Vienna University of Economics and Business Researcher in a number of FP7 funded research projects in the areas of sustainable development and corporate social responsibility. She formerly served as parliamentary assistant to a Member of the European Parliament and cross‐sector partnership broker at the United Nations Development Programme. She holds a Double Master of Science degree in Public Policy from the Maastricht Graduate School of Governance and United Nations University and a BA from University of Warwick. Members (in alphabetical order) Prof., PhD, FRCPath Maria Anvret (Sweden) Senior Advisor, The Sahlgrenska Academy, University of Gothenburg Maria Anvret has a well‐documented international scientific background within the area of life sciences. She received her PhD and professorship at the Karolinska Institute, was a fellow and Postdoc at Yale University. She has served as manager, director, global program director, clinical counsellor, scientific expert and leader, strategic leader, policymaker and spokesman in academia, industry as well as public, private organizations and NGOs. She is a member of numerous boards within life sciences organisations. She is a member of The Royal Swedish Academy of Engineering Sciences. Over the years she has been part of the knowledge community including research, education, technology and innovation. Her focus area is translation, development, implementation and communication of science to create added value and benefit for society. Dr. María Bustelo (Spain) Associate professor and director of the Equality Unity at the Complutense University of Madrid María Bustelo is director of the Equality Unit at the Complutense University of Madrid (UCM) and an associate professor in the area of public policies, political science and public administration. Director of the Master on Evaluation of Programmes and Public Policies (UCM), she served as President of the European Evaluation Society 2012‐2013, and Member of the Board of Director Committee of the Spanish National Agency for the Evaluation of Public Policies (2007‐2011). She currently integrates the UNWomen Global Evaluation Advisory Committee, and the Expert Group on Gender Training at the UN Women Training Centre. She is a leader of Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 86
several National and European research projects at the UCM on the quality of gender equality policies and structural change at universities and research organisatons, MAGEEQ (2003‐2005, FP5) QUING (2006‐2011, FP6) and GENOVATE (2013‐2016). She has a number of publications on evaluation theory and methodology as well as on gender equality policies. Prof. Dr. Eugenijus Butkus (Lithuania) Professor at Vilnius University Eugenijus Butkus previously served as the Chairman of the Research Council of Lithuania during 2003‐2013, Vice‐Rector for research of Vilnius University in 2013‐2015 and Vice President of the European Science Foundation (ESF) in 2012‐13. He has also been the Chair of the BONUS program steering committee (2013‐14). He has coordinated the FP7 collaborative project Servicing Policy for Resource Efficient Economy. In 2014 he has been awarded the National Science Prize. Eugenijus is a full member of the Lithuanian Academy of Sciences. Prof. Michel Cosnard (France) Professor, Université de Nice – Sophia Antipolis, Scientific Adviser, Inria Michel Cosnard has served as Chairman and CEO of Inria (2006‐2014), Chairman of the ERCIM board of directors (2011‐2014), Chairman of the French Alliance for Digital Sciences and Technologies (Allistene) (2009‐
2014). He has been Member of the FP6 IST Evaluation Committee (2006), Member of ISTAG (Information Society Technologies Advisory Group) (2008‐2012), Member of the advisory committee of NII and NICT (Japan), LIAMA (France‐China) and CWI (The Netherlands). In 1995, he received the IFIP Silver Core, in 2003 the IPDPS Babbage Award and in 2010 the Euro‐Par Outstanding Award. In 2007, he was awarded the title of Chevalier de la Légion d'Honneur and in 2013 Officier de l’Ordre National du Mérite. He received an Honorary Doctoral Degree from Ecole Polytechnique de Mons (Belgium) in 2009. Arvid Hallen, PhD (Norway) Director General of the Research Council of Norway Arvid Hallén is the head of Research Council of Norway, which is the key research funding agency in Norway. Among previous positions he has served as Executive Director of the Culture and Society Division of the Research Council and as Director of The Norwegian Institute for Urban and Regional Research. He has been involved in several research‐based evaluations and has served as member of international evaluation panels in the research policy field. He was at the panel of the Mid‐term Evaluation of FP7. He serves as a member of the Governing Board of Science Europe. Dr. Yuko Harayama (Japan) Executive Member, Council for Science, Technology and Innovation (CSTI), Cabinet Office, Government of Japan Yuko Harayama is an executive member of CSTI, which overlooks all of Japan’s science, technology and innovation (STI), formulates comprehensive and basic policies on STI and conducts their overall coordination. Prior to joining the CSTI, she spent two years at the Organisation for Economic Development and Cooperation (OECD) as the Deputy Director of the Directorate for Science, Technology and Industry and more than ten years at the Graduate School of Engineering of Tohoku University as a professor of Science and Technology Policy. For details see: http://www8.cao.go.jp/cstp/english/policy/cv/yukoharayama.pdf. Dipl.‐Ing, Dr. Sabine Herlitschka, MBA (Austria) Chief Executive Officer, Chief Technology Officer; Infineon Technologies Austria AG Sabine Herlitschka, an expert in Food‐ and Biotechnology by education, has been holding leadership positions in European & international research funding management, university executive management as well as industry. She has been frequently involved in European Research as of the 4th EU Framework Programme, and has coordinated several significant EU Framework Programme Projects herself. Furthermore, she served as advisor, proposal evaluator, Coordinating National Contact Point, and expert in various national, European and international expert groups. At EU level, she was Rapporteur in the “High‐level Expert Group on Frontier Research" that contributed to the development of the European Research Council, Rapporteur to the INCO Advisory Group, Chairwoman of the Expert Group on "Diversified Funding Streams for University‐based Research”, as is currently member of the CONNECT External Advisory Forum. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 87
Prof. Dr. Stefan Kuhlmann (Germany) Professor of Science, Technology, and Society; Chair dept. Science, Technology, and Policy Studies (STePS), University of Twente, The Netherlands Stefan Kuhlmann has been studying research and innovation, related systems and policies, at national and European levels (ex post and foresight) since 1980s. Until 2006 he served as Acting Director of Fraunhofer Institute for Systems & Innovation Research (ISI), Karlsruhe, and Professor of Innovation Policy at University of Utrecht. Member of many European and national expert advisory groups he is also publishing widely, he is the editor of 'Research Policy' (Elsevier). For details see: http://www.utwente.nl/bms/steps/people/scientific/kuhlmann/. a.Prof. Dr. Vesselina Nedeltcheva (Bulgaria) Associate Professor Emeritus at the Bulgarian Academy of Sciences, Economic Research Institute For more than 10 years Vesselina Nedeltcheva has been an evaluator and reviewer of projects funded by the EU Framework Programmes. In the last few years she has been an Expert in Horizon 2020 Advisory Group on Energy and in Horizon 2020 Advisory Group ‘Science with and for Society’. She is an author of about 10 individual and several joint monographs. She was a Visiting Professor at the Nihon University in Tokyo, a Visiting Fellow of Japan Foundation, Institute of Developing Economies in Tokyo, Central European University in Budapest, Politechnical University in Milano and Torino, Goethe University in Frankfurt, Leibniz University in Hannover, Netherlands Institute for Advanced Studies in Wassenaar etc. Her main areas of research are structural‐investment and scientific‐innovation policies, energy efficiency, green strategies, advanced technologies. Richard Fowler Pelly (United Kingdom) Advisor and Non‐Executive Director in the area of finance and development of SMEs, entrepreneurship and innovation Richard Pelly served as Chief Executive of the European Investment Fund (EIF) from 2008‐2014, and previously Managing Director of Structured Asset Finance at Lloyds TSB Bank; CEO of UK Business Finance within GE Commercial Finance; Chairman and CEO of Budapest Bank in Hungary. Prior to career at GE Commercial Finance, he worked for Barclays Bank in various functions in the UK and in France. He has earned his Bsc in Psychology from Durham University, Diploma from the British Institute of Bankers and MBA with distinction from INSEAD Fontainebleau. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 9.2.
88
Data sources processed in preparing this report
FP7 Monitoring Reports
European Commission (2010), Third FP7 Monitoring Report. Monitoring Report 2009. European Commission (2011), Fourth FP7 Monitoring Report. Monitoring Report 2010. European Commission (2012), Fifth FP7 Monitoring Report. Monitoring Report 2011. European Commission (2013), Sixth FP7 Monitoring Report. Monitoring Report 2012. Programme Evaluation Activities Reports
European Commission (2012), DG RTD Annual Report on Programme Evaluation Activities 2011. European Commission (2013), DG RTD Annual Report on Programme Evaluation Activities 2012. European Commission (2014), DG RTD Annual Report on Programme Evaluation Activities 2013. Reports of Supporting Experts
Peter, V. (2015), Synthesis report of the FP7 COOPERATION sub‐programme. Halme, K. (2015), CAPACITIES ‐ Summary Report of the Capacities Specific Programme of FP7 for the High Level Expert Group. Bonaccorsi, A. (2015), Ex‐post Evaluation of the Seventh Framework Programme. Support paper to the High Level Group. IDEAS Specific Programme Evaluation. Avramov, D. (2015), FP7 ex‐post evaluation People Specific Programme (2007‐2013): Rationale, implementation and achievements. Horizontal Studies
Campbell, D., Lefebvre, Ch., Picard‐Aitken, M., Côté, G., Ventimiglia, A., Roberge, G., Archambault, E. (2013), Country and Regional Scientific Production Profiles. Campbell, D., Ventimiglia, A., Archambault ,E. (2013), Scientific Output and Collaboration of Companies Publishing the Most in the ERA. Roberge, G., Campbell, D., Beauchesne, O, H., Ventimiglia, A., Picard‐Aitken, M., Haustein, S., Archambault, E. (2013), Scientific Output and Collaboration of European Research Public Organisations. Labrosse, I., Ventimiglia, A., Campbell, D., Haustein, S., Côt, G., Archambault, E. (2013), Scientific Output and Collaboration of European Universities. Campbell, D., Caruso, J., Archembault, E. (2013), Cross‐Cutting Analysis of Scientific Publications versus Other Science, Technology and Innovation Indicators. Zagamé, P., Fougeyrollas, A., Le Mouël, P. (2012), Consequences of the 2013 FP7 call for proposals for the economy and employment in the European Union. European Commission (2012), Success Stories 2012 ‐ Investing in Success. Deloitte (2014), Researchers’ Report 2014. IDEA Consult; iFQ; PPMI (2013), Study on assessing the contribution of the framework programmes to the development of human research capacity. Annerberg, R., Begg, I., Acheson, H., Borrás, S., Hallén, A.,Maimets, T.,Mustonen, R., Raffler, H., Swings, J‐P., Ylihonko, K. (2010), Interim Evaluation of the Seventh Framework Programme. Report of the Expert Group. Archambault, E., Campbell, D., Caruso, J., Côté, G., Hassan, E., Lavoie, R., Mitchell, B., Nievas, K., Rashid, M., Roberge, G., Lalonde, S.L., Ventimiglia, A., Kroll,H., Meyborg, M., Meyer, N., Møller, K., Light,D., Larsen,M., Bom, F., StryhnKoch, M., Hornnes,K. (2015), Network Analysis of FP7 participation. European Commission (2014), Pilot Study on using data mining to measure specialization patterns. everis (2014), Patent costs and impact on innovation: International comparison and analysis of the impact on the exploitation of R&D results by SMEs, Universities and Public Research Organisations. Politecnico Di Torno; ISMERI EUROPA; CERIS (2015), An analysis of the role an engagement of universities with regard to participation in the Framework Programmes. EY; Technopolis Group; Open Evidence (2015), An analysis of the role and impact of Research and Technology Organisations’ participation in the Framework Programmes. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 89
FP7-COOPERATION – Theme Health
European Commission (2011), Impact Assessment of the Health Theme and proposal for post‐2013 EU support for health research and innovation. European Commission (2013), Review of Public Health Research Projects Financed under the Commission’s Framework Programmes for Health Research. PPMI (2015), Ex‐post evaluation of the Health Theme in FP7: Preliminary Report. Preliminary analysis of FP7 projects portfolio and their outcome. FP7-COOPERATION – Theme KBBE
Horvat, M., Ricci, A., Casal, M., Griniecce, E., Pianta, M., Tjel, J.Ch., (2011), Impacts of EU Framework Programmes (2000‐2010) and Prospects for Research and Innovation in Food, Agriculture, Fisheries and Biotechnologies. Gherghinescu, O., Kraigher, H., Lazzeri, P., Murphy‐Bokern, D., Peeters, A., Schmid, O., Scudamore, J., Toepfer, S., Westergaard, J. (2011), Impacts of EU Framework Programmes (2000‐2010) and prospects for research and innovation in agriculture, animal health and welfare, and forestry. Vanhemelrijck, J., Lloyd‐Evans, M., Martins de Santos, V., Panopoulos, N., Querellou, J., Socaciu, C., Steinmetz, (2011), Biotechnologies Panel Report: Food Agriculture and Biotechnology. Impacts of EU Framework Programmes (2000‐2010) and Prospects for the Research and Innovation in Biotechnologies. Quantick, P., Neubert, H.J, Carrerras, J., Eberle, U., Holdsworth, M., Ohlsson, T., Vauchez, C., de Winter, K., (2011), Impact Assessment “Food”. Atkinson, D.B., Agius, C., Cahu, C., Kennedy, L., Laurec, A., Rodgers, P., (n.y.) Impacts of EU Framework Programmes (2000‐2010) and Prospects for Research and Innovation in Fisheries and Aquaculture. Baur, M., Serri, L. (2014) Ex‐post evaluation The Ocean of Tomorrow calls (2010‐2013). FP7-COOPERATION – Theme ICT
Dinges, M., Poel, M., Søndergaard Laugesen, N. (2011), Beyond Patents and Publications: Performance Monitoring Indicators for ICT. Ramboll Management A/S (2008), Ex‐post Evaluation of the IST Thematic Priority in the 6th FP. EPEC (2010), Interim evaluation of the ICT research in the 7th Framework Programme. Bravo, A., León, G., Luukkonen, T., Raffler, H., Truvé, S., Turk, Ž., Wright, S., Arnold, E. (2010), Interim evaluation of the ICT research in the 7th Framework Programme. Kuneva, M., Dózsa, C., Mann, E., Miles, I., Parent, A‐S., Vasconcelos, D., Millard, J. (2010), Interim Evaluation of the Ambient Assisted Living Joint Programme. Unlocking innovation in ageing well. Busquin, P., Aarts, E., Dózsa, C., Mollenkopf, H., Uusikylä, P., Sharpe, M. (2013), Final Evaluation of the Ambient Assisted Living Joint Programme. Abadie, F., Boehler, Ch., Lluch, M., Sabes‐Figuera, R. (2014), Monitoring and Assessment Framework for the European Innovation Partnership on Active and Health Ageien (MAFEIP). European Commission (2015), Ex‐post evaluation of ICT research in the Seventh Framework Programme. FP7-COOPERATION – Theme NMP
European Commission (2014), Exploitation Strategy and Innovation Consultants (ESIC) Services for NMP Projects. Medina, A., Thompson, D., Newton, R., Christensen, M., Drumond, C., Morrison, M., Gordeyev, S., Dingwall, K., O’Rourke, E., Robinson, D. (2013), Comparative Scoreboard and Performance Indicators in NMP. Research Activities between the EU and Third Countries. Heriot‐Watt University (2015), ITS‐NANO Report Summary. Periodic Report Summary 1 ‐ ITS‐NANO (Intelligent Testing Strategy for Engineered nanomaterials). FP7-COOPERATION – Theme Energy
Van der Veen, G., Altmann, M., Eparvier, P., Ploeg, M., Trucco, P., Hinico (2014), Evaluation of the impact of projects funded under the 6th and 7th EU Framework Programme for RD&D in the area of non‐nuclear energy. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 90
FP7-COOPERATION – Theme Environment
Helming, K., Amanatidou, E., Kalpazidou Schmidt, E., Kemp, R., Nilsson, M., Ricci, A. (2011), Ex‐ante Impact Assessment of the Common Strategic Framework (CSF) for Research and Innovation ‐ Environment Theme ‐ Expert Group "Stocktaking of results and Impacts of EU‐funded environmental research". Helming, K., Connolly, N., Amanatidou, E., Rem, P., Wilenius, M. (2014), Ex‐Post Evaluation of FP7 Cooperation Programme Theme: “Environment (including Climate Change)”. FP7-COOPERATION – Theme Transport
European Commission (2014), Synergies between the Transport Component of Horizon 2020 and the Cohesion Policy 2014 ‐ 2020. Expert Group Report. 7th Research Framework Programme Transport Programme Committee. Halvarsson, A., Rydell, I., Holmström, M., Blagoeva, T., Bain, R. (2014), Ex‐post Evaluation of the Loan Guarantee Instrument for the Transeuropean transport Network (TEN‐T) Projects. FP7-COOPERATION – Theme SSH
Watson, J., Kitchener, M., Gutheil, M., Ward, B., Zadrozny, T., Ackers, L., Harrap, K. (2010), Evaluation of the impact of the Framework Programme on the formation of the ERA in Social Sciences and the Humanities (SSH). IMPACT‐EV Coordinator Team (2015), Ex‐post evaluation of the FP7 Cooperation in Socioeconomic Sciences and Humanities (2007‐2013). FP7-COOPERATION – Theme Space
Booz & Company (2014), Evaluation of Socio‐Economic Impacts from Space Activities in the EU. Centre for Strategy & Evaluation Services (2011), Interim Evaluation of FP7 Space. FP7-COOPERATION – Theme Space
Bigo, D., Jeandesboz, J., Martin‐Maze, M., Ragazzi, F., (2014), Review of Security Measures in the 7th Research Framework Programme FP7 2007‐2013. Centre for Strategy & Evaluation Services (2011), Ex‐post Evaluation of PASR Activities in the field of Security. Interim Evaluation of FP7 Research Activities in the field of Space and Security. Aviation Security and Detection Systems ‐ Case Study. Centre for Strategy & Evaluation Services (2011), Ex‐post Evaluation of PASR Activities in the field of Security. Interim Evaluation of FP7 Research. CBRB Case Study. ECORYS (2012), Study on Civil Military Synergies in the field of Security. European Commission (2012), Communication from the Commission on Security Industrial Policy. Action Plan for an innovative and competitive Security Industry. COM(2012) 417 final. Bergkamp, L., Faure, M., Hinteregger, M., Philipsen, N. (2013), Study evaluating the status quo and the legal implications of third party liability for the European security industry. Centre for Strategy & Evaluation Services (2011), Ex‐post Evaluation of PASR Activities in the field of Security. Interim Evaluation of FP7 Research Activities in the field of Space and Security. SMEs and their Participation in Security Research ‐ Case Study. Centre for Strategy & Evaluation Services (2011), Ex‐post Evaluation of PASR Activities in the field of Security and Interim Evaluation of FP7 Security Research. Maritime Security and Surveillance ‐ Case Study. Centre for Strategy & Evaluation Services (2011), Ex‐post Evaluation of PASR Activities in the field of Security. Interim Evaluation of FP7 Research Activities in the field of Space and Security. Crisis Management ‐ Case Study. Centre for Strategy & Evaluation Services (2011), Ex‐post Evaluation of the Preparatory Action on Security Research (PASR). Interim Evaluation of FP7 Security Research. Technopolis Group (2015), Final Evaluation of Security Research under the Seventh Framework Programme for Research, Technological Development and Demonstration. JTIs
JTI Sherpas' Group (2010), Designing together the ‘ideal house’ for public‐private partnerships in European research. European Commission (2012), Annual Progress Report on the activities of the Joint Technology Initiatives Joint Undertakings (JTI JUs) in 2011. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 91
Gvillo, F., Rosenmöller, M., Andersen, T., Horvat, M., Keir, R., Wijnberg, B. (2011), First Interim Evaluation of the Innovative Medicines Initiative Joint Undertaking. European Commission (2011), First Interim Evaluation of the Fuel Cell & Hydrogen Joint Undertaking. Bertolini, E., Eckardt, D., Hecker, P., Herrera, I., Horvat, M., Huguet, M., (2010), CLEAN SKY. 1st Interim Evaluation. Hunter, J., Szumowski, M., Andersen, T., Rosaria, M., Nucci, D., Wijnberg, B.(2013), Second Interim Evaluation. IMI ‐ Innovative Medicines Initiative Joint Undertaking. European Commission (2013), Second Interim Evaluation of the Fuel Cell & Hydrogen Joint Undertaking. European Commission (2013) Second Interim Evaluation of CLEAN SKY Joint Undertaking. European Commission (2014), Second Interim Evaluation of the CLEAN SKY, FCH and IMI Joint Technology Initiatives Joint Undertakings. Bernotat, W.H., De Post, Ch., Eckstein, E., Georghiou, L., Luukkonen,T., Malcolm, B., Potier, D., Sangiovanni‐
Vincentelli, A. (2010), First Interim Evaluation of the ARTEMIS and ENIAC Joint Technology Initiatives. Goetzeler, M., Arden,W., Dormoy, J‐L., Jansz,M., Luukkonen,T.,De Prost,Ch., Sangiovanni‐Vincentelli,A., Wright, C.D. (2013), Second Interim Evaluation of the ARTEMIS and ENIAC Joint Technology Initiatives. European Commission (2013), Second interim evaluation of the ARTEMIS and ENIAC Joint Technology Initiatives. FP7-PEOPLE
Economisti Associati (2014), Marie Curie researchers and their long‐term career development: A comparative study. FP7-IDEAS
European Research Council (2012), Understanding and Assessing the Impact and Outcomes of the ERC and its Funding Schemes (EURECIA). FP7-CAPACITIES – Research Infrastructures
Calvia‐Goetz, A., Franciosi, A., Larsen, S., Marks, J., Tichmann, K., Wade, R., Zic Fuchs, M. (2013), Assessing the projects on the ESFRI roadmap. A high level expert group report. ESRIF (2009), ESRIF Final Report. European Commission (2013), The role of clusters in smart specialisation strategies. Fotakis, C., Rosenmöller, M., Brennan, J., Matei, L., Nikolov, R., Petiot, C., Puukka, J. (2014), The role of Universities and Research Organisations as drivers for Smart Specialisation at regional level. FP7-CAPACITIES – SMEs
European Commission (2013), Investing in European success Empowering European SMEs to Innovate and Grow. Makarow, M., Licht, G., Caetano, I., Czarnitzki, D., Elçi, S. (2014), Final Evaluation of the Eurostars Joint Programme. European Commission (2012), Eighth Progress Report on SMEs’ participation in FP7 2007‐2011. European Commission (2012), Ninth Progress Report on SMEs participation in the 7th R&D Framework Programme. European Commission (2013), Tenth Progress Report on SMEs participation in the 7th R&D Framework Programme June 2013. European Commission (2014), Small and Medium‐Sized Enterprises. Participation in FP7 projects in the Biotechnologies Activity. Van Elk, K., Snijders, J., Prince, Y., Gibcus, P., Doove, S., Simmonds, P., Warta, K., Good, B., Ruhland, S., Sheikh, S. (2014), Performance of SMEs within FP7. An Interim Evaluation of FP7 components. FP7-CAPACITIES – Regions of Knowledge
Bruno, N., Mahieu, B., Lacave, M., Köse, G. (2011), Assessment of the impact of the ‘Regions of Knowledge’ programme. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 92
FP7-CAPACITIES – Research potential of Convergence Regions
Claude, J., Flores, O., Frackowiak, K., Iacovou, C., Lehmann, A., León, G., Lindley, D., Miron, L., Russo, P., Smits, Y. (2011), Analysis and recommendations for the future of the “Research Potential” Activity. Expert Group report on Research Potential project portfolio. FP7-CAPACITIES – Science in Society
Technopolis Group; Fraunhofer (2012), Interim evaluation & assessment of future options for Science in Society Actions. Interim evaluation. Wilkinson, Ch., Franke, J. (2015), Ex‐post evaluation of Science in Society FP7. Support to the coherent development of research policies
Allinson, R., Arnold, E., Harper, J., Doranova, A., Giljum, S., Griniece, E., Kubeczko, K., Mahieu, B., Markandya, A., Miedzinski, M., Peter, V., Ploeg M., Stasiakowska, A., Van der Veen, G. (2013), Assessing Environmental Impacts of Research and Innovation Policy. Bason, Ch., Hollanders, H., Gomez, C., Kattel, R., Korella, G., Leitner, Ch., Le Masson, B., Mazzucato, M., Mungiu‐Pippidi, A., Pröhl, M., Oravec, J. (2013) Powering European Public Sector Innovation: Towards a new architecture. Hargreaves, I., Guibault, L., Handke, Ch., Valcke, P., Martens, B., Lynch, R., Filippov, S. (2014), Standardisation in the area of innovation and technological development, notably in the field of Text and Data Mining. Reynolds, J., Sundström, M., Agàrdi, I., Amaral, J., Bourlakis, M., Hirner, S., Martinez‐Ribes, L., Moiter, C., Schwartz, D., Werkhoven, T. (2014), Final Report from the Expert Group on Retail Sector innovation. Führer, A., Kaldos, P., Miranda, D., Enevoldsen, S., Fahl, K., Ghafele, R., Maguire, J., Szczepanowska‐Kozłowska, K., Pyis, L., Ryan, D., Wurzer, A.J., Zambon, S. (2013), Final Report from the Expert Group on Intellectual Property Valuation. Peter, V., Van der Veen, G., Doranova, A.,Miedzinski, M. (2013), Screening of regulatory framework. Peter, V., Van der Veen, G., Doranova, A.,Miedzinski, M. (2014), Regulatory screening. A short guide on the innovation effects of regulation. Andrée, D. (2009), Priority‐setting in the European Research Framework Programmes. Pattinson, M., Chevillard, S.V., Gagliardi, D., Marzocchi, Ch., Rigby, J., Li, Y., Cox, D., Brighton, R., Avigdor, G., Lucas, R. (2015), Final Report of the Framework Contract “Provision of Services in the Field of Research Evaluation and Research Policy Analysis" Lot 3. International Cooperation
Karhunen, P., Meyer, U., Kadochnikov, P., Popov, V., (2013), Review of the S&T Cooperation Agreement between the European Union and Russia. Acheson, H., León, G. (2013), Evaluation of the EU‐US Agreement on S&T. Khan‐Malek, Ch., Windsor, M. (2013), Final Report. Review of the EU‐NZ Agreement on Science and Technology. Ruigrok, I., Kahn, M. (2014), Review of the Science and Technological (S&T) cooperation Agreement between the European Union and South Africa. Horvat, M., Lundin, N. (2008), Review of the Science and Technology Cooperation between the European Community and the Government of the People’s Republic of China. Bernaras, A., Zickler, A. (2012), Review of S&T Cooperation between the European Union and the Federative Republic of Brazil. Basile, E., Régnier, P. (2012), Review of the S&T cooperation Agreement between the European Union and the Government of the Republic of India 2007‐2011. Bobe, B., Crehan, P. (2013), A Review of the S&T Agreement between the European Union and the Republic of Korea. Clar, G., Schmidt‐Lainé, C. (2011), Review of S&T Cooperation between the European Union and the Republic of Argentina 2006‐2010. 20 years of Trade & Economic Cooperation Agreement. 10 years of S&T Cooperation Agreement. Bernaras, A., Clar, G. (2011), Review of S&T Cooperation between the European Union and the Republic of Chile 2007‐2011. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 93
Remotti, A., Damvakeraki, T., Plooder, M., Sterner, C. (2014), International Science and Technology Cooperation in the EU’s 7th Framework Programme: the specific programme ‘Cooperation’ and its thematic areas. Vullings, W., Arnold, E., Boekholt, P., Horvat, M., Mostert, B., Rijnders‐Nagle, M. (2014), European Added Value of EU Science, Technology and Innovation actions and EU‐Member State Partnership in international cooperation. Deloitte; the evaluation partnership (2014), Geographical expansion of the EURAXESS‐Links Network. Sociedade Portuguesa de Inovação; The United Nations University ‐ Maastricht Economic and Social Research; Training Centre on Innovation and Technology; Austrian Institute of Technology (2014), Science, Technology and Innovation (STI) Performance of China. Joint Research Center
European Commission (2012), Joint Research Centre. The European Commission's in‐house science service. Annual Report 2011. European Commission (2013), Joint Research Centre. The European Commission's in‐house science service. Annual Report 2012. European Commission (2013), Joint Research Centre. The European Commission's in‐house science service. Annual Report 2013. Impacts
Mahieu, B., Arnold, E., Potau, X., Hammerschmidt, R. (2012), Evaluation of Pertinence and Impact of Research Infrastructure Activity in FP7 ‐ EPIRIA. Jansen, W., Warmenhoven, B., Fikkers, D.J., Poel, M. (2014), Study on Assessing the Research Management Performance of Framework Programmes Projects. Arnold, E., Mahieu, B., Stroyan, J., Campbell, D., Carlberg, M., Giaracca, F., Horvath, A., Jávorka, Z., Knee, P., Meijer, I., Sidiqi, S., Wagner, C. (2011), Understanding the Long Term Impact of the Framework Programme. Veugelers, R., Testoni, M. (2014), Social impact policy analysis of technological innovation challenges (SIMPATIC) ‐ Interim policy messages and relevant figures. Excellence
Bennetot Pruvot, E., Estermann, T. (2015), Define Thematic Report: Funding for Excellence. Barré, R., Hollanders, H., Salter A. (2011), Indicators of Research Excellence. Gender
European Commission (2013), She Figures 2012 Gender in Research and Innovation. Statistics and Indicators. Simplification
Booz & Company; SpaceTec Partners (2011), Evaluation of output based funding for the space and security themes in a future EU Framework Programme for Research. Deloitte (2011), Assessing the Effectiveness of Simplification Measures under FP7. ERA
Arundel, A., Es‐Sadki, N., Barjak, F., Perrett, P., Samuel, O., Lilischkis, S. (2013), Knowledge Transfer Study 2010 – 2012. European Commission (2013), European Research Area. Facts and Figures 2013. Debackere, K., Andersen, B., Dvorak, I., Enkel, E., Krüger, P., Malmqvist, H., Plečkaitis, A., Rehn, A., Secall, S., Stevens,W., Vermeulen, E., Wellen, D. (2014), Boosting Open Innovation and Knowledge Transfer in the European Union. Campbell, D., Roberge, G., Haustein, S., Archambault, E. (2013), Intra‐European Cooperation Compared to International Collaboration of the ERA Countries. European Commission (2014), European Research Area. Progress Report 2014 accompanied by Facts and Figures 2014. Dinges, M., Bouttier, R., Schiffbänker, H., Holzinger,F., Van der Giessen, A., Lehenkari, J. ,Deschryvere, M., Kuittinen, H., Rammer, Ch. (2014), Analysis of the state of play of the European Research Area in Member States and Associated Countries: focus on priority areas. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 94
European Commission (2014), R&D Investments and Structural Changes in Sectors – Quantitative and Qualitative Analysis; Policy Recommendations. PPPs
European Commission (2013), Future Internet Public Private Partnership (FI‐PPP). Response to the Interim Assessment. Bozier,P., Chirita, R., Gargani, F., MacGray,D., Newman, L., Odoardi R., Rytel M.(2013), Ex‐post Evaluation of EPEC (The European Public‐Private Partnership (PPP) Expertise Centre). European Commission (2012), Interim Assessment of the Future Internet Public‐Private Partnership. European Commission (2013), Final assessment of the research PPPs in the European Economic Recovery Plan. Factories of the Future Energy‐efficient Buildings European Green Cars Initiative. EIP
Aho, E., Serger, S.S., Mönig, V., Wilson, P.,Garmendia,Ch., Steinberg, M., Swieboda, P. (2014), Outriders for European Competitiveness. European Innovation Partnerships (EIPs) as a Tool for Systemic Change. Coordination of Research
Acheson, H., Annerberg, R., Dammert, R., Klusacek, R., Kraus, W., Lock, J. (2012), Review of the Joint Programming Process. Locatelli, P., Donards, O., Kaarls, R., Kolar, J., Hunter, A. (2012), Interim Evaluation of the European Metrology Research Programme (EMRP). FP6
European Commission (2008), Ex‐post Evaluation Joint Research Centre Direct Actions in the 6th Framework Programmes 2002‐2006. Rietschel, E., Arnold, E., Čenys, A., Dearing, A., Feller,I., Joussaume,S., Kaloudis, A., Lange, L., Langer, J., Ley,V., Mustonen, R., Pooley, D., Stame,N. (2009), Evaluation of the 6th Framework Programme for Research and Technological Development 2002‐2006. RSFF
Avice, E., Stevens, W., Mann, E., Figueiredo, J‐F., Maimets, T., (2013), Second interim evaluation of the RSFF. Ballaguy, R.L., D'Anglade, C., Monti,C., Dooms, A., Reyns, L., Van Geleuken,U., D'aversa,F., Szenci,K., Kruse,D., Demetri,D., Kauffmann,A. (2013), Second Evaluation of the Risk Sharing Finance Facility (RSFF). Innovation Union
European Commission (2011), State of the Innovation Union 2011. European Commission (2014), Research and Innovation performance. Innovation Union progress at country level in the EU 2014. Ruhland, S., Romanow, B. (2013), Innovation. How to convert Research into Commercial Success Story? Part 1: Analysis of EU‐funded research projects in the field of industrial technologies. European Commission (2011), Innovation Union Competitiveness report. European Commission (2014), Innovation Union Competitiveness report. Other studies
Ulmer‐Tuffigo, H., Harrap, K., Petrauskaite, R., Szendrak, E., Warrington, B., Vayssilov, G. (2014), Final Review of FP7‐COST II Grant Agreement 2014. European Commission (2011), External Evaluation of the European Institute of Innovation and Technology. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 9.3.
95
List of references
Österreichischer Forschungs‐ und Technologiebericht / Austrian Research and Technology Report 2014 Buitendijk, S., Corda, D., Flodstrom, A., Holdcroft, A., Hunter, J. Pollitzer, E., Rees, T. Rice, C., Schiebinger, L., Schraudner, M., Sjorup, K. & Tarrach, R. (2011), Women in Science and Medicine, The Lancet, 377, 9768, p. 811. Cecere, G., Corrocher, N. (2015). The Intensity of Interregional Cooperation in Information and Communication Technology Projects: An Empirical Analysis of the Framework Programme, Regional Studies, 49, 2, pp. 204–
218. CONSIDER Project (2015), Civil Society Organisations in Designing Research Governance. Final Publishable Report. Dimitrova, A., Dimitrova, A., Hametner, M., Martinuzzi, A. (2015), FP7 and the SDGs ‐ How did research in the 7th EU framework programme address the Sustainable Development Goals (SDGs)?, Vienna: Institute for Managing Sustainability, Vienna University of Economics and Business. ECORYS (2012), FP7 Marie Curie Life‐long Training and Career Development Evaluation: Individual Fellowships and Co‐Funding Mechanism. Final Report to the European Commission. Framework Contract (EAC/50/2009). Ehardt‐Schmiederer, M.; Brücker, J.; Milovanović, D.; Postl, V.; Kobel, C.; Hackl, F.; Schleicher, L.; Antúnez, A. (2014): 7. EU‐Rahmenprogramm für Forschung, technologische Entwicklung und Demonstration (2007–
2013), PROVISO‐Bericht Frühjahr 2014, Wien 2014 European Commission (2006), Communication from the Commission on Implementing the renewed partnership for growth and jobs. Developing a knowledge flagship: the European Institute of Technology, COM(2006)0077 final. European Commission (2009), Toolkit: Gender in EU‐funded research. Yellow Window Management Consultants, Engender, Genderatwork. 2009. Brussels: European Commission. European Commission (2010), Communication from the Commission on Europe 2020 Flagship Initiative Innovation Union. SEC(2010) 1161. European Commission (2011), Communication from the Commission on the Response to the Report of the Expert Group on the Interim Evaluation of the Seventh Framework Programme for Research, Technological Development and Demonstration Activities and to the Report of the Expert Group on the Interim Evaluation of the Risk‐Sharing Finance Facility. COM(2011) 52 final. European Commission (2012), Communication from the Commission an A reinforced European Research Area Partnership for Excellence and Growth, COM(2012) 392 final. European Commission (2012), Communication from the Commission on Enhancing and focusing EU international cooperation in research and innovation: A strategic approach. COM(2012) 497 final. European Commission (2012), Report on Structural Change in research institutions: Enhancing Excellence, Gender Equality and Efficiency in Research and Innovation. Report of the Expert Group on Structural Change. European Commission (2012), Communication from the Commission on Towards better access to scientific information: Boosting the benefits of public investments in research. COM(2012) 401 final. European Commission (2013), Investing in European success: HORIZON 2020, Research and innovation to boost growth and jobs in Europe. European Commission (2015), Communication from the Commission on A Global Partnership for Poverty Eradication and Sustainable Development after 2015. COM(2015) 44 final. European Commission (2015), Communication from the Commission on Better regulation for better results ‐ An EU agenda. COM(2015) 215 final. European Parliament (2004), European Parliament resolution on science and technology “Guidelines for future European Union policy to support research” (2004/2150(INI)). Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 96
European Parliament (2006) Decision No 1982/2006/EC of the European Parliament and of the Council of 18 December 2006 concerning the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007‐2013). European Parliament (2013), Regulation No 1291/2013 of the European Parliament and of the Council of 11 December 2013 establishing Horizon 2020 ‐ the Framework Programme for Research and Innovation (2014‐
2020) and repealing Decision No 1982/2006/EC; Regulation (EU) 2015/1017 of the European Parliament and of the Council of 25 June 2015 on the European Fund for Strategic Investments, the European Investment Advisory Hub and the European Investment Project Portal and amending Regulations (EU) No 1291/2013 and (EU) No 1316/2013 — the European Fund for Strategic Investments. European Research Council (2014), On the way to the top: providing equal opportunities for men and women in science and technology. Working Group on Gender Balance, final summary, 2 December 2013. European Union Council (2002), Decision 2002/835/EC: Council decision of 30 September 2002 adopting a specific programme for research, technological development and demonstration: "structuring the European Research Area" (2002–2006). European Union Council (2015), Investing in European projects: Council and Parliament agree rules on European fund for strategic investments. Press Release. Fougeyrollas, A., Le Mouël, P., Zagamé, P., (2012), Consequences of the 2012 FP7 call for proposals for the economy and employment in the European Union. Gago, J.M. (2004) Europe Needs More Scientists: Contribution High Level Group on Human Resources in Science and Technology to the EC Conference: Increasing Human Resources for Science and Technology. Brussels 2 April 2004. Holdcroft, A., Snidvongs, S., & Berkley K.J., (2011), Incorporating gender and sex dimension in medical research, Interdisciplinary Science Reviews, 36,2, pp. 180‐190. Owen, R., Heintz, M., Bessant, J. (Eds.), (2013), Responsible Innovation. Wiley. Rees, T. (2011), The Gendered Construction of Scientific Excellence, Interdisciplinary Science Reviews, Special Issue on Gender in Science, 36, 2, pp. 133–45. United Nations (2015), Zero draft of the outcome document for the UN Summit to adopt the Post‐2015 Development Agenda: Transforming Our World by 2030: A New Agenda for Global Action. Van der Hoven, J., Jacob, K., Nielsen, L., Roure, F., Rudze, L., Jack, S., Blind, K., Guske, A., Martinez Riera, C., (2013), Options for strengthening responsible research and innovation. Report of the Expert Group on the State of Art in Europe on responsible research and Innovation. Wild, A. (2014), ECSEL Joint Undertaking annual activity report, 2014. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 9.4.
97
Experts consulted in preparation of this report
More than 50 experts from the European Commission, EU Member States, stakeholder umbrella organisations and National Contact Points were consulted during the preparation of this report. They provided valuable insights during dedicated thematic hearings in one of the meetings or were consulted individually by one or more Members of the High Level Expert Group. Most extensive consultations were conducted with the following experts. Name Surname Position Organisation The Technology Centre of the Academy of Sciences of the Czech Republic Permanent Representation of Bulgaria to the EU Vladimír Albrecht DR&Ieputy Director Karina Angelieva First Secretary Muriel Attané Secretary General Tomaž Boh Head of Unit Martin Bohle European Commission DG R&I Unit J.5 Lidia Borrell‐Damian Head of Unit Director for Research and Innovation Jean Pierre Bourguignon President European Research Council Marco Brusati Fabienne Corvers Policy Officer European Commission DG R&I Unit A.5 Amanda Crowfoot Director Science Europe Kristin Danielsen International Director The Research Council of Norway Bert de Colvenaer Fuel Cells and Hydrogen JU Stijn Delaure Daniel Deybe Executive Director Head of unit, International Research Policy Policy Officer European Commission DG R&I Unit A.5 Andrea Erdei Policy Officer European Commission DG R&I Unit A.5 Jean‐Emmanuel Faure European Commission DG R&I Unit E.2 Peter Fisch Policy Officer Former Head of Unit "Evaluation and Monitoring of the Framework Programme" at DG R&I Dr. Jens‐Peter Gaul Director Paul Geurts Sr. Advisor International Michel Goldman Former Executive Director Innovative Medicines JU Martien Groenen Professor Animal Breeding and Genetics Wageningen University Michel Hordies Deputy Head of Unit European Commission DG CNECT Unit A.4 Brendan Howdon European Commission DG R&I Toshiyasu Ichioka Peter Jongebloed Head of Unit (At the time)Project Manager, (Now) Senior STI Cooperation Strategist Account Manager Europe Rein Kaarli European Association of Research and Technology Organisations (EARTO) Republic of Slovenia Ministry of Education, Science and Sport EUA (European University Association) CLEANSKY Joint Undertaking Adviser League of European Research Universities (LERU) (Formerly) European Commission DG R&I European Liaison Office of the German Research Organisations Wageningen University (At the time) EU‐Japan Centre for Industrial Cooperation, (Now) Japan Science and Technology Agency Wageningen University Republic of Estonia Ministry of Education and Research Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 98
Eeva Kaunismaa Policy Officer European Commission DG R&I Unit A.5 Per Magnus Kommandantvold Special Adviser The Research Council of Norway Sika Lauma Counsellor Permanent Representation of Latvia to the EU Terezia Lesayova Research Advisor Slovak Liaison Office for Research and Development Petros Macridis European Commission DG R&I Unit J.5 Iva Medvešek Statistical Assistant National NCP coordinator Jack Metthey Director European Commission DG R&I Directorate A Roxana Moisii Second Secretary Permanent Representation of Romania to the EU Tom‐Espen Møller Senior Adviser The Research Council of Norway Sergej Možina Counsellor for Research Joerg Niehoff Head of Sector Seán O'Reagain European Commission DG R&I Unit A.5 Roberta Pattono Deputy Head Of Unit B7 – Science with and for society Roberta Pattono Project Officer European Commission DG R&I Unit B.7 Neville Reeve Policy Officer European Commission DG R&I Unit A.3 Steve Ringer Reka Rozsavolgyi Head of EU Framework Programme Management Policy Officer Sebastian Serwiak Deputy Director Dr Rachel Smith Robert‐Jan Sverre Ministry of Science, Education and Sports Permanent Representation of the Republic of Slovenia to the EU European Commission DG R&I Unit B.2 European Commission, DG R&I UK Department of Business Information and Skills European Commission DG R&I Unit A.5 The National Contact Point for Framework Programme (Poland) Smits Partnership Funding Manager Director General European Commission DG R&I Sogge Special Adviser The Research Council of Norway Tobias Bade Strøm Special Adviser The Research Council of Norway Isabelle Suja Goffin Policy Officer European Commission DG R&I Unit R.4 László Szilágyi First Secretary Permanent Representation of Hungary to the EU Mika Tuuliainen Jan van den Biesen Ernst van den Ende Rosalinde van der Vlies Robert van Gorcom Philippe Vannson Head of Unit European Commission DG R&I Unit G.2 Jurate Vaznelyte Policy Officer European Commission DG R&I Unit A.5 Dr. Rolf. J. Vermeij Liaison Officer University of Twente Simon Vink Spokesperson Executive Board Wageningen University Head of Educational Affairs, The Federation of Finnish Enterprises Vice President Public R&D Programs Philips Research Director Plant Sciences Group Head of Unit Director RIKILT Food Safety Institute ITM Power European SMEs Representative Association (UEAPME ) representing BUSINESSEUROPE Wageningen University European Commission DG R&I Unit A.5 Wageningen University Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 9.5.
Abbreviations
ARTEMIS BRIC CIP COFUND CORDIS CSO DG DG R&I EC eCORDA EIT Embedded Computing Systems Joint Technology Initiative Brazil, Russia, India, China and South Africa Competitiveness and Innovation Framework Programme Marie Curie Co‐funding of Regional, National and International Programmes Community Research and Development Information System Civil Society Organisations Directorate General Directorate General for Research & Innovation European Commission External Common Research Data Warehouse European Institute of Innovation and Technology ENIAC ERA ERC ESFRI ETP EU EU SDS EU‐13 EU‐15 FCH FET FP FP5 FP6 FP7 GBWG GDP GERD HES IAPP ICPC ICT IDP IED IF IMI IRSES ISC ITN JRC JTI JU KBBE KET Nanoelectronics Technologies 2020 Joint Technology Initiative European Research Area European Research Council European Strategy Forum on Research Infrastructures European Technology Platforms European Union European Sustainable Development Strategy EU Member States who joined after 2004 EU Member States who joined before 2004 Hydrogen and Fuel Cells Initiative Future and Emerging Actions Framework Programme for Research and Innovation 5th Framework Programme for Research and Innovation 6th Framework Programme for Research and Innovation 7th Framework Programme for Research and Innovation Gender Balance Working Group Gross Domestic Product Global Effort in Research and Development Higher or Secondary Education Establishments Industry‐Academia Partnerships and Pathways International Cooperation Partner Countries Information and Communication Technologies Innovative Doctoral Programme European Industrial Doctorates Individual Fellows Innovative Medicines Initiative, Joint Technology Initiative International Research Staff Exchange Inter‐Service Consultation Initial Training Networks Joint Research Centre Joint Technology Initiative Joint Undertaking European knowledge‐based bio‐economy Key Enabling Technology 99
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme KIC MCA NCP NEST NMP openAIRE PIC PPPs PRC PUB PV QIPC REC RoK RRI RSFF RTD SDGs SESAM SiS SME SRAs SSH STI USA WP Knowledge and Innovation Community Marie Curie Actions National Contact Point New and emerging science and technology Nano‐sciences, Nano‐technologies, Materials and new Production Technologies FP7 project aimed to support the implementation of the EC and ERC Open Access policies Participant Identification Code Public‐Private Partnerships private‐for profit organisations Public Authorities Photovoltaics Quantum Information Foundations and Technologies Research Organisation Regions of Knowledge Responsible Research and Innovation Risk Sharing Finance Facility Research and Technological Development Sustainable Development Goals European Commission online reporting tool for Research and Technological projects Science in Society Small and Medium‐sized Enterprise Strategic Research Agendas Socio‐Economic Sciences and the Humanities Science, Technology and Innovation United States of America Work Programme 100
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 9.6.
101
Twelve myths about FP7
There are a number of “urban legends” about the Framework Programmes in general, and FP7 in particular. The following discussion sheds light on the twelve most fundamental ones that are repeatedly expressed in research and innovation communities: Myth Facts “FP7 is only suitable for large industry organisations and big universities” Due to their size, large industry organisations, as well as big universities and research organisations, are able to implement a number of projects simultaneously. Yet, FP7 was also able to remain an open and attractive programme for more than 12.400 SMEs, smaller research organisations and CSOs. Collaborative research consortia and networks were successful in initiating interaction and collaboration between organisations. “FP7 is only about science and technology; there are no funding opportunities for other scientific disciplines” It is a fact that a large share of themes in FP7‐COOPERATION was focused on science and technology. The sub‐programme SSH was rather small in comparison, with only about 1% of the overall budget. However, FP7 provided a range of additional funding opportunities for social and economic sciences and the humanities: in FP7‐IDEAS, about 15% of ERC grants were awarded to these disciplines; in FP7‐PEOPLE there were no disciplinary restrictions; and in FP7‐
CAPACITIES societally relevant themes were the main focus of the sub‐programme “Science in Society”. In HORIZON 2020, the role of social and economic sciences and the humanities was strengthened and implemented as a cross‐cutting issue across all thematic areas. “For a successful project proposal, it is necessary to include partners from almost all EU countries” Fostering collaboration between EU Member States and thus supporting the development of the European Research Area was an explicit goal of FP7. However, this principle should not be misinterpreted as a quota system or the conditionality of funding aimed at including specific EU countries in a research proposal. “In order to prepare a good project proposal, it is necessary to hire consultants” A successful FP7 proposal required scientific qualifications, an excellent idea and compliance with a particular proposal structure. Organisations with prior experience in developing Framework Programme proposals were able to apply their experiences to subsequent proposals, while inexperienced organisations may have made use of consultants to get acquainted with Framework Programme procedures. Engaging consultants was, however, neither a requirement nor did it constitute a competitive advantage. Review and evaluation of FP7 proposals was carried out by independent review “For winning a grant, it panels made up of top experts from all over Europe. This process ensured sound is important to have and objective evaluation and made lobbying impossible. What was subject to good lobbying influence from interest groups and lobbying, however, was the development of contacts” work programmes and individual topics. “FP7 proposals are not worth the effort, due to very low success rates” Success rates are not informative on their own, as they do not carry information about the actual competition for funding. A comparison of ERC grants and FP7‐
COOPERATION shows that competition depended on funding conditions as well as on the number of proposals. Funding conditions for ERC grants were less strict than those for FP7‐COOPERATION (e.g. no need for project consortia, independent selection of themes, fewer restraints in the application of funds); thus leading to a high number of proposals under the minimum threshold for ERC grants. Competition among proposals of high quality, however, was not any fiercer than in other specific programmes. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 102
“FP7 was biased against the New Member States (EU‐13)” The lower participation shares of EU‐13 is caused not by a bias against the new EU Member States, but rather by a comparably high number of weak proposals submitted by, or with partners from the EU‐13. Some of most important reasons are information and language barriers; lack of professional contacts and research networks; lack of leading Universities and research organisations; insufficient motivation to participate in FP7; generally low focus on R&D in policy and in business that is more prevalent in many EU‐13 countries. “FP7 projects involve so much administrative work that there is no time to actually do research” FP7 projects were generally large‐scale and received considerable amounts of public funding. Therefore, they were required to report on work packages and milestones, as well as to submit cost and time records. These are standard procedures in tendered projects. In the course of simplifying procedures for the second half of FP7 and HORIZON 2020, administrative burdens have been reduced to some extent. “FP7 projects are inflexible and can never be adjusted in the light of novel insights” FP7‐COOPERATION was set up according to the logic of public procurement projects. Work packages, tasks and milestones had to already be described in detail in the project proposal. While this procedure is somewhat contradictory to a creative research and learning process and thus is constrained by such a rigid work plan, the right wording of tasks and possible amendments to the project contract permitted a certain amount of flexibility. FP7‐IDEAS allowed for even more independence and flexibility. Until today, more than 165.000 scientific publications have been published acknowledging funding from FP7. This amounts to four publications per funded “FP7 projects are so project on average (with highest output per project in the Health and Environment work intensive that themes in FP7‐COOPERATION). 650 publications in the leading journals ‘Science’ they fail to produce and ‘Nature’ acknowledged funding from FP7. In the end, the project design scientific publications” determined if the focus was set on research (i.e. publications), innovation (i.e. patents) or impact (i.e. policy advice). “FP7 projects don’t allow researchers to conduct independent, sound and excellent scientific work as political targets and application orientation dominate” The specific programmes of FP7 followed a different orientation: FP7‐IDEAS addressed fundamental research, funded excellent researchers at leading universities worldwide and had a significant impact on scientific excellence. FP7‐
COOPERATION addressed applied research and innovation, involved the business sector as well and aimed for economic and societal impact. FP7‐CAPACITIES addressed policy objectives, involved civil society organisations and provided evidence for policy makers. In all areas scientific excellence was a key criterion for the selection of proposals. “FP7 does not fund the most innovative ideas but rather well established researchers” All FP7 proposals were evaluated according to their scientific and/or technological excellence. In addition, in some parts of FP7 the track record of the individual scientist was also considered in the review of proposals, especially in parts where the funding was addressing individuals (e.g. in FP7‐IDEAS and FP7‐PEOPLE). With the ERC starting grants and the Marie Curie Actions substantial support was provided to high potential early stage researchers. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 9.7.
103
Institutional setting of FP7
FP7 as the main instrument to implement the ERA is governed by a variety of institutions that are responsible for programme implementation and funding and others that are involved in a more consultative manner. The major institutions are summarized below: The Directorate‐General for Research and Innovation (DG R&I) in coordination with other concerned services (e.g. DG Connect, DG Energy, DG Transport, DG Agriculture) defines and implements European Research and Innovation policy and funds research through the Framework Programmes. The European Parliament’s ITRE Committee’s areas of competence cover research, industry and energy. The Competitiveness Council (COMPET) is one of ten configurations of the Council of the European Union. It deals with four major policy areas, one of which is research and innovation. The Research Executive Academy (REA) helps the European Commission manage EU programmes more efficiently. It has been fully operational since mid‐2009 and manages large parts of FP7. The National Contact Points (NCPs) are national structures established and financed by governments of the EU Member States and the states associated to FPs. They are the main structure providing guidance, practical information and assistance to researchers on all aspects of participation in the Framework Programmes. This includes advice on technical and administrative questions concerning the calls for proposals, partner search, national priorities, and matching national co‐financing possibilities, where applicable. External Advisory Groups: To draw up the work programmes, the Commission seeks and uses the best possible external advice. A series of Advisory Groups were established by the Commission in order to receive high‐level, independent and pluralistic advice. Programme Committees: The Commission services are assisted by Programme Committees. Under FP7 the Council created the following Programme Committees to assist the Commission in implementing the FP7 Specific Programmes: Cooperation, Ideas, People, Capacities. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 104
The European Research Council (ERC) is an implementing agency, which supports investigator driven 'frontier' research through competitive funding of individual researchers and implements specific programme FP7‐IDEAS. The ERC Scientific Council (ERC CsC) is the decision‐making body of the ERC and sets the ERC’s scientific funding strategy. It is composed of eminent scientists and scholars, who are appointed by the European Commission. The ERC Executive Agency (ERCEA) was established to manage FP7‐IDEAS. It implements the ERC strategy as set by the Scientific Council and is in charge of the day to day grant administration. The European Strategy Forum on Research Infrastructures (ESFRI) supports policy‐making on research infrastructures in Europe, and aims to facilitate multilateral initiatives leading to the better use and development of research infrastructures, at EU and international level. ESFRI's delegates are nominated by the Research Ministers of the Member and Associate Countries, and include a European Commission representative. The European Research Area Board (ERAB)110 is a consultative body advising the European Commission on development, promotion and evaluation policy initiatives and actions to meet the goals of the European Research Area. It consists of 22 high level experts from academia, business and industry. The European Research Area Committee (ERAC, “CREST” until 2010) is a strategic policy advisory body assisting the Council, the EC and EU Member States on research and innovation issues that are relevant to the development of the European Research Area. The ERAC can also meet in two additional configurations: The High Level Group for Joint Programming (ERAC GPC), which which contributes to debates and decisions of the Competitiveness Council on joint programming, and the Strategic Forum for International Science and Technology Cooperation (ERAC SFIC), which advises on issues in the area of international scientific and technological cooperation. The European Science Foundation (ESF) is an association of 75 member organizations in 30 European countries. It provides influence and advice on policy, promotes cooperation in basic research, and supports highest quality science in Europe to drive progress in research and innovation. Under FP7, ESF acted as managing agent for COST. It received funding from FP7, which was channelled to COST. The European Cooperation in Science and Technology (COST) is an intergovernmental network anticipating and complementing the activities of the Framework Programmes. It constitutes the connection with the scientific communities of emerging countries, aims to increase mobility of researchers across Europe and supports the establishment of “Networks of Excellence” in nine key scientific domains. EUREKA is an intergovernmental network launched in 1985, to support market‐oriented R&D and innovation projects by industry, research centres and universities across all technological sectors. It is composed of 41 member states, including the European Union represented by the Commission and three associated states – Canada, South Africa and South Korea. EUREKA offers project partners rapid access to skills and expertise across Europe and national public and private funding schemes. Independent associations provide advice and perform a consultative role regarding implementation of FP7. 110
today ERIAB; ERAB ran from 2007 until 2012 Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 9.8.
105
Who benefited from FP7 (by type of organization)
EC contribution (in million euro)
Theme 01 ‐ Health
Theme 02 ‐ KBBE
Theme 03 ‐ ICT
Theme 04 ‐ NMP
Theme 05 ‐ Energy
Theme 06 ‐ Environment
Theme 07 ‐ Transport
Theme 08 ‐ SSH
Theme 09 ‐ Space
Theme 10 ‐ Security
ERANET
JTI
Subtotal FP7‐COOPERATION
Total
4.792
1.851
7.877
3.239
1.707
1.719
2.284
580
713
1.295
313
1.966
28.336
% of EU contrib
small private companies
(SMEs)
844
277
1.064
709
270
215
343
19
97
268
0
327
4.434
Public Authorities (PUB)
134
74
139
42
95
80
100
26
37
75
14
30
847
other (incl. CSOs)
59
39
93
44
38
29
58
10
4
24
251
15
662
Univ.
(HES)
2.451
759
3.044
1.026
355
622
436
366
139
262
1
521
9.983
RTD Org.
(REC)
1.157
626
1.967
937
416
714
569
156
336
352
39
451
7.722
Large private companies (PRC)
147
76
1.570
480
533
59
778
3
100
314
8
621
4.689
35%
27%
17%
16%
3%
2%
FP7‐IDEAS
ERC Starting Grants
ERC Advanced Grants
ERC other activities
Subtotal FP7‐IDEAS
% of EU contrib
3.115
3.708
851
7.673
2.283
2.746
584
5.613
73%
798
909
256
1.963
26%
26
41
9
76
1%
0
3
1
4
0%
7
7
0
14
0%
0
2
2
4
0%
FP7‐PEOPLE
Initial Training
Career Development
Industry Academia Partnerships
World Fellowships
other activities of FP7‐PEOPLE
Subtotal FP7‐PEOPLE
% of EU contrib
2.175
1.482
415
665
40
4.777
1.440
859
174
497
12
2.982
62%
444
392
42
161
10
1.048
22%
150
18
67
3
1
240
5%
129
5
127
1
2
263
6%
10
150
3
2
6
172
4%
3
59
2
0
9
73
2%
FP7‐CAPACITIES
Res. Infrastructure
Res. for the benefit of SMEs
Regions of Knowledge
Res.Pot. of Conv. Regions
Science in Society
Coherent dev. of res. policies
International cooperation
Subtotal FP7‐CAPACITIES
% of EU contrib
1.528
1.249
127
378
288
28
173
3.772
440
19
22
212
140
1
27
861
23%
812
51
20
158
57
18
82
1.198
32%
39
40
14
0
8
0
6
108
3%
126
1.006
18
0
17
1
12
1.181
31%
52
16
20
2
15
6
35
146
4%
59
117
33
5
52
1
12
279
7%
Grand total
% of EU contrib
44.559 19.439 11.931 5.111 5.882 1.179 1.018
44%
27%
11%
13%
3%
2% Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 106 Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 9.9.
Success rates and success factors of FP7 sub-programmes
number of proposals per specific programme / sub-programme
requested EC-contribution per specific programme / sub-programme
107
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme FP7
108
number of proposals
proposals below threshold that did not receive EU funding
40.000
proposals above threshold that did not receive EU funding
30.000
proposals that received EU funding
20.000
10.000
0
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
10.000
20.000
30.000
FP7
EU funding (mio €)
50.000
proposals below threshold that did not receive EU funding
40.000
proposals above threshold that did not receive EU funding
proposals that received EU funding
30.000
20.000
10.000
0
10.000
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
20.000
30.000
40.000
50.000
60.000
In interpreting these results it should be considered, that 
thresholds were set for different criteria (e.g. scientific excellence, management, impact)  therefore the total score of a proposal is not the only criterion for being funded and some proposals with a comparable high total score might have missed the threshold for one criterion 
the total budget is allocated to different specific programmes, themes, areas and years  therefore in some years proposals with a lower score might get funding, while in other years proposals with a higher score might not get any funding. 
some parts of FP7 follow a top‐down approach publishing rather specific calls, while other specific programmes follow a bottom‐up approach (such as FP7‐IDEAS and FP7‐PEOPLE)  in areas where well defined topics are published the competition is lower and proposals with a lower score might be successful as well Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme FP7
number of proposals
coord by org. from EU‐15
109 FP7
number of proposals
coord by org. from EU‐13
30.000
proposals below threshold that did not receive EU funding
1.500
proposals below threshold that did not receive EU funding
proposals above threshold that did not receive EU funding
proposals above threshold that did not receive EU funding
20.000
proposals that received EU funding
1.000
proposals that received EU funding
10.000
500
0
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
0
non eligibe or
withdrawn
> 95% of
points
10.000
500
20.000
1.000
30.000
1.500
FP7
EU funding (mio €)
coord by org. from EU‐15
90‐95%
80‐90%
FP7
70‐80%
60‐70%
50‐60%
30‐40%
20‐30%
EU funding (mio €)
40.000
10‐20%
<10% of
points
non eligibe or
withdrawn
coord by org. from EU‐13
proposals below threshold that did not receive EU funding
1.500
proposals below threshold that did not receive EU funding
30.000
proposals above threshold that did not receive EU funding
proposals above threshold that did not receive EU funding
proposals that received EU funding
1.000
proposals that received EU funding
20.000
500
10.000
0
10.000
40‐50%
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
0
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
500
20.000
1.000
30.000
40.000
50.000
~ 113.700 proposals (205 billion euro), 48% below threshold, 37% adj. success rate
1.500
2.000
~ 7.700 proposals (11 billion euro), 66% below threshold, 34% adjusted success rate
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme FP7‐COOPERATION (incl. JTI)
number of proposals
coord by org. from EU‐15
110 FP7‐COOPERATION (incl. JTI)
number of proposals
coord by org. from EU‐13
8.000
proposals below threshold that did not receive EU funding
300
proposals below threshold that did not receive EU funding
proposals above threshold that did not receive EU funding
6.000
200
proposals above threshold that did not receive EU funding
proposals that received EU funding
proposals that received EU funding
4.000
100
2.000
0
> 95% of
points
0
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
2.000
200
4.000
300
6.000
400
8.000
500
FP7‐COOPERATION (incl. JTI)
EU funding (mio €)
coord by org. from EU‐15
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
100
FP7‐COOPERATION (incl. JTI)
EU funding (mio €)
25.000
coord by org. from EU‐13
proposals below threshold that did not receive EU funding
800
proposals below threshold that did not receive EU funding
20.000
proposals above threshold that did not receive EU funding
proposals above threshold that did not receive EU funding
proposals that received EU funding
600
proposals that received EU funding
15.000
400
10.000
200
5.000
0
0
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
5.000
10.000
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
400
600
15.000
800
20.000
1.000
~ 35.000 proposals (115 billion euro), 49% below threshold, 41% adjusted success rate
200
~ 1.900 proposals (4,6 billion euro) 73% below threshold, 30% adjusted success rate Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme FP7‐IDEAS
number of proposals
coord by org. from EU‐15
111 FP7‐IDEAS
number of proposals
coord by org. from EU‐13
4.000
100
2.000
0
> 95% of
points
100
0
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
200
2.000
300
4.000
400
500
6.000
8.000
proposals below threshold that did not receive EU funding
600
proposals above threshold that did not receive EU funding
700
proposals below threshold that did not receive EU funding
proposals that received EU funding
10.000
proposals above threshold that did not receive EU funding
proposals that received EU funding
800
12.000
900
FP7‐IDEAS
EU funding (mio €)
coord by org. from EU‐15
FP7‐IDEAS
EU funding (mio €)
coord by org. from EU‐13
10.000
200
5.000
0
> 95% of
points
0
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
200
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
400
5.000
600
10.000
800
15.000
proposals below threshold that did not receive EU funding
20.000
1.000
proposals above threshold that did not receive EU funding
1.200
proposals that received EU funding
25.000
~ 30.000 proposals (57 billion euro), 73% below threshold, 48% adjusted success rate
proposals below threshold that did not receive EU funding
proposals above threshold that did not receive EU funding
proposals that received EU funding
1.400
~ 2.000 proposals (3 billion euro), 91% below threshold, 45% adjusted success rate
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme FP7‐PEOPLE
number of proposals
coord by org. from EU‐15
112 FP7‐PEOPLE
number of proposals
coord by org. from EU‐13
800
20.000
proposals below threshold that did not receive EU funding
proposals above threshold that did not receive EU funding
15.000
proposals below threshold that did not receive EU funding
600
proposals above threshold that did not receive EU funding
proposals that received EU funding
proposals that received EU funding
400
10.000
200
5.000
0
> 95% of
points
0
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
5.000
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
200
400
10.000
600
FP7‐PEOPLE
EU funding (mio €)
coord by org. from EU‐15
FP7‐PEOPLE
EU funding (mio €)
coord by org. from EU‐13
300
10.000
8.000
proposals below threshold that did not receive EU funding
250
proposals above threshold that did not receive EU funding
200
proposals below threshold that did not receive EU funding
proposals above threshold that did not receive EU funding
proposals that received EU funding
proposals that received EU funding
150
6.000
100
4.000
50
0
2.000
50
0
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
100
150
2.000
200
4.000
~ 42.000 proposals (22 billion euro), 30% below threshold, 30% adjusted success rate
250
~ 2.300 proposals (1 billion euro) 44% below threshold, 36% adjusted success rate
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme FP7‐CAPACITIES
number of proposals
coord by org. from EU‐15
1.500
113 FP7‐CAPACITIES
number of proposals
coord by org. from EU‐13
250
proposals below threshold that did not receive EU funding
1.000
proposals above threshold that did not receive EU funding
proposals below threshold that did not receive EU funding
200
proposals above threshold that did not receive EU funding
150
proposals that received EU funding
proposals that received EU funding
100
500
50
0
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
0
> 95% of
points
50
500
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
100
1.000
150
200
1.500
250
2.000
300
FP7‐CAPACITIES
EU funding (mio €)
coord by org. from EU‐15
3.000
FP7‐CAPACITIES
EU funding (mio €)
coord by org. from EU‐13
500
2.500
proposals below threshold that did not receive EU funding
2.000
proposals above threshold that did not receive EU funding
proposals below threshold that did not receive EU funding
400
proposals above threshold that did not receive EU funding
proposals that received EU funding
300
proposals that received EU funding
1.500
200
1.000
100
500
0
500
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
0
> 95% of
points
90‐95%
80‐90%
70‐80%
60‐70%
50‐60%
40‐50%
30‐40%
20‐30%
10‐20%
<10% of
points
non eligibe or
withdrawn
100
1.000
200
1.500
2.000
2.500
~ 7.300 proposals (12 billion euro), 53% below threshold, 46% adjusted success rate
300
400
~ 1.500 proposals (2,3 billion euro), 57% below threshold, 14% adjusted success rate
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 114 9.10. Gender analyses
Source: data provided by the DG R&I Gender Unit based on CORDA, SESAM and RESPIR. Only projects included which submitted their final report until 26/03/2015. A Processed Final Report is one that has been submitted via SESAM, the corresponding assessment is signed and registered by the project officer in SESAM and final payment is available or the financial officer has finalized the calculation of the final payment. For the theme ICT in FP7‐COOPERATION no data was available. COOPERATION Specific Programme / Sub‐Programme CAPACITIES Total n. of reported worforce
% Females Health 430 26.735
52% KBBE 207 14.124
47% NMP 375 25.386
33% Energy 111 6.478
30% Environment 243 16.716
41% Transport 304 17.311
24% SSH 134 7.170
51% Space 124 6.580
30% Security 91 4.804
31% General Activities 15 1.263
32% Joint Technology Initiatives 135 1.674
21% 2.169 128.241
39% 104 500 48 114 97 16 68 11.877
17.223
1.732
3.991
3.716
180
2.201
29% 31% 44% 43% 54% 40% 47% 947 40.920
35% Total : COOPERATION Res. Infrastructures Res. for the benefit of SMEs Regions of Knowledge Res. Pot of Convergent Regions Science in Society Coherent dev. of res. policies International Cooperation Total : CAPACITIES Projects with a Processed Final Report (No.) Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 115 Percentage of women in each type of reported workforce COOPERATION Specific programme / Sub‐Programme Health KBBE NMP Energy Environment Transport SSH Space Security General Activities Joint Technology Initiatives Total : COOPERATION Res. Infrastructures CAPACITIES Res. for the benefit of SMEs Regions of Knowledge Res. Pot of Convergent Regions Science in Society Coherent dev. of res. policies International Cooperation Total : CAPACITIES Scientific % manager Females 1.135
32% 397
36% 1.076
28% 299
23% 517
33% 972
24% 290
43% 345
26% 233
26% 79
33% 193
18% 5.536
29% 360
25% 1.626
158
169
256
11
201
2.781
28% 40% 27% 50% 18% 45% 31% Experienced researcher
10.533
6.169
9.461
2.919
7.772
5.848
2.812
2.683
1.575
615
485
50.872
6.191
4.530
429
1.856
1.248
36
631
14.921
% Females 48% 43% 29% 28% 36% 22% 45% 27% 31% 27% 20% 36% 24% 30% 36% 40% 53% 33% 36% 31% PhD % student Females 3.988
56% 1.812
57% 3.201
37% 694
39% 2.069
51% 1.433
24% 1.086
56% 621
28% 356
37% 168
42% 125
18% 15.553
46% 1.351
31% 965
88
679
365
5
240
3.693
33% 43% 50% 61% 40% 54% 40% WP leader
3.156
1.488
3.049
764
1.946
2.289
953
1.028
663
229
328
15.893
1.046
% Females 31% 33% 27% 25% 33% 23% 39% 27% 23% 34% 22% 29% 24% 3.880
322
567
716
53
393
6.977
25% 43% 33% 49% 38% 48% 30% Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 116 Gender Aspects reported in the FP7 Projects by Priority Area COOPERATION Specific Programme / Sub‐Programme Health KBBE NMP Energy Environment Transport SSH Space Security General Activities Joint Technology Initiatives Total : COOPERATION Res. Infrastructures CAPACITIES Res. for the benefit of SMEs Regions of Knowledge Res. Pot of Convergent Regions Science in Society Coherent dev. of res. policies International Cooperation Total : CAPACITIES No. of projects with specific Gender Equality Actions No. % 144 34% 62 30% 116 31% 20 18% 70 29% 42 14% 41 31% 30 24% 15 16% 4 27% 29 21% 573 28 77 3 37 28 2 16 191 27% 27% 15% 6% 32% 29% 13% 24% 20% Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 117 Reported Gender Action Types COOPERATION Specific Programme / Sub‐Programme Health KBBE NMP Energy Environment Transport SSH Space Security General Activities Joint Technology Initiatives CAPACITIES Total : COOPERATION Res. Infrastructures Res. for the benefit of SMEs Regions of Knowledge Res. Pot of Convergent Regions Science in Society Coherent dev. of res. policies International Cooperation Total : CAPACITIES Design and implement an equal opportunity policy Effective % 129 30% 47 23% 90 24% 12 11% 44 18% 25 8% 27 20% 19 15% 14 15% 3 20% 21 16% 431 20% 20 19% 65 13% 2 4% 35 31% 20 21% 8 150 12% 16% Set targets to achieve a gender balance in the workforce Effective % 106 25% 41 20% 64 17% 9 8% 39 16% 28 9% 34 25% 15 12% 11 12% 3 20% 19 14% 369 17% 18 17% 48 10% 5 10% 32 28% 20 21% 1 6% 12 18% 136 14% Organise conferences Actions to improve and workshops on gender work‐life balance Effective % Effective
% 25 6% 84 19% 12 6% 36 17% 13 3% 64 17% 1 1% 11 10% 6 2% 35 14% 7 2% 22 7% 16 12% 27 20% 3 2% 12 10% 1 1% 8 9% 1 7% 3 20% 3 2% 21 16% 88 4% 323 15% 5 5% 12 12% 13 3% 62 12% 3 6% 4 4% 23 20% 15 15% 13 13% 3 40 4% 4% 8 121 12% 13% Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 118
Percentages of women by priority area and type of position (workforce), ranked from the highest to the lowest general participation.
FP7‐COOPERATION HEALTH SSH KBBE Environment NMP General Activities Security Energy Space Transport JTI TOTAL General Fem % % fem WP leaders‐ A % fem Exp. research‐ B % fem PhD students‐ C Diff. A‐B (**) Diff. A‐C
(**) 52 51 47 41 33 32 31 30 30 24 21 31 39 33 33 27 34 23 25 27 23 22 48 45 43 36 29 27 31 28 27 22 20 56 56 57 51 37 42 37 39 28 24 18 ‐17 ‐16 ‐10 ‐3 ‐2 ‐7 ‐8 ‐3 0 +1 +2 ‐24 ‐17 ‐24 ‐18 ‐10 ‐8 ‐14 ‐14 ‐1 ‐1 +4 39 29 36 46 ‐7 ‐17 % fem Exp. research‐ B % fem PhD students‐ C Diff. A‐B (**) Diff. A‐C
(**) FP7‐CAPACITIES Science in Society International Cooperation Regions of Knowledge Research Potential Coherent develop. of policies Research for the benefit of SMEs Research Infrastructures TOTAL General % fem WP Fem % leaders‐ A 54 47 49 48 53 36 61 54 ‐4 +12 ‐8 ‐8 44 43 40 43 33 38 36 40 33 43 50 40 +7 ‐7 +5 ‐6 ‐10 ‐7 31 25 30 33 ‐5 ‐3 29 24 24 31 0 ‐7 35 (*) 30 31 40 ‐1 ‐10 (*) This low mean is due to the fact that from the 947 reported projects in CAPACITIES, more than half ‐
500 ‐ are in the priority area of research for the benefit of SMEs. (**) A difference of zero or a slightly positive difference indicates themes with no vertical segregation. The higher the negative difference, the stronger is the vertical segregation. Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 119 9.11. Mobility of researchers in FP7-PEOPLE
… to EU member state
net mobility from EU‐member state …
UK
UK
x
IE
‐46
SE
‐30
DK
‐10
FI
‐32
FR
‐268
BE
‐45
NL
‐140
LU
3
DE
‐393
AT
‐116
IT
‐671
ES
‐492
EL
‐170
PT
‐103
PL
‐216
CZ
‐46
HU
‐66
SI
‐25
RO
‐51
SK
‐19
BG
‐39
HR
‐22
EE
‐6
LT
‐17
LV
‐4
CY
‐14
MT
‐3
Total ‐3.041
IE
SE
46 30
x
3
‐3
x
2
9
‐1
‐7
50 ‐33
2
‐2
7
‐15
0
0
‐1 ‐75
‐5
‐6
‐40 ‐38
‐1 ‐50
‐10 ‐26
‐5 ‐22
‐29 ‐31
‐1
‐4
‐6
‐5
0
‐4
‐7
‐3
0
‐2
‐1
‐3
‐3
‐1
0
3
‐2
‐3
0
‐4
‐1
‐4
‐3
‐1
‐12 ‐294
DK
10
‐2
‐9
x
‐4
‐18
‐4
‐9
1
‐53
‐5
‐55
‐30
‐9
‐19
‐12
‐2
‐8
‐3
‐2
‐1
‐5
‐4
‐3
0
‐1
0
0
‐247
FI
32
1
7
4
x
0
3
3
9
3
4
‐21
‐9
‐1
‐2
‐7
‐1
‐8
0
‐2
0
‐2
‐1
‐2
‐2
0
‐2
0
6
FR
268
‐50
33
18
0
x
54
47
31
88
14
‐223
‐75
‐27
‐11
‐60
‐7
‐13
‐2
‐26
‐5
‐19
‐7
‐1
‐6
‐2
2
0
21
BE
45
‐2
2
4
‐3
‐54
x
5
13
‐20
‐3
‐119
‐64
‐18
‐12
‐59
‐3
‐8
0
‐9
‐2
‐11
‐6
‐1
‐1
0
1
0
‐325
NL
140
‐7
15
9
‐3
‐47
‐5
x
0
‐83
0
‐158
‐92
‐35
‐39
‐50
‐8
‐12
‐5
‐19
‐3
‐13
‐6
3
‐1
1
‐6
0
‐424
LU
‐3
0
0
‐1
‐9
‐31
‐13
0
x
‐15
‐1
‐17
‐2
‐3
0
‐2
‐1
‐4
0
‐5
‐1
‐1
0
‐1
0
0
0
0
‐110
DE
393
1
75
53
‐3
‐88
20
83
15
x
0
‐262
‐107
‐90
‐25
‐106
‐17
‐38
‐9
‐42
‐14
‐24
‐20
2
‐10
‐6
‐3
‐3
‐225
AT
IT
ES
EL
PT
PL CZ HU
116 671 492 170 103 216 46 66
5
40
1
10
5
29 1
6
6
38
50 26 22 31 4
5
5
55
30
9
19 12 2
8
‐4
21
9
1
2
7
1
8
‐14 223
75 27 11 60 7 13
3
119
64 18 12 59 3
8
0
158
92 35 39 50 8 12
1
17
2
3
0
2
1
4
0
262 107 90 25 106 17 38
x
43
15
0
2
15 5
7
‐43
x
‐121 26 35 56 3 36
‐15 121
x
41 32 25 5
4
0
‐26
‐41
x
13
4
‐1 3
‐2
‐35
‐32 ‐13
x
‐1 ‐1 2
‐15 ‐56
‐25 ‐4
1
x
‐8 0
‐5
‐3
‐5
1
1
8
x
0
‐7
‐36
‐4
‐3
‐2
0
0
x
0
‐4
‐1
1
1
2
0
0
‐15 ‐20
‐23 ‐4
‐3
3
‐2 ‐11
‐9
‐3
‐7
‐1
‐2
2 ‐21 ‐1
0
‐12
‐8
‐2
0
‐1 ‐3 ‐1
‐10 ‐10
‐14
0
0
‐3
1 ‐2
‐3
6
‐2
2
0
0
2 ‐1
0
1
‐3
‐7
0
‐4
0
0
0
‐1
‐1
0
‐1
0
0
0
0
‐2
‐5
8
‐4
0
0
0
0
1
‐2
2
0
0
0
0
‐6 1.568 643 436 311 678 70 204
SI RO SK BG
25 51 19 39
0
7
0
1
4
3
2
3
3
2
1
5
0
2
0
2
2 26 5 19
0
9
2 11
5 19 3 13
0
5
1
1
9 42 14 24
0 15 9
0
4 20 3 12
1 23 7
8
‐1 4
1
2
‐1 3
2
0
‐2 ‐3 ‐2 1
0
2 21 3
0 11 1
1
x
0 ‐5 1
0
x
0
0
5
0
x
0
‐1 0
0
x
‐3 0
0
0
‐1 0
1
0
0
0
0
0
0
0
0
0
1
0
0 ‐1
0
0
0
0
50 241 85 145
HR
22
3
1
4
1
7
6
6
0
20
10
10
14
0
0
3
‐1
2
3
0
0
0
x
0
0
0
0
0
111
EE
6
0
‐3
3
2
1
1
‐3
1
‐2
3
‐6
2
‐2
0
0
‐2
1
1
0
‐1
0
0
x
‐1
‐1
0
0
0
LT LV CY
17 4 14
2 0 1
3 4 4
0 1 0
2 0 2
6 2 ‐2
1 0 ‐1
1 ‐1 6
0 0 0
10 6 3
0 0 0
‐1 1 2
3 1 5
7 0 ‐8
0 1 4
4 0 0
0 0 0
0 0 0
0 0 ‐1
0 0 0
0 0 0
0 0 1
0 0 0
1 1 0
x 0 0
0 x 0
0 0 x
0 0 0
56 20 30
Macro‐ Regions
MT
3
3.041
2
1
535
0
0
0
0
838
0
0
3
231
0
‐1
2
‐2.958 ‐2
0
0
0
0
0
0
0
‐1.699 0
0
0
0
0
0
x
8
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 120
9.12. International collaboration with ‘Third Countries’ in FP7
Associated Countries
SWITZERLAND
ISRAEL
NORWAY
TURKEY
ICELAND
REPUBLIC SERBIA
other (1)
total
EC % EC Participant % of total EC contribution contribution Project Costs contribution to (million to project (million Euro)
assoc. Countries
Euro)
costs
2.744 2.034
74%
51%
1.264 875
69%
22%
1.056 754
71%
19%
260 189
73%
5%
95 69
73%
2%
79 63
80%
2%
41 31
75%
1%
5.538 4.015
73%
100%
partici‐ pations
4.457
1.968
2.185
1.162
273
314
325
10.684
% of participations from assoc. Countries
51%
22%
19%
5%
2%
2%
1%
100%
EU contribution per participation
456.446
444.653
345.014
162.472
253.992
200.579
95.250
375.831
(1) Includes: MACEDONIA, MONTENEGRO, MODOVA, BOSNIA AND HERZEGOVINA, LIECHTENSTEIN, FAROER, ALBANIA. ICPC Countries
RUSSIAN FEDERATION
INDIA
CHINA
SOUTH AFRICA
BRASIL
UKRAINE
EGYPT
ARGENTINIA
MAROCCO
MEXICO
KENIA
TUNESIA
TANZANIA
CHILE
BURKINA FASO
GHANA
UGANDA
THAILAND
COLOMBIA
JORDAN
VIETNAM
SENAGAL
other (2)
total
EC % EC Participant % of total EC contribution contribution partici‐ Project Costs contribution to to project pations
(million (million Euro)
ICPC Countries
Euro)
costs
126 73
58%
14%
545
54 39
72%
8%
280
56 35
63%
7%
383
51 34
67%
7%
240
72%
6%
224
45 32
33 24
73%
5%
215
18 15
81%
3%
125
19 15
77%
3%
119
18 14
78%
3%
126
24 13
53%
3%
119
16 13
79%
2%
77
16 13
79%
2%
103
15 12
77%
2%
50
12 9
75%
2%
68
11 8
77%
2%
28
10 8
79%
2%
54
44
10 8
80%
2%
9 7
79%
1%
51
8 6
75%
1%
50
7 6
85%
1%
50
7 6
76%
1%
48
8 5
72%
1%
50
164 114
70%
22%
1.060
738 509
69%
100%
4.109
% of participations from ICPC Countries
14%
8%
7%
7%
6%
5%
3%
3%
3%
3%
2%
2%
2%
2%
2%
2%
2%
1%
1%
1%
1%
1%
22%
100%
EU contribution per participation
134.252
139.062
92.187
143.238
144.878
110.991
118.171
122.843
110.868
107.142
163.435
121.908
236.772
135.274
302.692
151.881
176.631
137.750
122.451
119.348
117.075
109.278
107.867
123.967
(2) includes AFGHANISTAN, ALGERIA, ANGOLA, ARMENIA, AZERBAIJAN , BANGLADESH, BARBADOS, BELARUS, BENIN, BHUTAN, BOLIVIA, BOTSWANA, BURUNDI, CAMBODIA, CAMEROON, CAP VERDE, CENTRAL AFRICAN REPUBLIC, CONGO, COSTA RICA, COTE D'IVORE, CUBA, DEMOCRATIC REPUBLIC OF CONGO, DOMINICAN REPUBLIC, ECUADOR, EL SALVADOR, ETHIOPIA, FIDJI, GABON, GAMBIA, GEORGIA, GUATEMALA, GUINEA, GUINEA BISSAU, GUYANA, HAITI, HONDURAS, INDONESIA, IRAN, JAMAICA, KAZAKHSTAN, KOREA, KOSOVO, KYRGYZSTAN, LAOS, LEBANON, LESOTHO, LIBYA, MADAGASCAR, MALAWI, MALAYSIA, MALEDIVES, MALI, MARSHALL ISLANDS, MAURITANIA, MAURITIUS, MOZAMBIQUE, MYANMAR, NAMIBIA, NEPAL, NICARAGUA, NIGER, NIGERIA, OMAN, PAKISTAN, PALESTINIAN TERRITORY, PANAMA, PAPUA NEW GUINEA, PERU, PHILIPPINES, RWANDA, SAMOA, SEYCHELLES, SOMALIA, SRI LANKA, SUDAN, SURINAME, SWAZILAND, SYRIA, TAJIKISTAN, TOGO, TRINIDAD and TOBAGO, TURKMENISTAN, URUGUAY, UZBEKISTAN, VANUATU, VENEZUELA, YEMEN, ZAMBIA, ZIMBABWE. High Income Countries
EC % EC % of total EC Participant contribution contribution contribution to Project Costs (million to project High Income (million Euro)
Euro)
costs
Countries
USA
AUSTRALIA
CANADA
JAPAN
NEW ZEALAND
SINGAPORE
other (3)
total
144
67
52
31
11
7
11
323
82
13
12
10
4
3
3
126
57%
19%
22%
32%
36%
41%
25%
39%
(3) Includes: HONG KONG, MONACO, TAIWAN, MACAO, SAN MARINO 65%
10%
9%
8%
3%
2%
2%
100%
partici‐ pations
514
194
200
113
48
26
56
1.151
% of participations from High Income Countries
65%
10%
9%
8%
3%
2%
2%
100%
EU contribution per participation
159.459
64.873
58.252
88.267
80.299
111.111
49.837
109.214
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 121
9.13. Annual Budget of FP6, FP7 and HORIZON 2020 by sub-programmes
Annual budget
FP6
FP7
H2020
I. EXCELLENT SCIENCE
European Research Council
(Frontier research by the best individual teams)
Future and Emerging Technologies
(Collaborative research to open new fields of innovation)
Marie Curie Actions
(Opportunities for training and career development)
Research infrastructures
(Ensuring access to world‐class facilities)
II. INDUSTRIAL LEADERSHIP
Leadership in enabling and industrial technologies
(ICT, nanotechnologies, materials, biotechnology,
manufacturing, space)
Access to risk finance
(Leveraging private finance and venture capital for research and innovation)
Innovation in SMEs
(Fostering all forms of innovation in all types of SMEs)
III. SOCIETAL CHALLENGES
Health, demographic change and wellbeing
Food security, sustainable agriculture,
marine and maritime research & the bioeconomy
Secure, clean and efficient energy
Smart, green and integrated transport
Climate action, environment resource efficiency and raw materials
Europe in a changing world
Secure Societies
OTHER
Science ~ Society
Spreading excellence and widening participation (Policy support & research potential)
European Institute of Innovation and Technology (EIT)
JRC direct non‐nuclear actions
Euratom
Sources: European Commission, Austrian Research Promotion Agency (FFG). Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 122
Commitment and Coherence – Ex‐Post Evaluation of the 7th EU Framework Programme 123