52013DC0830

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

Second interim evaluation of the ARTEMIS and ENIAC Joint Technology Initiatives

(Text with EEA relevance)

1.           Introduction

This report highlights the findings and recommendations of the panel of independent experts who conducted the second interim evaluation of the ARTEMIS and ENIAC Joint Technology Initiatives (JTIs). It furthermore provides the Commission's observations and sets out follow-up measures. ARTEMIS and ENIAC are the Joint Undertakings (JUs) implementing JTIs in the respective fields of embedded computing systems and nanoelectronics. In this manner, the Commission complies with the evaluation requirements as laid down in Article 11.2 of the founding acts of the JUs[1].

2.            Background

The JTIs, set up in the form of JUs under Article 187 of the Treaty on the Functioning of the EU (former Article 171 of the Treaty), were introduced in the Seventh Framework Programme (FP7) to support key areas of research and technological development of importance to Europe’s competitiveness. The ARTEMIS and ENIAC JUs were launched in February 2008.

The ARTEMIS and ENIAC JUs are public-private partnerships between industry, a number of EU Member States and associated countries (JTI member States)[2], and the European Union with the specific aim to define and implement a common Research Agenda by the European research communities (industry and academic/research organisations), achieve synergy and coordination of European R&D, promote the involvement of SMEs and create significant economic and social benefits. The main instrument is the mobilisation of funds from the EU, JTI member States and industry.

Since their establishment, the ARTEMIS and ENIAC JUs launched and evaluated, respectively, six (one per year) and nine calls for proposals (one in each of 2008, 2009 & 2010 and two in each of 2011, 2012 and 2013). By the end of 2012, 102 projects had been funded by the ARTEMIS and ENIAC JUs. The EU and JTI member States together have so far committed over €1.115 billion to both JTIs combined (2008-2012), in addition to private R&D efforts worth more than €1.670 billion.

The 52 ARTEMIS projects to date represent a total R&D investment of €990 million, comprising of €288 million national contributions, €163 million EU contribution and €539 million industry contribution. Those projects address all eight Sub-Programmes of the ARTEMIS Strategic Research Agenda in the areas of safety, healthcare, smart environments, manufacturing, computing, security, sustainability and embedded systems design. After a relative low commitment by ARTEMIS member States in the first four years, the introduction of the ARTEMIS Innovation Pilot Projects in 2012 gave a boost to the JU activities resulting from higher commitments by member States.

In the ENIAC case, the 50 projects to date represent a total R&D investment of €1.795 billion, comprising of €382 million national contributions, €283 million EU contribution and €1.130 million industry contribution. The projects cover all eight work areas of the ENIAC Strategic Research Agenda in the fields of automotive/transport, communications/lifestyle, energy, health, safety/security, semiconductor design, semiconductor manufacturing and the underlying equipment/materials. Since 2011, the ENIAC JU became an important vehicle for the implementation of manufacturing pilot lines following the recommendations of the High-Level Group on Key Enabling Technologies. This created a steep increase in the joint investments by the stakeholders, bringing the execution of the programme close to budget that was foreseen.

As foreseen in the Council Decisions establishing the ARTEMIS and ENIAC JU, the Commission has to carry out, by 31 December 2010 and by 31 December 2013, an interim evaluation with the assistance of independent experts. The evaluations cover the ARTEMIS and ENIAC JU quality, efficiency and progress towards the objectives set. The first interim evaluation of the ARTEMIS and ENIAC JTIs[3] was carried out after two years of the initiatives launch and it mainly assessed the efficiency, management and operations of the JUs. The main objective of the second interim evaluation is to assess the achievements of the ARTEMIS and ENIAC JUs after more than 5 years of operation.

3.            Evaluation process

The second interim evaluation examined the relevance, effectiveness, efficiency and research quality of the JUs. In addition, it performed a follow-up of the implementation of the recommendations of the first interim evaluation in 2010 and to the Commission's report[4] on this interim evaluation report.

The Commission appointed a panel of external and independent experts, chaired by Dr. Götzeler[5]. In the composition of the panel care was taken to ensure a good coverage of the technical domains as well as continuity with the panel in charge of the first interim evaluation. The panel operated by consensus and based its findings and recommendations on extensive desk research, peer review evidence and interviews with stakeholders[6].

The evaluation by the experts was conducted between September 2012 and February 2013. Their final report was issued in July 2013 and is available on the Digital Agenda website[7]. The evaluation ran in parallel with the drafting of the Commission proposal on the setting up of a new JU to replace and succeed the ARTEMIS and ENIAC JUs. The conclusions and recommendations of the evaluation of ARTEMIS and ENIAC JUs were inter alia analysed and served as input to the Impact Assessment for the new JTI’s regulation.

4.            The evaluation results and recommendations

The second interim evaluation report confirms the high value and significant achievements of the ARTEMIS and ENIAC JUs. In particular, the panel concludes that (i) the relevance of the JUs remains high, and considerable progress has been made to achieving their objectives; (ii) the effectiveness is high; (iii) the efficiency is good despite the rather heavy regulatory, administrative and financial burden; and (iv) the quality of reviewing, reporting and monitoring of projects as done by the JU is high. Overall, the panel is supportive to the tri-partite JTI instrument, pooling resources from industry, the EU and Member States. Their recommendations call for further simplification, improving administrative procedures and strengthening the governance structures. The Commission welcomes and supports that the panel strongly advocates the need for a single European strategy for Electronic Components and Systems (ECS) research, development and innovation. This strategy should be developed along with a matrix of Key European Industries (KEIs) to complement the Key Enabling Technologies[8] (KETs) as a means to identify, support and grow the industrial eco-systems required for a sustainable European activity in ECS in a manner that is compatible with the openness of the JU.

The detailed list of the panel's recommendations is given in the annex. The timeframe for the recommendations indicates whether they are to be implemented under the current JUs or should be considered for the future JU, i.e. Electronic Components and Systems for European Leadership (ECSEL) proposed by the Commission[9].

Regarding the implementation of the recommendations of the first interim evaluation, the panel is overall satisfied and indicates that most of the recommendations were addressed in the Commission's report on the interim evaluation report and overall implemented in a satisfactory manner. Nevertheless, the implementation of some recommendations that were addressed to the Member States is according to the panel less satisfactory. These recommendations relate to the harmonisation of MS funding practices, procedures, and multi-annual budgetary commitments which may cause process delays and administrative burden.

5.            Planned follow-up actions

The Commission welcomes the second interim evaluation report. The recommendations are based on a thorough and critical analysis. The Commission will as set out below implement the recommendations made to it and will work with industry, JTI member States and the Joint Undertakings to help them undertake their parts.

In particular, the Commission will undertake:

· short-term actions for the existing JUs (i.e. their activities until the launch of the new JTI) through their governance structures; and

· actions in the context of the legal framework of the next generation JTI ECSEL.

The Commission's response to the recommendations from the panel is outlined below. This section also presents the follow-up by the Commission of its report about the first interim evaluation.

5.1.         General observations

The Commission acknowledges the value of the tri-partite model and is in particular pleased with the positive findings on the effectiveness and efficiency of the current JUs. The launch and ramping up of the JUs has been difficult but the major hurdles have been overcome.

In operational terms, the ENIAC JU is on track to bring it close to the original ambition for the initiative, i.e. to leverage an industrial investment programme in nanoelectronics R&D of some €2.7 billion. The ARTEMIS JU will be further away from the initial target of €2.5 billion in R&D for embedded computing systems. However, the developments of the last two years indicate also for ARTEMIS a significant increase of its volume of activities. The Commission is confident that both initiatives will have delivered on their promises by the end of their mandate in 2017.

5.2.         Recommendations for the Industrial Associations

The Commission recognises the important role of the industrial associations being members of the JUs, especially in the buy-in of industry including SME's, in keeping the research agenda up-to-date and in promoting the JUs. The Commission stresses the importance of the involvement of industry at the highest level to show leadership and engagement. In this context, the positioning document[10] developed under the auspices of AENEAS[11] and CATRENE[12] provides significant perspectives and objectives for the European nanoelectronics industry by 2020.

In agreement with the observations of the second interim evaluation panel, the Commission considers the evolution of the JUs to support projects at higher Technology Readiness Level (TRL) as both timely and needed, putting more focus on innovation and exploitation, and bringing R&D closer to the market. In particular, the call for manufacturing pilot lines by the ENIAC JU in 2012 has been very successful. This call is the first large-scale implementation of the recommendations from the High-Level Group on KETs. Similarly, the ARTEMIS Innovation Pilot Projects are indeed 'a laudable intent of sustaining innovation from proof of concept and prototyping stage right through to a solid industrial platform'.

Furthermore, the good work done within ARTEMIS-IA[13] on the portfolio analysis and key performance indicators is acknowledged. Both industrial associations together with the JUs will be encouraged to continue working in this direction thereby addressing recommendation 15.

Nevertheless the Commission concurs with the evaluation panel that more can be done as stated in recommendations 1 and 2. With the new ECSEL JU, the industrial partners should take the opportunity of a more coordinated and proactive approach to reinforce the strategic dimension of their cooperation. They will be invited to come up with key orientations for the European electronic components and systems industry to become the engine for the promising field of "smart everywhere" products and services. To this effect, the proposal of the Council Regulation on ECSEL includes a requirement for broader stakeholder engagement.

The report from the Commission on the first interim evaluation commented extensively on the positioning of the JUs towards EUREKA. The Commission now acknowledges that significant progress has been made. In the area of nanoelectronics, CATRENE and ENIAC developed a common Vision, Mission and Strategy document for R&D in micro- and nanoelectronics in Europe. Similarly, in the area of embedded systems, ITEA2[14] and ARTEMIS hold co-summits and have set up a coordination mechanism[15]. These actions will contribute to the development of an overarching EU research, development and innovation strategy covering nanoelectronics, embedded computing and cyber-physical systems in line with recommendation 16 and should be reinforced.

5.3.         Recommendations for the Joint Undertakings

The Commission concurs with the findings of the panel of experts that the evaluation and selection procedures and the technical review process have been streamlined, are good and perceived by the participants as being very helpful. Projects launched at the start of the JUs are now coming to an end and the focus of the final reporting should be on the exploitation of the results. The Commission therefore welcomes recommendations 3 and 15 and will ask the Executive Directors to examine the possibility to implement them and ensure a proper reporting on exploitation activities, though limiting the burden on the beneficiaries. The development of an appropriate metrics for measuring the impact and success of JU projects is equally important.

As set out above, both JUs proved to be able to respond quickly and appropriately to changing requirements and steer the initiatives closer to the market. The various bodies in the JUs are to be commended for this. Therefore, the Commission is confident that the recommendations 4, 6 and 14 will find positive reception within the bodies of the JUs as they address issues of a more operational nature.

The implementation of an ex-post audit strategy by the JUs has been particularly difficult and subject to observations from the Court of Auditors. The Commission will continue to raise this issue in the Governing Boards in view of achieving a reasonable assurance that the financial transactions of the JUs are correct. Recommendations 10, 12 and 13 are addressed in the proposed Council Regulation on ECSEL to the extent in which these recommendations can be implemented within the applicable legal framework.

5.4.         Recommendations for the European Commission

The recommendations relating to the next generation JTI have been taken into account in the Commission's proposal for a Council Regulation on the ECSEL Joint Undertaking. This concerns in particular the recommendation to have a single JTI (recommendation 5) with a single integrated research and innovation agenda (recommendation 1) based on a simplified financial regulation (recommendation 7) and an increased strategic role for its Governing Board (recommendation 8). The ECSEL proposal furthermore incorporates a further harmonisation of rules (recommendation 9) in accordance with the Rules for Participation of HORIZON 2020.

Moreover the Commission adopted on 23 May 2013 a European strategy for micro- and nanoelectronics components and systems[16], in line with recommendation 16. A cornerstone of the strategy is the set-up of the new ECSEL JU with a focus on innovation and covering higher TRL.

5.5.         Recommendations for the Member States

The Commission recognises the difficulties experienced by the JU member States in implementing their part of the recommendations, in particular with respect to engaging in multi-annual commitments, the harmonisation of rules and funding rates, and the synchronisation of procedures. The experience and mutual understanding built up through their participation in the ARTEMIS and ENIAC JUs should allow making a fresh start with the ECSEL JU under the umbrella of Horizon 2020. This is already visible in the ECSEL Council Regulation proposed by the Commission in which these issues are addressed. In particular, it offers the possibility of a common approach for the implementation of the public contributions to the projects. The Commission urges the Member States to take all necessary measures to support the proposed harmonisation and simplification of the processes for the beneficiaries in the future actions being retained for funding.

The Commission also notes that much progress has been made recently within ARTEMIS and ENIAC JU in finding a balance between supporting national priorities and achieving common European objectives.

6.            Conclusion

While the first interim evaluation helped the JUs improve their operations and focus on their operational objectives, the findings of the second interim evaluation were a timely and significant input in the drafting of the Commission proposal for the future ECSEL Council Regulation. Although there are still administrative hurdles which can be overcome in the future ECSEL JU, the ARTEMIS and ENIAC JUs proved to be an effective and efficient tool to implement an R&D programme driven by industry and co-financed by the Member States and the EU.

The final evaluation of the ARTEMIS and ENIAC JUs is scheduled for 2017 as part of the interim evaluation foreseen for the ECSEL JU. That final evaluation should look into the overall impact of the projects supported by the JUs, and comment on their contribution in the overall context of the European strategy for micro- and nanoelectronics and embedded systems.

 Annex - List of recommendations by the evaluation panel

No[17] || Summary of recommendations || Timeframe

Recommendations for the Industrial Associations

1 || The JTI SRA and work programmes need to reflect more strongly a coherent European perspective, linking to an overarching European Electronic Components and Systems research, development and innovation strategy, as proposed in Recommendation 16. || Next Generation JTIs

2 || The Industrial Associations should play a more active role in the definition of the overall objectives and strategy of the JTIs and should engage more actively with stakeholders so as to promote and facilitate participation in project proposals, especially by SMEs, and to develop and keep up-to-date the Strategic Research Agenda. || Now/ Next Generation JTIs

Recommendations for the Joint Undertakings[18]

3 || JTI project reviews, including a final post-project review that should be held, the panel concludes, between 6 and 12 months after the end of a project, should monitor more closely and rigorously the actual and planned exploitation of project results, and the measures put in place by project partners to achieve such planned exploitation. || Now

4 || ARTEMIS projects should build, where appropriate, on previously developed ARTEMIS technology, making reference to what has been funded before and demonstrating, in addition to novelty, the appropriate re-use of previous project results combined with a suitable progression to higher TRL levels. The proportion of funding for projects targeting generic applications and services (Applications projects) should be increased. || Now/ Next Generation JTIs

6 || ENIAC and CATRENE calls for, and selection of, proposals should be more closely aligned (e.g. by the use of common and/or complementary calls), with the relevant funding awarding bodies retaining some flexibility over the assignment of the most appropriate funding stream. || Now/ Next Generation JTIs

8 || Focus the JU Governing Boards on strategic issues and reduce their administrative burden in order to attract participation from high-level industry representatives. || Now/ Next Generation JTIs

10 || The JUs should explore and develop appropriate mechanisms to create an ‘early warning system’ to identify potential delays, or restrictions to the availability, of funding from Member States. In order to bridge any financial gaps so identified, advanced funding by the EC (on behalf of a Member State) should be allowed for projects which are mission-critical. || Now

12 || Take steps (e.g. modification of evaluation criteria) during the proposal evaluation and selection process to improve the match of the project portfolio to strategic European aims and to ensure optimum coverage of key areas defined in the overarching EU ECS strategy (proposed in Recommendation 16) and the workplans derived from such a strategy. || Next Generation JTIs

13 || Specific support mechanisms for enhancing the project management processes in JTI projects should be developed and implemented. Management costs should be 100% funded by the EC for all JTI projects. || Now/ Next Generation JTIs

14 || JTI projects should be subject to only one (i.e. the JU) project review and reporting process. || Now

15 || Appropriate metrics for measuring the impact and success of JTI projects should be developed and used for both current and future JTIs. || Now

Recommendations for the European Commission

5 || The ENIAC & ARTEMIS JTIs, along with the European Technology Platform (ETP) on Smart Systems Integration (EPoSS), should be integrated into a single organisation (an ECS JTI). || Next Generation JTIs

7 || Construct the proposed new, integrated JTI (of Recommendation 5), or indeed any future JTI, as a PPP body as defined in Article 209 of the financial regulation. || Next Generation JTIs

13 || Specific support mechanisms for enhancing the project management processes in JTI projects should be developed and implemented. Management costs should be 100% funded by the EC for all JTI projects. || Now/ Next Generation JTIs

16 || A mid- to long-term overarching EU research, development and innovation strategy in Electronic Components and Systems (ECS) should be clearly defined and used as a key 'driver' for funding decisions. || Now

Recommendations for the Member States

9 || Member State participation rules, funding rates and procedures should be harmonised and synchronised wherever possible, adopting best practice as the guiding principle. || Now/ Next Generation JTIs

11 || Member States should commit to a multi-annual funding system. || Now/ Next Generation JTIs

14 || JTI projects should be subject to only one (i.e. the JU) project review and reporting process. || Now

[1]               Council Regulation No 74/2008 of 20 December 2007 on the establishment of the ‘ARTEMIS Joint Undertaking’ to implement a Joint Technology Initiative in Embedded Computing Systems. Council Regulation No 72/2008 of 20 December 2007 setting up the ENIAC Joint Undertaking.

[2]               ARTEMIS member States: Austria, Belgium, Cyprus (withdrew recently), Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Latvia, Netherlands, Norway, Poland, Portugal, Romania, Spain, Sweden, Slovenia, United Kingdom. ENIAC member States: Austria, Belgium, Czech Republic, Estonia, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Latvia, Malta, Netherlands, Norway, Poland, Portugal, Romania, Slovak Republic, Spain, Sweden, United Kingdom.

[3]               http://ec.europa.eu/dgs/information_society/evaluation/rtd/jti/index_en.htm

[4]               COM(2010) 752 final and http://ec.europa.eu/dgs/information_society/evaluation/rtd/jti/

[5]               The expert panel consisted of the following members: M. Götzeler (CEO Aixtron SE); W. Arden (former Infineon Technologies); Ch. de Prost (ATMEL); J.-L. Dormoy (EDF); M. Jansz (Technology Foundation STW); T. Luukkonen (Research Institute of the Finnish Economy); A. Sangiovanni-Vincentelli (University of California at Berkeley); D. Wright (University of Exeter).

[6]               The evidence base for the evaluation included an extensive desk review of relevant documents on legal and financial matters, research agendas, work programmes, participation statistics and project information. 104 interviews were conducted with a wide range of stakeholders: industry, academia, Commission, national Public Authorities, EUREKA clusters and the JUs.

[7]               https://ec.europa.eu/digital-agenda/en/news/second-interim-evaluation-artemis-and-eniac-joint-technology-initiatives

[8]               http://ec.europa.eu/enterprise/sectors/ict/files/kets/hlg_report_final_en.pdf

[9]               COM(2013) 501 final and http://ec.europa.eu/research/press/2013/pdf/jti/ecsel_factsheet.pdf

[10]             Innovation for the future of Europe: Nanoelectronics beyond 2020           http://www.aeneas-office.eu/web/downloads/strategic-docs/position_paper_final.pdf

[11]             The Association for European NanoElectronics ActivitieS         http://www.aeneas-office.eu/web/index.php

[12]             The Eureka Cluster for Application and Technology Research in Europe on NanoElectronics http://www.catrene.org/

[13]             The industrial Association for Advanced Research & Technology for EMbedded Intelligence and Systems   http://www.artemis-ia.eu/

[14]             The Eureka Cluster on Information Technology for European Advancement (ITEA)             http://www.itea2.org/

[15]             High-level vision 2030 ITEA-ARTEMIS - http://www.artemis-ia.eu/publications

[16]             COM(2013) 298 final

[17]             The numbering of the recommendations refers to the report of the experts.

[18]             These recommendations refer to a collective/joint responsibility of all parties involved in the JUs.