52011SC1072

COMMISSION STAFF WORKING PAPER

Report on the first interim evaluation of the Innovative Medicine Initiative Joint Undertaking Report on the first interim evaluation of the Clean Sky Joint Technology Initiative Joint Undertaking Report on the first interim evaluation of the Fuel Cells and Hydrogen Joint Undertaking

Accompanying the document

COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

Partnering in Research and Innovation

Overall introduction

Joint Technology Initiatives (JTIs) are a major novelty of the Seventh Framework Programme. On the basis of Article 187 of the Treaty on the Functioning of the European Union, five JTIs have been set up under 7th Framework Programme (FP7): Innovative Medicines initiatives (IMI), Advanced Research and Technology for Embedded Intelligence and Systems (ARTEMIS), Aaeronautics and Air Transport (Clean Sky), European Nanoelectronics Initiative Advisory Council (ENIAC) and Fuel Cells and Hydrogen (FCH).

In line with the request to the Commission under Article 11.2 of the Council Regulations setting up the Joint Undertakings to implement the Joint Technology Initiatives, independent interim evaluations of the operation of the Joint Undertakings have recently been carried out.[1]

The Commission response to the interim evaluations of the ARTEMIS and ENIAC Joint Undertakings has already been presented in COM(2010) 752 of 16 December 2010[2].

This Staff Working Document presents the detailed Commission response to the interim evaluations of the IMI, Clean Sky and Fuel Cells and Hydrogen Joint Undertakings.

Report on the First interim evaluation of the Innovative Medicine Initiative

1. Introduction

This report highlights the findings and recommendations of the first interim evaluation of the Innovative Medicines Initiative (IMI) Joint Undertaking (JU). The IMI JU was created between the European Commission and the European Federation of Pharmaceutical Industries and Associations (EFPIA), the umbrella organisation of the European pharmaceutical industry, to implement the Joint Technology Initiative in the area of Innovative Medicines. This report formulates the Commission's initial response to the evaluation's recommendations and sets out follow-up measures. The Commission thereby complies with the requirement to report on the first interim evaluation of the IMI JU, as laid down in the founding act.[3]

2. Background

In the field of innovative medicines, IMI was established in 2007 as a Joint Undertaking, based on Article 187 TFEU (ex Article 171 EC). This EU body constitutes a public-private partnership (PPP) between the pharmaceutical industry represented by EFPIA, and the European Union. The aim is to implement a research agenda defined by the European research communities (pharmaceutical industry, academic/research organisations, Member States) in the field of pre-competitive pharmaceutical research. This is to be achieved by means of in-kind contribution from the participating pharmaceutical companies, matched by a direct financial contribution from the EU. The JTI IMI thereby aims at increasing the investments in the biopharmaceutical sector in Europe and provide socio-economic benefits for European citizens, contribute to the health of European citizens, increase the competitiveness of Europe and help establish Europe as the most attractive place for biopharmaceutical research and development.

At the time of the first interim evaluation, the IMI JU had operated as an autonomous body for approximately one year, had launched its first two calls and was about to launch the third call. Fifteen projects from the first call had kicked-off and were ongoing, and a further 8 projects from the second call were under negotiation. These projects have been launched in the first few months of 2011. The in-kind contribution from the EFPIA companies to projects in the first two calls is € 198 million and the financial contribution from the EU budget, disbursed by the IMI JU is € 190 million. This is slightly lower than what was foreseen at the launch of IMI (€ 199.5 million).

3. Conduct of the evaluation

The objective of this evaluation was to assess the IMI JU against three criteria: the quality and efficiency of the IMI JU and its progress towards the objectives set. The Commission invited a panel of independent experts[4] to perform the first interim evaluation. The experts reviewed evidence and interviewed stakeholders[5].

The evaluation panel issued its report on 20 December 2010. The full report is available on the EUROPA website (http://www.imi.europa.eu/content/documents#imi_governance).

4. Evaluation results and recommendations

The overall appreciation of the panel for the first interim evaluation of IMI is positive. The experts state that:

"through the IMI JU, Europe has succeeded in establishing a new business model between public and private sectors" ...by bringing together the pharmaceutical industry, academia, SMEs as well as regulatory authorities and patient organisations.

This is considered a rare achievement by the experts.

This cooperation:

"enables mutual learning and the opportunity to build understanding of the respective rationales and approaches, with benefits to all parties."

Thereby IMI is a novel model for "open innovation", having achieved a unique scale in Europe.

The experts consider the scientific scope of IMI as well targeted and:

"The consortia formed so far carry out focussed research addressing problems of immediate relevance to industry and future public health. To have formed and embedded this new, applied research environment is a significant achievement for Europe."

As critical comments the evaluation panel underlines that it has heard evidence that the framework of IMI is not satisfactory for all stakeholders, especially concerning funding rates and sharing of intellectual property. The experts observed that the IMI:

"Executive Office is not yet fully staffed, but within this limitation is providing the necessary support to the programme."

The experts consider that the implementation of certain aspects of the IMI governance should be refined in order to better align the different actors in IMI, namely the Governing Board, the Scientific Committee, the Executive Office and the States Representatives Group.

In their performance assessment, as regards the effectiveness, the experts conclude that after the process of setting up the structure, IMI is now running well. Also, the first call projects running so far are working well. While the operation of IMI is overall considered as efficient, recommendations for improvements are given. Based on the large response to the calls, the positive reports about the evaluations from the independent observers as well as the potential of the projects and enthusiasm of the participants IMI is considered of high quality. The development of appropriate indicators to objectively measure the quality of IMI is considered important.

To put the assessment of IMI in a strategic context, the evaluation panel performed an analysis of the strength, weaknesses, opportunities and threats for IMI (SWOT analysis). The panel identified many strengths of IMI, such as the creation of a unique partnership between the public and private sectors that enables broad engagement of stakeholders and access to otherwise hidden resources in the pharmaceutical area. Further, the panel mentions the successful implementation of IMI, support from founding members, interest in participation and the high quality of the projects.

The panel considers the lack of key performance indicators and proactive communication activities, including limited engagement with policy makers as weaknesses. The fact that internal governance structures are not yet working optimally and lack of autonomy of the Executive Office are also considered as weaknesses.

The links with regulators, possible synergisms with programmes at national level, established through the States Representatives Group and the launching of larger, more strategic projects are all considered as opportunities for IMI.

The identified weaknesses can become threats, if not appropriately addressed. The economic crisis and the continued change in the pharmaceutical industry and the rapid pace of scientific change can potentially become threats for IMI. The move towards larger projects identified as an opportunity can also lead to new threats for IMI.

Based on the identified weaknesses in the areas of governance structures, lack of proactive communication, not optimum exploitation of the advisory potential of stakeholders such as the European Medicines Agency and finally the lack of key performance indicators, the evaluation panel has come up with seven recommendations to make IMI better:

1. Continuously improve stakeholder involvement in IMI supported research projects.

2. Continuously ensure EFPIA and Commission commitment to IMI’s success and sustainability.

3. Ensure excellence and exploit new ways to support IMI scientific objectives.

4. Improve IMI communication.

5. Reinforce and streamline decision making and well-functioning processes.

6. Ensure best use of IMI results and IMI sustainability.

7. Develop monitoring and evaluation processes.

Each of the top-level recommendations is broken down into several sub-recommendations, addressed to either one or several of the following: IMI Executive Director, IMI JU Governing Board, European Commission, EFPIA for a total of 28 recommendations. The detailed recommendations are listed in the annex.

5. Planned follow-up actions

The Commission welcomes the interim evaluation report about IMI JU. It takes note of the recommendation of the evaluation panel. The Commission is committed to working with EFPIA to implement the recommendations addressed to the IMI Governing Board, where the two founding members work together to give strategic direction and oversight to the IMI JU. The Commission is committed to implementing the recommendations addressed to it and is open to collaborating with EFPIA and the IMI Executive Director implementing the recommendations addressed to them.

Most of the recommendations can be implemented in the short to medium term and indeed many are already being addressed through actions from the Governing Board and the Executive Office.

The recommendation to setting up public private partnerships as a 'special body' can only be addressed in the context of the discussion about the next generation of public - private partnerships (PPPs).

(a) Recommendations addressed to the IMI Governing Board

The evaluation panel has issued the largest number of recommendations to the Governing Board of IMI, in which EFPIA and the Commission are equally represented.

The recommendation regarding concerns about the IMI IP policy has already been addressed by the Governing Board and the Executive Director. A clarification note to the IMI IP policy had been issued in 2009, supplemented by a guidance note published in November 2010. An IP Help Desk has been established by the Executive Director, and operating since December 2010

The experts recommend that the IMI Board should reflect on how to best account for resources contributed by EFPIA from outside of Europe. The Commission welcomes EFPIA company participation in IMI projects from outside of Europe. EFPIA can inform about this participation, which is in addition to the in kind contribution that is expected in proportion to the funding for academic and SME researchers contributed by the public side.

The issue of rates for indirect costs of academic and SME participants in IMI projects eligible to receive a financial contribution is addressed by a revision of the IMI grant agreement. In addition to the current flat rate for indirect costs participants eligible to receive a financial contribution from IMI will be able to charge actual indirect costs, including a method for simplified calculation of the actual indirect costs.

The Commission fully endorses the recommendation of continuing to ensure excellence and exploit new ways to support the scientific objectives of IMI by implicating the States Representatives Group and the Scientific Committee. This has been addressed in the ongoing revision of the Scientific Research Agenda and the preparation of the topics for the fourth call. Input from the Scientific Committee and the SRG will continue to be solicited for the definition of the topics for the remaining IMI calls. The fourth call already contains the first topics for "Think Big" projects, a trend that will continue. Bringing in industries converging with the pharmaceutical industry is already happening in several of the ongoing IMI projects. The Commission would welcome further examples of this, where scientifically justified.

The Commission agrees that more can be done as regards communicating about IMI. Together with the Executive Director, the Governing Board is working on refining the communication strategy for IMI and putting it into practice. It is clear that the communication strategy of IMI is led by the Executive Director with support from the founding members.

During the set-up of IMI and its early implementation, the IMI Governing Board had to address many organisational issues. As IMI is now moving into the phase of implementing the successive calls, the Governing Board has more time addressing strategic issues of IMI. In this context the Commission considers that in the spirit of a true partnership decisions are taken at the Governing Board in the appropriate time frame.

The Commission agrees that knowledge management processes need to be developed. This is being taken up in the fourth call and the Commission supports continued attention to this aspect throughout IMI. This will also support the sharing of knowledge between all IMI projects.

The experts recommend that the IMI Board should consider how to realise value from excellent but ultimately unfunded proposals through networking and knowledge management. The Commission agrees that the second-ranked and following Expressions of Interest that are considered of high quality in the evaluation represent significant potential. However, it is unlikely that such projects will be funded in the way they were proposed. Therefore the Commission considers that it will not be possible to implement the recommendation of the evaluation panel as proposed. More generally the Commission considers that it is a good proposal to open up certain aspects of the knowledge management system of IMI, to incorporate results from related research funded from other sources. This can for example be of relevance in cooperation between IMI and other partnerships in pharmaceutical research at national or international level.

Overall IMI is clearly a key initiative for the European Commission, which is fully in line with the Europe 2020 Strategy and the Innovation Union. The Commission is ready to further develop the partnership in pharmaceutical research to the benefit of patients and to improve the competitiveness of Europe.

(b) Recommendations addressed to the Executive Director

The Commission is ready to support the Executive Director of IMI and the Executive Office in implementing the recommendations addressed to him. Indeed, several of the recommendations, notably as regards improving the communication activities of IMI, are also addressed to the Governing Board.

The Commission agrees to the importance of harnessing the innovation potential of SMEs for IMI. The activities of the Executive Office to be in close contact with SME umbrella organisations are welcome. The Commission considers that it is worth further reinforcing these efforts. Meaningful involvement of patient organisations is key for biomedical research. Amongst the projects that will be launched from the third call one will specifically address this aspect.

Regulators and in particular the European Medicines Agency (EMA) are also an important stakeholder in IMI. The Commission welcomes their involvement in several ongoing projects and considers it relevant for them to continually be implicated in the further development of IMI. However, as EMA is also involved in IMI-funded projects, care needs to be taken to avoid conflicts of interest and for example EMA should not be a member of the IMI scientific committee.

During the setting up of the Executive Office a considerable number of staff had to be recruited and some delays were experienced. This phase is now ending and only the filling of a few more posts is envisaged. The Commission does not have particular concern in this context.

The continued evaluation of the IMI through Key Performance Indicators is fully supported by the Commission. Although the recommendations in this area are addressed to the Executive Director of IMI, the Governing Board has taken this up as being of key strategic importance and will work with the Executive Director implementing this aspect.

(c) Recommendations addressed to EFPIA

The Commission agrees that the leadership for the definition of the Scientific Research Agenda and its implementation through the launch of calls must come from EFPIA. The organisation of the pharmaceutical companies is changing constantly, resulting in changes of staff involved in IMI from EFPIA side. The Commission considers that after the successful setting up and launch of IMI there has been some challenge of moving ahead in the revision of the Scientific Research Agenda as fast as would have been desirable. This revision is now almost complete and the call topics for the 4th call have been defined. This call includes the first "Think Big" topics are considered a success by the Commission.

In light of the volume of funds still to be committed, for the Commission it will be very important that sufficiently high commitment for in kind contribution will be made by EFPIA companies in all next calls.

(d) Recommendations addressed to the European Commission

Both the Commission and EFPIA are recommended to dedicate enough staff to IMI. The Commission considers that both founding members have shown extraordinary commitment towards the successful setting up of IMI, including the launch of the IMI JU as an autonomous entity. Considering the number of processes that had to be established, the Commission considers that overall this has succeeded well, while recognising that some delays have occurred. It is inherent in a partnership between partners as different as EFPIA and the European Commission that it takes some time to agree on certain points, especially at the beginning. With the IMI JU Executive Office now almost fully staffed, all operational responsibility has moved to it. Indeed, the Commission is committed to giving the Executive Director full autonomy for this responsibility. Accordingly, the number of staff dedicated to IMI has been reduced at the Commission, without any loss of capacity to dealing with the issues at hand.

The recommendation to develop a simpler legal framework than the current set-up of IMI as a community body is in line with the recommendations made by the JTI Sherpas' Group[6]. Changes to the Financial Regulation applicable to the general budget of the European Union are required to make this possible.

Report on the First interim evaluation of the Clean Sky Initiative

1. INTRODUCTION

This document highlights the findings and recommendations of the first interim evaluation of Clean Sky. Clean Sky is a Joint Undertaking (JU) implementing the Joint Technology Initiative (JTI) in the field of environmentally friendly aeronautical technologies. It formulates the Commission’s initial response to the evaluation’s recommendations and sets out follow-up measures. Thereby the Commission complies with the requirement to report on the first interim evaluation of Clean Sky as laid down in its founding act[7].

2. BACKGROUND

The Clean Sky Joint Undertaking contributes to the implementation of the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007-2013) and particularly to the Specific Programme "Cooperation" on its theme "Transport". Its objective is to develop innovative technologies with low environmental impact for all flying segments of the Air Transport System, allowing substantial reduction of noise, fuel consumption and emission of noxious gases in line with the targets set by the Technology Platform for Aeronautics ACARE. Its activities will consist in technical development up to full scale final demonstrators covered by Grant Agreement to named beneficiaries and by further partners selected through Calls for Proposals.

The Clean Sky Joint Undertaking became autonomous on 19 November 2009, meaning that it has the operational capacity to implement its own budget.

After the first 2.5 years of operation, members in Clean Sky have performed activities for a cumulated value of € 232 million across all programme areas and the first flight tests of innovative technologies have been accomplished. In addition to the members' activities, seven Calls for Proposals have been launched so far for additional € 63 million of funding (€97.5 million total scope of activities), engaging some 374 partners in 201 projects with an average duration of three years.

While this represents a considerable engagement of facilities, resources and budget, delays with respect to the original time plan have accumulated, due to initial difficulties in establishing the Clean Sky Joint Undertaking internal procedures and regulations as well as building up the technical teams and unfolding the technical programme.

3. CONDUCT OF THE EVALUATION

The objective of this evaluation was to assess the effectiveness, efficiency and quality of the JTI operations, both with regard to the Joint Undertaking and its operating bodies and the technical activities carried out by its members and partners, as well as the progress of Clean Sky towards its objectives as set up in Article 2 of the Clean Sky Regulation. To this end, the Commission invited a panel of independent experts[8] to conduct the evaluation.

The evaluation was carried out over the period from 29 September to 7 December 2010, and the evaluation panel issued its report on 15 December 2010. The evidence base for this evaluation included legal, financial and other documents setting up Clean Sky, technical and annual reports, project information and participation statistics. Interviews were carried out with representatives from the JU, members and partners of Clean Sky, including industry (and SMEs), research organisations and universities.

4. THE EVALUATION RESULTS AND RECOMMENDATIONS

The evaluation panel recognises the appropriateness of the Clean Sky Joint Undertaking (CSJU) concept and acknowledges a number of achievements:

· The setting up of an entirely new Public Private Partnership.

· Successful stimulation of developments towards environmental targets.

· Wide high level participation from all EU key industries, and inclusion of a large number of SMEs, thereby forging new collaborations and enhancing European integration.

At the same time, the experts express their concern that Clean Sky has incurred significant delays in the technical programme and urge for a robust action to ensure that the final technical objectives are reached in full within the regulatory time frame. The panel notes the following:

· The start of the CSJU activities was slow as compared to initial plans due to the time needed to establish and implement the rules of procedures and the time required to build up the working teams for both Integrated Technology Demonstrators and the Joint Undertaking Executive Team.

· The panel sees clear requirements for special provisions for the implementation and the specific operating needs of Joint Undertakings for Joint Technology Initiatives such as Clean Sky and for developing a specific model regulation for PPPs as foreseen by Article 201 of the proposed new Financial Regulation.

· For future Joint Technology Initiatives, a clear separation should be made between formal establishment and formal start of technical activities. The Commission should ensure that prior to the formal start of technical activities, the resources and administrative tools are essentially available and that an in-depth review of the technical programme is carried out.

5. PLANNED FOLLOW-UP ACTIONS

The Commission welcomes the interim evaluation report and takes note of the evaluation panel’s recommendations. The Commission is committed to address its part of the recommendations and to work with the stakeholders on the private side of the partnership and the Joint Undertaking to help them address their parts. The Commission will undertake:

(1) immediate actions for the existing Joint Undertaking (i.e. the JTI activities in the period 2011-2013) to be followed up through their governance structures, such as strengthening the analysis of contents of traditional projects and JU topics, support a revision of the delegations granted to the Executive Director by the Governing Board and contributing to streamlining administrative requirements within the current reporting duties; and

(2) longer-term actions for the potential next generation of PPPs. An assessment of the role and options for PPPs within the Common Strategic Framework for Research and Innovation[9] is now underway.

(a) General recommendations

The Commission notes the positive opinion of the panel on the Clean Sky JU concept, in spite of the delays and underspending which have accumulated at programme start. It agrees that the JTI model should be analysed further and is committed to assess the operational aspects of the JTI in order to optimize its effectiveness in the context of determining the "ideal house" infrastructure for future PPPs.

(b) Recommendations to the European Commission

The Commission agrees with the panel's views that a thorough analysis of the "Community Body" legal framework should be carried out in line with the findings of the JTI Sherpas' report[10]. Nevertheless, the Commission considers that, after the initial period, the JTI JU appears to have increased its efficiency, and that the technical work plan has been reviewed thoroughly.

It recognises that the "Community Body" approach is too rigid in some of its aspects to be applied to a Joint Undertaking of the size of Clean Sky.

The Commission recognises the need to better coordinate the calls published by Clean Sky and those published by the Commission under the Framework Programme. It agrees with the panel which recommends formalising the process of coordination between the two work programmes. An appropriate procedure is being established.

It also recognises the need for streamlining the coordination between the activities of the SESAR Joint Undertaking and Clean Sky. It is noted that this task is under the responsibility of the JUs concerned and the Commission is prepared to act as a facilitator.

The Commission agrees with the panel conclusion that the Clean Sky JU should be continued with special attention towards adhering to the main objectives and work plans. It already closely monitors and stresses the need to achieve the objectives set out in the Clean Sky Regulation (article 2). However, the technical choices necessary to this end are under the responsibility of the JU and in particular its industrial members.

The Commission acknowledges the relevance of regulatory measures to accelerate the introduction of new 'green' technologies demonstrated in Clean Sky on aircraft. The inclusion of aviation in the EU Emissions Trading Scheme (ETS) as of 1 January 2012 constitutes a step in this direction.

The Commission agrees with the panel that EU funding should target high risk R&D activities where the market failure component is substantial. The programme of Clean Sky fulfils this condition while the necessary controls and monitoring activities are in place, including the possibility of carrying out technical audits. It is noted that the JU has the responsibility of ensuring that the activities undertaken are focused on meeting the Clean Sky objectives which include novel and advanced technological concepts requiring full scale demonstration before being considered for deployment.

(c) Recommendations to the Governing Board

As a member of the Governing Board, the Commission agrees to review the level of decisions this body is called upon in order for it to focus on strategic decision and to conduct reviews of research needs and policy priorities, when necessary with the involvement of Clean Sky's Scientific and Technical Advisory Board and/or external experts. The Commission would support to the extent possible a revision of the delegations granted to the Executive Director to enable the Governing Board to focus on strategic decisions, leaving the daily management to the Executive Director. The Executive Director should be fully responsible for the implementation of the budget, within the boundaries set out by the Annual Implementation Plan as adopted by the Governing Board.

In particular, the Commission supports the recommendation of the panel calling for an increase of staff levels to enable the Executive Team led by the Executive Director to fully exercise its coordinating and monitoring role. The Commission would welcome a proposal of the Executive Director on this point.

The Commission understands the need for a contingency budget as underlined by the panel to cover unforeseen changes, repairs etc. It looks forward to a proposal on this point from the Joint Undertaking, in line with the current Commission financial regulatory framework.

The panel recommends introducing multi-annual grant agreements for the members of Clean Sky. The Commission points out that the need to have annual amendment to the grant agreements of the members derives from the availability of the Commission budget which is subject to the annuality principle governing the general budget of the European Union, as well as for practical reasons of accounting transparency. However, the Commission has recognised the need to reduce the administrative burden which arises from the annual grant agreement cycle. In addition, after 2013, the Members' activities will already be covered by multi-annual grant agreements, as the end of FP7 will lead to a final commitment to cover all the remaining activities of the JU.

(d) Recommendations to the Joint Undertaking

The Commission agrees that a number of streamlining measures should be put in place in order to increase the effectiveness and control capability of the JU over technical activities carried out by members and partners. It notes further that some procedures were newly developed as Clean Sky is part of the first wave of JTIs being implemented, and considers that adequate processes and tools should be put in place in order to monitor and analyse the strategic achievements of Clean Sky. In particular, the panel recommends a number of measures to be implemented to ensure that all objectives are reached in full, including auditing strategies and the monitoring of roadmaps. While the Commission agrees with the aim of the recommendation, it is up to the JU to respond on practical issues of implementation.

The Commission agrees with the request of speedy and efficient implementation of the panel recommendations.

The panel highlights the importance of proper communication both within and outside Clean Sky. The Commission agrees to this, and notes that a Communication strategy document has been prepared by the JU to that effect.

Finally, the Commission shares the panel's opinion that the Technology Evaluator has a central role in Clean Sky, both technically and politically. It endorses the recommendation of raising the profile of the Technology Evaluator and agrees to a more proactive role for it at the later stages of the Clean Sky programme, also in view of future developments of the JTI concept.

(e) Recommendations to the Stakeholders

The Commission notes that the panel highlights the need for streamlining technical activities in order to ensure reaching the Clean Sky targets. The Commission supports this view. It also supports the position that it is imperative for Clean Sky to reach its objectives in full as a condition for developing further the JTI concept in the aviation domain.

(f) Conclusions

The Commission acknowledges the thorough and in-depth work carried out by the Interim Assessment panel. Its recommendations are valuable in order to identify and focus on specific areas the improvement of which would increase the effectiveness of the Clean Sky JU programme.

It is satisfied that the validity of the JTI approach is recognised by the panel and undertakes to implement corrective measures when appropriate within its powers of intervention and invites other bodies (JU, stakeholders, Governing Board members) to act accordingly.

The recommendations are taken on board and, when it comes to operational aspects (mostly addressed to the JU), actions have been proposed and, when appropriate, a deadline identified.

First interim evaluation of the Fuel Cells and Hydrogen Initiative

1. INTRODUCTION

This section highlights the findings, recommendations and conclusions of the first interim evaluation of the Fuel Cells and Hydrogen Joint Undertaking (FCH JU) performed by a panel of independent experts. It also formulates the Commission’s initial response and position to the panel’s recommendations and proposes follow-up measures.

2. BACKGROUND

The Seventh Framework Programme (FP7) for research, technological development and demonstration activities, the European Union provides for a financial contribution for the establishment of long term public-private partnerships in the form of Joint Technology Initiatives (hereinafter referred to as JTI) which could be implemented through Joint Undertakings within the meaning of Article 171 of the EC Treaty (now Article 187 of the TFEU) and defines specific criteria to identify them.

The Fuel Cells and Hydrogen Joint Undertaking (hereinafter referred as FCH JU) was established via Council Regulation[11]. The FCH JU is a public-private partnership that has as members the European Union represented by the European Commission, the fuel cell and hydrogen industries represented by the "NEW" Industry Grouping and the research community represented by the "N.ERGHY" Research Grouping. Its main aim is to accelerate the market introduction of these technologies, realising their potential as an instrument in achieving a carbon-lean energy system.

As for the other JTIs, the process of creation of the FCH JU was rather slow and complex due to the constraints and requirements arising from its status as a "Community body". The FCH JU became autonomous on 15th November 2010, when it reached the operational capacity to implement its own budget. During the period May 2008 - November 2010, the responsibility for managing the FCH JU was with the European Commission.

Three annual calls for proposals have already been completed up to date and a balanced portfolio of projects has been selected. Those from the first two calls for proposals (44 projects already on-going) will receive grants for a cumulative value of ~ 100 million Euros, engaging some 250 different partners. The negotiations for projects of the 2010 call for proposals (estimated 43 grants for a value of ~ 89 million Euros) should be completed by the end of July 2011. The 2011 call was published on 3rd of May 2011. Consequently, the operational delivery of the FCH JU to date is as originally planned.

On April 20th 2011, the Commission adopted a draft proposal for amending the Council Regulation (EC) N° 521/2008.This amendment recognises the important role of the Research grouping and the research community and at the same time will allow to better align the FCH JU maximum funding rates to those of FP7 making them more predictable and competitive than at present.

3. CONDUCT OF THE EVALUATION

The objective of this evaluation was to assess the effectiveness, efficiency and quality of the FCH JU operations, both with regard to the Joint Undertaking and its operating bodies and the technical activities carried out by its members and project participants. In particular, emphasis was put on the progress of the FCH JU towards its objectives as set up in article 2 of the FCH JU Regulation. To this end, the Commission invited a panel of independent experts[12] to produce a final report based on the review of evidence obtained from the background material procured by the Commission and, mainly, from interviewing a representative group of selected stakeholders.

The experts' evaluation was undertaken between mid-December 2010 and mid-April 2011, i.e. in the first few months of the FCH JU autonomous operation. During this time the Commission services provided the necessary secretarial assistance to the expert panel. The background material procured by the Commission for this evaluation consisted inter alia of legal, financial and other documents related to the setting up the FCH JU, technical and annual reports, minutes of meetings, and information and statistics on calls for proposals and projects. The main evidence base was obtained through interviews carried out with a representative and balanced group of people covering the whole spectrum of stakeholders of the FCH JU.

4. THE EVALUATION RESULTS AND RECOMMENDATIONS

The primary outcome of the experts' report is that the "FCH JU should be maintained and supported to implement its work as originally envisaged". However, the experts have also identified a number of issues and problems encountered by the FCH JU as well as some areas where its operation could be improved, and they had made a number of specific recommendations to this end.

First of all, the report states that the establishment of the FCH JU as an industry-led public-private partnership (PPP) is an achievement in its own right and represents a valuable instrument for the European Union. It is considered a unique platform and instrument at European level involving the most important stakeholders in defining objectives and implementing and monitoring activities towards deployment of fuel cells and hydrogen technologies. The evaluation found also that the overall technical objectives of the FCH JU as defined in the Multi-Annual Implementation Plan (MAIP) remain ambitious and competitive in comparison with efforts world-wide. It concluded that the FCH JU approach is generally regarded as a good mean to enhance public-private activities in the field and perceived as an improvement to the research landscape.

However, the experts consider that the set-up of the FCH JU took too long and conclude that the current legal framework as a “Community/Union body” is not best-suited to industry led public-private partnerships like JTIs and should be streamlined. They also highlight that funding rates for FCH JU projects have proved variable from year to year and, in addition, always considerably lower than those of FP7. Other expressed concerns requiring due attention are the inadequate resources of the Programme Office for effective project monitoring and management, the insufficient cohesion and collaboration with Member States’ related programmes and, the lack of a formal communications plan and an international cooperation strategy.

In order to address the different issues and difficulties identified in the report, the experts' panel makes a series of recommendations that aim to remove or reduce weaknesses as identified in the current operations of the FCH JU and to improve its effectiveness and quality. The recommendations have been grouped in five broad categories:

(1) Reinforce portfolio management

(2) Ensure high agility of operations and adaptability to changing competitive forces

(3) Improve visibility, communication and outreach

(4) Improve collaboration and alignment with Member States

(5) Ensure high efficiency of operations

The position and response of the Commission to these recommendations is presented in the following section.

5. PLANNED FOLLOW-UP ACTIONS

In general, the Commission agrees with the panel overall views and conclusions as well as with most of the specific recommendations. The Commission is committed to address its part of the recommendations and to work together with the other FCH JU stakeholders and bodies to treat the recommendations addressed to them. The Commission will undertake:

Short-term actions affecting the operational aspects of the FCH JU in the period 2011-2013; and

Longer-term actions of relevance for the potential next generation of PPPs within the Innovation Union initiative launched to address the Europe 2020 strategy.

The Commission notes the positive opinion of the panel on the FCH JU concept, in spite of the difficulties encountered and the issues identified. It also agrees that the potential continuation of the FCH JU, or an equivalent initiative, in a new phase of EU support following FP7 should be further reviewed at a later date.

5.1. Group of Recommendations 1: "Reinforce portfolio management" 5.1.1. The Commission acknowledges the panel comment on the need for a thorough review and update of the Multi Annual Implementation Plan (MAIP) before the production of the Annual Implementation plan (AIP) for 2012. However, it notes that such a revision is currently on going with a final version of the document expected in the third quarter of 2011. It also notes that the Commission lead service for the Technologies Information System (SETIS) has committed to produce a methodology for reviewing and testing the correlation between the project targets and the overall achievement of objectives and to contribute to the development of a roadmap for fuel cell and hydrogen technologies extending up to 2020. 5.1.2. The Commission is also aware that breakthroughs are needed for hydrogen production, storage and distribution and that the FCH JU current projects portfolio is light on this critical area. However, it notes that in the AIP 2011 a higher share of the funding is foreseen for this Application Area. The Commission will oversee that this is also the trend for AIPs 2012 and 2013. The FCH JU programme should also look at synergies between hydrogen and renewable electricity and integrated infrastructure solutions for fuel cell and battery electric vehicles (FCVs, BEVs). Also quality manufacturing methods should be considered. 5.1.3. The Commission agrees that the priorities for Regulations, Codes and Standards (RCS) should be formulated by industry. The Commission also agrees on the relevance of RCS to accelerate the introduction of fuel cell and hydrogen technologies in the market and feels that it is an area where more effort and coordination with international partners are needed. 5.1.4. The Commission fully agrees that the structure and composition of the annual calls for proposals should explicitly support the objectives of the FCH JU. The Programme Office should make sure that the call topics and subsequent projects are wholly in response to challenges identified as arising from the FCH JU objectives and this aspect should prevail over specific industrial interests and/or established competences of specialised expert communities. To that end, the Commission advices the Executive Director to make sure that the Project Managers of the FCH JU get involved early in the process of elaborating the Annual Implementation Plans. 5.1.5. The Commission recognises that basic research seeking radical innovation is not sufficiently addressed in the FCH JU Annual Implementation Plans and notes that it has already raised this point to the Governing Board. The Commission feels that complementary support to this kind of research could be explored through the "Future and Emerging Technologies" calls of the FP7 energy theme. The agreement of the Energy Programme Committee will be sought to that end. 5.2. Group of Recommendations 2: "Ensure high agility of operations and adaptability to changing competitive forces" 5.2.1. The Commission agrees that all stakeholders of the value chain should be involved in the FCH JU. In particular, the role of Regions and Municipalities (through “HyRaMP”) should be enhanced. Furthermore, the Executive Director could explore potential links with the financial sector - especially EIB and Venture Capital community - as well as trade associations for industry users and various potential consumer bodies. 5.2.2. The Commission agrees with the panel that the establishment of an SME contact point at the Programme Office would facilitate participation of SMEs in product development and early markets projects and encourages the Executive Director to provide the necessary means and resources to that end. 5.2.3. The Commission commits to work together with the FCH JU to explore opportunities for complementarity between fuel cell and battery electric vehicles (FCVs, BEVs) in the market place. The recent report on automotive applications “A portfolio of power-trains for Europe: a fact-based analysis” performed by McKinsey could be used as the guidance document. The relevant Commission Services, the Programme Office and the Industry Grouping should take the necessary steps in order to explore these opportunities. 5.2.4. The Commission recognises the need for a strategy to strengthen the development plans for stationary applications in a similar way to what has been done for automotive applications (i.e. the above-mentioned “McKinsey report “). However, the Commission notes that a public procurement for a contract to establish an independent fact base for fuel cell technology in stationary applications as well as a European based roll out plan is foreseen in the FCH JU Annual Implementation Plan 2011. 5.3. Group of Recommendations 3: "Improve visibility, communication and outreach" 5.3.1. & 5.3.2. The Commission acknowledges the need for an effective communication strategy and dissemination plan both within and outside the FCH JU in order to increase the "visibility" of hydrogen and fuel cell technologies identified by the panel. However, it notes the positive steps already taken since the appointment of the "Policy and Communications Officer" and the "Stakeholder Relationship Manager", which have resulted inter alia in the development of a new website and in the first draft of a FCH JU Communication (internal and external) strategy. The Commission also welcomes the proposal of the Programme Office of linking the Stakeholders General Assembly with the review of all FP7 ongoing projects on FCH and will contribute to the definition of a suitable approach. 5.3.3. The Commission agrees with the panel recommendation to best involve the States Representatives Group in the FCH JU communication and outreach activities. 5.3.4. The panel's views that an appropriate strategy and priorities should be developed and agreed for international cooperation are also shared by the Commission. The FCH JU should be able to define the level of international engagement or participation required to support the achievement of its programme objectives, building on previous experiences. Existing cooperation schemes such as the International Partnership for Hydrogen and Fuels Cells in the Economy (IPHE) and the EU-EU Energy Council should continue with an increasing support of the Programme Office. 5.3.5. The different services of the Commission involved in the activities of the FCH JU should coordinate their effort to ensure that the outputs from the FCH JU projects are integrated into, and used to support to the relevant EU policies, particularly those stemming from the EU 2020 Strategy such as the "Innovation Union" and "Resource efficient Europe" Flagship Initiatives. 5.4. Group of Recommendations 3: "Improve collaboration and alignment with Member States" 5.4.1. & 5.4.2. The Commission takes note of the panel's view that the States Representatives Group (SRG) needs to significantly improve the interaction and coordination between the activities of the FCH JU and the Member States’ programmes. To that end, the Commission agrees that the profile of the SRG members needs to be revised so that they are connected directly to their domestic policy and programme delivery and able to identify opportunities for alignment of national activities to those of the FCH JU. The Commission also recognises the need to further involve the SRG members in the activities of the FCH JU. 5.4.3. The Commission fully supports the panel recommendation to explore joint funding schemes between FCH JU and Member States, in particular, the FCH JU should consider using some of the funding mechanisms currently being tested by some of the European Industry Initiatives (EIIs) of the SET Plan. The mapping exercise currently being performed by the responsible Commission service is considered as an important first step. 5.5. Group of Recommendations 5: "Ensure high efficiency of operations" 5.5.1. The Commission acknowledges the panel's general view that the current legal framework as a Community/Union body is not best-suited to industry led public-private partnerships like JTIs and notes the panel's support to the related recommendations of the JTI’s Sherpa Group. In addition, the Commission recognises that the resources for project management and monitoring might have to be strengthened and agrees that the possibility of sharing resources for required administrative functions between the different JUs to reduce costs is worth to be considered. The Commission would welcome proposals from the Executive Directors on these points. 5.5.2. The Commission takes note of the panel's view that interaction and exchange between projects should be promoted and that the planned Stakeholder General Assemblies will include the review of all FP7 on-going projects (see position on recommendations R3.1 and R3.2 above). However, the Commission underlines that all FCH JU projects are still at their early stages. 5.5.3. The Commission recognises the need for the establishment of a robust system for project monitoring and assessment. However, it notes that a proposal received in the call 2010 for the development of a framework for technology monitoring and assessments (TMA) is currently being negotiated and, if successful, could provide the necessary tools for assessing progress of projects and of the programme. 5.5.4. The Commission acknowledges the need to establish international benchmarking against comparable schemes in order to help guide targets for the selection of future projects. The TMA framework, to be developed in the above-mentioned proposal, should provide the necessary tool also to that endeavour. 6. Conclusions

The Commission acknowledges the thorough and in-depth work carried out by the Evaluation Panel and is satisfied that the panel recognises the validity of the JTI approach. It also appreciates the positive views about the future prospects of the FCH JU.

The Panel recommendations are considered very valuable for removing or at least reducing weaknesses identified in the current operations of the FCH JU and for improving its effectiveness and efficiency.

The Commission undertakes to implement corrective measures when appropriate within its powers of intervention. It also commits to work together with the other FCH JU stakeholders and bodies to address the recommendations appropriately.

[1]               First Interim Evaluation of the ARTEMIS and ENIAC Joint Technology Initiatives (30 July 2010), Clean Sky 1st Interim Evaluation (15 December 2010), First Interim Evaluation of the Innovative Medicines Initiative Joint Undertaking (20 December 2010), First Interim Evaluation of the Fuel Cell & Hydrogen Joint Undertaking (13 April 2011)

[2]               http://ec.europa.eu/dgs/information_society/evaluation/rtd/jti/

[3]               Council Regulation No 73/2008 of 20 December 2007 setting up the Joint Undertaking for the implementation of the Joint Technology Initiative on Innovative Medicines.

[4]               The expert panel was composed of the following experts: Dr Fred Gvillo (chairman) – formerly of Schering AG, Prof. Magdalene Rosenmöller (rapporteur) - IESE Business School, Tom Andersen – European Investment Bank, Prof. Manfred Horvath - Vienna University of Technology, Ruth Keir – Archea Ltd., Bart Wijnberg – former employee at the Dutch ministry of health.

[5]               The IMI-related documents and information consulted included the legal documents for setting up IMI, documents describing the functioning of the IMI JU, the JTI Sherpa report and published articles about IMI as well as information including statistics about the projects selected so far.

[6]               Designing together the 'ideal house' for public-private partnerships in European research- JTI Sherpa’s' Group – Final Report – January 2010, ftp://ftp.cordis.europa.eu/pub/fp7/docs/jti/jti-sherpas-report-2010_en.pdf

[7]               Council Regulation No 71/2008 of 20 December 2007 setting up the Clean Sky Joint Undertaking

[8]               The expert panel consisted of the following members: Dr. Michel Huguet (chairman) (FR), formerly Technical Director of the JET Joint Undertaking; Prof. Enzo Bertolini (IT), Director of the Astronomical Observatory of Valle d'Aosta, formerly Head of the Power Supply Division of the JET Joint Undertaking; Dr. Dietrich Eckardt (DE),Senior Engineering Consultant, Alstom Switzerland Ltd; Prof. Peter Hecker (Rapporteur) (DE),Managing Director - Institute of Flight Guidance, Technical University of Braunschweig; Dr. Ivonne Herrera (NO),Research Scientist, SINTEF; Prof. Manfred Horvat (AT), Vienna University of Technology ;

[9]               COM(2011) 48: Green Paper: From Challenges to Opportunities: Towards a Common Strategic Framework for EU Research and Innovation funding

[10]             "Designing Together the 'ideal house' for public-private partnerships in European research", JTI Sherpas' Report, Final Report - January 2010

[11]             Council Regulation (EC) No 521/2008 of 30 May 2008

[12]             The expert panel consisted of the following members: Elisabet Fjermestad Hagen (Chair) (NO), formerly Director for business development of new energy markets of Norsk Hydro ASA; John Loughhead (Rapporteur) (UK), Executive Director of the UK Energy Research Centre; Dr. Jens Rostrup-Nielsen (DK), formerly Executive Vice President (R&D) at Haldor Topsøe A/S and adjunct professor at the Technical University of Denmark; Maria Rosaria Di Nucci (IT), Associate senior research fellow at the Environmental Policy Research Centre of the Freie Universität Berlin; Ana Sofia Caires Branco (PT), Technology Transfer expert having worked in several energy and transport agencies and research centres (e.g. AREAM, CIDAUT; Prof. Manfred Horvat (AT), Vienna University of Technology.