GLOSSARY

BPM tool : Business process management tool.

CIP : The competitiveness and innovation framework programme is a Commission programme which supports innovation activities (including eco-innovation), provides better access to finance and delivers business support services in the regions.

DFG : The German Research Foundation (Deutsche Forschungsgemeinschaft) funds research projects in all fields of science and the humanities. It has an annual budget of almost 2,5 billion euro.

DG : Directorate-general.

DG Communications Networks, Content and Technology (CNECT) : The Directorate-General for Communications Networks, Content and Technology at the Commission; former Directorate-General for Information Society and Media (DG INFSO).

DG Energy (ENER) : The Directorate-General for Energy at the Commission.

DG Enterprise and Industry (ENTR) : The Directorate-General for Enterprise and Industry at the Commission.

DG Mobility and Transport (MOVE) : The Directorate-General for Mobility and Transport at the Commission.

DG Research and Innovation (RTD) : The Directorate-General for Research and Innovation at the Commission.

EIB : European Investment Bank.

EMI database : The "Experts Management Back-office Internal for FP7" database is the major source of information used by research family Commission DGs for expert identification as well as checks for possible conflict of interest.

ERA : The European Research Area is composed of all research and development activities, programmes and policies in Europe which involve a transnational perspective. Together, they enable researchers, research institutions and businesses to increasingly circulate, compete and cooperate across borders. The overarching aim is to improve the organisation and coordination of the research effort in Europe so that transnational synergies and complementarities are fully exploited.

ERCEA : The European Research Council Executive Agency was created to manage exclusively the Ideas Specific Programme, following the strategic directions of the European Research Council.

Europe 2020 : Europe 2020 is the EU's growth strategy for the coming decade and aims to make the EU a smart, sustainable and inclusive economy.

Ex ante controls : Ex ante controls are preventive controls carried out either before grant signature or before payment.

Ex post controls : Ex post controls are detective controls, which are carried out after payment has been made.

FP7 : The seventh framework programme for research, technological development and demonstration activities is the European Union's main instrument for supporting research and innovation. The broad objectives of FP7 have been grouped into four specific programmes: Cooperation, Ideas, People and Capacities.

Horizon 2020 : Horizon 2020 is the EU’s new programme for research and innovation. It is FP7’s successor programme and will run from 2014 to 2020. For this period, the Commission has proposed a budget of 80 billion euro.

ICT : "Information and communication technologies" is one of the themes under the Cooperation Specific Programme.

JTIs : Joint technology initiatives are long-term public-private partnerships. They combine private sector investment with European public funding. JTIs have the legal status of joint undertakings within the meaning of Article 187 TFEU.

KBBE : "Food, agriculture and fisheries, and biotechnology" is one of the themes under the Cooperation Specific Programme.

KPI : Key performance indicator.

NSF : The National Science Foundation is an independent US federal agency, whose aim is to promote the progress of science, to advance national health, prosperity and welfare, and to secure national defence. It has an annual budget of about 6,9 billion dollars (5,2 billion euro, 2010 figures).

NMP : "Nanosciences, nanotechnologies, materials and new production technologies" is one of the themes under the Cooperation Specific Programme.

Participant Guarantee Fund : The Participant Guarantee Fund represents collateral on which the Commission can draw in case of financial losses linked to FP7 projects. It is owned by the beneficiaries, which contribute 5 % of the total EU contribution for their projects to the fund.

RCC : The Research Clearing Committee is set up between the directorates-general responsible for the implementation of indirect actions under FP7. The RCC has the mandate to take final positions on horizontal matters related to the implementation on which the services did not reach consensus.

RDI : Research, development and innovation

REA : Research Executive Agency.

RES : The Research Enquiry Service is an e-mail service that answers questions about any aspect of European research.

RSFF : The Risk Sharing Finance Facility is a financial instrument designed to improve access to debt financing of RDI investments on acceptable terms for private companies or public institutions promoting activities in the RDI field.

Rules for rarticipation : Rules for rarticipation refer to the collection of documents which together set out the conditions for funding research under FP7. It includes Regulation (EC) No 1906/2006 laying down the rules for participation, the implementing rules adopted by the Commission and non-binding guidance documents.

SME : Small and medium-sized enterprise.

SNF : The Swiss National Foundation (Schweizerischer Nationalfonds) is a Swiss federal agency for the promotion of scientific research. It has an annual budget of around 700 million Swiss francs (580 million euro).

TSB : The Technology Strategy Board is a UK innovation agency, whose aim is to accelerate economic growth by stimulating and supporting business-led innovation. It has an annual budget of around 250 million euro.

TTG : "Time to grant" is the length of time required to successfully select a proposal and negotiate a contract.

TTP : "Time to pay" is the length of time required to make a payment to a consortium of beneficiaries.

SUMMARY

I. The seventh framework programme for research, technological development and demonstration activities is one of the Union’s key instruments for funding research and is the EU’s contribution to the Europe 2020 strategy. It aims to strengthen industrial competitiveness and to meet the research needs of other Union pol icies, thereby contributing to the creation of a knowledge-based society. It covers the period 2007–13 and its total budget amounts to more than 50 billion euro. The vast majority of the budget is spent by the Commission directorates-general or its executive agencies in the form of grants to final beneficiaries (see paragraphs 1 to 13).

II. In order to examine whether the Commission has ensured efficient implementation of FP7, the Court’s audit addressed the following questions (see paragraphs 14 to 16):

(a) Have the rules for participation allowed efficient implementation of fp7?

(b) Has the Commission succeeded in streamlining the FP7 processes without compromising the quality of spending?

(c) Did the Commission manage well the setting up of new instruments?

III. Over the course of FP7 the Commission has introduced a number of changes which have simplified the rules for participation. In particular, the Commission has rationalised the requirements and improved its guidance documents for beneficiaries in a satisfactory manner. The Commission has been able to align FP7 provisions with beneficiaries’ practices in some cases but more needs to be done in the future. FP7 beneficiaries are faced with inconsistencies related to some aspects of the rules for participation. The establishment of the Research Clearing Committee is a step in the right direction to tackle these inconsistencies; however the mechanisms for identification of diverging practices are weak (see paragraphs 17 to 44).

IV. The Commission’s processes are geared to ensuring that funding is invested in high-quality research; however, there has been less focus on efficiency. The audit found that:

(a) the Commission’s management of FP7 processes is strong in three out of five areas, i.e. process design, improvement activities and management information, but less so as regards tools and resources. The existing tools do not allow efficient implementation and there are indications that too many staff resources are used for the implementation of certain themes under the Cooperation Specific Programme at the expense of other themes (see paragraphs 45 to 55).

(b) whilst processing times have shortened over the course of FP7, they have only approached 9 months in 2012. The audit has highlighted good practices to further shorten time to grant. Management attention to this problem and the existence of an environment which allows for the effective sharing of good practices are key (see paragraphs 56 to 66).

(c) the quality controls on the selection and follow-up of the projects are functioning well. However the FP7 financial control model does not sufficiently take into account the risk of errors. This means that low-risk FP7 beneficiaries are subject to too many controls (see paragraphs 67 to 83).

V. The Court’s examination of the Risk Sharing Finance Facility and the joint technology initiatives showed that both instruments have met the needs for which they were created. They have been successful in attracting specific groups of beneficiaries such as SMEs. However the implementation of joint technology initiatives has suffered from an overly complex legal framework and the Commission has not sufficiently demonstrated that funding provided by the Risk Sharing Finance Facility leads to investments above the level that beneficiaries would have undertaken without public money (see paragraphs 84 to 97).

VI. On the basis of these observations, the Court makes a number of recommendations addressed mainly to the Commission (see paragraphs 98 to 103):

(a) With regard to the rules for participation, the Commission should make further efforts to ensure that beneficiaries’ practices can be used in Horizon 2020 and manage FP7 in a more consistent manner.

(b) To strengthen process management, the Commission should deploy IT tools which will integrate all functionalities and it should examine the imbalances in staff workload.

(c) To reduce processing times, the Commission should make sure that the processes are automated and implemented consistently across its services.

(d) The Commission should make its control activities before and after payment more risk-driven, so as to better focus its control effort.

(e) The budgetary authorities and the Commission should bring the legal framework of the joint technology initiatives more into line with their staff complement. To maximise the impact of the Risk Sharing Finance Facility, the Commission should improve its targeting of those beneficiaries which have limited access to finance.

INTRODUCTION

BACKGROUND INFORMATION

1. The seventh framework programme for research, technological development and demonstration activities (FP7) represents one of the Union’s key instruments for funding research and is its contribution to the ​Europe 2020 strategy. It aims to strengthen industrial competitiveness and to meet the research needs of other Union policies [1] thereby contributing to the creation of a knowledge-based society, building on a European Research Area and complementing activities at national and regional level [2].

2. FP7 is made up of several funding schemes grouped into four specific programmes (see Figure 1), which are designed to support research-related activities including basic and applied research and educational and innovation activities. FP7 provides more than 50 billion euro [3] in support of these activities in the 2007–13 period. Most of the funds are spent on grants to final beneficiaries in the Member States as well as beyond the EU. The activities to be funded by the grants are determined on the basis of competitive calls for proposals. The beneficiaries usually work as consortia of partners on the basis of grant agreements with the Commission.

FIGURE 1

FP7 BUDGET 2007–13 (IN BILLION EURO)

+++++ TIFF +++++

Note: Dashed border lines indicate instruments which were newly introduced in FP7.

* A dedicated specific programme supports the non-nuclear activities of the Joint Research Centre, which is a directorate-general directly carrying out research activities for the Commission.

3. FP7 is a complex programme for funding research. Its annual budget increases year-on-year and is expected to approach 11 billion euro in 2013. This, together with the distinctive characteristics of FP7 (see Box 1), makes it one of the largest single research programmes in the world.

4. These characteristics have a direct impact on the way FP7 has to be managed. For instance, due to the transnational nature of FP7 and the combination of academic and industrial entities in individual projects, participants within the same consortium may be subject to different tax, legal, accounting or financial rules. The FP7 rules thus have to be sufficiently flexible to accommodate these differences and at the same time be precise enough to avoid misinterpretations. The rules of the national research funding schemes only need to take into account relevant national legislation.

BOX 1

CHARACTERISTICS OF FP7

FP7 serves beneficiaries from all EU Member States, associated and accession countries (14 [4]) as well as international cooperation partner countries.

It is managed by 19 independent entities: eight Commission services (six directorates-general and two executive agencies), the European Investment Bank, five joint technology initiatives and five Article 185 initiatives [5].

In the eight Commission services, more than 2500 staff members are employed to implement FP7.

The average EU contribution per project is 1,75 million euro. A typical project involves transnational collaboration between five or six participants. For collaborative projects, which represent around two thirds of the FP7 budget, the average number of participants is 11. The consortia in most projects are made up of participants from both the academic world and the private sector.

Every year, around 15000 proposals are evaluated, 2500 new grant agreements are concluded and around 10000 payments are made. The FP7 portfolio consists of around 14000 grant agreements (2011 figures).

Source: Fifth FP7 monitoring report, 2012 general budget of the EU and Commission communication on simplifying the implementation of the research framework programmes (COM(2010) 187).

5. FP7 has introduced a number of new instruments compared to FP6. The most important ones are support for frontier research [6], joint technology initiatives (JTIs) and the Risk-Sharing Finance Facility (RSFF). Support for frontier research was introduced to fund basic "blue-sky" research [7], whereas the other two instruments were conceived to support close-to-the-market innovation activities with the aim of fostering industry participation, which has steadily decreased since FP4 [8].

REGULATORY ENVIRONMENT

6. The objectives and activities to be financed under FP7 are defined in Decision No 1982/2006/EC and the decisions on four specific programmes. The FP7 regulation [9] sets out the rules for participation. This framework is complemented by the implementing rules adopted by the Commission and non-binding guidance documents (hereafter the term "rules for participation" will refer to the hierarchical structure in Figure 2).

7. The rules for participation define the conditions for project selection, participation of entities in the projects and financing of projects. Together they determine the funding conditions such as funding rates and cost eligibility requirements.

8. The current rules for participation are the result of the experience accumulated over 20 years of previous framework programmes. Simplification became the main driving force under FP6 [10] and has been a commitment for the Commission under FP7 [11].

FIGURE 2

HIERARCHY OF FP7 RULES FOR PARTICIPATION

+++++ TIFF +++++

Source: European Court of Auditors, CORDIS.

IMPLEMENTATION

9. The implementation of 58 % of FP7 by value falls under centralised direct management [12] by six directorates-general. For the remainder implementation has been delegated to two executive agencies (the Research Executive Agency (REA) and European Research Council Executive Agency (ERCEA)), the EIB, the JTIs and dedicated implementation structures for Article 185 initiatives. While the executive agencies follow the same procedures as the Commission, the other entities can organise their internal processes differently. A breakdown by management mode and entities responsible is provided in Figure 3.

FIGURE 3

ENTITIES RESPONSIBLE FOR IMPLEMENTATION

+++++ TIFF +++++

Source: European Court of Auditors.

10. Regardless of the implementing entity, grant implementation is typically accomplished by means of the following three main processes:

(a) identification of the scientific topics to be supported and setting out the work programmes for their implementation;

(b) project selection, typically based on a competitive selection process, and negotiation with successful applicants; and

(c) control of the scientific implementation of projects and reimbursement of the beneficiaries for their eligible costs.

11. Simplification under FP7 has not only concerned the rules, but also the internal processes of the DGs and executive agencies. Since the Court’s last audit in this field [13] in 2004, the Commission has introduced several new systems designed to simplify the grant implementation processes (e.g. participant portal, unique registration facility, negotiation facility).

NEW INSTRUMENTS UNDER FP7

12. The JTIs are long-term public–private partnerships. They combine private-sector investment with EU and national public funding. They aim to support cooperative research across Europe in fields of industrial research where there are clearly identified common technological and economic objectives. The EU contribution under FP7 for the JTIs is more than 3 billion euro. Bipartite JTIs are partnerships between the EU and industry associations, whereas tripartite JTIs include the EU, industry associations and participating Member States (see Table 1). For instance, the European Joint Undertaking for the implementation of the Joint Technology Initiative on Nanoelectronics (ENIAC) combines the Commission, participating Member States [14] and the Association for European Nanoelectronics Activities.

13. The RSFF is a financial instrument designed to improve access to debt financing of research, development and innovation (RDI) investments on acceptable terms for private companies or public institutions promoting activities in the RDI field. Target beneficiaries of the RSFF also include European research-intensive entities and research infrastructures. The European Commission, through DG Research and Innovation, monitors the RSFF in terms of eligibility of projects and budget allocation from FP7, while the EIB is in charge of the daily operations. The instrument is jointly financed by the EU and the EIB. The maximum EU contribution under FP7 for the RSFF was set at one billion euro.

TABLE 1

OVERVIEW OF THE JOINT TECHNOLOGY INITIATIVES

Source: Council decisions establishing the JTIs.

Joint technology initiative | Strategic objective | Maximum EU contribution (in million euro) |

Bipartite JTIs | Innovative Medicines Initiative (IMI) | To speed up the development of better and safer medicines for patients | 1000 |

Clean sky | To develop breakthrough technologies to significantly increase the environmental performance of airplanes and air transport, resulting in less noisy and more fuel- efficient aircraft | 800 |

Fuel, Cells and Hydrogen Joint Undertaking (FCH) | To accelerate the market introduction of fuel cell and hydrogen energy technologies in Europe | 470 |

Tripartite JTIs | ENIAC | Focusing on nanoelectronics to enhance the further integration and miniaturisation of devices and increase their functionalities | 450 |

Artemis | To tackle the research and structural challenges in the area of embedded computing systems | 420 |

AUDIT SCOPE AND APPROACH

14. The audit examined whether the Commission has ensured efficient implementation of FP7. This was addressed by the following three subquestions:

(a) Have the rules for participation allowed efficient implementation of FP7?

(b) Has the Commission succeeded in streamlining the FP7 processes without compromising the quality of spending?

(c) Did the Commission manage well the setting up of new instruments?

15. The audit covered the implementation of FP7 between 2007 and the first half of 2012. The audit criteria and the audit evidence collection methods are described in Annex I. The audit scope encompassed:

(a) for the first audit subquestion, the rules for participation for the entire FP7;

(b) for the second subquestion, internal processes established within DG Research and Innovation, DG Information Society and Media, DG Enterprise and Industry and REA and which were related to collaborative projects under the Cooperation Specific Programme [15]. In this way the analysis of internal processes covered the implementation of almost two thirds of the FP7 budget; and

(c) for the third subquestion, an analysis of two new instruments — JTIs and the RSFF.

16. The Court carried out a follow-up of its previous recommendations in "Special Report No 1/2004 on the management of indirect RTD actions under the Fifth Framework Programme for research and technological development (1998 to 2002)". The results including the references to the corresponding observations in this report are presented in Annex II.

OBSERVATIONS

THE COMMISSION HAS TAKEN A NUMBER OF STEPS TO SIMPLIFY THE RULES FOR PARTICIPATION, BUT MORE NEEDS TO BE DONE

17. At the inception of FP7 the Commission put forward a series of simplification plans affecting various aspects of the rules for participation. This part of the report uses the following principles as a basis for the assessment of the progress made in pursuing this simplification agenda:

(a) rationalising requirements by striking a better balance between risks and control;

(b) optimising the methods of reimbursing the beneficiaries for their research efforts;

(c) guiding the beneficiaries through the complexity of the scheme;

(d) aligning provisions with beneficiaries’ practices; and

(e) avoiding duplications and inconsistencies.

18. These principles were derived from various Commission documents [16] presenting plans for simplification and from the legislative authorities’ positions [17] relating to the rules for participation. Thus these principles have been driving the evolution of the rules for participation since 2005, when the Commission launched its preparation of FP7. For an overview of the progress in the simplification agenda refer to Table 2.

TABLE 2

PROGRESS WITH THE COMMISSION’S SIMPLIFICATION AGENDA

Source: European Court of Auditors.

Principles with examples of planned measures | Court's assessment |

Rationalising requirements: Replacement of bank guarantees by the Participant Guarantee FundLess use of audit certificates for payment requestsRationalisation of the reporting periods | Satisfactory Bank guarantees are no longer required, relieving the beneficiaries of the associated administrative burden. As a result of the introduction of a threshold, only 6 % of cost claims, representing a third of funded costs in FP7, require an audit certificate. In some aspects (e.g. length of reporting periods) FP7 compares favourably with national funding schemes. However, further rationalisation of reporting requirements is possible, if: reporting periods are also extended for projects under the ICT theme; andthe size of the reports is subject to strict limits, as in some national funding agencies (SNF, DFG, TSB). |

Optimising the methods of reimbursement: Broader use of flat ratesSimplifying the funding model based on actual costs | Partly satisfactory FP7 includes new methods for funding based on flat rates rather than actual costs. The FP7 funding model based on actual costs remains complex (see paragraph 20). |

Guiding the beneficiaries through the complexity of the scheme: Improvement of guidance materialsCommon electronic tools for interaction with beneficiariesAdditional guidance tools such as help desks | Satisfactory The guidance documents have become shorter and clearer compared to those of FP6, as confirmed by the beneficiaries’ responses to the Court’s survey. New IT tools facilitate interaction between the Commission and beneficiaries. Specialised help desks for beneficiaries have been introduced. |

Aligning provisions with the beneficiaries’ practices: Certification of beneficiaries’ costing methodologiesBroader acceptance of beneficiaries’ methodologies | Not satisfactory Take-up of the certification scheme by the beneficiaries was low, acceptance of the average personnel cost methodologies came late, beneficiaries have to bear the full risk of incorrect interpretation of FP7 provisions and FP7 provisions do not always reflect general practice in the research field (see paragraphs 21 to 29). |

Avoiding duplications and inconsistencies: An electronic registration desk for beneficiaries (unique registration facility)Consistent assessment of financial viabilitySingle clearing house (research enquiry service)Research Clearing Committee | Not satisfactory Most of the planned measures were introduced, but implementation is still characterised by inconsistencies (see paragraphs 30 to 44). |

GOOD PROGRESS WITH THE SIMPLIFICATION AGENDA

19. With regard to the five principles, the Commission has rationalised the requirements and improved the various guidance documents in a satisfactory manner. The Commission has been able to align FP7 provisions with beneficiaries’ practices in some cases (for example to accept average personnel costs) but more needs to be done in the future. FP7 beneficiaries are faced with inconsistencies related to some aspects of the rules for participation (see paragraphs 21 and 30 respectively).

20. As regards the funding model new modalities for funding based on flat rates instead of actual costs were introduced. The financing based on actual costs remains the main funding model in FP7. Compared to national funding agencies, the FP7 funding model is complex on account of the high number of combinations of cost categories, methods for calculating costs and reimbursement rates. This is a burden for the beneficiaries and has a number of important consequences for the administrative processes in the Commission. The Court recommended the simplification of the funding model in its Report 1/2004 (see Annex II). Simplification of the funding model is an important element of the proposal for the rules for participation in Horizon 2020 [18].

THE OBJECTIVE OF IMPROVING ALIGNMENT OF FP7 PROVISIONS WITH BENEFICIARIES’ PRACTICES HAS NOT BEEN FULLY MET

21. At the inception of FP7, the Commission indicated that the funding model based on actual costs would make use of the participants’ usual accounting and management principles [19]. The legislative authorities and the Commission itself have recently underlined the need to make FP7 rules more compatible with general business practices [20].

BETTER ALIGNMENT WITH BENEFICIARIES’ PRACTICES IS REQUIRED

22. To improve alignment of FP7 rules for participation with beneficiaries’ practices, the rules should provide for a mechanism to recognise these practices in good time and adjust interpretations and practices accordingly, provided this does not compromise the general and control objectives of FP7.

23. At the beginning of FP7 the Commission established a certification mechanism for the approval of cost methodologies to give assurance to beneficiaries on their methodologies. However, the stringent requirements have led to a low take-up of the certification scheme by the beneficiaries. The Commission has estimated that a maximum of 350 certificates would be issued [21]. As at March 2012 the Commission had accepted applications for certification of methodologies from 71 beneficiaries, which represents 20 % of such beneficiaries. The results of the Court’s survey indicate that the major reasons for not applying for ex ante certification are the length and complexity of the application process, the difficulty of meeting the certification criteria and the limited use of the certificate. The low participation undermined the Commission’s efforts to simplify procedures [22].

24. To facilitate the alignment of FP7 provisions with beneficiaries’ practices related to personnel costs, in 2011 the Commission introduced the possibility for them to use their internal practices for charging average personnel costs under certain conditions. The ECA’s interviews with research organisations and participants revealed a positive reaction to the wider acceptance of the average personnel cost methodologies. However, even though the problem with the acceptance of the average personnel costs was recognised relatively early [23], it took 21 months before the rules were modified. This is late for a 7-year programme that finances projects lasting on average 36 months.

25. Even if the FP7 financial rules are less prescriptive and more principles-based, there are situations where it is difficult to reconcile the reality faced by a beneficiary with FP7 principles. If the beneficiary encounters such a situation, three options are available. The beneficiary can either:

contact its project officer;

refer to the national contact point [24], which will forward the request to the Research Enquiry Service (RES); or

send it directly to the RES.

26. Project officers can only refer to existing rules and guidelines. The RES can provide more tailored guidance, which in case of complex legal questions is prepared by the legal department of DG Research and Innovation. However this is still subject to a legal disclaimer, which may significantly undermine its utility in case of later disagreements with the Commission. The correctness of the interpretation can be called into question up to 5 years after the closure of the project. Even though there may be valid reasons for having a legal disclaimer, the current system means the beneficiaries bear the full risk of incorrect interpretation of FP7 provisions.

FP7 PROVISIONS DO NOT ALWAYS REFLECT THE GENERAL PRACTICE OF BENEFICIARIES

27. To assess the extent of alignment of FP7 with general practice of beneficiaries, the Court compared FP7 practice with the practices used in national funding agencies and with the internal practices of the beneficiaries. The Court’s analysis focused on personnel costs as they often represent the most important category of direct costs incurred when carrying out a project.

28. Some national funding agencies accept lump sums or cost categories without time recording requirements (e.g. German Research Foundation, DFG) or use budgeted costs based on salary categories and predefined working time (Swiss National Science Foundation, SNF). Others follow time recording requirements similar to FP7’s (e.g. Department of Energy in the United States) and focus on a verifiable time allocation system for justifying personnel costs. However, in some cases national funding agencies’ rules make greater allowance for the beneficiaries’ usual management and accounting practices.

29. The Court’s survey of beneficiaries shows that, despite the positive reaction to the wider acceptance of the average personnel cost methodologies, the FP7 requirements relating to personnel costs are mostly not compatible with the beneficiaries’ practices In respect of time recording, 29 % of respondents confirmed that they are not able to use their internal system to fulfil the Commission requirements (see Figure 4). 30 % implemented a time recording system for participating in EU projects. Some beneficiaries indicated that for their internal purposes they use approaches other than time records (see Box 2).

FIGURE 4

TIME RECODING PRACTICES OF FP7 BENEFICIARIES

+++++ TIFF +++++

Source: European Court of Auditors.

BOX 2

EXAMPLES OF DIFFERENT APPROACHES TO FULFIL THE TIME RECORDING REQUIREMENTS

Output-based approach: specific research unit staff are fully assigned to research tasks, no time sheets are required. Method(s) of monitoring are for example progress of work, quality of results or achievement of milestones.

Alternative-evidence approach: sufficient evidence is obtained through work contracts (e.g. PhD graduate 100 % assigned to a specific project) or by internal declaration confirming full assignment to a project or task (e.g. for a specific period of time).

Source: European Court of Auditors, based on a review of the responses given to its questionnaire.

UNNECESSARY INCONSISTENCIES REDUCE EFFICIENCY

30. It is not always possible to ensure uniformity in implementation between programmes which might have different objectives. However, there should be only a limited number of differences in implementation of the same funding instrument within the same specific programme. At the inception of FP7, the Commission declared coherence to be one of three principles of FP7 implementation [25]. It also stated: "Uniform interpretation, particularly relating to legal and financial provisions of projects, is to be ensured across all of the Commission services" [26].

31. Coherence in implementation reduces the administrative burden placed on beneficiaries. It makes it easier for beneficiaries to be aware of the requirements and to respect them and for the Commission to manage the programme.

DESPITE THE INTRODUCTION OF SOME HARMONISATION MEASURES ...

32. To ensure more coherent implementation, the Commission set out in the preparatory acts [27] to:

establish an electronic registration desk (unique registration facility);

ensure consistent assessment of the financial viability of a participant; and

establish a single clearing house.

33. The unique registration facility prevents beneficiaries from being repeatedly asked to submit the same information on their legal status and financial situation for each project. The initial backlog, caused by many beneficiaries being registered at the same time, was cleared and currently there are no significant delays. According to the FP7 interim evaluation the unique registration facility has been a success [28]. The financial validation check is now performed by the central validation team in REA, which ensures consistency by using the same financial ratios [29].

34. A single clearing house [30] has not been established. Instead, the Commission first established the Research Enquiry Service to answer beneficiaries’ questions. In 2011, the Commission established the Research Clearing Committee (RCC) with a mandate to take final positions on horizontal matters relating to implementation on which the services cannot reach consensus [31].

… SOME ASPECTS OF FP7 IMPLEMENTATION ARE AFFECTED BY A LACK OF COHERENCE

35. Despite this progress, some aspects of FP7 implementation are affected by a lack of coherence. Considering only the collaborative projects in the Cooperation Specific Programme, the Court’s comparison of rules and procedures revealed practices that led to differences in treatment of essentially the same situation by different services or even by different units within a service (see Box 3).

BOX 3

EXAMPLES OF DIFFERING PRACTICES

1. Use of flat rates to cover travel-related subsistence costs — not accepted by DG Communications Networks, Content and Technology but accepted by other services

2. Hearings during evaluation of large projects — used by DG Communications Networks, Content and Technology, DG Enterprise and Industry and REA, but not DG Research and Innovation

3. The requirement to conclude and provide a consortium agreement — varies from theme to theme

4. Treatment of unforeseen subcontracting — varies from service to service

5. Definition of the management budget — varies from service to service

6. Procedure followed if the beneficiary changes its legal name — formal amendment in DG Communications Networks, Content and Technology only

7. Frequency of reporting — more frequent reporting in DG Communications Networks, Content and Technology than in other services

8. Use of scientific review for project monitoring — more common in DG Communications Networks, Content and Technology than in other services

9. Possibility to execute multiple payments per period — partial payments are used in DG Research and Innovation and DG Enterprise and Industry, but not in DG Communications Networks, Content and Technology and REA

Differing practices which were discussed and resolved at the RCC

10. Eligibility of costs for a certificate on financial statements, which was not mandatory

11. Treatment of adjustment in favour of beneficiary for closed projects

Source: European Court of Auditors. This list is based on the analysis of rules, internal Commission documents, responses of beneficiaries and interviews with the Commission managers.

36. The Court’s survey shows that the beneficiaries consider that there is a lack of coherence in FP7 implementation: 51 % of beneficiaries indicated that they were at least occasionally subject to different treatment which was not justified by the legal framework or particularities of the specific programme and 15 % of beneficiaries indicated that they experienced such differences in treatment in almost every project. The beneficiaries’ comments indicate that interpretation can vary depending on the project officer (see Box 4).

MECHANISMS TO IDENTIFY DIVERGING PRACTICES ARE WEAK

37. By September 2011, the RCC had discussed and resolved three topics related to better coordination between the services. Two were directly related to harmonisation of FP7 procedures. However, the Court’s analysis (see Box 3) identified nine examples of divergent rules and procedures, which were not discussed in the RCC. This suggests that even though the RCC is effective in finding uniform positions, the mechanisms put in place do not identify all divergent procedures in FP7 implementation.

BOX 4

EXAMPLES OF BENEFICIARIES’ COMMENTS ON DIFFERING PRACTICES

For every project, negotiations follow the rules as interpreted by each project officer. As a result, negotiations vary according to this interpretation and the lead organisation has to adapt and respond to the instructions as given by the project officer.

It has happened in some situations that different project officers of different units have given different treatment to the same situation.

The details requested in relation to the content of the application, technical (deliverables, milestones, progress reports) and financial reporting, as well as the use of FP7 IT tools and organisation of the processes, vary depending on the project officer. This applies for the all phases of project implementation.

Source: European Court of Auditors' questionnaire.

38. There are two main channels through which an issue can be brought to the attention of the RCC:

decision of a Commission internal working group or by directors general of the research family or their representatives; and

query posted by a beneficiary on the RES website.

39. In both the RCC relies on the active engagement of Commission staff or beneficiaries. There was only a limited attempt to map out all possible differences in the current FP7 procedures. There is potential to further harmonise FP7 procedures, as some will be in use up to 5 years after 2013 given the multiannual nature of many projects.

EXPERIENCE WITH FP7 SHOWS HOW DIFFICULT IT WILL BE TO PURSUE SIMILAR OBJECTIVES LINKED TO THE EUROPEAN RESEARCH AREA

40. The framework programme constitutes only a fraction of the public spending on research in Europe [32]. The basic notion lying behind the European Research Area (ERA) is that the research effort in Europe must be better organised and coordinated. Alignment of rules is one of the aspects related to better coordination within ERA. In relation to ERA, the FP7 rules for participation should provide a coherent framework to ensure the most efficient implementation possible [33]. FP7 should be implemented in such a fashion that mutual synergies and complementarity with Union policies and programmes are ensured [34].

41. FP7 includes instruments such as JTIs which have their own corpus of rules and procedures for calls for proposals and financial provisions (e.g. eligibility criteria or funding rules). Similarly Article 185 initiatives may adopt their own set of rules, which should be similar to FP7, but can still deviate in their detail.

42. As regards other programmes financed or co-financed by the Union, separate sets of rules for participation are used. The competitiveness and innovation framework programme (CIP) rules are based on different eligibility criteria, funding models, cost schemes and procedures ,etc. The shared management mode used for Structural Funds limits the Commission’s freedom of action to impose any alignment of rules which are set by the regions or Member States, with the rules for participation in FP7. However, there are still instances where more synergies could be achieved. Both the Structural Funds and FP7 allow for approval of cost methodologies for staff and indirect costs, but the Commission approval granted in one scheme is not valid for another. Non-recoverable VAT is eligible under Structural Funds, but not under FP7. While certain FP7 beneficiaries [35] can benefit from a flat rate of 60 % to cover their indirect costs, for the Structural Funds the relevant regulation [36] caps the flat rate for indirect costs at 20 %.

43. Some of the issues identified are expected to be resolved under Horizon 2020 when CIP and FP7 are merged. However, Structural Funds cover a wide range of activities, not necessarily related to research, and harmonisation for research activities could lead to additional inconsistencies with other activities.

44. The case of FP7 shows how difficult it is to achieve the ERA objective of better coordination through alignment of rules. FP7 beneficiaries are faced with differences which arise from the relative autonomy of either the Commission DGs or of the other implementing entities. Moreover, the funding agencies for Structural Funds or national programmes enjoy an even higher level of autonomy. It will therefore be very difficult to obtain a harmonisation of rules, considering that it has not been fully achieved in the context of the programmes where the Commission is in a decision-making position.

ATTENTION HAS FOCUSED MOSTLY ON ENSURING HIGH-QUALITY SPENDING, LESS ON EFFICIENCY

THE COMMISSION HAS DEVISED GOOD PROCESSES BUT HAS NOT FULLY ALIGNED THE RESOURCES TO SUPPORT THEIR EXECUTION

45. Good processes [37] make it easier to improve the efficiency of an organisation. To assess performance in this area, the audit focused on five elements [38] which characterise strong process management: process design, improvement, tools, information and staff (see Figure 5).

FIGURE 5

FIVE ELEMENTS OF STRONG PROCESS MANAGEMENT

+++++ TIFF +++++

Source: European Court of Auditors.

PROCESS DESIGN, IMPROVEMENT ACTIVITIES AND MANAGEMENT INFORMATION ARE STRONG AREAS WITH FEW WEAKNESSES

46. All services use the process approach for the management of their operational activities. Procedures and the roles of all users involved in the internal processes have been documented and disseminated. All services have established internal committees or assigned staff resources, which are dedicated to pursuing improvement activities. A number of improvement activities have taken place since the inception of FP7 and are ongoing. Examples of good practice in process design and improvement exist across all services.

47. Information is also one of the stronger areas. In line with the Court’s previous recommendation (see Annex II), the Commission has established a system for collecting a set of FP7-wide key performance indicators, which form the backbone of the yearly FP7 monitoring reports. For internal purposes, process-related KPIs or other management information such as budget consumption, time to pay, time to grant and staff vacancies are reported to managers on a quarterly, monthly or even real time basis. However, certain important metrics such as workload or use of staff resources are not collected in a way which would allow an easy comparison across services. Furthermore, the interdependencies between the different process metrics have not been sufficiently analysed, such as the benefit in terms of lower error rates or better follow-up of projects if more frequent reporting or scientific reviews are used [39].

EXISTING TOOLS DO NOT ALLOW EFFICIENT EXECUTION OF PROCESSES

48. The processes should be supported by an integrated IT system [40]. The systems should be interoperable and possible synergies between the DGs' information systems should be exploited [41]. Therefore, the audit focused on how the front-office, back-office and business process management (BPM) IT tools [42] support FP7 implementation and to what extent tools common to the whole research family exist.

49. On the front-office side, the tools have been improved by the participant portal and interaction with beneficiaries occurs mostly through the same tools across the research family. However, corresponding back-office tools have not been harmonised to the same extent. The maintenance and further development of multiple back-office tools, which have essentially the same purpose, is not an economical use of resources.

50. The BPM tools are the least developed with the exception of iFlow in DG Communications Networks, Content and Technology, which is an electronic workflow tool that interacts with the other back-office tools and automates the processes. This tool contributes to the shortening of the time taken by processes (see paragraph 63). The other services (DG Research and Innovation, DG Enterprise and Industry and REA) rely more on paper workflows or local tools covering specific phases of grant implementation or use Excel spreadsheets.

51. The different tools used for grant management have been developed as largely independent modules without considering how they should interlink with each other. This has created a need to establish synchronisation mechanisms between the systems. Furthermore, if a manager wants to get an overview of the projects in the portfolio under their responsibility, they must often manually compile data from up to six different systems.

52. Although FP7 electronic tools generally compare favourably with those of most national funding agencies visited, some national tools are examples of good practice. For instance, the grant management tool of the Swiss National Research Foundation (SNF) integrates all front-office and back-office functionalities. This reduces complexity and data redundancy and improves just-in-time data management. SNF and the National Science Foundation (NSF) use external data sources such as citation databases to improve the accuracy of their systems.

VARIATIONS IN THE UTILISATION OF STAFF RESOURCES ACROSS THE SERVICES NEED TO BE EXPLAINED

53. The allocation of staff should be based on the DG's objectives and priorities. Whenever necessary — at least once a year — management should align the organisational structures and staff allocations with priorities and workload [43].

54. In the eight Commission services, more than 2500 staff members are employed to implement FP7, out of whom 1500 (60 %) are directly assigned to manage the implementation of the Cooperation Specific Programme. This proportion corresponds to the share of budget allocated to this programme (61 %). However, within this programme, there are indications that the utilisation of staff resources varies significantly from one theme to another (see Table 3), even though the complexity of the project management is not significantly different.

TABLE 3

STAFF UTILISATION INDICATORS FOR THEMES IN THE COOPERATION SPECIFIC PROGRAMME

Note: The staff utilisation indicators were calculated on the basis of all staff allocated to the themes according to information provided in the annual activity reports.

Source: European Court of Auditors.

Theme | Budget per staff member (in million euro) | Contracts per staff member | Participants per staff member |

Energy DG Research and Innovation, DG Energy) | 20,4 | 2,2 | 25 |

Environment (DG Research and Innovation) | 13,8 | 2,5 | 39 |

Health (DG Research and Innovation) | 35,7 | 4,1 | 48 |

ICT (DG Communications Networks, Content and Technology) | 12,5 | 2,1 | 20 |

KBBE (DG Research and Innovation) | 16,0 | 2,9 | 41 |

NMP (DG Research and Innovation) | 24,5 | 3,6 | 48 |

Social sciences and humanities (DG Research and Innovation) | 9,9 | 2,8 | 29 |

Space and security (REA, DG Enterprise and Industry) | 22,5 | 2,7 | 32 |

Transport (DG Research and Innovation) | 35,0 | 3,8 | 47 |

55. The Commission does not have a sufficient basis for comparing in a transparent and objective way the differences in staff utilisation between the themes in the Cooperation Specific Programme.

PROCESSES CAN BE SHORTENED FURTHER

56. The DG's processes and procedures used for the implementation and control of its activities should be efficient [44]. One of the key characteristics of process efficiency is timeliness. The importance of timeliness was stressed in the FP7 decision: "Participation in FP7 should be facilitated through (...) the appropriate use of (...) quick procedures" [45], and also in the rules for participation: "It is necessary that the Union financial contribution reaches the participants without undue delay" [46]. For this reason, this section focuses on two key time indicators related to FP7 processes:

(a) time to grant (TTG) — the length of time required to successfully select a proposal and negotiate a contract; and

(b) time to pay (TTP) — the length of time required to make a payment to a consortium of beneficiaries.

TIME TO GRANT IS DECREASING, BUT THERE HAVE BEEN DIFFERENCES BETWEEN SERVICES DURING THE FIRST 5 YEARS OF FP7

57. At the inception of FP7, the Commission stated in its impact assessment that "efforts should be made to reduce the length of time required to successfully negotiate a contract [47]". For projects with similar characteristics (e.g. complexity, size and experience of beneficiaries) the time to grant should not significantly deviate from project to project. Short time to grant is important because:

(a) it enables projects to start earlier, which is very important in the fields of research and innovation where ideas can easily become outdated; and

(b) it makes funding available earlier and thus increases "the fluidity of funds for all beneficiaries" [48] including SMEs.

58. Time to grant for all successfully negotiated projects in FP7 has been on average 333 calendar days [49]. Over the course of FP7, there has been a clear trend towards shortening the time to grant. Also, with the continuing improvement of internal procedures, TTG statistics per service are converging, but there are still differences between the services. Even if only similar projects are taken into account (i.e. projects in the Cooperation Specific Programme), the TTG is around 260 days in DG Communications Networks, Content and Technology, 315 in DG Research and Innovation and 380 in REA and DG Enterprise and Industry [50] (see Figure 6).

59. The time to grant in FP7 is still longer than that of the national funding agencies. Their average time to grant is in the range of 5 to 7 months. However, the projects funded by the national funding agencies tend to be smaller in terms of budget and often involve only one participant.

FIGURE 6

EVOLUTION OF TTG IN COOPERATION SPECIFIC PROGRAMME

+++++ TIFF +++++

Source: European Court of Auditors.

SOME OF THE CAUSES OF TTG DELAYS ARE WITHIN THE COMMISSION’S CONTROL

60. Commission staff mentioned a number of factors that in their view can influence time to grant [51]. For those factors where quantification is possible and data were readily available, the Court used a multiple regression analysis to estimate the relative magnitude of their impact. This suggests that among the possible factors with an impact on TTG, the assignment to a particular service has been an important factor. For instance, if a proposal was negotiated by DG Communications Networks, Content and Technology, TTG decreased by 29 %, all other factors being equal. The role of other factors (e.g. staff workload, number of participants, consortium‘s experience with FP7) was not material.

61. The finding that the assignment to a service is an important variable influencing TTG indicates that there is no single cause to explain the differences in TTG. The assignment to a service is primarily linked with the various characteristics of the services such as the service’s internal procedures, IT systems, internal organisation structure, management prioritisation or specificities not present in other services. Thus it combines a multitude of factors which all impact upon TTG.

62. The Court has also analysed the main causes of long time to grant at the level of individual projects. Annex III summarises these causes, dividing them into causes that were unavoidable, causes that could have been partially avoided by the Commission and causes that could have been avoided. The analysis shows that there are only a few fully unavoidable causes, even though these can have substantial impact on TTG. The Commission has started to address some of the avoidable issues during the course of FP7; however some persist, such as poor communication between, or changes in, the officials responsible for project negotiation, or generous deadlines for the submission of information.

CERTAIN PRACTICES HELP TO AVOID DELAY IN TIME TO GRANT

63. Sharing good practices between services is one way of improving the timeliness of procedures. During the interviews, many of the Commission staff pointed out practices that help to mitigate the causes of delays. There are also lessons that can be learnt from the national funding agencies. Box 5 summarises good practices identified during the audit.

BOX 5

PRACTICES WHICH HELP TO REDUCE TIME TO GRANT

Process automation

Measuring progress against the processing targets, which are set for each process step

Automatic tracking tools, which make collecting monitoring information easy

Process consistency

Ensuring that each proposal or project passes the same steps and is subject to the same controls, which are performed in the same manner

Alignment of the organisational structure with the processes

Short organisational distance between the different persons involved in negotiation

Giving full responsibility for proposal selection and negotiation to a single team

Centralisation of support functions

Parallel processing

Decisive management culture

Terminating negotiations which exceed negotiation deadlines

Putting emphasis on the efficiency of processes

Source: European Court of Auditors.

64. The managers of the operational units in the Commission indicated that lessons learnt from the negotiations are passed on within their units or directorates. There is evidence that such information is partially shared through internal committees in a given service. However, the continuing differences in TTG between services demonstrate that there are inherent organisational barriers which prevent effective sharing and timely implementation of the lessons learnt. For instance, the distribution of responsibilities within the operational directorates in DG Research and Innovation is not standardised to the extent found in other DGs. The different organisational structures prevent effective sharing of good practices, because what works in one directorate may not always be relevant for other directorates, which are organised differently.

THERE ARE CONSIDERABLE DIFFERENCES IN TIME TO PAY BETWEEN DIRECTORATES

65. In general, payments should be made as fast as possible [52]. Interim and final payments should be made within 90 calendar days from the date when the periodic report is considered to be complete [53]. The beneficiaries are required to submit a periodic report together with a cost claim within 60 days of the end of a reporting period. The Commission services determine when the periodic report is acceptable for (partial) payment and can suspend the time limit in case of corrections or if further information is needed from the consortium.

66. On average more than 85 % of payments related to research grants are made within the time limits. However, this indicator does not on its own give a full picture of timeliness. For instance, Directorates E and I of DG Research and Innovation have very similar results. Both made around 95 % of payments in 2010 and 92 % in 2011 within the legal time limit set. However, coordinators of projects managed by Directorate I receive payments for their cost claims on average less than 7 months after the end of the reporting period; whereas in Directorate E it is almost 10,5 months. A difference of more than 3 months over a portfolio of projects is difficult to attribute to outside factors, since the directorates manage similar types of projects with similar participation patterns.

QUALITY CONTROLS ARE SOUND WITH ONLY A FEW WEAKNESSES

67. Internal control is broadly defined as a process intended to provide reasonable assurance to the management on the achievement of the DG's objectives. The quality-related controls aim to ensure that FP7 finances projects of high quality.

THE QUALITY CONTROLS WORK WELL

68. Under FP7, funds are allocated on the basis of competitive calls for proposals and through independent and merit-based peer review [54]. The more reliable the controls of the project selection and project implementation are, the more likely it is that FP7 will finance projects of high quality. The following key controls critically impact upon the quality of the FP7 selection process and project implementation:

(a) Projects are selected by experts who are knowledgeable in the field and do not face a conflict of interest which could impair their objectivity.

(b) Experts are guided through the evaluation process by the Commission.

(c) The evaluation follows international best practices in the area.

(d) Recommendations made during evaluation are adequately addressed before the grant signature.

(e) The project officers follow-up the scientific implementation of the projects.

(f) Scientific progress achieved by the portfolio of projects is measured against the objectives.

69. The Commission evaluates proposals with the assistance of independent experts in order to ensure that only those of the highest quality are selected for funding. The Commission’s approach is very similar to those followed by the national funding agencies, although evaluation procedures do differ on some points. For instance, only the Commission sets strict criteria concerning the mix of evaluation experts (e.g. geographical mix, proportion of industrial experts, gender balance).

70. Over the course of FP7, quality controls have improved in terms of guidance provided to the experts and the documentation of the project negotiation phase, during which the experts’ recommendations are addressed. The Commission has also experimented with tools for detecting cases of plagiarism and in this respect it is ahead of the national funding agencies.

THE MAIN TOOL USED FOR SELECTION OF EXPERTS HAS CERTAIN WEAKNESSES

71. Experts used for the assessment of the proposals submitted are chosen from the EMI database that has been established by the Commission for this purpose. All experts who take part in the evaluation exercises are required to register in this database. Operational units also need to ensure that the experts who carry out the evaluations do not face a conflict of interest that would undermine their objectivity. As the EMI database is the major source of information used by all services for expert identification, as well as checks for possible conflict of interest, this database should contain accurate and complete information. It contains some 100000 experts.

72. Registration in the database is entirely voluntary, as is the updating of information on the expert’s background. There is a risk that experts encode unreliable information on their professional experience or that it is out of date. Therefore, other sources [55] are frequently used to identify suitable experts. Some services use EMI as the primary source for expert identification; others rely both on the EMI and other sources or use primarily the other sources to find suitable experts, who then have to register in the EMI.

73. The review of the EMI database revealed some weaknesses that undermine its usefulness both as a tool for selecting experts, as well as for ensuring that chosen experts do not face a conflict of interest. In particular:

(a) there is no information on the performance of experts who have taken part in past evaluation exercises. This can be a major weakness particularly in cases where experts were found not to have complied with the evaluation rules and/or the required code of conduct;

(b) services need to ensure that experts are "not involved in a contract or research collaboration with an applicant organisation, or had been so in the previous 3 years" [56]. Nevertheless, there is no requirement for the experts who register in the EMI database to provide information on their employment history that goes back 3 years.

74. It is not cost-effective to carry out a quality check on the information recorded in the EMI by each individual expert. However, some national funding agencies have developed cost-effective ways to improve the reliability of their experts’ databases:

(a) Linking the experts’ database with citation databases or other external sources of information: For instance, SNF uses a web-crawler module, which is linked with the expert database and which improves the reliability of the data through searching selected sources on the Internet.

(b) Using internal intelligence on the experts: For instance, when selecting the experts, the project officers in the NSF can also see the list of projects in which the expert has already participated.

75. At the time of the audit, the Commission DGs were planning to upgrade the tools for the management of experts to include new features such as the start and end dates of the experts’ previous employment. The Commission expects that the new features will improve the quality of data.

FINANCIAL CONTROLS HAVE BEEN RATIONALISED BUT ARE STILL NOT RISK-DRIVEN

76. The objective of the financial controls is to verify whether FP7 transactions comply with the eligibility requirements and that costs incurred are directly related to the objective of the project.

FP7 FINANCIAL CONTROL MODEL DIFFERS SIGNIFICANTLY FROM THOSE FOUND IN NATIONAL FUNDING AGENCIES; IN PARTICULAR IT ATTACHES LESS IMPORTANCE TO THE LEVEL OF RISK

77. The control system should be designed in a cost-effective manner [57]. This implies that the benefits of controls should outweigh the costs of control [58]. The Commission has a responsibility to ensure not only that funds are well spent — which entails robust controls and effective performance measurement — but also to reduce the administrative burden for recipients of funds and to cut administrative costs, wherever possible.

78. Under FP6 and FP7 a decision was taken by the Commission to move away from detailed ex ante controls [59] and place greater reliance on ex post controls [60] in order to ease the beneficiaries’ burden with regard to providing evidence before payments are made. As a consequence, most of the significant errors that are discovered are detected through the ex post controls. As noted by the Court [61], the ex ante controls are often limited to an administrative review of the cost claims and related deliverables. All payment transactions are essentially subject to the same ex ante controls.

79. In general, most of the national funding agencies carry out more in-depth ex ante checks than the Commission. Although more in-depth ex ante checks imply better control, they require more staff and result in a greater administrative burden for the beneficiaries. Some national funding agencies use ex ante checks differentiated either by type of beneficiary [62] or by the beneficiary’s risk profile [63]. This allows the agencies to focus their limited resources on those beneficiaries who are considered to constitute a higher risk. This also has the advantage that it reduces the burden of ex ante controls for less risky beneficiaries.

80. The Commission’s system of ex post controls differs significantly from control models typically found in national funding agencies. No other funding agency uses a representative sample of ex post controls to such an extent. Instead, they use a risk-based ex post control strategy. Some funding agencies [64] put more emphasis on institution-based controls, which allow them to obtain a certain level of assurance on the beneficiary’s control environment and to reduce the audit burden. The Commission’s ex post controls focus on individual projects.

81. The ex post controls can provide intelligence on beneficiaries’ practices and risk. For instance, SNF assigns risk ratings to beneficiaries according to their past performance. These ratings allow the agency to extend the ex ante checks for riskier beneficiaries and to increase the audit effort in areas that are considered to be problematic (e.g. salaries, equipment). The Commission does not use the information obtained from ex post controls to vary the intensity and focus of ex ante controls with a view to improving their effectiveness.

THE INTRODUCTION OF A SINGLE REPRESENTATIVE SAMPLE CONSTITUTES AN IMPROVEMENT

82. The research family DGs have adopted a common ex post control strategy [65]. Until the end of November 2011, the sampling methodology resulted in eight separate representative samples of 161 ex post controls being taken for each service of the research family [66]. This ex post control strategy was widely criticised by stakeholders for creating an unnecessary administrative burden for the beneficiaries [67]. It led to a situation where the same beneficiary could be audited several times during the course of FP7.

83. There is now one common single representative sample across the research family. The Commission expects that this will reduce the audit burden on the beneficiaries and minimise situations where the same beneficiary is audited repeatedly by multiple services of the research family. However, the single representative sample covers only the Commission services (92 % of the FP7 budget). It will not include those parts of FP7 which are implemented by independent bodies (e.g. JTIs, Article 185 initiatives), which will be still responsible for carrying out their own ex post controls.

NEW INSTRUMENTS HAVE MET THE NEED FOR WHICH THEY WERE CREATED, BUT THERE ARE DEFICIENCIES IN IMPLEMENTATION

BOTH INSTRUMENTS ENHANCED THE RESEARCH FUNDING LANDSCAPE ...

84. The JTI and the RSFF were created to achieve objectives which could not be readily achieved with traditional grant-based instruments alone. The JTIs have been set up to support long-term industrial investment in defined research areas. The RSFF’s main objective [68] was to foster "private sector investment across Europe in research, technological development and demonstration as well as innovation". This was to be achieved by leveraging that private investment with a target of 10 billion euro in additional RDI loans.

85. Between 2008–11, the JTIs committed 1,4 billion euro on research projects. This represents almost 90 % of the corresponding budget. However, due to the multiannual nature of projects financed by JTIs, payments are far lower [69]. The participation patterns show that JTIs have been able to attract a wide variety of beneficiaries and their calls for proposals have redressed the balance between the academic and the private sector by attracting more private participants than calls under the Cooperation Specific Programme.

86. Some JTIs have been particularly successful in getting SMEs involved in their projects. For instance, 38 % of the EU’s clean sky funding has gone to SMEs; more than 40 % of the projects financed by ENIAC have included an SME, even though the share of ENIAC funding for SMEs is only around 19 %. On average, 21 % of funding provided by the JTIs has gone to SMEs.

87. Under RSFF, 96 operations amounting to 9,5 billion euro had been approved up until the end of 2011. As not all approved operations resulted in an actual signature of the loan operation between the EIB and the borrower, the loan amount of signed projects reached 7,3 billion euro and disbursements were almost 5 billion euro [70]. According to DG Research and Innovation, the leverage effect of the EU contribution is 6,6 (each euro of EU contribution has led to more than 6 euro of the RSFF loan finance). DG Research and Innovation further estimates that the RSFF has a multiplier effect of 28 on total RDI investment (each euro from the EU budget contributed to total financing of 28 euro of RDI investment). The RSFF has also a wide country diversification. As of the end of 2011, the RSFF portfolio included signed projects from 21 countries [71]. However, three Member States (Germany, Spain and Sweden) accounted for more than half of total RSFF transactions.

88. The Court asked industry experts (in the case of JTIs) and beneficiaries (in the case of RSFF) for their opinion on the two instruments. The general reaction was positive and the respondents agreed that the instruments were useful. For instance, the industry experts involved in the JTIs indicated that JTIs added value in encouraging research activities and succeeded in building mutual trust between the JTIs’ stakeholders.

... BUT IMPLEMENTATION OF JTIS SUFFERED FROM AN OVERLY COMPLEX LEGAL FRAMEWORK

89. The set-up must be tailored to the nature of the technological challenge to be addressed, the characteristics of the relationship between the stakeholders and the financing needs of the initiative [72].

JTIS ARE SUBJECT TO REQUIREMENTS WHICH ARE NOT APPROPRIATE FOR THEIR SIZE

90. JTIs are small organisations, established as Union bodies. The legal status of a Union body obliges them to meet requirements which would normally only apply to much larger organisations in terms of staff resources. As a consequence on average 52 % of their staff are devoted to work on administrative tasks. In comparison, in the case of EU agencies, which are subject to the same legal requirements, only 22 % of staff [73] are devoted to administrative tasks.

91. A JTI is required to apply a financial regulation, which is similar to the one used by the EU institutions. The Staff Regulations [74] apply to the staff of the JTIs. This regulatory framework implies that each JTI is expected to have a certain number of functions: a data protection officer, a local information security officer, an internal control coordinator, an accounting officer (and a back-up), an authorising officer (and back-up), initiating agents (and back-ups), validating agents (and back-ups), an internal auditor, an IT manager and a staff committee. These requirements are difficult to satisfy in an entity which employs between 13 and 31 staff.

92. The financial regulation does not permit a JTI to conclude a service level agreement with another JTI. This prohibits the sharing of services between JTIs to reduce costs. For instance, the internal auditor function could have been shared between the JTIs. Furthermore, the legal framework requires the approval of the governing board for a long list of decisions. Consequently, too many administrative decisions (such as catering orders and purchase of office stationery) are brought up to the level of the governing board, thus hampering its focus on strategic issues [75].

THE REQUIREMENTS IMPOSED ON JTIS LED TO DELAYED START-UPS

93. Due to the time needed to establish and implement the rules of procedure and the time needed to recruit staff, the start-up of the JTIs was slower than planned [76]. On average, it took 2 years to grant financial autonomy to a JTI. This meant that the Commission remained responsible for an average of one third of the expected operational lifetime of the JTIs. Industry experts overwhelmingly deemed the time taken to set up the operating structure of the JTIs to be unreasonable.

... AND THE RISK AND CROWDING OUT EFFECT OF THE RSFF MUST BE CAREFULLY CONSIDERED

THE COMMISSION HAS NOT SUFFICIENTLY DEMONSTRATED THAT A RSFF FUNDING LEADS TO INVESTMENTS ABOVE THE LEVEL THAT BENEFICIARIES WOULD HAVE UNDERTAKEN WITHOUT PUBLIC MONEY

94. The purpose of the RSFF is to finance riskier European research and innovation actions that would not be possible without such Community support [77]. To satisfy this condition, the EIB aims to finance RSFF projects with an internal grading of D- and below. Projects with such risk would normally not be funded by the EIB.

95. The availability of debt financing for riskier RDI projects has been particularly valuable in times of financial crisis. The RSFF remained one of the few financial instruments available for public companies to maintain RDI activities. This was confirmed by the responses of RSFF beneficiaries to the Court’s survey. The RSFF beneficiaries also expressed the opinion that the loans contributed to avoiding the "brain drain" of highly qualified persons and sustained EU employment.

96. A survey among the RSFF beneficiaries revealed that access to finance was not a major barrier to beneficiaries investing in RDI. More than half of the RSFF beneficiaries (56 %) stated that the lower interest rate was a decisive or major factor for taking a RSFF loan (see Figure 7). This indicates that RSFF could have a crowding out effect and that a number of RSFF projects would have been financed by commercial banks [78], albeit at higher cost to the beneficiaries. Given the relatively long maturity of the loans, it is too early to assess the risks taken by the EIB under the RSFF. However, the current low impairment [79] rate could indicate that the RSFF loans are not necessarily as risky as expected, when the RSFF was set up.

FIGURE 7

REASONS FOR TAKING A RSFF LOAN

+++++ TIFF +++++

Source: European Court of Auditors' questionnaire to the RSFF beneficiaries.

REVISION OF THE RSFF PARAMETERS IN 2011 SHIFTED THE RISK BALANCE TOWARDS THE EU BUDGET

97. Under the initial RSFF cooperation agreement between the EIB and the European Commission, the EU budget supported a maximum loss of 50 % of the nominal loan. An amendment in December 2011 introduced a new approach based on a "portfolio first loss piece". Applying this new approach, the EIB may withdraw from the EU contribution an amount of 95 % of the impaired loan amount within the limit of the overall EU contribution. The EIB is therefore exposed to the remaining risk of 5 % of the impairment amount and to all impaired amounts above the EU contribution. Thus, the revised cooperation agreement has shifted the risks from the EIB to the EU. If the loans turn out to be indeed of low risk, the cumulative amount of losses will not exceed the maximum EU contribution and the EU will end up bearing 95 % of the losses on impaired loans (for illustrative example see Box 6). The Council decision is not entirely clear on the level of risk that could be accepted by the EU.

BOX 6

EXAMPLE COMPARING INITIAL AND REVISED COOPERATION AGREEMENT

Initial cooperation agreement

The maximum EU contribution to cover a loss for a given loan was set at 50 %. That is, for a loan of 100 million euro, a maximum of 50 million euro was blocked to cover the risk of default. If the borrower defaults on the loan, the loss for the EU would amount to 50 million euro while the EIB would bear the remainder.

Revised cooperation agreement

For the same loan, the amount to be blocked from the EU contribution is set at 95 million euro. If the borrower defaults, the loss for the EU would be 95 million euro while the EIB would bear a loss of 5 million euro only.

CONCLUSIONS AND RECOMMENDATIONS

98. Over the course of FP7 the Commission has introduced a number of changes which have simplified the rules for participation. The Commission has been successful in particular in rationalising the requirements and improving its guidance documents for beneficiaries. The FP7 funding model remains complex, but the simplification of the funding model is an important element of the Commission’s proposal for the rules for participation under Horizon 2020 (see paragraphs 17 to 20).

99. The Commission has been able to align FP7 provisions with beneficiaries’ practices in some cases, but more needs to be done in future to maximise the advantages. The Court notes that despite the importance given by the Commission to coherence in the FP7 preparatory acts, some aspects of FP7 implementation are affected by a lack of coherence. FP7 beneficiaries are faced with differences which arise from the relative autonomy of the Commission services. The establishment of the RCC as a high-level committee with overall responsibility is a step in the right direction to tackle incoherent rules; however, the mechanisms for identification of diverging practices are weak (see paragraphs 21 to 44).

RECOMMENDATION 1

With a view to better aligning FPs’ provisions with beneficiaries’ practices, the Commission should in Horizon 2020 aim to:

- make further efforts to ensure that beneficiaries’ practices can be accepted;

- reduce the requirements that the beneficiaries have to fulfil to have their cost methodology certified;

- ensure that certificates on methodology provide clear benefits in terms of reduced administrative burden, so that obtaining such a certificate has clear value for money for the beneficiaries;

- accept, where justified, existing national practices, such as average personnel cost methodologies, with the aim of further simplification for those beneficiaries that use them.

RECOMMENDATION 2

The Commission should improve coherence in FP7 management. The introduction of RCC is a step forward. The Commission should carry out in-depth analysis of the procedures in order to identify and eliminate any divergences. Exceptions should be clearly justified. The Commission should make the implementation of other research-related instruments (JTIs, Structural Funds) more consistent with framework programmes (FP7 and Horizon 2020), as a good example of how to address the European Research Area objective of better harmonisation through alignment of rules.

100. The Commission’s management of FP7 processes is not sufficiently robust in certain key areas. On the one hand, process design, improvement and information are strong areas with few weaknesses. On the other hand, the Commission has not fully aligned resources to support FP7 processes. The lack of harmonisation of the back-office tools and the absence of business process management tools may hamper efficient implementation. There are some examples of advanced tools in the Commission, which other services could learn from. Furthermore, there are indications that staff utilisation varies between the themes of the Cooperation Specific Programme. In its analysis the Court used basic workload indicators which indicate that too many staff resources are used for the implementation of some themes at the cost of other themes or other Commission activities (see paragraphs 45 to 55).

RECOMMENDATION 3

To improve process efficiency, the Commission should ensure that resources are properly aligned:

- The Commission has already started to develop a new generation of IT tools for Horizon 2020. To enable processes to be implemented efficiently, the Commission should put in place tools which integrate front-office, back-office and workflow functionalities and whose architecture is flexible to allow for rapid upgrades without negative impact on the smooth running of operations.

- As regards staff, the Commission DGs should agree on workload indicators which will provide a basis for an objective assessment of staff needs for the management of the framework programme. Based on these indicators, the Commission should conduct a study to identify overstaffed services or directorates, from which staff can be reallocated to understaffed areas or to other Commission activities.

101. Whilst processing times have shortened over the course of FP7, they have only approached 9 months in 2012. The audit has highlighted a number of causes of longer time to grant as well as good practices on how to deal with those causes that are avoidable. Management’s attention to this problem and the existence of an environment which enables effective sharing of good practices are key. The important differences in total time to pay between different directorates also suggest possible scope for improvement (see paragraphs 56 to 66).

RECOMMENDATION 4

Reducing the processing times, in particular of time to grant, requires that processes be automated to the maximum extent possible and implemented consistently across the Commission. To that end, the Commission should review whether more harmonised distribution of responsibilities between services would improve process efficiency. The Commission should ensure that the distribution of responsibilities is optimised given the changes envisaged under Horizon 2020. This will allow for an easy exchange of best practices across the services and will create a level playing field, in which the performance of different organisational entities implementing the same programme can be directly compared. It will also bring the benefit of reduced complexity, which in turn can make the framework programme more efficient (for instance, the set-up of electronic workflow tools will be easier).

102. The quality controls applied to the selection and follow-up of the projects function well. The Commission’s evaluation approach follows best international practice. The audit identified one specific gap in quality controls, which relates to the main tool for expert selection. As regards the financial controls, the FP7 control model differs significantly from those found in national funding agencies. It places greater reliance on controls after payment and attaches less importance to the risk of errors. The risk-based control strategies of national funding agencies allow them to focus their control effort on those beneficiaries which constitute a higher risk and ease the burden of controls on less risky beneficiaries (see paragraphs 67 to 83).

RECOMMENDATION 5

The Commission should make its control activities more risk-driven. The ultimate objective should be to focus its controls on those beneficiaries which are considered to constitute a higher risk and ease the burden on less risky beneficiaries. The beneficiaries should be classified into categories based on their general risk profile and the Commission’s experience with the beneficiary’s control environment. Cooperation with national funding agencies could provide further intelligence on beneficiaries’ risk and could also potentially lead to a reduction in the control burden placed on some beneficiaries. As regards the ex post controls, the Commission should extend its single representative sample of controls so that it covers all parts of FP7.

103. Both new instruments analysed by the Court in this audit showed their potential to enhance the research funding landscape. They have met the needs for which they were created. Sometimes they have been very successful in attracting a specific group of beneficiaries (e.g. SMEs). However, the implementation of JTIs has suffered from an overly complex legal framework (see paragraphs 84 to 93).

RECOMMENDATION 6

The Court recommends that the budgetary authorities and the Commission bring the legal framework of JTIs (and of any future similar instruments) into line with their staff complement. In particular, requirements which do not directly serve the purpose of ensuring proper control should be rationalised. The powers of the executive director responsible for the day-to-day management should be increased, enabling the governing board to focus solely on operational and strategic matters.

104. The risk and possible crowding out effect of the RSFF must be carefully considered. The Commission has not sufficiently demonstrated that RSFF funding leads to investments above the level that beneficiaries would have undertaken, without public money anyway (see paragraphs 94 to 97).

RECOMMENDATION 7

To maximise the impact of the RSFF and minimise the risk of crowding out, the Commission needs to demonstrate that it targets those beneficiaries which are unable to secure loan financing from commercial and other lenders. Furthermore, the risk-sharing arrangements between the Commission and the EIB should be more clearly set out in the legal basis as is the case for some other financial instruments managed by the Commission.

This Report was adopted by Chamber IV, headed by Mr Louis GALEA, Member of the Court of Auditors, in Luxembourg at its meeting of 19 March 2013.

For the Court of Auditors

+++++ TIFF +++++

Vítor Manuel da Silva Caldeira

President

[1] Such as sustainable development, climate change, energy, transport, public health, information technologies and biotechnology.

[2] Annex I to Decision No 1982/2006/EC of the European Parliament and the Council of 18 December 2006 concerning the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007-2013) (OJ L 412, 30.12.2006, p. 1).

[3] This amount does not include the FP7 Euratom budget, which will provide an additional 5,3 billion euro over the same period.

[4] Switzerland, Israel, Norway, Iceland, Liechtenstein, Turkey, Croatia, the former Yugoslav Republic of Macedonia, Serbia, Albania, Montenegro, Bosnia and Herzegovina, Faeroe Islands and Moldova.

[5] Initiatives under Article 185 TFEU (Article 185 initiatives) allow the EU to provide support to the joint implementation of research and development programmes of the Member States. The implementation is managed by dedicated implementation structures.

[6] Funded through the Ideas Specific Programme, which is implemented by the European Research Council and its executive agency.

[7] The term "blue sky" or "frontier" research denotes research in science and technology beyond the frontiers of understanding, progressing on new areas, and is characterised by an absence of disciplinary boundaries.

[8] From 39 % in FP4 to 31 % in FP6. Source: "Interim evaluation of the seventh framework programme — Report of the Expert Group", 12 November 2012.

[9] Regulation (EC) No 1906/2006 of the European Parliament and of the Council of 18 December 2006 laying down the rules for the participation of undertakings, research centres and universities in actions under the Seventh Framework Programme and for the dissemination of research results (2007-2013) (OJ L 391, 30.12.2006, p. 1).

[10] For instance, the delegation of management to coordinators and the use of audit certificates or simplification of Marie Curie actions. The Marie Curie actions are now part of the People Specific Programme.

[11] SEC(2005) 431 final.

[12] Under centralised management, the budget is implemented either directly by the Commission or indirectly by the agencies or other bodies to which the Commission has delegated its powers.

[13] Special Report No 1/2004 on the management of indirect RTD actions under the Fifth Framework Programme (FP5) for research and technological development (1998 to 2002), together with the Commission’s replies (OJ C 99, 23.4.2004, p. 1).

[14] ENIAC Member States are Belgium, Czech Republic, Germany, Estonia, Ireland, Greece, Spain, France, Italy, Latvia, Hungary, the Netherlands, Austria, Poland, Portugal, Romania, Slovakia, Finland, Sweden, the United Kingdom and Norway.

[15] The identification of scientific topics and setting out of the work programmes was outside the scope of this audit.

[16] SEC(2005) 431, COM(2005) 705 final, SEC(2005) 430 final, COM(2010) 187 final.

[17] Conclusions on simplified and more efficient programmes supporting European research and innovation, 3 016thCompetitiveness Council meeting of 26 May 2010; European Parliament report on simplifying the implementation of the research framework programmes, 2010/2079 (INI).

[18] " The Court considers that the radically simplified cost- funding model will improve the reliability of the model, decrease the risk of irregularities in beneficiaries’ cost claims, make project accounting less complex, and eliminate some of the verification steps required under the current FP7 funding model, thereby facilitating and accelerating the application process." For more details, refer to the Court’s Opinion No 6/2012 on the proposal for a Regulation of the European Parliament and of the Council laying down the rules for the participation and dissemination in "Horizon 2020 — the Framework Programme for Research and Innovation (2014-2020)" (OJ C 318, 20.10.2012, p. 1).

[19] SEC(2005) 431 final, section 2.5.

[20] E.g. paragraph 117 of 2007 European Parliament general budget discharge, 2008/2186(DEC); COM(2010) 187 final; European Parliament report on simplifying the implementation of the research framework programmes (2010/2079(INI)); Conclusions on simplified and more efficient programmes supporting European research and innovation, 3016 th Competitiveness Council meeting of 26 May 2010.

[21] The intended scope of the certification mechanism was limited to participants with multiple and sizeable participations: those with eight participations above 375000 euro.

[22] European Court of Auditors’ Annual Report concerning the financial year 2010 (OJ C 326, 10.11.2011).

[23] In April 2009, the European Parliament pointed out that beneficiaries should be allowed to use their average cost methodologies and asked "the Commission to start a procedure making seventh framework programme rules compatible with general business practices that allow for calculation and charging of average hourly rates per cost centre" (Source: 2007 European Parliament general budget discharge, 2008/2186(DEC)). The decision (C(2011) 174) addressing this request was adopted in January 2011.

[24] The national contact points exist in all Member States. They are the FP7 support structures that provide practical information to researchers and entrepreneurs on their participation in FP7.

[25] SEC(2005) 431.

[26] COM(2005) 705.

[27] SEC(2005) 431; COM(2005) 705.

[28] FP7 interim evaluation, p. 56.

[29] The financial validation check is based on a combination of several financial ratios of liquidity, solvency, profitability, autonomy, cash-flow and net operating profit.

[30] The purpose of the single clearing house was to guarantee that the interpretation, in particular of the legal and financial provisions of the contracts, given out to beneficiaries by the Commission is consistent and uniform.

[31] Decision No 1982/2006/EC.

[32] FP7 will represent around 10 % of funding by Member States in 2013, Source: Interim Evaluation of the Seventh Framework Programme — Report of the Expert Group.

[33] Recital 5 of FP7 Regulation (EC) No 1906/2006.

[34] Recital 21 of FP7 decision.

[35] Non-profit public bodies, secondary and higher education establishments, research organisations and SMEs.

[36] Article 7(4) of Regulation No 1080/2006 of the European Parliament and of the Council of 5 July 2006 on the European Regional Development Fund and repealing Regulation (EC) No 1783/1999 (OJ L 210, 31.7.2006, p. 1).

[37] A process is defined as "a structured, measured set of activities designed to produce a specific output for a particular customer." Source: Davenport, T. H., Process innovation: Reengineering work through information technology, Harvard Business School Press.

[38] These five elements are based on process management practices in both the public (i.e. Commission internal control standards, UK National Audit Office’s framework for maturity of process management) and private sectors (i.e. Hammer, M., "The process audit", Harvard Business Review, April 2007; 2011–12 criteria for performance excellence, Baldrige Performance Excellence Programme).

[39] More frequent reporting and scientific reviews are used in DG Communications Networks, Content and Technology.

[40] Hammer, M. "The process audit", Harvard Business Review, April 2007.

[41] Commission internal control standard No 7 "Operational structure" — Tips for effectiveness.

[42] The front-office IT tools facilitate interaction with the beneficiaries. The back-office IT tools support the Commission staff in exercising their back-office functions. The BPM IT tools aim to automate the workflows and thereby make the administrative processes faster. They also allow for a better monitoring of the workflows by management.

[43] Commission internal control standard No 3 "Staff allocation and mobility", Requirements.

[44] Commission internal control standard no 8 "Processes and procedures".

[45] Recital 23 of Decision No 1982/2006/EC.

[46] Recital 19 of Regulation (EC) No 1906/2006.

[47] SEC(2005) 430, Annex 1, p. 42.

[48] SEC(2009) 477 final, p. 3.

[49] Webcorda, projects successfully negotiated before the end of May 2012.

[50] REA and DG Enterprise and Industry are treated together as they share the management of the same themes in Cooperation Specific Programme. Their TTG is higher partly due to the security clearance procedure, which all projects in the security theme have to undergo. The recent average TTG for projects in the space theme where no security procedure delays the negotiation process was 350 days.

[51] For instance, "the simpler management of smaller consortia bears a potential for reducing average time to grant." Source: COM(2010) 187.

[52] SEC(2009) 477 final.

[53] Starting from the third quarter of 2009, the Commission reduced the time limits for making interim and final payments to 90 days to help economic recovery. The legal basis sets the limit at 105 days. (Article 106 of Commission Regulation (EC, Euratom) No 2342/2002 of 23 December 2002 laying down detailed rules for the implementation of the Financial Regulation (OJ L 357, 31.12.2002, p. 1). Articles II.14 to II.19 of Annex 2 to the model grant agreement.)

[54] Regulation (EC) No 1906/2006.

[55] These include identification of experts through scientific publications that relate to the call’s theme (e.g. nature, science), scientific websites (such as PubMed, ISI web of knowledge and Scopus), participation in conferences, use of social media and Internet to check an expert’s background.

[56] General conditions applicable to the appointment of independent experts as specified in the "Rules for submission".

[57] Paragraph 21 of Court’s Opinion No 1/2010.

[58] Paragraph 12 of Court’s Opinion No 1/2010.

[59] Ex ante controls are financial controls carried out before grant signature or before payment. They can be institution-based (e.g. legal validation and financial viability checks) or project-based.

[60] Ex post controls are done after beneficiaries receive reimbursement for their cost claims.

[61] Paragraph 8.20 of Court’s Annual Report concerning the financial year 2011.

[62] DFG and French National Research Agency.

[63] SNF.

[64] SNF, research councils in the United Kingdom.

[65] FP7 ex post audit strategy 2009–16.

[66] In addition to representative ex post controls, the services can also make use of the corrective ex post controls to obtain additional assurance of the proper functioning of their control systems.

[67] Both the European Parliament and Council, several Member States (e.g. Germany, United Kingdom, EU-12, Finland) and enlargement countries called for a reduction in audit intensity during the preparation of Horizon 2020.

[68] Annex III to Council Decision No 2006/971/EC of 19 December 2006 concerning the Specific Programme Cooperation implementing the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007 to 2013) (OJ L 400, 30.12.2006, p. 86).

[69] Reports on the annual accounts of the joint undertakings for the financial year 2011, together with the replies of the joint undertakings (OJ C 6, 10.1.2013, pp. 1, 9, 18, 27, 36, 46 and 56).

[70] RSFF annual ex post operational report 2011.

[71] 19 Member States, Israel and Turkey.

[72] European Commission's Report on ETP and JTIs: fostering public–private R & D partnerships to boost Europe’s industrial Competitiveness (revised March 2006), SEC(2005) 800.

[73] Figure based on the European Court of Auditors own analysis covering 22 agencies for the years 2008, 2009 and 2010.

[74] Staff Regulations and the Conditions of Employment of Other Servants, applicable to the staff of the EU Institutions.

[75] As reported during interviews with industry experts.

[76] This was confirmed by the Commission: "[JTIs'] internal structures were not yet working optimally and they have still to recover from the initial operational delays", Source: Commission staff working document accompanying the annual report on the progress achieved by the joint technology initiatives joint undertakings in 2010, SWD(2012) 105.

[77] Annex III to Decision 2006/971/EC.

[78] 72 % of the RSFF beneficiaries either disagree or strongly disagree with the statement that they had no access to other debt finance.

[79] Only two loans of a combined value of 70 million euro have been impaired. This represents 7 % of EU contribution to cover the losses from RSFF and 1,4 % of total disbursements.

--------------------------------------------------

ANNEX I

OVERVIEW OF THE AUDIT APPROACH

+++++ TIFF +++++

Note: Numbers in brackets indicate the paragraph in the annex with description of methodology.

DESCRIPTION OF THE AUDIT EVIDENCE COLLECTION METHODS

PRELIMINARY STUDY

1. Prior to undertaking the main research study, a preliminary study was conducted to identify the key risks related to efficient implementation of FP7. This exploratory work was based on desk research and interviews with Commission staff involved in the main areas of FP7 implementation, staff dealing with research topics in a number of permanent representations, staff from national research and innovation liaison offices in Brussels and representatives of key beneficiaries such as universities, research organisations and private companies.

BENCHMARKING VISITS TO NATIONAL FUNDING AGENCIES

2. Five countries were selected for comparative analysis, taking into account the quality of the country’s funding system for research, with the aim of comparing to FP7 rules for participation and Commission’s processes. In these five countries, a total of 10 agencies for funding research were identified using three main selection criteria: the budgetary size of the funding schemes managed, the similarity of funding approach and the recognised excellence of the schemes. The following funding agencies were visited:

in Germany: the German Science Foundation (DFG), the Federal Ministry of Economics and Technology and the Ministry for Economics and European Affairs of Brandenburg (Managing Authority for Structural Funds);

in the United Kingdom: research councils, the Technology Strategy Board and regional development agencies;

in France: the French National Research Agency;

in the United States of America: the National Science Foundation (NSF) and the Department of Energy; and

in Switzerland: the Swiss National Science Foundation (SNF).

QUESTIONNAIRE TO BENEFICIARIES

3. A sample of more than 300 beneficiaries was asked to comment on aspects important to FP7 implementation, via an electronic questionnaire. The sample was drawn from the key beneficiaries of FP7 within each category of beneficiaries (higher education sector, research institutions, industry, private entities, SMEs). The selected beneficiaries are recipients of almost half of the FP7 budget. Around 40 % of the addressees submitted their responses.

SEMI-STRUCTURED INTERVIEWS WITH THE COMMISSION STAFF INVOLVED IN FP7 IMPLEMENTATION

4. In total eight Commission directorates and/or services were selected to be visited during the audit:

Directorate H — Aerospace, Global Monitoring for Environment and Security, Security and Defence in DG Enterprise and Industry;

Directorate E — Digital Content and Cognitive Systems in DG Information Society and Media (now Directorate A in DG Communications Networks, Content and Technology);

Directorate G — Components and Systems in DG Information Society and Media (now Directorate G in DG Communications Networks, Content and Technology);

Directorate E — Biotechnologies, Agriculture, Food in DG Directorate Research and Innovation

Directorate F — Health in DG Research and Innovation;

Directorate G — Industrial Technologies in DG Research and Innovation;

Directorate I — Environment in DG Research and Innovation;

the Research Executive Agency — Space and security research.

5. In each directorate, the audit team arranged for semi-structured interviews with the managers of the administrative and financial unit, the horizontal coordination unit (if existing) and two operational units responsible for conducting the evaluation and monitoring of the scientific progress of the projects. Furthermore, the audit team conducted a set of interviews with the key personnel in each DG/service with the aim of obtaining audit evidence on how the DGs/services manage their internal processes.

DESK REVIEW OF A SAMPLE OF PROJECTS

6. A sample of 30 coordinators with projects from multiple services was drawn to analyse the Commission procedures from the perspective of the beneficiaries. The analysis was used to identify cases which can illustrate the variability of procedures from the beneficiaries’ perspective, to classify common causes of delays in negotiation of projects and to assess the operation of the key controls related to ensuring the quality of spending.

QUESTIONNAIRE TO BENEFICIARIES OF RSFF LOANS AND REPRESENTATIVES OF JTIS

7. During the audit, all 26 beneficiaries of RSFF loans in the EU window were sent a questionnaire. The aim of the questionnaire was to obtain an overview of the beneficiaries’ motivations for using the RSFF instrument and the results generated by the financed investment. 18 out of 26 contacted beneficiaries provided their reply. This represents a response rate of almost 70 %.

8. The JTI representatives and the representatives of the industry associations were asked to fill in a questionnaire in order to obtain a balanced view of the aspects relating to the organisational structures of the JTIs.

--------------------------------------------------

ANNEX II

FOLLOW-UP OF THE COURT'S PREVIOUS RECOMMENDATIONS

Court’s key findings in SR 1/2004, which were followed up in the audit | Court’s recommendations in SR 1/2004 | Commission’s replies to recommendations in SR 1/2004 | Court’s assessment of the situation in 2011/2012 |

The complexity of the rules for participation hinders the effective contract management of FP5 and is a complication for beneficiaries. (paragraphs III, 15, 17, 27 to 29). | (...) The Commission should (...) review from first principles the rules for setting the Community’s participation, aiming for significant simplification whilst allowing adequate control over expenditure in a future FP7. The Court’s previous recommendation of a single cost reimbursement system with three main categories of costs should be one of the main elements to be considered. (paragraph 115) | [The Commission] will ensure that the framework programmes’ management model continues to develop, including when the proposal is put forward for the future FP7, focusing in particular on streamlining procedures. | Partially implemented. Some simplifications have been proposed in FP7. Simplification of the funding model is an important element of the Commission’s proposal on Horizon 2020 (see paragraph 20). |

The fragmentation of programmes into five DGs resulted in dilution of responsibilities, duplication of functions and increased need for coordination and consultation (paragraphs V, 46 to 57, 63, 116). | The Court recommends that the structure of the framework programme and the organisation of the Commission services which manage it be brought into line. The basic design of research programmes as single programmes should be reflected in the organisational structure of the Commission. Horizontal aspects, such as support for small and medium-sized enterprises or international cooperation, should be integrated as far as possible within the thematic programmes. (paragraph 119) The Court recommends that the Commission analyses the possibility of merging the RTD framework programmes in a single entity. (paragraph 123) | (...) The Court’s recommendations concerning horizontal aspects have largely been adhered to. (...) One "Programme Management Committee" oversees all thematic priorities in each specific FP6 programme. (...) The Commission shares the Court’s view that effective coordination (...) is crucial to efficient management. (...) The Commission considers that a centralised management model is not appropriate. The action plan adopted in January 2003 by the Commission (...) includes a proposal to set up a permanent interdepartmental group to ensure better coordination and cohesion between Commission departments. | Partially implemented. The number of Commission services involved in the management of FP7 implementation has even increased. This situation has complicated the implementation of FP7. For instance, FP7 beneficiaries are faced with some differences which arise from the relative autonomy of the Commission services (see paragraphs 35 to 36). The Research Enquiry Service was established at the very beginning of FP7. In 2011, the Commission established the Research Clearing Committee with a mandate to take final positions on horizontal matters related to the implementation on which the services did not reach consensus (see paragraph 34). Support for SMEs and international cooperation has been substantially integrated within the thematic programmes. |

The Court recommends that the Commission considers setting up centralised support functions for: "procedural and legal advice","information technology","registration and clearing", and"controlling and reporting". (paragraph 122) | (...) Depending on the operations to be undertaken, coordination within FP6 is ensured by means of appropriate arrangements (...). | Mostly implemented. To reduce unnecessary duplications, the Commission has implemented several measures, such as the Unique Registration Facility, Research Enquiry Service and Research Clearing Committee (see paragraphs 25, 32 to 34). The support function "information technology" was not centralised. Instead, the Commission services developed IT tools with similar functionalities independently of each other (see paragraph 49). |

Certain rules and practices unnecessarily extend the grant selection process (paragraphs VI, 58 to 61, 68 to 69). | The Commission should further improve the flexibility of the RTD framework programme and streamline its internal procedures and practices. The Court recommends that administrative procedures be simplified, with the aim of halving the "time to contract" to between 5 and 6 months.(paragraph 120) | (...) The Commission will continue to simplify its administrative procedures in accordance with the new arrangements laid down by the FP6 participation rules. | Partially implemented. Time to grant has only been reduced in 2012 to around 270 days, for most of the services (see paragraphs 56 to 64). |

The Commission experienced difficulties in employing staff with required profiles resulting in uneven caseload, bottlenecks or insufficient monitoring (paragraphs V, 70 to 77). | (...) Particular attention should be paid to reducing the number of vacancies and achieving an equitable allocation of staff resources, based on actual and expected caseload figures, between Directorates-General and between programmes. This would require the implementation of appropriate monitoring instruments in the framework of Activity Based Budgeting (ABB). (paragraph 121) | In line with the Court’s recommendations, the Commission has adopted a package of measures designed to integrate its research staff into the mainstream of the Commission’s personnel policy and to reduce vacancy rates. (...) The Commission has deployed its Activity- Based Budgeting system [which] (...) will enable systematic monitoring of resource allocations (...) | Partially implemented. The number of vacancies has been reduced. The staff resources continue to be an issue in the management of FP7. It is not certain that staff resources are equitably allocated between services. (see paragraphs 53 to 55) |

IT systems deployed were inadequate and not sufficiently integrated with each other. (paragraphs VIII, 82 to 86, 127) | (...) The Commission should continue its implementation of a common and integrated IT system. In particular, the development of the IT system should be based on clearly defined user requirements and provide for comprehensive testing of the software before the programmes actually become operational. (paragraph 125) As far as possible, paper-based transactions should be phased out during FP6, with the aim of achieving complete electronic interaction in FP7. (paragraph 126) | The Commission accepts however that further integration of its IT systems would have improved the efficiency of the management of FP5 indirect RTD actions (...) | Partially implemented. The Commission has put into operation a number of tools to support FP7 implementation. On the front-office side, the tools have been harmonised across the Commission services; harmonisation of the corresponding back-office tools has not progressed at the same pace (see paragraph 49). The tools are not yet fully integrated (see paragraph 51). |

Not all important internal controls were operated effectively. (paragraphs 26, 89 to 94, 100) | Internal control activities should be harmonised and their implementation strengthened across the framework programme. In particular, the Commission services should jointly define the relevant and necessary controls which address the known risks in this budgetary area. (paragraph 130) The use of preventive controls should be intensified. (paragraph 131) | Further efforts will be made to guarantee that the Commission’s procedures are sufficiently standardised within the RTD framework programmes and adequately documented within each directorate-general and programme. The Commission will continue its efforts in the areas indicated by the Court. | Partially implemented. The procedures have been documented (see paragraph 46), the controls applied to the projects are sound and some controls have been improved (see paragraph 67 to 70). The use of preventive controls has not been intensified. The Commission has moved away from detailed ex ante controls and places greater reliance on ex post controls (see paragraph 78). Overall, the Court’s 2011 Annual Report assesses the control systems for the research framework programmes to be partially effective. |

Deficiencies in the management information system prevent effective monitoring of the framework programme in its entirety. (paragraphs XI, 101 to 110) | The Court recommends that the Commission develop a comprehensive reporting and controlling system for the RTD framework programmes. (paragraph 135) The Commission should agree on common (...) objectives for the research area and the implementation of the RTD framework programmes. (paragraph 136) | As regards the overall monitoring of programme implementation, the legal requirements are complied with by the Commission. The establishment of harmonised internal control standards has contributed to the regular production of management reports in addition to those required by the rules. The Commission will continue to focus on ongoing improvements to management, setting clear objectives and clear priorities. | Implemented. The Commission has established a system for collecting a set of FP7-wide key performance indicators (see paragraph 47). |

--------------------------------------------------

ANNEX III

CAUSES OF LONG TIME TO GRANT

BEYOND THE CONTROL OF THE COMMISSION

Drop-out of partners from the consortium; finding new partners to replace those who have dropped out can take 1–2 months

Delays in the submission of the requested documentation by beneficiaries or delays due to extensive time taken by beneficiaries to analyse the Commission documents

Lack of experience with FP7 by the consortium

Negotiation with financially weak or uncooperative beneficiaries

PARTIALLY AVOIDABLE BY THE COMMISSION

Delayed legal and financial validation of partners due to peaks in validation requests, due to the complexity of validation of SMEs or partners from non-European countries, or due to discussion of the status of particular beneficiaries

Complexity of rules regarding some specific aspects (e.g. subcontracting, use of third parties) and the need to explain these concepts to beneficiaries

Periodicity of programme committee meetings

Under the 'Security' theme, security scrutiny takes around 10 weeks. (The process is stalled because this step determines which service will be responsible for the project.)

AVOIDABLE BY THE COMMISSION

Lack of staff resources with appropriate profiles (e.g. financial officers) in some services at a certain point in time

Deficiencies in the IT tools such as NEF (in 2009) and CPM (in 2010)

Extended time needed by the Commission service to react to or analyse the documents submitted by the consortium

Changes in the officials responsible for project negotiation with the consequence that some checks were redone

Poor communication between the different officials involved in negotiation

Fragmentary requests or request presented very late by the Commission service for changes to grant preparation forms or description of work

Generous deadlines or lack of strict deadlines set by the Commission service for the submission of updated description of work and grant preparation forms

Linearity of certain process steps (e.g. between panel conclusion and formal opening of negotiation with the consortium)

Differences in the structure of proposals and the description of work requires an unnecessary extra time for the consortia in preparing their proposals

Note: The information was collected through interviews with the managers in the Commission and through a desk analysis of 29 projects.

--------------------------------------------------