52009SA0013

Abbreviations and glossary

AAR : annual activity report

AWP : annual work programme

CBA : cost–benefit analysis

COBU : Committee on Budgets of the European Parliament

CREA : Regulatory Committee for Executive Agencies

DG : Directorate-General (of the European Commission)

DG BUDGET : Directorate-General for the Budget

DG ECFIN : Directorate-General for Economic and Financial Affairs

DG ENTR : Directorate-General for Enterprise and Industry

DG ENV : Directorate-General for the Environment

DG INFSO : Directorate-General for the Information Society and Media

DG SANCO : Directorate-General for Health and Consumers

EACEA : Education, Audiovisual and Culture Executive Agency

EACI : Executive Agency for Competitiveness and Innovation (formerly IEEA)

EAHC : Executive Agency for Health and Consumers (formerly PHEA)

ERC : European Research Council

ERCEA : European Research Council Executive Agency

Framework Regulation : Council Regulation (EC) No 58/2003 of 19 December 2002 laying down the statute for executive agencies to be entrusted with certain tasks in the management of Community programmes (OJ L 11, 16.1.2003, p. 1)

IEEA : Intelligent Energy Executive Agency (became EACI in 2008 after extension of its mandate)

MFF : multiannual financial framework — contains the financial perspectives of the EU budget divided into expenditure "headings". The MFF for the current period (2007–13) contains the following headings: 1. Sustainable growth and employment, 2. Preservation and management of natural resources, 3. Citizenship, freedom, security and justice, 4. The EU as a global partner, 5. Administration, 6. Compensations

Parent DG : The Commission department responsible for the Community programmes the management of which is delegated to the agencies

PHEA : Public Health Executive Agency (became EAHC in 2008 with the extension of its mandate to new programmes)

REA : Research Executive Agency

TAO : Technical Assistance Office

TEN-T EA : Trans-European Transport Network Executive Agency

Executive summary

I. Executive agencies are Community bodies with legal personality established by the Commission in order to implement by delegation all or part of EU spending programmes. The Commission remains responsible for the policy tasks and the supervision of the activities delegated.

II. Executive agencies are a relatively recent phenomenon in the EU institutional landscape. Since 2003, six executive agencies have been created, responsible for managing a financial envelope of around 32 billion euro for the period until 2013. The number of personnel authorised for 2009 is 1 339. They are mainly contract staff.

III. The Court’s audit aimed at assessing whether the delegation of management tasks to the executive agencies had proved to be a successful option for implementing the European budget.

IV. To this purpose, it examined in particular:

(i) whether the decision to create an executive agency was supported by an adequate analysis of the needs and the potential gains;

(ii) what benefits have been achieved in terms of financial savings, improved service and other efficiency gains;

(iii) whether the Commission has effectively carried out its supervisory role over the agencies’ activities.

V. The Court found that:

(i) the initiative of setting up the executive agencies was mainly driven by constraints on employment within the Commission rather than being based on the intrinsic features of the programmes themselves;

(ii) the cost–benefit analyses (CBAs) required by the legislation in order to support the decision to create the agencies took little account of non-financial aspects and omitted some important factors on the side of costs. The picture provided, though undeniably positive from a financial point of view, was not entirely accurate. The contribution of the CBAs to the decision-making process was rather limited;

(iii) in terms of benefits achieved, there are clear cost savings stemming from the prevalence of lower paid contract staff, even when one considers the additional costs of the new posts created for supervision and support at both the Commission and the agencies. However, the actual amount of the savings depends on the redeployment of the Commission staff who were previously doing the work taken over by the agencies, and on the suppression of the contract staff posts within the corresponding programme areas at the Commission. The lack of reliable information on the ex ante situation at the Commission does not allow the extent of the savings to be verified;

(iv) as a result of their specialisation in identifiable and specifically defined tasks, the agencies provide better service delivery in terms of reduced time for contracting, more rapid approval procedures for technical and financial reports and lower payment delays. Other qualitative improvements are the simplification of processes as well as increased external communication and dissemination of results, which contribute to enhance the visibility of the EU. On the other hand, the expected flexibility in hiring staff is not demonstrated;

(v) the Commission’s supervision of the agencies’ work is quite limited: the annual work programmes (subject to the Commission’s approval) are scarcely used for setting targeted objectives; the monitoring is mainly focused on indicators related to how the tasks are carried out rather than to the results produced; the reports are usually confined to budgetary execution and omit to measure progress made on a multi-annual basis and identify corrective actions for the future.

VI. The Court’s main recommendations are that the Commission should:

(i) reconsider its procedures for identifying the potential for externalisation and for considering the establishment or the extension of agencies;

(ii) improve the quality of the cost–benefit analyses to allow them to contribute fully and effectively to the decision-making process;

(iii) ensure that it has relevant, reliable data on workload and productivity related to the implementation of the delegated tasks, both before and after externalisation;

(iv) identify the success factors which have led to better results at the executive agencies and apply similar factors to programmes which continue to be managed in-house;

(v) supervise the agencies by setting results-oriented and targeted objectives, using a limited number of relevant performance indicators which form the basis for next years’ objectives.

Introduction

1. Executive agencies are Community bodies with legal personality established by the Commission in order to implement all or part of EU programmes on its behalf and under its responsibility.

2. Delegating the day-to-day management of spending programmes to specialised agencies has been a leitmotiv of public service reform at national level over the last two decades [1]. This had essentially two aims: improving service delivery (through reduced red tape, increased specialisation and enhanced commitment to specific results) and enabling central authorities to concentrate on "core functions" such as policy design and supervision.

3. The setting-up of executive agencies in the EU context became an issue in 1999, when the Commission undertook a major review of its externalisation policy as part of a package of overall administrative reform [2]. This was in response to the Parliament’s concerns about the management of spending programmes by more than one hundred bodies, known as technical assistance offices (TAOs), operated by private contractors [3].

4. In 2000, the White Paper on reform [4] affirmed the need for the Commission to refocus on its core tasks by externalising some of the administrative work when externalisation:

(a) is a more efficient and cost-effective means of delivering the services or goods concerned;

(b) does not involve the exercise of discretionary powers (i.e. it does not imply political choices);

(c) is justified on its own merits rather than being a mere substitute for compensating for Commission staff shortages.

5. Furthermore, the White Paper required that the decision to externalise should be taken on a consistent basis across the European Commission, so that similar instruments are used in similar cases [5]. The externalisation of management tasks would include "testing a new type of implementing body headed by Commission staff" [6]; i.e. the executive agencies.

6. The use of TAOs was therefore phased out. The management of some programmes (mainly related to external aid) was taken over by the Commission, through significant recourse to contract staff; some programmes were discontinued and others were entrusted to executive agencies [7].

7. The 2002 Financial Regulation makes provision for creating executive agencies [8]. Their statute is defined by a common legal framework [9]. They are established by the Commission, after endorsement by the Member States under the comitology regulatory procedure [10]. The European Parliament is also associated in the decision: it gives its opinion on the need for any proposed agency.

8. Six executive agencies have been established since 2003 (see Table 1).

9. At the time of the Court’s audit, four of the six agencies were operational (EACI, EAHC, EACEA and TEN-T EA, the latter since April 2008).

Table 1

Executive agencies established since 2003

Executive agencies | Total budget managed under the MFF 2007–13 (billion euro) | Initial and final term | Number of staff (full-time equivalent units) Budget 2009 | Programmes managed [] | Logo |

+++++ TIFF +++++

Executive Agency for Competitiveness and Innovation (EACI), former IEEA | 1,7 | 1.1.2004– 31.12.2008 (extended until 31.12.2015) | 147 | Implements part of the "Competitiveness and innovation" framework programme and Marco Polo II | |

+++++ TIFF +++++

Executive Agency for Health and Consumers (EAHC), former PHEA | 0,5 | 1.1.2005– 31.12.2010 (extended until 31.12.2015) | 50 | Implements EU "public health" programme, "Consumer policy" programme and "Better training for safer food" initiative | |

+++++ TIFF +++++

Education, Audiovisual and Culture Executive Agency (EACEA) | 3,7 | 1.1.2005– 31.12.2008 (extended until 31.12.2013) | 394 | Implements programmes and actions in the fields of education and training, active citizenship, youth, audiovisual and culture | |

+++++ TIFF +++++

Trans-European Transport Network Executive Agency (TEN-T EA) | 8,0 | 26.10.2006– 31.12.2008 (extended until 31.12.2015) | 99 | Implements the "Trans-European transport network" (TEN-T) programme | |

+++++ TIFF +++++

Research Executive Agency (REA) | 6,5 | 1.1.2008– 31.12.2017 | 349 | Implements the "People" programme and parts of the "Capacities" and "Cooperation" programmes under FP7. Carries out support services for a large part of FP7 | |

+++++ TIFF +++++

European Research Council Executive Agency (ERCEA) | 7,5 | 1.1.2008– 31.12.2017 | 300 | Implements the "Ideas" programme under FP7 | |

| 27,9 | | 1339 | | |

10. In several cases the original duration or mandate of an agency has been extended:

(a) EACEA mandate has been extended four times to deal with the new generation of programmes 2007–13 and to add new programmes in the field of education;

(b) PHEA has become EAHC by adding the "Consumer policy" programme 2007–13 [11] and the "Food safety training" programme to the new "Health" programme 2008–13 [12], and

(c) IEEA has become EACI, by adding parts of the CIP and the Marco Polo II programme to the "Intelligent energy" programme.

11. Executive agencies implement EU spending programmes totalling around 27,9 billion euro for the current multiannual financial framework (MFF 2007–13). Taking into account also the tasks managed under the previous MFF, the total amount is around 32 billion euro.

12. The administrative budget of the executive agencies is drawn from the financial envelope of the programmes under their management. The corresponding appropriations are therefore outside the ceiling on resources set by heading 5 of the MFF, which refers to the "general" administrative expenditure of the EU institutions.

13. The Commission is responsible for supervising the executive agencies’ activities. The agencies’ directors are officials appointed by the Commission; the members of the steering committees are also appointed by the Commission [13]. The agencies’ annual activity reports are annexed to the reports of their parent DGs.

14. Executive agencies differ in various ways from the so-called "regulatory agencies", which have been mainly developed since the 1990s, to deal with tasks of regulation at EU level related to the expansion of the internal market [14]. The main differences are that executive agencies implement spending programmes and are directly dependent on the Commission, whereas regulatory agencies mainly provide common rules and services and operate under a management board composed of Member States’ representatives (for more details see Annex II).

Audit objective and approach

15. The overall objective of the audit was to assess whether the delegation of operational tasks to executive agencies had proved to be a successful instrument for implementing the EU budget.

16. This was done by addressing the following questions:

(a) Were the decisions to create the executive agencies supported by adequate analyses of the needs and the potential gains?

(b) What benefits have been achieved in terms of cost savings, improved performance and other efficiency gains?

(c) Has the Commission effectively carried out its supervisory role?

17. The audit was carried out from April to November 2008. Audit evidence was collected through documentary analyses, file reviews and interviews at all executive agencies and at the Commission. National good practices were used as a benchmark in the areas of objective-setting and performance measurement.

Audit observations

The creation process

Initiative to set up the executive agencies mainly driven by constraints on employment within the Commission

18. As reflected in the Framework Regulation [15], the rationale for outsourcing implementation tasks to executive agencies was for them to provide better service and efficiency gains and for the Commission to focus primarily on its institutional tasks such as policy-making and strategic management. Specific reasons for setting up an executive agency should be established through a needs assessment based on these principles.

19. More recently, the general guidance issued by DG BUDG [16] spelled out the essential factors on which a needs assessment should be based:

(a) the need for the Commission to focus on legislative and strategic tasks in policy formation and monitoring, including those connected with Community programmes;

(b) a clear separation between policy programming (the Commission's core business) and implementation of technical projects, where no discretionary powers implying political choices are involved;

(c) the need for a high level of technical expertise throughout the project cycle;

(d) the possibility of economies of scale through a high level of specialisation or the regrouping of similar programmes or activities within one agency;

(e) the need to carry out certain activities with increased visibility.

20. The Court assessed how the Commission had put these principles into practice and how it had identified possible drivers for devolution of tasks based on the nature and specificities of the programmes, such as the level of technical expertise required, the possibility of reaching standardisation of time-consuming tasks and the grouping of similar activities, leading to synergies.

21. The Court found that the initiative to set up agencies was mainly taken by the Commission in response to practical problems, in particular the need to compensate for discontinuing TAOs and to allow for the continuation of the programmes they managed when no new resources were made available under the ceiling on the Commission’s administrative budget set by the MFF (paragraph 12). Despite the intentions set out in the White Paper and the guidance documents, staff shortages (in number and specialisation) at the Commission were the main driver for externalisation (see Box 1). Recent documents confirm that it continues to be the Commission’s practice to explore the option of creating new executive agencies when faced with a shortage of resources [17].

Box 1

The CBA for the Marco Polo programme (implemented by EACI) explicitly states that the reason for delegating implementation to an executive agency was the absence of sufficient human resources at the Commission.

The creation of PHEA followed considerable implementation problems and increasing payment delays related to the need for additional resources within DG SANCO after the dismantling of the former corresponding TAO. As a consequence of those problems, when they came to adopt a new "Public health" programme in 2002, the Council and the European Parliament insisted that "appropriate structural arrangements" should be made to carry out the programme. This was the argument to support the establishment of PHEA.

The focus on staff shortages has also produced a wide range of implementation modes within the same programme. The three specific programmes within the "Competitiveness and innovation" framework programme (CIP) are managed in various ways: the "Intelligent energy Europe" programme continues to be managed largely by EACI. For the "Information and communication technnology — policy support" programme, the Commission opted to continue direct management by DG INFSO. Regarding the "Entrepreneurship and innovation" programme (EIP), DG ENTR delegated major parts of its share to EACI and DG ENV delegated the EIP eco-innovation projects. For the EIP parts of DG ECFIN, the CIP legal base stipulated the continuation of the implementation of the CIP financial instruments by the European Investment Fund.

Limited contribution from cost–benefit analyses to the decision-making process

22. The Framework Regulation requires that the setting-up of an executive agency should be based upon a cost–benefit analysis (CBA) [18], providing justification for the outsourcing and taking into account a number of factors such as the costs of coordination and checks, the resulting impact on human resources at the Commission, possible financial savings, the benefits afforded by additional efficiency and flexibility in the implementation of outsourced tasks, simplification of the procedures used, proximity of outsourced activities to final beneficiaries, visibility of the Community as promoter of the Community programme concerned and the need to maintain an adequate level of know-how inside the Commission [19].

23. The Court assessed the quality of the CBAs supporting the decisions to set up the agencies, analysing whether all relevant factors had been adequately considered.

24. It found that, in general, the CBAs were restricted to the financial aspects and took little account of other factors justifying the outsourcing as set out in paragraph 22. Emphasis was placed on the savings deriving from employing cheaper contract staff rather than permanent officials [20]. The aspects of improved performance and efficiency gains were scarcely considered, though including such elements in the analysis might have strengthened the case for the creation of agencies. As such, the analyses were mainly cost comparisons, rather than CBAs in the proper sense of the term.

25. On the side of costs, some elements related to the functioning and the winding-up of the agencies are missing. Thus, the costs of the additional staff needed at the Commission to supervise the agencies and at the agencies for horizontal functions were not or not accurately included (except for TEN-T EA); nor were the costs of running the programmes beyond the established life-cycle of the agency.

26. Moreover, the time needed to set up a new agency was underestimated and the transition costs of the overlapping Commission/agencies were not included in the analyses, even though it was known that it took about two years before an agency reaches its full operational autonomy [21]. For PHEA, the transition costs were included in the analysis but the initial forecast of three to six months for the start-up of the agency was significantly over-optimistic [22].

27. In addition, there are some elements of uncertainty in the analyses. First, the comparison is generally made using a single average unit cost for the various categories of contract staff whereas in practice they vary in grade and therefore in cost [23]. This average is based on the Commission’s staff composition, which shows a prevalence of lower grades compared to the (specialised) personnel to be recruited by the agencies and leads to an understatement of the costs for the agencies option.

28. Second, the estimates used to calculate the staff needs of the agencies and the resulting impact on the Commission’s human resources (in terms of permanent posts saved and re-allocated and contract posts suppressed) were not backed up by reliable data on workload and productivity (except for the TEN-T EA). Not only does this compromise the accuracy of the process, but it also hinders an effective ex post evaluation of the actual level of the benefits (as explained in paragraph 37 below).

29. In most cases, correcting the CBAs to take account of the weaknesses set out above would not affect the conclusion in favour of setting up an agency (given the large margin of savings on staff costs): the exception is the smallest agency (PHEA — now EAHC), where the balance of purely financial arguments could have cast doubt on the recommendation to proceed. Nonetheless the CBAs are open to the criticism that they do not give a satisfactory overall picture of the financial consequences and other benefits in terms of efficiency and effectiveness.

30. In practice, the Court did not identify any case of CBAs advising against setting up an executive agency. However, not all cases of positive recommendations were followed by the creation of an agency [24] as the final decision was based on a more comprehensive evaluation of the arguments for and against. Therefore, the CBAs brought little added value to the process and did not provide an adequate basis to allow decision-makers to form an opinion about different alternatives.

The benefits achieved

Savings from lower staff costs but overall quantification difficult

31. Concerning the financial benefits, the Commission’s assumption is that savings are achieved by recruiting a 75 % proportion of contract staff instead of permanent officials at the agencies [25].

32. In fact, a total of 1339 posts were authorised for the agencies in the budget for 2009, of which 1227 were newly created: 941 contract staff, 256 temporary staff and 30 national experts [26]. This represents a real proportion of 70 % contract staff. Assuming a post-per-post average salary difference of about 50 %, it is clear that considerable benefits stem from the outset from the prevalence of lower paid staff compared to the staff who would have been hired at the Commission.

33. However, there are some elements of uncertainty which make the exact amount of the savings difficult to quantify. Firstly, even assuming that the Commission would have hired exactly the same number of people as the agencies, the "all Commission permanent officials" hypothesis does not consider that (at least some) contract staff might also have been recruited. Secondly, actual data on staff costs available for one agency show that the average cost for contract staff is around 10 % higher than that used by the Commission for general purposes (see Box 2) [27]. This is due to the fact that agencies’ contract staff are mostly high grade whereas the Commission’s average cost reflects the composition of its personnel, with prevalence of contract staff in mainly low grades. In view of a more satisfactory comparison, it would be necessary to have average staff costs differentiated by category of personnel [28].

34. In addition, the staff savings need to be reduced by the additional costs of the new supervisory functions performed by the parent DGs and the overhead costs related to the horizontal functions in each agency [29]. The Commission reports that the proportion of staff assigned to such supervisory and support functions is around 13–14 % of the total staff of the agencies [330].

35. On the other hand, for those programmes previously managed directly by the Commission services, any saving is dependent on the effective redeployment of the staff previously assigned to the work that has been transferred to the agencies and on the suppression of the contract staff posts within the corresponding programme areas at the Commission [31].

Box 2

For legislative proposals, DG BUDG uses the following average unit costs, estimated for the Commission as a whole:

122000 euro per year for permanent/temporary officials, and

64000 euro for contract staff.

Recruiting contract staff instead of permanent officials would therefore allow for per capita theoretical savings of about 50 %.

36. Following the creation of the six executive agencies, the Commission planned to free 267 staff (159 permanent posts + 108 contract agents) within the corresponding DGs. Of these staff, 209 (131 permanent posts + 78 contract agents) had been actually freed as of 1 January 2009 (see Table 2). For ERCEA, no staff are planned to be freed as the activities delegated are totally new.

37. However, the adequacy of these plans cannot be verified as there is no reliable information on the ex ante situation at the Commission concerning the staff allocated to the programmes taken over by the agencies or on the split in the workload between policy tasks (which remain with the Commission) and implementation tasks. Nor does the Commission have such figures for staff who currently carry out similar tasks within its Directorates-General.

38. In this respect, the Court notes that the separate presentation of the operating budgets of the agencies in the Community general budget has increased transparency. Indeed, this provides relevant information about the resources (and related costs) needed for running programmes which is not necessarily available when the Commission runs them itself.

39. Even including the impact of the uncertainties above, the savings margin still remains significant.

Table 2

Freed posts

| Total number of staff to be freed by 2012 | Staff actually freed as of 1.1.2009 [100] |

Permanent posts | Contract agents | Permanent posts | Contract agents |

EACEA | 15 | 18 | 30 | 18 |

IEEA/EACI | 28 | 5 | 30 | 0 |

PHEA/EAHC | 22 | 0 | 15 | 0 |

TEN-T EA | 56 | 6 | 56 | 6 |

ERCEA | 0 | 0 | 0 | 0 |

REA | 38 | 79 | 0 | 54 |

Total | 159 | 108 | 131 | 78 |

Better service delivery

40. Better service delivery is one of the benefits expected from concentrating limited and well-defined tasks in a single agency (the specialisation factor).

41. The Court assessed the management aspects of programme implementation using quantitative indicators to compare the performance of the agencies with that of the Commission prior to the externalisation. This concerned the evaluation of proposals and contracting, the monitoring of contractors’ performance and the payment procedures. Other aspects of a qualitative nature were also considered.

42. The audit showed that the executive agencies generally perform better than their parent DGs before. This is demonstrated by (see Box 3):

(a) a shorter time-to-contract (evaluation and negotiation of proposals);

(b) more rapid approval procedures for both technical and financial reports on the projects; and

(c) shorter time-to-payments (from the payment request to payment).

Box 3

As an example, the contracting time for the "Public health" programme was reduced from 345 days when the programme was managed by the parent DG to 219 days; the payment period went down from 503 to 91 days; and the time required for approving the technical and financial reports was shortened from 90 to 42 days.

43. In addition, the agencies simplified the management procedures, thereby reducing the administrative burden for applicants and project promoters (see Box 4).

44. The audit also noted that the agencies enhanced communication with potential applicants and ensured the dissemination of results to a wider public, thus providing more visibility for EU actions. The agencies organise several events (Infodays, information sessions, workshops), either directly or through "national contact points", to inform potential beneficiaries about new programmes, guiding them through the procedures and providing data on previous programmes (see Box 5).

45. Further examples are shown in Annex III.

Box 4

As an example, EACEA simplified the call for proposals and the application forms, moved to online submission, clarified the selection criteria, reduced the number of contract annexes, extended the use of flat rate and lump sums and simplified the reporting obligations.

Box 5

All EACI activities are supported by a communication plan that defines, for each of the programmes, which activity is planned in which period of the year. A website, a logo, leaflets and news alerts are designed for each programme. In a previous report (Special Report No 7/2008 — Intelligent Energy 2003-2006) the Court observed that these initiatives had a positive impact on the degree of applicants’ satisfaction.

46. As regards extending the mandate of existing agencies, synergies were achieved by using tools already developed for similar programmes. This concerns in particular the communication tools used for sending out calls and informing potential applicants, the procedures to find and manage experts and the contract management.

Limited efficiency gains for recruitment

47. Executive agencies were also supposed to provide efficiency gains through more flexibility in hiring staff, the capacity to attract highly qualified personnel and more staffing stability.

48. On the first point, the recruitment of contract staff can by definition adjust more easily to agencies’ specific needs. However, the process of selecting adequate personnel proved to be more difficult than the Commission had assumed. Problems in hiring the staff are the main reason for the agencies’ slow start-up (see paragraph 26).

49. The appointment of the directors was a lengthy enterprise and finding a suitable candidate took an average of eight months [32]. For the post of accountant, the rule that the accounting officer should be an official seconded by the Commission had to be amended [33] because no suitable candidates were found for EACI and PHEA.

50. Hiring staff with specialised expertise also proved to be a more time-consuming task than expected. The agencies make use of the same lists of candidates as the Commission and the establishment plans are also subject to the approval of DG ADMIN (besides the approval of the parent DG). Moreover, recruitment at the agencies is made at lower grades for temporary posts and more years of experience are requested for contract staff compared to contract staff with similar tasks at the Commission. This reduces the attractiveness of the job positions, despite the offer of renewable contracts, in contrast to the three years’ maximum applicable to Commission contract staff.

51. The agencies can also use employment agency staff. Such staff are used extensively at EACEA, which has hired an average of 55 interim units per year. While this solution provided greater flexibility in coping with peak period problems, the high number of such staff reflects more fundamental problems of recruitment and conflicts with the expectations of experienced and stable staff.

The commission’s supervision of the agencies

A focus on budgetary execution

52. As mentioned above (paragraph 13), the Commission retains a supervisory role in respect of the executive agencies’ activities [34]. This should include the definition of clear priorities and result-oriented goals in the annual work programmes (AWPs) approved by the Commission and an assessment of the activities carried out by the agency through the annual activity reports (AARs) [35].

53. The Court assessed the use that was made of the agencies’ AWPs as a tool for objective-setting, the appropriateness of the performance indicators established for 2006 to 2008, as well as the quality of the reporting for the years 2006 and 2007.

54. The timing of the AWPs does not enable the Commission to make use of them to define the agencies’ priorities and objectives. The executive agencies’ Framework Regulation requires the adoption of the agencies’ AWPs no later than the beginning of each year [36], whereas the AWPs of the Community programmes delegated to the agencies, according to the Financial Regulation, are only adopted by the first quarter of the year. This calendar is not suitable for ensuring consistency. As a matter of fact, except for EAHC, the AWPs were approved by the Commission late in the middle of the year (EACI, 2008), at the end of the year (EACEA, 2007 and 2008), or not at all (EACEA, 2006).

55. With regard to the content of the AWPs, the executive agencies have generally been assigned tasks without results-oriented objectives and related targets.

56. The agencies’ AWPs set out too many indicators (from 52 to 109 for 2008, see Chart 1) whereas good national practices suggest using only a limited number of key indicators.

57. Moreover, the indicators used are mostly related to the management activities (tasks and workload) rather than to the results of the programmes managed. Only a limited number of indicators aim at measuring delivery against objectives (effectiveness) and there are no indicators to measure the relationship between the resources employed and the results achieved (efficiency).

Chart 1

Number and type of indicators in AWPs

+++++ TIFF +++++

58. As regards reporting, this is usually confined to the budgetary data (e.g. the consumption of commitment appropriations and payment delays). The AARs are not always consistent with the indicators of the AWPs, make no systematic comparison with all the targets set in the AWPs and make no reference to progress achieved from year to year, or to corrective actions required for the future.

59. Due to the weaknesses set out in paragraphs 54 to 58, the Commission's control over the agencies’ activities is not fully effective. On an operational level, there are informal contacts on a regular basis between staff at the Commission and the agencies (facilitated by their geographical proximity). If this ensures communication on day-to-day management issues, it does not take the place of well-structured relationships based on clear performance measurement instruments and reports.

60. This is also crucial in the longer term for the Commission’s strategic functions (its "core business"), which require knowledge of project implementation on the ground (especially for policy domains that rely heavily on evidence-based project results to develop new initiatives).

Conclusions and recommendations

61. The Court’s audit showed that the initiative to set up the six executive agencies created since 2003 was mainly driven by the need to compensate for staff shortages at the Commission.

62. The Commission made pragmatic use of the ability to set up executive agencies, responding to problems as they arose. On the other hand, the Commission’s own guidance suggests a more systematic approach to the creation of agencies; and it is suboptimal to consider setting up agencies only when forced to do so by staff shortages or other external constraints.

63. The cost–benefit analyses accompanying the decisions to create the agencies focused on cost comparisons, took little account of other relevant benefits, and were not backed up by reliable data on workload and productivity. In addition, some of the costs were incorrectly estimated. The CBAs brought limited added value to the decision-making process.

Recommendation 1

The Commission should reconsider its procedures for identifying the potential for externalisation and for considering the establishment or the extension of agencies.

The Commission should ensure that the CBAs are more complete to allow them to contribute fully and effectively to the decision-making process.

64. Financial savings clearly result from recruiting contract staff at lower grades compared to the Commission’s officials. However, the precise amount of the savings cannot be accurately quantified. Moreover, the adequacy of the plans concerning the posts needed by the agencies and the posts to be freed at the Commission as a consequence of the externalisation cannot be assessed in the absence of reliable data available on the ex ante situation at the Commission.

65. As a result of their specialisation in well-defined tasks, the executive agencies are delivering better service than their parent DGs did before. They conclude contracts, make payments and approve technical and financial reports on the projects more rapidly. They have also improved the processes and increased external communication and dissemination of results, thus contributing to enhance the visibility of the EU. On the other hand, the Court found that elements of rigidity in the recruitment of staff can hinder the attainment of the expected efficiency gains.

Recommendation 2

In order to fully evaluate the benefits produced by the executive agencies, the Commission should ensure that it has relevant, reliable workload and productivity data related to the implementation of the delegated tasks, both before and after externalisation. This is also essential to ascertain the capacity of the existing agencies to cope with additional tasks.

The Commission is also invited to identify the success factors which have led to better results at the executive agencies and to apply similar factors to programmes which continue to be managed in-house.

The Commission should reflect upon measures to simplify the recruitment of the agencies’ staff.

66. The Commission’s supervision of the executive agencies’ activities is not fully effective. The timing of the annual work programmes does not enable the Commission to make use of them to define the agencies’ priorities and objectives. The executive agencies have generally been assigned tasks without results-oriented and targeted objectives. Monitoring, whilst making use of a large number of indicators, is restricted to the management activities and does not cover the key aspects of effectiveness and efficiency. The reporting requirements do not extend beyond simple budgetary data.

Recommendation 3

The agencies should be supervised more specifically on a results basis. The timing for the adoption of the annual work programmes should be made more consistent. The Commission should set SMART objectives and monitor their achievement by a limited number of key performance indicators which form the basis for next years’ objectives. This is also crucial in the longer term for the Commission’s strategic functions (its "core business"), which require knowledge of project implementation on the ground and evidence-based results.

This report was adopted by the Court of Auditors in Luxembourg at its meeting of 15 and 16 July 2009.

For the Court of Auditors

+++++ TIFF +++++

Vítor Manuel da Silva Caldeira

President

[1] This has been part of the modernisation agenda for better governance in the UK, Canada and New Zealand since the early 1990s. More recently, other European countries (Sweden, France, Germany and Italy) have followed these examples.

[2] "Orientations for the Commission's externalisation policy", Communication from Mrs Schreyer and Mr Kinnock, SEC(1999) 2051/7, 14 December 1999.

[3] TAOs were criticised in particular for the vague definition of the tasks, the poor control exercised by the Commission over their activities, the threat to budget transparency due to the use of operational appropriations to fund administrative costs borne by the TAOs, and non-compliance with the relevant accountability requirements. See European Parliament, "COBU Working Documents 10, 11 and 12 on the future of the TAOs: supervision or dismantling?", 4 November 1999.

[4] "Reforming the Commission" — A White Paper — Part I, COM(2000) 200 final/2, 5 April 2000.

[5] "Reforming the Commission", cit., p. 11.

[6] "Reforming the Commission", cit., p. 11.

[7] Six out of 126 TAOs existing in 2000 were replaced by executive agencies (three by EACEA, one by EAHC, two by EACI).

[8] Articles 54(2)(a) and 55 of Council Regulation (EC, Euratom) No 1605/2002 (OJ L 248, 16.9.2002, p. 1).

[9] Regulation (EC) No 58/2003.

[10] This is done through the Regulatory Committee for Executive Agencies (CREA), provided for in Article 24 of the Framework Regulation, which is composed of the representatives of the Member States and chaired by the representative of the Commission.

[] More detailed information on the programmes managed by the agencies can be found in Annex I.

[11] Decision No 1926/2006/EC of the European Parliament and of the Council (OJ L 404, 30.12.2006, p. 39).

[12] Decision No 1350/2007/EC of the European Parliament and of the Council (OJ L 301, 20.11.2007, p. 3).

[13] Article 7 et seq. of the Framework Regulation (EC) No 58/2003.

[14] The Court has recently issued a special report on the performance of regulatory agencies. See Special Report No 5/2008 — The European Union's agencies: Getting results.

[15] See especially the preamble (Whereas 4 to 6) of the Framework Regulation (EC) No 58/2003.

[16] SEC(2006) 662 final, 31 May 2006, "Guidelines for the establishment and operation of executive agencies financed by the general budget of the European Communities".

[17] SEC(2007) 530 "Planning and optimising Commission human resources to serve EU priorities", p. 15, and COM(2008) 135 final "European agencies — The way forward", p. 3. The screening did not report a need for new executive agencies in the near future. However, externalisation of a limited number of other programmes to existing agencies was envisaged.

[18] The cost–benefit analysis compares all the relevant costs and benefits over time to determine whether the benefits outweigh the costs, and if so, by what proportion.

[19] Article 3 of the Framework Regulation (EC) No 58/2003. Article 25 of the Framework Regulation provides for such analyses to be repeated every three years in the framework of the external evaluation reports to be drawn up on the operations of each agency.

[20] Following an earlier indication from the European Parliament, the personnel of executive agencies should be composed of 75 % contract agents and 25 % officials seconded from the Commission and temporary staff (see COBU Working Documents 10, 11 and 12, 4.11.1999, cit., p. 25). For TEN-T EA, however, the CBA assumed that the composition would be 55 % contract staff and 45 % officials and temporary staff. A staff composition of 60 % contract staff and 40 % officials was taken into account in EACEA CBA for programme strands relating to external relations (EMECW, Tempus, EM II Action 2).

[21] In 2005 the Commission recognised that: "The most important lesson is that the time needed to plan and set up an executive agency should not be underestimated. The first agencies established point to a lead time of about two years from the Commission decision creating the agency to the moment when the agency is fully operational" (Issue paper ref. SG.C.2 D (2005) 11582, part 4).

[22] It took an average of almost two years from the decision to establish an agency and its full operation. In particular, this period was 24,5 months for PHEA; 24 months for EACI, 18 months for TEN-T EA; 12 months for EACEA. REA and ERCEA are expected to be operational by mid-2009, which is after a period of 19,5 months.

[23] Only EACEA used several rates to take into account the distribution of the personnel by grade.

[24] This was for instance the case for the envisaged Eurostat agency.

[25] See footnote 20. This percentage is applied to all the agencies, with the exception of TEN-T EA, for which a composition of 65 % contract and 35 % temporary and seconded staff has been authorised.

[26] Source: General Budget 2009. The remaining 112 posts (technically known as full-time equivalent (FTE)) are Commission staff seconded to the agencies (the so-called "frozen" posts): they generate a corresponding reduction of the Commission’s budget administrative appropriations, so they have no impact on the benefits. As regards the temporary posts, it can be assumed that the costs would be the same whoever the employer (the Commission or the agencies), so their impact on benefits can also be considered neutral. This calculation does not take into account the people previously employed by the TAOs.

[27] See the recent Interim evaluation report of the EACEA (paragraph 64).

[28] See also paragraph 27.

[29] The parent DGs have staff specifically devoted to the supervision of the agencies. On the other hand, all executive agencies have a director, a data protection supervisor, an internal auditor and an accountant.

[330] This percentage has decreased with the increase in size of the agencies, which has allowed for economies of scale.

[31] Article 13(6)(c) of Framework Regulation (EC) No 58/2003.

[100] Source: Working Document — Part III to the 2010 PDB — COM(2009) 300

[32] This was a key milestone, as the director is in charge of recruiting the rest of the staff.

[33] Article 30 of Commission Regulation (EC) No 1653/2004 of 21 September 2004 on a standard financial regulation for the executive agencies (OJ L 297, 22.9.2004, p. 6), amended by Commission Regulation (EC) No 1821/2005 (OJ L 293, 9.11.2005, p. 10): the accountant may now be recruited on the private market.

[34] Article 20 of the executive agencies’ Framework Regulation. More in general, Article 54(1) of the Financial Regulation applicable to the general budget of the European Communities (Council Regulation (EC, Euratom) No 1605/2002), "The implementing tasks delegated must be clearly defined and fully supervised as to the use made of them".

[35] Article 9(2) of the executive agencies’ Framework Regulation requires the annual work programme to entail detailed objectives and indicators. In its Opinion No 8/2001 (OJ C 345, 6.12.2001, p. 1) on the executive agencies, the Court had already stressed that "when the work programme is adopted at the beginning of the year, clear objectives should also be adopted for the agency together with performance indicators to assess how well the agency is carrying out its tasks". The Court had also pointed out that the annual activity reports "should contain sufficient information and analysis for an appreciation of the extent to which the objectives of the agency concerned have been met, and of the efficiency of the agency’s management".

[36] Article 9(2) of the executive agencies' Framework Regulation.

--------------------------------------------------

ANNEX I

DETAILED OVERVIEW OF PROGRAMMES AND BUDGETS MANAGED BY EXECUTIVE AGENCIES

Executive agency | Parent DG | Programmes | Amount managed by the EA in the MFF 2007–13 (million euro) |

Executive Agency Competitiveness and Innovation (EACI, previously IEEA) | DG TREN | CIP (IEE 2007–13) | 690,60 |

DG ENTR | CIP (EIP) | 375,80 |

DG ENV | CIP (EIP: Eco-innovation) | 181,00 |

DG INFSO | CIP (ICT) not managed by EACI | 0,00 |

DG TREN | Marco Polo II | 450,00 |

| | Total 2007–13 | 1697,40 |

| | | |

Education, Audiovisual and Culture Executive Agency (EACEA) | DG EAC | Lifelong learning | 728,66 |

DG EAC | Youth in action | 130,13 |

DG EAC | Culture | 320,88 |

DG EAC | Europe for citizens | 177,18 |

DG INFSO | Media 2007 | 696,97 |

DG AIDCO | Erasmus Mundus 2004–08 | 182,34 |

DG EAC | Erasmus Mundus 2009–13 (Actions 1 and 3) | 467,01 |

DG AIDCO | Erasmus Mundus 2009–13 (Action 2) | 336,00 |

DG RELEX | Erasmus Mundus 2009–13 (Action 2) | 19,00 |

DG ELARG | Erasmus Mundus 2009–13 (Action 2) | 57,00 |

DG ELARG | Erasmus Mundus — IPA windows 2009–13 | 37,50 |

DG AIDCO | Erasmus Mundus external cooperation window Asia–Latin America DCI 2008 | 47,82 |

DG AIDCO | Cooperation in Central Asia and Middle Eastern countries DCI 2007–08 | 15,39 |

DG AIDCO | European neighbourhood and partnership financial cooperation with Mediterranean and eastern European countries | 61,99 |

DG EAC | USA–EU cooperation in higher education and vocational education and training 2006–13 | 38,53 |

DG EAC | Canada–EU cooperation in higher education and vocational education and training 2006–13 | 16,50 |

DG RELEX | ICI bilateral educational projects | 13,67 |

DG ELARG | Erasmus Mundus Balkan window IPA 2007 | 4,00 |

DG ELARG | Erasmus Mundus external cooperation window IPA 2008 | 6,00 |

DG AIDCO | Tempus IV action | 204,20 |

DG ELARG | Tempus IV action | 94,50 |

| | Total | 3655,27 |

Executive Agency for Health and Consumers (EAHC, previously PHEA) | DG SANCO | Public health programme 2008–13 | 325,20 |

DG SANCO | Consumer policy programme 2007–13 | 101,50 |

DG SANCO | Better training for safer food initiative | 70,80 |

| | Total | 497,50 |

| | | |

Trans-European Transport Network Executive Agency (TEN-T EA) | DG TREN | Trans-European networks — transport | 8013,00 |

| | | |

Research Executive Agency (REA) | DG RTD | People programme (including Marie Curie fellowships) and parts of the Capacities programme (research for the benefit of SMEs) under FP7 | 6500,00 |

DG ENTR | Cooperation programme, themes "Space" and "Security" (multi-partner projects) under FP7 | |

| FP7 support services for the four research DGs (RTD, ENTR, INFSO, TREN): proposal submission and evaluation, payment of experts, validation of legal and financial documentation submitted by applicants, helpdesk for enquiries about FP7 | |

| | | |

European Research Council Executive Agency (ERCEA) | DG RTD | Ideas programme under FP7 2007–13 | 7510,00 |

| | | |

Total amount of programmes managed by EAs under the MFF 2007–13 | 27873,17 |

--------------------------------------------------

ANNEX II

DIFFERENCES BETWEEN EXECUTIVE AND REGULATORY AGENCIES

| Executive agencies | Regulatory agencies |

Setting up | Created by the Commission under the regulatory procedure | Created by the Council or by the Council and the Parliament |

Legal basis | Basic act of the programme Commission decision under Council Regulation (EC) No 58/2003 | Regulation of the Council or regulation of the Parliament and the Council |

Control | Under the control of the Commission | Under the control of a board, consisting of representatives of the Member States, the Commission and sometimes other interested parties |

Seat | Brussels/Luxembourg (close to Commission headquarters) | Seat decided by the Council |

Scope | Executive and operational tasks related to a specific spending programme | Assistance to Member States and the Commission in various forms (e.g. adoption of common binding rules, issuing of technical/scientific opinions, establishment of inspection reports, networking and exchange of information) and help in implementing Community regulation |

Life/time | Limited | In general, unlimited, but depends on the legal base |

Budget | Fully subsidised as a part of the programme financial envelope | Combination of EC subsidies and self-financing |

Discharge | Subject to individual discharge for their administrative budget while the operational expenditure is part of the general discharge given to the Commission | Subject to individual discharge for their own budget (administrative and operational expenditure) |

--------------------------------------------------

ANNEX III

MEASURES TAKEN IN RESPECT OF SIMPLIFICATION, COMMUNICATION, VISIBILITY AND DISSEMINATION OF RESULTS

| EACI | EACEA | EACH |

Simplification of procedures | Simplification measures were mainly aimed at the final beneficiaries: Reduction of bank guarantees requiredExtension of reporting period for technical reportsRevision of threshold for audit certificates for non-public bodiesElectronic submission of proposalsSimplification of forms, guidelines and selection criteriaSecond pre-financing instead of interim payment based on verified incurred costsUse of flat ratesCreation of specific databases for experts and projects | Simplification measures were taken at various phases of the project cycle, such as: Simplification of forms, guidelines and selection criteriaReduction in the number of application forms and of requested documentsUse of declarations on honour instead of extensive financial documentationUse of standard grant calculation forms in ExcelIntroduction of multiannual calls and permanent programme guides (Youth, Citizenship, Culture)Extended use of decisions rather than bilateral grants agreements Gradual shift toward online applications and submission (already in use for Citizenship); target: extension to LLP in 2008 and to all EACEA programmes by 2009–10Simplification of the final reports and of the beneficiaries' obligations in the reporting phaseExtended use of flat-rate financing and lump sums, mainly in Citizenship and Youth areas of activity | Simplification measures were taken on the internal side: Centralised organisational structure based on the Commission standard Financial circuit model 4 (Full centralisation of financial transactions)More streamlined and coordinated selection procedure bringing in more qualityCreation of specific databases for experts, projects, and applicantsDetailed handbooks with clear rules and well explained criteria |

Communication | Functional mailboxes to reply to queriesCommunication plan defining the activities planned during the year for each programme and for the networkWebsite, logo, poster leaflets and news alerts designed for each programme | Functional mailboxes to reply to queriesPublications on the website and in the calls for proposals of phone numbers to receive feedback and guidanceCooperation with national contact points to provide information on the programmes | Dedicated helpdesk to provide guidance to potential applicants (250 e-mails and 80 phone calls in 2008)Cooperation with the national focal points to provide information on the programmesDedicated website to inform on the calls for proposals and forms to useProduction of info-packages, CDs and posters |

Visibility of the Community as promoter of the Community programmes | More than 30 Infodays/yearDedicated website with about 200000 hits on the call for proposal sectionOrganisation of national Infodays in addition to European Infodays in Brussels | Several Infodays/information sessions/workshops organised (eg. Infodays for LLP, culture; media festivals; kick-off meetings with LLP project promoters, annual meetings of Erasmus Mundus consortia)Support to events organised by national contact pointsDedicated website with more than 300000 hits in 2008 | 16 Infodays all over Europe in 2008 with 2400 participantsDedicated website with an ever increasing number of hits23 technical meetings with expert groups organised in 2007 (1 600 persons invited)Creation of a dedicated network (national focal points) to exchange information on the programmes |

Dissemination of results | Publication of projects’ fact sheets on the websitePublication of project brochures and video reports on selected subjectsPresentation of examples of projects during the Infodays | Publication of selection results on the website immediately after selection decisions, followed by individual notification of resultsIndividual information on the reasons for non-selectionPublication on the website of best practices and compendiaInformation collected from project reporting used for dissemination to a wider publicInvolvement of the agency in the development of dissemination tools (EVE project) | Substantial improvement of feedback sent to successful and unsuccessful applicantsGuidelines for applicantsGuidelines for beneficiaries on how to prepare a final payment |

--------------------------------------------------