EUROPEAN COMMISSION

Brussels, 29.4.2020

COM(2020) 184 final

REPORT FROM THE COMMISSION

TO THE EUROPEAN PARLIAMENT, THE COUNCIL AND THE COURT OF AUDITORS



Evaluation of the Consumers, Health, Agriculture and Food Executive Agency, the Executive Agency for Small and Medium-Size Enterprises, the Innovation and Networks Executive Agency, the Education, Audiovisual and Culture Executive Agency, the Research Executive Agency and the European Research Council Executive Agency

{SWD(2020) 73 final} - {SWD(2020) 74 final} - {SWD(2020) 75 final} - {SWD(2020) 76 final} - {SWD(2020) 77 final} - {SWD(2020) 78 final}

Introduction

To achieve its goals the European Union supports a wide field of policy areas through a range of funding programmes. The programmes provide grants and other forms of funding for Member States, businesses, researchers, non-governmental organisations and others. In accordance with Article 317 of the Treaty on the Functioning of the EU, the Commission is responsible for budget implementation. In accordance with Article 62 of the Financial Regulation, the Commission shall implement the budget directly, under shared management with Member States or indirectly by entrusting budget implementation tasks to third parties (such as third countries, Member State organisations, international organisations, public-private partnership bodies).

In the case of directly managed programmes, the Commission decided in 2003 that it would be adequate to delegate the management of certain tasks in the implementation of some programmes to executive agencies, while allowing the Commission to concentrate on its core tasks. The agencies were created 1 to be specialised entities for executing some specific tasks throughout the lifetime of a given project, implementing the budget and providing relevant information to the policymakers. The executive agencies are autonomous entities under the supervision of the Commission. 

To achieve their objectives, the agencies execute grant and public procurement procedures, manage contracts, and implement the necessary budgetary procedures. In addition, the agencies provide support in programme implementation, such as collecting, processing and making available information on the programme implementation. They monitor the progress and impact of the projects. They inform potential applicants about funding opportunities and support applicants and beneficiaries in applying the procedures. They inform the Commission on programme implementation, projects results and impact. Outsourcing of management tasks should nevertheless stay within the limits set by the institutional system as set out in the Treaty. This means that tasks assigned to the institutions by the Treaty, which require discretionary powers in translating political choices into action should not be outsourced. Outsourcing should, moreover, be subject to a cost-benefit analysis.

Currently six executive agencies manage funding programmes for research and innovation, education, citizenship, culture, environment, climate action, transport, energy, telecommunications, agriculture, food, health, consumers, competitiveness and small and medium-sized businesses.

Before outsourcing tasks to these executive agencies the Commission performed a prior cost-benefit analysis (ex ante). In this analysis it considered several aspects: (1) the costs of coordination and checks; (2) the impact on human resources; (3) efficiency and flexibility in the implementation of outsourced tasks; (4) simplification of the procedures used; (5) proximity of outsourced activities to final beneficiaries; (6) visibility of the EU as promoter of the EU programme concerned; and (7) the need to maintain an adequate level of know-how inside the Commission 2 .

According to the cost-benefit analysis carried out in 2013, the delegation of certain programme management tasks to the agencies was estimated to be more cost-effective than keeping the activities in-house. It anticipated that the alignment of more coherent programme portfolios with the agencies’ core competences and their brand identities would bring qualitative benefits. In addition, it projected that bringing together the management of different EU programmes would bring synergies, simplification and economies of scale.

To assess how the agencies perform, key performance indicators have been set and are monitored continuously. Three years after the creation of each agency 3 and every three years after this, the Commission has to assess whether each agency functions well. It assesses whether the results anticipated by the ex ante cost-benefit analysis have been achieved. On this basis, it can identify possible areas for improvement and possibly review the scope of the tasks it delegates.

The Commission has evaluated the performance of all six executive agencies individually 4 in 2018/2019. All evaluations were supported by external studies 5 following the same methodology and applying the Commission’s better regulation principles 6 . The evaluations assessed whether the agencies had fulfilled their tasks in an effective and efficient way, whether there were overlaps, gaps or inconsistencies in the management of the programme portfolio by the agencies, and whether there was a clear delineation of tasks between each agency and the parent DGs or other executive agencies 7 . The evaluations include a retrospective cost-benefit analysis comparing the actual performance during the three-year evaluation period with the expected performance in the ex ante cost-benefit analysis.

The individual evaluations cover 2014 to 2016 for CHAFEA, INEA and EASME, 2015 to 2017 for EACEA and mid-2015 to mid-2018 for ERCEA and REA. The evaluation period varies because the agencies were created at different times.

Results

In general the evaluations showed that the six executive agencies were effective and efficient over the period analysed. They reached their objectives and the vast majority of the targets in their key performance indicators. They managed to cope with a challenging environment, including new mandates, changes of portfolio, changes of organisational structure and changes in their governance as well as peaks of very high workload coupled with constraints on human resources. Stakeholders were interviewed and gave a generally very positive assessment of the relationship between the executive agencies and the Commission departments, which allow the Commission to concentrate on its core tasks and to be kept informed about the implementation and results of the programmes. Beneficiaries and experts working with the agencies rated the agencies very positively and staff in the agencies were generally positive about the agencies as employers.

Alongside this positive general picture, the individual evaluations identified a number of challenges and related shortcomings. These challenges include: (1) achieving their objectives; (2) the fact that some agencies manage parts of the same programme; (3) the diversity of programme portfolios; (4) the relationship with parent DGs; (5) the relationship with beneficiaries; (6) cost-effectiveness; (7) human resources management; and (8) the change of mandate. These eight challenges are discussed in detail in the following chapters. The Commission will take the lessons learned into account when preparing to delegate tasks to executive agencies in programmes under the 2021-2027 multiannual financial framework.

1.Good results for (almost all) key performance indicators

Overall, the six executive agencies managed the delegated programmes efficiently and achieved very good results for most key performance indicators 8 . The indicators for executive agencies relate to financial management, in particular to timely evaluation and grant finalisation — time-to-inform, time-to-contract, time-to-grant and timely execution of payments. Some agencies consistently achieved all or most of their targets for timely evaluation, conclusion of grant agreements and execution of payments — REA, ERCEA, EACEA, CHAFEA — or even surpassed them — time-to-pay in INEA. Very few did not meet some of their targets — EASME at the start of the programming period and INEA for the time-to-pay for one programme from the previous programming period during one year (2015). Several agencies also achieved yearly full execution of their operational budget — REA, ERCEA. This efficient handling of financial management might leave room for more ambitious targets for some indicators.

With regard to the legality and regularity of transactions, the error rates of the 2014-2020 programmes are considered better than those from the previous generation of programmes (2007-2013). Agencies are however still managing projects from the 2007-2013 programmes and so, during the evaluation period, three agencies — REA, EASME and EACEA — still had reservations in their Annual Activity Reports for error rates above the 2% materiality threshold for 2007-2013 programmes. This is a legacy issue that will progressively disappear when all the projects of those programmes will have been closed 9 . Only one programme of the current programming period — the programme for small and medium-sized enterprises, COSME — had a temporary high error rate that required a reservation in 2018 from the Agency — EASME. The Agency has put in place corrective measures that improved the situation and as of 2019 the reservation was lifted.

The evaluations identified improvements in information technology tools and in procedures as key factors that contributed to improving the agencies’ key performance indicators. By bringing in new information technology tools, CHAFEA improved operational efficiency as well as providing better delivery of tasks under its project management systems. Administrative simplifications — a shift to electronic reporting and partially automated workflows — implemented by ERCEA and REA led to efficiency gains. EACEA further simplified and streamlined its internal and programme management arrangements. Payment procedures were improved and simplified in INEA 10 .

The evaluations show that agencies have put in place appropriate processes and procedures and that they have internal control systems that contribute to their efficiency and effectiveness. The agencies have developed strong internal control standards (with an exception for EACEA that qualified its internal control system as partially functioning in 2017 and 2018), including the management of financial and human resources. The agencies have put in place a large number of control and reporting mechanisms that allow progress against objectives to be closely monitored and risks to their operations to be prevented and mitigated in a timely and effective way.

EACEA’s evaluation highlighted serious shortcomings identified in two audits of the Internal Audit Service on grant management — from call to signature of contracts and from project monitoring to payment. Those shortcomings concerned among others the design and implementation of the Agency’s selection procedures for Erasmus+ and Creative Europe. The audit recommendations identified the need for action on the control environment for grants, the evaluation process, evaluation committees and the role of experts, as well as the contracting phase. The Agency drew up and implemented an action plan for each audit to address the issues and changed its procedures for the evaluation of proposal. One of the main actions taken by the new management of the Agency was a broad organisational change to improve and better structure the internal control framework of the Agency. The situation has now improved — the related recommendations for the first audit (from call to signature of contracts) have been closed by the Internal Audit Service. As a consequence, the reservation entered by the Agency because of those internal control weaknesses has been lifted as of the 2019 Annual Activity Report. For the second audit (from project monitoring to payment), the closure of all outstanding recommendations by the Internal Audit Service is expected by mid-2020. The Commission and notably the parent DGs of EACEA closely monitored how the actions to strengthen internal controls have been implemented through clarified and reinforced supervision mechanisms.

2.One EU programme managed by several agencies — how to ensure consistent implementation across the board

In the post-2014 period, the Horizon 2020 research programme is implemented by several executive agencies. Horizon 2020 is the biggest programme to be delegated to executive agencies in terms of financial allocation. In 2013, the Commission decided to delegate different strands of Horizon 2020 to four different agencies — REA, ERCEA, INEA and EASME. To ensure consistent implementation of Horizon 2020, the Common Support Centre (now named Common Implementation Centre) was created in 2014 and is hosted by a Commission department (DG Research and Innovation). It aims to provide the Commission DGs, executive agencies and joint undertakings dealing with the programme with common support for business processes linked to grant management, information technology, audit, data management and legal services.

While the need for a more standardised approach across the research family was generally recognised by the four executive agencies concerned, it represented a challenge for ERCEA due to the specificities of the European Research Council. The Agency demonstrated flexibility on how some of the more differentiated aspects of its work could be integrated into a common approach. These included: single beneficiary grants rather than institutions only; differences in the selection and management of experts; the requirement to serve the Scientific Council and resulting differences in terms of project reporting and follow-up. While constraints were encountered in how quickly the Common Support Centre was able to develop new information technology tools to support the introduction of new instruments, it has been assessed as responsive in addressing specific needs.

REA was also mandated to provide administrative and logistical support services to all entities involved in Horizon 2020 management. The evaluation observed that the extension of REA’s remit to include the provision of common administrative and logistical support services for the research and innovation family has strengthened consistency, particularly through the introduction of a common approach across Horizon 2020. REA was also mandated to provide participant validation services, not only for Horizon 2020 but also to a wide variety of programmes: Erasmus+, Creative Europe, Europe for Citizens, EU Aid Volunteers, Competitiveness of Small and Medium-sized Enterprises (COSME), Health and Consumer Programmes, Research Fund for Coal and Steel, Competitiveness and Innovation Framework Programme, Internal Security Fund, Asylum, Migration and Integration Fund, Rights, Equality and Citizenship, Justice Programme and the fund for information provision and promotion measures concerning agricultural products implemented in the internal market and in third countries.

As of 1 January 2018, as part of the single electronic data interchange area (SEDIA), the Commission centralised in REA the legal validation of third parties and the preparation of financial capacity assessments for all Commission departments and executive agencies that implement grants and procurements. REA delivered a high quality and effective service to its clients and other stakeholders for the management and provision of central support services. It achieved or exceeded the targets set in its key performance indicators. The Agency coped well with the increased workload following the expansion of its mandate to SEDIA. Overall, these developments demonstrate that REA is increasingly becoming a central provider of support services to the Commission departments and the executive agencies.

3.One agency managing several EU programmes — how to deal with diversity

The evaluations showed that working with a relatively high number of diverse thematic areas and tasks can pose a challenge to agencies’ effectiveness. For example, the broadness of activities delegated to EACEA (Erasmus +; Creative Europe, Europe for Citizens, European Solidarity Corps and EU Aid Volunteers) means that the Agency was confronted with quite diverse actions in terms of scope and scale. CHAFEA manages programmes in the fields of consumers, health, agriculture, and food safety. Its tasks include procurement, grant management and organising political events. This diversity of programmes and tasks resulted in the different units in CHAFEA being organised in rather divergent ways. This led to a complex working environment with limited possibilities for synergies.

EASME manages a wide portfolio of programmes covering COSME, Horizon 2020, the programme for environment and climate action (LIFE) and the European Maritime and Fisheries Fund (EMFF). Although the Agency’s performance was assessed as effective overall, a diverse portfolio resulted in a high workload and the need to coordinate with a large number of parent DGs. As a result, the management of new calls and subsequent activities was more complex.

In INEA, the evaluation identified difficulties linked to the portfolio’s diversity: for example better call planning between the three sectors — transport, energy and telecoms — of the Connecting Europe Facility (CEF) during the preparation of the work programmes was mentioned as a potential gain in efficiency.

Programme portfolios of agencies were assessed as coherent even if further synergy gains were expected. In 2016 under the CEF, the Commission and INEA launched a call looking for synergy projects in the fields of energy and transport. This joint call was not completely successful, as the eligibility conditions under the CEF Regulation were too restrictive. Synergies were also expected between the CEF and the transport and energy strands of Horizon 2020 but due mainly to differing timescales (research takes time to develop into infrastructure projects) there were fewer synergies than expected 11 .

For EASME, the evaluation reported that there is a high potential for synergies between the various programmes managed by the Agency due to their thematic affinities. Despite the Agency’s organisational set-up of multi-programme departments facilitating cross-programme cooperation, the evaluation identified a tendency to work in silos. This was due to different objectives and procedures that made it hard to exploit synergies. The Enterprise Europe Network used both for the Horizon 2020 instrument for small and medium-sized enterprises and for the COSME programme, on the other hand, was identified as a good example of synergy. Including LIFE and Horizon 2020 in the portfolio of a unique agency allowed also additional synergies across projects on sustainability and transition to climate neutral economy.

In some cases, synergies through the use of similar information technology tools and processes should also be improved, for example in CHAFEA and INEA. The CEF telecoms strand has different types and numbers of beneficiaries to those of the two other CEF strands limiting the scope for synergies. In EASME, there have been attempts to harmonise procedures and templates between the different programmes. The application of the Horizon 2020 common approach and common tools to other programmes has started with some success in INEA with CEF and in EASME with EMFF and COSME and with limited gains in CHAFEA. Their application to LIFE in EASME has just started and no measurement is available yet.

The diversity of thematic areas goes together with a higher number of parent DGs. This has consequences on the governance of the individual agencies but also on the practical collaboration between the agencies and the DGs.

4.Relationship with parent DGs — the challenge of working together

The Framework Regulation 12 establishes general provisions for all executive agencies. Each agency is established through an individual Commission decision (act of establishment) and programmes are delegated through acts of delegation that identify their parent DGs. A memorandum of understanding between each agency and its parent DG(s) sets out flexible provisions to ensure overall policy coherence and communication between the agency and its parent DGs, with the aim of avoiding duplication and micro-management. The Commission also provided further guidance in guidelines on executive agencies 13 .

Generally, no evidence of duplication, overlaps or inconsistencies were identified between the agencies and their parent DGs. The delimitation of responsibilities between the agencies and their parent DGs were found to be generally clear. The evaluations found the formal and informal communication mechanisms between the agencies and the Commission sufficiently frequent and effective to ensure that parent DGs are kept informed about the agencies’ performance and the state of play of implementation of the EU programmes. The secondment of Commission officials to occupy management positions in the agencies as well as a significant proportion of staff having a previous experience in the Commission departments were also mentioned as assets, providing the agencies with a high degree of expertise and knowledge of the parent DGs’ programmes and policies (in particular in the transition period). This also helped to promote a closer relationship between both sides.

In a limited number of cases, evidence of some duplication of work and micro-management were reported (CHAFEA). In other occasions, the supervision role of the parent DGs needed to be clarified or updated (EACEA). Similarly, when EASME was mandated with new programmes previously managed by the Commission and when for example Commission staff were recruited in EASME, in some cases the division of tasks between the Agency and its parent DGs was blurred. In some cases, the evaluations suggested that the memorandums of understanding and the guidelines on executive agencies could be improved in some instances by better defining the role of the parent DGs in the governance and the division of the tasks between agency and parent DG. In 2019-2020, the Internal Audit Service of the Commission also analysed the current situation as regards governance and supervision of the executive agencies. Based on this analysis, it identified a certain number of best practices or advices (e.g. clearer definition of supervision, role of the lead parent DG, guidance on internal control supervision and reporting, etc.) that the Commission services will take into account when revising the framework of supervision of the executive agencies.

Distance was also considered as an obstacle for the good relationship between the agencies and the parent DGs. For CHAFEA, the distance between the Agency in Luxembourg and some of its parent DGs based in Brussels was presented as a reason for some difficulties between the Agency and the Commission departments. The EACEA evaluation showed that the relocation of the Agency within Brussels close to the buildings of the parent DGs helped dialogue and improved the collaboration between them.

Communication between agencies and Commission departments is key: various formal measures (meetings of Steering Committees) or informal measures (regular meetings between a Commission unit and its contact unit in the agency or direct access to databases) have been adopted to facilitate effective communication and direct contact between the agency and the parent DGs. Communication is particularly important in situations where the agencies have to report on the results of programme implementation.

One objective of setting up the executive agencies was to allow the Commission to focus primarily on its core tasks. The evaluations confirmed that the existence of the executive agencies and their role in implementing programmes have enabled the Commission to concentrate on policymaking. Moreover, policymaking relies in part on feedback from the programmes and project implementation. The executive agencies play a key role in gathering this information on the ground and transforming it into policy feedback needed by the Commission departments. The evaluations showed that all six executive agencies provide policy feedback to the Commission, sometimes already very extensively on specific topics.

Agencies provide policy feedback through different channels: meetings, reports, briefings or data sharing. The policy support can be provided on a regular basis or as response to specific requests by the parent DG. REA developed a comprehensive and structured policy feedback mechanism, created a project monitoring and policy feedback task force and produced a catalogue of options to assist their parent DGs in better formulating their policy support needs. Specific initiatives to strengthen the relevance of policy feedback, such as events to review lessons learned and research results across thematic clusters of projects and the setting up of a staff network were organised. REA progressed on this matter by developing annual policy feedback plans tailored to each of the delegated programme parts that are agreed between the Agency and its parent DG. In July 2016, EACEA adopted a strategy for knowledge management. This was followed by mapping of the relevant practices that existed in different units in the Agency, examining the potential for improvement, and defining what is needed to realise this potential. Some indicators for measuring policy support were also proposed. INEA regularly submitted country reports to its parent DGs with details of projects that are supported by the CEF programme as well as reports for each transport project covering several countries (Trans-European Transport Network corridors). CHAFEA shared its monitoring database with its parent DGs. ERCEA provided useful feedback to Commission departments through briefings and data analysis. This has tended to be in response to specific requests for support to provide information and/or analysis about particular policy topics, such as climate change, artificial intelligence and gravitational waves. EASME policy feedback on energy efficiency was considered as good practice because of its content (the information provided by EASME was well targeted, of high quality, and always very useful) and its timeliness.

The evaluations demonstrated however that despite these good practices, policy feedback from executive agencies tends to lack standardisation and is developed in an ad hoc manner. Wider recognition and better awareness among the Commission departments about the policy feedback offered by the agencies is needed. The definition of the content, nature, quantity and regularity of this policy feedback needs to be further agreed between the agencies and the Commission. The evaluation highlighted that one of the key challenges for the next programming period is the need for agencies and the Commission to work more closely together and agree on information sharing.

One example of such an approach comes from the Common Support Centre for Horizon 2020. The Common Support Centre has begun the process of defining policy feedback as a business process with the executive agencies dealing with Horizon 2020. This will require input from both the parent DGs — to better define and formulate their needs for the extraction of policy-relevant information — and from the executive agencies — to determine what can usefully be provided.

In 2019-2020, the Internal Audit Service of the Commission analysed the way executive agencies and joint undertakings implementing Horizon 2020 provide policy feedback. Based on this analysis, it identified a number of best practices or advices that could be implemented in the future design of the policy feedback mechanisms: e.g. defining policy feedback, defining the policy feedback process, defining the policy feedback needs or facilitating the policy feedback from all types of programmes (not only the policy driven ones but also the bottom up ones). This analysis specifically targeted Horizon 2020 but it should be extended to the management of the other EU programmes.

5.Proximity to beneficiaries — dedicated one-stop-shops in constant search for a better service provided

All the agencies achieved their objectives for proximity to beneficiaries and visibility of EU programmes. All six agencies benefited from high satisfaction rates among beneficiaries of programmes managed by the agencies: 76% for EASME, 80% for CHAFEA, 86% for REA, 87% for INEA, 89% for EACEA and ERCEA. Some agencies — REA, EACEA and CHAFEA — also make use of independent, external experts to evaluate proposals. The satisfaction of those experts with the services provided by the agencies is also very high. Satisfaction rates among unsuccessful applicants are obviously lower. CHAFEA also measures a satisfaction rate for service providers due to its procurement activities. The relatively low satisfaction (58%) among service providers is linked both to the nature of this financial instrument and certain aspects of the quality of the service delivered by CHAFEA, for instance the timeliness of the application process, feedback on the proposal, the contract negotiation procedure and other aspects of the Agency’s service.

In the surveys carried out in the various evaluations, beneficiaries praised a number of simplifications (CHAFEA), the high standard for professionalism (INEA), the service-oriented attitude (EASME), the strong competence of the staff (EACEA), efficient procedures (INEA), the improved and simplified procedures for payments (INEA).

Beneficiaries and unsuccessful applicants also identified areas for improvement. Among others it is worth mentioning: better communication and feedback (INEA); further consolidation of processes related to selection of independent experts, validation of expert lists and participation of Commission staff in project monitoring activities (REA); user-friendliness and functionality of the existing IT tools (REA, CHAFEA, EACEA, INEA); external communication activities (EACEA); time to reply to tenders (EASME); or the ethics review and monitoring process (ERCEA).

The agencies implemented several actions to increase programme visibility. These included information days, project management workshops, proactive use of social media tools and good quality graphic and communication materials. Agencies improved their websites by providing relevant materials related to the programmes such as project information, brochures, maps and statistics. The many channels of communication used by the agencies have resulted in better visibility for the programmes.

6.Significant savings and simplifications achieved — in constant search for more cost-effectiveness

The evaluations revealed substantial differences in the programme management costs of the executive agencies. The executive agencies implementing Horizon 2020 were found to be more cost-effective than EACEA and CHAFEA. The programme management cost values in this group of agencies (the ratio between their administrative and operational budgets for payments) were below the 4.72% average for all agencies, ranging in 2016 from 0.89% (INEA) to 3.61% (REA including the provision of common administrative and logistical support services 14 ). By contrast, the 2016 programme management costs of CHAFEA were 10.11%, while those of EACEA were 7.20%. These two executive agencies also have the lowest values for budget ‘per head’, which in 2016 amounted to EUR 1.46 million in EACEA and EUR 1.37 million in CHAFEA (compared to the average for all agencies of EUR 3.65 million). For EACEA it is mainly related to the complexity and diversity of its programme portfolio, the diverse nature of its applicants and beneficiaries, the high volume of applications and grants, as well as their small average size. In CHAFEA it is related to the diversity of its programme portfolio, the small size of the Agency and the higher average cost of staff linked to their high level of expertise and the location of the Agency.

Chart: Cost of programme management and budget per head

Source: Triennial evaluations of the executive agencies

The retrospective cost-benefit analyses for the executive agencies revealed that in all cases the executive agency scenario was a cost-effective solution compared to an in-house scenario, in which the Commission would implement the programmes. In the table below, actual savings are compared to the two sets of estimations drawn up in 2013. The first set was provided by the external contractor who performed the ex ante cost-benefit analysis before the Commission decision to delegate the 2014-2020 programmes to executive agencies 15 . The second one was set out in the Specific Financial Statements 16 by the Commission for each of the executive agencies taking into account further synergies to be achieved (higher savings expected).

Chart: Savings during the tri-annual evaluation period estimated vs actual, million EUR

Source: Triennial evaluations of the executive agencies 17

The results of the six analyses show that the savings achieved differ from the 2013 estimates. Most executive agencies (with the exception of CHAFEA) exceeded the initial cost savings estimations to a very large extent. EASME and REA achieved savings much higher than the initial expectations, followed by ERCEA, EACEA and INEA 18 . CHAFEA achieved much lower savings than initially estimated (EUR 0.4 million compared to EUR 4.4 million for the 2013 cost-benefit analysis and EUR 2.3 million for the Specific Financial Statement estimates).

The reports identify a number of reasons for savings that were common to most of the executive agencies. The savings resulted primarily from a higher share of lower-cost staff (contract agents) employed within the executive agencies, which confirms the findings of the ex ante cost-benefit analysis. There were also significant savings on expenditure for infrastructure and operating costs (INEA, EASME, REA). Another reason for savings was that fewer staff were actually employed in the agencies compared to the 2013 forecasts. These lower staff numbers are due to reductions in staff numbers in the Specific Financial Statements compared to the ex ante cost-benefit analyses and high vacancy rates in the early years of implementation (2014-2015).

Lastly, a common feature for all agencies during the evaluation period is that they are constantly taking operational measures to further improve their efficiency. In particular the agencies, in cooperation with the Commission, continued to improve their procedures and programme management functions and introduced a number of simplifications. Procedures became more electronically-managed (paperless workflows, e-submission of proposals, automatic check functions integrated into the electronic forms, automation of project reports for the beneficiaries, etc.) and new IT tools were made available to streamline procedures. In the four executive agencies managing Horizon 2020, the improvements and simplifications included: wider use of remote evaluation of proposals; the introduction of a single set of rules; electronic signature of grant agreements; the Participant Portal as a one-stop-shop for interactions with participants; a single reimbursement rate; and a flat rate for indirect costs.

However, the reports showed that in all cases staff expenditure increased more than initial estimates in 2013 for the evaluation period. This was due in part to the agencies’ need to attract and recruit more specialised (and thus more costly) staff than initially envisaged. Another reason is that the costs of staff expenditure estimated in the Specific Financial Statements did not take into account salary indexation, promotions and increasing staff seniority. For CHAFEA, staff expenditure was also affected by the need to hire staff in higher function groups to sustain the attractiveness of the Agency’s location in Luxembourg. These are some of the lessons to be learned and taken into account in reflections in advance of future delegation to the agencies to implement 2021-2027 EU programmes.

7.Human Resources management — positive results but challenges ahead

The level of staffing in executive agencies is set out in the EU budget and each programme delegated to an agency generates a corresponding number of staff. This link between programme and number of staff creates some rigidity that has prevented the agencies from using their workforce flexibly to cope with work peaks 19 . Nonetheless, during the period evaluated some agencies experienced good practices of dynamic reassignment of staff within the same agency, within the limits of the applicable rules.

The large majority of the agencies’ staff are allocated to programme implementation and only around 10% to support functions — a lower figure than in the Commission 20 . Given their task of implementing a large part of the EU budget, some 25% of agencies’ staff work in the financial domain, compared to 9% of Commission staff. These figures seem to demonstrate that the agencies’ focus on operational programme management and the related financial tasks is adequately mirrored in the organisational roles of their staff.

The staff opinion survey carried out in 2016 demonstrated a level of staff engagement 21 that was above the Commission average for five out of the six executive agencies. EACEA EASME, ERCEA, INEA and REA all had staff engagement values between 65% and 70%, compared to the Commission average of 64%. For CHAFEA, the staff opinion survey reported lower staff wellbeing compared to other executive agencies and a decrease in staff engagement. The evaluation gives as explanations the Agency’s small size and relative isolation from other agencies due to its location in Luxembourg. This did not allow CHAFEA staff to participate to the same extent in the job market of the six agencies.

In general, the survey showed that staff perceive executive agencies as stimulating and dynamic places to work with excellent internal communication and efficient processes and procedures in place. Staff are also of the opinion that agencies encourage collaborative working as well as new and better working methods.

In the same survey, however, staff were less positive about their career development and mobility opportunities. In four of the six agencies, this indicator is below the Commission average: especially in the smaller agencies, staff find it difficult to move to another job that matches their skills and competencies.

Based on mature participatory processes, the agencies have put in place strategies that have helped them to effectively follow-up on the key human resources matters identified in the staff opinion survey. Among the positive developments, it is important to highlight the increased differentiation of the recruitment channels, involving the European Personnel Selection Office for more generalist profiles and executive agencies’ own processes for specialist needs. This differentiation helped to avoid bottlenecks in the recruitment processes in particular when wide scale and fast recruitments were needed.

The results of the 2018 staff opinion survey show that in general, the efforts of the agencies have paid off as the results for the main indicators have further improved with the exception of EACEA linked partly to difficulties in the area of internal control in 2018. The evaluations identified the need to create more synergies and efficiency in the way some human resources matters are handled, building on current common rules and procedures. The evaluations have however identified challenges that still need to be addressed such as developing staff skills further in the area of policy feedback, as well as investing more in career guidance and development.

8.Change of mandate — a crucial moment

The start of a new programming period (as in 2014 and in 2021) is a challenging time for the agencies. Their mandates are changed to entrust them with the management of a new generation of programmes. They have to put in place the procedures for the start of the new programmes while still following and executing payments for projects from the previous programmes. In 2014, management of the new programmes meant an increased financial allocation requiring the development of new management structures, information technology systems and support services to support the expansion, and coping with a phase of major staff recruitment. This transitional period is characterised by managerial complexities associated with implementing two different sets of rules, processes and procedures. Despite this challenging environment, the agencies were considered effective for the start of the new programming period (2014-2015), because they invested significant resources in streamlining and documenting their procedures for the new programmes. The agencies most impacted were EASME and INEA, whose staff increased by 175% and 147% respectively between 2013 and 2016, and whose budget increased by approximately 220% and 350% over the same period. Due to this intense starting phase, EASME and INEA failed to meet several of their key performance indicators 22 .

With their new mandates, the agencies were confronted with more thematically diverse portfolios (see section 3) and were supervised by more parent DGs (see section 4). This was particularly the case for CHAFEA and INEA whose parent DGs increased from one to four and for EASME, supervised by seven parent DGs compared to four in the previous programming period. The evaluations also found that collaboration between agencies and parent DGs delegating programme implementation tasks for the first time had to grow during the first years after delegation in order to create a relationship of trust between the agencies and the parent DGs. This affected the governance (see section 4) and the established practices of the agency with its parent DGs. For CHAFEA, INEA and EASME, the agencies also changed names and acronyms, which are important for visibility and proximity with beneficiaries, but the evaluations showed that such changes did not seem to have any major impact (see section 5). The changes also affected the organisational structures of the agencies. INEA and EASME introduced a new management layer. In general, the evaluations assessed the agencies’ internal organisation to be closely aligned with the tasks entrusted to them. The evaluations of EACEA and EASME mentioned that the structure of these agencies could be improved and the evaluation of CHAFEA identified its small size as an issue, in particular for business continuity. The Agency’s relatively small size has obvious consequences on its ability to maintain knowledge and business continuity when key staff unexpectedly leave the Agency or change assignment internally.

The mandates of REA, EASME, INEA and EACEA were changed during the programming period. For EACEA, several mandate extensions took place over the period analysed. The evaluations showed that the affected agencies were flexible and effective in addressing the extension of their mandates. All four agencies managed to cope with the increased workload and/or additional tasks. The evaluations showed however that the delegation procedure (cost-benefit analysis, information to the Council committee for executive agencies, amendments to the acts of delegation, etc.) proved to be too complex for limited extensions of mandate. Indeed, there is currently no simplified delegation procedure for limited extensions of mandate during a programming period. Those limited extensions resulted in an unnecessary burden for the Commission departments and the agencies.

Conclusion

The evaluations of the six executive agencies confirmed their good performance and their added value in managing EU programmes. At the same time, they revealed some challenges the agencies still face. Some challenges are common to all agencies. One of them is the quality of the policy feedback the agencies deliver to the Commission, another is the need to update some aspects of the legal framework (memoranda of understanding or guidelines on executive agencies). Other challenges are more agency specific stemming from the agency’s size, location, the diversity of portfolio of programmes to manage, or related to internal control issues. The Commission and the agencies are preparing and implementing action plans to remedy the shortcomings identified and to face the challenges ahead.

Some of the replies to those challenges will also need to be addressed as part of the reflections on the future of the executive agencies. The Commission has presented its proposals for the next generation of EU programmes (2021-2027) and it is currently undertaking a cost-benefit analysis to determine the adequate role that executive agencies should play in implementing them. To this end, the executive agencies need to be able to continuously improve their performance, to continue operating in a cost-effective manner and to deepen cooperation with the Commission. The lessons learned from this common evaluation process will feed the reflections on the delegation of the next generation of EU programmes to the executive agencies.

(1)

Council Regulation (EC) No 58/2003 of 19 December 2002 laying down the statute for executive agencies to be entrusted with certain tasks in the management of Community programmes (OJ L11, 16 January 2003, page 1).

(2)

Cost-benefit analysis for the delegation of certain tasks regarding the implementation of Union Programmes 2014-2020 to the executive agencies — Final report for the Commission of 19 August 2013.

(3)

Article 25 of Council Regulation 58/2003.

(4)

Evaluation of the Consumers, Health, Agriculture and Food Executive Agency (CHAFEA), SWD(2020) 75;

Evaluation of the Executive Agency for Small and Medium-Size Enterprises (EASME), SWD(2020) 76;

Evaluation of the Innovation and Networks Executive Agency (INEA), SWD(2020) 73;

Evaluation of the Education, Audiovisual and Culture Executive Agency (EACEA), SWD(2020) 78;

Evaluation of the Research Executive Agency (REA), SWD(2020) 77;

Evaluation of the European Research Centre Executive Agency (ERCEA), SWD(2020) 74.

(5)

Study supporting the evaluation of the Consumers, Health, Agriculture and Food Executive Agency (CHAFEA) 2014-2016, Public Policy and Management Institute (PPMI), Centre for Strategy & Evaluation Services (CSES) and Maastricht University;

Evaluation of the Executive Agency for Small and Medium-sized Enterprises (EASME) 2014-2016, Valdani Vicari & Associati and Deloitte;

Study supporting the Evaluation of the Innovation and Networks Executive Agency (INEA) 2014-2016, Centre for Strategy & Evaluation Services (CSES), Public Policy and Management Institute (PPMI) and Centre for Industrial Studies (CSIL);

Study supporting the evaluation of the Education, Audiovisual and Culture Executive Agency (EACEA) 2015-2017, Public Policy and Management Institute (PPMI);

Study supporting the Research Executive Agency (REA) 2015-2018, Public Policy and Management Institute (PPMI), Centre for Strategy & Evaluation Services (CSES) and IDEA Consult;

Study supporting the evaluation of the European Research Council Executive Agency (ERCEA) 2015-2018, Centre for Strategy & Evaluation Services (CSES) and Public Policy and Management Institute (PPMI).

(6)

   Guidelines SWD: https://ec.europa.eu/info/files/better-regulation-guidelines_en

Toolbox: https://ec.europa.eu/info/better-regulation-toolbox_en

(7)

The Commission’s better regulation guidelines were applied in a proportionate way. There was no need to assess the criteria of ‘EU value added’ and of ‘relevance’ as the agencies carry out tasks, which the EU legislator had entrusted upon the Commission.

(8)

In parallel to the evaluations, the European Court of Auditors carried out an audit on the performance of INEA in implementing the Connecting Europe Facility programme (Special Report No 19/2019 of 7 November 2019 “INEA: benefits delivered but CEF shortcomings to be addressed”). As regards the KPIs of INEA, the Court acknowledged that INEA met the targets but recommended to set more results-oriented goals and indicators as future KPIs. The Commission has argued in its reply to the report of the Court that INEA’s KPIs are adequate for the measurement of the performance of INEA in accordance with the mandate that it has received from the Commission and should be distinguished from the KPIs measuring the results of the programmes themselves.

(9)

Due to the small proportion of the financial impact on the overall financial portfolio of the agencies, these reservations were lifted by the three executive agencies as of their 2019 Annual Activity Reports.

(10)

The Special Report of the Court of Auditors on INEA’s performance confirmed the good management results of INEA thanks to the use of standardised procedures.

(11)

The Special Report of the Court of Auditors on INEA’s performance confirmed that the potential for further synergies between CEF and Horizon 2020 and among the various CEF strands has not materialised fully yet.

(12)

Council Regulation (EC) No 58/2003 of 19 December 2002 laying down the statute for executive agencies to be entrusted with certain tasks in the management of Community programmes (OJ L11, 16 January 2003, page 1).

(13)

Guidelines for the establishment and operation of executive agencies financed from the Union budget (C(2014)9109).

(14)

By excluding the provision of common administrative and logistical support services, the REA ratio between administrative and operational budget becomes 2.58%.

(15)

Cost-benefit analysis for the delegation of certain tasks regarding the implementation of Union Programmes 2014-2020 to the executive agencies — Final report for the Commission of 19 August 2013.

(16)

Communication of the Commission SEC(2013) 493 and the last amendments of the Specific Financial Statements accompanying the establishment acts of the executive agencies, as follows: C(2017)4900 of 14 July 2017 for REA; C(2013)801 of 23 December 2013 for INEA; C(2013)9048 of 17 December 2013 for ERCEA; C(2014)6944 of 2 October 2014 for EASME; C(2018)1716 of 13 November 2018 for EACEA and C(2014)927 of 17 December 2014 for CHAFEA.

(17)

REA’s mandate was substantially changed since the beginning of the evaluation period, hence any comparison of the initial CBA estimates with the SFS estimates and the actual figures must be considered carefully, given the change in the tasks and workload level. This graph illustrates the savings of all agencies for the purpose of comparability among them.

(18)

The Special Report of the Court of Auditors on INEA’s performance also confirmed the lower administrative costs (though slightly higher staff costs).

(19)

The Special Report of the Court of Auditors on INEA’s performance also identified a need for more flexibility in the allocation of staff among programmes within the same agency.

(20)

Taking into account all jobs in the Commission, 27% of jobs were allocated to administrative support and coordination roles in 2018. This percentage has been relatively stable since 2012 and represents well the responsibilities and related institutional tasks.

(21)

Staff engagement is an aggregate indicator. The full list of sub-questions is provided in the staff working documents of the different evaluations.

(22)

Time to pay for TEN-T programme in 2015 for INEA, execution of commitment appropriations for TEN-T and Marco Polo II for INEA in 2014, time to grant for Horizon 2020 (except Energy efficiency) in 2014-2015, LIFE (2015 only) and Fast Track Innovation (2014-2016) for EASME.