EUROPEAN COMMISSION

Brussels, 15.12.2017

SWD(2017) 601 final

COMMISSION STAFF WORKING DOCUMENT

Evaluation of the 11th European Development Fund

Accompanying the document

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL

Midterm Review Report on the External Financing Instruments

{COM(2017) 720 final}
{SWD(2017) 463 final}
{SWD(2017) 600 final}
{SWD(2017) 602 final}
{SWD(2017) 604 final}
{SWD(2017) 605 final}
{SWD(2017) 606 final}
{SWD(2017) 607 final}
{SWD(2017) 608 final}
{SWD(2017) 609 final}

Contents

Summary    

1. Introduction    

Purpose of the evaluation    

Scope of the evaluation    

2. Background to the initiative    

Description of the 11th EDF and its objectives    

Baseline    

3. Method    

4. Implementation state of play    

5. Responses to the evaluation questions    

Relevance    

Effectiveness    

Efficiency    

Coherence    

Added value    

Leverage    

6. Conclusions    

Annex 1. Evaluation questions    

Annex 2. 11th EDF reconstructed intervention logic    

Annex 3- Procedural information    

Annex 3a Response to the opinion of the Regulatory Scrutiny Board    

Annex 4 -Synopsis report of the stakeholders' consultation    

Annex 4a. External Evaluation Summary of Open Public Consultation Contributions    

Annex 5. Acronyms    

Annex 6. External evaluators' report, including Annexes    

Summary

This Staff Working Document evaluates the performance of the 11th European Development Fund (EDF) at mid-term, for the period January 2014 to mid-2017.

With a budget of EUR 30,506 million for the period 2014-2020, directly financed by European Union Member States outside of the Union's budget, the EDF is the largest financing instrument contributing to turn into action the European Union's development policy and external action. Its objective is to help 79 countries in Africa, the Caribbean and the Pacific alleviate poverty and to support the 25 Overseas Countries and Territories that all have a special link with some EU Member States to improve their competitiveness and reduce their vulnerability.

The 11th EDF is a relevant instrument to meet the objective of poverty reduction in our partner countries. Its flexibility largely enables the EU to respond to changing needs of its partners and changing EU priorities while offering a stable medium-term framework.

The various factors underlying persistent poverty in African Caribbean and Pacific States, international commitments such as the United Nations Sustainable Development Goals translated into EU policy through the 2017 European Consensus for Development and unprecedented migration/refugee flows justified an increased emphasis on security concerns and migration issues by addressing the root causes of irregular migration.

It is difficult to measure the performance of the 11th EDF in the absence of indicators in the legal base beyond the Millennium Development Goals/Sustainable Development Goals. Despite progress, there is still not enough focus on results. Its effectiveness largely depends on the partner countries policies and reforms. The use of National/Regional Authorising Officers has not always proven to be a satisfactory way to ensure ownership and partnership, which are key principles of the EDF. Their functioning is considered to have often hindered effectiveness and efficiency. Despite some inefficiencies at the level of the procedures to implement projects and programmes, the 11th EDF is an overall efficient instrument. For example, it is on-track in terms of budget execution, compared with the preceding EDF.

The coherence among the various components of the 11th EDF is satisfactory despite missed opportunities for synergy. There is also some overlap between the thematic component of the Development Cooperation Instrument and the 11th EDF.

The 11th EDF has a proven added-value among the set of the EU’s External Financing Instruments, and also compared to the instruments of Member States and other Development Partners. For example, the instrument has the flexibility to cover issues that other External Financing Instruments cannot, such as support to peace and security through the African Peace Facility; it also has valuable financial flexibility through its reserve to respond to unforeseen needs and the non-application of the budgetary principle of annuality.

The elements highlighted in this evaluation will feed into the reflection on how to improve the implementation of the 11th EDF for the remaining period until 2020, and on the future set of External Financing Instruments for the post-2020 period.

1. Introduction

This Staff Working Document 1 constitutes the 11th European Development Fund (EDF) Performance Review. It assesses whether the 11th EDF, which covers the period 2014-2020 is fit for purpose to deliver on its objectives 2 of reducing and eventually eradicating poverty in the African, Caribbean and Pacific (ACP) States and of attaining sustainable development of Overseas Countries and Territories (OCTs). It is largely based on an external evaluation 3 .

Purpose of the evaluation

This evaluation is retrospective and responds to Article 18 of Council Regulation (EU) 2015/322 of 2 March 2015 on the implementation of the 11th EDF which requires a Performance Review at the latest by the end of 2018. This evaluation takes place in 2017 so as to be aligned with and ensure consistency with the Mid-Term Review of the EU's External Financing Instruments 4  under Heading 4 'Global Europe' 5 of the 2014-2020 Multiannual Financial Framework 6 , even if the EDF is not funded from the EU budget. The Mid-Term Review of the instruments will culminate in a Mid-Term Review Report, as defined in the Common Implementing Regulation 7 .

Scope of the evaluation

The evaluation covers the period from 1 January 2014 to 1 June 2017. It includes all the 11th EDF components managed by the European Commission 8 , i.e. cooperation with ACP States at national, regional level and intra-ACP levels, and cooperation with Overseas Countries and Territories at territorial and regional levels. 

This mid-term evaluation concentrates on the 11th EDF as an instrument of the EU's development policy and external action. This means that the evaluation focuses, to the extent possible, on the elements contained in the Regulation on the implementation of the 11th EDF 9 (e.g. its principles, scope, flexibility and complementarity with other instruments) rather than on the programmes and projects that have been put in place on the basis of the instrument. Development Cooperation policies associated with the EDF legal base are not within the scope of this evaluation: they are the object of other evaluations 10 .

The geographic scope of the evaluation corresponds to that referred to in the Regulation on the implementation of the 11th EDF 11 .

In accordance with the EU Better Regulation guidelines 12 , the following evaluation criteria are used: relevance, effectiveness, efficiency, coherence, EU added value and leverage.

2. Background to the initiative

The External Financing Instruments make up a major part of the above-mentioned Multiannual Financial Framework's Heading 4 which, together with the EDF which functions outside the EU budget, provides the EU with the tools necessary to fulfil its role on the world stage and to ensure that it is able to live up to its ambitions in promoting its interests and universal values and principles such as democracy, human rights, peace, solidarity, stability and poverty reduction and to help safeguard global public goods.

Adopted early 2014, the External Financing Instruments were designed to ensure policy implementation, with the intention of remaining relevant for the entire duration of the Multiannual Financial Framework and therefore enabling the EU to implement external action policies as needed.

Description of the 11th EDF and its objectives

The European Union maintains privileged relations with the ACP group of 79 countries under the EU-ACP Partnership Agreement also known as 'Cotonou Agreement' 13 . The main objective of the Cotonou Agreement is to reduce poverty towards its eradication. The Agreement is based on fundamental principles: (i) equality of the partners and ownership of the development strategies, (ii) participation of various kinds of actors, (iii) pivotal role of dialogue, and the fulfilment of mutual obligations, and accountability, (iv) differentiation (according to partners' situation) and emphasis on the regional dimension.

The EU also enjoys a close relationship with 25 Overseas Countries and Territories (OCTs) that have constitutional links with four Member States 14 . The OCTs are associated to the EU through a regime based on the provisions of Part IV of the Treaty on the Functioning of the EU and the detailed rules and procedures laid down in the Overseas Association Decision of 25 November 2013 15 . The Overseas Association Decision seeks to strengthen OCTs' sustainable development by enhancing their competitiveness and their resilience, reducing their economic and environmental vulnerability and promoting cooperation with other partners.

The successive European Development Funds, funded by the EU Member States outside of the EU budget, have been the EU's main financing instrument for providing development aid to the ACP countries and the OCTs since the 1st EDF in 1959. The objectives of the 11th EDF are to reduce and eventually eradicate poverty in the ACP countries, and to attain sustainable development of the OCTs.

The 11th EDF was created by an intergovernmental agreement (the Internal Agreement) 16 signed in June 2013 that entered into force on 1 March 2015 after ratification by all EU Member States. While the High Representative's role is to ensure the overall political coordination of the Union's external action, the Commission implements the EU budget and thus also the Union’s external cooperation programmes 17 . An EDF Committee of Member States has been set up for the 11th EDF resources that the Commission administers 18 .

The 11th EDF intervention logic has been reconstructed on the basis of (i) the above-mentioned Internal Agreement and the Regulation on the implementation of the 11th European Development Fund, (ii) logical steps that lead from the EU’s development cooperation policy objectives to EDF instrument, as well as (iii) explicit and implicit assumptions that underlay each of these different steps. The intervention logic features in Annex 2 to the present document.

With total financial resources of EUR 30.5 billion for the period 2014-2020 19 , the 11th EDF, which is extra-budgetary, is the largest of all EU's External Financing Instruments in terms of volume, representing the equivalent of almost half of the Multiannual Financial Framework Heading 4 'Global Europe' 20 of EUR 66,262 million for the same period. As funds under the 10th EDF could no longer be committed beyond 31 December 2013, a Bridging Facility 21 with limited resources (EUR 1.630 billion) was put in place to cover the period from 1 January 2014 until the entry into force of the 11th EDF. Funds committed under the Bridging Facility were accounted for under the 11th EDF. Because of these inception conditions, implementation of the 11th EDF reached cruising speed only towards the end of 2016.

The EDF resources are mainly channelled via non-reimbursable assistance, broken down into different components as shown hereafter in Table 1.

The Commission determines 22 the multiannual indicative resource allocations for each ACP country and region and for intra-ACP cooperation on the basis of criteria laid down in the Cotonou Agreement 23 . In line with programming instructions 24 , the 11th EDF funds are programmed per country and per region, jointly with the partner countries, regions and territories. Non-allocated resources constitute reserves that help ensure a level of financial flexibility and can be used for countries' or regions' unforeseen needs. The initial amount of the non-allocated reserve in 2014 was EUR 5 800 million, meaning 19% of the EDF envelop was initially unallocated compared to 6% for the Development Cooperation Instrument 25 .

Table 1: 11th EDF components 26

Baseline

ACP States

As this is a mid-term evaluation, the baseline has been set at January 2014, which corresponds to the 11th EDF (2014-2020) inception. Therefore, the evaluation compares, to the extent possible, the situation in January 2014 with the situation as of June 2017. It should be noted that the objective of the instrument, poverty reduction, remains the same as in the previous EDFs.

Pursuant to the implementing Regulation of the 11th EDF, the indicators used to measure the achievement of the 11th EDF objectives 32 are the indicators of the Millennium Development Goals (MDGs) adopted following the September 2000 UN Summit 33 and, subsequently, of the Sustainable Development Goals (SDGs) 34 . There are no specific indicators for the EDF instrument to measure its overall performance. This is due to the fact that (i) the fight against poverty in developing countries is a global effort and (ii) the success of development cooperation largely rests on the willingness and capacity of partner countries to implement relevant policies and institutional frameworks conducive to development.

At the time of design and adoption of the 11th EDF, poverty remained a major problem in the ACP countries. The 2015 deadline for the MDGs' achievement was looming with insufficient global progress being achieved. Table 2 on the following page with selected MDGs/SDGs indicators shows the difficult situation faced by the ACP States 35 . Moreover, for certain countries that had been able to move out of poverty, progress was often temporary. Security issues, economic shocks, environmental degradation and climate change, food insecurity are among factors that directly threaten the sustainability of hard-won gains over poverty.

Overseas countries and territories

The financial support provided by the EU to OCTs is not based solely on their socio-economic situation. OCTs are all islands, some of them isolated and, in a majority of cases, with small populations. Most of the OCTs are dependent on fossil fuels and on a limited range of economic resources, making their economy vulnerable to external shocks. The OCTs are host to wide terrestrial and marine biodiversity: climate change constitutes a threat undermining their sustainable development. Some OCTs are particularly vulnerable to natural disasters.

The 2013 Overseas Association Decision 36 reflects a paradigm shift, from a focus on poverty reduction to a reciprocal relation focused on cooperation on mutual interests, and places special emphasis on priorities which are relevant for the OCTs, such as the enhancement of their competitiveness, the strengthening of their resilience, reduction of their vulnerability, and promotion of the cooperation between OCTs and their regional, European and international partners.

Table 2 - Selected Millennium Development Goal and Sustainable Development Goal indicators 37

3. Method

This evaluation is informed by an external evaluation carried out by a team of consultants from August 2016 to May 2017 43 . There was no divergence compared to the roadmap for this evaluation 44 , published in November 2015. Various stakeholders 45 have provided input into the 12-week open public consultation on the draft external evaluation report that is also taken into account in this evaluation. A total of 320 stakeholders were met through a series of meetings organised during the period of the open public consultation. More information can be found in Annex 2.

As mentioned in the "Scope of the Evaluation" this Staff Working Document is set at instrument-level. In terms of the method used, a focus was put on reviewing strategic documents such as country/thematic evaluations and programming documents, rather than project-level documents.

Unless mention is made to the contrary, this Staff Working Document relies on and concurs with the findings and conclusions of the external evaluation. Examples of where the text disagrees or brings nuances to the external evaluation can be found in Section 6: Leverage.

Limitations – Robustness of process and findings

The process of this evaluation is robust and the evidence reasonably solid. However, specific limitations faced by the external evaluation need mentioning in particular (see Annex 2 for further details):

-The evaluation was pitched at instrument-level meaning project-level data was not the focus, although such data could have further informed the overall findings. The instrument-level focus also means that the development policies which are set outside the instrument, but upon which the instrument is based, were not analysed.

-There was no comprehensive monitoring and evaluation system defined in the EDF legal base.

-The 11th EDF's implementation was in its early stages when the collection of evidence was made. To mitigate this, evidence has also been collected when necessary from the 10th EDF.

-There was a very limited number of recent data regarding the EDF performance in OCTs.

-The very broad nature of the partner countries where interventions take place (over 100 countries and territories) made aggregation and comparisons between data difficult.

When evaluating the instrument, it must be taken into consideration the fact that its support can only be seen as a contributing factor towards any results achieved:

-The fight to eradicate poverty in ACP countries is a highly ambitious agenda. Many factors, both internal and external, affect the development of those countries. An important number of official development cooperation providers play an active role, together with an increasing number of private donors including foundations.

-In addition, it is for the partner countries to adopt and implement the necessary reforms and policies that are the driving force behind these achievements.

Moreover, there are number of characteristics that pose constraints when analysing the EDF. For example: the broad nature of the objectives that are pursued.

4. Implementation state of play

This section looks at the progress made in implementing the 11th EDF since 2014 and the monitoring systems used to measure progress.

11th EDF implementation

As of June 2017, with 50% of its budget committed 46 , 31% contracted and 13% paid, and considering the fact that the 11th EDF reached its cruising speed only towards the end of 2016, implementation of the 11th EDF is deemed efficient.

Cooperation with ACP States

Most of the 11th EDF National Indicative Programmes 47  were signed in 2014 and the first part of 2015 48 and all Regional Indicative Programmes between May and June 2015. The duration of the programming process was long which, in a fast-changing environment, and combined with other factors, could have jeopardised the relevance of EDF-funded actions 49 .

Table 3 50  Level of implementation of the 11th EDF – ACP countries, illustrates the level of 11th EDF implementation with ACP partners as of June 2017.

The level of commitments after 3 and a half years of implementation out of 7 is at almost 50% of allocated funds for regional and intra-ACP cooperation and already at almost 60% for cooperation at national level, thus pointing to efficient cooperation with partner countries and regions in identifying actions for support. The relatively low amount paid is not a concern as it is mainly due to most programmes and projects becoming operational mostly from 2016 onwards and spanning over a few years.

The non-allocated resources (reserve funds) initially amounted to EUR 5,800 million. Top-ups to National Indicative Programmes occurred for Somalia and Chad 51 , thus responding to all such requests made by partner countries. Also, the reserve funds have been extensively used up until June 2017 to cater for unforeseen needs at country and regional level. This includes the mobilisation of EUR 1.5 million for the European Emergency Trust Fund for stability and addressing root causes of irregular migration and displaced people in Africa 52 . This capacity to respond to emerging needs shows the financial flexibility of the instrument, especially when compared to other external financing instruments: 20% of the 11th EDF geographic programme is unallocated compared with only 6% for the Development Cooperation Instrument (the geographic component). The consumption of 48.1% of the 11th EDF reserves midway through it’s implementation also points to an efficient use of resources.

Territorial and regional cooperation with OCTs

Despite some initial delays 53 , after two years and a half of the instrument's existence, two-thirds of the EUR 229.5 million territorial envelope for OCTs is either signed or in the final stage before signature, which demonstrates the relative efficiency of the processes in place under the 11th EDF. The programming of the OCT regional programmes has required a longer timeframe because of the requirement for a more intensive co-ordination exercise between OCTs to agree on priorities and the difficulty of conciliating diverse local realities such as the nature of economic drivers, the level of development, and the population size.

A reserve of EUR 21.5 million has been set aside. It has not yet been used but, similarly as for ACP countries, its existence participates in the flexibility of the instrument.

Costs and benefits

Costs to manage the 11th EDF are linked to human resources, infrastructure including EU Delegations and offices in ACP States and OCTs, evaluations and results oriented monitoring, development and management of IT systems, studies, technical cooperation and accompanying measures, audit and financial obligations of the Commission linked to the implementation of the EDF, and communication activities.

The support expenditure amounts to 3.45% of the total 11th EDF budget 54 , which represents a low level of functioning costs, slightly below the support expenditure for the Development Cooperation instrument amounting to 3.81% of the DCI budget. As of June 2017, 38.9% of the available amount for support expenditure has been committed which shows a conservative use of these funds. The EU has a Delegation in most ACP partner countries and one office in an OCT (New Caledonia). These costs are considered by Commission services to be proportionate to meet the principle of partnership, which requires a presence in the partner countries and regions, and to deliver efficiently while ensuring the adequate protection of the EU's financial interest.

The benefits of the cooperation with ACP States and OCTs are considerable. Sustainable prosperity and peace in the ACP States and OCTs that are home to a total of 1.5 billion people 55 benefit Europe by contributing to addressing mutual challenges such as climate change and demographic unbalances, by offering new economic, trade and investment opportunities for both sides and by reducing mutual security threats.

Procurement contracts statistics demonstrate the economic benefits of the 11th EDF for EU-based companies. The 2016 figures report that 52.8% of contracts and grants from the 11th EDF, irrespective of untied aid, were attributed to EU-based companies and organisations.

Monitoring and Evaluation

In the Regulation on the implementation of the 11th EDF, no comprehensive monitoring or evaluation system is mentioned for the purpose of measuring the instrument's overall performance (e.g. its flexibility and complementarity with other instruments). Neither was an intervention logic established at the time the instrument was adopted, which would have shown the external and internal assumptions on which it was based, thus making it easier to measure changes.

The only references to indicators in the Regulation are for impact-level indicators (namely those from the Millennium Development Goals and subsequently the Sustainable Development Goals). A number of these indicators are included in the EU International Cooperation and Development Results Framework which was created in 2015 56 . However, these indicators can only show the EU's contribution to development results as many factors such as partner government policies and interventions of other donors will influence what has been achieved.

The EU Results Framework mentioned above defines quantitative indicators for the collection, aggregation and presentation of three types of result data: (level 1) wider development progress made by partner countries (see impact-level indicators mentioned above); (level 2) partner country results to which the EU contributed through EU-funded projects; (level 3) the European Commission's own organisational performance in respect to international cooperation and development.

In terms of level 2 (outcome and output indicators), it is possible to aggregate these indicators at instrument level. However, they have their limitations for the purpose of this evaluation. For example, they only report results from projects that have closed from mid-2013 to mid-2016 and they only measure quantitative aspects such as the number of teachers trained and the number of children immunised. This means in reality that the results are coming from projects implemented under the previous EDF and do not, for the time being, show the overall performance of the current instrument.

Situation in terms of poverty in ACP States

The ACP States are still facing very serious poverty situations. There are disparities in their level of development. For example, while Mauritius ranks 64 out of 188 countries in the human development index 57 as reported in 2016, the Central African Republic is at the very bottom ranking 188. The MDGs were far from being achieved by the end of 2015 despite the fact that the number of people in extreme poverty declined by an estimated 130 million 58 .

Table 2 above shows progress albeit limited in the fight against poverty. This progress cannot be directly attributed to the EDF although the external evaluation report quotes some multilateral aid reviews that found the EDF to have been critical for progress in MDGs and poverty reduction 59 , pointing at the instrument’s relevance and effectiveness.

5. Responses to the evaluation questions

Relevance

To what extent did the overall objectives and principles as foreseen in Articles 1 and 2 of the Cotonou Agreement as well as in Articles 1 and 2 of the 11th EDF Implementation Regulation and the design of the EDF respond to EU priorities and beneficiary needs in 2014? To what extent can they accommodate changed parameters since then?

This Staff Working Document agrees with the external evaluation report and various stakeholders’ opinion 60 that the 11th EDF is relevant to the EU's objective of poverty reduction.

The programming 61 of the aid appropriately reflected the EU development policy in existence at its inception, as developed in the Agenda for Change 62 and the 2005 European Consensus on Development 63 . The new concepts 64 of the Agenda for Change were overall well promoted in the programming instructions. To enhance ownership and facilitate opportunities to encourage Joint Programming 65 the EU intended to synchronise the duration of the National Indicative Programmes with that of the partner country. However, this did not occur in the vast majority of cases because of the choice made by the Commission to simplify procedures by adopting National Indicative Programmes for the whole MFF period: 2014-20.

It is too early for this Staff Working Document to conclude whether the 11th EDF is a sufficiently enabling instrument to fully implement the new European Consensus on Development adopted on 7 June 2017, which focuses on the implementation of the internationally agreed Sustainable Development Goals, although there is a large overlap between the 11th EDF and the policy orientation of the new Consensus. It is also the case with recent international commitments, such as the Agenda 2030 66 , the 2015 Paris Agreement on Climate Change 67 , or with the Global Strategy for the European Union’s on foreign and security policy 68 .

Following lessons learned from the 10th EDF which experienced a complex and lengthy programming process, the 11th EDF programming based itself on the partners’ development plans, both at national and regional level instead of requiring the elaboration of Country or Regional Strategy Papers. Nevertheless, stakeholders (such as EU platforms) expressed the views that consultation during programming should be more inclusive as they have been mostly limited to some organisations that may not be sufficiently representative 69 .

The external evaluation points at a tension between the ownership and partnership principles of the 11th EDF and agendas perceived as deviating from the fundamental objective of poverty alleviation such as security and migration that have been financially supported by the 11th EDF. However, this evaluation assesses that peace building (as funded through the African Peace Building Facility under the intra-ACP component) and addressing root causes of migration (as with the European Emergency Trust Fund for stability and addressing root causes of irregular migration and displaced people in Africa 70 ) are fundamental aspects of sustainable development. The evaluation nevertheless takes note from the external evaluation and stakeholders’ opinion 71 that the governance rules of the Trust Fund and recourse to accelerated procedures may require attention.

The flexibility of the 11th EDF is an important element of relevance as partners' changing needs and EU changing priorities can be addressed. As much as the instrument is fundamentally a strong anchor for medium- to long-term development strategies, it contains several elements of flexibility: (i) the possibility of ad hoc reviews of the programming 72 ; (ii) the reserve funds to respond to unforeseen needs and to mitigate adverse short-term effects of exogenous shocks 73 , the importance of which was pointed out by EU Member States during the open public consultation 74 ; (iii) its multiannual time perspective and the fact that the budgetary principle of annuality 75 does not apply to it, (iv) the ability to use funds for non-Official Development Assistance actions, such as supporting peace and security operations through the African Peace Facility. Overall, the 11th EDF has more flexibility than the other instruments under the EU budget 76 .

Effectiveness

To what extent does the EDF deliver results against the instrument's objectives, and specific EU priorities?

Performance of the 11th EDF is difficult to measure in the absence of indicators attached to the instrument itself. As mentioned above 77 , the results of the fight against poverty in developing countries depends on many factors.

Most partner countries have experienced progress in poverty reduction and human and economic development over the last ten years. For example, the last Millennium Development Goal Report (2015) 78 shows that sub-Saharan Africa has seen a 6% reduction in extreme poverty since 2011. However, it is difficult to measure the direct impact of the 11th EDF on development outcomes such as poverty reduction because those indicators reflect development progress over the long-term and the 11th EDF has only been operational for two years. As a result, the data available is also resulting from actions taken under the previous EDF. Moreover multiple actors contributed to these development outcomes; they cannot be directly attributable to the EDF. This shows that because the Millennium Development Goal and Sustainable Development Goal indicators are the only measures of performance mentioned in the EDF legal base, there will only be a partial picture emerging of what the EDF's performance has been.

The Commission has strongly enhanced results reporting on projects and programmes with the launch of the EU International Cooperation and Development Results Framework 79 in 2015. It produced its first report on selected results in July 2016 80 . This evaluation nevertheless agrees with the external evaluation and views expressed by some public authorities 81 that a focus on results is lacking at the instrument level.

This Staff Working Document agrees with the external evaluation report that the 11th EDF institutional structures and processes are propitious for effectiveness 82 . The situation of staff in Delegations both in numbers and qualifications has improved but remains an area for careful monitoring.

Following on from the 2011 Impact Assessment 83 , it can be concluded that several of the problem drivers mentioned in the assessment have been addressed in the 11th EDF making it a more able to meet its objectives. For example, in response to problems raised of fragmented support: (1) the 11th EDF established a differentiation approach: least developed countries, low-income countries and countries in crisis/fragile situations were given priority in the resource allocation process in order to focus more effectively on where EU assistance could have the greatest impact; and (2) sector concentration was implemented so that the average number of sectors was three per country/region. However, according to the external evaluation report 84 , the selection of sectors was sometimes made through a top-down approach from the Commission, which was difficult to reconcile with the ownership and partnership principles. In response, the Commission services consider that partner countries commitment was ensured through the co-signature of the programming documents which set out the sectors of intervention, after dialogues took place.

In terms of mainstreaming EU priorities 85 , progress has been noted in the areas of climate change and environment 86 . The emergence of some tools, such as the Environment and Climate Change Mainstreaming Facility 87 has helped step-up efforts on mainstreaming into EDF-funded cooperation. Some stakeholders point at insufficient attention to these issues from development partners 88 . On mainstreaming human rights issues, despite the creation in 2014 of a Toolbox on a rights-based approach encompassing all human rights for EU development cooperation 89 , mainstreaming is still considered a "work-in-progress" by EU staff in Headquarters. Plus, evidence 90 suggests that partner governments' lack of interest or resistance to human rights often represents a major obstacle in mainstreaming such issues. Some stakeholders expressed their disappointment at the limited mainstreaming of issues such as rights of persons with disabilities 91 .

This evaluation notes issues linked to the existence and functioning of NAOs and RAOs 92 . It is the role of the NAO/RAO to represent the country/region in the activities related to EU cooperation funded with the 11th EDF and the European Investment Bank, which can include: planning EU assistance, identifying and designing interventions, and assessing and auditing those interventions. While this structure has been created to ensure ownership, some stakeholders point at the weaknesses and limits of the system as, for example, in some cases, they can represent a barrier between line ministries benefiting from EU support and EU delegation staff 93 . The evaluation finds that their functioning is very often not prone to effectiveness, in particular at regional level. NAOs and RAOs also prove to be challenging in terms of efficiency. An explanation often given relates to capacity issues despite regular support in building capacity 94 .

Efficiency

To what extent is the EDF delivering efficiently?

This evaluation assesses that the 11th EDF is overall efficient. Lessons learnt from the previous EDF were followed by action:

·At regional level, efficiency has considerably improved thanks to the possibility for national administrations and a wider range of regional actors to implement regional projects against the monopoly of the Duly Mandated Regional Organisations in the previous EDFs.

·The process of project formulation has substantially improved with a considerably reduced timeline from project submission to approval while maintaining the same internal quality control process 95 . The duration of the formulation cycle is satisfactory.

Overall budget execution of the first two years of the 11th EDF shows high efficiency 96 , especially in comparison to the 10th EDF as pointed by the external evaluation report 97 .

The relatively long timeframe for the entry into force of the 11th EDF could have impacted the overall efficiency, but it did not, mainly thanks to the Bridging Facility in place between January 2014 and March 2015 98 .

An additional new feature has been the alignment of the Regulation on the implementation of the 11th EDF 99 with the provisions of the Common Implementing Regulation applicable to all other external financing instruments 100 . There is insufficient evidence to be conclusive on the derived efficiency gains beyond the tenderers benefitting from harmonised rules of origin across financing instruments 101 .

Inefficiencies have been pointed out by stakeholders regarding procedures. Despite additional efficiency gains mentioned by the external evaluation, such as the responsibility of amending contracts and proceeding to recommitments being entrusted to EU Delegations, the EDF procedures are still perceived as cumbersome by staff in Delegations and public authorities who shared their views during the open public consultation 102 . Some public authorities also echo the external evaluation report finding according to which efficiency gains obtained by increasing the size of grant contracts have come at the expense of civil society organisations participation 103 .

The external evaluation report mentions the ‘one size fits all’ nature of the EDF procedures that generates inefficiencies considering the existing differences between ACP countries and OCTs, and the specificities of the latter 104 . The relative small size and associated limited administrative capacity of some OCTs can be a source of explanation for the perceived barrier represented by the structure and procedures of the EDF in terms of effectiveness and efficiency.

Coherence

To what extent does the EDF facilitate coherence, consistency, complementarity and synergies both internally between its own set of objectives and programmes and vis-à-vis other EFIs?

The 11th EDF intervention logic (Annex 2) includes a number of assumptions on coherence that were tested by the external evaluation. For example, assumptions were made on the coherence within the 11th EDF components, coherence between external financing instruments, and coherence with other policies other than development policy (Policy Coherence for Development 105 ).

This evaluation finds that the internal coherence between the components of 11th EDF is satisfactory, in particular because the programming instructions prevent duplication by providing that sectors funded from the national envelope cannot be funded from the regional envelope. However, this may have negative consequences in terms of synergies that appear limited among the ACP components as highlighted by organisations and associations that participated in the open public consultation 106 . Complementarity of regional cooperation between OCTs and ACP countries has improved with the relative alignment, in the Pacific region, of the sectors of concentration of the 11th EDF regional programmes for OCTs and ACP countries focusing on climate change and biodiversity. The existing provisions favouring regional cooperation among the different categories of regional actors (OCTs, ACP countries and their neighbouring outermost regions and non-ACP developing countries) have not been sufficiently used until now.

In terms of coherence between the EDF and other instruments, there are three other EU EFIs operating in ACP countries 107 : the Development Cooperation instrument 108 , through its thematic programmes (Global Public Goods and Challenges, and the Civil Society Organisations and Local Authorities) and its Pan-African Programme, the European Instrument for Democracy and Human Rights (EIDHR) 109 , and the Instrument contributing to Stability and Peace 110 . Evidence used by the external evaluation shows that these external financing instruments support actions that the 11th EDF is not best placed to support, such as direct cooperation with civil society organisations on human rights issues. However, some duplications exist between the intra-ACP component and the DCI thematic programme 111 , as also highlighted by some stakeholders 112 . Complementarity of the 11th EDF and other EFIs for which OCTs are eligible is limited.

As stated in the recent evaluation on Joint Programming (2017) 113 , there has been progress on Joint Programming with Member States where it has proven to be valuable for the EU and its Member States, although it is still in its early stages and faces challenges such as ensuring ownership of the process and results by partner countries. For example, in Burkina Faso and Kenya, it has improved EU aid coordination and coherence and has led to complementarities and synergies 114 .

Policy coherence for development has gained momentum under the 11th EDF. For example, the analysis of trade policy support to development in regional cooperation shows an increasing attention to the trade/development nexus 115 .

Added value

To what extent do the EDF programmes add value compared to interventions by Member States or other key donors?

This evaluation agrees with the external evaluation report that the main factors for the 11th EDF's added-value are the nature of the partnership anchored in the Cotonou Agreement, the instrument's geographical scope, the different levels at which it operates 116 combined with the volume of funds available, their predictability and the instrument flexibility described under ‘relevance’. This evaluation also agrees with the public authorities who highlighted the EDF's specific added value in fragile contexts where there are fewer development partners 117 .

For the OCTs, the 11th EDF adds value by linking them to Europe and by reinforcing the OCTs as a group and thus their prominence in regional and global fora 118 .

The 11th EDF's added value compared to EU Member States and other Development Partners mainly rests with its capacity to mobilise high volumes of funding over a long period, and in sectors that need a critical mass of funds 119 . This is less the case in the Pacific where EU funding is significantly smaller than funding from regional donors, particularly Australia. Expectedly, both the regional and intra-ACP components, as illustrated by the African Peace Facility, potentially have a strong added value in addressing issues beyond the reach of national cooperation instruments and on issues of regional integration or of a global nature. Supporting the African Union efforts in the area of peace and security was possible only because the EDF is not financed from the Union's budget 120 .

Leverage

To what extent has the EDF leveraged further funds and/or political or policy engagement?

The 11th EDF is set to support domestic revenue mobilisation (DRM) in particular to reflect the commitment made in 2015 to collectively double support for DRM in developing countries by signing the Addis Tax initiative 121 . Tracking expenditure on DRM has been fully in place only since July 2016 with the creation of a separate DRM DAC code that will be applied for the future. There is therefore insufficient evidence on the 11th EDF leverage on DRM.

This evaluation disagrees with the external evaluation report finding that there is too limited evidence to draw definite conclusions as to whether the 11th EDF has contributed to leverage additional funds through blending 122 . Indeed, a dedicated external evaluation on blending as an EU aid delivery mechanism 123 released in March 2017 concluded that blending 124 , through its leverage of loans, was associated with around twenty times more development funding. This mobilisation of additional funding has been confirmed in half of the cases reviewed; 44% of which were EDF-funded.

Similarly, this Staff Working Document differs from the finding of the external evaluation regarding a possible departure from the EDF fundamental principles of ownership and partnership in the context of blending 125 . National and regional authorities remain fully involved through strategic meetings and the regional steering committees established in the context of the different Regional Indicative Programmes and the ACP-EU Private Sector Development platform 126 . The EU Platform for Blending in External Cooperation 127  pays attention to the respect of those principles.

Conditions are in place for the 11th EDF to continue its predecessor's positive track on policy dialogue, especially at bilateral level and as the EU plays an important role in donor coordination. The use of the budget support modality plays a key role in this respect and also in reinforcing the political dialogue foreseen in the Cotonou Agreement, as the respect for fundamental values is a prerequisite where a Good Governance and Development Contract 128 is foreseen 129 .

6. Conclusions

This Staff Working Document shares the opinion of the external evaluation that the 11th EDF is a relevant instrument with appreciated flexibility features at instrument level. It is overall an efficient instrument with proven added-value and satisfactory internal coherence. This Staff Working Document also shares the opinion of the external evaluation on challenges raised in external evaluation that need attention in the process on how to improve the implementation of the 11th EDF for the remaining period until 2020, and on the future external financing instrument(s) for the post-2020 period. For example, in order of magnitude:

1.There is a tension around the principles of ownership and partnership of the 11th EDF and agendas perceived as deviating from the fundamental objective of poverty alleviation such as security and migration. This emerged through the reported top-down approach during programming of funds, during consultation processes that are not sufficiently inclusive, and in the context of new aid modalities. This tension around the principles is part of the result of an evolving set of policies which has seen more focus put on emerging priorities and challenges.

2.There is evidence of overlaps between the 11th EDF and the thematic component of the Development Cooperation Instrument as both programmes are able to support similar sectors. There are also missed opportunities of synergies between the various EDF components.

3.Data on results is limited. The absence of a monitoring and evaluation system at the level of the instrument beyond the Millennium Development Goals/Sustainable Development Goals does not make it possible to measure changes accurately. For example, no indicators were provided on how to measure the instrument's flexibility or the level of consistency between instruments.

4.The NAO/RAO structure which was initially created to ensure the key principles of ownership and partnership has not fully fulfilled that role and their functioning is often seen as hampering effectiveness and efficiency.

5.Inefficiencies exist at the level of implementation procedures that have consequences on cooperation’s effectiveness, in particular with OCTs.

6.The potential for cooperation with Member States, notably through Joint Programming, is still not fully exploited.

Annex 1. Evaluation questions

Relevance

To what extent did the overall objectives and principles as foreseen in Articles 1 and 2 of the Cotonou Agreement as well as in Articles 1 and 2 of the 11th EDF Implementation Regulation and the design of the EDF respond to EU priorities and beneficiary needs in 2014? To what extent can they accommodate changed parameters since then?

Information sought in this area includes:

· A timeline showing congruence/divergence of the instrument against evolving context, including global challenges, and institutional policy changes e.g. to what extent does the EDF respond to the demands of Agenda 2030, including the need to co-operate with emerging countries on implementing the SDGs.

Effectiveness, impact, sustainability

To what extent does the EDF deliver results against the instrument's objectives, and specific EU priorities? 130

Information sought in this area includes:

·To what extent do EDF programmes contribute towards poverty reduction, and more specifically towards:

ofostering sustainable economic, social and environmental development;

oconsolidating and supporting democracy, rule of law and good governance, human rights and relevant principles of international law (,Cotonou agreement, EDF Implementation Regulation )

oimplementing a rights-based approach encompassing all human rights with a focus on the poor and vulnerable groups (e.g. persons with disabilities, children, indigenous groups) (Cotonou agreement, EDF Implementation Regulation)

·To what extent has the EDF contributed to the European Union's priorities for smart, sustainable and inclusive growth?

·To what extent does the EDF mainstream EU policy priorities (e.g. gender, climate change) and other issues highlighted for mainstreaming in the instrument, and, where relevant, deliver on the commitments including the financial allocations (Cotonou Agreement, part 3, chapter 2, section 4)

·To what extent does the EDF promote principles of aid effectiveness, such as ownership and joint programming (Cotonou agreement, part1, title 1,chapter 1, art 2, Implementation Regulation)

·To what extent are the processes conducive to programming, identification/formulation of effective actions at national, regional and continental levels (Cotonou Agreement Annex IV)? To what extent have the specific OCT guidelines facilitated the programming process?

·To what extent the process of differentiation has been effectively managed e.g. have countries most in need been given priority in the resource allocation process, have differentiated partnerships with new forms of strategic cooperation been developed as set out in Agenda for Change, have any negative effects been minimised (Cotonou Agreement, Part 1, Title 1, Chapter 1, Article 2) ?

·To what extent is the EDF flexible enough to respond to changing needs? (e.g. changed policy priorities,changed contexts) (Cotonou Agreement, Annex IV, Chapter 1, Articles 5, 11 and 14)

·To what extent does the EDF contribute to the aims and objectives of the Overseas Association Decision? To what extent does the split of fiancial resources between regional and territorial programmes contribute to the aims and objectives of the Overseas Association Decsion?

·To what extend are the OCT programmes accompanyed by national/terrotorial policies that support sustainable development?

Efficiency

To what extent is the EDF delivering efficiently? 131

Information sought in this area includes:

·What is the ratio of support expenditure to EDF overall budget?

·How efficient is budget execution in terms of time taken from commitments to payments?

·Have the changes made to EDF 2014 – 2020 from the previous EDF 2007 – 2013 brought efficiency gains? E.g. Has the new regional programmes management brought positive change in terms of efficency of delivery?

·Are there areas, such as administrative/management procedures, where the EDF can be simplified to eliminate unnecessary burden?

·To what extent is the EDF in line with its implementation Regulation? Specifically in terms of :

- Objectives and general principles,

- Programming and allocation of funds,

- Implementation,

- Decision-making procedures,

- Final provisions

· To what extent are the following in place and functioning:

oappropriate monitoring processes and indicators for measurement of the performance of the EDF instrument

orelevant strategic and operational indicators to measure results achieved by the EDF?

Added value

To what extent do the EDF programmes add value compared to interventions by Member States or other key donors?

Information sought in this area includes:

·Where the EDF is operating in the same field as other donors, does it offer added value in terms of size of engagement, particular expertise, and/or particular weight in advocacy?

Coherence, consistency, complementarity and synergies

To what extent does the EDF facilitate coherence, consistency, complementarity and synergies both internally between its own set of objectives and programmes and vis-à-vis other EFIs?

Information sought in this area includes:

·To what extent are the different EDF programmes coherent/overlapping with one another?

·To what extent are the different EDF programmes aligned with EU development policy?

·To what extent are the programmes consistent with EU external action policies?

·To what extent do the programmes complement/overlap/stimulate synergies with other external action financing instruments?

·To what extent does the EDF complement/overlap with other EU instruments outside of development policy?

·To what extent does the EDF complement/overlap with interventions of other donors?

Leverage

To what extent has the EDF leveraged further funds and/or political or policy engagement?

Annex 2. 11th EDF reconstructed intervention logic

Annex 3- Procedural information

This evaluation was initiated in 2015. DEVCO Management took the decision to align the 11th EDF Performance Review on the Mid-Term Review of the MFF 2014-2020 EFIs so as to ensure coherence and consistency.

The evaluation roadmap was published in November 2015. An Inter-Service Group (ISG) was constituted to specifically steer the work of the 11th EDF evaluation. It is composed of representatives from Commission services and the EEAS.

A team of six external evaluators was contracted by the Commission in August 2016. The induction period commenced immediately with a series of interviews with Commission and EEAS staff. The work of the external evaluators was closely monitored by DG DEVCO’s Unit in charge of ACP coordination, and reviewed by the ad hoc ISG. The latter met on seven occasions with a view in particular to reviewing the inception report, desk report, the emerging messages further to the country visits, the drat external evaluation report, the post-open public consultation revised draft evaluation report and assess the quality of the external evaluators’ work.

There were also four meetings of all the external evaluators with the services in charge of monitoring their work and relevant EU staff. The objective of these meetings was to promote understanding and exchange on complementarity and synergy between the EFIs under evaluation.

The evaluation was carried out fully in line with the indications provided in the roadmap. The evidence base in particular consisted of primary sources, secondary sources, and a consultation. A total of eight countries were visited by the evaluation team. Those countries were selected by the ISG to ensure coverage of the three main geographical areas where the EDF operates including a range of countries in Africa, as the continent is the biggest EDF recipient. Attention was paid to include countries which are facing challenges that are high on the European agenda (migration, terrorism) and countries hosting regional organisations. Two OCTs were also included considering the lack of recent evaluation material on OCTs and the need to better understand their needs and challenges.

List of countries visited by the external evaluation team:

The consultation took the shape of an on-line 12-week consultation of stakeholders between 8 February and 3 May 2017 that yielded 125 responses on the 11th EDF. In addition, DG DEVCO organised a series of face-to-face meetings with key stakeholders at which the external evaluators presented their draft report and received direct feedback. The meetings were as follows:

·Technical Workshop with representatives of the European Parliament (Secretariat and assistants to MEPs), Council of Ministers (Secretariat) and EDF Committee (Member States representatives);

·Civil society and Local Authorities representatives gathered in the Policy Forum on Development 134 ;

·ACP States’ Ambassadors and Ministers;

·OCTs representatives.

The Council of Ministers’ ACP Working Party also added the 11th EDF Performance Review on the agenda of its 21.03.2017 meeting and had a robust discussion with the external evaluators.

The ISG reviewed the quality of the external evaluators' work. The external evaluation report is found to be overall of a good quality especially (despite diverging views from the Directorate General for international cooperation and development on blending), considering two main limitations faced by the external evaluators:

·the timeframe to complete their work was relatively short considering the complexity of the EDF instrument i.e. six months between inception (August 2016) and the first draft external evaluation report (January 2017). This short timeframe had an impact on the depth in which some aspects of the instrument could have been examined. The external evaluators proved to be very flexible in accommodating additional factual input from Commission services until the very last stage of their report preparation.

·secondary sources of information concerning the Overseas Countries and Territories were very limited. The mitigation measure was to provide as much information and data as possible from the relevant DEVCO service

The reliability of the data used by the external evaluators is found to be high. The quality of the data can easily be verified through the sources that are systematically mentioned. In case of information emanating exclusively from EU sources, the evaluators attempted as much as possible to triangulate with external sources of information to alleviate any possible bias. In addition, information was collected through interviews with key internal and external interlocutors. A number of evaluations were reviewed. As these were commissioned by the EU, they are quality assured. The quantity of information analyses also appears satisfactory. The overall evidence used by the external evaluation was quality assured by the ISG.

Annex 3a Response to the opinion of the Regulatory Scrutiny Board

Opinion of the Regulatory Scrutiny Board

Overall opinion: POSITIVE

Annex 4 -Synopsis report of the stakeholders' consultation

The stakeholder consultation for the performance review of the 11th EDF began in late 2015 and came to an end in May 2017. The majority of the consultation activities took place during the Open Public Consultation at the beginning of 2017. As highlighted in the evaluation Roadmap 135 , the consultation approach involved collecting input from a wide range of stakeholders on the 11th EDF at its mid-point.

1.Evaluation Roadmaps

The consultation process began with the publication of the evaluation Roadmap, which was published on the European Commission website November 2015. As per the Better Regulation guidelines 136 , the aim of the Roadmap was to give stakeholders and the general public an early opportunity to provide feedback on the evaluation and its approach. However, no feedback was received.

2.Interviews

Whilst the evaluation approach used by the external evaluation maximised the use of available secondary information, importance was given to collecting primary data through interviews and country visits 137 . The external evaluation report mentions that "The interviews were a critical source of evidence for the evaluation" 138 .

The external evaluators proceeded to a stakeholders' analysis and consultation strategy 139 under the guidance of the Inter-Service Group that steered their work. Sampling techniques were applied to ensure a selective approach amongst each stakeholder group, given the many thousands of stakeholders directly involved with the European Development Fund.

The objectives of the interviews were to (i) address gaps in the documentation reviewed, (ii) better understand realities on the ground, especially during the eight field visits, and (iii) triangulate findings especially when the evidence collected was based on internal EU documentation and sources.

Interviews took place at three levels 140 :

·With global informants to complement areas that were inadequately covered from desk review;

·With regional and country level informants by phone to complement areas not covered from desk review;

·During the eight field visits including focus group discussions with stakeholders groups.

The external evaluation team conducted over 170 semi-structured interviews with more than 300 informants, including over 120 interviews at country level 141 . These interviews were undertaken progressively, throughout the evaluation phases leading to the draft report. All interview notes were included in an interview compendium. These notes and compendium have been treated as confidential by the external evaluators. The European Commission services have not had access to them. However, it is possible to assert that the external evaluation has included numerous elements from those interviews in the report as sources quote Meeting Notes with the abbreviation MN 142 .

3.Open Public Consultation – 7 February to 3 May 2017

The Open Public Consultation on the draft evaluation report took place during 12 weeks and closed on the 3rd of May. It was carried out at the same time as the consultation activities for the set of ten External Financing Instruments. The aim was to gather feedback from the broadest possible range of stakeholders, including those in beneficiary countries and in the EU Member States, on the emerging conclusions from the evaluations.

This Annex focuses on the retrospective aspects that were covered under the Open Public Consultation. However, there was also a forward-looking element to the consultation which aimed to gather preliminary ideas on the future External Financing Instruments after the current ones have expired by 31 December 2020.

The Consultation took the shape of (i) an online consultation which included some guiding questions to facilitate providing feedback and (ii) face to face meetings organised with key stakeholders. In that respect, a technical workshop with representatives of Council working groups and of the EDF committee, together with representatives of the European Parliament, took place end of March 2017.

a.OPC online contributions

In total, 125 respondents reacted to the OPC through the web-based platform concerning the performance reviews of the European Union’s (EU) External Financial Instruments; of these, 62 addressed all or some of the four EDF guiding questions. The main category of respondents were organisations and associations, both at global level (all External Financing Instruments) and for the EDF: 28 associations responded to the four guiding questions specifically targeting the EDF. The EDF also received a high level of interest from public authorities (17 responses). Very little input came from the private sector (2) or citizens (2); a few more from EU platforms or networks (8). Responses came from within 15 EU Member States, only 4 ACP countries and 8 other countries.

The four guiding questions asked in line with the findings and conclusions of the external evaluators were as follows:

o1) How well do you think the 11th EDF has addressed its objectives?

o2) Has the 11th EDF, for which partner country ownership is a specific feature, reflected the views of beneficiary countries and the full range of their constituencies?

o3) Do you think the regional and intra-ACP cooperation is efficient, effective and coherent with country level actions?

oThe fourth question was open for any further comment.

b.Summary of OPC contributions

Annex 4a below provides a detailed analysis of the input received through the on-line consultation, and the manner in which this input was taken into account for the evaluation. The summary below presents the main thrust of the responses received.

The question that received most interest was question one on whether the 11th EDF has met its objectives (56 responses), followed by the question on consultation process (45), then the open question (37). The least interest was paid to the question on regional and intra-ACP cooperation (29).

For question one, the vast majority of the contributions came from organisations based in Europe.

The contributions of the public authorities - 18 in total, including one Overseas Country/Territory (OCT) - were largely positive and reflected agreement with the main findings of the evaluation against the different evaluation criteria.

There was overall agreement that the EDF continues to be highly relevant for poverty reduction. There was also recognition of the tension highlighted in the report around the pressure of responding to other agendas, although various respondents (Member States) were of the view that EDF reserves have been critical in addressing emerging priorities of the EU (migration and humanitarian crises).

The contributions of organisations and associations constituted the majority of the responses to this question (23). Overall the responses showed agreement with the findings of the evaluation. Various comments also showed agreement that there are still insufficient synergies between different levels of the EDF, combined with a lack of transparency and coherence in regional programming.

Other comments from public authorities or organisations were examined by the external evaluation, but not in detail by this Staff Working Document because they were not directed at the instrument as such but at its implementation (where there are other legal bases, e.g. the Financial Regulation 143 apply). For example: the limits of a one-size fits all approach with implementation procedures, in particular for the OCTs; how trust funds and blending contribute to fragmentation and lack of transparency; and how efficiency gains have been at the expense of CSOs).

There were also a few observations which while interesting are not reflected in the external evaluation, such as:

•    Comments on the lack of an EU instrument for Middle Income Countries - which is beyond the scope of this instrument evaluation.

•    Observations on the lack of opportunities for European contractors - these were not taken into account as they do not pertain to the realm of an instrument-level assessment of this kind.

On question two: forty-five responses were received to the question about ownership, most of them stemming from organisations and associations or EU platforms. Unfortunately, not a single respondent was a government representative of the ACP countries, who are directly concerned by the finding that ownership has suffered under the 11th EDF. However, the subject was discussed and commented upon at the meeting with the ACP ministers in May 2017.

Results of these discussions and online input from Europe supported the findings of the evaluation that: (i) despite strong existing features (such as alignment with the partner countries' development priorities, and the existence of the National Authorising Officer role), there was a certain loss of ownership from the 10th to the 11th EDF and (ii) there was poor progress in widening public participation in the different phases of the cooperation cycle – especially for Civil Society Organisations and cooperatives.

A total of 29 responses addressed question three, related to efficiency, effectiveness and coherence of regional and intra-ACP cooperation. Overall the comments support the evaluation findings, without disagreements. The feedback included: limited synergies between EDF component programmes, as well as limited dialogue and consultation. Several comments pointed to the limited engagement of the regional cooperation with civil society, in contrast with the legal dispositions of the Cotonou Partnership Agreement. There were also some positive comments addressing regional and intra-ACP cooperation. These recognised the importance of the African Peace Facility.

Under the fourth, open question, contributions touched on a wide variety of different issues including some input on the tension between EDF's principles and EU Trust Funds and blending.

A total of 39 responses were received to this question. Most responses were provided by organisations (16), followed by public authorities (13) and EU platforms (6). In terms of origin, a total of ten responses were from organisations or entities based in Belgium. The remaining responses were equally distributed among different countries. Three contributions were made by ACP countries and OCTs covered by the EDF. Since this question was very open, the contributions are difficult to summarise, but in general, the responses tend to confirm the findings of the evaluation.

4.Face-to-face meetings

During the Open Public Consultation phase, a series of face-to-face meetings were organised as follows 144 :

·Technical workshop with representatives of the European Parliament (Secretariats of the Committees and assistants to MEPs), Council working groups and EDF Committee;

·Civil society and Local Authorities representatives gathered in the Policy Forum on Development 145 ;

·ACP States’ Ambassadors and Ministers;

·OCTs representatives.

A total of 320 stakeholders participated in these face-to-face meetings.

All the above-mentioned meetings provided an opportunity for the external consultants to present their preliminary results and engage in a discussion with key stakeholders as well as creating an opportunity for the stakeholders to give input and ask clarifying questions before submitting their final feedback.

More questions than feedback on the findings of the external evaluation were provided at the technical workshop. A few questions touched upon the evaluation methodology such as on the choice of countries visited; another question was aimed at clarifying the sources of information. The majority of the comments concerned the future direction of the instrument and will not be dealt with in this Staff Working Document.

The gathering of the Policy Forum for Development 146 with Civil Society Organisations and Local Authorities did not directly result in changes into the external evaluation report that already contained a series of considerations related to these actors, for example on their involvement in consultations. Participants raised a number of comments regarding access to funds, including that more opportunities should be made available for CSOs/LAs to apply for projects across the set of external financing instruments. Technical questions on the evaluation methodology were raised such as the extent to which CSOs/LAs were consulted during the field trips that took place. Questions were raised on mainstreaming commitments such as climate change and whether the targets were being met. Others also wanted to know more about the role of private sector in the instruments.

Representatives of ACP States welcomed the opportunity of exchanging with the external evaluators. As with the other face-to-face meetings, clarifications were sought. The report was well received. The ACP States representatives stressed the importance of the EU-ACP partnership as a fundamental element of the EDF, and explained what they viewed as a lack of equality of partnership in terms of programming and implementation. The flexibility of the EDF as an instrument was acknowledged with the mentioning that the rolling programming that enables the necessary adaptation to changing needs. On the role of the NAO, it was mentioned that the situation varies from country to country and that the relationship with EU Delegations should also be considered in the equation. The external evaluation team used the input from ACP States to nuance some of their findings, as evidenced for example in paragraph 33 on page 11 147 .

The OCTs and EU Member States that participated in the meeting (including through video-conference) briefly mentioned some of their specificities. They asked clarifications from the external evaluators, mainly as to whether some findings applied to OCTs as well or solely to ACP States, for example on complementarity among donors. Experience was shared about EDF response to emergency situations. The information thus gathered did not lead to modifications to the draft external evaluation report.

The Council of Ministers’ ACP Working Party also discussed the 11th EDF Performance Review (28 March 2017) with the external evaluators. The draft external evaluation report was globally well received as balanced and clear. Questions were raised on the evaluation process, and on the content of the findings. Discussions provided their views on the potential risk for the EDF principles of partnership and ownership identified in the draft report regarding the European Emergency Trust Fund 148 governance rules and recourse to accelerated procedures. Comments were made on the balance between the instrument's predictability and flexibility. Cross-cutting issues were felt to deserve more attention in the report. Concern was expressed about the finding on the difficulty to assess the results at instrument level. The external evaluators increased the information on cross-cutting issues in their report, and further verified information regarding the above-mentioned Trust Fund, partly as a follow up to this meeting and partly as a result of further interviews with European Commission staff.

Annex 4a. External Evaluation Summary of Open Public Consultation Contributions

This report has been prepared by

Report authors: Muriel Visser (Team Leader), Karolyn Thunnissen, Javier Pereira, Paolo Scalia, Ana García Femenía, Fran Girling

Consortium composed by DAI, Mokoro, Geotest, ICF International, HCL, Technopolis, Europe Limited, ETI Consulting

FWC COM 2015

EuropeAid/137211/DH/SER/Multi

Specific Contract No°2015/368452

This evaluation was commissioned by the Evaluation Unit of the Directorate-General for International Cooperation and Development (European Commission)

Table of Contents

List of acronyms and abbreviations    

1    Introduction                                    

2    Overview                                    

3    Question 1: Addressing EDF objectives                    

3.1.1    Summary of contributions                        

3.1.2    Response of the evaluation team                    

4    Question 2: Extent to which the 11th EDF has taken account of the views of stakeholders                                        

4.1.1    Summary of contributions                        

4.1.2    Response of the evaluation team                    

5    Question 3: Efficiency, effectiveness and coherence between country, regional and Intra-ACP cooperation                                

5.1.1    Summary of contributions                        

5.1.2    Response of the evaluation team                    

6    Question 4: Other views on the EDF                        

6.1.1    Summary of contributions                        

6.1.2    Response of the evaluation team                    

7    Question 5: Other views common to several or all instruments.        

7.1.1    Summary of contributions                        

7.1.2    Response of the evaluation team                    

Annex 1.    Outcome of OPC responses and extent of disagreement with performance review assessment                                    

Tables

Table 1.    Response rate to the OPC on EU’s EFIs by category of respondent        

Table 2.    Number of responses received by category of respondent and question.    

Table 3.    Number of responses to the question about whether the EDF has addressed its objectives, by type of respondent and country of origin of respondent            

Table 4.    Number of responses to the question about ownership, by type of respondent and country of origin of respondent                                

Table 5.    Number of responses to the question about regional and intra-ACP cooperation, by type of respondent and country of origin of respondent            

Table 6.    Number of responses to the question “other views”, by type of respondent and country of origin of respondent                                

Table 7.    Number of responses to the question about EFIs, by type of respondent and country of origin of respondent                                

Table 8.    Outcome of the OPC responses and extent of disagreement with performance review assessment on addressing EDF objectives                        

Table 9.    Outcome of the OPC responses and extent of disagreement with performance review assessment on the issue of ownership                            

Table 10.    Outcome of the OPC responses and extent of disagreement with findings for regional and intra-ACP cooperation                                

Table 11.    Outcome of the OPC responses and extent of disagreement with performance review assessment on the issue of EFIs                            

Figures

Figure 1.    Number of responses received by category of respondent and question    

List of acronyms and abbreviations

1. Introduction

1.The Open Public Consultation (OPC) on the nine evaluations of the European Union (EU) External Financing Instruments (EFIs) was launched on the 7th of February 2017, and closed on the 3rd of May 2017. It included an open access platform for responses to the evaluation reports. This report presents a summary of the responses received to the report of the Performance Review of the European Development Fund (EDF) over the OPC period.

2. Overview

2.In total, 125 respondents reacted to the OPC through the web-based platform concerning the performance reviews of the European Union’s (EU) External Financial Instruments (EFI); of these, half addressed all or some of the four European Development Fund (EDF) guiding questions. The main category of respondents were organisations and associations, both at global level (for the 9 performance reviews) and for the EDF: 28 associations responded to the four guiding questions specifically targeting the EDF. The EDF also received a high level of interest from public authorities and EU platforms/networks or associations. Table 1 shows the response rate by category of respondent.

Table 1.Response rate to the OPC on EU’s EFIs by category of respondent

Source: ET calculation based on OPC responses as of 8th of May 2017

3.Looking at the respondents to EDF questions by country of origin, responses were received from 27 different countries, 15 EU countries, 3 European non-EU countries, 4 African Caribbean and Pacific (ACP) countries and 5 other countries. Belgium was the most represented country in terms of respondents.

4.It is to be noted that not all respondents responded to all the EDF questions. As illustrated Table 2 and Figure 1 below, there is a strong decrease of response rate moving from the first question (concerning the extent to which EDF11 managed to address its objectives) to the second question (asking whether the EDF11 reflected the views of beneficiary countries and the full range of their constituencies) and then to the third question (looking into the efficiency and effectiveness of intra-ACP and regional cooperation and its coherence with country cooperation). For the last, open, question for EDF, there was an increase in responses. The very last question asked for any other views common to several or all instruments.

Table 2.Number of responses received by category of respondent and question.

Source: ET calculation based on OPC responses as of 10th of May 2017

Figure 1.Number of responses received by category of respondent and question

Source: ET, based on OPC responses as of 10th of May 2017

5.The next section of the report summarizes the contributions for each of the questions. It provides an overview of the main contributions in terms of their origin, the key messages that were conveyed, as well as of any changes that this entailed in the evaluation report.

Question 1: Addressing EDF objectives

Summary of contributions

6.This question covered a very wide range of possible contributions, across all the evaluation criteria covered by the report. Organisations or associations (21 responses) and public authorities (18 responses) formed the bulk of the 56 responses to this question. As noted above, the vast majority of the contributions came from organisations based in Europe.

Table 3.Number of responses to the question about whether the EDF has addressed its objectives, by type of respondent and country of origin of respondent

Source: ET, OPC analysis of 10th of May 2017

Views of public authorities

7.The contributions of the public authorities –- 18 in total, including one Overseas Country/Territory (OCT) –- were largely positive and reflected agreement with the main findings of the evaluation against the different evaluation criteria.

8.There was overall agreement that the EDF continues to be highly relevant for poverty reduction. There was also recognition of the tension highlighted in the report around the pressure of responding to other agendas, although various respondents (member states –- MS) were of the view that EDF reserves have been critical in addressing emerging priorities of the EU (migration and humanitarian crises). Others, however, were concerned about the increasing lack of transparency on decision-making (which is noted in the report). Some tension is therefore noted here. Various respondents suggested that the use of EU funds for these agendas should be evaluated against the EDF’s poverty reduction objectives, with one observation of the need for being vigilant on how the private sector might be benefiting from blending arrangements.

9.Two respondents noted that while EDF is relevant, the EU lacks an adequate basis for development cooperation with Upper Middle Income Countries (MICs), although poverty is a concern in these countries, and that there is insufficient focus on private sector development.

10.Various responses echoed agreement on the territorial scope, concerns about the one-size fits all approach, and the need to review procedures, as well as broad agreement with the concerns raised in the evaluation report on insufficient lesson learning, monitoring (too much focus on procedures, too little on results), challenges to sustainability and absence of exit strategies. Broad agreement was also voiced through various comments that flexibility remains a challenge, although some changes have been made, and a particular concern that these changes have not made a difference at operational level, and in particular for OCTs. Concerns by respondents echoed those reported by the evaluation about the fact that efficiency gains have come at the expense of decline in partner participation, especially Civil Society Organisations (CSOs) (see also Q2).

11.Two respondents felt more prominence could be given to the important achievements at regional level in the Pacific, and also noted the importance of Budget Support (BS) for OCTs, but with the observation that there is too much focus on procedures, and too little focus on anticipating challenges with BS and ensuring OCTs can handle these. There were various comments of agreement that there is insufficient understanding of OCTs, with Headquarters (HQ) and expertise that is put at the disposal of OCTs not sufficiently cognizant of OCT specific concerns and procedures.

12.Most respondents agreed that added value of the EDF lies mainly in the volume of money (as well as geographical scope and long-term horizon, and presence of EU in countries where MS are not present). Some responses also highlighted the added value of the EU as an honest broker and providing a balance in the position of the Member States, but regretted that there is insufficient linking with the discussions in Brussels which would allow for stronger agenda setting and better coherence. A number of respondents singled out the specific added value in fragile contexts where there are fewer Development Partners (DPs), and the EU has a specific role to play, and noted that this was not specifically highlighted in the evaluation report.

13.In terms of complementarities, some respondents supported the evaluation finding of incoherence between the Development Cooperation Instrument (DCI) and EDF which needs to be resolved. There was also broad agreement that leveraging is weak and below what one would expect of an instrument of this size. Some contributors noted that trust funds and blending have contributed to fragmentation and lack of transparency. Respondents noted the value of regular consultation with other MS through the EU, but agreed with the evaluation’s findings that potential for joint strategies between MS and EU is insufficiently explored. Various respondents noted that there is not always good collaboration between delegations and MS, and insufficient willingness to take on a joint agenda. In practice, dialogue is reported to be limited to sector concentration and not enough on other critical (cross-cutting) agendas, e.g. climate change.

Organisation or associations

14.The contributions of organisations and associations constituted the majority of the responses to this question (23). Overall the responses showed agreement with the findings of the evaluation around insufficient planning for exit from sectors where the EU had an added value and corresponding loss of expertise (combined with reduction in staff). Various comments also showed agreement that there are still insufficient synergies between different levels of the EDF, combined with a lack of transparency and coherence in regional programming.

15.As was the case for the public authorities, this category of contributors also noted the lack of transparency and accountability mechanisms to civil society (in line with evaluation findings), and included suggestions that there should be a more active role for CSOs in monitoring impact, as well as much more active engagement of CSOs in regional priority setting. Various comments also showed agreement with the EDF11 findings around priority setting being driven by Brussels, the notion of joint programming having been lost.

16.Various responses expressed concerns about the blending facilities and whether these are fit for purpose, and in particular whether they respond to sustainable development objectives in line with country ownership.

17.Many of the responses from this category of contributors focussed on specific cross-cutting agendas, on specific commitments under Cotonou, and on specific areas of focus within the mandate of the organisations providing the response. Thus various organisations concurred with the evaluation’s observations regarding insufficient attention to environment and climate change, although with the specification that the main shortcoming of the EDF was insufficient attention to these issues at national level and that these issues had been well integrated into the Regional Indicative Programmes (RIP), as well as at the Intra-ACP Cooperation Strategy. Attention was also drawn to the need to include attention to child rights (in line with various United Nations (UN) conventions on children and on disability), to human rights, and to gender equality in EDF programming priorities as well as in data collection and monitoring. Two organisations also noted insufficient information on what the EDF is doing for persons with disabilities, including whether adequate attention is being placed on infrastructure for the rights of people with disability. One organisation stressed ‘meagre performance’ on population issues and that this goes against the Cotonou Agreement which provides for the integration of ‘population issues into development strategies in order to improve reproductive health, primary health care, family planning’. The same organisation also highlighted concerns that there was a progressive withdrawal by EDF from support to basic sectors (education etc.). Various suggestions were made that these issues should be systematically mainstreamed and prioritized, notwithstanding the choice of sectors of concentration per partner country and that European Union Delegations (EUDs) should be more actively encouraged to look at opportunities integrating them in the National Implementation Plans (NIPs).

18.There was some agreement voiced among this category of respondents with the evaluation’s findings on the European Union Trust Fund for Africa (EUTFA). One organisation specifically stressed the need for a much longer-term approach to genuine problems of conflict, fragility and resilience which addressed the root causes of migration including conflict, impoverishment and human rights violations, and that this should be clearly reflected in the EUTFA’s allocations which should serve to address these factors.

19.The important role of the EDF in supporting the African Peace Keeping Facility (APF) noted in the evaluation report was seconded by one organisation, although noting that this support does not extend to national efforts.

EU platform, network or association

20.Comments from this group covered a total of six entities. The responses concurred around a number of points made in the evaluation report related to weaknesses of the consultation processes, flexibility, and simplifications, with selected respondents making suggestions that centred around the importance of creating mechanisms and capacity for consultation with non-state actors, ensuring a say and access to EDF funds by subnational governments and local organisations, building capacity of the EUDs to be able to manage consultation processes, and the importance of simplifying procedures to allow this to happen. A specific set of comments also concurred with the evaluation’s recommendation on conducting a political economy analysis, with respondents highlighting the existence of other analytical documents by the EU (on Human Rights etc.) as possible sources for such an analysis.

21.Various observations were made on cross-cutting issues, reflecting disappointment with the limited degree to which EDF11 planning documents include references to gender and to the achievement of climate targets (as reported in the evaluation), as well as disappointment on the lack of attention in the evaluation report itself to the rights of persons with disabilities.

Industry, business or worker’s association

22.Two comments were received under this category of contributors and they converged around the need to have programmes for MICs as these include most of the world’s poor. There was also agreement expressed with the evaluation team findings on loss of expertise in transportation and the lack of adequate exit strategies, and an acknowledgement of the lack of sufficient progress in this sector due to insufficient government commitment (also mentioned in the evaluation report).

Research/academia

23.Four comments came from this category of respondents. Respondents concurred with the conclusions around the important role of the EDF in poverty alleviation. One organisation highlighted in particular the stronger focus of the EDF11 on least developed countries (LDCs) and low-income countries (LICs), which is also noted in the evaluation report. The same respondent noted that in its view the Cotonou Agreement had been substantially eroded with the move of two its key pillars (trade and political cooperation) to regional structures, and stated that this had affected the instrument’s capacity to apply an integrated approach (a point not noted in the evaluation report).

24.While the evaluation’s findings on sector concentration were noted, there was also a call by one of the organisations for attention on the need to be cognisant of the limitations of the donor concentration, another point not mentioned in the report, and specifically that “Donor’s policy of graduation, combined with sector concentration and deployed in a context of an imperfect division of labour, may result in: larger volumes of aid directed towards sectors with limited absorption capacity, overcrowding or saturating those sectors, sector saturation, aid inefficiency and opportunity costs … (and may) compromise EU desire to improve impact, notably engaging in sectors where there is insufficient traction for reform, reducing its effectiveness and limiting EU leverage to facilitate and support partner-led change”.

25.Various specific recommendations were made across the contributions related to improving the programming process; broadening participation in planning and implementation; strengthening monitoring and evaluation in particular of regional integration; and addressing the weaknesses of blending.

Citizen/individual

26.The two citizen/individual contributions expressed agreement with the EDF conclusions on simplifying processes, strengthening monitoring, and improving sustainability.

Other

27.Only one organisation responded in this category to provide a list of recommendations in line with comments from other categories of respondents on the need to strengthen political dialogue, build capacity for meaningful multi-stakeholder governance, simplification, capacity building of EUDs for effective dialogue, as well as some specific recommendations that the assessment/evaluation of trust funds should be done against the same goals and principles as the EDF, and the need to include target country constituencies (e.g. government) in the decision-making structures of trust funds and blending facilities.

Response of the evaluation team

28.The contributions across different stakeholder groups broadly confirmed the evaluation team’s findings on the extent to which the EDF has addressed its objectives. There was no disagreement on major points, but some contributions prompted further fine-tuning or specific nuancing/clarification of points of detail/exemplification. This includes the following points:

·That the EDF programmes and projects can have a particular role in fragile contexts where there are few MS/DP (the role that the EDF already plays in contexts with few MS/DP had already been mentioned in the report, but not specifically in relation to fragile states).

·That there are concerns that there may be efficiency implications of the proliferation of instruments and tools under the EDF (in addition to lack the lack of transparency of these instruments which is already in the report).

·That the performance review’s approach of working with countries with recent evaluation evidence contributed to an under-representation of fragile, conflict and post-conflict countries in the sample of the evaluation, which will be mentioned in the report as one of the limitations.

·That there is a strong view from various constituencies about the need to strengthen delegations’ capacity in planning and consultation processes, together with a more robust and transparent approach to consultation in general.

·There is a related view by many constituencies on the need for the EDF to respect more consistently commitments to gender, climate change, reproductive health (there are also commitments related to reproductive health that arise from the Cotonou agreement which have not been followed through in implementation choices), etc., and for these to be monitored. Many constituencies made recommendations related to the need for capacity strengthening of delegations to ensure climate change, gender equality, inclusion, and other cross-cutting issues can be fully mainstreamed and not just taken into account for certain sectors, and that results achievement is adequately monitored (with for climate change special attention needed in energy, agriculture and private sector development programmes).

·A number of respondents noted that the lack of political weight of the EDF, and its poor added value in political dialogue, reflects insufficient link-up with political discussions in Brussels, which requires better coordination among different services in Brussels to enhance the capacity of the EU to fully utilize external tools efficiently.

·That Blended finance (and the same was mentioned with respect to Trust Funds) should be assessed against the same principles as development cooperation in general and the EDF overall – poverty reduction objectives, the perspective of people living in poverty, gender equality, sustainability in its three dimensions (economic, social and environmental), and cost-effectiveness.

·That there are views that local and regional government associations should be represented in the governing structures of Trust Funds and blending instruments.

·The need to reflect in the evaluation report the recent findings of the blending evaluation and the coherence with the conclusions of the DCI evaluation, a recommendation which the evaluation has taken on board.

·That there are various points related to OCTs that can be further emphasized in the report, namely: a) the important achievements at regional level in the Pacific (already mentioned); b) that expertise put at disposal by the EU for preparation of proposals often doesn’t understand the specific conditions of the OCTs; and c) that complementarity of the EDF and other EFIs for OCTs (even when eligible) is limited by lack of visibility of these opportunities, lack of expertise (human resources), and parallel but unlinked programming exercises.

·That the specific analysis of the attention in EDF11 programming to the rights of persons with disabilities (currently in an Annex) should be brought forward in the main report. The same will be done for other key cross-cutting issues – also an observation that the evaluation has taken on board by including a box on cross-cutting issues in the revised report.

29.There were a few observations which while interesting are not reflected in changes in the report. These are listed below together with the evaluation’s rationale for not including them:

·The observations made on the lack of an EU instrument for MICs, which we see as beyond the scope of this instrument evaluation. It should perhaps be an issue for the coherence report.

·Observations were made on the lack of opportunities for European contractors – these were not taken into account as they do not in the view of the EDF Evaluation Team (ET) pertain to the realm of an instrument-level assessment of this kind.

30.Table 8 in 0 attempts to capture, in quantitative terms, the outcome of the OPC in terms of globally agreeing or not with the findings and recommendations of the evaluation team, and shows that there was broad support for the main findings and recommendations of the evaluation. The assessment was challenging for this question because of the broad nature of the question which involved many different dimension, resulting in many respondents expressing agreement with various points, but perhaps having just one (usually minor) point of disagreement or nuancing. Thus, the column with confirmed findings reflects overall agreement with the findings, where opposition to a finding (next column) reflects whether a specific finding was contested (while the respondent might have agreed in his/her contribution to other findings). Also, many respondents commented only indirectly on areas of recommendation so the table reflects on explicit agreement with recommendations.

Question 2: Extent to which the 11th EDF has taken account of the views of stakeholders

Summary of contributions

31.Forty-five responses were received to the question about ownership, most of them stemming from organisations and associations or EU platforms, as seen in Table 4 below. Unfortunately, not a single respondent was a government representative of the ACP countries, who are directly concerned by the finding that ownership has suffered under the EDF11. However, the subject was discussed and commented upon at the meeting with the ACP ministers in May 2017. Both the OPC survey and the OPC discussions provided ample evidence to back up the findings discussed and analysed in the performance review in terms of loss of ownership between EDF10 and EDF11 and poor progress in widening public participation in the different phases of the cooperation cycle (programming, implementation, monitoring, evaluation). The OPC responses were mostly geared towards the participation of entities other than central Government (local government, CSOs, cooperatives, local communities, …); they provide additional evidence that enabled the evaluators to refine some of the statements made, as seen below when presenting the responses by type of respondent.

Table 4.Number of responses to the question about ownership, by type of respondent and country of origin of respondent

Source: ET, OPC analysis of 10th of May 2017

Views of public authorities

32.All public authorities that responded to the second question confirmed that the EU processes ensure a good ownership by the Partner Country Government, due to the programming and execution processes and the existence of the National Authorizing Officer (NAO). However, they also saw a diminished ownership with EDF11, and, excepting one MS, they also expressed their concerns regarding insufficient CSO, local authority and private sector involvement and eventual ownership. A few respondents also pointed out that the heavy administrative procedures of the EDF undermine ownership, especially for OCTs.

Organisation or association

33.All of the 17 associations that responded to this question bore witness to the lack of full engagement of CSOs, Local Authorities (LAs), cooperatives and the private sector with the EU at the various stages of the programme cycle. It was recognised that the practice is variable between countries, with some having made good progress in involving and widening public participation. However, on the whole, the existing engagement is felt to be insufficient as consultations are limited to specific associations which are not necessarily representative, either in sectoral or in geographical terms. Engagement and ownership are put even more at risk with the shrinking space for CSOs in some countries (restrictive legal frameworks and/or practices) and the use of new EU funding instruments that do not involve consultations. Several respondents also called attention to the importance of involving local communities in order to fulfil the Sustainable Development Goal (SDG) agenda. Finally, some actors called for increased attention to the inclusion of specific thematic/sector support such as transport, gender and disabilities.

34.Some of the associations extended the discussion of ownership to that of the funding of CSOs, pointing out here again that practices are variable between countries and that whilst some NIPs have included CSO envelopes, others have not and volumes and shares of funding are equally variable. Some examples illustrating this variability were provided, pointing out the discrepancy between CSO support provided and CSO needs in view of their important role in the development of the country.

EU platform, network or association

35.Although it was recognised that the EDF put much more emphasis on stakeholder consultations than other EFIs, the various organisations ranked as EU platforms, networks or associations confirmed the general lack of truly representative multi-stakeholder consultations and participation and above all the one directional nature of the consultation processes (top down approach). As with the responses in the previous category, the fact that consulted CSOs were not particularly representative and that their views were not reflected in decisions taken, was highlighted.

36.Importantly, the eight respondents in this category also drew attention to two issues missed out by the evaluators:

·the importance of involvement of local and regional governments (LRGs): local policy priorities are deemed to be insufficiently taken account of in national planning and in joint-programming with donors. The evaluation failed to pick up on this issue: it mentions national government, CSOs, the private sector, trade unions, religious leaders and representatives of political parties, but not local and regional governments, which are major players in those partnerships. The evaluation also failed to mention cooperatives as major local players.

·The fact that, where consultations are held, they are always about country programmes, not about regional or intra-ACP programmes.

37.Another aspect that was possibly not sufficiently highlighted in the performance review was the fact that the ACP countries are not involved in the decision-making process relating to the use of the reserve funds (limited to the EDF Committee) thus also limiting ownership, equal partnership, transparency and accountability. Finally, some aspects were highlighted such as the need to include the National Organisations of Persons with Disabilities in consultations, the need for improved communication between ministries and specialists that focus in particular on cross-cutting issues such as gender equality, human rights and climate change.

38.Finally, there was also the interesting comment that budget support should be accessible to LRGs.

Industry, business or workers’ association

39.Only one European international trade union responded to the question of ownership. It confirmed the view held by previous respondents: consultations of non-state actors are sparse, very formal and without much influence on political choices already made by the EU and their access to EU funding is constrained by heavy procedures. The trade union calls for the promotion of a wider civil and social dialogue in all aspects of design, implementation, monitoring, trade agreements and political dialogue.

Research/academia

40.Of the four responses received from researchers, two provided confirmation of findings, one just confirmed that beneficiary countries’ views are represented in EDF and the other was not related to EDF. One of the responses cited independently realised studies that confirmed the performance review’s findings: EDF shows good alignment with country programmes but found it difficult to comply with EU policy priorities emanating from HQ; the NAO structure is not a guarantee of strong ownership, and may even sometimes be a stumbling block for reforms or tackling sensitive issues; focalising the cooperation discussion on a centralised NAO can run contrary to a more inclusive sector-based dialogue and cooperation; lastly, the centralised governance of cooperation and shrinking CSO space make it difficult for non-government actors to engage.

41.A suggestion was made to integrate local researchers in the programme cycle and dialogue with the EUDs so that local specificities are taken proper account of.

Citizen/private individuals

42.Only two individuals responded to this question and agreed that civil society is generally weak and not sufficiently supported and that in most cases the EDF has reflected the views of countries and their constituencies.

Other

43.There was one single response in the ‘other’ category. This respondent drew strong attention to the fact that the EDF did not include the LRGs in its consultations (backed up by data from an independent survey amongst 37 countries) and has thus not delivered on its promise of an inclusive policy dialogue. The respondent cited evidence that local priorities are not necessarily taken account of in national programming and donor joint programming which should not be just top-down but also bottom-up.

44.The respondent also confirmed the bottleneck represented by some NAOs (again on the basis of independent studies) and the need to review their role, especially in countries where space for CSOs and/or political opposition is reduced. Wider programme steering committees were called for, and again the respondent called for the evaluation to extend its recommendation of inclusiveness not only to national government, CSOs, the private sector, trade unions, religious leaders and representatives of political parties, but also to local and regional governments. Again, similarly to one of the earlier responses, it was suggested that budget support could be provided directly to local governments to support the decentralisation processes and local development.

Response of the evaluation team

·The contributions confirmed the evaluation team’s findings and conclusions on ownership and partnership. Whilst most respondents recognised that alignment of cooperation with national plans was realised, all respondents highlighted the uneven and partial stakeholder consultations (in geographic and sectoral scope, in type of stakeholders consulted). Some also noted the directive manner in which end decisions were taken and regretted the top-down approach to programming and implementation. Table 9 in 0 attempts to capture, in quantitative terms, the outcome of the OPC in terms of agreement or disagreement with the findings and recommendations of the evaluation team.

·Some respondents also took the opportunity of this question to:

§underline the problems surrounding the NAO structure, which is too centralised and sometimes undermining possibilities of closer and more inclusive dialogue with sector players;

§comment on the uneven funding of CSOs, with great variation between countries and apparent delinking between needs/role of CSOs in development at local level and support provided.

·Important points that came to the fore and which prompted the team to refine their analysis, conclusions and recommendations concerned testimonies as to:

§the absence of consultations with cooperatives and local governments, which are important local players

§the important role of devolution and local communities in reaching the SDGs and hence the need for the EDF to pay even more heed to full involvement of local players in programming and execution of cooperation programmes

§the absence of discussion of the intra-ACP and regional programmes.

·Some recommendations were also made, to:

§Enhance social dialogue with inclusion of local governments, trade unions, cooperatives, etc.

§Extend the provision of budget support to regional and local governments

§Revisit the role of the NAO so as to decentralise the cooperation dialogue.

Question 3: Efficiency, effectiveness and coherence between country, regional and Intra-ACP cooperation

Summary of contributions

45.A total of 29 responses addressed question 3, related to efficiency, effectiveness and coherence of regional and intra-ACP cooperation (see Table 5 below). Overall the comments support the evaluation findings, without disagreements. The majority of comments evidence specific issues and weaknesses of regional and intra-ACP cooperation. Several of the shortcomings addressed by the evaluation are also pointed out by the OPC, including:

·the lack of synergies and coherence across national, regional and intra-ACP levels;

·limited capacities to address supranational goods;

·limited transparency, dialogue and consultation. Several comments point to the limited engagement of the regional cooperation with civil society, in contrast with the legal dispositions of the Cotonou Partnership Agreement (CPA).

There were a few positive comments addressing regional and intra-ACP cooperation. These recognized the importance of the APF although advocating for increased attention to conflict prevention and peace building.

Table 5.Number of responses to the question about regional and intra-ACP cooperation, by type of respondent and country of origin of respondent

Source: ET from OPC, 10th of May 2017

Views of public authorities

46.A total of 12 responses were received from public authorities, including 7 MS, Morocco and New Caledonia. In all cases evaluation findings, conclusions and recommendations were not challenged and the majority of the comments converged with evaluation findings, evidencing critical issues of effectiveness and performance of regional and intra-ACP cooperation.

47.Among the responses there was: mention of the outdated nature and lack of adaptation of the intra-ACP structure to global challenges; recognition of the potential of regional cooperation but regret that this potential is not being been tapped; agreement with evaluation findings on the lack of synergies across national, regional and intra-ACP cooperation; and on the lack of transparency in the intra-ACP decision making. Two contributors comment on the APF, with one advocating for an increased involvement of the African Union and long term provisions for the APF, and another advocating for a more robust and strategic approach. The negative impact on regional cooperation resulting from the Economic Partnership Agreements (EPAs) is also pointed out.

Organisation or association

48.Comments from the 10 respondents converged in supporting evaluation findings particularly in recognizing untapped opportunities while evidencing issues of coherence, complementarity and effectiveness of the regional and intra-ACP mechanisms, and poor transparency of regional cooperation. The inadequate role of the CSOs in regional and intra-ACP cooperation was mentioned across various responses as affecting the relevance and coherence of EU cooperation, while it was considered of particular importance in the context of environmental and climate change challenges.

49.Other responses pointed to the need to strengthen specific features of the regional and intra-ACP instrument, including in terms of weaknesses of programming and organisational set-up and the limited synergies across the national, regional and intra-ACP levels – issues which the evaluation report clearly raises. Particular mention was made of the weaknesses of programming "EDF11 intra-ACP programming was not well coordinated and lacked consultation with EUDs and Regional Programmes", and of the limited consultation with the ACP Secretariat. In terms of specific priorities the need to strengthen support to health (sharp decrease of EDF11 support to the health sector, not compensated by other instruments) and lack of capacity of regional and intra-ACP cooperation to address acute cross-border and global needs was mentioned by respondents. A final comment related to the increased attention to new political priorities and the how this is affecting EDF11 capacity to address goals for poverty reduction: "There should be no more depletion of original development programmes towards growing political concern, under the risk of undermining the EDF objective to reduce poverty". A specific note on AFP was also made by one respondent, pointing to the disproportionate focus of AFP on peace operations, which favours short-term and military management of crises and insufficient attention to conflict prevention and peace building.

EU platform, network or association

50.This group of respondents provided three comments. The responses converged with evaluation findings and addressed specific issues of EDF regional and intra-ACP mechanisms. One comment stressed that while some level of complementarity has been achieved across the EDF envelopes, complementarities are missing with other mechanisms that EDF is resourcing, including the Africa Investment Facility, the soon to be approved African Investment Platform under the European Fund for Sustainable Development (EFSD) or the EU Trust Fund for Africa. It was noted that these mechanisms constrain transparency and do not provide evidence of contribution to additionality and impact toward development goals, pointing to the need to strengthen monitoring and evaluation arrangements. A second comment supported earlier observations from the public organisations on the limited focus of the AFP approach with insufficient attention to conflict prevention and peace building.

Research/academia

51.Two responses were received under this group which broadly echo the observations from other organisations. Additional points concern the lack of ACP involvement beyond funding relationships and no real impact outside Brussels in ACP cooperation "with the notable exception of effective ACP coalitions in the WTO. The multilateral trading system has proved a fertile ground for collective ACP action and is widely considered as a success story." One of the comments highlights the need to improve the theory of change and the measurability of regional and intra-ACP cooperation, and the need to have a temporal timeline beyond the five-year cycle in order to achieve results related to regional integration.

Citizen/private individuals

52.Two short comments were received with limited relevance to the evaluation.

Response of the evaluation team

53.OPC responses converge toward the instrument evaluation and the assessment of regional and intra-ACP cooperation mechanisms. Several comments across the different categories of respondents recognize the opportunities and potential value added of regional and intra-ACP cooperation. However, the comments point out how these mechanisms are not meeting expectations. Overall the responses confirm the concerns of the evaluation about structural weaknesses of regional and intra-ACP cooperation. Several of the evaluation recommendations have been explicitly mentioned by the comments, including the need to strengthen result orientation, to build synergies across national, regional and intra-ACP envelopes, the need to increase transparency and of an increased involvement of civil society. Of particular interest is the discussion by one of the research organisations which presents an in-depth analysis of cooperation mechanisms and of shortcomings of regional and intra-ACP cooperation as well as the changes brought under EDF11. Table 10 in 0 attempts to capture, in quantitative terms, the outcome of the OPC in terms of agreement or disagreement with the findings and recommendations of the evaluation team.

Question 4: Other views on the EDF

Summary of contributions

54.A total of 39 responses were received to this question. Most responses were provided by organisations (16), followed by public authorities (13) and EU platforms (6). In terms of origin, a total of ten responses were from organisations or entities based in Belgium. The remaining responses were equally distributed among different countries. Three contributions were made by ACP countries and OCTs covered by the EDF. A full breakdown of the responses can be found in Table 6 below. Since this question was very open, the contributions touch upon many different issues and are difficult to summarise. In general, the responses tend to confirm the findings of the evaluation. More details are provided below in the discussion of the responses by type of respondent.

Table 6.Number of responses to the question “other views”, by type of respondent and country of origin of respondent

Source: ET, OPC analysis of 7th of May 2017

Views of public authorities

55.The contributions from the 13 public authorities that responded to question 4 show a great diversity in terms of content and approach. Several comments supported some of the findings and recommendations of the evaluation in the following areas:

·Difficulties in accessing EDF funding for certain stakeholders (e.g. CSOs)

·Coherence between the EDF and DCI

·Limited consultation with CSOs, the private sector and other stakeholders

·Tension between EDF’s core values and the Trust Fund and emerging instruments such as blending

56.The contributions also reflected on some aspects that are specific to the role of EU MSs in the EDF. There were references to the need to pay more attention to internal EDF procedures to ease the burden on MSs and allow for a meaningful participation, and to the importance of increasing consultation between EUDs and MSs at country level.

57.Public authorities also commented on some information they would have liked to see reflected in the evaluation. These issues are discussed in the following section:

·Sectoral breakdown of EDF support

·Evidence of impact of EDF11

·Consideration of EDF11 added value in relation to DCI

58.The contributions also include a number of considerations about the future of the EDF that are beyond the scope of the evaluation. These include comments on the need to merge EDF and DCI, and the level of connection between EDF support and EU political objectives in areas such as migration.

Organisation or association

59.There were 15 contributions from organisations and associations. These contributions confirm or support the findings of the evaluation in relation to:

·Complementarity between the EDF and other instruments and specific challenges in some cases (e.g. DCI).

·Contradictions between EDF‘s core values and new and innovative mechanisms (blending and trust funds).

·Limitations of one-size fits-all approach in the EDF.

·Tensions between what the EDF was designed to do and what it is being asked to do, in particular in relation to the EU migration and security agenda.

60.The contributions also raise some issues that could deserve additional attention:

·Weaknesses in the EDF’s approach to people with disabilities.

·Role of CSOs in domestic accountability processes.

61.Finally, the contributions also contain a number of considerations and recommendations about areas the EDF should support that do not relate to the contents of the evaluation report (e.g. biodiversity, increased support to certain actors, etc.). Some of the more general recommendations are aligned with the recommendations in the report (e.g. capacity to adapt to local circumstances vs. one-size-fits-all approach, increased transparency and strong monitoring and evaluation (M&E) systems.

62.One business association also replied to this question. The response highlighted the potential contribution of infrastructure building to inclusive development and defended the use of the “doing business report” as a useful tool to measure good governance reforms. This contribution is beyond the scope of the evaluation.

EU platform, network or association

63.A total of 6 different organisations in this category responded to question 4 of the OPC. The contributions confirm and support the evaluation findings on:

·challenges in supporting CSOs and other stakeholders such as political parties or cooperatives in their role as watchdogs (accountability) and democratic actors;

·uncertainties and potential drawbacks of blending and trust funds;

·insufficient synergies and complementarity among the different levels (intra-ACP, regional and country) and lack of clarity about how Intra-ACP cooperation works;

·added value of the EDF in terms of budget.

64.The contributions also highlight certain areas of interest:

·strong focus on CSOs, but not to other stakeholders (e.g., cooperatives and political parties) in the debate about the EDF (not restricted to the evaluation);

·the lack of connection between EDF support to OCTs and other forms of EU support available to these territories.

Industry, business or workers' organisations

65.One European international trade union responded to this question. The response discussed the implications of some SDGs and the related International Labour Organization (ILO) work on labour rights and included a set of recommendations and suggestions that could be relevant for the future of the EDF, but which do not relate to the contents of the evaluation. Certain specific aspects related to “ownership” are discussed above.

Research/academia

66.One research/academic institution responded to question 4. The reply confirmed the analysis made in the evaluation in relation to expertise, programming and sector concentration. The response also made a number of comments about the future of the EDF and the Cotonou agreement that are beyond the scope of the evaluation.

Citizen/private individuals

67.Only one response from a private citizen was received. The response does not provide any useful information in relation to the evaluation.

Response of the evaluation team

68.The diversity of the responses to question 4 of the OPC makes it difficult to follow the approach used for previous questions and build a comparative table of level of agreement/disagreement with the contents of the evaluation.

69.In general, those responses that referred to the contents and the analysis of the evaluation supported the findings and/or recommendations made in the report. More information on this can be found in the discussion per type of respondent above.

70.Some of the responses referred to issues that respondents would have liked to see addressed in the report. The different issues and the team’s responses are listed below:

·Sectoral breakdown of EDF support: this was not necessary to answer any of the evaluation questions in the report. After a detailed analysis, the team concluded that the request responds to a specific interest of the organisation making the comment in a certain sector.

·Evidence of impact of EDF11: as explained in the report and the methodology, implementation of EDF11 is just starting and it is too early to measure impact.

·Consideration of EDF11 added value in relation to DCI: dealing with this would require an in-depth understanding of the DCI. It would be addressed in the coherence report.

71.There are also some important points that came to the fore and which prompted the team to refine their analysis, conclusions and recommendations concerned testimonies as to:

·The lack of connection between EDF support to OCTs and other forms of EU support available to these territories.

·Support to people with disabilities. This is an issue that was reviewed in the EDF’s contribution to the coherence report. The team is making some revisions to ensure the discussion is more visible in the main report.

·Visibility of EU funding. This issue was also addressed in the EDF’s contribution to the coherence report.

Question 5: Other views common to several or all instruments.

Summary of contributions

72.Thirty three responses from 19 countries were received for the final question which invited comments across the EFIs. The majority of respondents were from organisations or associations (18) and also from European countries (11); see a full breakdown of respondents in Table 7.

Table 7.Number of responses to the question about EFIs, by type of respondent and country of origin of respondent

Source: ET, OPC analysis of 9th of May 2017

Views of public authorities

73.There were six public authority respondents. The one non-European country respondent (Jordan) is not a beneficiary of the EDF. Comments were mainly addressed to the EFIs broadly or to specific EFIs (ENI, IPA, PI and IcSP), with little specific feedback on the EDF. There was limited consistency among responses as they addressed different issues. However, two respondents agreed that, in looking forward, the EFIs should be closely tied to the EU Global Strategy and evolving policy framework.

74.Two responses emphasised a lack of suitable dialogue instruments to facilitate contribution from Trade Promotion Organisations (TPOs) given the increase in EFI projects dedicated to improving trade exchanges and FDI. In terms of lesson-learning between EFIs, one respondent recommended that a sideways view on complementarities and interactions should be included within the recommendation section of the EFI evaluations (complementarity and synergies between EDF and the EFIs’ regulatory framework are examined in Annex 20 of the Performance Review). One respondent agreed that the EDF should be tailored more to the OCTs’ particular contexts and levels of development, emphasising the lack of connection between EDF funding in OCTs with other potential sources of EU funding (as also highlighted in responses to Question 4). The respondent suggested that OCTs’ access to other EFIs should be facilitated and the notion of creating a dedicated instrument for the OCTs should be considered. Furthermore, it was agreed that the EU’s leverage and added value with OCTs was in part achieved through the visibility of its partnership, and that initiatives to increase this should be supported.

Organisation or association

75.The contributions of organisations and associations made up the majority of the responses to this question (18). Many of the responses targeted specific non-EDF instruments. Some were not relevant or were too general/brief to be useful.

76.Respondents concurred around a number of the evaluation findings concerning CSOs and the need for the EU to play a greater enabling role. There was a consensus among respondents that the EU is perceived to favour engagement with large international organisations, MS development agencies and private sector companies to the detriment of smaller CSOs and NGOs (both local and European). One respondent highlighted a trend where CSOs are increasingly excluded from participation and EU funding opportunities and the need to move dialogue and initiatives such as Roadmaps beyond a tick-box approach to more genuine participation. One respondent suggests that funding predictability for CSOs would improve if the EU systematically published its Annual Action Programmes and Annual Work programmes.

77.There was agreement with the evaluation findings around the increasing pressure on EFIs to tackle political priorities, particularly migration and security issues. One respondent noted the insufficient transparency on how, and from where, additional funding is being channelled to finance the new migration agenda with initiatives such as the EU Trust Fund for Africa. Another respondent noted the lack of coordination between different entities (instruments and MS activities) managing funds related to security which leads to a risk of duplication and a lack of transparency.

78.There were also contributions from organisations advocating for greater spending and commitment to biodiversity and disability-related assistance.

79.Contributions also corroborated the evaluation finding around the cumbersome and time-consuming nature of EU procedures, which are subject to frequent change, and the lack of consistent understanding among EU delegation staff of EU rules and procedures.

EU Platform, network or association

80.The contributions of EU platforms, networks and associations echoed some of the issues raised by organisations and associations, including calls for greater commitment to climate funding and facilitating an enabling environment for CSOs. In particular, responses spoke to the increase in the grant sizes of EU calls for proposals that means cooperation is being concentrated on a smaller number of large NGOs. Suggestions to improve local civil society’s access to EU funding include further improving consultation and dialogue, improved forecasting of calls for proposals across all instruments, and ring-fencing calls for proposals between European and local CSOs to increase equitable participation. One respondent stressed the need to improve and recognise the added value of EU MSs’ linkages to regions and countries, and that MSs’ historical and privileged positioning could be further leveraged by the EU.

Research/academia

81.One research organisation responded, concurring with the evaluation’s recommendation (R4) that programming should be based on robust political economy analysis, stressing the need to face up to political incentives which often sit behind EFI programming and implementation. EFI programming that is informed by a political economy analysis, which balances various disbursement pressures, was noted as being important to improve the chance of impact, and is a recommendation that was made by the evaluation.

Citizen/private individuals

82.One individual responded briefly to this question suggesting a radical change of approach was necessary.

Other

83.There was one single entry for the ‘other’ category, with recommendations applicable to all instruments including increasing EU staff capacity, and multi-stakeholder in-country dialogues.

Response of the evaluation team

84.OPC contributions across EFIs understandably drew broad responses and it was often unclear which specific EFI was being referred to. This therefore limits the relevance to the EDF evaluation and the team’s ability to respond. However a few clear points emerged:

·There is a perceived shrinking operational space for smaller European NGOs and local CSOs to engage with EU funding opportunities and participation, in part given the increasing size of EU grants being awarded and the lack of genuine dialogue and participation. (See also Question 4 as well as EQ1 and encompassed in R1)

·A key message was the need to improve transparency around financing of the new political agendas. Similarly, the need to couch EFI programming in robust political economy analysis was noted and is already one of the evaluation’s recommendations.

·Suitability of the EDF instrument for the OCTs, the need to facilitate OCT access to other EU sources of finance and the notion of examining the possibility of a dedicated instrument for OCT cooperation which would take into account the specific needs and level of development experienced by the OCTs was highlighted and is reflected in one of the conclusions of the evaluation. (C7)

·There was a clear view from some respondents around the need for EFI programming to reflect EU commitments to climate change and issues of biodiversity.

Table 11 in 0 attempts to capture, in quantitative terms, the outcome of the OPC in terms of agreeing or not with the findings and recommendations of the evaluation team.

Annex 1

Outcome of OPC responses and extent of disagreement with performance review assessment

Table 8.Outcome of the OPC responses and extent of disagreement with performance review assessment on addressing EDF objectives

Source: ET, OPC analysis of 10th of May 2017

Table 9.Outcome of the OPC responses and extent of disagreement with performance review assessment on the issue of ownership