EUROPEAN COMMISSION

Brussels, 29.1.2021

SWD(2021) 8 final

COMMISSION STAFF WORKING DOCUMENT

Evaluation Report

Accompanying the document

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL

Evaluation of Decision N° 534/2014/EU of the European Parliament and of the Council of 15 May 2014 providing macro-financial assistance to Tunisia

{COM(2021) 33 final}

Table of contents

Glossary2

1.Introduction4

   Purpose of the evaluation    4

   Scope of the evaluation    4

2.Background to the intervention5

   Description of the intervention and its objectives    5

   Points of comparison    7

3.Implementation / state of play7

   Implementation of the MFA operation    7

   Economic situation    9

   MFA-I and MFA-II                             12

4.Methodology2

   Evaluation techniques used    2

   Risks and limitations    4

5.Analysis and answers to the evaluation questions15

   Evaluation Question 1: Relevance of the operation    15

   Evaluation Question 2: Effectiveness    17

   Evaluation Question 3: Efficiency of the operation    19

   Evaluation Question 4: EU added-value of the operation    

   Evaluation Question 5: Coherence of the operation    

   Evaluation Question 6: Social impact of the operation    3

   Evaluation Question 7: Public Debt Sustainability of the operation    24

6.Conclusions25

Annex 1: Procedural information28

   Organisation, design and timing    28

Annex 2: Stakeholder consultation strategy……………………………………………………. 29

    Mapping of stakeholder groups                 29

   Methods and tools for engaging with stakeholders             30

ANNEX 3: METHODS AND DATA SOURCES32

ANNEX 4: LIST OF KEY MFA-I DOCUMENTS REVIEWED……………………………………………………..37

Annex 5: Timeline of the Tunisia MFA-I operation39

Glossary

Term        Meaning or definition

AA        Association Agreement

ACAA        Agreement on Conformity Assessment and Acceptance

AFD        Agence Francaise de Développement

AfDB        African Development Bank

ANCSEP    Agence Nationale de Contrôle Sanitaire et Environnemental des Produits

ANER        Agence Nationale d'Evaluation des Risques

CAD        Current Account Deficit

CBT        Central Bank of Tunisia

CdC        Cour des Comptes

CONECT    Conféderation des Entreprises Citoyennes de Tunisie

CPR        Congress for the Republic

CRES        Centre de Recherches et d'Etudes Sociales

DCFTA        Deep and Comprehensive Free Trade Agreement

DG ECFIN    Directorate-General for Economic and Financial Affairs

DG NEAR    Directorate-General for Neighbourhood and Enlargement Negotiations

DSA        Debt Sustainability Analysis

EBRD        European Bank for Reconstruction and Development

EC        European Commission

EEAS        European External Action Service

EFF        Extended Facility Fund

EIB        European Investment Bank

EIDHR        European Instrument for Democracy and Human Rights

EIU        Economic Intelligence Unit

ENI        European Neighbourhood Instrument

ENP        European Neighbourhood Policy

ENPI        European Neighbourhood and Partnership Instrument

EQ        Evaluation question

EU        European Union

EUR        Euro

FDI        Foreign Direct Investment

FSAP        Financial Sector Assessment Programme

FX        Foreign exchange

GDP        Gross Domestic Product

GIZ        German Society for International Cooperation

GNI        Gross National Income

GOJ DPL    Governance, Opportunities and Jobs Development Policy Loans

IACE        Institute Arabe des Chefs d’Entreprises

IDA        International Development Association

IFI        International Financial Institution

IMF        International Monetary Fund

INS        Institut National de la Statistique

ISG        Inter-Service Steering Group

LHS        Left hand scale

LOB        Loi Organique Budgétaire

MDIIC        Ministry of Development, Investment and International Cooperation

MENA        Middle East and North Africa

MFA        Macro-financial assistance

MOF        Ministry of Finance

MoU        Memorandum of Understanding

NEET        Not in Employment, Education or Training

NGO        Non-governmental organisation

NIS        National Institute of Statistics

OA        Operational Assessments

ODA        Official Development Assistance

PAI        Programme d'Appui a l'Intégration

PAR        Programme d’Appui à la Relance

PFM        Public Finance Management

PNAFN        Programme national ď aide aux familles necessiteuse

PPP        Public Private Partnership

RHS        Right Hand Scale

SAO        State Audit Office

SBA        Stand-By Agreement

SIA        Social Impact Analysis

SCF        Stand-By Credit Facility

SSN        Social Safety Net

SWD        Staff Working Document

TA        Technical Assistance

TND        Tunisian Dinar

ToR        Terms of Reference

UGTT        Tunisian General Labour Union

UN        United Nation

US        United States

USAID        United States Agency for International Development

VAT        Value Added Taxes

WB        World Bank

WB DPO    World Bank Development Policy Operations

1.Introduction

Purpose of the evaluation

This Staff Working Document (SWD) presents an evaluation of the Macro-Financial Assistance (MFA) operation to Tunisia provided by the European Union (EU) over the period 2015-2017. It largely draws on an independent, ex-post evaluation 1 conducted by an external contractor and a consultation process that involved targeted stakeholders in Tunisia as well as EU staff.

The aim of the evaluation is to assess whether the MFA operation of 2015-2017 met its objectives to support Tunisia in addressing its balance-of-payments problems and implementing economic and structural reforms that would stabilise its economy and enhance the sustainability of its external position. The purpose of the evaluation is to support decision-making by identifying areas of improvement for similar on-going or future MFA operations, while also ensuring transparency and accountability.

Scope of the evaluation

MFA is a policy-based financial instrument of untied and undesignated 2 balance-of-payments support to partner third countries. It is designed to assist third countries that are geographically, economically and politically close to the EU. MFA takes the form of medium/long-term loans, grants or a combination of the two. Unlike other, regular development aid provided by the EU, MFA is exceptional in nature and is mobilised on an ad-hoc case-by-case basis. Its objective is to help restore a sustainable external financial situation, while encouraging economic adjustment and structural reforms in the partner country. MFA always complements (and is conditioned on) financing provided in the context of a reform programme agreed with the International Monetary Fund (IMF).

Since the Arab Spring began, the EU has committed to supporting Tunisia in its economic and political reform process. This support has remained consistent with the EU’s policy towards the Southern Neighbourhood region, set out in the context of the European Neighbourhood Policy.

In May 2014, the European Parliament and Council adopted a decision 3 to provide EUR 300 million - as part of a wider package of international assistance - to support Tunisia’s economic and political transition following the 2011 revolution. The aim of the MFA was to help Tunisia cover its external financing needs and to support structural reforms. The assistance was disbursed in three tranches between May 2015 and July 2017, alongside IMF assistance and contributions from other donors like the World Bank and the African Development Bank. The MFA disbursements were linked to the fulfilment of nine specific structural reform conditions related to reforms in six areas, namely: taxation, public finance management, social safety net, financial sector, public statistical system and trade.

In accordance with Article 34(1) of the Financial Regulation 4 , MFA operations in third countries are subject to an ex-post evaluation. In turn, the aforementioned MFA Decision for Tunisia stipulates that the European Commission is required to “submit to the European Parliament and to the Council an ex post evaluation report”.

To this end, the Directorate-General for Economic and Financial Affairs (DG ECFIN) engaged an external contractor to complete an independent assessment, which informs this Staff Working Document and which ran from September 2018 until July 2019, with the objectives of:

1.Analysing the impact of the MFA on the economy of Tunisia and, in particular, on the sustainability of its external position;

2.Assessing the added-value of the EU’s intervention. In general, the evaluation seek to draw lessons with respect to the EU’s financial assistance, i.e.

a.Whether the ex-ante considerations determining the design and terms of the operation were appropriate, taking due account of the economic, political and institutional context; and

b.Whether the outcome of the programme met the objectives.

These areas were assessed along the following key evaluation criteria: relevance, effectiveness, efficiency, EU added-value, and coherence with both other EU policies and interventions from international donors. In addition, the evaluation assessed the social impact of the MFA and the impact on the sustainability of Tunisia’s public debt. This is further specified in the Evaluation Roadmap. 5

2.Background to the intervention

Description of the intervention and its objectives

Following the Jasmine Revolution and due to the conflict in neighbouring Libya, the Tunisian economy experienced a recession in 2011, with GDP contracting by 2% during the year. While economic growth resumed in 2012, persistently high unemployment rates (notably among the youth), a deepening political crisis and widening external and fiscal imbalances created considerable macroeconomic uncertainty and prompted the need of international financial support.

As regards the external sector, the current account deficit continued to widen after the 2008 global financial crisis, driven by a deteriorating trade balance. The turmoil following the 2011 revolution caused severe disruptions to industrial production and services, with investor sentiment weakening while the EU, Tunisia´s main trading partner, was hit by the sovereign-debt crisis. The conflict in neighbouring Libya also prompted a loss of remittance income, owing to the return of a large number of migrant workers. Buoying imports (especially energy and capital goods) and declining oil and phosphate exports in 2013 and 2014 worsened the trade balance and added further strains. As a result, the current account deficit stood at almost 9% of GDP, on average, between 2012 and 2014.

Turning to the fiscal situation, after 2008, the Tunisian government resorted to various fiscal policy measures with the aim of reinvigorating the economy, including increases in public sector wages and subsidies on food and energy. By 2012, the fiscal deficit had increased to nearly 6% of GDP. Faced with economic stagnation and rising unemployment, the transition government in Tunisia opted for further expansionary fiscal measures. These contributed to a budget deficit widening further to almost 7% of GDP in 2013 and public debt increasing to almost 47% of GDP in the same year.

In this difficult macroeconomic context, in June 2013, Tunisia entered into a USD 1.75 billion (400% of quota), 24-month Stand-by Arrangement (SBA) with the IMF. Shortly upon the conclusion of the SBA in December 2015, a 48-month Extended Fund Facility (EFF) of USD 2.9 billion was signed between the two parties, in May 2016. The IMF EFF programme aimed to promote stronger and more inclusive growth by consolidating macroeconomic stability, reforming public institutions—including the civil service, facilitating financial intermediation, and improving the business climate. In addition, Tunisia was in receipt of development policy loans provided by the World Bank. The World Bank loans aimed to help lay the policy foundations for a more competitive business environment, a strengthened financial sector, more inclusive and accountable social services, as well as more transparent public governance.

To complement Tunisia’s arrangements with the IMF, the EU pledged to grant MFA. Upon Tunisia’s request and following an ex-ante evaluation, the European Commission proposed 6 in December 2013 up to EUR 250 million in macro-financial assistance (MFA) loans. The European Parliament and Council approved the MFA on 15 May 2014 and agreed to increase the amount of the assistance to EUR 300 million 7 .

Following the consultation of the Member States' Committee on MFA on 9 July 2014, a Memorandum of Understanding (MoU) 8 and Loan Facility Agreement related to this assistance were signed, while the ratification by the Tunisian Parliament took place in March 2015. Annex 5 of this report summarizes the timeline of the operation.

The assistance was provided in three tranches of EUR 100 million each, disbursed between May 2015 and July 2017, and conditional to good progress under the IMF's SBA, political prerequisites 9 , and to the fulfilment of the set of policy conditions specified in the Memorandum of Understanding.

The MFA operation aimed to help Tunisia move forward with its planned economic reforms while also underpinning its political reform efforts. The general intervention logic of the MFA-I operation, applicable to the 2015-2017 Tunisia programme, is summarised in the graph below.

Figure 2.1 Intervention logic of MFA-I operation to Tunisia

Source: ICF

Points of comparison

The points of comparison, against which the MFA operation of 2015-2017 is assessed, refer to the situation in Tunisia (1) prior to the intervention, on one hand, and (2) during and immediately after the implementation of the MFA operation, on the other hand. As noted in the previous subsection, which describes the situation prior to the MFA operation, particular attention is paid to the external sector and the fiscal situation of the Tunisian economy. Section 3 will describe the implementation and following phase of the MFA operation. It will focus both on the main developments in Tunisia in the areas of political reform covered by the programme, and on the wider economic development of the country.

3.Implementation / state of play

This section briefly describes the implementation of the MFA-I operation to Tunisia, looking at the key developments on the policy reform areas supported by the programme. It will then describe the economic situation in Tunisia during and immediately after the implementation of the MFA operation, with particular reference to the points of comparison in the external sector and the fiscal situation. The assessment of the effectiveness of the MFA operation will be then addressed in section 5.

Implementation of the MFA operation

To achieve the objectives detailed in section 2, and as per the usual MFA procedure, disbursements under this operation were tied to the fulfilment of political pre-condition (the respect of human rights, effective democratic mechanisms, including a multi-party parliamentary system, and the rule of law), as well as good progress with the implementation of the IMF programme. The disbursement of the second and third tranches was also subject to the fulfilment of a set of country-specific policy conditions, specified in the MoU and related to six structural reform areas 10 .

The operation was disbursed in full, in three instalments, over the period May 2015 – July 2017:

·First tranche of EUR 100 million loan, disbursed in May 2015;

·Second tranche of EUR 100 million loan, disbursed in December 2015; and

·Third tranche of EUR 100 million loan, disbursed in July 2017.

MFA loans were provided on highly favorable terms. Maturity and interest rates differed for each tranche, varying between (1) the coupon of 0.49 per cent and the maturity of 12 years for the first instalment, and (2) the coupon of 1.25 per cent and the maturity of 15 years for the last instalment. For all the three instalments, it was envisaged a bullet capital repayment (i.e. a lump sum of the full outstanding amount) in the last year of maturity. With respect to the interest rates, no grace period was applied.

The operation covered nine specific conditions under six reform areas:

Figure 3.1 Structural reform areas under MFA-I to Tunisia

Tunisia satisfactorily fulfilled all the policy conditions necessary for the disbursement of the three instalments, with the exception of condition 8 on trade policy, for which a waiver was granted by DG ECFIN 11 . The condition required the publication of the decrees implementing the new technical regulations converting the existing system of industrial compulsory standards (normes homologuées) into a system aligned with that of the EU, for two priority sectors: building materials, and electrical and electronic products.

According to the external evaluation report, the relative complexity of the reform was coupled both with:

·shortages in the qualified staff in some of the departments at the Ministry of Industry and SMEs, involved in the technical aspects of the implementation; and

·regular changes of the Tunisian governments, that clearly affected the configuration and collaboration among the multiple ministries involved. Furthermore, continuous updates of the European directives (i.e. norms related to the EU acquis) made the legislative work of the Tunisian side more challenging.

In light of the delay, and due to efforts made by the Tunisia authorities to achieve progress in relation to this condition, a waiver was deemed fully justified.

In the area of public finance management, MFA policy conditionality supported the adoption of an Organic Law of Cour des Comptes (CdC), strengthening the external audit of public accounts and ensuring the financial independence of the Court. The Law was first adopted by the Tunisian Government in 2016, but only became effective with the adoption by the local Parliament on the 16th of April 2019. Regarding the social safety net, MFA conditions have helped to improve the management of the system, through better targeting by setting up a dedicated database and launch a dedicated survey. In the financial sector, MFA conditions have supported the strengthening of the Central Bank’s governance and a new banking law to strengthen the banking sector. In terms of trade and competition policies, MFA conditions have supported export performance and strengthened the competition regulation by fostering a general alignment with the EU in this area, including a law on security of industrial products and a law on food security. On fiscal management, MFA supported the adoption of a decree reducing the number of economic activities eligible for the regime forfaitaire with a view to increasing the tax collection and make the system more equitable. The conditions also strengthen the statistical system through the adoption of a national chart of public statistics (consistent with the UN fundamental principles of statistics) by the National Institute of Statistics.

The Tunisian authorities were generally effective in the implementation of the conditionality, notwithstanding the waiver on condition 8. On the EU’s side, the monitoring process was adequate and the Commission complied with all checks ensuring that Tunisia had satisfactory fulfilled the reform measures supported by the MFA. Implementation of MFA policy conditions is assessed in detail in section 5 of the present document, as part of the relevance and effectiveness of the MFA operation of 2015-2017.

Economic situation

During the review period and due to the negative effects of two terrorist attacks, Tunisia’s economy remained stagnant until the end of 2016, then registering a slight recovery starting from the following year, with a real GDP growth rate of 1.9% in 2017 and of 2.7% in 2018.

Tunisia’s external imbalances persisted, bringing down foreign exchange reserves. Despite a slight recovery in 2015-16, a number of factors maintained the pressure on the current account. These included: 1) the worsening energy deficit following reduced domestic energy production because of the maturation of oil fields and reduced exploration during the revolution period; (2) reduced external demand from the EU until 2018, a region that accounts for 65% of trade with Tunisia; (3) negative supply shocks in the phosphate mining sector because of labour disruptions and social unrest; and (4) reduced FDI inflows and tourism revenue in the wake of the 2015 terror attacks.

The balance of payments deteriorated in 2018 and the current account deficit reached 11.2% of GDP at the end of 2018, against 10.2% one year earlier. The trade account deficit reached about 16.3% at the end of November 2018, up from 15% a year later. Growing volumes of imported energy, coupled with the increasing price of oil and the depreciating dinar, translated into a widening import bill overall, which was not offset by exports in spite of the latter’s increasing prices. The tourism sector experienced a revival in 2018 while remittances from abroad decreased slightly throughout the year. The combined inflow was 22% higher than in 2017.

Figure 3.2 Current account balance and trade balance, as % of GDP

Source: National Institute of Statistics

Tunisia’s public debt continued to rise sharply in the years following the 2011 revolution. The debt ratio increased from 41% of GDP in 2010 to 71% of GDP at the end of 2017. Tunisia's elevated fiscal deficits drove up government debt and adverse exchange rate dynamics, as more than 68% of Tunisia's government debt is denominated in foreign currency, also had a heavy impact on debt. Debt service costs had once again a considerable impact on total expenditures in 2018.

Figure 3.3 Public debt and public deficit, as % of GDP

Note: Figure 3.3 illustrates the trend of Tunisian public debt and public deficit (% of GDP) between 2000-2017. Measurement scales on the y-axis reported on the left and right side of the graph refer to public debt and public deficit, respectively.

Source: IMF

In terms of the fiscal situation, the deficit increased dramatically during the reference period, as shown in Figure 3.3. An increase in public spending in the immediate aftermath of the revolution was followed by some fiscal restraint in 2014. This increase mainly reflects higher government spending (notably public sector wages and social spending) even if fiscal consolidation was meant to be one of the key elements of the second agreement with the IMF.

Figure 3.4 Macroeconomic indicators

Source: IMF; Central Bank of Tunisia; Tunisia National Statistical Institute

MFA-I and MFA-II

In 2016, Tunisia requested a second MFA operation from the EU, which would accompany the successor IMF Extended Fund Facility (EFF) programme of USD 2.9 billion, which followed the Stand-by Arrangement that was concluded in December 2015 12 .

In view of the strong impact the deteriorated security situation was having on Tunisia's economy; and after an updated assessment of the country’s external financing needs conducted in liaison with the IMF, the EU adopted a second MFA (MFA-II) to the Republic of Tunisia amounting to a maximum of EUR 500 million, in the form of medium-term loans.

The policy-reform measures included in the MoU for MFA-II build on sectors that had been identified as priorities in the policy programme attached to MFA-I. The conditionality for MFA-II focused on: improving PFM; reforming the tax system to increase tax collection while enhancing tax equity; reinforcing and better targeting the social safety net; strengthening the banking system; promoting investment and supporting the recovery of the tourism sector; and enhancing active labour market policies so as to reduce Tunisia’s high unemployment rate.

4.Methodology

Evaluation techniques used

The methodology for evaluating the MFA operation in Tunisia over the period 2015-2017 was guided by the Commission’s Better Regulation Guidelines 13 and the Guidelines for the Ex-Post evaluation of Macro-Financial Assistance Operations 14 .

For the evaluation conducted by the external contractor, the evidence and the data were collected through several complementary approaches, including (i) a theory based approach; (ii) quantitative and qualitative research methods; and (iii) triangulation 15 . Overall, the quality of the collected evidence by the external contractor (data, documentation, interviews and survey results) for this evaluation can be assessed as very good, within the limitations mentioned below.

The qualitative research was grounded in logic and economic theory, whilst the quantitative fieldwork was based on reliable statistical data, and purposeful sampling was used for the interviews and the focus group discussion. To collect a broad, multi-dimensional and triangulated picture of the economic, financial and structural issues surrounding the programme, a wide range of relevant stakeholders and civil society organisations was also involved. Based on this and the triangulation of evidence, this evaluation can be considered reliable and valid.

Methods used to build the evidence for the evaluation are explained below 16 . Overall, triangulation of findings obtained using different techniques has helped to increase validity of the evaluation results. The methodology employed was comprehensive and responded to the very specific and unique nature of the MFA operation.

a.Documentary Review

The main documents used for the evaluation were the Ex-ante assessment of the MFA, the Commission proposal and MFA Decision, the MoU (MFA I and II), and mission reports drafted by Commission’s Directorate-General for Economic and Financial Affairs (DG ECFIN). Additional sources included other EU documents (such as the EU-Tunisia Neighbourhood Agreement), reports of the IMF, World Bank and other international organisations, as well as analyses carried out by research institutes.

b.Macroeconomic data Analysis (including DSA)

For quantitative analysis, the evaluation used data from the national sources (mainly, the Ministry of Finance and the National Bank of Tunisia) as well as from international organisations such as the IMF and the World Bank. The quantitative analysis notably covered macroeconomic fundamentals, fiscal indicators, external sustainability variables, financial sector variables, and structural reforms (e.g. variables measuring socio-economic performances). Additionally, a macroeconomic tool developed by the IMF was used for Debt Sustainability Analysis (DSA), 17 while fiscal savings (resulting from concessional terms of the MFA operation in question, as compared to market-based alternatives) were calculated by comparing the face value and the net present value of the operation.

c.Semi-structured interviews with key stakeholders

A series of 31 in-depth semi-structured interviews 18 were conducted, aiming at gathering information on the design, implementation and results of the MFA operation. The focus was primarily on the macroeconomic and fiscal situation in Tunisia, on structural reforms, on the social impact and debt sustainability. The interviews were confidential, and pre-interview questionnaires were used to improve the quality. Interviews were held with key stakeholders representing: the European Commission and the interest of the European Union; International Financial Institutions; and Tunisia. Annex 2 of the present report provides further details about the stakeholder groups and the interviews-.

d.Focus group

Half a day focus group discussion 19 with locally-based civil society and business representatives was organised during the second mission to Tunisia (March 28th, 2019). It complemented and crosschecked information gathered from desk research and targeted stakeholder interviews.

e.Delphi survey

A Delphi survey was undertaken with a panel of 82 experts, representing the business society, think thanks, financial and macroeconomic analysts, and the Academia. The scope of the survey was to assess on the contribution of the MFA in achieving macroeconomic stability, easing external financing constraints and alleviating Tunisia’s balance of payments and budgetary needs 20 . Experts were interviewed using a structured questionnaire and were asked to elaborate on plausible scenario would MFA-I not have been available, and the potential implications. The recruitment to the panel was carried out with the support of the local economic experts and with the advice from DG ECFIN and the EU Delegation in Tunis.

f.Case studies

Two in-depth case studies on MFA-promoted reforms were developed in the following areas: (1) social safety net reforms and (2) tax policy (and more specifically “regime forfaitaire”). They were mainly based on the desk research, targeted stakeholder interviews and interaction of the local economic experts.

g.Qualitative counterfactual analysis

A qualitative counterfactual analysis was preferred by the evaluation team to a quantitative approach, due to the difficulties of isolating the effects of MFA from other interventions (i.e. the IMF programme, other EU interventions and supports from other donors) and other exogenous and/or unobservable factors. It applied a theory-based approach to draw inferences regarding the role and contribution of the MFA in promoting macroeconomic stabilisation. Information gathered through the other methods was used to deduce wat might have happened in the absence of the MFA (alternative 1) and in the absence of the MFA + IMF assistance programmes (alternative 2).

h.Social Impact Analysis (SIA)

For the Social Impact Analysis, the evaluation analysed trends of key indicators prior to, during and after the MFA operation: (i) wages, (ii) poverty, (iii) household expenditure, (iv) employment, (v) unemployment, (vi) education, (vii) health. Counterfactual reasoning was applied to deduce the extent to which the MFA operation contributed to the observed outcomes.

Risks and limitations

While the overall reliability and validity of the evaluation is strong, a number of methodological limitations and challenges affected the evaluation:

·While generally data coverage is good, it is limited in some areas of interest (notably, as regards social indicators) and longer time series are not always available to carry out robust analyses. An additional limitation relates to the lack of sufficient debt sustainability and fiscal statistics.

·Another limitation relates to the reliability of the judgment provided by Delphi experts and key stakeholders, as some participants had very limited awareness and knowledge of the MFA operation. Moreover, the time elapsed between the first disbursement under MFA-I and the external evaluation in question caused some ‘memory loss’.

·The changing economic environment over an extended period and the fact that the MFA operation was implemented in parallel with IMF and other international support programmes also made it difficult to disentangle the impacts of the MFA operation from the impacts resulting from other factors (therefore limiting the counterfactual analysis).

The Inter-Service Steering Group, as well as other staff from the Directorate-General for Neighbourhood and Enlargement Negotiations (DG NEAR), the European External Action Service (EEAS) and the EU delegation in Tunisia, actively participated in dealing with these challenges, including by providing missing data to the evaluators and multiple rounds of feedback.

In turn, the identified risks and limitations do not put into question the overall reliability of the evaluation analysis, as they were mitigated by the fact that information was obtained from a wide range of sources, using different evaluation techniques, alternative scenarios and multiple rounds of feedback. Therefore, the conclusions reached in the evaluation can be considered as valid.

5.Analysis and answers to the evaluation questions

Evaluation Framework

The ex-post evaluation covered five evaluation criteria used in the assessment of EU programmes, namely: (1) relevance, (2) effectiveness, (3) efficiency, (4) EU added-value, and (5) coherence. The evaluation considered two additional criteria: (6) social impact and (7) debt sustainability.

Answers to questions 21

Question 1.1 Based on the MFA Decision 22 and the Memorandum of Understanding 23 , the objectives of the MFA operation were, essentially, to alleviate short-term external financing pressure and help Tunisia return to a sustainable path.

The overall design of the MFA operation was relevant to its objectives. More specifically, the criteria for the first instalment (ratification of the MoU and the IMF programme being on track) allowed for a swift disbursement. This was in line with the objective to provide short-term relief. The second and third instalments were additionally subject to the implementation of several policy conditions, which provided a suitable means for encouraging structural reforms.

Question 1.2 The amount of the EU contribution under MFA-I to Tunisia corresponded to circa 0.5% and 0.3% of the country’s GDP in 2015 and 2017, respectively. In absolute terms, this was the third largest MFA operation since 2000 (leaving aside the MFA I, II and III to Ukraine, which were of unprecedented size). There was consensus among the consulted stakeholders that an increase of the MFA size from initially proposed EUR 250 mln to EUR 300 mln, advocated strongly by some Member States, was an appropriate decision. The envelope covered 14.8% of the residual financing gap in 2015 and around 11.3% for the whole period 2015-2016. 24 The EU contribution, was considered to be appropriate (in terms of burden sharing with other donors) and proportional (limited to the minimum necessary to achieve short-term macroeconomic stability in Tunisia).

The first two instalments were disbursed in 2015 when the external financing needs of Tunisia (as reassessed by the Commission 25 and the IMF 26 in 2014) were still growing. As for the third instalment disbursed in 2017, it helped to close the financing gap identified as part of the EFF programme agreed with the IMF, mostly in form of replenishing Tunisia’s foreign exchange reserves.

Regarding the terms of the MFA operation of 2015-2017, stakeholders have acknowledged this coherence during the interviews. The MFA financing was provided in the form of EUR 300 million loan on the highly concessional terms that could not have been obtained on the market. The initial assessment of the guiding principles including inter, alia, per capita income of Tunisia, debt sustainability and poverty level excluded correctly the grant component from the consideration. The form of the MFA (entirely loan) was therefore deemed appropriate.

Question 1.3 As confirmed by the civil society during the focus group, and by the EU Delegation and the World Bank, the areas of MFA policy conditionality (public finance management, financial sector, social safety net, statistics, trade and taxation policies) covered the most relevant reform challenges in Tunisia. The reforms promoted by the MFA were found to be in line with the country priorities and backed by thorough analytical work conducted by DG ECFIN, which also comprised consultations with other donors present in Tunisia, in particular the World Bank and the IMF.

Likewise, at the level of specific conditions, the focus of most of the conditions was also (highly) relevant with some emblematic examples such as the reform of the simplified income tax declaration system (regime forfaitaire). In case of two particular conditions (namely, the trade related condition concerning the approximation of the industrial compulsory standards with the EU acquis, and the social safety net reform, concerning the progress in the establishment of a unified database and targeting system based on a single Social Identification Number), it appears that their focus may have been too broad/ ambitious, albeit their relevance remained high.

Overall, the focus of the MFA reforms was found to be right. While some selected reforms were ambitious and arguably surpassed the capacity of the Tunisian authorities, they related to areas were fast and very meaningful improvements have been urgently needed.

The impact of the timeline (from proposal to disbursement) of the operation on its relevance (Question 1.4) was limited. Overall, despite protracted negotiations and the subsequent lag between 2nd and 3rd tranche, the relevance of the MFA remained high, given the prevailing macro-economic conditions and increasing budgetary needs of the Tunisian State, following the terrorist attacks in 2015. The objectives of macroeconomic stabilisation and supporting structural reforms were just as relevant in 2014 (when the MoU was negotiated) as in 2017 (when the last tranche was disbursed). In terms of macroeconomic challenges, Tunisia was facing political turmoil and a regional crisis that affected both its balance-of-payments and fiscal situation. In terms of structural reforms, the MFA operation under evaluation not only supported them directly during a difficult period but also (indirectly) created space for reforms by easing the macroeconomic adjustment. These views have been confirmed by the Delphi survey and the targeted stakeholder interviews, which broadly agreed that MFA I helped making the necessary economic adjustment in Tunisia less harsh and abrupt.

Question 2.1. In order to assess the role of MFA-I in promoting macroeconomic stability, easing external financing constraints and alleviating Tunisia balance of payments and budgetary needs, the external evaluation report, at first, analysed the developments achieved in the country, irrespective of the actual role played by the MFA 27 . Subsequently, it inferred the contribution given by the operation in question, by implementing two alternative counterfactual analyses (Alternative 1 and Alternative 2).

Counterfactual analyses were based on:

-Literature review covering DG ECFIN and IMF documents related to the operation;

-Trend analysis of macroeconomic data;

-Scoping interviews with targeted stakeholders (DG ECFIN officials involved in design and implementation of the programme, representatives of the IMF, World Bank and Tunisian institutions);

-Discussions with the Steering Group;

-Results from the Delphi survey;

-Insights from local economic experts and NGOs.

In the absence of the MFA (Alternative 1), the evidence suggests that obtaining alternative financing from the international financial markets would have been the most plausible course of action. The difference in the cost of the MFA loan versus the one obtained from the international financial markets would have oscillated around EUR 110 mln accrued over 15 years, due to higher interest rates for the latter option and shorter maturities available. From the debt sustainability perspective, this increase would not have been large enough to drastically change the debt burden; and in terms of the impact on the real economy, Tunisia would have registered a slightly lower real GDP growth between 2015-2017.

Figure 5.2.1 Summary – alternative scenarios for obtaining financing had MFA I not been available (but with IMF support continuing)

Source: ICF

The hypothetical absence of both the MFA and IMF support programme (Alternative 2) could have had far more severe implications. In fact, given the ‘catalytic effect’ of the IMF assistance, its absence could have also resulted in the absence of African Development Bank and World Bank budget support operations. Under this scenario, this would have resulted in the absence of EUR 1099 mln in 2015, EUR 726 mln in 2016, and EUR 835 mln in 2017, or circa 3.1%, 2.1% and 2.5% of GDP in these three consecutive years.

In this case, the authorities would have had to resort to few options including: far less concessionary lending from bilateral donors (Saudi Arabia and Qatar) and international financial markets (with no certainty that these latter would have financed Tunisia without an IMF programme); and potentially some cuts of capital expenditures of limited size.

Consequences would have been an increase in the cost of debt servicing, loss of contracting power for financing from Gulf countries, and decrease in public investments, with a subsequent worsening of the GDP performance, inflation level and depreciation of Tunisian dinar.

Question 2.2 With respect to the short and medium-term structural effects of the MFA operation of 2015-2017, we should distinguish between: progress in the reform implementation within the narrower scope, as defined in the text of the specific MFA conditions; and the overall progress in the broader sense, as demonstrated by tangible results achieved. In the first instance, the MFA has been broadly successful. Conditions related to the new banking law had the law on Central Bank made positive contributions and reinforced the efforts of the IMF. Yet, when we consider the broader perspective (beyond the narrow definition of conditionality), progress has been considerably slower than expected in a number of areas including SSN reforms, trade or régime forfaitaire 28 , and there is little evidence of tangible results.

This, however, has to be viewed in the context of a challenging economic background, characterized by weak institutional capacity, fragmented political landscape and frequently changing governments, all of which reduced the political ownership. It should also be noted that the overall level of ambitiousness of the MFA conditionality package (as compared to past MFA operations) was high, and so were the benchmarks for its effectiveness. Therefore, this should be taken into consideration, when making an overall judgment on the most challenging conditions.

Question 3.1 To answer to this question, the evaluation primarily focused on the ownership of the programme by the Tunisian authorities, on their capacity to reform, and on the effectiveness of monitoring activities. Flexibility of the operation to adjust to contextual changes, effectiveness of dialogue among concerned parties and visibility of the MFA itself were also considered.

As assessed by some interviewed stakeholders, the sense of ownership of the MFA was lower than initially expected by the European Commission. One of the main impediments was the political instability resulting from frequent changes of government, which led Tunisian authorities to focus more on short-term objectives. Some of the reduced ownership had been anticipated by DG ECFIN at the design stage of MFA, leading to more contained and less ambitious conditionalities. However, the discrepancy between actual and expected ownership was still high, stressing the need for a better ex-ante assessment of local capabilities and political stability for future MFA interventions in the country.

The study team found some reduced capacity in some Tunisian institutions exposed throughout the design and implementation of the MFA, which hindered reforms. The impaired capacity was partially caused by budgetary pressures faced by the local institutions, which led to an outflow of experienced staff. It was, moreover, amplified by a challenging political context (frequent changes of the governments), influential vested interests’ groups having often a disproportionate say in the policy discussions (i.e. some trade unions), and by the design of some conditions, requiring cross-ministerial collaboration, that added difficulty to the task.

However, it is important to notice that young Tunisian institutions have operated in the democratic context since only several years, a very limited time to expect full adjustment to materialize. Furthermore, it is also warranted to recognize that the level of ambitiousness of some of the MFA reforms was rather high and there have been also numbers of other conditions promoted by alternative donors that absorbed resources of the state apparatus.

Field missions led by DG ECFIN staff constituted a primary tool for monitoring the MFA operation; four missions took place between the ratification of the MFA by Tunisian Parliament in early 2015 and the closure of the operation. Some weaknesses were found in the coordination of the MFA on the Tunisian side, led by the Ministry of Development, Investment and International Cooperation. Nonetheless, overall, the outcomes were deemed appropriate.

Finally, the focus group and stakeholder consultation suggested that the visibility of the MFA has been negligible, and limited to narrow groups of experts. Indeed, the review of the visibility of the operation and the EU communication activity indicated scope for improvement. Being this evidence largely in line with the findings from other past MFA evaluations, the European Commission (EC) decided to improve the visibility and public understanding of all MFA programmes. To his end, since mid-2014, the EC started to publish on the web site of DG ECFIN the Memoranda of Understanding that lay down the reform measures related to each MFA operation.

Question 4.1 The EU intervention primarily lead to financial benefits for the Tunisian economy, with the highly concessional terms of the MFA loans granting fiscal savings for the local government and a gradual adjustment of the primary public deficit. The financial added-value of MFA operations also derives from the fact that the EU could mobilise and coordinate a wider amount of resources, as compared to any other individual donor country. Moreover, MFA included conditions on SSN reforms, granting a politically reinforcing effect that contributed to the sustained mobilisation of local authorities around this reform area.

Question 4.2 In certain reform areas, MFA added additional endorsement to the reforms driven by the IMF (i.e. financial sector) and the World Bank (i.e. SSN reforms). It was justified to give those reform further push, given their importance and the historical delays in their progress. In a context of political transition marked by limited capacity of Tunisian institutions, it has been also crucial from a donors’ coordination point of view that both priorities set and the number of reform areas promoted across all donors remain manageable (to avoid overwhelming the young Tunisian institutions).

There were also areas where MFA conditions promoted reforms that were not addressed by other donors. In three areas (statistics, public finance management and trade), MFA conditions provided further leverage to reform efforts promoted under some EU budget support operations. Furthermore, in the case of the simplified income tax declaration system (regime forfaitaire), the MFA-I was the only instrument promoting related reforms.

Question 4.3 The level of coordination with the international donor community was considerable and it proved to be crucial for achieving faster and more effective results. There has been a high degree of reform interdependence/ cross-conditionality, especially in the areas of SSN and financial sector. As such, the MFA gave additional leverage for the advancement of reforms seen as key at the EU and international level, but where progress was not materializing fast enough.

Over the period 2013-2017, EU deployed a total of EUR 388 million to Tunisia, via budget support programmes. While all types of aid are coherent elements of the broader support provided to Tunisia, only budget support type of assistance is meant to stabilize the macro-economic situation while encouraging the reform process. The most sizable budget support programmes implemented in Tunisia are known as Programmes d’Appui à la Relance (PARs), which are general budget support programmes running for a maximum of two years, to which the EU contributed in a grant form.

Since 2011, there has been five consecutive PARs, under which disbursements were conditional upon pre-agreed reform progress 29 . Over the five programmes, the conditions covered a wide range of areas, including: (i) transparency, democratic participation, justice and the fight against corruption, (ii) public finance, (iii) regional disparities, (iv) unemployment and social inclusion, (v) micro-finance, and (vi) economic growth.

Figure 5.5.1 MFA I and PARs conditionalities – mapping

MFA conditions often complemented the reform package associated with the IMF’s arrangements with beneficiary countries, as well as those related to the World Bank support programmes. There were two main areas with synergies among donors, namely financial sector and SSN. Some cases of cross-conditionality at same or different points in time can be observed:

ŸFor instance, in the case of the development of a unified database (MFA Action 2 and, implicitly 6 on SSN), the IMF introduced similar condition calling for an “establishment of a databank on vulnerable households” as from the beginning of the EFF programme (in June 2016). The World Bank was providing a crucial technical assistance in parallel.

ŸWhen it comes to the cash transfer support programme to compensate vulnerable households affected by the reform of the energy price subsidies, while the MFA called for the adoption or reinforcement of such programme (Action 6), the IMF required the “submission to the Council of Ministers of a new targeted household support programme to accompany the reform of the generalized energy subsidies”, from the beginning of the SBA until the fourth review 30 .

ŸIn relation to the central bank law, the IMF had among its structural benchmark in its 1st and 2nd review (May 2014) the “Submission to the CBT board of the draft of the new central banking law in line with best international practices” while the EU had for the 2nd tranche of the MFA (disbursed in December 2015) the condition that the central banking law meant to be submitted to the Parliament.

In other reform areas, there was more of a split of roles.

ŸFor example, investment climate was an area addressed by both IMF and WB but not by MFA-I 31 .

ŸIMF focused on several aspects of PFM reforms while the MFA focused exclusively on external audit and the World Bank on transparency and public procurement.

ŸTax reform was also widely addressed by the IMF (in relation to the Large Taxpayer Unit for instance) but there was nothing specifically on the regime forfaitaire 32 in IMF programmes.

At the level of specific conditions, the desk research and the interviews with the World Bank and the IMF concluded that there may have been five specific conditions where the EU was the sole promoter of the reforms, namely: the condition related to the regime forfaitaire and the external audit (CdC); discrete pieces of broader reforms promoted by the IMF and WB; the two trade related conditions; and the condition on statistics.

Overall, it can be stated that the MFA was aligned with the broad policy framework guiding EU-Tunisia relations, either complementing or strengthening parallel programmes and boosting the effectiveness of simultaneous international interventions. Notwithstanding the original intention of reducing as much as possible cross-conditionality with IMF/WB, the evaluation proved the choice of stressing further on already tackled conditions to be a successful one in view of its systemic effect. The finding is deemed relevant also for future interventions.

Question 6.1 The direct effects of the MFA-I stem from the conditionalities on SSN outlined in the Memorandum of Understanding. The analysis found that increases 33 in monthly disbursements of the existing social security system, the Programme national d'aide aux familles nécessiteuses (PNAFN), coupled with the expanding household coverage of this policy are likely to have compensated households for the reduction in energy subsidies on average. This finding is predicated on evidence of consumer fuel price increases, which have been relatively modest 34 . However, given the shortfalls in the targeting of PNAFN, there is no guarantee that vulnerable households have been receiving the support required. Indeed, until the social security database is operationalized and used to implement well-designed policies, the targeting is likely to remain fundamentally weak.

Question 6.2 The indirect effects of the MFA operation to Tunisia can be defined as those effects which stem, more broadly, from wider macroeconomic stability.

Unemployment, although lower than rates observed during the crisis, was still stubbornly high by the end of 2018, especially among women and recent graduates. Inflation has also been following an upwards trajectory since 2000, and spiked at 7.8% in June 2018. Unless addressed, this could have considerable implications for the affordability of key household commodities and living standards.

A counterfactual analysis was performed to formulate a view of what the social situation might have been in the absence of MFA operation, together with the IMF support. This analysis was based on inputs from various key stakeholders from Tunisian government departments and international institutions, such as IMF and the World Bank.

The analysis suggested that, if MFA was not provided but IMF support continued, the indirect effects on the social situation in Tunisia would have been limited. Having access to international markets, Tunisia would likely have been able to replace the missing funds through a variety of domestic and international sources. Changes to fiscal policy, including public sector pay and employment, would probably not have occurred under this scenario, and the public sector wage bill was expected to remain roughly unchanged. Impacts to the social situation would therefore have come through direct channels, namely the MoU conditions related to the social safety net.

If both MFA and IMF support were revoked, then this would have presented an exceptionally challenging situation for the Tunisian authorities, likely triggering a national crisis. Tunisia would probably have borrowed from bilateral donor countries such as Qatar and Saudi Arabia. The rising debt, cuts in public spending and destabilized economy would have had repercussions on the Tunisian labour market and threatened the living standards of local citizens.

Question 7.1 The external evaluation assessed the contribution of the EU’s MFA to the sustainability of Tunisia’s public debt, relying on a counterfactual analysis in which the possible paths of the debt burden indicators, (1) debt-to-GDP ratio, and (2) gross financing need-to-GDP ratio, are evaluated and compared across different scenarios. Specifically, the evaluation compared:

-what concretely happened (the baseline scenario);

-what would have happened without the MFA operation but with the IMF programme still in place (Alternative 1: no MFA scenario); and

-what would have happened without either the MFA operation or the IMF programme (Alternative 2: no MFA and no IMF scenario).

The analysis highlighted that, if Tunisia had not received MFA funding but retained access to the IMF’s rescue programme (Alternative 1), debt sustainability is unlikely to have been adversely affected because there would not have been large changes in many of the variables which affect debt dynamics and sustainability. The most likely outcome would have involved the government replacing the MFA funds by borrowing from international financial markets, leading to an increase in debt due to the higher cost of borrowing from financial markets compared to the MFA’s concessional rates. That said, because the MFA was relatively small in the absolute amount, this increase in debt would not have been large enough to drastically change the evolution of the debt burden indicators.

The Alternative 2, under which Tunisia receives neither the EU’s MFA nor IMF funding, had the potential for more serious consequences for debt sustainability. The IMF’s involvement in providing financial assistance to countries tends to attract other donors. A likely outcome, therefore, of Tunisia not having access to the MFA and IMF programmes is that other multilateral and bilateral donors would not have intervened, with the result being increased risks to Tunisia’s debt sustainability and its economy more generally. Many of these donors tend to rely on an IMF programme being in place to ascertain that a recipient country is undertaking the necessary reforms.

Comparing the course of events under the three scenarios, it is clear that risk to debt sustainability is highest in the absence of both MFA and IMF assistance. In pure financial terms, the analysis does suggest that the MFA made a positive contribution to Tunisia's debt sustainability. Moreover, the provision of EU MFA is likely to have also helped spur additional support from other sources, over and above the financial contribution of the MFA itself.

6.Conclusions

In May 2014, the European Parliament and Council adopted a first Macro-Financial Assistance (MFA) operation of EUR 300 million to support Tunisia. The EU agreed on this operation to support the country in addressing a growing balance-of-payment crisis, which was triggered by the economic and political transition that followed on the 2011 Arab spring revolution. The operation was disbursed in full, in three tranches over the period May 2015 and July 2017.

This Staff Working Document is based on an external ex-post evaluation by ICF, in collaboration with Cambridge Econometrics 35 , which covered the relevance, efficiency, effectiveness, coherence, and EU value added of this MFA. It also explored the social impact of MFA and its effect on Tunisia’s public debt sustainability. Limitations encountered relate predominantly to data coverage in some areas (social indicators), the changing economic environment over the extended period during which the MFA-I was implemented, and the fact that MFA-I was implemented in parallel with other EU and international support programmes (making it somewhat difficult to disentangle the impact of the MFA operation specifically). Nevertheless, these limitations do not put into question the overall reliability of the evaluation analysis as they were mitigated by the wide range of sources, the use of different evaluation techniques, alternative scenarios and multiple rounds of feedback.

The evaluation found that the MFA-I was relevant in terms of its objectives, form (loans), timing (notwithstanding delays in the third disbursement) and design. The size of the envelope (300 million EUR) was deemed meaningful, but not critical, especially if compared to the budget assistance provided by the World Bank and IMF over the same period.

MFA policy conditionality covered the most relevant reform challenges in Tunisia, namely: (i) taxation, (ii) public finance management, (iii) social safety net, (iv) financial sector, (v) public statistical system and (vi) trade. Particularly satisfactory proved to be the introduction of a simplified income tax declaration system (régime forfaitaire), as well as the progress achieved in the area of public finance management, where MFA policy conditionality supported the adoption of an Organic Law of Cour des Comptes, strengthening the external audit of public accounts and ensuring the financial independence of the Court. Some difficulties have been encountered in the field of social safety net (SSN) and trade reforms. SSN reform has been technically complex and logistically challenging. It tackled a long-lasting problem that required a considerable in-house expertise at the Ministry of Social Affairs, close coordination across donors on the ground, and the successful completion of a number of sequential tasks with some conditioning the progress of others. Similarly, the trade reform relating to advancing with the process of converting the existing system of industrial compulsory standards (normes homologuées) into a system aligned with that of the EU also proved ambitious, given, inter alia, its technical complexity and the reduced capacity of the Tunisian institutions involved in its implementation.

These findings highlight the need for designing reforms in a way that secures achievable and sustainable results, taking into account the challenges faced by the still young Tunisian institutions in terms of administrative capacity and the context of political instability that can delay the implementation of reforms. The European Commission managed to mitigate part of these risks affecting the efficiency of the operation. It designed reform actions that were ambitious, yet achievable and realistic within a two-year implementation period, fostering a constant dialogue with the concerned parties and adjusting to contextual changes.

The MFA was deemed coherent with the broad policy framework guiding the EU-Tunisia relations. There is a high degree of consistency in a number of reform areas for budget support programmes and MFA conditionality. There is also a high level of coherence and continuity between this and the subsequent MFA operation. Finally, as far as external coherence is concerned, the MFA not only contributed to ‘burden sharing’ with the International Monetary Fund (IMF) and other donors in financial terms, but also reinforced reforms promoted by the IMF and World Bank (WB) through the use of cross conditionality and complementary conditions. Notwithstanding the original intention of reducing cross-conditionality with the IMF/WB, the evaluation proved the choice of stressing key reform areas, even if already tackled elsewhere, as a successful one in view of their systemic effects. Similar considerations are encouraged for possible future interventions.

The evaluation found that MFA-I was effective in helping to improve Tunisia’s balance-of-payments situation, as well as supporting fiscal consolidation and structural reforms. The MFA covered around 11.3% of the residual financing gap for the whole period 2015-2016 and increased confidence in the Tunisian economy (by decreasing the market-based financing cost, as well as by helping to stabilise the national currency).

Apart from contributing to the sustained mobilisation of local authorities around specific (sometimes new, as was the case for the regime forfaitaire) reform areas, the EU’s added-value primarily concerned the financial benefits granted to the Tunisian economy, with the highly concessional terms of the MFA loans allowing for fiscal savings and a more gradual adjustment of the primary public deficit (thereby avoiding a more disruptive budgetary adjustment path). The financial added-value of MFA operations also derives from the fact that the EU could mobilise and coordinate a wider amount of resources, as compared to any other individual donor country.

A counterfactual analysis of what the social situation might have been in the absence of MFA suggested that, if MFA was not provided but IMF support continued, the broad effects on unemployment, public sector wage and fiscal policy in Tunisia would have been limited. On the contrary, MFA had more impact through direct channels, namely the specific MoU conditions relating to social safety net. The positive impacts registered also stem from the strong coordination and complementarity existing between MFA’s and IMF’s interventions. This finding highlights the need for further coordination with MFA and IMF conditionality on social impact issues, to take advantage of their positive systemic effects.

Finally, the evaluation found that the MFA-I had a positive effect on the sustainability of Tunisia’s public debt and enabled fiscal savings. The fiscal savings resulted from very favourable financial conditions of the MFA operation and helped to smooth Tunisia’s adjustment path and create fiscal space for reforms and sustained social spending. However, given its relatively limited size, in the absence of the MFA, Tunisia’s debt sustainability is unlikely to have been significantly adversely affected.

In conclusion, MFA-I was relevant to support Tunisia's economic recovery after the Jasmine Revolution in 2011 and the ensuing political transition, providing fiscal savings and financial benefits, as well as boosting private sector confidence. The MFA conditionality package was fully aligned with the related IMF programme and created a politically reinforcing effect that contributed to the mobilisation of local authorities around crucial reforms, even in areas not covered by other international donor programmes.

Annex 1: Procedural information

A1.1 Organisation, design and timing

The ex-post evaluation assessed the EUR 300 million MFA operation in Tunisia, which was approved in 2014 and implemented between 2015-2017. The assessment was in line with article 34(1) of the Financial Regulation 36 and the relevant MFA Decision 37 , which required the European Commission to submit an ex-post evaluation report to the European Parliament and the Council. The objective of the evaluation was to draw lessons with respect to the EU’s financial assistance, in particular the design and implementation of the programme and the way it contributed to achieving macroeconomic stabilisation and fostering structural reforms. Apart from identifying areas of improvement for similar on-going or future possible interventions, the evaluation also aimed at ensuring better transparency and accountability of the Commission’s activities.

The evaluation looked at various aspects of this particular EU intervention (relevance, effectiveness, efficiency, EU added-value, coherence with other EU policies towards Tunisia, social impact and the impact on the sustainability of Tunisia’s public debt). In order to ensure validity, the analysis and conclusions are based on the evidence obtained using several evaluation methods (documentary review, macroeconomic data analysis, targeted stakeholder interviews, case studies, focus groups, qualitative counterfactual analysis, a Delphi survey and a Social Impact analysis).

The lead DG to carry out and manage this evaluation has been the Directorate General for Economic and Financial Affairs (DG ECFIN). DG ECFIN chaired the ISG that was set up to manage the evaluation. Apart from DG ECFIN, the ISG comprised of representatives of other Commission services (the Secretariat General and the Directorate-General for Neighbourhood and Enlargement Negotiations) and the EEAS.

The indicative roadmap for the ex post evaluation of MFA for Tunisia was published in June 2018 38 . In the context of the framework contract for the provision of evaluation services related to MFA programmes, on the 4th of March 2016 the Commission awarded the specific contract to undertake the external evaluation to Consortium ICF (ICF Consulting Services Ltd. and Cambridge Econometrics Ltd.).

A kick-off meeting, where the ISG and the external contractor discussed the deliverables and the evaluation methods, took place in September 2018. This was followed by meetings on the inception and interim reports in, respectively, December 2018 and March 2019. The draft final report was submitted in July 2019 with updates provided for a final version approved in October 2019. In addition to meetings, ISG members were continuously informed and consulted (via email and by phone) during the evaluation. The work of the external contractor was complemented by internal analysis from Commission services.

Annex 2: Stakeholder consultation strategy, including summary of results

The stakeholder consultation strategy was developed with the overall objective to collect factual information and opinions on various aspects of the MFA operation in Tunisia, structured around the main evaluation criteria. It was designed to capture as much relevant information as possible with regard to the MFA operation in addition to the information collected through key documentation review, interviews with targeted stakeholders and data analysis.

The consultation was developed in line with the Better Regulation Guidelines on stakeholder consultation 39 and it focused on (i) extracting recollections from the period in which the operation was designed and implemented, and also (ii) on collecting views on the period after the MFA had ended, to assess its impact and sustainability.

Consultation tools were tailored to each targeted stakeholder group to collect information most appropriate to their knowledge and associated with different aspects of the MFA operation in Tunisia. As for primary data collection activities, these principally included: (1) a Delphi survey; (2) depth interviews with key (EU and Tunisian) stakeholders; (3) (internal and external) workshops; and (4) a focus group with civil society organisations and other non-governmental stakeholders (based in Tunisia).

A2.1 Mapping of stakeholder groups

Consultation was targeted to specialists – either people who have been closely involved in the development and/or the implementation of the MFA-I operation or people with expert knowledge in the areas related to the objectives of the MFA-I operation (i.e. macroeconomic and fiscal policy, structural reforms in the areas of public finance management, social policy, trade and statistics.

In turn, three core stakeholder groups were identified as relevant: (1) the EU side; (2) the Tunisian side; and (3) the International Financial Institutions (IFIs) / donor community.

In total, close to 100 people were consulted through a Delphi survey, depth interviews and a focus group.

   Figure A2.1 Relevant stakeholder groups 

A2.2 Methods and tools for engaging with stakeholders

A2.2.1 Delphi Survey

The Delphi Survey seek to establish views on the role and contribution of the MFA in achieving macroeconomic stability, easing external financing constraints and alleviating Tunisia’s balance of payments and budgetary needs. Responses gathered from the Delphi panel fed into the analysis of the relevance, effectiveness, and efficiency of the MFA operation as well as the debt sustainability and social impact analyses.

The Delphi Survey was carried out on the basis of structured questionnaires. Specifically, participants were asked to elaborate on the plausible scenario(s) had (i) the EU MFA, or (ii) the joint MFA-IMF action not been implemented. As mentioned above, the Survey also covered aspects relating to the role of the MFA operation in promoting structural reforms and their social impacts.

A2.2.2 In-depth interviews

The evaluator conducted intensive (individual) interviews with key stakeholders. An important aim of these targeted interviews was to explore selected stakeholders’ perspectives on the MFA operation in Tunisia. Specifically, detailed information was gathered on the operation, including (but not restricted to): its design and implementation, perceived / realised impact(s), notably its effectiveness in driving reform and / or macroeconomic stability, its ‘added-value’ when compared to other/similar support, and potential future improvements.

The objective was to interview various officials and representatives of the European Commission, Tunisian national authorities (and other public entities involved in the implementation of the operation) and the wider donor community. Additional interviews were conducted with representatives of the business community, the research sphere and the general population (notably in the recipient country). Officials/representatives who were still or no longer in employment with the relevant institutions were targeted, to ensure the consultation of stakeholders still closely involved in the MFA operation.

A2.2.3 Focus group

A half-day focus group discussion with a wider group of non-government / private sector representatives, based in Tunisia, was organised.

The aim of the focus group was to capture additional insights, perspectives, and thoughts in relation to the MFA operation in Tunisia. The discussion was steered to specific topics, including (but not restricted to):

A2.2.4 Stakeholder validation workshop

A half-day validation workshop was organised, with relevant EC officials, Member State representatives, Tunisian authorities and selected IFIs and donors. Video-conferencing facilities were provided to allow Tunisian stakeholders (and other stakeholders if necessary) to attend virtually.

A primary objective of the workshop was to bring together selected stakeholders to critically assess and validate emerging findings, initial conclusions and recommendations. The workshop took place at the Draft Final Report stage. Workshop materials were shared with participants prior to the workshop to allow them to familiarise themselves with the content, thereby enabling a more informed / interactive discussion on the day.

Annex 3: Methods and data sources

An evaluation matrix was developed to guide the choice and design of specific research methods, as well as to provide a framework for subsequent data analysis and interpretation. The table below provides a high-level overview of the data collection methods and analytical techniques that were used to address each evaluation criteria. Further details are provided in the sub-sections that follow.

Table A3.1 Overview of the methods and techniques used for the evaluation

●●● a very important method for addressing the evaluation criterion

●● an important method for addressing the evaluation criterion

● a complementary method

A3.1 Documentary review

Table A3.2 below provides an overview of the sources and types of documentary evidence assembled and reviewed. It also provides an assessment of the usefulness of each of the different types of documentary evidence for the evaluation.

Table A3.2 Documentary sources of evidence for the evaluation

●●● very useful ●● somewhat useful

A3.2 Macroeconomic data analysis

Table A3.3 Key Macroeconomic Indicators and Data Sources

A3.3 Case studies

Two in-depth case studies on MFA promoted reforms in the following areas were developed: (1) social safety net reforms and (2) tax policy (and more specifically “regime forfaitaire 40 ”).

The case studies addressed the following aspects:

ŸThe rationale behind the selection of specific MFA conditions in the above areas as well as the relevance and added-value of the MFA conditionality;

ŸThe significance of MFA conditionality in the context of the overall need for reform in a particular thematic area/ sector;

ŸHow the MFA conditions were implemented and whether the authorities encountered any obstacles in implementing these conditions (e.g. lack of capacity, political or public resistance to change etc.);

ŸThe role and contribution of the MFA in promoting reforms including identification of the key ‘causal links’

ŸShort, mid and long-term benefits of the MFA conditions.

In addition, the case studies attempted also to draw on some lessons learnt from the design and implementation of similar reforms in the past by two donors, the World Bank and the IMF.

The case studies were mainly based on the desk research, targeted stakeholder interviews and interactions of the local economic experts.

A3.4 Focus Group with the civil society

Table A3.4 Focus group participants

A3.5 Delphi survey

The Delphi survey sought to establish views on the role and contribution of the MFA in achieving macroeconomic stability, easing external financing constraints and alleviating Tunisia’s balance of payments and budgetary needs. In particular, participants were asked to elaborate on plausible scenario would MFA-I not have been available, and the potential implications. The survey also covered aspects related to the role of the MFA-I operation in promoting structural reforms.

The survey panel included 82 representatives of the following groups / institutions:

ŸBusiness representatives and financial / macroeconomic analysts from the private sector (e.g. research departments of commercial banks and credit rating agencies); and

ŸResearchers from think tanks, experienced commentators of Tunisian economic policies (i.e. specialized press), independent fiscal policy experts, and academic experts.

During the first round, 32 respondents provided the valid feedback which resulted in 40% response rate. The first round of survey results yielded fairly consistent views among respondents, thought there was still no consensus on a few aspects.

Therefore, a second round was conducted among experts who responded to the first survey. Among 32 experts who received the second-round questionnaire, 23 responded resulting in 72% response rate.

Table A3.5 Details of the Delphi Panel that were invited and responded

Source: ICF

Annex 4: List of key MFA-I documents reviewed

Association Agreement between the European Union and the European Atomic Energy Community and their Member States, of the one part, and Tunisia, of the other part, Official Journal of the European Union.

Compliance Statements of the Government of Tunisia, 2014.

Decision No 534/2014/EU of the European Parliament and of the Council of 15 May 2014 providing macro-financial assistance to the Republic of Tunisia. 

European Commission (2015), Report on mission to Tunisia: review of compliance with conditions for 2nd tranche of МFА and possible further МFА (Tunis, 31 August - 4 September 2015)

European Commission (2016), Report on mission to Tunis (8-11 November 2016)

European Commission (2017), Report on mission to Tunis (18-21 April 2017): Review of compliance with conditions for 3rd tranche of MFA-I; and discussions on launching of MFA-II

European Commission DG ECFIN (2011a). Proposal for a Decision of the European Parliament and of the Council providing macro-financial assistance to Tunisia.

European Commission DG ECFIN (2011b). Ex-ante evaluation statement on further macro-financial assistance to Tunisia, final.

European Commission DG ECFIN (2013a), Ex-ante evaluation statement on EU macro-financial assistance to the Republic of Tunisia.

European Commission DG ECFIN (2013b), EU-Tunisia Macroeconomic dialogue (Brussels, 12 February 2013).

European Commission DG ECFIN (2013c), Proposal for MFA to the Republic of Tunisia for 2014-2015.

European Commission DG ECFIN (2013d), Proposal for the Decision of the EP and the Council. Providing Macro-Financial Assistance to Tunisia.

European Commission DG ECFIN (2014). Report on mission to Tunisia: Memorandum of Understanding negotiations for Macro-Financial Assistance to Tunisia.

European Commission DG ECFIN (2014a), Memorandum of Understanding for the MFA-I.

European Commission DG ECFIN (2014b), Report on Mission to Tunisia. Macroeconomic Dialogue and launching of MoU negotiations for Macro-Financial Assistance.

European Commission DG ECFIN (2015a), Macro-financial assistance to the Republic of Tunisia: disbursement of the first tranche (2015).

European Commission DG ECFIN (2015b), Macro-financial assistance to the Republic of Tunisia: disbursement of the second tranche (2015). 

European Commission DG ECFIN (2015b). Macro-Financial Assistance to Tunisia Disbursement of the First Tranche, Information Note to the European Parliament and the Economic and Financial Committee.

European Commission DG ECFIN (2016), Nouvelle opération d'assistance macro-financière (AMF-II) de l'UE et décaissement de la dernière tranche de l'opération en cours (AMF-I).

European Commission DG ECFIN (2017), First macro-financial assistance to the Republic of Tunisia: disbursement of third tranche.

European Commission DG ECFIN (2017a). Macro-Financial Assistance to Tunisia Disbursement of the Second Tranche, Information Note to the European Parliament and the Council. Ref. Ares(2015).

European Court of Auditors (2017), “Special Report: EU Assistance to Tunisia” No 03/2017, Publications Office of the European Union.

ICF (2019) Ex-post evaluation of the first Macro-Financial Assistance operation in Tunisia over the period 2014 – 2017.

Macro-Financial Assistance for Tunisia Grant Agreement, 17.12.2014.

Macro-Financial Assistance for Tunisia Loan Facility Agreement, 11.12.2014.

Memorandum of Understanding between the European Union as Donor and Tunisia as Beneficiary and the National Bank of Tunisia as Beneficiary's Financial Agent, 2014.

Annex 5: Timeline of the Tunisia MFA-I operation

Table A5.1

(1) Ex-post evaluation of the first Macro-Financial Assistance operation in Tunisia over the period 2014-2017, available at: https://ec.europa.eu/info/evaluation-reports-economic-and-financial-affairs-policies-and-spending-activities_en

(2) In terms of the use of funds.

(3) Decision No 534/2014/EU of the European Parliament and of the Council of 15 May 2014 providing macro-financial assistance to the Republic of Tunisia https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1476964844762&uri=CELEX:32014D0534

(4) https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32018R1046

(5) https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/1812-Ex-post-evaluation-of- macro-financial-assistance-to-Tunisia

(6) https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1476964783648&uri=CELEX:52013PC0860

(7) https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1476964844762&uri=CELEX:32014D0534

(8) https://ec.europa.eu/economy_finance/eu_borrower/mou/tunisia_mfa_mou_signed_en.pdf

(9) https://ec.europa.eu/info/business-economy-euro/economic-and-fiscal-policy-coordination/international-economic-relations/macro-financial-assistance-mfa-non-eu-partner-countries_en

(10) For the detailed list of policy conditions, please refer to the Memorandum of Understanding, available at: https://ec.europa.eu/economy_finance/eu_borrower/mou/tunisia_mfa_mou_signed_en.pdf

(11)  DG ECFIN, 2017. Report on mission to Tunis (18-21 April 2017).

(12) https://www.imf.org/en/News/Articles/2015/09/14/01/49/pr16238  

(13)      European Commission, May 2015. Better Regulation Guidelines. Available at: http://ec.europa.eu/smart-regulation/guidelines/toc_guide_en.htm .

(14)      Available at: http://ec.europa.eu/dgs/economy_finance/procurement_grants/calls_for_tender/2015/015d/annex4-methodological_orientations_en.pdf .

(15) For a brief presentation thereof, please refer to section 4.2 of the external evaluation report, available at: https://ec.europa.eu/info/evaluation-reports-economic-and-financial-affairs-policies-and-spending-activities_en

(16) Further information is available in chapter 4 of the external evaluation report and in Annex 3 of the present Staff Working Document.

(17)      IMF Staff Guidance Note for Debt Sustainability Analysis in Market-Access Countries, available at: https://www.imf.org/external/np/pp/eng/2013/050913.pdf .

(18)      See the list of completed interviews in Annex III of the external evaluation report.

(19) See the summary note from focus group discussion in Annex VIII of the external evaluation report.

(20)      Detailed results are available in Annex IX of the external evaluation report.

(21)      For more detail, please refer to chapter 5 of the external evaluation report, available at: https://ec.europa.eu/info/evaluation-reports-economic-and-financial-affairs-policies-and-spending-activities_en

(22) https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1476964844762&uri=CELEX:32014D0534

(23) https://ec.europa.eu/economy_finance/eu_borrower/mou/tunisia_mfa_mou_signed_en.pdf

(24)      Gap remaining after the contributions from the IMF and the World Bank.

(25) https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52015SC0148&from=it

(26) https://www.imf.org/external/pubs/ft/scr/2014/cr14362.pdf

(27) Please refer to section 3 of the present Staff Working Document, on the economic situation in Tunisia, during and immediately after the implementation of the MFA operation in question.

(28) The simplified income tax declaration system

(29) In addition, to be eligible for the budget support programmes, the following eligibility criteria need to be met: (i) a well-defined national or sectorial development or reform policy and strategy; (ii) a stable macroeconomic framework; (iii) good public financial management or a credible and relevant programme to improve it; (iii) transparency and oversight of the budget (budget information must be made publicly available).

(30) The plan of setting up a new programme was abandoned meanwhile.

(31) It is addressed under MFA-II, under labour market.

(32) The simplified income tax declaration system

(33) Monthly disbursements increased from 56.7 dinars to 115 dinars in 2014, and again to 150 dinars in 2015.

(34) Average consumer price inflation for electricity, gas and other fuels was 2.3% over 2010-2018 (1.9% over 2015-2018).

(35) Ex-post evaluation of the first Macro-Financial Assistance operation in Tunisia over the period 2014-2017, available at: https://ec.europa.eu/info/evaluation-reports-economic-and-financial-affairs-policies-and-spending-activities_en

(36) https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32018R1046

(37) https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1476964844762&uri=CELEX:32014D0534

(38) https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/1812-Ex-post-evaluation-of- macro-financial-assistance-to-Tunisia

(39) https://ec.europa.eu/info/law/law-making-process/planning-and-proposing-law/better-regulation-why-and-how/better-regulation-guidelines-and-toolbox_en

(40)  The simplified income tax declaration system