Contents

Glossary    

1.    Introduction    

Purpose    

Scope    

2.    Background to the intervention    

Description of the intervention and its objectives    

3.    Implementation / State of Play    

3.1. Modality and implementation of the MFA programme    

3.2. Economic developments during the lifecycle of MFA (Q4 2011- Q2 2016)    

3.3. Economic developments after the implementation of MFA    

3.4 Continued support from the EU after the implementation of MFA    

4.    Method    

Documentary review    

Macroeconomic data analysis (including Debt Sustainability Analysis)    

Interviews with key informants/ stakeholders    

Focus Group with development partners    

Qualitative counterfactual analysis    

Social Impact Analysis (SIA)    

Limitations and Methodological Issues    

5.    Analysis and answers to the evaluation questions    

5.1    Relevance    

5.2    Effectiveness of MFA    

5.3    Efficiency    

5.4    EU Value Added    

5.5    Coherence    

5.6    Social Impact Analysis    

5.7    Debt Sustainability Analysis    

6.    Conclusions    

Annex 1: Procedural information    

Annex 2: References    

Annex 3: Discussion of validity and reliability of the evaluation findings    

Annex 4: Comparison of MFA, IMF and World Bank conditionalities    

Annex 5: Evaluation Framework    

Annex 6: Synopsis Report    

Annex 7: Summary of findings from stakeholder consultation activities    

Glossary

ADB        Asian Development Bank

CPI        Consumer price index

CU        Customs Union

DCI        Development Cooperation Instrument

DFID        Department for International Development (UK)

DG DEVCO    Directorate-General for International Cooperation and Development

DG ECFIN    Directorate-General for Economic and Financial Affairs

DG ECHO    Directorate General for European Civil Protection and Humanitarian Aid Operations

DIPECHO    Disaster-Preparedness ECHO

DPO        Development Policy Operations

DSA        Debt Sustainability Analysis

EACU        Eurasian Customs Union

EAEU        Eurasian Economic Union

EBRD        European Bank for Reconstruction and Development

ECF        Extended Credit Facility

EDB        Eurasian Development Bank

EEAS        European External Action Service

EEU        Eurasian Economic Union

EIB        European Investment Bank

EIDHR    European Initiative for Democracy and Human Rights

EIU        Economist Intelligence Unit

EU        European Union

EUR        Euro

EURASEC    Eurasian Economic Community

FATF        Financial Action Task Force

FDI        Foreign direct investment

FMIS        Financial Management Information System

GDP        Gross Domestic Product

GIZ        Deutsche Gesellschaft für Internationale Zusammenarbeit

GSP+        Generalised Scheme of Preferences

GTAC        Governance Technical Assistance Credit

IBC        International Business Council

IcSP        Instrument contributing to Stability and Peace

IDA        International Development Association

IDB        Islamic Development Bank

IFCA        Investment Facility for Central Asia

IMF        International Monetary Fund

INTOSAI    International Organization of Supreme Audit Institutions

KGS        Kyrgyz national currency (Som)

KZT        Kazakhstani tenge

LGBTI        Lesbian, gay, bisexual, transgender, and intersex

MDBF        Medium Term Budget Framework

MDTF        Multi-Donor Trust Fund

MFA        Macro Financial Assistance

MTDS        Medium Term Development Strategy

NBKR        National Bank of the Kyrgyz Republic

NSI        Nuclear Safety Instrument

NSSD        National Strategy for Sustainable Development

OA        Operational Assessment

OSCE        Organization for Security and Cooperation in Europe

PCA        Partnership and Cooperation Agreement

PEFA        Public Expenditure and Financial Accountability

PFM        Public Finance Management

PIP        Public Investment Programme

RCF        Rapid Credit Facility

RIP        Regional Indicative Programme

RUR        Russian national currency (Ruble)

SDPK        Social Democratic Party of Kyrgyzstan

SDR        Special Drawing Rights

SECO        State Secretariat for Economic Affairs

SIA        Social Impact Analysis

SIDA        Swedish International Development Cooperation Agency

SRC        Sector Reform Contracts

TAF        Technical Assistance Facility

US        United States

USAID    United States Agency for International Development

USD        United States Dollars

VAT        Value Added Tax

WB        World Bank

WTO        World Trade Organization

1.Introduction 

Purpose

This Staff Working Document presents an assessment of the Macro-Financial Assistance (MFA) operation provided by the European Union (EU) to the Kyrgyz Republic over the period 2015-2016. It largely draws on an independent, ex-post evaluation conducted by an external contractor and extensive consultations with targeted stakeholders in the Kyrgyz Republic as well as Commission specialists on this field. The aim is to assess the results and efficiency of the completed MFA and to draw lessons for future decision-making and identify areas of improvement for similar on-going or future possible interventions.

Scope

Macro-financial assistance , or MFA, is an EU financial instrument designed to address exceptional external financing needs of countries that are geographically, economically and politically close to the EU. It takes the form of medium/long-term loans or grants, or a combination of these, and is only available to countries benefiting from a disbursing International Monetary Fund (IMF) programme. The assistance is released in instalments strictly dependent on the successful implementation of reform measures aimed at putting the beneficiary country’s economy back on a long-term sustainable path. In this way, MFA complements regular EU cooperation assistance and contributes to the wider goal of preserving stability and promoting prosperity beyond the EU.

In October 2013, the European Parliament and the Council approved a EUR 30 million MFA operation to the Kyrgyz Republic. The operation was disbursed in full, in two instalments, over the period June 2015 and April 2016.

In accordance with the Financial Regulation (article 30 point 4), MFA operations in third countries are subject to an ex-post evaluation. In turn, the MFA Decision on Kyrgyzstan 1 stipulates that the European Commission is required to "[...] submit to the European Parliament and to the Council an ex-post evaluation report, assessing the results and efficiency of the completed Union’s macro-financial assistance and the extent to which it has contributed to the aims of the assistance."

The ex-post evaluation was undertaken by an external contractor and ran from March 2018 until April 2019. The ex-post evaluation assessed the implementation of the MFA programme to Kyrgyzstan that was initiated in 2011 and disbursed in full in 2015 and 2016. The external report has assisted in the preparation of the Staff Working Document (SWD). Because of the nature of the assistance, the SWD evaluation looks at both the immediate and medium-term impact of the MFA programme on the country's economy and state of play with structural reforms.

The objectives of this ex-post evaluation are two-fold:

·to analyse the impact of the MFA on the economy of Kyrgyzstan, and in particular, on the sustainability of its external position; and

·to assess the added value of the European Union’s (EU) intervention. In general, the evaluation seeks to draw lessons with respect to the EU's financial assistance, i.e.

owhether the ex-ante considerations, determining the design and terms of the operation were appropriate, taking due account of the economic, political and institutional context; and

oand whether the outcome of the programme met the objectives.

In turn, these main areas of analysis have been assessed conform the five Better Regulation evaluation criteria: relevance, effectiveness, efficiency, EU added-value, and coherence with other EU policies. In addition, the evaluation assesses the impact of the MFA on the social sector and the public debt sustainability in the Kyrgyz Republic. This is further specified in the Evaluation Roadmap 2 .

2.Background to the intervention

Description of the intervention and its objectives

The Kyrgyz Republic was strongly hit by the global economic crisis in 2008-2009. Output growth declined from a buoyant 7.6% in 2008 to 2.9% and -0.5% in 2009 and 2010 respectively. This was largely due to external shocks, including a fall in remittances from migrant workers, lower demand for exports and shrinking foreign investments 3 . In addition, mounting political instability further plunged Kyrgyzstan into an economic slowdown. In April 2010, the regime of President Bakiyev was overthrown under allegations of corruption and lack of democracy. In a vacuum of power, inter-ethnic violence erupted in June 2010 in the South of the country, leaving over 450 people dead and hundreds of thousands displaced. In spite of these tragic events however, the new, interim government secured an approval vote for democratic reforms in a constitutional referendum held on 27 June 2010. Subsequently, free parliamentary elections were held in October 2010 establishing the first parliamentary democracy in Central Asia.

The 2010 revolutions became known as the Second Kyrgyz Revolution, and had adverse effects on economic activity, confidence within the private sector, the government's fiscal position, and other key economic indicators. 4 Figures 1 to 4 overleaf illustrate. On the fiscal side, public debt to GDP ratio grew to 60% in 2010, while the fiscal deficit rose from 3.5% of GDP in 2009 to 5.9% of GDP in 2010, reflecting the budgetary cost of the crisis-related measures and the negative effect of weaker economic activity on tax revenues. On the external side, Kyrgyzstan’s trade balance had worsened due to a disruption in trade flows, primarily driven by border closures (notably with Kazakhstan), and an alarming security situation that markedly affected tourism and partly disrupted agricultural production. Indeed, the current account moved from a temporary surplus in 2009 to a deficit of 2.2% of GDP in 2010. On the monetary side, the central bank of the Kyrgyz Republic had tightened monetary policy substantially since the mid-2010, responding to an acceleration in inflation due to a sharp increase in the global prices of fuel and food.

In response to these events and their economic implications, a donor conference “Emergency assistance to the Kyrgyz Republic: Reconciliation and Recovery” was organised in Bishkek in July 2010. At this conference, donors committed to providing USD 1.1 billion in emergency support by the end of 2011. The EU was among the major donors, pledging about EUR 118 million for support in a number of critical areas, through various instruments including:

The IMF extended help to the Kyrgyz Republic in 2010 with a three-month Rapid Credit Facility in the amount of around USD 33.7 million, implemented in September-December 2010. As a follow-up, in June 2011, the IMF agreed with the Kyrgyz Republic authorities on a follow-up arrangement under the Extended Credit Facility for USD 106 million, in support of a comprehensive economic adjustment and reform programme for the period mid-2011 to mid-2014. Other major donors at the conference included the World Bank (pledged USD 200 million), and the Asian Development Bank (ADB) (pledged USD 100 million).

Shortly after the donor conference, in October 2010, the President and the Minister of Finance of the Kyrgyz Republic formally requested the EU MFA to complement the financing from the IMF. In this context, the Commission assessed the macroeconomic situation and financing needs of the Kyrgyz Republic and concluded that the 2010 political events and the related social and reconstruction expenditure had left important external and fiscal financing needs for the period 2010-12. Furthermore, given the EU’s strong political support for the Kyrgyz Republic’s incipient parliamentary democracy, the Commission concluded that the political and economic pre-conditions for a MFA operation of a moderate amount are satisfied.

The Kyrgyz Republic is out of the normal geographical scope of MFA, yet in exceptional and duly justified circumstances the possibility of approving operations outside that area is possible. In this case, the Commission concluded that this MFA operation was justified by “the strength of the political and economic reform momentum in the country and by its position in a region of strategic economic and political importance for the EU. It would provide a political signal of strong EU support to democratic reforms in Central Asia, consistent with the EU policy towards the region [...]” 5

Against this background, and in order to supplement the resources made available by the IMF, the European Commission, on 20 December 2011, adopted a proposal to provide up to EUR 30 million of MFA to the Kyrgyz Republic, in the form of a grant of EUR 15 million and a medium-term loan of EUR 15 million. This was accompanied by a detailed assessment of the Kyrgyz Republic's needs (See footnote 5). Following the entry into force of the Lisbon Treaty, a procedural disagreement between the Council and European Parliament on the decision-making process of MFA operations delayed the adoption of the decision  by the co-legislators to 22 October 2013.

The objectives of the EU’s MFA was to:

·Contribute to covering the external financing needs of the Kyrgyz Republic and to alleviating budgetary financing needs;

·Support the fiscal consolidation effort and external stabilisation in the context of the IMF programme;

·Support structural reform efforts aimed at improving the overall macroeconomic management (e.g. targeting the transparency and efficiency of public finance management and the financial stability) and at improving conditions for sustainable growth.

The intervention logic for the EU's MFA to Kyrgyzstan is detailed in Figure 5.

Figure 5: MFA Intervention Logic

3.Implementation / State of Play 

3.1. Modality and implementation of the MFA programme

To achieve the objectives detailed in Section 2, and as per the usual MFA procedure, disbursements under this operation were tied to the fulfilment of country-specific policy conditions, as laid down in the Memorandum of Understanding. The design of the conditionality for this programme was subject to negotiations with the Kyrgyz authorities, and additionally took into account discussions with the EU Delegation, the European External Action Service (EEAS) and other DGs, as well as the IMF and the World Bank. In addition, the Commission (DG ECFIN) commissioned an external Operational Assessment in order to ensure that the public finance management (PFM) system provided sufficient safeguards for the MFA programme.

The operation covered seven specific conditions under four reform areas: (i) Public Finance Management (PFM), (ii) Financial sector, (iii) Trade policy and Investment, and (iv) Business environment. Table 1 below lists the exact structural reforms attached to the second disbursement. 6 As for most of the MFA operations, policy measures were not attached to the first instalment, which was conditional only on a satisfactory track record of the IMF programme and the fulfilment of the so-called political pre-condition for MFA - the existence of effective democratic mechanisms in place and to respect the rule of law and human rights.

Table 1: Overview of MFA to Kyrgyz Republic – Size, form, terms and attached reform conditions

This operation was characterised by unusual, long delays throughout the process. The Commission adopted the proposal in December 2011, yet the approval by the co-legislators was delayed until October 2013 as mentioned above. Subsequently, the signature of the MFA documents (i.e. the Memorandum of Understanding, the Loan Facility Agreement and the Grant Agreement) only took place in October 2014 due to legal issues involving the National Bank of the Kyrgyz Republic (NBKR). The MFA documents were subsequently ratified by the Kyrgyz Parliament in February 2015, yet by this time, the 2011 IMF programme had expired (in July 2014). A new three-year Extended Credit Facility (ECF) was agreed with the IMF in April 2015 and from then onwards, disbursements followed more swiftly.

The operation was disbursed in full, in two instalments, over the period June 2015 - April 2016:

ŸThe first instalment: a EUR 10 million grant disbursed in June 2015 and a EUR 5 million loan disbursed in October 2015;

ŸThe second instalment: a EUR 5 million grant disbursed in February 2016 and a EUR 10 million loan disbursed in April 2016.

Both loan tranches were characterised by very long maturity (14 years and 15 years, respectively) and low interest rate (1.375% and 0.70%, respectively). See Table 1.

As MFA is an emergency instrument without legal base, its legislative process consists of an ad-hoc agreement on every single MFA operation between the European Parliament and the Council under the ordinary legislative procedure (TFEU Art. 212), which on average requires eight months, after which the MFA documents also necessitate sometimes-lengthy approval procedures in the beneficiary country. This operation, however, was characterised by an exceptionally long timetable between approval and disbursement, largely for reasons beyond the Commission’s control. The lifecycle of this MFA programme thus spanned an unusually long period from late 2011 to early 2016.

Figure 6 below summarizes the timeline for this MFA operation, from the events that motivated the Commission proposal (See Section 2) to the last disbursements under MFA and IMF.

Figure 6: Timeline for approval and disbursement of IMF and MFA support

Points of comparison

The points of comparison, against which the MFA operation of 2011-2016 is assessed, refer to the situation in the Kyrgyz Republic (i) prior to the intervention on the one hand (Section 2), and (ii) during and after the implementation of the MFA operation on the other hand (Sections 3.2 – 3.3).

The role of MFA in promoting macroeconomic stability, easing external financing constraints and alleviating Kyrgyzstan’s balance of payments and budgetary needs, as well as the effectiveness of the structural reforms attached to this MFA programme are analysed in Section 5.

3.2. Economic developments during the lifecycle of MFA (Q4 2011- Q2 2016) 7

Following the lingering effects of the global economic crisis of 2008-2009 and inter-ethnic violence of 2010, the Kyrgyz Republic GDP dropped by 0.5% in 2010. The next year was characterized by a rebound of 6.0% in economic activity, owing to a full resumption of trade and services flows, continued reconstruction works, and improved investor confidence. The external evaluation asserts that this renewed confidence in the Kyrgyz economy could have been due to, though not the sole result of, the IMF’s announcement of a three-year ECF arrangement (over the period June 2011 - July 2014) and the European Commission’s proposed MFA to Kyrgyzstan, signalling important structural reform actions in key areas. Nevertheless, 2011 was still characterized by a high fiscal deficit and a very high external debt (around half of its GDP), making the economy highly vulnerable to external shocks.

While the peaceful transfer of power in 2011 and the swift formation of a new government in 2012 laid the foundation for growth, low gold production nevertheless caused GDP to contract in 2012. The external current account deficit widened substantially, reflecting lower export volumes and a slight fall in gold prices. The trade deficit is estimated to have increased significantly, as higher purchases of gasoline and diesel fuel caused imports to rise, while gold exports fell 8 . Nonetheless, strong and positive growth was observed in other sectors of the economy, such that the non-gold GDP growth rate was estimated at a buoyant 6.3% for 2012.

In 2013, economic growth soared to almost 11.0%, and the current account deficit narrowed further. This was driven by a rebound in gold production 9 , lower prices for imported food and fuel, and a sustained improvement in the rest of the Kyrgyz economy. 10 This strong growth was however short lived. In 2014, GDP growth slowed to 4.0%, reflecting a sharp fall in gold production and other major supply-side constraints. At the same time, the external environment (notably the neighbouring economies of the Russian Federation and Kazakhstan) weakened, while strong local currency depreciation pushed inflation up. At the end of 2014, external public debt surged to an estimated 51.0% of GDP from 43.7% a year earlier, whilst the current account deficit is estimated to have risen to around 15% of GDP from 14.1% in 2013. 11  

In 2015, GDP growth slowed to 3.9%. Continued recession in the Russian Federation and a slowdown in neighbouring Kazakhstan weighed heavily on the economy. External public debt soared to 63.0% of GDP from 51.0% a year earlier, mainly reflecting local currency depreciation. Kyrgyzstan’s vulnerable balance of payments situation was reflected in the structurally large and sustained current account deficit, which widened to 17% in 2015. 12

The Kyrgyz economy further contracted in the first half of 2016, as a sharp decline in gold output (-35.6%) coincided with weak performance in the non-gold sector, in particular industry and service.

The Commission disbursed the first instalment of the MFA programme to the Kyrgyz Republic (comprising of EUR 10 million in grants and EUR 5 million in loans) in June and October 2015, respectively. The second and final instalment (comprising EUR 5 million in grants and EUR 10 million in loans) was disbursed in February and April 2016, respectively. The disbursements coincided with a time of sluggish economic growth in Kyrgyzstan as the Russian economic crisis began unfolding (See Figure 7).

Figure 7 below illustrates an overview of Kyrgyzstan’s economic situation, measured by GDP growth rate over the lifecycle of this MFA, including also specific events that help clarify the developments observed over these periods.

Figure 7: Trends in quarterly GDP growth rates observed for Kyrgyzstan, 2011(Q4)-2016(Q2)

Source: ICF External evaluation report

Furthermore, Figures 8 to 13 illustrate the economic developments over 2011-2017, as described above.

Figure 8. Decomposition of real GDP trend, 2011 - 2017, in % of GDP

Source: NSC

Figure 9. Current account balance and its main components, in USD billion

Source: NBKR

Figure 10. General government revenue & expenditure (left axis), and deficit (rights axis), in % of GDP

Source: IMF

Figure 11. Public debt, in % of GDP

Source: MoF

Figure 12. Deposits and loans by banks and micro-financial organizations, in %

Source: NBKR

Figure 13. Inflation rate (left axis) and official exchange rate (right axis)

Sources: NSC, NBKR

3.3. Economic developments after the implementation of MFA

Economic growth in Kyrgyzstan over 2016 and 2017 was close to the long-term average (2000 - 2015) GDP growth value of about 4.5%. This was supported by some recovery in the neighbouring economies (especially Russia and Kazakhstan) and increase in the inflow of remittances. The debt and fiscal situations have somewhat improved, and inflation remains below 5% per annum.

In 2018, the economy continued showing signs of recovery, benefiting from stable regional and external environment. GDP growth was estimated at 3.5 percent, whilst inflation averaged 1.5 percent for the year. Fiscal deficit stands moderately at 1.3 percent of GDP, gross official foreign exchange reserves at 4.0 months of imports and a stable banking sector point to the success of stabilization policies, carried out by the government and National Bank of the Kyrgyz Republic (NBKR). 13  

The latest assessment of the IMF (IMF, 2019) positively notes that, ‘’[…] While significant risks remain, debt and financial sector vulnerabilities have subsided. The authorities are committed to prudent macroeconomic policies and implementation of structural reforms, and see them as essential to promoting higher and more inclusive growth and to increase economic resiliency. Monetary policy remains appropriately focused on maintaining price stability.’’

Looking ahead, growth in Kyrgyzstan is expected to recover to around 4.0% in 2019, as per the latest forecasts of the IMF (World Economic Outlook Database, April 2019), the World Bank (Global Economic Prospects, June 2019 14 ) and the Asian Development Bank (Asian Development Bank Outlook, 2019 15 ). The medium-term growth prospects are shaped however by downside risks (i.e. slowdown in the region, especially in Kazakhstan and the Russian Federation, the country’s two main regional partners; lower remittance inflow; difficulties attracting foreign direct investments etc.). While debt sustainability has improved, the IMF assesses the country as facing moderate risk of debt distress because of continuing vulnerability involving currency stability and possible deterioration in the fiscal balance. For 2020-2024, the IMF predicts less buoyant activity, with GDP growth averaging around 3.0%. The latter value may be related to the gradual closure of Kumtor gold mine (amongst the largest contributors to the GDP of Kyrgyzstan), expected to start in 2022-2023.

3.4 Continued support from the EU after the implementation of MFA

Kyrgyzstan will continue to be an important partner for the EU. The EU operates in the Kyrgyz Republic on the basis of the Partnership and Cooperation Agreement which has been in force since 1999. In July 2019, the EU and Kyrgyzstan initialled a new Enhanced Partnership and Cooperation Agreement (EPCA). This is the second EPCA to be concluded with a Central Asian country (after Kazakhstan), creating the conditions for strengthened trade relations in a region of strategic interest for the EU, as outlined by the recent Joint Communication on the EU and Central Asia 16 . Moreover, Kyrgyzstan is one of the countries where the EU implements its new EU strategy on Central Asia, adopted in June 2019.

Meanwhile, for the timeframe 2014 - 2020, the EU will have allocated EUR 174 million of bilateral aid to the Kyrgyz Republic in grants. The focal sectors were rule of law, education and integrated rural development/social protection and public finance management. The most important implementation modality is budget support (with operations on education, social protection, and electoral reform). Kyrgyzstan is also the main beneficiary of the Investment Facility for Central Asia (IFCA). During the same period, it received a total of EUR 73.4 million in grant contributions, which catalysed investments of over EUR 300 million.

4.Method 

This ex-post evaluation has been undertaken by an external contractor and was carried out from March 2018 to April 2019. The external evaluation, whose progress and quality was monitored by an Inter-service Steering Group (ISG), included the following steps: an inception report (which explained how the evaluation design would deliver the information required), field visits to Kyrgyz Republic, Belgium and the United States for discussions with key, targeted stakeholders, an interim report and a final report (providing responses to evaluation questions). 17 The Evaluation was carried out in line with the principles commonly applied for the evaluation of EU initiatives, as laid down in the Better Regulation Guidelines. 18 This included the publication of an evaluation roadmap 19 between 22 December 2017 – 19 January 2018 and the incorporation of feedback, where relevant, in the external evaluation report. The methodology employed was comprehensive and responded to the very specific and unique nature of the MFA operation.

For the evaluation conducted by the external contractors, the evidence and the data were collected through several complementary approaches, including (i) a theory based approach (See Section 2 from external evaluation report), (ii) participatory approaches to data collection and analysis; (iii) quantitative and qualitative research methods; and (iv) triangulation. Overall, the quality of the collected evidence by the external contractor (data, documentation, interviews and survey results) for this evaluation can be assessed as very good, within the limitations mentioned below.

The qualitative research was grounded in logic and economic theory, whilst the quantitative fieldwork was based on reliable statistical data, and purposeful sampling was used for the interviews and the focus group discussion. To collect a broad, multi-dimensional and triangulated picture of the economic, financial and structural issues surrounding the programme, a wide range of relevant stakeholders and civil society organisations was also involved. Based on this and the triangulation of evidence, this evaluation can be considered reliable and valid.

The data collection methods and analytical techniques used to address the evaluation criteria’s are explained below.

Documentary review

The evaluation is based on an in-depth review of documentation assembled from a variety of sources, including the European Commission, IMF, WB, Kyrgyz authorities and others. A full list of references is available in Annex 2.

Macroeconomic data analysis (including Debt Sustainability Analysis)

The evaluation examined how the key macroeconomic indicators have evolved over time and specifically, the direction and magnitude of the changes observed over the period of interest (before, during and after the MFA implementation) as well as any deviations from the initial projections made by the IMF and the underlying reasons for these. Similarly, the Debt Sustainability Analysis (DSA) involved an analysis of key debt-burden indicators and macroeconomic variables, which influence the path of a country’s debt and its capacity to manage this burden sustainably in the medium term to long term. This was carried out using the DSA toolkit developed by the IMF and the WB that is the benchmark tool for analysing debt sustainability of countries.

Interviews with key informants/ stakeholders

Interviews with key stakeholders constituted an essential part of the fact-finding work. A total of 41 interviews were held in several rounds over the course of 2018. They covered a wide variety of issues related to the design and implementation of the MFA programmes, their impact, the overall financial support package to Kyrgyz, progress with macroeconomic stabilisation and implementation of structural reforms, and others.

Focus Group with development partners

Towards the end of the evaluation, a one-day focus group discussion was organised with locally-based development actors to complement as well as cross-check information collected from other sources such as desk research and key informant interviews.

Qualitative counterfactual analysis

The external contractor adopted a qualitative approach in this evaluation to counterfactual analysis. Specifically, the external evaluators applied counterfactual reasoning within a qualitative framework (using a theory-based approach) to draw inferences regarding the role and contribution of the MFA in promoting macroeconomic stabilisation and cushioning the social impact of the crisis. Evidence and insights collected from desk research, interviews, focus group and expert opinions were used to deduce what might have happened in the absence of the MFA (and IMF). The external report also paid particular attention to ensuring that the counterfactual scenario(s) and any inferences drawn were grounded in logic and economic theory. This was an appropriate approach given the difficulty of constructing reliable counterfactual scenarios using econometric techniques or macro-economic models in a crisis context.

Social Impact Analysis (SIA)

The evaluation examined the role and contribution of the MFA operation in cushioning the social impacts of the crisis. This was done by analysing trends in a number of social indicators prior to, during and after the MFA operation, and then applying counterfactual analysis to determine outcomes.

The detailed approach to the SIA is presented in Annex 13 of the external evaluation report.

Limitations and Methodological Issues 

While the overall reliability and validity of the evaluation is strong 20 , a number of methodological limitations and challenges effected the evaluation:

·The access and the quality of the macroeconomic data in the Kyrgyz context has been satisfactory, yet there were exceptions whereby the contractors could not trace particular events. The external contractor specifies that “[…] the data related to the financial assistance provided by Russia is somehow obscure and the study required an assumption regarding the specific grant disbursement in the late 2015 that was announced by the Kyrgyz officials, but without any evident traces in the data published by the Ministry of Finance.”

·When organising and conducting interviews, the contractors were faced with ‘memory loss’ as in certain cases stakeholders were unable to recall in detail aspects related to the programmes or/and the relevant context due to the time that had elapsed. Moreover, there was a consistent and relatively pronounced reluctance among interviewees to express their views on matters that could have been seen by them as potentially sensitive. Additionally, with respect to the one-day focus group discussion organised with locally-based development actors, most participants had very limited awareness and knowledge of the MFA.

·Evaluating the outcomes of MFA programmes is particularly challenging, since financial support normally has a lagged impact on the real economy. Furthermore, it was difficult to isolate the impact of MFA from other interventions (IMF, WB, other EU interventions, support from other donors etc.) and other exogenous factors, particularly during the abnormally longer timeframe of the operation.

·Similarly, with respect to the Debt Sustainability Analysis, the un-earmarked nature of MFA implies that it was not possible to trace down how MFA funding was exactly used (i.e. maintain public expenditure; meet the foreign debt repayment obligations etc.).

The ISG as well as other staff from DG DEVCO, EEAS and the EU delegation in Kyrgyzstan actively participated in dealing with these challenges, including by providing missing data to the evaluators and multiple rounds of feedback.

In turn, the identified risks and limitations do not put into question the overall reliability of the evaluation analysis as they were mitigated by the fact that information was obtained from a wide range of sources, using different evaluation techniques, alternative scenarios and multiple rounds of feedback. Therefore, the conclusions reached in the evaluation can be considered as valid.

5.Analysis and answers to the evaluation questions 

5.1    Relevance

Question 1: To what extent was the MFA operation design (including adequateness of financing envelope, focus of conditionality) appropriate in relation to the outputs to be produced and objectives to be achieved?

By 2010, following the lingering effects of the global economic crisis of 2008-2009 and inter-ethnic violence of 2010, Kyrgyzstan was facing pressing external and fiscal financing needs. The size of Kyrgyzstan’s financing gap was estimated at USD 271 million and USD 149 million for 2011 and 2012 respectively. However, as a result of procedural delays (see Section 3), the MFA disbursements took place in 2015 (EUR 15 million) and 2016 (EUR 15 million), where the size of the country’s financing gap was slightly improved.

As indicated in Table 2 below, actual MFA disbursements corresponded to around 10% and 22 % of the financing gaps in 2015 and 2016. The amounts accounted for 14% and 12% of the total assistance package (budget support financing exclusively) provided by the multilateral and bilateral donors in 2015 and 2016, and corresponded to around 0.3% of the GDP each year 21 . In this context, the external evaluation report confirm that the size of the MFA operation in Kyrgyzstan was of a satisfactory amount, particularly since it was only intended to supplement the assistance offered by the IMF and other international and bilateral donors.

Table 2: Kyrgyzstan’s financial requirements (2011-2016)

Source: ICF Evaluation & Internal European Commission calculations

With respect to timing, some stakeholders argued that the MFA was somehow less vital during the disbursement period. Nevertheless, the external evaluation confirms that despite the delays in disbursement, the MFA was still relevant in 2015-2016 as they coincided with important debt repayment obligations and with the time Kyrgyzstan faced difficulties as the Russian economic crisis began unfolding.

The MFA comprised an equal mix of grants and loans, reflecting the compromise reached between the Council of the European Union (who strongly opposed grant-only financing to Kyrgyzstan) and the European Parliament. The interviewed stakeholders and the external contractors judged this appropriate, particularly in the context of the overall package of EU support (which comprised a significant grant component). Furthermore, Kyrgyz government officials have regarded the un-earmarked nature of MFA disbursements and the highly concessional terms of the MFA loans as important and attractive characteristics of this MFA operation.

The MFA conditionality reform package aimed to address structural weaknesses affecting Kyrgyzstan. In turn, the seven structural conditions attached to the MFA covered four reform areas: (1) public finance management (PFM); (2) banking; (3) trade; and (4) business and investment. The specific conditions that were part of the MFA programme to Kyrgyzstan are detailed in Annex 10 of the external evaluation report.

The evaluation confirmed that all seven MFA conditions included in the MoU were considered relevant back in 2013 at the time of the MoU negotiations. The Kyrgyz authorities were effective in the implementation of the conditionality, meeting six out of seven conditions prior to the signature of the MoU in October 2014. 22 In turn, a number of interviewed stakeholders from the EU delegation questioned (1) the appropriateness of the conditions by the time the MFA was formally approved and disbursed, and (2) the extent to which the EU could have re-opened the negotiations and revised the specific MFA conditions. Nevertheless, swift implementation of MFA reforms is not uncommon, as authorities speed up the reform process to ensure that MFA will be disbursed, and convey the right commitment signals to the donor community. In addition, any re-opening of discussions related to this programme could have resulted in further delays with the disbursements, thus seriously undermining the relevance of the entire operation and triggered further tensions between the EU and the Kyrgyz government. This argumentation is shared by the external contractor in their detailed assessment of the relevance of this MFA in Section 7 and Section 8.2.2 of their final report.

Finally, MFA reforms were generally in line with the priorities set by the Kyrgyz government as well as other donors / IFIs (as part of their support programmes). These were also consistent with wider objectives associated with EU MFA operations, notably to promote structural measures/reforms capable of driving and maintaining macroeconomic stability.

The evaluation concludes that overall, at a thematic level, all areas of conditionality were relevant and well chosen.

5.2    Effectiveness of MFA 

Question 2: To what extent have the objectives of the MFA operation been achieved?

The objectives of MFA to Kyrgyzstan are, as set out, inter alia, in the Memoranda of Understanding (MoU) to “support the restoration of a sustainable external financial situation for the Kyrgyz Republic, to alleviate its balance of payments and budgetary needs and thereby support its economic and social development.” Beyond this, the objectives are also to support structural reforms.

The external evaluators relied on two strands of analysis to answering this question:

Part 1: The role of MFA in promoting macroeconomic stability, easing external financing constraints and alleviating Kyrgyzstan’s balance of payments and budgetary needs

The exact impact of the EU financial support on economic outcomes is inherently difficult to quantify, in particular due to external factors and the fact that MFA was only part of the international support package. Nevertheless, the evaluation employs a set of quantitative and qualitative analysis tools to get to plausible alternative scenarios.

With the disbursements corresponding to around 0.3% of GDP in both 2015 and 2016, the EU MFA was not negligible and contributed to the stabilisation of the Kyrgyz economy. At the same time, the absence of the MFA (‘Scenario 1’) would not have caused significant, adverse economic impacts.

The impact of this MFA on the sustainability of the Kyrgyz debt was considered positive, yet not too significant. Results from the Debt Sustainability Analysis (DSA) (Section 5.7) show that having the MFA operation will lower the present value of the debt-to-GDP ratio by 1.6 percentage points on average over 2015 - 2025, compared to not having the MFA. On the other hand, results from the Social Impact Analysis (SIA) (Section 5.6) show that the impact of this MFA on the social situation was considered less prominent, mostly because none of the MoU conditionality specifically refers to social policies and/or outcomes. Yet, evidence gathered from interviews with Kyrgyz’s civil society points to the MFA operation providing the EU with some leverage in its discussions on the controversial ‘anti-LGBTI ‘and ‘foreign agents’ bills with the Kyrgyz authorities. The Kyrgyz government ultimately did not pursued these laws for adoption in the parliament. 23

Furthermore, the evaluation did not find evidence suggesting that the sole absence of the MFA financing could have had any material impact on business and investor confidence. This could be explained in part by the low awareness/visibility surrounding this MFA operation (due to the experienced delays and the smaller magnitude of the MFA compared to assistance from other donors). In this context, a presumed negative impact on foreign exchange rates, inflation and households’ disposable incomes, in the absence of the EU MFA, is also unlikely.

With respect to financing, the Kyrgyz government could potentially have resorted to other options had MFA not been implemented, thus containing the expected negative impact on aggregate demand. Figure 14 overleaf presents these options together with the likelihood of each happening. Overall, there was a high level of consensus among all stakeholders regarding these hypothetical scenarios, with the most likely being that the government would have withdrawn some of its reserve deposits at the central bank, undertaken cuts in public expenditure, and/or sought increased assistance from Russia.

Figure 14. Alternative scenarios for obtaining finance had MFA not been available (but with IMF continuing)

Source: ICF External Evaluation Report

With respect to structural reforms, the MFA programme was effective in bringing about additional stimulus for reform in priority areas, and encouraging the implementation of specific, short-term, measures that may have otherwise been overlooked by national authorities, primarily owing to a lack of financing. At the same time, the reforms were not found to have had a significant impact on economic outcomes, predominantly due to the limited scope and since some were intended to pave the way for more extensive reforms (driven by the EU and/or other international donors) over the medium to longer term.

The hypothetical absence of both the EU’s MFA and IMF’s ECF (‘Scenario 2’) would have had more severe implications. This is not surprising, given that by its nature, MFA is meant to compliment IMF programmes, and that, it usually comes in smaller magnitudes compared to the IMF loans.

The evaluation finds that the absence of the EU’s MFA and the IMF’s ECF would have most certainly affected the financing provided by other donors, particularly the World Bank who requires a stable macroeconomic framework to deploy its budget support operations (e.g. Development Policy Operation(s)). In nominal terms, the absence of the assistance provided by the EU, the IMF and the WB in 2015 (i.e. USD 29 million from the IMF, USD 16.5 million from the WB and USD 16.5 million from the EU via the MFA), would have corresponded to a financing gap of USD 62 million, or ~1 % of GDP in 2015 only. In turn, this would have resulted in consequences that are more significant. According to the IMF, the depth of the spending cuts would have been greater, most likely affecting social spending. A potentially deeper loss of business and investor confidence would have triggered inter alia, an even greater depreciation of the national currency and a further hampering of FDI. In addition, a ‘no MFA and IMF’ alternative would have also resulted in foregone benefits from the IMF reform package.

Part 2: Effectiveness of structural reforms

The Kyrgyz authorities fulfilled all the economic and structural policy conditions related to the disbursement of the MFA, without the need for waivers. Implementation was swift, as six out of seven conditions were fulfilled prior to the approval and signature of the MoU in October 2014. Some of the interviewed stakeholders questioned the appropriateness and effectiveness of the MFA conditions and whether the Commission should have re-negotiated more meaningful reforms with the Kyrgyz. The evaluation finds fully justified that the Commission did not re-open negotiations to avoid further delays that would have only amplified tensions that prevailed at the time between the Kyrgyz government and the EU 24 . Furthermore, interviewed stakeholders from the Commission explained why the effectiveness of the reforms attached to this MFA need to be assessed in the light of the exceptional and highly political nature of this operation. Indeed this MFA was considered an additional political gesture of the EU in endorsing the country’s efforts towards becoming a fully-fledged parliamentary democracy (rather than purely an anti-crisis support mechanism aiming to stabilise the economy). The MFA reform package conferred added leverage to the EU in its discussions of the anti-LGBTI and foreign agents bills with the Kyrgyz authorities that allowed for the withdrawal of these laws back in 2015/2016.

5.3    Efficiency

Question 3: Was the disbursement of the financial assistance appropriate in the context of the prevailing economic and financial conditions in the beneficiary country?

Question 4: In what way has the design of the MFA assistance conditioned the performance of the operation in respect to its cost and its objectives?

The timing of the disbursements of this MFA operation (over June 2015 and April 2016) were still relevant and appropriate given the economic difficulties that prevailed in Kyrgyzstan at the time (See Section 2).

Throughout the process, the frequent dialogues between the EU (via DG ECFIN and the EU Delegation) and Kyrgyz authorities were critical in supporting the understanding and commitment to MFA conditionality and management of disbursements. In general, stakeholders consulted during the evaluation acknowledged a smooth running of the negotiation process, with respondents from the EU delegation emphasising the importance of technical assistance support accompanying the implementation of certain reforms (for both this MFA operation and other EU’s budget support programmes).

The Commission also closely coordinated its activity with other international creditors, in particular the IMF and the WB, thereby ensuring that actions/reforms under the EU MFA complemented those promoted by other donors on the ground. The main synergies were achieved in the public financial management, banking and trade policy areas.

On a national level, synergy was also existent between the MFA conditionality and national policies. Interviewed stakeholders acknowledge that in some instances MFA opened a window of opportunity to some government departments in promoting/accelerating their own reform plans. Overall, the Kyrgyz’s reform ownership was appropriate and there was no need for waivers. On the EU’s side, the monitoring process was frequent and adequate and the Commission complied with all checks ensuring that Kyrgyzstan had satisfactory fulfilled the reform measures supported by the MFA.

In terms of visibility, evidence gathered from local stakeholders and various development partners indicates poor visibility of the EU MFA in Kyrgyzstan. This is partly explained by the delays experienced during the negotiations and disbursements, which, equally, did not receive much attention at national level. In this context, with the aim to improve the visibility and public understanding of MFA programmes more generally, the Commission started in mid-2014 to publish on the web site of DG ECFIN the Memorandum of Understanding that lays down the reform measure related to the operations.

5.4    EU Value Added

Question 5: What was the rationale for an intervention at EU level and to what extent did the MFA operation add value compared to other interventions by other international donors? Did the operation actually lead to the expected impacts and added value of international cooperation, and what can be learnt for future operations?

The response to the economic and socio-political crisis hitting Kyrgyzstan around 2010 (See Section 2) required active co-operation of all international partners, both in terms of financial commitments and agreement on the type of conditionalities proposed. The EU was a constructive party to these discussions, and overall, the added value from the EU’s MFA to Kyrgyzstan was judged positive by the external evaluators.

The size of the MFA operation (EUR 30 million or 0.3% of GDP in 2015 and 2016), though relatively smaller when compared to the IMF programme, made a significant contribution in terms of financial value added. An important attribute of the EU MFA versus alternative sources of financing was its highly concessional terms, i.e. relatively low interest rates, long maturity and long grace period. As pointed out in the evaluation, this generated fiscal space for the Kyrgyz government at a time when the country was faced with various economic difficulties as well as preparing for parliamentary elections.

The external evaluation finds that the EU added value was most apparent in strengthening the promotion of structural reforms in several priority areas. The MFA conditions generally supplemented reforms driven by existing EU support programmes 25 and by key international donors, such as the IMF and the WB 26 . This reinforced the rationale for adjustment measures and provided local authorities with an impetus to reform swiftly in targeted sectors. More broadly, however, interviewed stakeholders from the IMF, WB, DG ECFIN and the EU Delegation reported low and varying level of reform ownership among Kyrgyz authorities. In turn, various projects led by the IMF, such as the Financial Management Information System (FMIS) and the Banking Code, were not materialising.

Beyond economic reforms, the EU MFA served wider political objectives in the Kyrgyz context, enhancing bilateral political relations. Moreover, the actual disbursements (coincidentally) came at a politically important moment ahead of parliamentary elections and thus served as a signal of EU support to democratic reforms in Central Asia, consistent with the EU policy towards the region. Furthermore, though the EU MFA did not specifically target social reform, interviews conducted with EU and Kyrgyz stakeholders suggest that the MFA played an important role in driving the Kyrgyz government to prevent the enactment of two controversial laws back in 2015 and 2016, notably the ‘anti-LGBTI’ and ‘foreign agents’ laws.

Overall, however, the EU’s value added was hampered by the delays experienced during the implementation of this MFA operation. Interviewed stakeholders felt that these delays had adversely impacted the extent of awareness and visibility surrounding the operation. Moreover, this was a completely new operation for Kyrgyzstan and the conditions associated with it were viewed as “too technical” and difficult to convey to “laymen.” 27

5.5    Coherence

Question 6: Were the measures of the MFA operation in line with key principles, objectives and measures taken in other EU external actions towards Kyrgyz Republic?

The European Union and the Kyrgyz Republic have been partners since the country's independence in 1991, increasing their cooperation and areas of interaction year-on-year. The EU’s support is channelled through various instruments, including the Development Cooperation Instrument, the Regional Indicative Programme (RIP) for Central Asia, the Investment Facility for Central Asia (IFCA), and various other thematic programmes, humanitarian and budget support operations, including MFA.

As a sign of renewed commitment, in May 2019, the Commission and the High Representative of the Union for Foreign Affairs and Security Policy adopted a new Joint Communication 28 that sets out a fresh vision for a stronger partnership with the five countries of Central Asia, including the Kyrgyz Republic. The Commission and the High Representative/Vice-President presented the new EU Strategy on Central Asia at the 15th EU-Central Asia Ministerial meeting, which took place on 7 July in Bishkek, Kyrgyz Republic.

More specifically on MFA, over the period 2012 - 2016, between 30 and 50% of EU assistance to the Kyrgyz Republic was delivered in the form of budget support or MFA grants. Figure 15 below illustrates.

Figure 15 - EU external aid to Kyrgyzstan, by type, over 2010 - 2016 (in EUR million)

Source: Annual Report on the implementation of the European Union’s instruments for financing external actions and DEVCO disbursement data. Please note that there is little information available on the types of EU aid provided to Kyrgyzstan prior to 2012.

In turn, evidence gathered by the external contractor suggests that the MFA was coherent with key principles, objectives and measures taken as part of other EU external actions towards Kyrgyzstan. Furthermore, MFA conditionalities were coherent with those in other EU and international donor support programmes, notably in the PFM and banking areas. 29 Annex 4 provides a comparison of reform areas driven by the EU through MFA, and the IMF and the WB through their respective support programmes in Kyrgyzstan.

Overall therefore, the MFA operation was both, internally as well as externally coherent.

5.6    Social Impact Analysis

MFA is a balance of payments support instrument aiming primarily at covering external financing needs and at restoring the sustainability of the external position. It is not directly targeting specific social outcomes like other EU programmes do, such that the EU’s MFA to Kyrgyzstan had no explicit conditions on social outcomes. This presents a challenge when assessing the exact social impact of the operation. In light of this challenge, the evaluation employs a comprehensive counterfactual analysis to compare what happened to the Kyrgyz Republic’s social situation during and after the MFA operation against two alternative scenarios: of no MFA (Alternative 1) and that of no MFA nor support from the IMF (Alternative 2). 30

Evidence from the counterfactual analysis suggests that the social consequences stemming from the absence of EU’s MFA (Alternative 1) would have been relatively limited, both in absolute terms and relatively when compared to those under Alternative 2. The shortfall of the EUR 30 million would have likely put more pressure on government procurement, yet significant impacts on the labour market, the cost of living and provision of key public services (e.g. health, education, pensions) are deemed unlikely. However, the hypothetical absence of both the EU’s MFA and IMF’s ECF (Alternative 2) suggest that the social consequences would have been more severe. As explained in previous sections, this is not surprising since by its nature, MFA is meant to compliment IMF programmes, and that, it usually comes in smaller magnitudes compared to IMF loans.

More broadly, the MFA supported social development through its assistance in preserving macro-economic stability in Kyrgyzstan. Indeed, a more aggravated balance of payments crisis could lead to the depletion of foreign reserves, an increase in uncertainty, and a depreciation in local currency, which would in turn have negative implications on the country’s standard of living, growth potential, unemployment and social benefits.

5.7    Debt Sustainability Analysis 

The EU MFA’s contribution to Kyrgyzstan’s public debt sustainability was clearly positive, particularly when combined with the IMF assistance, which it complemented 31 . This was confirmed by the results of the evaluation report which used the IMF's Debt Sustainability Analysis (DSA) 32 framework to quantify the effect of the IMF and MFA on public debt sustainability for 2015-2017 (medium term) and beyond (longer term). To this end, the evaluation analyses the possible paths of debt-burden indicators under different scenarios, comparing what happened (the baseline) with scenarios of no MFA financing (Alternative 1), and no MFA and IMF financing (Alternative 2). 33  

The direct impact on Kyrgyzstan’s debt dynamics comes from the concessional component of this MFA, which entails a grant element and loans with favorable terms. Specifically, the two loan tranches of this MFA bared long maturity (14 and 15 years) and very low interest rates (1.375% and 0.750%), this leading to improvement of both the maturity and interest profile of the public debt in Kyrgyzstan. The favorable terms also suggest that the Kyrgyz Republic may have benefited from savings on the cost of borrowing, and so limited the accumulation of debt to some extent.

More broadly, the impact of EU’s MFA and the IMF assistance could be seen through their positive impact in supporting economic recovery, which is key to bringing debt metrics to a sustainable path. Indeed, over 2015 to 2017, a period when MFA funds and most of the IMF’s second tranche were disbursed, Kyrgyzstan’s economic performance was relatively satisfactory. Real GDP grew at just over 4% annually on average, inflation stood at 3.2% per annum, while public debt-to-GDP ratio averaged 59.7%.

This performance is, at least in part, due to financial assistance received from the EU and IMF during this period. The interventions reduced the need for further fiscal adjustments by the authorities (tax hikes, cuts in wages and social payments), and funded an increase in (or maintaining of) government spending levels in a way that may have contributed towards stabilising the economy. In addition, the analysis estimates that over 2015-2025, the debt-to-GDP ratio is considerably higher (+ 11 percentage points) in the no MFA and IMF scenario ratio, compared to the baseline case of having the MFA and IMF operations in place. Figure 16 illustrates.

Figure 16: Present value of the debt-to-GDP ratio (%) (Alternative 2: no MFA and IMF)

Sources: Cambridge Econometrics calculations; IMF datasets, DSA toolkit and Country Reports on the Kyrgyz Republic.

Overall, therefore, the analysis reveals that the absence of MFA and IMF assistance would likely have negatively affected confidence in the Kyrgyz economy and the impact of reforms that were implemented as part of IMF conditionalities. Overall, the public debt outlook in the Kyrgyz Republic following the MFA operation has been, and is expected to remain, stable in part because of support that the country received from international partners such as the EU and the reinforcing role those donors played in driving through necessary reforms associated with financial aid from donors.

6.Conclusions

Hit by the lingering effects of the global economic crisis of 2008 and the interethnic violence of 2010, Kyrgyzstan’s economy plunged in 2010, resulting in significant balance of payments and budgetary financing needs. In response, the European Commission, among other major international donors, announced the mobilisation of EUR 118 million of assistance through various instruments, as well as a EUR 30 million package of MFA. The MFA was disbursed in full, in two instalments, over the period June 2015 and April 2016.

The EU’s MFA programme to Kyrgyzstan was exceptional in nature, as the Kyrgyz Republic is out of the normal geographical scope of MFA. This MFA operation was however justified by the strength of the political and economic reform momentum in the country at the time and by its position in a region of strategic economic and political importance for the EU. Indeed, this MFA operation bore great political significance at times of severe political and economic crisis.

The evaluation finds that the design of the MFA operation, both in terms of financing envelope and focus of reforms, was relevant to Kyrgyzstan's economic challenges and coherent with EU priorities. MFA disbursements accounted for around 0.3% of Kyrgyzstan’s GDP in both 2015 and 2016, and though delayed, were still relevant given the economic difficulties that prevailed in Kyrgyzstan at the time.

The EU’s added value was most apparent in, firstly, reinforcing the promotion of structural reforms in several priority areas, including public finance management, financial sector, trade policy and investment, and business environment. Indeed, evidence suggests that the MFA encouraged the implementation of specific, short-term, measures that may have otherwise been overlooked / further delayed by national authorities. Additionally, MFA reforms were closely aligned to and coherent with those in other EU support programmes (including budget support programmes) and those set by other important donors, including the IMF and the WB. This reinforced the rationale for adjustment measures and provided local authorities with an impetus to reform swiftly in targeted sectors.

More broadly, the evaluation finds that MFA was effective in contributing to the stabilisation of the Kyrgyz economy, particularly when combined with the IMF’s support programme, which it complemented. Due to its grant component and loans bearing long maturity (14 and 15 years) and very low interest rates (1.375% and 0.750%), MFA contributed to improving Kyrgyzstan’s public debt sustainability. The favorable terms also suggest that the Kyrgyz Republic may have benefited from savings on the cost of borrowing, and so limited the accumulation of debt to some extent. More broadly, the impact of EU’s MFA and the IMF assistance could be seen through their positive impact in supporting economic recovery, which is key to bringing debt metrics to a sustainable path.

Beyond contributing to economic stabilisation, the EU MFA served important political objectives in the Kyrgyzstan. The actual disbursements (coincidentally) came at a politically important moment ahead of parliamentary elections and thus served as a signal of EU support to democratic reforms in Central Asia, consistent with the EU policy towards the region. Furthermore, though the EU MFA did not specifically target social reform, interviews conducted with EU and Kyrgyz stakeholders confirm that the MFA provided the EU with (added) leverage to prevent the enactment of contested legislations in 2015 and 2016, notably the ‘anti-LGBTI’ and ‘foreign agents’. Indeed, the perceived high risk of losing the MFA financial envelope constituted a critical determining factor in the Kyrgyz government’s decision to not pursue the two laws for adoption in parliament.

Finally, the evaluation confirms that the MFA operation was implemented efficiently and in close coordination with the Kyrgyz’s authorities, the IMF and the World Bank. This enabled the EU to draw on the expertise of partner institutions. The Kyrgyz authorities were swift and effective in the implementation of the conditionality, fulfilling all conditions, without the need for waivers.

Overall therefore, the evaluation finds evidence that the EU MFA to Kyrgyzstan was relevant, coherent, effective and efficiently implemented. Nevertheless, the operation was characterised by an exceptionally long timetable throughout the process, which may have hampered the EU added value and the visibility surrounding the operation. Indeed, the review of the visibility of the operation and the EU communication activity indicates scope for improvement. Since then, with the aim to improve the visibility and public understanding of MFA programmes, the Commission started in mid-2014 to publish on the web site of DG ECFIN the Memorandum of Understanding that lays down the reform measure related to the operations.

Nevertheless, the external evaluation confirms that despite the delays in disbursement, the MFA was still relevant in 2015-2016 as they coincided with important debt repayment obligations and with the time Kyrgyzstan faced difficulties as the Russian economic crisis began unfolding.

A number of the conclusions and limitations identified will require further consideration and may benefit from cross-comparison with other recently completed MFA operations. A meta-evaluation is planned for 2020 and it will explore these points in more detail. The meta-evaluation will provide a synthesis of the results of the evaluation studies of MFA operations carried out in the last ten years and will consider the reliability and relevance of the methodology, process, outputs, and outcomes of these evaluations. In addition the exercise will assess the timeline of MFA operations and potential linkages to added-value and operational visibility.

Annex 1: Procedural information

1. Organisation and Timing

This evaluation assesses the Macro-financial assistance (MFA) programme for Kyrgyz Republic of EUR 30 million, which was launched in October 2013 and completed in April 2016. The assessment is in line with the Financial Regulation (article 30 point 4) and the relevant MFA Decision that call for a submission of an ex post evaluation report to the European Parliament and the Council. 34

The objective of the evaluation is to draw lessons with respect to the EU’s financial assistance, in particular the design and implementation of the programmes and the way they contributed to achieving macroeconomic stabilisation and fostering structural reforms. Apart from identifying areas of improvement for similar on-going or future possible interventions, the evaluation also aims at ensuring better transparency and accountability of the Commission's activities. The evaluation looks at various aspects of this particular EU intervention such as relevance and efficiency, effectiveness, coherence and EU value added. In addition, the evaluation explores the social impact of the MFA operations and their effect on Kyrgyzstan’s public debt sustainability.

The work of the external consultant was complemented by internal analysis. The indicative roadmap for the ex post evaluation of MFA for Kyrgyz Republic was published in December 2017. In the context of the framework contract for the provision of evaluation services related to MFA programmes, the Commission awarded on 30 January 2018 the specific contract to Consortium ICF (ICF Consulting Services Ltd. and Cambridge Econometrics Ltd.)

The lead DG to carry out and manage this evaluation has been the Directorate General for Economic and Financial Affairs (DG ECFIN). DG ECFIN chaired the Inter-service Steering Group (ISG) that was set up to manage the evaluation. Apart from DG ECFIN, the ISG comprised of representatives of other Commission services (namely the Secretariat General and the Directorate-General for International Cooperation and Development – DG DEVCO) as well as the European External Action Service (EEAS) and the EU Delegation in Kyrgyzstan.

A kick-off meeting at which the ISG and the external consultant discussed the foreseen deliverables and the evaluation techniques in depth, took place in March 2018. It was followed by meetings on the inception and interim reports in May 2018 and in July 2018, respectively. Overall, the ISG met three times. In addition, ISG members were continuously informed and consulted, through e-mails and phone calls, during the various stages of the evaluation.

2. Consultation of stakeholders

The consultation strategy included a range of appropriate tools to reach key, relevant stakeholders in order to collect pertinent evidence on the implementation of the MFA in Kyrgyzstan. In turn, this did not include an open public consultation, but was rather targeted to specialists - people who have either been closely involved in the development and/or the implementation of the MFA-I operation or persons with expert knowledge in the areas related to the objectives of the MFA operation. In addition, an indicative evaluation roadmap was published in December 2017 to seek wider feedback 35 . Finally, the results of this evaluation will, in due course, feed into a wider meta-evaluation of EU MFA operations, which will include an open public consultation.

3. Evidence, sources and quality

The evidence used for the ex-post evaluation of MFA for Kyrgyz Republic included a variety of documents, official data sources, media outlets as well as interviews and surveys. Approximately 60 documents were reviewed and analysed. These could be grouped in four main types - (i) documents directly related to the design and implementation of the MFA programmes, (ii) EU policy-related documents, (iii) documents published by the Kyrgyz authorities, and (iv) documents prepared by various international financial institutions and economic researchers. In addition, a variety of data sources was used and analysed such as Kyrgyz authorities (including Ministry of Finance and Central Bank of Kyrgyzstan), the World Bank, the World Economic Outlook of the IMF, and the EuroAsian Development Bank among others.

Overall, the quality of the collected evidence (data, documentation, interviews and survey results) for this evaluation can be assessed as very good, demonstrating a satisfactory degree of confidence regarding the various findings of this evaluation. The quantitative fieldwork was based on reliable statistical data, while purposeful sampling was used for the interviews, the Delphi survey and the focus group discussion. To collect a broad, multi-dimensional and triangulated picture of the economic, financial and structural issues surrounding the programme, a wide range of civil society organisations was also involved.

4. Limitation and Challenges

Aside from the limitations and challenges detailed in Section 4 of this SWD, the main challenge surrounding this exercise appeared to be the timeline of the evaluation as initially envisaged. Given the number, complexity and depth of the data treatment, the organisation of several missions, and the availability of the persons to be interviewed, the time line of the evaluation proved somewhat tight. Therefore, to address these pressures an extension of the evaluation time line was deemed necessary.

Annex 2: References

Bibliography

OECD (2010), Kyrgyz Republic, Aid Effectiveness 2011: Progress in Implementing the Paris Declaration

Ministry of Finance of the Kyrgyz Republic (2010), Reporting document, information letter

European Commission DG ECFIN (2011), Ex-Ante evaluation statement on EU macro-financial assistance to the Kyrgyz Republic, Staff Working Paper

Council of the European Union (2011), Possible Macro-Financial Assistance to the Kyrgyz Republic Further Information, Information note for the Financial Counsellors Working Party

European Commission DG ECFIN (2011), Proposal for a decision of the European Parliament and of the Council providing macro-financial assistance to the Kyrgyz Republic. COM(2011) 925 final

European Commission (2012), Operational Assessment of the financial circuits and procedures in the Kyrgyz Republic, Report (July 2012)

European Commission (2012), Second instalment review, Policy Support Programme, Social Protection and Public Finance Management, Aide Memoire – Review Mission (May 2012)

European Commission (2013), Decision of the European Parliament and the Council of 22 October 2013 providing macro-financial assistance to the Kyrgyz Republic, DECISION No 1025/2013/EU

European Commission DG ECFIN (2014), Macro-Financial Assistance to the Kyrgyz Republic, MS Committee meeting of 26 June 2014, meeting minutes (June 2014)

European Commission DG ECFIN (2014), Memorandum of understanding between the European Union and the Kyrgyz Republic on the Macro Financial Assistance for the Kyrgyz Republic, Grant and Loan from the EU of up to EUR 30 million, Memorandum of Understanding (July 2014)

European Parliament (2015), Joint motion for a resolution on Kyrgyzstan homosexual propaganda bill (January 2015)

European Commission (2015), European Parliament urges Kyrgyzstan to drop anti-LGBTI bill, press release (January 2015)

European Commission DG ECFIN (2015), Macro-Financial Assistance the Kyrgyz Republic: Disbursement of the first tranche, Information note to the European Parliament and to the Council (May 2015)

European Commission DG ECFIN (2015), Commission Decision on the borrowing of the EU for the macro-financial assistance loan to the Kyrgyz Republic, C(2015) 3582 final

European Commission DG ECFIN (2015), Commission Decision on the release by the EU of Macro Financial assistance to the Kyrgyz Republic, C(2015) 3581 final

European Commission DG ECFIN (2016), Macro-Financial Assistance the Kyrgyz Republic: Disbursement of the second tranche, Information note to the European Parliament and to the Council (May 2015)

ECORYS (2017), Ex-post evaluation of Macro-Financial Assistance to Jordan, Report (October 2017)

EC Mission reports

European Commission (2011), Discussion on possible macro-financial assistance for Kyrgyzstan. Staff Report on mission to the Kyrgyz Republic (23-27 May 2011, Bishkek)

European Commission (2012), Mission report to the Kyrgyz Republic in the framework of the MFA Operational Assessment, Staff Report (11-15 June 2012)

European Commission (2013), IMF conference on Caucasus and Central Asia 19-21 May 2013, and discussions on the conditionality for the macro-financial assistance operation with the Kyrgyz Republic. Staff Report on mission to the Kyrgyz Republic (21-24 May 2013, Bishkek).

European Commission (2013), Discussions on the legal documents associated with the macro-financial assistance operation with the Kyrgyz Republic. Staff Report on mission to the Kyrgyz Republic (8-11 October 2013, Bishkek).

European Commission (2013), Report on mission to the Kyrgyz Republic to finalize an agreement on EUR 30 MM macro-financial assistance operation. Staff Report on mission to the Kyrgyz Republic (8-11 October 2013, Bishkek).

European Commission (2016), Report on the review mission to Bishkek, Kyrgyzstan, 17-19 November 2015 related to the disbursement of the 2nd tranche of the MFA. Staff Report on mission to the Kyrgyz Republic (17-19 November 2016, Bishkek)

IMF mission reviews

International Monetary Fund (2011), Public Finance Management Reforms, Report (August 2011)

International Monetary Fund (2011), Kyrgyz Republic: First Review Under the Three-Year Arrangement Under the Extended Credit Facility and Request for Modification of Performance Criteria, Staff Report (December 2011)

International Monetary Fund (2014), Kyrgyz Republic: Sixth Review under Three Year Agreement under ECF.

International Monetary Fund (2015), Staff Report for the 2015 Article IV consultation, Staff Report (November 2015)

International Monetary Fund (2016), Kyrgyz Republic: Second Review Under the Three-Year Arrangement Under the Extended Credit Facility and Request for Modification of Performance Criteria, Staff Report (June 2016)

International Monetary Fund (2017), Kyrgyz Republic: Third Review Under the Three-Year Arrangement Under the Extended Credit Facility and Request for Modification of Performance Criteria, Staff Report (June 2017)

International Monetary Fund (2018), Kyrgyz Republic: Fourth and Fifth Review Under the Three-Year Arrangement Under the Extended Credit Facility and Request for Modification of Performance Criteria, Staff Report (February 2018)

MFA annual reports and SWD

European Commission (2012), Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2011, Staff Working Document (June 2012)

European Commission (2012), Background analysis per beneficiary, accompanying the document: Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2011, Staff Working Document (June 2012)

European Commission (2013), Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2012, Staff Working Document (June 2013)

European Commission (2013), Background analysis per beneficiary, accompanying the document: Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2012, Staff Working Document (June 2013)

European Commission (2014), Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2013, Staff Working Document (June 2014)

European Commission (2014), Background analysis per beneficiary, accompanying the document: Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2013, Staff Working Document (June 2014)

European Commission (2015), Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2014, Staff Working Document (June 2015)

European Commission (2015), Background analysis per beneficiary, accompanying the document: Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2014, Staff Working Document (June 2015)

European Commission (2016), Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2015, Staff Working Document (June 2016)

European Commission (2016), Background analysis per beneficiary, accompanying the document: Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2015, Staff Working Document (June 2016)

European Commission (2017), Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2016, Staff Working Document (June 2017)

European Commission (2017), Background analysis per beneficiary, accompanying the document: Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2016, Staff Working Document (June 2017)

Key data sources / statistics

Centre for International Development of Harvard University, Atlas of Economic Complexity. Available at: http://atlas.cid.harvard.edu/

IMF, (2018). World Economic Outlook April/July. Available at: https://www.imf.org/en/Publications/WEO/Issues/2018/07/02/world-economic-outlook-update-july-2018 

World Bank (2018). Available at: https://data.worldbank.org/ 

Various statistics from the Ministry of Finance and Economy

Various statistics from the CIA World Factbook

Various statistics from the National Statistical Committee of the Kyrgyz Republic

Various statistics from the National Bank of the Kyrgyz Republic

Various IMF datasets, and Country Reports on the Kyrgyz Republic

Annex 3: Discussion of validity and reliability of the evaluation findings

In the Commission’s and the external evaluator’s judgement, the overall reliability and validity of the evaluation is strong. There are multiple lines of evidence and inquiry contributing to answering each evaluation question, mitigating the limitations associated with individual research activities. Moreover, a series of measures were undertaken by the study team to ensure the reliability and validity of the overall findings of this evaluation as indicated in the table below.

Overview of the main elements underpinning reliability and validity of findings

Annex 4: Comparison of MFA, IMF and World Bank conditionalities

Annex 5: Evaluation Framework

Annex 6: Synopsis Report

Introduction

Stakeholder consultation is a key element in the ex-post evaluation of MFA to the Kyrgyz Republic and Commission minimum standards have been met, as a comprehensive range of key stakeholders have been reached as part of the consultation.

The implementation of the stakeholder consultation strategy followed the publication of the ex-post evaluation roadmap of MFA to the Kyrgyz Republic in December 2017. The strategy was developed with the overall objective to capture as much information as possible with regard to the MFA programme, in addition to information collected via desk research and data analysis. The consultation focused on extracting recollections from the time when the operations were designed and implemented – See Table A6. The consultation also focused on collecting views on the period after the MFA was ended to assess its impact on achieving its key objectives and drawing lessons for ongoing and future similar EU interventions.

Table A6: General Timeline of MFA operation

This consultation strategy described below sets out the objectives of the consultation, maps key stakeholders, presents the consultation methods and tools which are used and demonstrates how the stakeholder consultation fits in the evaluation framework.

The diagram below presents a timeframe for the implementation of this consultation during 2018 and 2019. The following section elaborates.

Objective setting

The objective of stakeholder consultation was to collect as much valuable and relevant information as possible from the key stakeholders involved to construct an ex-post assessment of the design, implementation and impact of the MFA operation. Stakeholders were consulted on the following key aspects:

·Relevance of the MFA operation: the relevance of the objectives, the financial envelop and the conditionality, both at the time of designing the MFA operation and currently with the benefit of hindsight;

·Effectiveness of the MFA operation: the direct results of the MFA operation, i.e. the results of the MFA operation on macroeconomic level and the results in the area of the structural reforms;

·Efficiency of the MFA operation: the design and process of the MFA in terms of value-for-money;

·EU-added value of the MFA operation: the added value of the operation when considering other possible scenarios and alternatives;

·Coherence of the MFA operation: alignment with other support initiatives implemented at the time of the MFA;

·Social impact: the more indirect impact of the MFA operation in the context of social development in Kyrgyzstan, in particular alleviation of the impact of the refugee crisis;

·Debt sustainability: the longer-term result of the MFA I in terms of implications for Kyrgyzstan’s fiscal sustainability.

Stakeholder mapping

Consultation was targeted to specialists – either people who have either been closely involved in the development and/or the implementation of the MFA-I operation or persons with expert knowledge in the areas related to the objectives of the MFA-I operation (i.e. macroeconomic and fiscal policy, structural reforms in the areas of PFM, social and labour policy, investment climate and trade, and energy).

Therefore, the following groups of stakeholders have been central in this consultation strategy: (1) Kyrgyzstan authorities; (2) International Financial Institutions, (3) Other donors, (4) external (i.e. non-governmental) experts.

In addition, EU officials, including current and former representatives of the European Commission, representatives of the European External Action Service (EEAS) and representatives of the EU Delegation in Kyrgyzstan played a key role in the information gathering process.

The full list of stakeholders is detailed in Annex 4 of the external report.

Consultation methods

Interviews with key informants

A series of face-to-face, in-depth interviews were conducted with all key informants and stakeholders, including non-governmental actors such as business representatives and civil society organisations. The table below indicates the number and types of stakeholders interviewed.

Overview of the interviews conducted:

The focus of the interviews varied significantly depending on the stakeholder type. The interviews were used to analyse all evaluation questions - relevance, effectiveness, efficiency, coherence, EU added value. The topics covered included, among others, design and coordination of the MFA programmes, assessment of the economic impact of the MFA, views on progresses made in various reform areas and role of MFA in achieving these, articulation of the MFA with other EU instruments/as part of the wider EU-Kyrgyzstan relationships, implementation aspects such as domestic political and institutional constraints as well as timeliness of disbursements). The insight gained through the interviews was a key input for the assessment of the five evaluation questions.

The key findings from the interview discussions are detailed in Annex 7.

Focus group

The focus group discussion was organised with a wider group of non-government stakeholders (not directly involved in the operations) to collect their views and opinions on various aspects of the MFA operation. Representatives of the following organisations participated in the focus group discussion:

ŸEurasian Development Bank;

ŸWorld Bank;

ŸAsian Development Bank;

ŸUnited Nations Development Programme - Kyrgyzstan;

ŸDeutsche Gesellschaft für Internationale Zusammenarbeit - Kyrgyzstan;

ŸEuropean Bank for Reconstruction and Development.

The full list of participants is detailed in Annex 4 of the external report.

The findings from this exercise were therefore used to complement as well as cross-check information collected from other sources such as desk research and key informant interviews. The key findings from these discussions are detailed in Annex 3.

Workshop

The stakeholder validation workshop was organised in January 2019 via video-conference from Brussels in order to validate the emerging findings with stakeholders closely. The specific agenda comprised a presentation of the main findings by the external consultant, followed by a discussion. Potential recommendations as regards the design and implementation of future MFA operations were also discussed.

The full list of participants is detailed in Annex 12 of the external report.

Annex 7: Summary of findings from stakeholder consultation activities

Interviews with EU and Kyrgyz stakeholders

Relevance of the EU MFA

There was general consensus among the stakeholders consulted that the EU MFA was sizable, though to a lesser extent when compared to other forms of EU and international assistance provided to the Kyrgyz Republic (e.g. IMF support). Most stakeholders recalled the delays that surrounded the design and implementation process, which had some bearing on EU-Kyrgyz relations owing to the uncertainty that prevailed as regards the actual disbursement of the MFA. Eventually, the disbursement of the MFA was however coincidentally timely owing to new economic difficulties the Kyrgyz economy faced from 2015 (i.e. a sharp fall in exports to Russia (2015) and an important contraction in gold production (2016)). On the form of the MFA (i.e. the grant-loan combination), this was considered appropriate by both EU and Kyrgyz stakeholders. Some Kyrgyz stakeholders did however highlight their preference for grant-only arrangements. Various EU stakeholders commented on this possibility and confirmed that the likelihood for such an arrangement was low at the time, given the (existing) realm of EU aid operations in place in Kyrgyzstan.

The EU MFA was accompanied by a reform package agreed between the EU and Kyrgyz authorities. Target reform areas were informed by an in-depth operational assessment conducted by the EU that highlighted priority areas for action. EU and Kyrgyz stakeholders unanimously agreed that the MFA reform package was in line with the priorities set by the Kyrgyz government as well as other donors / IFIs (as part of their support programmes). The reforms were also consistent with wider objectives associated with EU MFA operations, notably to promote structural measures/reforms capable of driving and maintaining macroeconomic stability. However, owing to various procedural delays, most of the reforms had been implemented by the time the MoU was agreed and signed. Various stakeholders felt that the fulfilment of MFA conditionalities prior to the signature of the MoU de facto made the conditionalities redundant, wrongly signalling to Kyrgyz authorities that the EU was handing ‘free’ money. There was thus strong support for re-negotiations.

Effectiveness of the EU MFA

Stakeholders were asked about various aspects (pertaining to the effectiveness) of the EU MFA package, notably: the nature and extent of reform implementation and the (economic and social) impacts of such reforms.

Some stakeholders remarked that, in spite of the delays encountered in finalising the terms of the MFA package and disbursing the assistance to the Kyrgyz Republic, the EU MFA coincidentally came at the right moment (i.e. 2015). The Kyrgyz economy was facing new economic difficulties, while parliamentary elections were nearing. Nonetheless, it was generally recognised that the EU MFA was not “indispensable” and that the Kyrgyz government would have been able to find/seek alternative sources of finance in these difficult times (e.g. Russian support). In addition, many stakeholders believed that the Kyrgyz government would have also resorted to cutting public expenditure (especially among unprotected categories).

It was generally recognised that target reform areas under the EU MFA were appropriate. However, other donors (notably the IMF and the WB) were also active in these areas and promoted longer-term reforms (with potentially more significant macro-economic impacts than the EU MFA). Most stakeholders confirmed that Kyrgyz authorities fulfilled the economic and structural policy conditions related to the disbursement of the MFA in a satisfactory way. Various stakeholders however questioned the potential redundancy of MFA conditions, given that most of the reforms had been implemented prior to the MoU being approved. There was general preference for re-negotiations to ensure that reforms were still fit for purpose at the time of the disbursements. Some stakeholders praised the IMF’s negotiation process, which (as opposed to the EU) offers greater flexibility to review and adjust reforms / conditions.

It was generally recognised that the EU MFA entailed additional impacts. First, its political dimension contributed to strengthening EU-Kyrgyz relations at a time when the Kyrgyz Republic faced important political and economic difficulties. This impact was however potentially reduced owing to the delays that surrounded the disbursement of the EU MFA. Additionally, the magnitude of the impacts entailed by the EU MFA may have been less important when compared to other EU programmes. These are known to have had a desired positive impact, notably in terms of advancing legislation and promoting certain values, such as democracy, free press, rule of law and good governance.

Furthermore, a number of stakeholders recognised that, even though the EU MFA did not explicitly target particular areas of social development, it played an important role alongside other stakeholders (notably civil society) in preventing the enactment of the ‘foreign agents’ and ‘anti-LGBTI’ bills in 2015 and 2016.

Efficiency of the EU MFA

In general, stakeholders consulted during this research acknowledged a smooth running of the negotiation process (facilitated through various mission visits). This also comprised effective coordination between the EU and other international donors to ensure that actions/reforms under the EU MFA complemented those promoted by other donors on the ground. There were particular synergies in the public financial management (PFM) and financial areas. The general level of reform ownership among Kyrgyz authorities was seen by many stakeholders as having been varying, partly in sync with the pendulum of the political cycle. Although some stakeholders saw a substantial pick up in the appetite to embark on MFA reforms in late 2014, more difficulties were reported closer to the parliamentary elections in the late 2015. In terms of monitoring, formal and regular missions constituted an important channel of dialogue between the Commission and Kyrgyz authorities. These were generally led by the Commission in close collaboration with the EU Delegation to the Kyrgyz Republic. The monitoring process was judged appropriate, though less stringent when compared to the extent of supervision undertaken in other recipient countries. A commonly-cited reason for reduced scrutiny in the case of Kyrgyzstan was that it was “outside of the traditional scope of MFA.”

EU added value of the EU MFA

There was general recognition that the EU’s added value, as a result of the MFA, was high at the beginning of the negotiation/implementation process. However, with the numerous delays, it was further recognised that the overall EU added value was arguably reduced. In particular, various stakeholders felt that these delays had adversely impacted the extent of awareness / visibility surrounding the operation.

Coherence of the EU MFA

It was generally recognised that the MFA conditionalities / reforms were designed to be coherent with the objectives of the operation as well as other EU and international donor support programmes, notably in the PFM and banking areas. As such, the MFA operation was both, internally as well as externally coherent.

Focus group with development partners

Theme 1: Awareness of the role played by the EU support and specifically the MFA operation

Familiarity with the MFA operation

The general consensus among the workshop participants was that “publicity” surrounding the EU MFA has been negligible in Kyrgyzstan. There is little awareness/understanding among the donor community (and the general population) of the MFA instrument, its objectives and intended outcomes.

Several participants (e.g. GIZ, EBRD) explained that the EU MFA is only viewed as “some form of EU support.” The EU MFA is not necessarily attributed to budget support. Specific conditions attached to the operation are also not widely known and information is lacking as regards the concrete outcomes/impacts the operation is seeking to achieve.

Other participants also stressed that awareness of the MFA operation is almost inexistent among the general population (in Bishkek as well as other provinces, such as Osh).

Extent of media coverage

The EU is generally perceived as an important donor partner in Kyrgyzstan. However, the MFA operation has received less attention (by the media) when compared to other EU support programmes, especially those dedicated to reforms in the areas of the rule of law, the judiciary, the electoral system.

A representative of the UNDP explained that the international donors’ conference (organised in Bishkek back in July 2010) promulgated the EU’s work in / assistance to Kyrgyzstan, but little was said about the MFA operation specifically This was because the MFA negotiations had not yet been initiated around the time the conference took place. In contrast, the focus was on the EU’s continued assistance to Kyrgyzstan to tackle important societal issues/challenges (e.g. deficiencies of the electoral system).

In the same vein, representatives from the EBRD acknowledged the need to publicise the role of the EU more widely. Various initiatives of the EBRD are often co-managed by the EU and, in these instances, the role of the EU becomes more apparent – e.g. via press releases, EBRD/EU logo displays, etc. However, in the case of the EU MFA, media coverage is lacking, and other communication channels are not fully exploited.

Various participants were in favour of social media as a means to showcase EU initiatives (such as the EU MFA), increase visibility, and stimulate interest. Other communication channels, such as local radio, television, were also suggested to increase visibility and reach.

General perceptions of the EU and relative visibility of its assistance programmes compared to those offered by other international donor organisations

Visibility of the EU MFA could be significantly improved. As the negotiations surrounding the terms/conditions of the MFA financing often take place between the EU and senior officials of the government only, the donor community, local stakeholders (e.g. the media, the civil society) and the general public do not have all the necessary information to fully understand the purpose and intended objectives of the EU MFA. Additionally, participants recognised that, in the case of the EU MFA, the outcomes/impacts/benefits are not “tangible” per se. As a result, further down the line, it becomes even more difficult to attribute certain reforms/impacts to the MFA operation.

In that respect, a representative of GIZ warned that the general public tends to attribute specific projects/reforms to the organisations who ultimately implement them. For instance, GIZ (as opposed to Germany) is generally perceived to be the provider of financing within the projects it oversees on the ground.

Some participants also indicated that the extent of the role played by donor organisations in implementing certain projects/reforms on the ground is often “minimised.” As such, the opening of a project overseen by GIZ was highly-politicised in the run up to last year’s elections, which suggests that donors often allow their projects to be used by for the purposes of domestic politics.

In terms of visibility, the EU MFA operation is not on par with other support programmes, notably the IMF’s. The IMF has a longstanding relationship with Kyrgyzstan, which could nonetheless explain the relative importance attributed to its programmes (and associated conditionalities / reform areas) among the media, and local stakeholders, including the general public.

Raising awareness is thus imperative when it comes to the EU MFA. As alluded to before, communication channels (specifically digital media) ought to be fully exploited. Participants also suggested that the process of negotiating and agreeing the terms of the MFA financing be publicised. Ratification of the MFA agreement by Parliament, for instance, could be more widely covered. This would not only increase awareness of the operation but also the level of transparency around the amount of funds being negotiated with the EU and their intended use(s).

Theme 2: Review of the impacts of the MFA

Economic impact

Participants were asked to assess the most probable course of action for Kyrgyzstan under various scenarios, namely: (i) Alternative 1: what would have happened had MFA not been disbursed in 2015 and 2016? (ii) Alternative 2: what would have happened had MFA and IMF assistance not been disbursed in 2015 and 2016?

Under Alternative 1, several scenarios were discussed, including:

Scenario 1: Financing from other key donor / bilateral partners;

Scenario 2: Financing from domestic markets;

Scenario 3: Public cuts.

Scenario 1 was generally seen as the preferred course of action for Kyrgyzstan had MFA assistance not been provided. According to participants, Russia would have been the “easiest” partner to reach out to for budget support. The extent to which the financing offered would have been “tied” (i.e. specific conditions would be established prior to disbursement) was however difficult to ascertain. As such, Russia currently provides many (varying) forms of assistance to Kyrgyzstan and the extent to which their disbursement is conditional on the implementation of reform is not known.

Scenario 2 could potentially have been contemplated by the Kyrgyz government, though it could have proved more expensive (as opposed to seeking grant support from donor/bilateral partners) and further exacerbated the crisis faced by the country. In that regard, some participants recalled Armenia’s experience. In 2010, the Armenian government engaged in large-scale borrowing from domestic markets to finance public spending and boost economic activity. This led to a rapid and substantial increase in interest rates and the “crowding out” of private investment (or a reduction in private investment spending). The government eventually had to adjust the fiscal rule to contain the pressures caused by their overspend, but with difficulty.

Scenario 3 was considered plausible, though it was generally agreed that the government’s room for manoeuvre would most likely have been small. As such, some government expenditures are “protected” and there would not have been much flexibility to reduce them. On the other hand, a reduction in or (most likely) a reallocation of capital expenditures could have been considered. In sum, the government would have most probably favoured greater budget / fiscal discipline.

In the event that both IMF and EU assistance would not have been provided to Kyrgyzstan (Alternative 2), participants generally agreed that bilateral partners, such as Russia and China, would have most likely played a greater role (as regards the provision of financing). In addition, a representative of the Eurasian Development Bank (EDB) reckoned that the Kyrgyz government could have issued Eurobonds, i.e. It could have resorted to more non-concessional borrowing. This approach was recently trialled by the Tajik government.

Political impact

On the political significance of the EU MFA in Kyrgyzstan, a prevalent observation from participants was that the operation did not “catch eyes and ears.” This also applies to the delays experienced during the negotiations and disbursements which, equally, did not receive much attention at national level.

The lack of visibility around the EU MFA was, to some extent, attributed to the EU’s insufficient presence on the ground (as opposed to countries, such as Russia, which invest considerably in Kyrgyzstan and heavily publicise their support programmes).

Nonetheless, participants remarked that other EU programmes, notably those targeted at specific reforms (e.g. electoral system, rule of law, democracy, etc.), tend to be more widely covered and have sparked greater interest over the years among local stakeholders and the general public. It is easier to promote such programmes/projects as the objectives/intended outcomes of the assistance offered are clearer and “more targeted.” On the contrary, conditions associated with the EU MFA are often “too technical” and difficult to convey to “laymen.”

To further boost visibility, participants suggested that the EU increase support designed to tackle important (and pressing) societal issues, such as bridal kidnapping, gender inequality, promotion of women’s rights, and promotion of mass media.

Theme 3: Review of the MFA contribution to Kyrgyz reform efforts

General assessment

Mass media and civil society were viewed as critical in driving national reforms in Kyrgyzstan. Specifically, they can help better promote donor assistance at national, local, and regional level. When assisting the implementation of reforms in the country, donors (such as the EU) ought to engage more heavily with them to boost visibility of their support programmes.

As far as the EU MFA is concerned, participants suggested that conditions be developed in collaboration/consultation with civil society and other lobby groups. Additionally, the language used for describing the conditions ought to be simplified in order to reach out / appeal to a larger audience. These conditions should also be publicised and made available within the public domain. This would increase transparency around how the EU MFA is being channelled into public sector reform.

On the design of external support/assistance, participants were generally in favour of the idea of having conditionalities attached to financing (akin to the EU MFA). In a way, they ensure that “government is accountable” and “pressured into reforming priority areas.”

Participants highlighted Kyrgyzstan’s longstanding track record of successfully implementing reforms (when compared to its neighbours). However, in 2010, the Constitution was amended and required that reforms be ratified in Parliament, which largely reduced the government’s ability/power to “move reforms quickly in the short term.” Additionally, some participants remarked that “frequent changes in government” (as experienced by Kyrgyzstan in recent years) tend to limit policy / reform continuity. Further, corruption is rampant and, to a large extent, inhibits reform. Corruption scandals were particularly cited in the judiciary which, according to participants, have furthered the level of distrust towards the courts.

Participants were asked about potential reform areas for the EU to focus on going forward. The judiciary, public governance (or corruption), and private sector development (including competitiveness) were most commonly cited. Other key reform areas identified were: trade and banking. As regards the former, one participant explained that Kyrgyzstan’s membership to the Eurasian Economic Union (EEU) has not reaped the benefits intended. Many trade barriers still exist. As regards the latter, access to loans (especially by businesses) was flagged as a major concern, a key impediment being high interest rates. Finally, policies/reforms aimed at furthering digital transformation were also recommended. Digital transformation of the economy has the capacity to spur innovation and boost more sustainable growth. Participants warned that Kyrgyzstan should not lag behind its neighbours in this area but instead channel efforts into positioning itself as a “regional digital hub” or “digital champion.”

Finally, participants suggested that the EU focus on social reforms as well, notably women’s healthcare services, which were reported to be lacking in many parts of the country. More widely, the needs of vulnerable groups (such as children, the elderly, etc.) should be thoroughly assessed and reforms should be focused on improving their well-being.

(1)

Decision No 1025/2013/EU (OJ L 283, 25.10.2013, p. 1–6)

(2)

Available at: https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-6341347_en

(3)

European Commission, 2011. Ex-ante evaluation statement on EU macro-financial assistance to the Kyrgyz Republic. Available at: https://eur-lex.europa.eu/legal content/EN/TXT/PDF/?uri=CELEX:52011SC1619&from=EN

(4)

The ‘Joint Economic Assessment: Reconciliation, Recovery and Reconstruction (JEA)’ by the IMF, WB, ADB (and the participation of the European Commission) provides a detailed assessment of the impact of tumultuous events of April and June. Available at: https://www.imf.org/en/Publications/Policy-Papers/Issues/2016/12/31/Kyrgyz-Republic-Joint-Economic-Assessment-Report-for-the-Kyrgyz-Republic-Donor-Conference-PP4466

(5)

 European Commission, 2011. Ex-ante evaluation statement on EU macro-financial assistance to the Kyrgyz Republic. Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52011SC1619&from=EN

(6)

The specific conditions attached to the MFA programme to Kyrgyzstan are further detailed in Annex 10 of the external evaluation report.

(7)

For ease of reference and comparison, and unless otherwise stated, this section uses the same dataset as per the external evaluation report.

(8)

Asian Development Outlook (2013). Available at: https://www.adb.org/sites/default/files/publication/30205/ado2013-kyrgyz-republic.pdf

(9)

In 2013, the Kyrgyz government reached an agreement with the Canadian–based gold mining company, Centerra, allowing for joint ownership of the Kumtor mine. The agreement led to the resolution of a long-standing dispute, which drove gold mining and production to resume.

(10)

Asian Development Outlook (2014). Available at: https://www.adb.org/sites/default/files/publication/31241/ado2014-kyrgyz-republic.pdf

(11)

Asian Development Outlook (2015). Available at:
https://www.adb.org/sites/default/files/publication/154508/ado-2015.pdf

(12)

Asian Development Outlook (2016). Available at: https://www.adb.org/sites/default/files/publication/182221/ado2016.pdf

(13)

IMF (2019). Available at: https://www.imf.org/en/News/Articles/2019/03/20/pr1981-kyrgyz-republic-imf-staff-completes-2019-article-iv-mission-to-the-kyrgyz-republic

(14)

World Bank (2019). Available at: http://pubdocs.worldbank.org/en/956691557323120176/Global-Economic-Prospects-June-2019-Analysis-ECA.pdf

(15)

Asian Development Bank (2019). Available at: https://www.adb.org/sites/default/files/publication/492711/ado2019.pdf

(16)

  https://eeas.europa.eu/sites/eeas/files/joint_communication_-_the_eu_and_central_asia_-_new_opportunities_for_a_stronger_partnership.pdf

(17)

The evaluation’s procedural information is detailed in Annex 1.

(18)

European Commission, July 2017, https://ec.europa.eu/info/files/better-regulation-guidelines_en

(19)

  https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-6341347_en  

(20)

For a more detailed overview of the main elements underpinning reliability and validity of findings, refer to Annex 3.

(21)

For more detail, see Section 7.1.2 of the external evaluation report.

(22)

The MoU policy conditions were negotiated on the assumption that they would have to be met by June 2014, i.e. before the expiry of the 2011 IMF Extended Credit Facility (ECF).

(23)

For more detail, refer to Box 4 of the external evaluation report.

(24)

The interviewees (EU Delegation in Bishkek, the Presidential Office, the Ministry of Finance, EEAS and IMF staff) confirmed that the protracted implementation of this MFA constituted a strain on the political relationship between the EU and the Kyrgyz authorities.

(25)

PFM reforms, for instance, were central to the disbursement of specific EU support packages, including Sector Reform.

(26)

The World Bank, for instance, indicated during the stakeholder consultation that the EU MFA laid the groundwork for the PEFA, which was eventually undertaken in 2014.

(27)

Workshop with development partners.

(28)

Joint Communication available at: https://eeas.europa.eu/sites/eeas/files/joint_communication_-_the_eu_and_central_asia_-_new_opportunities_for_a_stronger_partnership.pdf

(29)

For a detailed presentation of the approach to the Social Impact Analysis, see Annex 13 of the external evaluation report.

(30)

The EU’s MFA complemented an IMF rescue package of about $92.4m provided to the Kyrgyz Republic under the Extended Credit Facility (ECF) programme which began in April 2015. Some of the funds from the IMF programme were disbursed over a three-year period starting with about $26.8m in 2015, $26.6m in 2016, and $26.5m in 2017, with the final tranche not disbursed.

(31)

 The IMF developed the DSA framework as a tool to better detect, prevent, and resolve potential payment crises. The framework consists of two complementary components: the analysis of the sustainability of total public debt and that of total external debt. For details see: https://www.imf.org/external/pubs/ft/dsa/ .

(32)

 For a detailed presentation of the DSA methodology, see Annex 14 of the external evaluation report.

(33)

Decision No 1025/2013/EU of the European Parliament and of the Council of 22 October 2013 providing macro-financial assistance to the Kyrgyz Republic

(34)

Evaluation roadmap available at: https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-6341347_en