EUROPEAN COMMISSION

Brussels, 20.7.2016

SWD(2016) 251 final

COMMISSION STAFF WORKING DOCUMENT

Accompanying the document

Report from the Commission to the European Parliament and the Council on evaluating the implementation of Decision No. 406/2009/EC pursuant to its Article 14

{COM(2016) 483 final}

Table of Contents

1.EXECUTIVE SUMMARY

2.INTRODUCTION

2.1.Purpose of the evaluation

2.2.Scope of the evaluation

3.BACKGROUND TO THE INITIATIVE6

3.1.Description of the initiative and its objectives

3.2.Baseline9

4.EVALUATION QUESTIONS

5.METHOD

5.1.Process and methodology

5.2.Limitations – robustness of findings

6.IMPLEMENTATION STATE OF PLAY

7.ANSWERS TO THE EVALUATION QUESTIONS

7.1.Relevance

7.2.Effectiveness14

7.3.Efficiency

7.4.Coherence23

7.5.EU added value

7.6.Competition24

8.CONCLUSIONS

9.ANNEXES

Annex 1: Procedural information

Annex 2: Stakeholder consultation

Annex 3: Methods and analytical models used in preparing the evaluation

Annex 4: Provisions for flexibility instruments and monitoring and compliance in the Effort Sharing Decision

1.    EXECUTIVE SUMMARY

Article 14 of Decision No. 406/2009/EC (also called the Effort Sharing Decision, ESD) requires the European Commission to draw up a report evaluating the implementation of the ESD by 31 October 2016. The evaluation was undertaken in 2015.

Adopted in 2009 as part of the Climate and Energy package, the Effort Sharing Decision establishes greenhouse gas (GHG) emission limits for all Member States for 2020. It covers emissions in the transport, buildings, agriculture, small industry and waste sectors. These sectors accounted for more than 55% of total EU GHG emissions in 2013. The objective of the ESD is to reduce GHG emissions in the EU by 10% by 2020 compared to 2005 and promote reductions of emissions within its scope in a fair and cost-effective manner.

While the first year of reporting by Member States under the Decision was 2015, at the time of the evaluation most of the provisions of the Decision had not yet been applied, making it more difficult to draw conclusions about their appropriateness and to consider any changes based on lessons learned. However, the evaluation could use valuable evidence deriving from the process of legal implementation of the Decision and the preparatory work undertaken by Member States, including national policies and measures reported by them to the European Commission.

There is strong and consistent evidence that the EU is on track to meet its 2020 GHG emissions reduction target in the ESD sectors. Based on Member State reports submitted in 2015, GHG emissions in 2013 and 2014 in all Member States were below their annual limits for these years. Total 2013 emissions covered by the ESD at EU level were 9.7% lower than the 2005 emissions. In 2014 EU emissions covered by the ESD further decreased to a level 12.9% below 2005 levels, which was also below the EU-wide ESD target for 2020.

Most emission reductions since 2009 have occurred from technological changes and policies which have resulted in increased uptake of less carbon-intensive technologies. This effect has been reinforced by the fact that the ESD was launched together with a number of other EU climate and energy initiatives as part of the 2020 package, in particular on energy efficiency and renewable energy.

The economic recession has also had an effect on GHG emissions in some ESD sectors to date, and these effects will continue to resonate until 2020. However, many sectors covered by the ESD are not directly impacted by the fluctuations in GDP and are more prone to policy influence. For several of the ESD sectors, including buildings, transport, agriculture and waste, part of the emissions reductions to date can be attributed to factors that are influenced by policy interventions at EU and member state level related to the 2020 package. Whilst it was possible to identify that the ESD has had some effect in stimulating new national policies in some Member States, there was insufficient evidence to quantify the overall impact of the ESD on GHG emissions at this stage.

The Effort Sharing Decision was found to add value through action on EU level. There was a strong level of agreement among stakeholders representing Member States that the ESD raised awareness of mitigation potential in ESD sectors and contributed to establishing new national institutional and legal frameworks. It also improved coordination on GHG mitigation across the ESD sectors and between national and regional or local governments.

Stakeholders highlighted that an important outcome of the ESD to date has been the improvement in the quality of the emissions data and projections relating to ESD sectors at a national level, which has helped improving policy preparation. The annual emission limits, combined with annual and biennial requirements for reporting emissions, policies and measures, and projections, keep Member State better informed about the progress not only of GHG emissions but also of climate and energy policies. The reporting also gives them and other EU stakeholders a tool to compare their performance with that of other Member States. Finally, transparency of reporting helped building trust and confidence among Member States that each of them is contributing a fair share to the overall EU climate policy.

Evidence on the direct costs and benefits and the efficiency of national policies implemented in response to the ESD is very limited; it was not possible to assess these costs with confidence. This is partly due to the fact that national policies and measures reported by Member States so far have provided insufficient information on expected and actual costs and benefits.

The ESD was found to have resulted in limited additional administrative burden on Member State level, although there may be opportunities for reducing administrative costs at EU level, for example by simplified or less frequent compliance controls.

Stakeholders did not present any evidence that national policies resulting from the ESD have unduly distorted competition in the EU internal market.

The objectives of the ESD remain relevant and still correspond strongly with the needs of the EU. The European Council Conclusions in October 2014 on the 2030 Climate and Energy Policy Framework confirmed the importance of the ESD and of its continuation after 2020 with all its main elements.

The evaluation found a strong consensus among stakeholders that there was a continued need for an instrument such as the ESD also for the period after 2020. Evidence from stakeholder interviews, and to a lesser extent the literature review, suggested that the objectives of the ESD are coherent with other EU climate and energy policies, such as the ETS, energy efficiency and renewable energy.

2.    INTRODUCTION

Adopted in 2009 as part of the Climate and Energy package 1 , Decision No. 406/2009/EC 2 (also called the Effort Sharing Decision, ESD) establishes greenhouse gas (GHG) emission limits for all Member States for 2020. It covers emissions in the transport, buildings, agriculture, small industry and waste sectors. These sectors accounted for more than 55% of total EU GHG emissions in 2013. 3 The objective of the ESD is to reduce GHG emissions in the EU by 10% by 2020 compared to 2005 and promote reductions of emissions within its scope in a fair and cost-effective manner. The ESD entered into force in June 2009.

While the first year of reporting by Member States under the Decision was 2015, at the time of the evaluation, most of the provisions of the Decision had not yet been applied, making it more difficult to draw conclusions about their appropriateness and to consider any changes based on lessons learned. However, the evaluation could use valuable evidence deriving from the process of legal implementation of the Decision and the preparatory work undertaken by Member States, including national policies and measures reported by them to the European Commission.

2.1.    Purpose of the evaluation

This evaluation was performed as required by Article 14 of the Effort Sharing Decision and it covers the period from the adoption of the Decision in 2009 until November 2015. 4 It examined the actual implementation and the achievements of the Effort Sharing Decision at Member State and EU level, including to what extent it is contributing to the overall EU 2020 GHG emissions target as expected.

It provides conclusions on how the Effort Sharing Decision has performed so far, what experience has been gained, and what lessons can be learned with respect to policies and measures implemented by Member States to reduce greenhouse gas emissions in the sectors covered by the Decision, and their effect on their national emissions.

The results of the evaluation were also used in the impact assessment of the Commission’s legislative proposal for continuing effort sharing after 2020 within the 2030 Climate and Energy Framework. 5

2.2.    Scope of the evaluation

The evaluation explored the impacts of the Effort Sharing Decision both at EU and Member State level. Impacts on third countries were expected to be limited and mainly associated with the use of international project credits as part of the flexibility instruments of the ESD. However, since the use of such credits by Member States has, to date, not occurred, the evaluation did not include this point.

In accordance with Article 14 of the ESD, the scope of the evaluation included all the provisions and requirements of the Decision, including how the implementation affected competition. In addition, it implicitly covers reporting and verification procedures laid down in Regulation No 525/2013 (Monitoring Mechanism Regulation, MMR) as the information needed for the compliance assessment under the ESD is submitted by Member States as required in the regulation.

The general approach of the evaluation was to focus on Member State actions from when the ESD entered into force in June 2009 as it is reasonable to assume that climate mitigation policies and measures adopted in Member States from 2009 and onwards were influenced by the Decision.

It should be noted that some of the main requirements in the ESD were only due for the first time in 2015, and others only in 2016.

3.    BACKGROUND TO THE INITIATIVE

3.1.    Description of the initiative and its objectives

The Climate and Energy Package sets EU-wide targets for 2020, requiring a 20% reduction in EU total greenhouse gas emissions from 1990 levels, a 20% share of EU final energy consumption from renewable sources, and a 20% improvement in its energy efficiency.

The 20% GHG emission reduction target for the European Union by 2020 compared to 1990 is equivalent to a 14% reduction compared to 2005. This effort has been divided between the sectors covered by the Emission Trading System (ETS) and sectors under the ESD. In the EU ETS emissions are required to decrease by 21% relative to 2005 levels, while from sectors in the ESD a 10% reduction relative to 2005 is required.

To share the required EU-wide effort of 10 % GHG emission reduction by 2020 under the ESD, national targets for 2020 were set for Member States according to economic capacity on the basis of their relative wealth (measured by 2005 gross domestic product per capita). This principle for sharing the effort ensures fairness. The national targets range from a 20% emissions reduction by 2020 (from 2005 levels) for the richest Member States to a 20% increase for the least wealthy one. (See Figure 1).

The ESD does not set specific emission targets for the individual sectors covered by the ESD, but leaves it to Member States to choose where and how to achieve the necessary reductions according to their national situation

To meet their commitments Member States are supposed to limit their greenhouse gas emissions in the sectors covered by the ESD, and they are expected to implement national policies and measures to fulfil their obligations. Member States can also apply flexibility instruments to meet their obligations and enhance cost-effectiveness, if needed.

The ESD also defines a linear trajectory of corresponding binding emission limits (annual emission allocations, AEAs) for each year from 2013 to 2020. Progress towards the 2020 targets is ensured through annual reporting obligations and compliance checks. Member States are obliged to report on their greenhouse gas emissions and projected progress towards meeting their ESD obligations.

Member States are responsible for implementing policies and measures to meet their obligations under the ESD and are supported by a number of EU measures 6 , some of which also are expected to help achieving the EU’s 2020 renewable energy and energy efficiency targets. These supporting EU policies are important to stimulate EU-wide emission reductions in the sectors under the ESD. But it is clear that emission reductions have to be delivered through national policies and measures, in particular in sectors such as transport and buildings.

In order to provide flexibility for Member States in implementing their commitments and as a means to enhance the overall cost-effectiveness of reaching the EU-wide 2020 target, the ESD provides a range of flexibility instruments. These concern Member States’ possibilities to manage their own AEAs within the compliance period and engage in transfers of AEAs among each other. Should a Member State’s GHG emissions exceed its AEAs for a given year, it can borrow 5% of its AEAs from the next year, buy AEAs from other Member States, or use international project credits in order to meet its annual limits. Should a Member State reduce its emissions by more than needed thereby overachieving its target for a given year, it can keep the surplus AEAs for later use within the commitment period or transfer it to other Member States. 7  

The ESD has an annual reporting and compliance cycle consisting of Member States reporting their GHG emissions in national inventory reports to the European Commission, which then performs an emission inventory review to validate the reported emissions, and a compliance check (i.e., comparing the actual emission of Member States with their annual emission allocations for a given year). If a Member State's emissions exceed its annual emission allocation, even after accounting for flexibility instruments, they will be subject to certain penalties and have to take corrective measures.

Figure 1: Member State greenhouse gas emission limits in 2020 under the ESD

Emission limits in 2020 are set in relation to 2005 emission levels. Source: Decision No. 406/2009/EC.

As outlined in the intervention logic diagram that was developed for the evaluation, as a result of the ESD Member States are expected to report on their annual greenhouse gas emissions and projected progress towards meeting their obligations, and implement national policies and measures in sectors covered by the ESD limiting their GHG emission by 2020 to meet their national targets. They are expected to apply flexibility instruments to enhance cost-effectiveness and compliance, if needed.

Intervention Logic

In the diagram all actions and consequences are shown as linear. In practice, some steps will occur in parrallel, and there will also be some feedback loops between the different steps. For example, depending upon their expected emissions performance at a given point in time relative to their target, Member States may choose to implement more or less national policies.

In terms of impacts with more long term effects Member States are expected to prepare national policies to reduce greenhouse gas emissions beyond 2020 and also promote low-carbon energy resources and energy efficiency. Other expected long-term impacts are reduced air pollution and health costs, green jobs and cleaner production.

3.2.    Baseline

In 2008 the Commission undertook an extensive analysis of the GHG emissions trends in EU and in Member States up to 2020 as part of the impact assessment accompanying the package of implementation measures for the EU's objectives on climate change and renewable energy for 2020 8 . According to the impact assessment it was expected that, by 2020, EU-wide ESD emissions under a business-as-usual scenario would increase by 2.4 % relative to the 2005 emission levels. Projections submitted by Member States for a number of selected sectors under the ESD in 2009 showed a similar trend in projected emissions to 2020, with emissions from energy use and transport in particular expected to increase across the EU. (Figure 2)

Figure 2:

Note: WEM = with existing (current) measures; WAM; with additional (planned) measures)

•Projections based on data reported by Member States under the Monitoring Mechanism Decisions by May 2009, so represents projections from 2008 and 2009.

•For Industrial processes and waste the WEM scenario is identical to the WAM scenario

Source: GHG emission trends and projections in Europe 2009, European Environment Agency 2009.

It was concluded that for the majority of Member States additional national policy measures would need to be implemented in order for them to meet their emission limits if they were to be achieved with domestic action alone. As mentioned earlier, a wide range of EU-wide policies were expected to help Member States to meet their emission limits under the ESD. However, the expected quantitative contribution of the individual EU policies towards the delivery of Member States’ targets was not determined.

It was also expected that some Member States would use flexibility mechanisms, including the use of international project credits, in order to deliver their ESD targets at least cost.

4.    EVALUATION QUESTIONS

The main evaluation questions were agreed by the Commission steering group set up for this evaluation and part of the evaluation mandate. The questions relate to the relevance, effectiveness, efficiency, coherence and EU added value of the ESD and follow the established evaluation framework that is used for Commission evaluation methodology. An additional question was included on how the implementation of the ESD affected competition as stated in Article 14 of the ESD.

The main evaluation questions were further elaborated and complemented in the study supporting the evaluation. 9  

Relevance

How well do the objectives of the Decision still correspond to the needs within the EU?

Effectiveness

How do the observed effects correspond to the objectives of the ESD? To what extent can the effects reasonably be credited to the ESD?

To what extent is the EU on track to meet its 2020 GHG emission reduction target for the ESD? To what extent are Member States on track to meet their 2020 emission limits?

What drivers and barriers (including actions by Member States and other stakeholders) contributed to or stood in the way of achieving the objectives? How are these factors addressed in the Decision? For example: How well are co-operation between central and regional/local authorities working in Member States?

Efficiency

To what extent are the costs resulting from the implementation of the ESD proportionate to the results and benefits that can be linked to the observed effects of the ESD? This should include analysis of secondary impacts concerning administrative and other costs, including for SMEs, employment etc.?

What factors influenced the efficiency of the ESD to achieve its objectives?

Which role did the flexibility instruments play until now? Are Member States using or planning to use them?

Coherence

How well does the ESD fit with other EU climate and energy policies, including on CO2 emission standards for light-duty vehicles, on non-CO2 gases, energy efficiency and renewable energy sources? To what extent are the different requirements under the ESD working coherently as they were expected to?

EU added value

How is the ESD contributing as expected to the EUs international commitments under the second commitment period of the UNFCCC's Kyoto Protocol? To what extent could the changes brought by the Decision have been achieved by national or individuals' measures only?

What would be the most likely consequences of stopping or not continuing the ESD?

Competition

To what extent have the policies and measures implemented by Member States in order to achieve obligations under the Effort Sharing Decision affected competition on national, Union and international level?

5.    METHOD

5.1.    Process and Methodology 10

The evaluation of the Effort Sharing Decision was performed in 2015. A steering group consisting of members from the concerned directorate-generals in the Commission was set up for the evaluation. The European Environment Agency also participated in this group. A study supporting the evaluation was carried out by a team of external consultants for the European Commission from April to December 2015. 11 . The steering group agreed on the evaluation mandate and on the intervention logic, guided the work of the consultants and made a quality assessment of the evaluation report produced by the consultants. 12

As far as possible, evidence was collected from multiple sources to enable the comparison and triangulation of key findings. The evaluation used evidence from emissions and emission trends reported by Member States, adopted national policies and measures, policy literature and case studies. It used questionnaires and structured interviews with Member State experts and other key stakeholders involved in implementing ESD on national level to gather additional evidence. 13 The evaluation also drew on the results of a public consultation on the preparation of a legislative proposal on reducing Member States’ GHG emissions in sectors under the ESD from 2021 to 2030. 14

The evaluation benefited from the well-established EU monitoring framework for GHG emissions, projections and policies measures and also previous research on GHG emission drivers. 15 This reduced the need for extensive collection of new data or modelling. Member States report GHG emissions data to the European Commission and the United Nations Framework Convention on Climate Change (UNFCCC) every year. In addition to GHG inventories, Member States have to report every second year on the policies and measures implemented in order to achieve their targets under ESD and on their projection. At the moment of the evaluation GHG inventory information for all years between 2009 and including 2013 was available. The reports on projections included data reported by Member States in 2015. The information in these reports provided important data sources for the evaluation.

Two targeted questionnaire surveys were organised in order to gather the views of stakeholders on the evaluation questions. A general survey which covered all of the evaluation questions was shared with all stakeholder groups. A specific survey on administrative burdens associated with the implementation of the ESD’s reporting requirements was targeted at Member States.

The Commission launched a public consultation on the preparation of a legislative proposal on the effort of Member States to reduce their GHG emissions to meet the European Union's GHG emission reduction commitment in a 2030 perspective. The consultation ran from 26 March 2015 to 18 June 2015 and it included questions relating to the current ESD. Each question relevant for the evaluation received between 78 and 100 responses that have been taken into account as part of the analysis.

A number of stakeholder interviews were held in order to allow the more detailed exploration of the evaluation questions and to provide contextual information to the data and indicators collected for the evaluation. The interviews provided a valuable form of evidence, particularly for those evaluation questions that cannot be tested empirically. Furthermore, the interviews helped identifying additional sources of research, as well as validating and further explaining the findings from the literature review. 

Based on the general survey and drawing on the stakeholders' interviews, three case studies were developed as practical examples exploring the wider impacts of the Decision. Each case was used to complement answers to evaluation questions together with the other evidence.

5.2.    Limitations – robustness of findings

The main limitation of this evaluation is linked to the timing. Because of the requirement of the ESD to evaluate and report on the implementation of the ESD by October 2016 and the fact that the European Council in October 2014 endorsed a continuation of the ESD after 2020, the evaluation had to be carried out already in 2015. This was at an early stage of the compliance period and the first year of the compliance check, which was not yet completed at the time of the evaluation. 16 Consequently, at the time of the evaluation, most of the provisions of the Decision had not yet been applied, making it more difficult to draw conclusions about their appropriateness and to consider changes based on lessons learned.

Another limitation is related to the fact that policies and measures reported by Member States provide insufficient information on their expected and actual costs and benefits.

Further limitations are inherent to the methodology used for the evaluation. The sample size for both questionnaires and stakeholders interviews was relatively small. This was not surprising given the nature of the Decision which mostly concerned a limited number of specialists and experts in national administrations in Member States. The risk of biased results by unrepresentative sample was mitigated by ensuring the coverage of stakeholder from most groups and types. For example, both the interviews and surveys included governmental representatives from large and small Member States, from Member States who are projected to meet their ESD target easily, and from those that may need to implement additional measures.

Despite the efforts to increase the response rate during the evaluation, for other stakeholder groups the sample size cannot be considered representative. In particular, the views of business associations are potentially under-represented, although the lack of interest from this group may indicate that views are less strong in general. This view was supported by the interviews themselves, with a generally lower level of understanding and awareness of the ESD from this group. This potentially reflects the lack of a direct impact of the ESD on businesses, with any impacts arising through other EU and national policies. It should be noted that a public consultation preparing a proposal for continuing the ESD after 2020 was organised in parallel to the evaluation. The consultation received responses from a number of business associations and it appears that they focused their feedback on this consultation.

While validation of the research conclusions with other sources was performed it has not been possible within the scope of the evaluation to critically appraise these studies and their limitations.

6.    IMPLEMENTATION STATE OF PLAY

As shown in the intervention logic, the Commission was supposed to undertake a number of actions to implement the provisions of the ESD. All these actions have been completed.

The Effort Sharing Decision mandates the Commission to determine annual emission allocations for each Member State for the years 2013 to 2020. Annex II of the ESD sets the limits to be achieved by each Member State by 2020 expressed in percentages while Article 3 provides the methodology for these percentages to be translated into annual emissions allocations (AEAs) expressed in CO2 equivalents in a Commission decision. According to Article 10 of the ESD the AEAs set by this decision also have to be adjusted by the Commission in order to account for any scope correction in the ESD due to any changes in the scope of the ETS in the period 2013-2020. 17  

The Commission adopted both these measures in 2013 following an initial review of Member States GHG inventories for the years 2005, 2008, 2009 and 2010 performed by the Commission with the assistance of the European Environmental Agency. 18

The Commission has also adapted existing legislation and established the necessary infrastructure for Member States ESD accounts in the Union Registry. 19 .

With respect to Member State obligations under the ESD, they must each year report their GHG emissions. They must also, every second year, report on their implemented national policies and measures and submit updated projections in 2015 with expected progress to their 2020 targets. Each year the Commission compiles the information reported by Member states and publishes a progress report where it analyses and presents Member States’ progress towards their 2020 ESD targets. An assessment of Member States’ progress towards their targets is also part of the European Semester country report published by the Commission every spring. Emission trends and projections in the EU and its Member States are also published each year by the European Environment Agency which assists the Commission with the implementation of the ESD.

Member States have so far fulfilled their reporting obligations and the exchange of information with the Commission is working well. The Commission and Member States expert meet several times every year in working groups under the Climate Change Committee to follow up the implementation of the reporting obligations under the ESD.

So far, no Member State has used any of the flexibility instruments provided in the ESD as all countries appear to be meeting their annual emission limits for the first two years of the compliance period. In the future it can be expected that flexibility instruments, such as trade with other Member States, will be used by some Member States that are projected to emit above their limits by 2020. Although the flexibility instruments under the ESD are untested they remain widely supported and were further endorsed by Member States in the stakeholder consultation.

7.    ANSWERS TO THE EVALUATION QUESTIONS

This section provides a synopsis of the answers to the evaluation questions presented in section 3. A detailed analysis of the answers to all questions is provided in chapter 6 of the supporting evaluation study.

7.1.    Relevance

The ESD is part of the package of measures adopted under the climate and energy package in 2009 addressing the need to cut emissions to a level that limit the global average temperature increase to below 2 degrees Celsius compared to pre-industrial levels. It was designed to ensure that this objective is met in a cost effective and fair manner. In order to answer the question on relevance, the evaluation assessed how well the objectives of the Decision still correspond to the needs within the EU.

The relevance of the objectives of the ESD and the need for continuing this initiative was confirmed by the European Council in its conclusions in October 2014 on the 2030 Climate and Energy Policy Framework which agreed of its continuation to 2030 in a cost-efficient and fair manner with all the elements as applied in the Effort Sharing Decision for 2020.

The adoption in December 2015 of the Paris Agreement confirmed the commitment at the highest level of the EU to implement domestically greenhouse gas emissions reductions also for 2030 and beyond. As laid down in its intended nationally determined contribution (INDC), the EU commits to cut emissions in its territory by at least 40% below 1990 levels by 2030.

The evidence from both literature review and stakeholder consultation suggests that the objectives of the ESD of ensuring emissions reductions in a cost efficient and fair manner still correspond strongly with the needs within the EU. Therefore, while the policy and economic context has changed as compared to the time when the Decision was first introduced, the problems that it was designed to address remain as important, as do the objectives of the Decision itself.

Out of the 34 responses received in the stakeholder consultation to the question whether there continues to be a need for the ESD or a similar instrument to promote reductions of GHG emissions in the ESD sectors across the EU, in a cost-effective and fair manner, the large majority of stakeholder including national governments, NGO and academia agreed (17) or strongly agreed (14), with no respondents disagreed with the statement.

A further need addressed by the Effort Sharing Decision was to overcome the market failure which means that prices do not reflect the full social costs of the GHG emissions (i.e. the economic costs of climate change), so fail to provide appropriate price signals to deliver the required level of abatement and stimulate cost-effective mitigation. The binding nature of the Decision helps to address this need by providing appropriate price signals for emissions abatement, by providing Member State limits on emissions in the ESD sectors and allowing trading between member States as part of the flexibility mechanisms. This need largely remains as strong. Whilst some EU wide policies will in part correct for this market failure, for example, by regulating the CO2 performance of new vehicles, or by requiring minimum levels of energy taxation, the market corrections are not applied equally across all sectors of the economy.

7.2.    Effectiveness

The evaluation questions explored if the EU and its Member States are on track to meet their 2020 obligations, to what extent Member States have implemented national policies and measures any emission reductions to reduce emissions, and whether any emission reductions can reasonably be credited to the ESD. The questions also covered how cooperation between central and regional authorities was working with respect to implementing the ESD.

Based on data reported by Member States to the Commission, there is strong and consistent evidence that the EU is on track to meet its 2020 GHG emissions reduction target in the ESD sectors. 20 Total 2013 GHG emissions covered by the ESD at EU level were 9.7% lower than the 2005 emissions. In 2014 EU emissions covered by the ESD further decreased to a level 12.9% below 2005 levels, which was below the EU-wide ESD target for 2020. 21

The achieved emission reduction represents a large improvement in performance relative to the business-as-usual scenario when the 2020 targets were first agreed. Emission reductions have so far been much deeper than expected in 2007 when the European Council agreed on the overall EU climate targets for 2020 and the Commission performed the impact assessment of the Climate and Energy Package. According to the business as usual scenario in the impact assessment prepared for the Climate and Energy Package, EU-wide ESD emissions were expected to increase by 2.4% between 2005 and 2020. 22  

In the EU total emission reductions between 2005 and 2013 were achieved in all sectors, ranging from -3 % in agriculture to -25 % in the waste sector. (Figure 3) In this period there was also a convergence of GHG emission intensities across Member States, both per capita and per GDP.

ESD emissions per Member State have also decreased significantly since 2005. In all Member States ESD emission were below their annual limits in 2013 and 2014. (Figure 4) Overachievements were typically larger in those countries that were allowed to increase emissions compared to 2005. Countries that experienced a particularly severe economic recession (e.g., Greece, Portugal and Spain) also had emissions significantly below their 2014 limits.

According to Member State projections submitted in 2015, total ESD emissions are expected to keep decreasing until 2020, continuously being below target at the EU level. (Figure 5) The EU-wide ESD emissions are expected to decrease by 13.3% in 2020 relative to 2005 emission levels (under a scenario taking into consideration existing measures) which is overachieving the 10% reduction attributed to the ESD sectors.

A total of 24 Member States are projected to meet their national targets domestically, while four Member States (Austria, Belgium, Ireland and Luxembourg) are expected to need additional measures or use flexibility instruments within the ESD to reach their targets. 23  These Member States may exceed their emission limits for one or more years of the 2013-2020 period with existing measures in place. However, they are expected to comply with their legal obligations for each year in 2013-2020, thanks to AEA surplus from earlier years in the 2013-2020 period, which Member States can use to cover any emissions gap up to 2020 or by implementing additional measures at home. 24  

Figure 3: Achieved EU-wide ESD emission reductions 2005-2013

Figure 4

The figure shows that in all Member States ESD emission were below their annual limits in 2014 under the ESD. The emissions are based on based on approximated inventory data reported in 2015 by Member States or estimated by the EEA on behalf of the Commission, where needed. Adapted from Figure 3 in the Climate Action Progress Report 2015. 25  

Figure 5: Actual and projected total ESD emissions 2005-2020

Source: Trends and projections in Europe 2015, EEA (2015). Updated with emission inventory figures submitted by March 2016. The black dotted line represents the linear trajectory of the annual emission limits under the ESD for the period 2013-2020.

Impact of the Effort Sharing Decision on observed GHG emissions reductions

The Effort Sharing Decision was supposed to limit GHG emissions in the EU and in the Member States. While the previous analysis shows a significant drop in GHG emissions at EU and Member States level, it is unclear from the evidence how much of these reductions can be attributed to national measures adopted as a direct result of the ESD. It was not possible to quantify the extent to which the observed reductions in historical emissions, and the expected future emissions trend, can reasonably be credited to specific policies. In particular, it is difficult to isolate the impact of the ESD on national climate and energy policies, and from other EU-wide initiatives under the Climate and Energy Package, which have a more direct impact on the different drivers of emissions.

The economic recession has also had an effect on GHG emissions in some ESD sectors to date (especially freight transportation), and these effects are expected to resonate until 2020. However, many sectors (such as buildings and agriculture) covered by the ESD are not directly impacted by the fluctuations in GDP and are more prone to policy influence which gives reasons to conclude that some of the GHG reductions in the ESD sectors can be attributed to policy interventions at EU and Member State level.

The results from a study decomposing the historical drivers of GHG emissions in the EU show that for the ESD sectors, the economic recession did not play as important a direct role in the observed emission reductions even if for the whole economy 30% of the change in emissions can be attributed to decrease in GDP. 26

While GDP is particularly relevant for the non-ETS industry and freight transport, the decomposition analysis revealed that for emissions from buildings, passenger transport, agriculture and waste, a significant part of the reductions could be attributed to factors that are influenced by policy intervention. However, it was not possible to quantify the extent to which the observed effects on historical emissions, and the expected future emissions trends, can reasonably be credited to the ESD alone. In particular, it is difficult to isolate the impact of the ESD from other policies, including other EU policies 27 .

Another decomposition analysis was carried out for the 2005-2012 period covering CO2 emissions from fossil fuel combustion, which account for about 80 % of total GHG emissions, in both ETS and ESD sectors. The analysis concluded that technological changes contributed most to drive down emissions, by far outweighing the contribution of the shift within and between economic sectors, and by far overcompensating GDP related emission drivers in times of economic crisis.  28

The results showed that CO2 emissions decreased overall by 11.5 % between 2005 and 2012. Technological changes had the most significant effect on driving down emissions, leading to an 18.5 % decrease and by far outweighing the contribution of the shift between economic sectors. The policies implemented in the field of climate and energy contributed significantly to the take-up of less carbon-intensive technologies, including renewable energy. Growth in economic activity (GDP) caused a 6.8 % increase in emissions. Structural changes in the economy (at constant GDP and CO2 z\intensity in every economic sector) caused a small increase in emissions, by 1.7 %.

Stakeholders expressed mixed views on how the ESD contributed to the observed emission reductions. Respondents from all stakeholders groups identified the implementation of national policies (which may have been introduced in response to the ESD) as an important driver in some cases, however, the effects of the recession, and other EU wide policies were also considered important. While some stakeholders considered that the ESD has been an important driver for new national policies and measures in certain Member States, others considered that the ESD may have had little or no influence on national policy developments so far. The level of influence of the ESD also appears to vary by Member State, which may reflect the different situation of Member States in relation to their ESD targets, i.e., whether they need to take further action or not.

Based on the findings of the supporting study of the evaluation, Table 1 summarises different that have so far contributed to changes in GHG emissions in ESD sectors.

Table 1: Summarising factors contributing to changes in emission levels in ESD sectors 29  

Importance is ranked from (0) marginal effect to +++ very significant effect

Implementation of new national policies in ESD sectors

The evaluation found an increase in the implementation of national policies in the ESD sectors in most years starting from 2007, when the European Council agreed on the overall EU climate targets for 2020.

All sources of evidence indicated that the ESD has been a driver for new (i.e., planned after 2009) national policies and measures (PAMs) in over half of Member States (based on the EEA PAMs database: Belgium, Croatia, Denmark, Estonia, France, Germany, Greece Latvia, Lithuania, Malta, Romania and Spain, based on the stakeholder consultation also Austria, the Netherlands and Hungary). The PAMs cover all sectors: transport (39), energy consumption (25), waste (12), agriculture (7), industrial processes (2), energy supply (6), and cross-cutting (28).

According to the stakeholder consultation, the ESD has been successful in reinforcing or strengthening existing PAMs in some Member States, as well as raising awareness of the GHG abatement potential and cost of mitigation measures (23 out of 34 of respondents agreed or strongly agreed) and supporting the development of new legal frameworks or legislative actions for limiting ESD emissions (19 respondents out of 34 agreed or strongly agreed). Specific policy examples that were cited included Austria (Climate Change Act studies), Hungary (Domestic Decarbonisation Roadmap), Lithuania (Strategy for the National Climate Change Management Policy by 2050), Latvia (Climate Change Financial Instrument), the Netherlands (various fiscal measures), Romania (initial steps towards new national legal framework) and Spain (2020 Roadmap for ESD emissions). However, some respondents disagreed that the ESD had resulted in new policies (9 out of 34 respondents disagreed) or in the strengthening of existing policies (4 out of 34 respondents disagreed).

It is therefore apparent that the strength of the ESD as a driver varies between Member States, and in some of them the ESD may have had little or no influence on national policy developments. Analysis of the reports submitted by them to the Commission suggests that this difference does not necessarily reflect the status of Member States with respect to the expected performance against their ESD targets. However, this is not born out by the stakeholder interviews, where this status was considered more important. This indicates an apparent inconsistency in the reporting of linked EU policies under the MMR, and on the views expressed by stakeholders.

The number of PAMs implemented in response to ESD is only a rough indicator of policy action. The expected GHG savings from the PAMs would be more informative. However, this information is scarce. In the reporting of GHG savings by Member States, the impacts of single PAMs are frequently aggregated and reported for a group of policies. In Figure 6 reported emission savings expected by 2020 have been aggregated per EU policy based on an assessment made by the European Environment Agency. 30 This shows that the reported impact of the ESD is relatively small compared to other EU policies, such as the Renewable Energy Directive or the ETS. The result is perhaps not surprising given that the ESD only sets an overall cap for emissions in Member States, whereas most other EU-wide policies included in the comparison were designed for specific measures or sectors. It should also be noted that this analysis only can give a rough indication, considering the incompleteness of reporting of emission savings by the Member States.

There may be several reasons as to why, according to their reporting, the ESD does not appear to have been a major driver of policies in some Member States to date. Firstly, the reporting may not represent a complete and accurate reflection of the influence of EU policies on the development of national climate policies. In their reporting, Member States might link national policies only to the most obvious EU policy and not necessarily to all EU policies that have had an influence. Secondly, there may be a lag in policy development in response to the ESD, for instance the case studies show that in Belgium and Spain the governments have installed plans to put them on track to achieve the ESD target, but this is not yet fully reflected in either concrete new policies or in their reporting of 2015. Finally, Member States do not have to achieve their ESD target with domestic action alone, but can instead use the flexibility instruments of the ESD to compensate for any shortfalls in emission reductions compared to the target.

The evaluation produced reasonable evidence that the ESD has contributed to a certain extent to the cooperation between national and regional or local authorities. Nevertheless such influence is likely to be different between Member States. In some Member States little or no cooperation or coordination has taken place between central and regional or local level. This is supported by evidence from the sixth National Communications 31 , where 16 Member States did not explicitly mention coordination between central and regional or local level. This however does not mean that mitigation actions at local or regional level are not taking place.

Figure 6: Aggregated emission savings from EU policies (source: EEA) 32

7.3.    Efficiency

The costs and benefits due to the implementation of measures in ESD sectors are hard to quantify, as a direct link between many national climate and energy policies and emission reductions in sectors under the ESD so far is difficult to establish. This is also due to the fact that policies and measures reported by Member States provide insufficient information on their expected and actual costs and benefits.

The relative youth of climate change policies that were introduced as a result of the ESD and the climate and energy package means there are currently few assessments of ex-post cost effectiveness. In coming years, this should change and a more complete picture of the extent to which the ESD has delivered cost effective abatement across sectors should become more apparent.

Benefits associated to the Effort Sharing Decision

The main benefit associated with the ESD relates to the GHG emissions reductions in the sectors covered by the Decision. In addition to any direct costs, the implementation of the ESD has potentially also delivered a number of economic co-benefits, such as improvements in air quality and energy security (i.e., reduced costs of fuel imports). 33

A further qualitative benefit of the ESD appears to be that it helps Member States in setting up institutional frameworks, targets and data collection and reporting procedures for reducing emissions in the ESD sectors. This benefit is more apparent in those Member States where the policies reducing emissions in these sectors are devolved to a regional level.

Costs from the implementation of the ESD come from two main sources – implementation of policies and measures in relevant sectors and ESD reporting.

Costs associated with the implementation of national GHG mitigation policies in the ESD sectors

Data gathered during the evaluation suggests that of four main ESD sectors (transport, direct emissions from buildings, waste and agriculture) the most cost-effective abatement is found in buildings, with measures in the other sectors being more expensive. In particular some measures in the agriculture sector have a very high cost per tonne of CO2 reduced. 34  

However, these findings have a high level of uncertainty as the evidence comparing cost effectiveness in ESD sectors is relatively scarce and where it does exist has often been done on different timeframes and potentially with differing methodologies making comparison difficult.

Data reported by Member States includes information on the cost, and cost-effectiveness of GHG mitigation policies when available but provide insufficient information on their expected and actual costs of such policies. Of the 123 individual policies reported by Member States that were identified as being linked with the ESD, cost data was only provided for 9 policies. This severely limited the value of this data source in assessing the cost-effectiveness of policies implemented in response to the ESD.

Furthermore, the uncertainty is increased by a number of factors that make comparisons of costs of climate policies challenging, including uncertainty around the extent to which additional costs incurred by businesses are passed on to consumers, use of different models and differences in definitions of what is included in the term ‘costs’ (e.g. investment costs, operation and maintenance costs, subsidy payments, government administrative costs and revenues from energy savings).

Costs associated with reporting

The most tangible costs related to the ESD are those related to the reporting and compliance requirements for national governments. The cost for the Commission and the European Environment Agency of the current system for reviewing emission inventories and checking compliance is in the order of 650.000 EUR per year, while the annual costs for all 28 Member States taken together is estimated to be in the order of 500.000 EUR per year. These costs include administration of other reporting obligations under the ESD, which are separate from the emissions reporting and compliance check. 35 .

The ESD has no direct reporting obligations for business or SMEs.

Role of the flexibility instruments under the Effort Sharing Decision

The flexibility instruments, including transfers of AEAs among Member States, together with legally binding annual targets in the ESD were expected to be an important driver of cost efficiency under the ESD. This was also generally acknowledged by stakeholders although some expressed concern that the flexibilities may undermine some of the stimulus that the ESD provides to take domestic action.

When Member States reported in 2014 on their intentions to engage in AEAs transfers only 3 Member States indicated that they were planning to purchase AEAs, whereas 10 Member States intended to sell.

One of the reasons for the lack of a market developing under the ESD might be linked to the fact that most of the Member States are on track to meet or overachieve their targets. The low demand for AEAs might also be due to the fact that the ESD is still in the early years of the implementation and that there might be other drivers, not investigated by the evaluation, that would prompt Member States to prefer the implementation of domestic actions rather than buying AEAs from other Member States.

The fact that Member States only to a limited extent intend to use these instruments for compliance with their emissions limits might indicate that the ESD has missed on its objective of promoting mitigation actions in the places where this would be the most cost efficient. The supply indicated by Member States in 2014 (around 80 million) of AEAs was 8 times higher than the indicated demand (around 10 million). This reflects the expectation of most Member States that they will meet their 2020 targets under the ESD without any need for acquiring AEA transfers from other countries in the EU.

7.4.    Coherence

The objective of the ESD is expected to be supported by other EU-wide climate and energy policies. As noted earlier, the implementation of the Decision is still at a relatively early stage and consequently there is yet limited evidence on how coherently the different climate and energy initiatives have been implemented. There is a lack of studies specifically relating to the ESD and its coherence with other PAMs in the published literature. Thus the evaluation to a large extent relied on views expressed in the stakeholder consultation, which in itself was a relatively small sample size.

Evidence from stakeholder interviews, and to a lesser extent the literature review, suggests that the objectives of the ESD are largely coherent with other EU climate and energy policies, such as the ETS, energy efficiency and renewable energy. Stakeholders identified a strong coherence with the EU objectives relating to energy efficiency and renewable energy, although some stakeholders queried the coherence of the targets themselves.

By providing flexibility as to how Member States meet their emission limits, the objectives of the ESD was found to be largely coherent with national policy making.

There is a potential lack of coherence between the ESD and other interventions in relation to agriculture and land use, land use change and forestry (LULUCF). Part of the issue with coherence relates to other international commitments, rather than coherence with EU policies; agriculture is included in the ESD and Kyoto Protocol, while LULUCF is in the Kyoto Protocol but not the ESD.

Coherence with other reporting obligations was identified as strong, both with EU internal requirements under the MMR and with international reporting requirements under the UNFCCC. Some streamlining opportunities of reporting obligations under the ESD and under EU energy policies were identified. One such example is the requirements to report on projections both under EU climate and energy legislation, which possibly could be streamlined into one single obligation.

7.5.    EU Added Value

The evaluation examined to what extent the changes brought by the Decision could have been achieved by national or individuals' measures only, and how it is contributing to the EU’s international commitments under the Kyoto protocol’s second commitment period (2013-2020).

There was a strong indication in the stakeholder consultation that binding national targets, and robust requirements for monitoring and compliance in the ESD were important features to stimulate national action, For a large majority of EU Member States there were no or only weak domestic policy drivers before the ESD.

The evaluation indicated that Member States (e.g., Germany, UK, Denmark, Sweden and Finland) having more ambitious domestic targets than the ones under the ESD enshrined in their own national legislation may have taken action anyway in response to those national laws and associated targets. Although the evaluation did not account for the fact that the EU discussions and agreement on the EU 2020 targets in 2007 may have helped support the setting of national targets in the first place, by providing certainty that other EU Member States will be taking a certain minimum level of action. However, for a large majority of EU Member States this domestic driver did not exist before the ESD, which suggests that without the Decision the actions may not have been taken, or may have been taken at a slower pace. Even where other drivers of action were in place within Member States, the stakeholder consultation concluded that the ESD might have exerted an additional positive influence, even if it was not a primary driver.

Another important added value of the ESD to date has been the improvement in the quality of the emissions data and projections relating to ESD sectors at a national level, which has helped improving policy preparation. The annual reporting of emissions, combined with biennial requirements for reporting of policies and measures, and projections, keep Member State better informed about the progress not only of GHG emissions but also of climate and energy policies. The reporting obligations also give them and other EU stakeholders a tool to compare their performance with that of other countries in the EU.

The ESD is important instrument to ensure that the EU and its Member States deliver on their international obligations under the UNFCCC’s second Kyoto protocol, which has the same commitment period as the ESD (2013-2020). 36 As shown in Section 7.2, the EU is well on track to meet their Kyoto commitments.

7.6. Competition

The evaluation examined whether domestic policies and measures implemented by Member States in response to the ESD may have distorted the internal EU market. Due the lack of specific data regarding the impact of such national measures on the EU internal market, the evaluation outcome is based solely on stakeholder's opinions.

The majority of stakeholders indicated that the ESD has had no or limited impact on competition within the EU. Two respondents indicated that the national mitigation policies induced by the ESD may restrict the potential for growing output from the agriculture sector because of a perceived lower mitigation potential in that sector. However, no stakeholders offered examples of where national policies and measures resulting from the ESD had any adverse impact on specific enterprises or parts of a sector.

8.    CONCLUSIONS

The Effort Sharing Decision is still in the early stages of implementation. Nevertheless, it seems clear from the evidence gathered so far that the ESD targets together with the other initiatives under the Climate and Energy Package have been effective in stimulating new national policies and measures promoting effective reductions of GHG emissions within the scope of the ESD.

The objectives of the ESD remain relevant and still correspond strongly with the needs of the EU. The European Council Conclusions in October 2014 on the 2030 Climate and Energy Policy Framework confirmed the importance of the ESD and of its continuation after 2020 with all its main elements.

There was strong consensus among stakeholders that the ESD as an EU instrument to promote GHG emission reductions in a cost-effective and fair manner should be continued. There was also a strong indication that binding national targets, flexibility instruments, and robust requirements for monitoring and compliance were important features to promote action in Member States.

With respect to effectiveness, the EU is on track to meet its 2020 GHG emissions reduction target in the ESD sectors. Based on Member States’ reports submitted in 2015, GHG emissions in 2013 and 2014 in all Member States were below their annual limits for these years. Total 2013 emissions covered by the ESD at EU level were 9.7% lower than the 2005 emissions. In 2014 EU emissions covered by the ESD further decreased to a level 12.9% below 2005 levels, which was below the EU-wide ESD target for 2020. This means that the EU is on track to meet its ESD target in 2020 and that Member States have contributed to the reduction in GHG emissions.

Most emission reductions since 2009 have come from technological changes and policies which have resulted in increased uptake of less carbon-intensive technology. This effect has been reinforced by the fact that the ESD was launched together with a number of other EU climate and energy initiatives as part of the 2020 package, in particular on energy efficiency and renewable energy.

The economic recession has also had an effect on GHG emissions in some ESD sectors to date, and these effects will continue to resonate until 2020. However, many sectors covered by the ESD are not directly impacted by the fluctuations in GDP and are more prone to policy influence. For several of the ESD sectors, including buildings, transport, agriculture and waste, part of the emissions reductions to date can be attributed to factors that are influenced by policy interventions at EU and member state level related to the 2020 package. Whilst it was possible to identify that the ESD has had some effect in stimulating new national policies in some Member States, there was insufficient evidence to quantify the overall impact of the ESD on GHG emissions at this stage.

The evaluation of the ESD showed an increase in the implementation of national policies in the ESD sectors in most years starting from 2007, when the European Council agreed on the overall EU climate targets for 2020. Without the ESD, actions to mitigate emissions in the ESD sectors at Member States level may not have been taken, or may have been taken at a slower pace.

Evidence on the direct costs and benefits and the efficiency of national policies implemented in response to the ESD is very limited; it was not possible to assess these costs with confidence. This is partly due to the fact that national policies and measures reported by Member States so far have provided insufficient information on expected and actual costs and benefits.

With respect to reporting costs, the ESD was found to deliver the outcomes efficiently, although there may still be some opportunities for reducing administrative burdens. Costs related to reporting and compliance are modest and mostly fall on the Commission and the European Environment Agency. There was no major variation in terms of administrative costs apparent between Member States. There may be opportunities for reducing such costs at EU level, for example by simplified or less frequent compliance controls.

Evidence from stakeholder interviews, and to a lesser extent the literature review, suggested that the objectives of the ESD are coherent with other EU climate and energy policies, such as the ETS, energy efficiency and renewable energy.

By providing flexibility as to how Member States deliver their emission limits, the objectives of the ESD are largely coherent with national policy making.

Coherence with other reporting obligation was also identified as strong especially with the MMR and the reporting requirements under UNFCCC. However, streamlining opportunities of reporting obligation under the ESD and under EU energy focused legislation were identified.

The ESD was found to add value through action on EU level. There was a strong level of agreement among Member State stakeholders that the ESD raised awareness of mitigation potential in ESD sectors and contributed to establishing new national institutional and legal frameworks. It also improved coordination on GHG mitigation across the ESD sectors and between national and regional or local governments.

There was strong consensus from stakeholders that there was a continued need for an instrument such as the ESD also for the period after 2020.

Stakeholders did not present any evidence that national policies resulting from the ESD have unduly distorted competition in the EU internal market.

9.    ANNEXES

Annex 1: procedural information

The Directorate-General (DG) for Climate Action was leading the evaluation of Decision No. 406/2009/EC (the Effort Sharing Decision, ESD) within the European Commission.

An inter-service Steering Group chaired by DG Climate Action was set up in January 2015 for guiding the evaluation. The following DGs and services were invited to the Steering Group: Secretariat General (SG), Legal Service (SJ), Agriculture and Rural Development (AGRI), Competition (COMP), Economic and Financial Affairs (ECFIN), Energy (ENER), Environment (ENV), Internal Market, Industry, Entrepreneurship and SMEs (GROW), Joint Research Centre (JRC), Mobility and Transport (MOVE), Regional and Urban Policy (REGIO), Research and Innovation (RTD), Taxation and Customs Union (TAXUD), Trade (TRADE).

The Steering Group agreed on the evaluation mandate and the terms of reference, including the intervention logic, for the consultants contracted for carrying out an external evaluation. The European Environmental Agency was also invited to participate in the Steering Group. The group met five times in the period from March 2015 to December 2015.

The evaluation mandate was included in the Commission’s agenda planning and work programme for 2015 and was published on the Commission’s Europa web pages. 37  

The external evaluation of the implementation of the Effort Sharing Decision was carried out from April to December 2015 by a team of external consultants led by Ricardo Energy & Environment and contracted by the Commission. 38

The steering group provided guidance to the external consultants at different stages of the evaluation and quality assessed the evaluation report. The group’s quality assessment of the evaluation report considered its relevance, design, use of data and sources, analysis and conclusions.

The external evaluation report and the quality assessment are available on the Commission’s Europa web pages. 39

Annex 2: stakeholder consultation

The stakeholder consultation was carried out as part of the external study supporting the evaluation. A detailed description of how the consultation was organised and its result is provided in Appendix 1 and Appendix 2 to the study, 40  

 Organisation of the consultation

Stakeholders have been consulted using four mechanisms: general questionnaire survey, administrative burdens questionnaire, EU public consultation, stakeholder interviews. The scope and audience for each of the mechanisms is described below.

All stakeholder types were given the opportunity to participate in the evaluation. However, the following stakeholder groups were specifically targeted:

1.EU policy makers i.e. individuals within the Commission who may be involved in the development or implementation of the ESD and related policies, and individuals from the EEA involved in the monitoring of the ESD.

2.National ESD administrators i.e. the individuals involved in the implementation of the ESD in Member States.

3.National policy makers i.e. the individuals within national, regional and local governments who may be involved in the development or implementation of national GHG mitigation policies.

4.National inventory compilers i.e. individuals within Member States who are involved in compliance activities associated with the monitoring and reporting of emissions data.

5.Wider interest groups (Industry associations, academia and civil society).

The general public was not actively targeted by the evaluation. However, this stakeholder group was able to provide views by responding to the EU public consultation.

The evaluation sought to achieve a balanced representation of views from within each of the different stakeholder groups. For example, the representation of Member States aimed to cover the following different circumstances:

Member States with more stringent domestic emissions reduction targets than those required under ESD.

Member States that have, following the introduction of the ESD, been projected to meet their ESD emission limits without the need for further domestic action.

Member States that have, following the introduction of the ESD, been projected to miss their ESD emission limits without further domestic action.

Member States whose ESD emissions are biased towards a single source or sector e.g. agriculture.

Member State where control of GHG emission is a regional or local responsibility.

In relation to the wider interest group, the evaluation sought to achieve an equal balance between non-governmental organisations and industry associations. Likewise, representation was targeted from organisation representing each of the main sources/sectors within the scope of the ESD (i.e. transport, agriculture, buildings).

Invitations to participate in the evaluation were sent to a total of 224 individuals, representing each of the different stakeholder types. To achieve the required balance in representation, additional targeting and encouragement to participate in the evaluation was provided to certain groups.

Each of the main consultation tools are summarised below.

1.General questionnaire survey

Scope: This web-based questionnaire sought views from stakeholders on a number of the evaluation questions. The questionnaire therefore provided a structured response, based on closed multiple-choice answers. Open-response answers were allowed for certain questions.

Audience: Invitations to complete the survey were sent to stakeholders in Member States, Industry Associations and NGOs. The questionnaire was accessible behind a password protected website.

2.Administrative burdens questionnaire

Scope: This was a separate MS Word questionnaire that sought views on just the administrative burdens associated with the implementation of the ESD.

Audience: The survey targeted those individuals within Member States who are involved in the implementation of the ESD at national level, in particular monitoring and reporting.

3.EU public consultation 41

Scope: The Commission launched a public consultation on the preparation of a legislative proposal on the effort of Member States to reduce their GHG emissions to meet the European Union's GHG emission reduction commitment in a 2030 perspective.

Audience: Open to all stakeholders, including the general public.

The responses to this consultation, where relevant to the evaluation questions, have been taken into account as part of the analysis. This comprised between 78 and 100 responses (depending upon the question).

4.Stakeholder interviews

Scope: There is a need to explore certain issues relevant to the evaluation in more depth. The interviews covered all of the evaluation questions, but were tailored to the audience concerned.

Audience: This involved a detailed discussion with a limited number of key stakeholders including stakeholders in the EC, Member States, Industry Associations and NGOs. The list of stakeholders was informed by the data and literature review, for example, if an NGO had commissioned research or prepared a position paper relating to the ESD then this demonstrated that they had a good understanding of the ESD so were on the priority list of stakeholders. Likewise, for the business associations prioritisation was given to organisations representing business within the main ESD target sectors (e.g. transport, agriculture, buildings, industry) as well as organisations representing small and medium sized enterprises. However, ultimately, interviews could only be held with stakeholders who were interested in participation, and it proved challenging to persuade some stakeholders, particularly business associations to participate.

Results from the questionnaire survey and stakeholder interviews

Overview of respondents to the questionnaire survey

In total, 35 complete responses were received to the questionnaire survey, including one written response. In addition 7 partial responses were submitted, but have not been taken into account. In almost all cases of partial responses the respondents failed to complete more than the initial registration pages i.e. they stopped once they got to the technical questions, which indicates that they may not feel sufficiently knowledgeable to provide views on the questions posed. Given that the questions were specific to the ESD, and there is only a relatively small community of individuals who are actively involved in ESD implementation, this is an unsurprising outcome. It also helps to explain the relatively low response rate overall in comparison to say policies with a more clear and direct impact on multiple stakeholder groups e.g. product regulations.

The majority of respondents were from national government, which was the primary target group for the questionnaire. The second largest group was non-governmental organisations with 9 respondents. A further 3 respondents were from academia or research organisations, and another 3 organisations represented business. There were 2 respondents from international administration, both from the EC. One further respondent described themselves as ‘Other, public services’.

On one hand the overall response rate was low compared to the number of invitations to participate that were sent out to (224). However, given that only a relative small cohort of individuals who have been directly involved in the implementation of the Decision, the response rate is considered reasonable. In particular, responses were received from 18 (including one written response) of the 28 Member States.

One potential barrier, which may help to explain the low response rate, was the timing of the questionnaire. The questionnaire was open from May until the end of August, which coincided with the holiday season. To address the time constraint the original deadline for the consultation period was extended (the original deadline was the end of July). In addition, the questionnaire was launched at the same time that stakeholders were asked to provide responses to the public consultation. There was therefore a potential risk that stakeholders may have got confused between the different consultation activities. This was mitigated as far as possible by clearly explaining about the different surveys in all communication. A related issues was the risk of consultation fatigue – whereby respondents may have been less willing to participate in the questionnaire survey having recently spent time responding to the public consultation. It is not clear if this was a real issue, but may explain the low initial response rate.

Note: does not include one written response from a national government

In relation to the ESD related experience of the respondents, the experience most frequently stated was involvement in the implementation of the ESD, involvement in climate change policy making and involvement in ESD monitoring and compliance. This response mirrors the stakeholder distribution of stakeholder groups described above, since all of these activities are the responsibility of the national government actors.

One respondent indicated that they had no direct involvement in ESD implementation to date. These views are nevertheless still considered valid – since the questionnaire allowed respondents to answer don’t know to questions they didn’t feel knowledgeable enough to answer.

Overview of audience for the stakeholder interviews

In total 24 interviews were held with stakeholders. Of these, 18 were follow-up interviews with selected respondents to the questionnaires survey. The specific stakeholders were selected based on their response to the questionnaire e.g. if they made an important point that was worth following up, or if they didn’t provide explanation of their views so more detail was required. Consideration was also given to the type of stakeholder (to ensure a balance of stakeholder views) in the selection process. The remaining 6 interviews were with stakeholders on the priority list who had not accepted the invitation to complete the questionnaire survey. The interviews were therefore offered as an alternative means by which they could participate in the evaluation. These are highlighted as new stakeholders in the table below.

The interviews were distributed amongst the stakeholder groups as follows:

Relevance of the Decision

Are the objectives of the ESD still relevant?

This question formed part of the stakeholder interview and views from all stakeholder groups.

There was a strong consensus that the overall objectives of the ESD are still relevant. Stakeholders expressed the view that these objectives are consistent with the overarching EU 2020 targets, and that an instrument such as the ESD with binding national targets is important in ensuring the EU wider targets are met. Some respondents, however, suggested that the ESD was less relevant currently as most Member States can meet their target without any additional measures though it was recognised that the ESD will be more important in the future. One stakeholder indicated that this is consistent with the European Council Conclusions that call for the ESD to be continued for a further 10 years.

Effectiveness of the Decision

A series of questions were asked in relation to the effectiveness of the ESD.

Do you think the EU is on track to deliver its 2020 GHG emissions reduction target for the non-traded i.e. ESD sectors?

There was a strong consensus that the current evidence indicated the EU was on track to meets its 2020 target for the ESD sectors. Member States latest emissions projections, as well as EU wide modelling results from the PRIMES and GAINS models, were cited as evidence to support this view. However, it was noted that the situation differs between the individual Member States, with some Member States being more on track than others. These conclusions also represent the current expectations and may change in the future.

Stakeholders also highlighted that one of the outcomes of the ESD to date is an improvement in the quality of the emissions data and projections relating to ESD sectors at a national level. This has itself provided more robust data to enable stakeholders to more accurately draw conclusions on whether Member States are on track.

What do you consider are the main factors that have influenced whether the EU is currently on track or not?

Responses were received from all the stakeholder groups, with 12 responses received in total.

Stakeholders identified a range of factors which have influenced whether Member States are currently on track to deliver their emissions limits under the ESD. The introduction of national policies, which is one of the expected impacts of the ESD, was one of the four main factors cited. The other main factors were the economic recession, the impacts of EU wide policies, and the overall stringency of the limits themselves. There was no clear factor that was cited more frequently that the others, and respondents frequently cited more than one factor.

How effective has the ESD been in promoting reductions of GHG emissions to help deliver the target?

Has the ESD been more effective in promoting reductions in some sectors than in others? If so why?

In practical terms, what have been the main activities or developments that have been stimulated (e.g. within Member States) by the ESD?

Has the ESD supported national and regional GHG policy developments?

Do you think the existence of the ESD has made it easier to agree additional mitigation actions in the EU and/or Member States?

This question was asked as part of the stakeholder interviews. Responses were received from 7 stakeholders, including representatives from national governments, the EC, non-governmental organisations and industry associations.

There was agreement amongst stakeholders that the ESD has had a positive influence on additional mitigation actions at Member State level, although the strength of this influence appears to vary from one Member State to the next. The strength varied between being perceived as an important stimulus for additional action, to only stimulating discussions amongst different government department on mitigation actions in ESD sectors.

Do you think the ESD has had any influence on the cooperation between central and regional/local authorities working in Member States on GHG mitigation issues?

Aside from the impacts on GHG emissions, what other positive impacts have the ESD had at Member State level?

Has the ESD had any negative impacts at Member State level?

Only a few examples were identified, and none more than once. These were: potential for competitive distortions, negative air quality impacts from a switch to biomass, the use of flexibilities reducing the motivation for domestic action, and potential conflicts with other policy objectives. One broader issue that was highlighted by a stakeholder was the potential conflict at Member State level between economic development objectives, and the need to limit emissions. It was considered that this has not been a major conflict to date, however, it may become more so in the future.

Has the ESD affected competition within the EU, for example, by affecting EU internal markets?

What other wider impacts has the ESD had within Member States that are not captured in the lists above?

It was suggested that ESD has given a better awareness of GHG mitigation measures. Also, implemented policies such as building regulations were stated as being beneficial to SMEs as they carry out most of the installations. One respondent noted that it potentially reduces the impact that individuals can have on emission reductions via a change in lifestyle or product choice.

Has the ESD promoted GHG emissions reductions in a cost-effective and fair manner?

Efficiency

As described in the section above, there are a number of benefits associated with the delivery of the objectives of the ESD. At the same time, there may be certain costs associated with the implementation of the ESD. The following questions refer both to the costs incurred to date, and to predicted costs to 2020.

What costs have been associated with the implementation of ESD?

Are there any costs which would have been incurred anyway due to existing GHG reporting obligations?

Three respondents indicated that most of the administrative costs associated with the ESD would have occurred anyway in response to other reporting requirements. However, some stakeholders identified specific costs, including the annual inventory review, as being additional. Two respondents stated that none of the costs would have been incurred anyway, although no further details were provided on the specific costs incurred. This would suggest that the ESD is associated with additional administrative costs, but at least a proportion of these would have been incurred anyway.

Are the costs of ESD implementation proportionate to the benefits from the ESD?

What opportunities exist to reduce the costs of implementation, whilst also delivering a similar level of benefits?

Several stakeholders suggested options for reducing the costs of implementation of the ESD, whilst also delivering a similar level of benefits. Most of these related to the reporting and compliance requirements, although some others related to wider issues. In relation to the reporting and compliance requirements, several stakeholders suggested that the compliance checks could be made less frequent, although one cautioned that this may reduce the effectiveness. Other respondents suggested there may be options to develop tools for more efficient reporting, and further harmonisation with other international reporting requirements. In relation to the wider costs of delivery, one stakeholder suggested that the creation of a more transparent and liquid market for AEAs would help reduce the overall costs. Another stakeholder suggested that raising awareness of the ESD at EU level would result in greater discussion at national level, which would facilitate greater action. Another suggestion was to share best practice more widely amongst national administrators.

What role did the flexibility mechanisms (e.g. banking and borrowing) play until now?

Stakeholders were in agreement that flexibilities had played a limited role to date, although several respondents stated that this role will increase in the future. The reasons for the limited role were that Member States had not been required to use them for compliance with their emissions limits. The influence of the economic crises on emissions was cited as an important influence on this outcome, and this had not been expected at the outset.

Do you think the use of the flexibility mechanisms has helped (or will help) the ESD to deliver emissions reduction in a cost effective and fair manner?

All stakeholders agreed that the flexibility mechanisms have an important role in delivering emissions reductions at the EU28 level in a cost-effective manner. However, two stakeholders expressed concern that the flexibilities may undermine some of the stimulus that the ESD provides to take domestic action, with the use of external credits reducing this pressure further. This is consistent with concerns expressed by other stakeholders in response to other evaluation questions. The importance of flexibilities in evening out inter-annual fluctuations in GHG emissions (e.g. in response to weather) was identified as a key factors requiring the need for such flexibilities.

Coherence

The emission sources and sectors within the scope of the ESD are also targeted by other EU wide GHG PAMs, which were introduced with the ESD as part of a package of instruments to reach the EU's 2020 targets. A series of questions were asked to explore whether stakeholders considered that the ESD was coherent with this package of policies, and their individual requirements.

Is the ESD, and its associated activities, coherent with the current EU climate and energy policies in the 2020 package?

Is the ESD coherent with national policies and strategies in the target sectors?

Are the reporting obligations under the ESD coherent with other GHG monitoring and reporting obligations under EU legislation?

Added value of EU action

The consultation also explored the added-value of EU action as compared to Member State action alone. A number of the following questions are related to those explored in the effectiveness section, as they concern the overall added value of the Decision.

Could the specific actions that have been taken, since the ESD came in to force (June 2009), to limit EU-wide GHG emissions by 2020, have been achieved by national-level measures alone?

Does there continue to be a need for an instrument, such as the ESD, to promote reductions in GHG emissions in the ESD sectors across the EU?

What would be the most likely consequences of stopping or not continuing the ESD?

Annex 3: Methods and analytical models use in preparing the evaluation

The ESD came into force in June 2009, and the scope of the evaluation concerns the impacts of the Decision to November 2015. While the ESD has been in place for some time, some of the main requirements have only happened for the first time in 2015, and others will not happen until 2016. The methodology has been designed to gather evidence on the performance of the ESD to date, but the limited evidence on some elements of the ESD needs to be recognised.

The following methods were selected as most appropriate for the external evaluation, and were subsequently implemented, including analysis of evidence:

Policy literature and data review

Questionnaire surveys

EU public consultation

Stakeholder interviews

Case studies

As far as possible, evidence has been gathered from multiple sources, to enable the comparison and triangulation of key findings. Each of the methods is described briefly below.

I.Policy literature and data review

This is a key source of evidence. It includes policy documents, data reported by Member States as part of the on-going monitoring of the Decision and wider literature relevant to the intervention logic.

1.Sources of evidence

A key source of evidence that has informed the evaluation are the reports submitted by Member States under the Monitoring Mechanism Decision and, starting from 2015, Monitoring Mechanism Regulation (MMR). Member States are required to submit information on GHG inventories (Art 7), projections (Art 14), PAMs (Art 13), projections and information on the use of flexible mechanisms. This information is aggregated by the EEA in, among others, the ‘Trends and projections in Europe’ report and the EEA database on national climate change mitigation policies and measures (PAMs) in Europe. 42

Policy documents provide evidence on the EU climate policy context, which is valuable for assessing the relevance and coherence of the ESD. For example, the 2008 Impact Assessment of the Climate and Energy Package provides important evidence on the expected impact of the Decision prior to its implementation, and its interaction with the other policies in the Climate and Energy Package.

The literature review has also drawn upon existing research into the historical drivers of GHG emissions across Europe, including modelling studies. This included previous research by the EEA on the different drivers of GHG emissions in the EU, as well as the initial results from an on-going CLIMA study on the same issue. The existence of this research also removed the need for any new modelling work to be performed as part of this evaluation study. 43  

Finally, there are number of previous studies that have reviewed and compiled information on the implementation of PAMs within Member States.

The evaluation was therefore able to benefit from a well-established monitoring framework in relation to GHG emission trends and policies measures, and also a strong body of previous research on GHG emission drivers, and national policy measures. This reduced the need for extensive collection of new data, or modelling. However, it was still necessary to check and synthesise the data, and then validate the research conclusions with other sources. It was also necessary to identify additional data sources to fill certain gaps in data.

To fill data gaps a wider review of relevant publications and research reports was performed. This included a review of relevant publications, articles and web sites. 44

2.Limitations in evidence and key uncertainties

As far as possible, evidence was gathered for each of the evaluation questions. However, for some of the questions, for example, that relating to the wider impacts of the ESD, the evidence from the literature was more limited. This in particular related to evidence that was specific to the ESD i.e. studies that specifically related to the ESD, rather than climate policies more generally. To address this limitation, the analysis drew upon broader literature on the impacts of climate change policies more generally, in sectors within the scope of the ESD. This assumes that this broader literature is representative of the impacts associated with the ESD, including impacts associated with national policies introduced in response to the ESD. Whilst this is a plausible assumption, given that the ESD does not specify how Member States should meet their emission limits, it has not been possible to test it. Therefore, where evidence has drawn upon these more general sources the conclusions should be treated with greater uncertainty. In particular, it is uncertain whether the impacts may have arisen as a result of the ESD, or as a result of other drivers.

The literature review also aimed to gather evidence on the effectiveness of the ESD for the EU as a whole, but then to validate the findings with evidence for individual Member States. This validation was possible for some evaluation questions and not all. Likewise, where the analysis has drawn upon results for individual Member States the results cannot be assumed to be representative for other Member States, or the EU as a whole. Nevertheless, the results from the national studies are generally supportive of the results from the EU wide analysis, and where discrepancies have been identified these have been clearly described. Where validation has not been possible, conclusions should be considered more uncertain.

Some key sources of evidence are reports and the results from previous modelling work, which have been produced by other organisations. It has not been possible within the scope of the current study to critically appraise these studies, and their limitations.

II.Questionnaire surveys

1.Sources of evidence

To gather the views and opinions of stakeholders in relation to the individual evaluation questions two targeted questionnaire surveys have been used. The first was a general survey which covered all of the evaluation questions, and was targeted at all stakeholder groups (as listed below). The second was a specific survey on the administrative burdens associated with the implementation of the ESD’s reporting requirements by Member States, such as reporting obligations.

Based upon previous studies, for example capacity building workshops on GHG mitigation policies in ESD sectors that DG CLIMA organised for Member States in 2014, it was recognised that the population of individuals directly involved in ESD implementation at a national level was rather limited. Therefore, it was not expected that the questionnaire survey would have a very high response rate (in comparison to consultations on policies with a more direct impact on different stakeholder groups, such as product regulations, for example). The priority for the general questionnaire was, therefore, to achieve a good coverage of those individuals who have been directly involved in the ESD implementation. At the same time, the aim was to ensure other stakeholders groups, including non-governmental organisation and business association, also had the opportunity to participate. It was though recognised, based on experiences with the capacity building workshops, that the interest from these groups was likely to be weaker.

A priority list of stakeholders was developed, comprising individual and organisations that were known to have been directly involved in ESD implementation, along with a secondary list of wider stakeholders. The stakeholder list drew upon the contacts list for the capacity building workshops, and was supplemented through further desk research.

General survey

The general survey was aimed at four stakeholder groups:

EU policy makers i.e. individuals within the EC and the EEA who may be involved in the development or implementation of the ESD and related policies;

National ESD administrators i.e. the individuals involved in the implementation of the ESD in Member States;

National policy makers i.e. the individuals within national, regional and local governments who may be involved in the development or implementation of the policy; and

Wider interest groups (industry associations, academia and civil society)

Administrative burdens survey

A separate survey was used for gathering data on the administrative costs, in particular with respect to Member States’ reporting obligations under the ESD. Since the cost of implementing most of the requirements of the ESD is only likely to be known by a small group of individuals in each Member State, a tailored questionnaire survey was developed and sent to national administrators with each of the Member States.

In the assessment of the administrative burdens it was important to take account of the overlaps with other Member State reporting obligations. In particular, it was necessary to determine what activities were Member States obliged to perform anyway as part of their EU and International reporting obligations, and what requirements were solely arising from the ESD.

A total of 9 Member States replied to this survey: Austria, Cyprus, Denmark, Estonia, Finland, Malta, The Netherlands, Sweden and Slovenia. Estimates were also provided by the EEA and the EC of their administrative costs.

2.Limitations in evidence and key uncertainties

The main limitation with the general questionnaire is that it is based on the views and opinions of the participating respondents. This presents a risk that the results can be biased both by unrepresentative samples, but also by protest responses. In relation to the first point the sample size for the general questionnaire was quite low, but this was not surprising given the specialist nature of the policy. Reasonable coverage was achieved from the primary target group (national governments), with responses from 17 of the 28 EU Member States. This included older and newer Member States, larger and smaller Member States, and a good geographic spread. Also a good response rate was received from the non-governmental organisations active in the climate change policy area. A more limited response rate was received from business associations. This means that the results are more strongly biased towards the views of national governments, and secondly non-governmental organisations. The view of business associations may be under represented, although these organisation were given equal opportunity to respond which may suggest weaker interest in the evaluation from this stakeholder group.

The general survey was sent initially to a short-list of priority target stakeholders including those individuals involved in the implementation of the ESD within national governments, individuals within the EC and its agencies that have been involved in ESD related activities, and also non-governmental organisations and business associations that had been involved in previous ESD related activities (e.g. workshops). However, the initial response rate to the questionnaire was very low. This could be linked to two factors 1) the questionnaire was issued shortly after a related public consultation (see below) so respondents were asked to complete two surveys on similar issues 2) the questionnaire was issued during the summer holiday season which is typically a more difficult time to achieve high response rates. To address the low response rates the timescales for the questionnaire were extended, and the invitation to participate in the questionnaire was issued to an extended mailing list (224 invitees in total). In addition, where responses were not received from the priority stakeholder list, a telephone interview was offered to the stakeholders (see below) as alternative means of contributing to the evaluation.

Despite these efforts a large number of respondents who were invited to participate rejected the offer. In relation to the primary target audience some national governments (2 Member States) stated explicitly that they did not wish to participate in the survey. In the remaining cases no response was received despite numerous prompts. In 7 cases respondents begun to complete the survey, but then did not proceed further once they had understood better the main topics and questions that were covered by the survey. These incomplete responses largely came from stakeholders from the expanded list. This reinforces the point that the subject matter was rather specialist in nature, and the cohort of individuals with relevant expertise was relatively small.

No obvious responses were identified that could be considered a protest response.

Similar limitations exist with the administrative burdens questionnaire. The overall response rate was relatively low (9 of the 28 Member States). This may have reflected the timing of the survey. The individuals within national government whose input was required were very busy finalising their national inventories for submission to the UNFCCC at the time of the consultation. A further limitation relates to the potential bias in the responses. One response to the questionnaire included a much larger estimate of the number of days required to perform the activities, than for the other responses. To address this potential bias, this estimate was excluded when aggregating the costs at an EU level. See Appendix 2.

III.EU public consultation

1.Sources of evidence

The Commission launched a public consultation on the preparation of a legislative proposal on the effort of Member States to reduce their GHG emissions to meet the European Union's GHG emission reduction commitment in a 2030 perspective. The consultation ran from 26 March 2015 to 18 June 2015. As part of this consultation there were questions relating to the current ESD. The responses to this consultation, where relevant to the evaluation questions, have been taken into account as part of the analysis. This comprised between 78 and 100 responses (depending upon the question)

2.Limitations in evidence and key uncertainties

Similar limitations exist to those described for the general survey above, with the potential for bias in the results. However, since the results from the public consultation were not focussed on specific evaluation questions, the evidence is generally less relevant, so has had a more limited influence on the overall conclusions.

IV.Stakeholder interviews

1.Sources of evidence

Stakeholder interviews allow a more detailed exploration of the evaluation questions. These interviews therefore provide important contextual information to explain the wider statistics/indicators. The interviews were also used to develop case studies which illustrate specific outcomes.

An interview plan was developed with targeted representation from each of the different stakeholder groups, as well as stakeholders in different circumstances. For example, interviews with Member States included those countries that had been projected to meet their ESD targets without the need for new policies since its introduction, but also countries which are expected to miss their targets without further action.

In many cases the stakeholder interviews were held with the same individuals that completed the general questionnaire. In total 24 interviews were held with stakeholders. Of these, 18 were follow-up interviews with selected respondents to the questionnaires survey. In these cases the interviews allowed more detailed exploration of the issues raised in the questionnaire. The remaining 6 interviews were with stakeholders who had not completed the questionnaire survey. These are highlighted as new stakeholders in the table below.

2.Limitations in evidence and key uncertainties

The main limitation with the evidence from the stakeholder interviews is that is based on the opinions of the stakeholders, and not empirical evidence. Nevertheless, these views still provide a valuable form of evidence, particularly for those evaluation questions that cannot be tested empirically. Furthermore, the interviews were valuable in identifying additional sources of research, as well as validating and further explaining the findings from the literature review.

As with the general questionnaire, the results from the stakeholder interviews can suffer from potential bias. This was mitigated by ensuring that the sample covered each of the relevant stakeholder groups, and also relevant stakeholder types within each stakeholder group. For example, the interviews with national governments included representatives from large and small Member States, from EU-12 and EU16, and from Member States who are projected to meet their ESD target easily, and those that need to implement additional measures. This distribution helps to ensure that views are representative. However, for other stakeholder groups the sample size cannot be considered representative. In particular, the views of business associations are potentially under-represented, although the lack of interest from this group may indicate that views are less strong in general. This view was supported by the interviews themselves, with a generally lower level of understanding and awareness of the ESD from this group. This potentially reflects the lack of a direct impact of the ESD on businesses, with any impacts arising indirectly though national policies

The overall sample size for the stakeholder interviews was relatively low, so there remains a risk of bias towards the views held by certain stakeholders. However, the views from stakeholder did not indicate a strong bias for most evaluation questions. A possible exception was that non-governmental organisations tended to highlight more strongly the benefits of climate policy and the need for stronger mitigation action. However, this view was not exclusive to non-governmental organisations, and was balanced out in most cases by views of other stakeholder groups.

V.Case Studies

1.Sources of evidence

Case studies were used to illustrate interesting responses to some of the issues covered in the evaluation. The case studies were identified based on responses to the general questionnaire, and also through the review of wider literature. The evidence to complete the case studies was therefore based on published literature, which was then updated and validated though further engagement with relevant stakeholders.

The case studies were used to illustrate practical examples relating to the different aspects of the intervention logic. In particular, the case studies were used to explore the wider impacts of the Decision – since these impacts are difficult to quantify and may be context specific.

Three cases studies were developed, as shown below. Each was used to provide answers to some of the evaluation questions.

Table 1: Case studies for ESD evaluation

2.Limitations in evidence and key uncertainties

The main limitation with the case studies is that the findings are relevant for the particular case, but cannot be consider representative of other Member States, or the EU as a whole. The case studies are therefore illustrative of specific examples, but cannot be considered to cover either the full range of responses from Member State, or necessarily reflect a representative response. Indeed the selection of the case studies has an inherent bias towards those Member States that have taken action in response to the ESD. A case study on a Member State that are taking no or limited action has not been prepared. Thus the case studies are necessarily biased towards cases where action has been taken, and the intervention logic can be demonstrated. Whilst this is valid – and provides valuable insights into how the intervention logic has operated, this potential bias should be recognised.

For certain issues, such as the use of flexibility mechanisms there is insufficient experience to date to provide a useful case study. The case studies were therefore selected to cover the aspects of the intervention logic where further evidence from case studies was considered to be most valuable and sufficient evidence available to develop valuable findings. Nevertheless, the case study evidence only covers certain issues and is not comprehensive.

Annex 4: Provisions FOR FLEXIBILITY INSTRUMENTS AND MONITORING AND compliance in the Effort Sharing Decision

Flexibility instruments

The ESD allows a Member State to carry over (“bank”) any surplus annual emission allocations (AEAs) from a given year to subsequent years within the commitment period (until 2020). It can also carry forward (“borrow”) up to 5 % of its annual emission allocation from the following year. In this way, Member States can either cover emissions in excess of their AEAs in a given year by borrowing AEAs from allocations for future years, or, where the emissions of a Member State are below that annual emission allocation, it can bank unused AEAs from one year in order to offset future emissions in excess of the AEAs allocated in future years (Article 3.3 in the ESD).

These instruments provide flexibility to Member States within their own schedule of annual allocations in 2013-2020 and help them managing the use of their AEAs over the whole commitment period to cover any AEA shortage in specific years, for example, due to weather fluctuations. A higher borrowing rate is available to Member States for the years 2013-2014 under the current ESD in the event of “extreme meteorological conditions” (Article 3.3).

Member States can also trade with each other in the ESD. A Member State is allowed to transfer up to 5 % of its AEAs for a given year to another Member State at any time (Article 3.4). After the annual compliance check has been concluded it can transfer its entire surplus AEAs for a given year (Article 3.5). A receiving Member State may use these acquired AEAs for complying with its obligation under ESD for the given year or any subsequent years of the compliance period. A Member State cannot transfer any part of its annual emission allocation if, at the time of transfer, that Member State is not in compliance with the requirements of the ESD. Member States are obliged to report any concluded transfers to the European Commission, which then publishes a summary of such transfers as part of its annual progress report.

Finally, under Article 5 of the ESD, Member States can also use international project credits under the Kyoto Protocol’s Clean Development Mechanism and Joint Implementation to meet their commitments under the ESD. The use of such credits is limited on a yearly basis up to 3% of 2005 non ETS emissions in Member State. Member States that do not use their 3% limit in any specific year can transfer their unused part for that year to other Member States or bank it for own future use. Member States, which fulfil further criteria (AT, BE, DK, FL, IE, ES, IT, CY, LU, PT, SI, SE) may use CDM project credits up to an additional 1% of their verified emissions in 2005; these credits are not bankable and not transferable. 45

Monitoring of progress and compliance

Monitoring of progress and assessing compliance is based on a comprehensive framework of monitoring, reporting and verification (MRV) laid down partly in the ESD itself (Article 6) and partly in the Monitoring Mechanism Regulation 46 (MMR) and its implementing provisions. Member States are obliged to annually report their greenhouse gas emissions to the European Commission. The MMR and ESD also require Member States to report every second year on national policies and measures implemented in order to achieve their targets under the ESD and on their emission projections.

To ensure that the compliance assessment relies on accurate and verified data, the GHG emissions inventories submitted by Member States are reviewed by the Commission with support by the European Environment Agency as laid down in the MMR. The review includes quality assurance and quality control of Member States greenhouse gas emissions inventories. A team of technical review experts co-ordinated by the Agency checks the transparency, accuracy, consistency, comparability and completeness of the information submitted.

The annual compliance assessment is performed by the Commission by comparing the reported and reviewed emissions with the annual emission limits under the ESD. Before the compliance check, Member States can make use of the flexibility instruments (e.g., banking and borrowing, AEA transfers from other countries) to close any gap between their actual emission and emission limits for a particular year.

If a Member State would still not be in compliance after applying the flexibility instruments, a deduction from its emission allocation of the following year equal to the excess emissions (in tonnes of carbon dioxide equivalent) multiplied by an abatement factor of 1.08 is applied.

(1)

http://ec.europa.eu/clima/policies/strategies/2020/index_en.htm

(2)

Decision No. 406/2009/EC of the European Parliament and of the Council of 23 April 2009 on the effort of Member States to reduce their greenhouse gas emissions to meet the Community's greenhouse gas reduction commitment up to 2020; http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32009D0406&from=EN  

(3)

The current ESD covers the same greenhouse gases as the Kyoto Protocol, with the exception of nitrogen trifluoride NF3. Emissions or removals from land use, land-use change and forestry (LULUCF) are not included in the ESD. Emissions from international aviation and shipping are not covered by the ESD.

(4)

Article 14 of the Effort Sharing Decision states that the Commission shall draw up a report evaluating the implementation of the ESD, including how it has affected competition, and present the results to the European Parliament and to the Council by 31 October 2016.

(5)

SWD(2016) 247

(6)

Examples of relevant EU legislation are the Regulation of CO2 emissions from passenger cars and vans, the Landfill Directive, the Energy Performance of Buildings Directive, the Energy Efficiency Directive, the Renewable Energy Directive, the F-gas Regulation and Mobile Air-Conditioning Systems Directive, the eco-design framework, the Nitrates Directive, and the greening of the Common Agricultural Policy.

(7)

For more details of the flexibility instruments and the rules for monitoring and compliance, see Annex 4.

(8)

SEC (2008) 85/3 EC. Commission Staff Working Document: Impact Assessment accompanying the Package of Implementation measures for the EU's objectives on climate change and renewable energy for 2020. Available at http://ec.europa.eu/clima/policies/strategies/2020/documentation_en.htm  

(9)

 Supporting study for the Evaluation of Decision No. 406/2009/EC (Effort Sharing Decision), Ricardo Energy and Environment with Trinomic and Vito.

(10)

The methodology applied for the evaluation is presented in more detail in Annex 3 to this Staff Working Document.

(11)

 Supporting study for the Evaluation of Decision No. 406/2009/EC (Effort Sharing Decision), Ricardo Energy and Environment with Trinomic and Vito. Available at http://ec.europa.eu/clima/news/articles/news_2015060802_en.htm  

(12)

See Annex 1 for information on the evaluation procedure.

(13)

The results of the stakeholder consultation are presented in Annex 2 of the Staff Working Document.

(14)

For more information on this consultation, see http://ec.europa.eu/clima/consultations/articles/0025_en.htm  

(15)

DG CLIMA (2015) Decomposition analysis of the changes in GHG emissions in the EU and Member States.

(16)

Because of technical difficulties with reporting software provided by the UNFCCC, Member State submissions of GHG emissions inventories by the Member States, the first compliance check under the ESD was further delayed into 2016.

(17)

Decision 2013/162/EU and Decision 2013/634/2013.

(18)

These four years are the reference years for determining the AEAs for the period 2013-2020 as laid down in the ESD.

(19)

Regulation No 389/2013 establishing a Union Registry lays down rules in order to insure the accounting of transactions under the ESD. Member States accounts in the Union Registry were prepared in several steps and completed in 2016.

(20)

 Greenhouse gas emission trends and projections in Europe 2015. Tracking progress towards Kyoto and 2020 targets, European Environment Agency, 2015.

(21)

ESD emissions in 2013 based on inventory data and in 2014 based on approximated inventory data reported in 2015 by Member States or estimated by the EEA, where needed, on behalf of the Commission.

(22)

SEC (2008) 85/3 EC

(23)

Climate action progress report, COM(2015) 576 final.

(24)

For example, Luxembourg is projected to fall short of its 2020 national target and expects to cover this gap by using the flexibility instruments provided for in the ESD. Its ESD emissions are mostly from road transport, which represents more than two-thirds of its ESD emissions due to low excise duties on motor fuel as well as a large number of cross-border commuters.

(25)

  http://ec.europa.eu/clima/policies/strategies/progress/docs/com_2015_576_en.pdf (For details see Approximated EU GHG inventory: Proxy GHG estimates for 2014, report of the European Environment Agency: http://www.eea.europa.eu/publications/approximated-eu-ghg-inventory-2014 ).

(26)

 Why did greenhouse gas emissions decrease in the EU between 1990 and 2012? European Environment Agency, 2014:

(27)

For more information on the findings from decomposition analysis please refer to Appendix 4 in Supporting study for the Evaluation of Decision No, 406/2009 (Effort Sharing Decision), Ricardo Energy and Environment with Trinomic and Vito.

(28)

Climate action progress report, COM(2015) 576 final, p.9.

(29)

 Supporting study for the Evaluation of Decision No, 406/2009 (Effort Sharing Decision), p.46.

(30)

 Synthesis of the data on GHG PAMs reported by Member States under the Monitoring Mechanism Regulation in 2015, European Environment Agency, 2015.

(31)

Member States regularly submit national communications to the UNFCCC including information on climate policies and measures to fulfil their international obligations.

(32)

WEM = with existing policies and measures, WAM = with additional measures (not yet implemented)

(33)

The impact assessment prepared for the climate and energy package that was presented by the Commission in 2008 estimated reduced air pollution costs at 10 to 11 billion EUR by 2020 as a result of the EU’s 20% GHG emission reduction target (i.e., including both the ESD and the ETS).

(34)

 Supporting study for the Evaluation of Decision No, 406/2009 (Effort Sharing Decision), Section 6.3.4.1.

(35)

Source: Supporting study for the Evaluation of Decision No. 406/2009/EC (Effort Sharing Decision), Ricardo Energy & Environment with Trinomics and Vito, December 2015.

(36)

Member States commitments for reducing or limiting their GHG emission under the second Kyoto protocol is equal too their 2020 targets under the ESD but also includes any emissions or removals in LULUCF.

(37)

  http://ec.europa.eu/clima/news/articles/news_2015060802_en.htm  

(38)

 Supporting study for the Evaluation of Decision No. 406/2009/EC (Effort Sharing Decision), Ricardo Energy and Environment with Trinomic and Vito.

(39)

  http://ec.europa.eu/clima/news/articles/news_2015060802_en.htm  

(40)

 Supporting study for the Evaluation of Decision No. 406/2009/EC (Effort Sharing Decision), Ricardo Energy and Environment with Trinomic and Vito.

(41)

A synopsis of the public consultation is provided in Annex 8.2 to the impact assessment accompanying the proposal for continuing effort sharing in the non-ETS sectors in the period 2021-2030, SWD(2016) 247.

(42)

A detailed analysis of national policies and measures is provided in appendix 3 to the study supporting the evaluation, Supporting study for the Evaluation of Decision No. 406/2009/EC (Effort Sharing Decision), Ricardo Energy and Environment with Trinomic and Vito.

(43)

The results of the decomposition analyses are discussed in more detail in Appendix 4 to the study supporting the evaluation, Supporting study for the Evaluation of Decision No. 406/2009/EC (Effort Sharing Decision), Ricardo Energy and Environment with Trinomic and Vito.

(44)

A full bibliography of all references used for the study supporting the evaluation is provided in Appendix 6 to that study.

(45)

Such additional CDM project credits must be from less developed countries and small island states.

(46)

Regulation (EU) 525/2013 of the European Parliament and the Council of 21 May 2013 on a mechanism for monitoring and reporting greenhouse gas emissions and for reporting other information at national and Union level relevant to climate change and repealing Decision No 280/2004/EC.